From d7c840cf32a7443170f66b61d681cdbc7040e6dd Mon Sep 17 00:00:00 2001 From: Cole Medin Date: Sat, 15 Nov 2025 18:41:25 -0600 Subject: [PATCH] Ultimate Validation Command --- validation/README.md | 102 +++++++++++++++++++++ validation/example-validate.md | 61 +++++++++++++ validation/ultimate_validate_command.md | 116 ++++++++++++++++++++++++ 3 files changed, 279 insertions(+) create mode 100644 validation/README.md create mode 100644 validation/example-validate.md create mode 100644 validation/ultimate_validate_command.md diff --git a/validation/README.md b/validation/README.md new file mode 100644 index 0000000..eb6a7c3 --- /dev/null +++ b/validation/README.md @@ -0,0 +1,102 @@ +# Ultimate Validation Command + +**One command to analyze your codebase and generate a comprehensive validation workflow.** + +## Quick Start + +### 1. Generate Your Validation Command +```bash +/ultimate_validate_command +``` + +This analyzes your codebase and creates `.claude/commands/validate.md` tailored to your project. + +### 2. Run Complete Validation +```bash +/validate +``` + +This runs all validation: linting, type checking, style checking, unit tests, and comprehensive end-to-end testing. + +## What It Does + +### Analyzes Your Codebase +- Detects what validation tools you already have (ESLint, ruff, mypy, prettier, etc.) +- Finds your test setup (pytest, jest, Playwright, etc.) +- Understands your application architecture (routes, endpoints, database schema) +- Examines existing test patterns and CI/CD configs + +### Generates validate.md +Creates a validation command with phases: + +1. **Linting** - Using your configured linter +2. **Type Checking** - Using your configured type checker +3. **Style Checking** - Using your configured formatter +4. **Unit Testing** - Running your existing unit tests +5. **End-to-End Testing** - THIS IS THE KEY PART + +## The E2E Testing Magic + +The generated E2E tests are designed to be SO comprehensive that you don't need to manually test. + +### For Frontend Apps +Uses Playwright to: +- Test every user flow (registration, login, CRUD operations) +- Interact with forms, buttons, navigation +- Verify data persistence and UI updates +- Test error states and validation +- Cover all routes and features + +### For Backend Apps +Uses Docker and custom scripts to: +- Spin up full stack in containers +- Test all API endpoints with real requests +- Interact directly with database to verify data +- Test complete workflows (auth → data operations → verification) +- Create test utilities or API endpoints if needed + +### For Full-Stack Apps +- Tests complete flows from UI through API to database +- Verifies data consistency across all layers +- Simulates real user behavior end-to-end + +## Key Philosophy + +**If `/validate` passes, your app works.** + +The E2E testing is creative and thorough enough that manual testing becomes unnecessary. It tests the application exactly how a real user would interact with it. + +## Example + +For a React + FastAPI app, the generated command might: + +1. Run ESLint, TypeScript check, Prettier +2. Run pytest and Jest +3. **E2E:** + - Use Playwright to test user registration → login → creating items → editing → deleting + - Spin up Docker containers for backend + - Use curl to test all API endpoints + - Query database directly to verify data integrity + - Test error handling, permissions, validation + +**Result:** Complete confidence that everything works. + +## How It's Different + +Traditional testing: +- ❌ Unit tests in isolation +- ❌ Manual E2E testing +- ❌ Gaps in coverage +- ❌ Time-consuming + +This approach: +- ✅ Automated everything +- ✅ Tests like a real user +- ✅ Comprehensive E2E coverage +- ✅ One command to validate all + +## Get Started + +Run `/ultimate_validate_command` to generate your validation workflow, then use `/validate` whenever you need complete confidence in your code. + +The generated command adapts to YOUR codebase and tests it thoroughly. diff --git a/validation/example-validate.md b/validation/example-validate.md new file mode 100644 index 0000000..43a5693 --- /dev/null +++ b/validation/example-validate.md @@ -0,0 +1,61 @@ +--- +description: Comprehensive validation for this codebase +--- + +# Validate Codebase + +> **Example generated validation command** for a React + FastAPI + PostgreSQL app + +## Phase 1: Linting +!`cd frontend && npm run lint` +!`cd backend && ruff check src/` + +## Phase 2: Type Checking +!`cd frontend && npx tsc --noEmit` +!`cd backend && mypy src/` + +## Phase 3: Style Checking +!`cd frontend && npm run format:check` +!`cd backend && black --check src/` + +## Phase 4: Unit Testing +!`cd frontend && npm test -- --coverage` +!`cd backend && pytest tests/unit -v --cov=src` + +## Phase 5: End-to-End Testing + +### Setup +!`docker-compose up -d` +!`timeout 60 bash -c 'until curl -f http://localhost:8000/health; do sleep 2; done'` + +### Frontend E2E (Playwright) +!`cd frontend && npx playwright test` + +**Tests:** +- User registration → email verification → login +- Create item → edit item → delete item +- Search and filter functionality +- Error handling and validation +- All main user workflows + +### Backend E2E (API + Database) + +**Test all API endpoints:** +!`curl -X POST http://localhost:8000/api/auth/register -d '{"email":"test@test.com","password":"Test123!"}'` +!`TOKEN=$(curl -X POST http://localhost:8000/api/auth/login -d '{"email":"test@test.com","password":"Test123!"}' | jq -r '.token')` +!`curl http://localhost:8000/api/items -H "Authorization: Bearer $TOKEN"` +!`curl -X POST http://localhost:8000/api/items -H "Authorization: Bearer $TOKEN" -d '{"name":"Test"}'` + +**Verify database:** +!`docker exec postgres psql -U user -d db -c "SELECT COUNT(*) FROM users;"` +!`docker exec postgres psql -U user -d db -c "SELECT * FROM items WHERE name='Test';"` + +**Test error handling:** +!`curl -w "%{http_code}" http://localhost:8000/api/items/invalid-id` # Should be 404 +!`curl -w "%{http_code}" http://localhost:8000/api/admin -H "Authorization: Bearer $TOKEN"` # Should be 403 + +### Cleanup +!`docker-compose down -v` + +## Summary +All validation passed! Ready for deployment. diff --git a/validation/ultimate_validate_command.md b/validation/ultimate_validate_command.md new file mode 100644 index 0000000..48b2678 --- /dev/null +++ b/validation/ultimate_validate_command.md @@ -0,0 +1,116 @@ +--- +description: Generate comprehensive validation command for this codebase +--- + +# Generate Ultimate Validation Command + +Analyze this codebase deeply and create `.claude/commands/validate.md` that comprehensively validates everything. + +## Step 0: Discover Real User Workflows + +**Before analyzing tooling, understand what users ACTUALLY do:** + +1. Read workflow documentation: + - README.md - Look for "Usage", "Quickstart", "Examples" sections + - CLAUDE.md/AGENTS.md or similar - Look for workflow patterns + - docs/ folder - User guides, tutorials + +2. Identify external integrations: + - What CLIs does the app use? (Check Dockerfile for installed tools) + - What external APIs does it call? (Telegram, Slack, GitHub, etc.) + - What services does it interact with? + +3. Extract complete user journeys from docs: + - Find examples like "Fix Issue (GitHub):" or "User does X → then Y → then Z" + - Each workflow becomes an E2E test scenario + +**Critical: Your E2E tests should mirror actual workflows from docs, not just test internal APIs.** + +## Step 1: Deep Codebase Analysis + +Explore the codebase to understand: + +**What validation tools already exist:** +- Linting config: `.eslintrc*`, `.pylintrc`, `ruff.toml`, etc. +- Type checking: `tsconfig.json`, `mypy.ini`, etc. +- Style/formatting: `.prettierrc*`, `black`, `.editorconfig` +- Unit tests: `jest.config.*`, `pytest.ini`, test directories +- Package manager scripts: `package.json` scripts, `Makefile`, `pyproject.toml` tools + +**What the application does:** +- Frontend: Routes, pages, components, user flows +- Backend: API endpoints, authentication, database operations +- Database: Schema, migrations, models +- Infrastructure: Docker services, dependencies + +**How things are currently tested:** +- Existing test files and patterns +- CI/CD workflows (`.github/workflows/`, etc.) +- Test commands in package.json or scripts + +## Step 2: Generate validate.md + +Create `.claude/commands/validate.md` with these phases (ONLY include phases that exist in the codebase): + +### Phase 1: Linting +Run the actual linter commands found in the project (e.g., `npm run lint`, `ruff check`, etc.) + +### Phase 2: Type Checking +Run the actual type checker commands found (e.g., `tsc --noEmit`, `mypy .`, etc.) + +### Phase 3: Style Checking +Run the actual formatter check commands found (e.g., `prettier --check`, `black --check`, etc.) + +### Phase 4: Unit Testing +Run the actual test commands found (e.g., `npm test`, `pytest`, etc.) + +### Phase 5: End-to-End Testing (BE CREATIVE AND COMPREHENSIVE) + +Test COMPLETE user workflows from documentation, not just internal APIs. + +**The Three Levels of E2E Testing:** + +1. **Internal APIs** (what you might naturally test): + - Test adapter endpoints work + - Database queries succeed + - Commands execute + +2. **External Integrations** (what you MUST test): + - CLI operations (GitHub CLI create issue/PR, etc.) + - Platform APIs (send Telegram message, post Slack message) + - Any external services the app depends on + +3. **Complete User Journeys** (what gives 100% confidence): + - Follow workflows from docs start-to-finish + - Example: "User asks bot to fix GitHub issue" → Bot clones repo → Makes changes → Creates PR → Comments on issue + - Test like a user would actually use the application in production + +**Examples of good vs. bad E2E tests:** +- ❌ Bad: Tests that `/clone` command stores data in database +- ✅ Good: Clone repo → Load commands → Execute command → Verify git commit created +- ✅ Great: Create GitHub issue → Bot receives webhook → Analyzes issue → Creates PR → Comments on issue with PR link + +**Approach:** +- Use Docker for isolated, reproducible testing +- Create test data/repos/issues as needed +- Verify outcomes in external systems (GitHub, database, file system) +- Clean up after tests + +## Critical: Don't Stop Until Everything is Validated + +**Your job is to create a validation command that leaves NO STONE UNTURNED.** + +- Every user workflow from docs should be tested end-to-end +- Every external integration should be exercised (GitHub CLI, APIs, etc.) +- Every API endpoint should be hit +- Every error case should be verified +- Database integrity should be confirmed +- The validation should be so thorough that manual testing is completely unnecessary + +If /validate passes, the user should have 100% confidence their application works correctly in production. Don't settle for partial coverage - make it comprehensive, creative, and complete. + +## Output + +Write the generated validation command to `.claude/commands/validate.md` + +The command should be executable, practical, and give complete confidence in the codebase.