Ultimate Validation Command

validation/README.md

@@ -0,0 +1,102 @@
+# Ultimate Validation Command
+
+**One command to analyze your codebase and generate a comprehensive validation workflow.**
+
+## Quick Start
+
+### 1. Generate Your Validation Command
+```bash
+/ultimate_validate_command
+```
+
+This analyzes your codebase and creates `.claude/commands/validate.md` tailored to your project.
+
+### 2. Run Complete Validation
+```bash
+/validate
+```
+
+This runs all validation: linting, type checking, style checking, unit tests, and comprehensive end-to-end testing.
+
+## What It Does
+
+### Analyzes Your Codebase
+- Detects what validation tools you already have (ESLint, ruff, mypy, prettier, etc.)
+- Finds your test setup (pytest, jest, Playwright, etc.)
+- Understands your application architecture (routes, endpoints, database schema)
+- Examines existing test patterns and CI/CD configs
+
+### Generates validate.md
+Creates a validation command with phases:
+
+1. **Linting** - Using your configured linter
+2. **Type Checking** - Using your configured type checker
+3. **Style Checking** - Using your configured formatter
+4. **Unit Testing** - Running your existing unit tests
+5. **End-to-End Testing** - THIS IS THE KEY PART
+
+## The E2E Testing Magic
+
+The generated E2E tests are designed to be SO comprehensive that you don't need to manually test.
+
+### For Frontend Apps
+Uses Playwright to:
+- Test every user flow (registration, login, CRUD operations)
+- Interact with forms, buttons, navigation
+- Verify data persistence and UI updates
+- Test error states and validation
+- Cover all routes and features
+
+### For Backend Apps
+Uses Docker and custom scripts to:
+- Spin up full stack in containers
+- Test all API endpoints with real requests
+- Interact directly with database to verify data
+- Test complete workflows (auth → data operations → verification)
+- Create test utilities or API endpoints if needed
+
+### For Full-Stack Apps
+- Tests complete flows from UI through API to database
+- Verifies data consistency across all layers
+- Simulates real user behavior end-to-end
+
+## Key Philosophy
+
+**If `/validate` passes, your app works.**
+
+The E2E testing is creative and thorough enough that manual testing becomes unnecessary. It tests the application exactly how a real user would interact with it.
+
+## Example
+
+For a React + FastAPI app, the generated command might:
+
+1. Run ESLint, TypeScript check, Prettier
+2. Run pytest and Jest
+3. **E2E:**
+   - Use Playwright to test user registration → login → creating items → editing → deleting
+   - Spin up Docker containers for backend
+   - Use curl to test all API endpoints
+   - Query database directly to verify data integrity
+   - Test error handling, permissions, validation
+
+**Result:** Complete confidence that everything works.
+
+## How It's Different
+
+Traditional testing:
+- ❌ Unit tests in isolation
+- ❌ Manual E2E testing
+- ❌ Gaps in coverage
+- ❌ Time-consuming
+
+This approach:
+- ✅ Automated everything
+- ✅ Tests like a real user
+- ✅ Comprehensive E2E coverage
+- ✅ One command to validate all
+
+## Get Started
+
+Run `/ultimate_validate_command` to generate your validation workflow, then use `/validate` whenever you need complete confidence in your code.
+
+The generated command adapts to YOUR codebase and tests it thoroughly.

validation/example-validate.md

@@ -0,0 +1,61 @@
+---
+description: Comprehensive validation for this codebase
+---
+
+# Validate Codebase
+
+> **Example generated validation command** for a React + FastAPI + PostgreSQL app
+
+## Phase 1: Linting
+!`cd frontend && npm run lint`
+!`cd backend && ruff check src/`
+
+## Phase 2: Type Checking
+!`cd frontend && npx tsc --noEmit`
+!`cd backend && mypy src/`
+
+## Phase 3: Style Checking
+!`cd frontend && npm run format:check`
+!`cd backend && black --check src/`
+
+## Phase 4: Unit Testing
+!`cd frontend && npm test -- --coverage`
+!`cd backend && pytest tests/unit -v --cov=src`
+
+## Phase 5: End-to-End Testing
+
+### Setup
+!`docker-compose up -d`
+!`timeout 60 bash -c 'until curl -f http://localhost:8000/health; do sleep 2; done'`
+
+### Frontend E2E (Playwright)
+!`cd frontend && npx playwright test`
+
+**Tests:**
+- User registration → email verification → login
+- Create item → edit item → delete item
+- Search and filter functionality
+- Error handling and validation
+- All main user workflows
+
+### Backend E2E (API + Database)
+
+**Test all API endpoints:**
+!`curl -X POST http://localhost:8000/api/auth/register -d '{"email":"test@test.com","password":"Test123!"}'`
+!`TOKEN=$(curl -X POST http://localhost:8000/api/auth/login -d '{"email":"test@test.com","password":"Test123!"}' | jq -r '.token')`
+!`curl http://localhost:8000/api/items -H "Authorization: Bearer $TOKEN"`
+!`curl -X POST http://localhost:8000/api/items -H "Authorization: Bearer $TOKEN" -d '{"name":"Test"}'`
+
+**Verify database:**
+!`docker exec postgres psql -U user -d db -c "SELECT COUNT(*) FROM users;"`
+!`docker exec postgres psql -U user -d db -c "SELECT * FROM items WHERE name='Test';"`
+
+**Test error handling:**
+!`curl -w "%{http_code}" http://localhost:8000/api/items/invalid-id` # Should be 404
+!`curl -w "%{http_code}" http://localhost:8000/api/admin -H "Authorization: Bearer $TOKEN"` # Should be 403
+
+### Cleanup
+!`docker-compose down -v`
+
+## Summary
+All validation passed! Ready for deployment.

validation/ultimate_validate_command.md

@@ -0,0 +1,116 @@
+---
+description: Generate comprehensive validation command for this codebase
+---
+
+# Generate Ultimate Validation Command
+
+Analyze this codebase deeply and create `.claude/commands/validate.md` that comprehensively validates everything.
+
+## Step 0: Discover Real User Workflows
+
+**Before analyzing tooling, understand what users ACTUALLY do:**
+
+1. Read workflow documentation:
+   - README.md - Look for "Usage", "Quickstart", "Examples" sections
+   - CLAUDE.md/AGENTS.md or similar - Look for workflow patterns
+   - docs/ folder - User guides, tutorials
+
+2. Identify external integrations:
+   - What CLIs does the app use? (Check Dockerfile for installed tools)
+   - What external APIs does it call? (Telegram, Slack, GitHub, etc.)
+   - What services does it interact with?
+
+3. Extract complete user journeys from docs:
+   - Find examples like "Fix Issue (GitHub):" or "User does X → then Y → then Z"
+   - Each workflow becomes an E2E test scenario
+
+**Critical: Your E2E tests should mirror actual workflows from docs, not just test internal APIs.**
+
+## Step 1: Deep Codebase Analysis
+
+Explore the codebase to understand:
+
+**What validation tools already exist:**
+- Linting config: `.eslintrc*`, `.pylintrc`, `ruff.toml`, etc.
+- Type checking: `tsconfig.json`, `mypy.ini`, etc.
+- Style/formatting: `.prettierrc*`, `black`, `.editorconfig`
+- Unit tests: `jest.config.*`, `pytest.ini`, test directories
+- Package manager scripts: `package.json` scripts, `Makefile`, `pyproject.toml` tools
+
+**What the application does:**
+- Frontend: Routes, pages, components, user flows
+- Backend: API endpoints, authentication, database operations
+- Database: Schema, migrations, models
+- Infrastructure: Docker services, dependencies
+
+**How things are currently tested:**
+- Existing test files and patterns
+- CI/CD workflows (`.github/workflows/`, etc.)
+- Test commands in package.json or scripts
+
+## Step 2: Generate validate.md
+
+Create `.claude/commands/validate.md` with these phases (ONLY include phases that exist in the codebase):
+
+### Phase 1: Linting
+Run the actual linter commands found in the project (e.g., `npm run lint`, `ruff check`, etc.)
+
+### Phase 2: Type Checking
+Run the actual type checker commands found (e.g., `tsc --noEmit`, `mypy .`, etc.)
+
+### Phase 3: Style Checking
+Run the actual formatter check commands found (e.g., `prettier --check`, `black --check`, etc.)
+
+### Phase 4: Unit Testing
+Run the actual test commands found (e.g., `npm test`, `pytest`, etc.)
+
+### Phase 5: End-to-End Testing (BE CREATIVE AND COMPREHENSIVE)
+
+Test COMPLETE user workflows from documentation, not just internal APIs.
+
+**The Three Levels of E2E Testing:**
+
+1. **Internal APIs** (what you might naturally test):
+   - Test adapter endpoints work
+   - Database queries succeed
+   - Commands execute
+
+2. **External Integrations** (what you MUST test):
+   - CLI operations (GitHub CLI create issue/PR, etc.)
+   - Platform APIs (send Telegram message, post Slack message)
+   - Any external services the app depends on
+
+3. **Complete User Journeys** (what gives 100% confidence):
+   - Follow workflows from docs start-to-finish
+   - Example: "User asks bot to fix GitHub issue" → Bot clones repo → Makes changes → Creates PR → Comments on issue
+   - Test like a user would actually use the application in production
+
+**Examples of good vs. bad E2E tests:**
+- ❌ Bad: Tests that `/clone` command stores data in database
+- ✅ Good: Clone repo → Load commands → Execute command → Verify git commit created
+- ✅ Great: Create GitHub issue → Bot receives webhook → Analyzes issue → Creates PR → Comments on issue with PR link
+
+**Approach:**
+- Use Docker for isolated, reproducible testing
+- Create test data/repos/issues as needed
+- Verify outcomes in external systems (GitHub, database, file system)
+- Clean up after tests
+
+## Critical: Don't Stop Until Everything is Validated
+
+**Your job is to create a validation command that leaves NO STONE UNTURNED.**
+
+- Every user workflow from docs should be tested end-to-end
+- Every external integration should be exercised (GitHub CLI, APIs, etc.)
+- Every API endpoint should be hit
+- Every error case should be verified
+- Database integrity should be confirmed
+- The validation should be so thorough that manual testing is completely unnecessary
+
+If /validate passes, the user should have 100% confidence their application works correctly in production. Don't settle for partial coverage - make it comprehensive, creative, and complete.
+
+## Output
+
+Write the generated validation command to `.claude/commands/validate.md`
+
+The command should be executable, practical, and give complete confidence in the codebase.