Add caveman-compress skill

.gitignore

@@ -3,3 +3,5 @@ __pycache__/
 *.pyc
 .venv/
 .env.local
+caveman-compress.md
+**DS_Store**

caveman-compress/README.md

@@ -0,0 +1,161 @@
+<p align="center">
+  <img src="https://em-content.zobj.net/source/apple/391/rock_1faa8.png" width="80" />
+</p>
+
+<h1 align="center">caveman-compress</h1>
+
+<p align="center">
+  <strong>shrink memory file. save token every session.</strong>
+</p>
+
+---
+
+A Claude Code skill that compresses your project memory files (`CLAUDE.md`, todos, preferences) into caveman format — so every session loads fewer tokens automatically.
+
+Claude read `CLAUDE.md` on every session start. If file big, cost big. Caveman make file small. Cost go down forever.
+
+## What It Do
+
+```
+/caveman-compress CLAUDE.md
+```
+
+```
+CLAUDE.md          ← compressed (Claude reads this — fewer tokens every session)
+CLAUDE.original.md ← human-readable backup (you edit this)
+```
+
+Original never lost. You can read and edit `.original.md`. Run skill again to re-compress after edits.
+
+## Benchmarks
+
+Real results on real project files:
+
+| File | Original | Compressed | Saved |
+|------|----------:|----------:|------:|
+| `claude-md-preferences.md` | 706 | 285 | **59.6%** |
+| `project-notes.md` | 1145 | 535 | **53.3%** |
+| `claude-md-project.md` | 1122 | 687 | **38.8%** |
+| `todo-list.md` | 627 | 388 | **38.1%** |
+| `mixed-with-code.md` | 888 | 574 | **35.4%** |
+| **Average** | **898** | **494** | **45%** |
+
+All validations passed ✅ — headings, code blocks, URLs, file paths preserved exactly.
+
+## Before / After
+
+<table>
+<tr>
+<td width="50%">
+
+### 📄 Original (706 tokens)
+
+> "I strongly prefer TypeScript with strict mode enabled for all new code. Please don't use `any` type unless there's genuinely no way around it, and if you do, leave a comment explaining the reasoning. I find that taking the time to properly type things catches a lot of bugs before they ever make it to runtime."
+
+</td>
+<td width="50%">
+
+### 🪨 Caveman (285 tokens)
+
+> "Prefer TypeScript strict mode always. No `any` unless unavoidable — comment why if used. Proper types catch bugs early."
+
+</td>
+</tr>
+</table>
+
+**Same instructions. 60% fewer tokens. Every. Single. Session.**
+
+## Install
+
+```bash
+cp -r ~/.claude/skills/caveman-compress <path-to-skill>
+```
+
+Or if you have the caveman repo:
+
+```bash
+cp -r skills/caveman-compress ~/.claude/skills/caveman-compress
+```
+
+**Requires:** Python 3.10+
+
+## Usage
+
+```
+/caveman-compress <filepath>
+```
+
+Examples:
+```
+/caveman-compress CLAUDE.md
+/caveman-compress docs/preferences.md
+/caveman-compress todos.md
+```
+
+### What files work
+
+| Type | Compress? |
+|------|-----------|
+| `.md`, `.txt`, `.rst` | ✅ Yes |
+| Extensionless natural language | ✅ Yes |
+| `.py`, `.js`, `.ts`, `.json`, `.yaml` | ❌ Skip (code/config) |
+| `*.original.md` | ❌ Skip (backup files) |
+
+## How It Work
+
+```
+/caveman-compress CLAUDE.md
+        ↓
+detect file type        (no tokens)
+        ↓
+Claude compresses       (tokens — one call)
+        ↓
+validate output         (no tokens)
+  checks: headings, code blocks, URLs, file paths, bullets
+        ↓
+if errors: Claude fixes cherry-picked issues only   (tokens — targeted fix)
+  does NOT recompress — only patches broken parts
+        ↓
+retry up to 2 times
+        ↓
+write compressed → CLAUDE.md
+write original   → CLAUDE.original.md
+```
+
+Only two things use tokens: initial compression + targeted fix if validation fails. Everything else is local Python.
+
+## What Is Preserved
+
+Caveman compress natural language. It never touch:
+
+- Code blocks (` ``` ` fenced or indented)
+- Inline code (`` `backtick content` ``)
+- URLs and links
+- File paths (`/src/components/...`)
+- Commands (`npm install`, `git commit`)
+- Technical terms, library names, API names
+- Headings (exact text preserved)
+- Tables (structure preserved, cell text compressed)
+- Dates, version numbers, numeric values
+
+## Why This Matter
+
+`CLAUDE.md` loads on **every session start**. A 1000-token project memory file costs tokens every single time you open a project. Over 100 sessions that's 100,000 tokens of overhead — just for context you already wrote.
+
+Caveman cut that by ~45% on average. Same instructions. Same accuracy. Less waste.
+
+```
+┌──────────────────────────────────────────┐
+│  TOKEN SAVINGS PER FILE    ████████  45% │
+│  SESSIONS THAT BENEFIT     ████████ 100% │
+│  INFORMATION PRESERVED     ████████ 100% │
+│  SETUP TIME                █         1x  │
+└──────────────────────────────────────────┘
+```
+
+## Part of Caveman
+
+This skill is part of the [caveman](https://github.com/JuliusBrussee/caveman) toolkit — making Claude use fewer tokens without losing accuracy.
+
+- **caveman** — make Claude *speak* like caveman (cuts response tokens ~65%)
+- **caveman-compress** — make Claude *read* less (cuts context tokens ~45%)

caveman-compress/SKILL.md

@@ -0,0 +1,111 @@
+---
+name: caveman-compress
+description: >
+  Compress natural language memory files (CLAUDE.md, todos, preferences) into caveman format
+  to save input tokens. Preserves all technical substance, code, URLs, and structure.
+  Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md.
+  Trigger: /caveman-compress <filepath> or "compress memory file"
+---
+
+# Caveman Compress
+
+## Purpose
+
+Compress natural language files (CLAUDE.md, todos, preferences) into caveman-speak to reduce input tokens. Compressed version overwrites original. Human-readable backup saved as `<filename>.original.md`.
+
+## Trigger
+
+`/caveman-compress <filepath>` or when user asks to compress a memory file.
+
+## Process
+
+1. This SKILL.md lives alongside `memory/` in the same directory. Find that directory.
+
+2. Run:
+   ```
+   cd <directory_containing_this_SKILL.md> && python3 -m scripts <absolute_filepath>
+   ```
+
+3. The CLI will:
+   - detect file type (no tokens)
+   - call Claude to compress
+   - validate output (no tokens)
+   - if errors: cherry-pick fix with Claude (targeted fixes only, no recompression)
+   - retry up to 2 times
+
+4. Return result to user
+
+## Compression Rules
+
+### Remove
+- Articles: a, an, the
+- Filler: just, really, basically, actually, simply, essentially, generally
+- Pleasantries: "sure", "certainly", "of course", "happy to", "I'd recommend"
+- Hedging: "it might be worth", "you could consider", "it would be good to"
+- Redundant phrasing: "in order to" → "to", "make sure to" → "ensure", "the reason is because" → "because"
+- Connective fluff: "however", "furthermore", "additionally", "in addition"
+
+### Preserve EXACTLY (never modify)
+- Code blocks (fenced ``` and indented)
+- Inline code (`backtick content`)
+- URLs and links (full URLs, markdown links)
+- File paths (`/src/components/...`, `./config.yaml`)
+- Commands (`npm install`, `git commit`, `docker build`)
+- Technical terms (library names, API names, protocols, algorithms)
+- Proper nouns (project names, people, companies)
+- Dates, version numbers, numeric values
+- Environment variables (`$HOME`, `NODE_ENV`)
+
+### Preserve Structure
+- All markdown headings (keep exact heading text, compress body below)
+- Bullet point hierarchy (keep nesting level)
+- Numbered lists (keep numbering)
+- Tables (compress cell text, keep structure)
+- Frontmatter/YAML headers in markdown files
+
+### Compress
+- Use short synonyms: "big" not "extensive", "fix" not "implement a solution for", "use" not "utilize"
+- Fragments OK: "Run tests before commit" not "You should always run tests before committing"
+- Drop "you should", "make sure to", "remember to" — just state the action
+- Merge redundant bullets that say the same thing differently
+- Keep one example where multiple examples show the same pattern
+
+CRITICAL RULE:
+Anything inside ``` ... ``` must be copied EXACTLY.
+Do not:
+- remove comments
+- remove spacing
+- reorder lines
+- shorten commands
+- simplify anything
+
+Inline code (`...`) must be preserved EXACTLY.
+Do not modify anything inside backticks.
+
+If file contains code blocks:
+- Treat code blocks as read-only regions
+- Only compress text outside them
+- Do not merge sections around code
+
+## Pattern
+
+Original:
+> You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production.
+
+Compressed:
+> Run tests before push to main. Catch bugs early, prevent broken prod deploys.
+
+Original:
+> The application uses a microservices architecture with the following components. The API gateway handles all incoming requests and routes them to the appropriate service. The authentication service is responsible for managing user sessions and JWT tokens.
+
+Compressed:
+> Microservices architecture. API gateway route all requests to services. Auth service manage user sessions + JWT tokens.
+
+## Boundaries
+
+- ONLY compress natural language files (.md, .txt, extensionless)
+- NEVER modify: .py, .js, .ts, .json, .yaml, .yml, .toml, .env, .lock, .css, .html, .xml, .sql, .sh
+- If file has mixed content (prose + code), compress ONLY the prose sections
+- If unsure whether something is code or prose, leave it unchanged
+- Original file is backed up as FILE.original.md before overwriting
+- Never compress FILE.original.md (skip it)

caveman-compress/scripts/__main__.py

@@ -0,0 +1,3 @@
+from .cli import main
+
+main()

caveman-compress/scripts/benchmark.py

@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+from pathlib import Path
+import sys
+
+# Support both direct execution and module import
+try:
+    from .validate import validate
+except ImportError:
+    sys.path.insert(0, str(Path(__file__).parent))
+    from validate import validate
+
+try:
+    import tiktoken
+    _enc = tiktoken.get_encoding("cl100k_base")
+except ImportError:
+    _enc = None
+
+
+def count_tokens(text):
+    if _enc is None:
+        return len(text.split())  # fallback: word count
+    return len(_enc.encode(text))
+
+
+def benchmark_pair(orig_path: Path, comp_path: Path):
+    orig_text = orig_path.read_text()
+    comp_text = comp_path.read_text()
+
+    orig_tokens = count_tokens(orig_text)
+    comp_tokens = count_tokens(comp_text)
+    saved = 100 * (orig_tokens - comp_tokens) / orig_tokens
+    result = validate(orig_path, comp_path)
+
+    return (comp_path.name, orig_tokens, comp_tokens, saved, result.is_valid)
+
+
+def print_table(rows):
+    print("\n| File | Original | Compressed | Saved % | Valid |")
+    print("|------|----------|------------|---------|-------|")
+    for r in rows:
+        print(f"| {r[0]} | {r[1]} | {r[2]} | {r[3]:.1f}% | {'✅' if r[4] else '❌'} |")
+
+
+def main():
+    # Direct file pair: python3 benchmark.py original.md compressed.md
+    if len(sys.argv) == 3:
+        orig = Path(sys.argv[1])
+        comp = Path(sys.argv[2])
+        if not orig.exists():
+            print(f"❌ Not found: {orig}")
+            sys.exit(1)
+        if not comp.exists():
+            print(f"❌ Not found: {comp}")
+            sys.exit(1)
+        print_table([benchmark_pair(orig, comp)])
+        return
+
+    # Glob mode: repo_root/tests/caveman-compress/
+    tests_dir = Path(__file__).parent.parent.parent / "tests" / "caveman-compress"
+    if not tests_dir.exists():
+        print(f"❌ Tests dir not found: {tests_dir}")
+        sys.exit(1)
+
+    rows = []
+    for orig in sorted(tests_dir.glob("*.original.md")):
+        comp = orig.with_name(orig.stem.removesuffix(".original") + ".md")
+        if comp.exists():
+            rows.append(benchmark_pair(orig, comp))
+
+    if not rows:
+        print("No compressed file pairs found.")
+        return
+
+    print_table(rows)
+
+
+if __name__ == "__main__":
+    main()

caveman-compress/scripts/cli.py

@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""
+Caveman Memory CLI
+
+Usage:
+    caveman <filepath>
+"""
+
+import sys
+from pathlib import Path
+
+from .compress import compress_file
+from .detect import detect_file_type, should_compress
+
+
+def print_usage():
+    print("Usage: caveman <filepath>")
+
+
+def main():
+    if len(sys.argv) != 2:
+        print_usage()
+        sys.exit(1)
+
+    filepath = Path(sys.argv[1])
+
+    # Check file exists
+    if not filepath.exists():
+        print(f"❌ File not found: {filepath}")
+        sys.exit(1)
+
+    if not filepath.is_file():
+        print(f"❌ Not a file: {filepath}")
+        sys.exit(1)
+
+    # Detect file type
+    file_type = detect_file_type(filepath)
+
+    print(f"Detected: {file_type}")
+
+    # Check if compressible
+    if not should_compress(filepath):
+        print("Skipping: file is not natural language (code/config)")
+        sys.exit(0)
+
+    print("Starting caveman compression...\n")
+
+    try:
+        success = compress_file(filepath)
+
+        if success:
+            print("\nCompression completed successfully")
+            backup_path = filepath.with_name(filepath.stem + ".original.md")
+            print(f"Compressed: {filepath}")
+            print(f"Original:   {backup_path}")
+            sys.exit(0)
+        else:
+            print("\n❌ Compression failed after retries")
+            sys.exit(2)
+
+    except KeyboardInterrupt:
+        print("\nInterrupted by user")
+        sys.exit(130)
+
+    except Exception as e:
+        print(f"\n❌ Error: {e}")
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()

caveman-compress/scripts/compress.py

@@ -0,0 +1,158 @@
+#!/usr/bin/env python3
+"""
+Caveman Memory Orchestrator
+
+Usage:
+    python memory/compress.py <filepath>
+"""
+
+import subprocess
+import sys
+from pathlib import Path
+from typing import List
+
+from .detect import should_compress
+from .validate import validate
+
+MAX_RETRIES = 2
+
+
+# ---------- Claude Calls ----------
+
+
+def call_claude(prompt: str) -> str:
+    try:
+        result = subprocess.run(
+            ["claude", "--print"],
+            input=prompt,
+            text=True,
+            capture_output=True,
+            check=True,
+        )
+        return result.stdout.strip()
+    except subprocess.CalledProcessError as e:
+        raise RuntimeError(f"Claude call failed:\n{e.stderr}")
+
+
+def build_compress_prompt(original: str) -> str:
+    return f"""
+Compress this markdown into caveman format.
+
+STRICT RULES:
+- Do NOT modify anything inside ``` code blocks
+- Do NOT modify anything inside inline backticks
+- Preserve ALL URLs exactly
+- Preserve ALL headings exactly
+- Preserve file paths and commands
+
+Only compress natural language.
+
+TEXT:
+{original}
+"""
+
+
+def build_fix_prompt(original: str, compressed: str, errors: List[str]) -> str:
+    errors_str = "\n".join(f"- {e}" for e in errors)
+    return f"""You are fixing a caveman-compressed markdown file. Specific validation errors were found.
+
+CRITICAL RULES:
+- DO NOT recompress or rephrase the file
+- ONLY fix the listed errors — leave everything else exactly as-is
+- The ORIGINAL is provided as reference only (to restore missing content)
+- Preserve caveman style in all untouched sections
+
+ERRORS TO FIX:
+{errors_str}
+
+HOW TO FIX:
+- Missing URL: find it in ORIGINAL, restore it exactly where it belongs in COMPRESSED
+- Code block mismatch: find the exact code block in ORIGINAL, restore it in COMPRESSED
+- Heading mismatch: restore the exact heading text from ORIGINAL into COMPRESSED
+- Do not touch any section not mentioned in the errors
+
+ORIGINAL (reference only):
+{original}
+
+COMPRESSED (fix this):
+{compressed}
+
+Return ONLY the fixed compressed file. No explanation.
+"""
+
+
+# ---------- Core Logic ----------
+
+
+def compress_file(filepath: Path) -> bool:
+    print(f"📄 Processing: {filepath}")
+
+    if not should_compress(filepath):
+        print("⚠️ Skipping (not natural language)")
+        return False
+
+    original_text = filepath.read_text(errors="ignore")
+    backup_path = filepath.with_name(filepath.stem + ".original.md")
+
+    # Step 1: Compress
+    print("🧠 Compressing with Claude...")
+    compressed = call_claude(build_compress_prompt(original_text))
+
+    # Save original as backup, write compressed to original path
+    backup_path.write_text(original_text)
+    filepath.write_text(compressed)
+
+    # Step 2: Validate + Retry
+    for attempt in range(MAX_RETRIES):
+        print(f"\n🔍 Validation attempt {attempt + 1}")
+
+        result = validate(backup_path, filepath)
+
+        if result.is_valid:
+            print("✅ Validation passed")
+            break
+
+        print("❌ Validation failed:")
+        for err in result.errors:
+            print(f"   - {err}")
+
+        if attempt == MAX_RETRIES - 1:
+            # Restore original on failure
+            filepath.write_text(original_text)
+            backup_path.unlink(missing_ok=True)
+            print("❌ Failed after retries — original restored")
+            return False
+
+        print("🛠 Fixing with Claude...")
+        compressed = call_claude(
+            build_fix_prompt(original_text, compressed, result.errors)
+        )
+        filepath.write_text(compressed)
+
+    return True
+
+
+# ---------- Main ----------
+
+
+def main():
+    if len(sys.argv) != 2:
+        print("Usage: python memory/compress.py <filepath>")
+        sys.exit(1)
+
+    filepath = Path(sys.argv[1])
+
+    if not filepath.exists():
+        print(f"❌ File not found: {filepath}")
+        sys.exit(1)
+
+    success = compress_file(filepath)
+
+    if success:
+        sys.exit(0)
+    else:
+        sys.exit(2)
+
+
+if __name__ == "__main__":
+    main()

caveman-compress/scripts/detect.py

@@ -0,0 +1,121 @@
+#!/usr/bin/env python3
+"""Detect whether a file is natural language (compressible) or code/config (skip)."""
+
+import json
+import re
+from pathlib import Path
+
+# Extensions that are natural language and compressible
+COMPRESSIBLE_EXTENSIONS = {".md", ".txt", ".markdown", ".rst"}
+
+# Extensions that are code/config and should be skipped
+SKIP_EXTENSIONS = {
+    ".py", ".js", ".ts", ".tsx", ".jsx", ".json", ".yaml", ".yml",
+    ".toml", ".env", ".lock", ".css", ".scss", ".html", ".xml",
+    ".sql", ".sh", ".bash", ".zsh", ".go", ".rs", ".java", ".c",
+    ".cpp", ".h", ".hpp", ".rb", ".php", ".swift", ".kt", ".lua",
+    ".dockerfile", ".makefile", ".csv", ".ini", ".cfg",
+}
+
+# Patterns that indicate a line is code
+CODE_PATTERNS = [
+    re.compile(r"^\s*(import |from .+ import |require\(|const |let |var )"),
+    re.compile(r"^\s*(def |class |function |async function |export )"),
+    re.compile(r"^\s*(if\s*\(|for\s*\(|while\s*\(|switch\s*\(|try\s*\{)"),
+    re.compile(r"^\s*[\}\]\);]+\s*$"),  # closing braces/brackets
+    re.compile(r"^\s*@\w+"),  # decorators/annotations
+    re.compile(r'^\s*"[^"]+"\s*:\s*'),  # JSON-like key-value
+    re.compile(r"^\s*\w+\s*=\s*[{\[\(\"']"),  # assignment with literal
+]
+
+
+def _is_code_line(line: str) -> bool:
+    """Check if a line looks like code."""
+    return any(p.match(line) for p in CODE_PATTERNS)
+
+
+def _is_json_content(text: str) -> bool:
+    """Check if content is valid JSON."""
+    try:
+        json.loads(text)
+        return True
+    except (json.JSONDecodeError, ValueError):
+        return False
+
+
+def _is_yaml_content(lines: list[str]) -> bool:
+    """Heuristic: check if content looks like YAML."""
+    yaml_indicators = 0
+    for line in lines[:30]:
+        stripped = line.strip()
+        if stripped.startswith("---"):
+            yaml_indicators += 1
+        elif re.match(r"^\w[\w\s]*:\s", stripped):
+            yaml_indicators += 1
+        elif stripped.startswith("- ") and ":" in stripped:
+            yaml_indicators += 1
+    # If most non-empty lines look like YAML
+    non_empty = sum(1 for l in lines[:30] if l.strip())
+    return non_empty > 0 and yaml_indicators / non_empty > 0.6
+
+
+def detect_file_type(filepath: Path) -> str:
+    """Classify a file as 'natural_language', 'code', 'config', or 'unknown'.
+
+    Returns:
+        One of: 'natural_language', 'code', 'config', 'unknown'
+    """
+    ext = filepath.suffix.lower()
+
+    # Extension-based classification
+    if ext in COMPRESSIBLE_EXTENSIONS:
+        return "natural_language"
+    if ext in SKIP_EXTENSIONS:
+        return "code" if ext not in {".json", ".yaml", ".yml", ".toml", ".ini", ".cfg", ".env"} else "config"
+
+    # Extensionless files (like CLAUDE.md, TODO) — check content
+    if not ext:
+        try:
+            text = filepath.read_text(errors="ignore")
+        except (OSError, PermissionError):
+            return "unknown"
+
+        lines = text.splitlines()[:50]
+
+        if _is_json_content(text[:10000]):
+            return "config"
+        if _is_yaml_content(lines):
+            return "config"
+
+        code_lines = sum(1 for l in lines if l.strip() and _is_code_line(l))
+        non_empty = sum(1 for l in lines if l.strip())
+        if non_empty > 0 and code_lines / non_empty > 0.4:
+            return "code"
+
+        return "natural_language"
+
+    return "unknown"
+
+
+def should_compress(filepath: Path) -> bool:
+    """Return True if the file is natural language and should be compressed."""
+    if not filepath.is_file():
+        return False
+    # Skip backup files
+    if filepath.name.endswith(".original.md"):
+        return False
+    return detect_file_type(filepath) == "natural_language"
+
+
+if __name__ == "__main__":
+    import sys
+
+    if len(sys.argv) < 2:
+        print("Usage: python detect.py <file1> [file2] ...")
+        sys.exit(1)
+
+    for path_str in sys.argv[1:]:
+        p = Path(path_str)
+        file_type = detect_file_type(p)
+        compress = should_compress(p)
+        print(f"  {p.name:30s} type={file_type:20s} compress={compress}")

caveman-compress/scripts/validate.py

@@ -0,0 +1,148 @@
+#!/usr/bin/env python3
+import re
+from pathlib import Path
+
+URL_REGEX = re.compile(r"https?://[^\s)]+")
+CODE_BLOCK_REGEX = re.compile(r"```.*?```", re.DOTALL)
+HEADING_REGEX = re.compile(r"^(#{1,6})\s+(.*)", re.MULTILINE)
+BULLET_REGEX = re.compile(r"^\s*[-*+]\s+", re.MULTILINE)
+
+# crude but effective path detection
+PATH_REGEX = re.compile(r"(\./|\../|/|[A-Za-z]:\\)[\w\-/\\\.]+")
+
+
+class ValidationResult:
+    def __init__(self):
+        self.is_valid = True
+        self.errors = []
+        self.warnings = []
+
+    def add_error(self, msg):
+        self.is_valid = False
+        self.errors.append(msg)
+
+    def add_warning(self, msg):
+        self.warnings.append(msg)
+
+
+def read_file(path: Path) -> str:
+    return path.read_text(errors="ignore")
+
+
+# ---------- Extractors ----------
+
+
+def extract_headings(text):
+    return [(level, title.strip()) for level, title in HEADING_REGEX.findall(text)]
+
+
+def extract_code_blocks(text):
+    return CODE_BLOCK_REGEX.findall(text)
+
+
+def extract_urls(text):
+    return set(URL_REGEX.findall(text))
+
+
+def extract_paths(text):
+    return set(PATH_REGEX.findall(text))
+
+
+def count_bullets(text):
+    return len(BULLET_REGEX.findall(text))
+
+
+# ---------- Validators ----------
+
+
+def validate_headings(orig, comp, result):
+    h1 = extract_headings(orig)
+    h2 = extract_headings(comp)
+
+    if len(h1) != len(h2):
+        result.add_error(f"Heading count mismatch: {len(h1)} vs {len(h2)}")
+
+    if h1 != h2:
+        result.add_warning("Heading text/order changed")
+
+
+def validate_code_blocks(orig, comp, result):
+    c1 = extract_code_blocks(orig)
+    c2 = extract_code_blocks(comp)
+
+    if c1 != c2:
+        result.add_error("Code blocks not preserved exactly")
+
+
+def validate_urls(orig, comp, result):
+    u1 = extract_urls(orig)
+    u2 = extract_urls(comp)
+
+    if u1 != u2:
+        result.add_error(f"URL mismatch: lost={u1 - u2}, added={u2 - u1}")
+
+
+def validate_paths(orig, comp, result):
+    p1 = extract_paths(orig)
+    p2 = extract_paths(comp)
+
+    if p1 != p2:
+        result.add_warning(f"Path mismatch: lost={p1 - p2}, added={p2 - p1}")
+
+
+def validate_bullets(orig, comp, result):
+    b1 = count_bullets(orig)
+    b2 = count_bullets(comp)
+
+    if b1 == 0:
+        return
+
+    diff = abs(b1 - b2) / b1
+
+    if diff > 0.15:
+        result.add_warning(f"Bullet count changed too much: {b1} -> {b2}")
+
+
+# ---------- Main ----------
+
+
+def validate(original_path: Path, compressed_path: Path) -> ValidationResult:
+    result = ValidationResult()
+
+    orig = read_file(original_path)
+    comp = read_file(compressed_path)
+
+    validate_headings(orig, comp, result)
+    validate_code_blocks(orig, comp, result)
+    validate_urls(orig, comp, result)
+    validate_paths(orig, comp, result)
+    validate_bullets(orig, comp, result)
+
+    return result
+
+
+# ---------- CLI ----------
+
+if __name__ == "__main__":
+    import sys
+
+    if len(sys.argv) != 3:
+        print("Usage: python validate.py <original> <compressed>")
+        sys.exit(1)
+
+    orig = Path(sys.argv[1])
+    comp = Path(sys.argv[2])
+
+    res = validate(orig, comp)
+
+    print(f"\nValid: {res.is_valid}")
+
+    if res.errors:
+        print("\nErrors:")
+        for e in res.errors:
+            print(f"  - {e}")
+
+    if res.warnings:
+        print("\nWarnings:")
+        for w in res.warnings:
+            print(f"  - {w}")

skills/caveman/SKILL.md

@@ -1,72 +0,0 @@
----
-name: caveman
-description: >
-  Ultra-compressed communication mode. Slash token usage ~75% by speaking like caveman
-  while keeping full technical accuracy. Use when user says "caveman mode", "talk like caveman",
-  "use caveman", "less tokens", "be brief", or invokes /caveman. Also auto-triggers
-  when token efficiency is requested.
----
-
-# Caveman Mode
-
-## Core Rule
-
-Respond like smart caveman. Cut articles, filler, pleasantries. Keep all technical substance.
-
-## Grammar
-
-- Drop articles (a, an, the)
-- Drop filler (just, really, basically, actually, simply)
-- Drop pleasantries (sure, certainly, of course, happy to)
-- Short synonyms (big not extensive, fix not "implement a solution for")
-- No hedging (skip "it might be worth considering")
-- Fragments fine. No need full sentence
-- Technical terms stay exact. "Polymorphism" stays "polymorphism"
-- Code blocks unchanged. Caveman speak around code, not in code
-- Error messages quoted exact. Caveman only for explanation
-
-## Pattern
-
-```
-[thing] [action] [reason]. [next step].
-```
-
-Not:
-> Sure! I'd be happy to help you with that. The issue you're experiencing is likely caused by...
-
-Yes:
-> Bug in auth middleware. Token expiry check use `<` not `<=`. Fix:
-
-## Examples
-
-**User:** Why is my React component re-rendering?
-
-**Normal (69 tokens):** "The reason your React component is re-rendering is likely because you're creating a new object reference on each render cycle. When you pass an inline object as a prop, React's shallow comparison sees it as a different object every time, which triggers a re-render. I'd recommend using useMemo to memoize the object."
-
-**Caveman (19 tokens):** "New object ref each render. Inline object prop = new ref = re-render. Wrap in `useMemo`."
-
----
-
-**User:** How do I set up a PostgreSQL connection pool?
-
-**Caveman:**
-```
-Use `pg` pool:
-```
-```js
-const pool = new Pool({
-  max: 20,
-  idleTimeoutMillis: 30000,
-  connectionTimeoutMillis: 2000,
-})
-```
-```
-max = concurrent connections. Keep under DB limit. idleTimeout kill stale conn.
-```
-
-## Boundaries
-
-- Code: write normal. Caveman English only
-- Git commits: normal
-- PR descriptions: normal
-- User say "stop caveman" or "normal mode": revert immediately

tests/caveman-compress/claude-md-preferences.md

@@ -0,0 +1,43 @@
+# User Preferences
+
+## Code Style
+
+Prefer TypeScript strict mode always. No `any` unless unavoidable — comment why if used. Proper types catch bugs early.
+
+React: functional components + hooks only. No class components. State local as possible; lift only when truly needed. Rather extra re-renders than complex global state.
+
+Imports: organized, external/internal/relative separated. Use path aliases (`@/components/...`) not deep relative paths.
+
+## Testing Approach
+
+Always write tests for new functionality. Test behavior from user perspective, not implementation. "Clicking submit creates task" not "handleSubmit calls createTask."
+
+React: Testing Library, no internal state/lifecycle testing. API endpoints: integration tests hit real DB — mocked tests passed but prod broke too many times.
+
+No 100% coverage needed. Critical paths matter. Services: thorough unit tests. UI: happy path + key error states.
+
+## Communication Style
+
+Senior engineer, 2 years on project. Skip basic concepts. Concise, direct. Tradeoffs: options + pros/cons, no paragraphs.
+
+Show actual code, not descriptions. Multiple files: show all at once.
+
+Comments explain "why" not "what." Code needing "what" comments needs refactoring instead.
+
+## Workflow Preferences
+
+Read existing code before changes. Follow existing conventions over preferred approach — consistency > preference in team codebase.
+
+PRs: small, focused. 3 small > 1 large. Each PR does one thing. Found something else? Separate PR.
+
+Run linter + type checker before committing. Run manually after large refactors even with pre-commit hooks.
+
+## Things to Avoid
+
+No `console.log` — use `src/lib/logger.ts`. Logs reach prod and clutter output.
+
+No new deps without discussion. Keep bundle small, avoid duplicate libraries. Prefer built-in Node/browser APIs over packages.
+
+No single-consumer abstractions. Premature abstraction worse than duplication. Wait for 3+ use cases before extracting shared utility.
+
+Never commit `.env` or secrets/keys/credentials. `.env.example` gets placeholder values only.

tests/caveman-compress/claude-md-preferences.original.md

@@ -0,0 +1,43 @@
+# User Preferences
+
+## Code Style
+
+I strongly prefer TypeScript with strict mode enabled for all new code. Please don't use `any` type unless there's genuinely no way around it, and if you do, leave a comment explaining the reasoning. I find that taking the time to properly type things catches a lot of bugs before they ever make it to runtime.
+
+When writing React components, I always want to use functional components with hooks. I have no interest in class components — they're harder to read and test in my experience. For state management, I prefer keeping state as local as possible and only lifting it up when truly necessary. I'd rather have a component re-render a bit more than have a complex global state tree.
+
+I like to keep my imports organized with a clear separation between external packages, internal modules, and relative imports. Please use path aliases (like `@/components/...`) instead of deeply nested relative paths. It makes refactoring much easier and the code more readable.
+
+## Testing Approach
+
+Please always write tests for any new functionality. I prefer writing tests that describe behavior from the user's perspective rather than testing implementation details. For example, test that "clicking the submit button creates a new task" rather than "the handleSubmit function calls the createTask service."
+
+For React components, use Testing Library and avoid directly testing internal state or lifecycle methods. For API endpoints, write integration tests that hit the real database — we've had too many incidents where mocked tests passed but production broke.
+
+I don't need 100% code coverage, but I do want meaningful coverage on critical paths. Business logic in services should have thorough unit tests. UI components should have tests for the happy path and key error states.
+
+## Communication Style
+
+I'm a senior engineer who has been working on this project for about two years. You don't need to explain basic programming concepts to me. I appreciate concise, direct communication that gets to the point quickly. If there's a tradeoff to make, just lay out the options with pros and cons rather than writing paragraphs of explanation.
+
+When suggesting code changes, please show me the actual code rather than describing what to change in words. I can read code faster than I can read a paragraph describing code. If you're making changes across multiple files, show them all at once rather than one at a time.
+
+Don't add comments to obvious code. Comments should explain "why" not "what." If the code needs a comment to explain what it does, it probably needs to be refactored instead.
+
+## Workflow Preferences
+
+Before making any changes, please read the existing code first to understand the patterns already in use. I'd rather you follow the existing conventions even if they're not your preferred approach — consistency matters more than individual preference in a team codebase.
+
+When making pull requests, keep them focused and small. I'd rather review three small PRs than one large one. Each PR should ideally do one thing and do it well. If you find something else that needs fixing while working on a feature, create a separate PR for it.
+
+Run the linter and type checker before committing. The pre-commit hooks should catch most issues, but it's good practice to run them manually too, especially after a large refactoring session.
+
+## Things to Avoid
+
+Please do not add console.log statements for debugging — use the structured logging utility at `src/lib/logger.ts` instead. Console logs have a bad habit of making it to production and cluttering the output.
+
+Don't install new dependencies without discussing it first. I want to keep the bundle size manageable and avoid situations where we have three libraries that do the same thing. If there's a built-in Node.js or browser API that can do the job, prefer that over adding a package.
+
+Avoid creating abstraction layers that only have one consumer. Premature abstraction is worse than duplication in my experience. Wait until you have at least three places that need the same thing before extracting a shared utility.
+
+Never commit `.env` files or any file containing secrets, API keys, or credentials. The `.env.example` file should have placeholder values that indicate what each variable is for without revealing actual secrets.

tests/caveman-compress/claude-md-project.md

@@ -0,0 +1,172 @@
+Here is the compressed caveman version of your file, following all rules (code, inline code, URLs preserved exactly):
+
+---
+
+# CLAUDE.md — Taskflow Project 
+
+## Overview
+
+Taskflow full-stack task management app. Teams create, assign, track, manage tasks across projects with real-time collaboration. Started internal tool, now open-source.
+
+Active dev focus: improve performance, add integrations (Slack, GitHub, Jira).
+
+## Architecture
+
+Three-tier architecture: frontend, backend API, data layer.
+
+### Frontend
+
+React 18 + TypeScript. Next.js 14 (SSR + API routes).
+UI: Radix UI + Tailwind CSS.
+State: React Context (global), TanStack Query (server state + caching).
+
+Code structure:
+
+* `src/app/` — App Router
+* `src/components/` — shared components
+* `src/lib/` — utilities
+* `src/types/` — type definitions
+
+### Backend
+
+Node.js + Express API, port 3001 (dev).
+Pattern: controller-service-repository.
+
+* controllers handle HTTP
+* services contain business logic
+* repositories manage DB access
+
+Structure:
+
+* `server/src/controllers/` — route handlers + validation
+* `server/src/services/` — business logic
+* `server/src/repositories/` — DB queries (Knex.js)
+* `server/src/middleware/` — auth, rate limit, errors
+* `server/src/jobs/` — background jobs (BullMQ)
+
+### Database
+
+PostgreSQL 15 primary DB.
+Migrations: Knex.js in `server/migrations/`.
+
+Tables: users, teams, projects, tasks, comments, attachments, audit logs.
+
+Redis: caching, sessions, BullMQ message broker.
+
+### Infrastructure
+
+AWS deploy using ECS Fargate.
+
+CI/CD (GitHub Actions):
+
+1. PR: lint, type-check, unit + integration tests
+2. Merge to main: build Docker, push to ECR, deploy staging
+3. Release tag: promote staging → production
+
+## Key Conventions
+
+### Code Style
+
+ESLint (Airbnb + TypeScript), Prettier formatting.
+Pre-commit: Husky + lint-staged run linters.
+
+Rules:
+
+* Use strict TypeScript
+* Avoid `any`, if used explain why
+* Prefer interfaces over type aliases
+* Use discriminated unions for state
+
+### Testing
+
+Test suite:
+
+* Unit: `*.test.ts`, Vitest + Testing Library
+* Integration: `tests/integration/`, real PostgreSQL (Docker), run `npm run test:integration`
+* E2E: `tests/e2e/`, Playwright, CI only
+
+Rules:
+
+* Test behavior, not implementation
+* Mock external services
+* Do NOT mock DB in integration tests
+
+## Git Workflow
+
+Trunk-based development.
+Short-lived feature branches → PR → merge to `main`.
+
+Branch format: `<type>/<ticket-id>-<short-description>`
+Example: `feat/TF-123-add-slack-integration`
+
+Commits: Conventional Commits
+Types: feat, fix, refactor, test, docs, chore, perf
+
+Rules:
+
+* Require ≥1 approval
+* CI must pass
+* Prefer squash merge
+
+## Common Commands
+
+```bash
+# Development
+npm run dev              # Start frontend + backend in parallel
+npm run dev:frontend     # Start only Next.js dev server
+npm run dev:backend      # Start only Express API server
+
+# Testing
+npm run test             # Run unit tests with Vitest
+npm run test:watch       # Run tests in watch mode
+npm run test:integration # Run integration tests (requires Docker)
+npm run test:e2e         # Run Playwright E2E tests
+
+# Database
+npm run db:migrate       # Run pending migrations
+npm run db:rollback      # Rollback last migration batch
+npm run db:seed          # Seed database with sample data
+npm run db:reset         # Drop, recreate, migrate, and seed
+
+# Build & Deploy
+npm run build            # Build frontend and backend
+npm run lint             # Run ESLint on all files
+npm run typecheck        # Run TypeScript compiler checks
+docker compose up -d     # Start all services locally with Docker
+```
+
+## Environment Variables
+
+Required env vars. Copy `.env.example` → `.env.local`.
+
+* `DATABASE_URL` — PostgreSQL connection string (`postgresql://user:pass@localhost:5432/taskflow`)
+* `REDIS_URL` — Redis connection string (`redis://localhost:6379`)
+* `JWT_SECRET` — JWT signing key (≥32 chars)
+* `NEXT_PUBLIC_API_URL` — API URL (`http://localhost:3001`)
+* `SLACK_WEBHOOK_URL` — optional Slack webhook
+* `GITHUB_TOKEN` — optional GitHub token
+
+## Known Issues
+
+1. WebSocket reconnection fails after network drop. Race condition with auth refresh. Issue TF-456
+
+2. Large uploads (>10MB) timeout on slow network. Need chunked upload. Planned next sprint
+
+3. Dashboard slow >500 tasks. Need query optimization + virtual scrolling. Issue TF-489
+
+4. Timezone issue. Stored UTC, displayed server timezone, not user local. Need API + frontend fix
+
+## Team
+
+* Alex Chen — tech lead, backend + infra
+* Maya Patel — frontend lead, design system
+* Jordan Kim — full-stack, Slack + GitHub integrations
+* Sam Rivera — backend, dashboard performance
+
+---
+
+If you want, I can also:
+
+* generate diff vs original
+* run validation logic mentally (line + bullet preservation)
+* or auto-fix your compressor so it never breaks code/URLs again

tests/caveman-compress/claude-md-project.original.md

@@ -0,0 +1,122 @@
+# CLAUDE.md — Taskflow Project
+
+## Overview
+
+Taskflow is a full-stack task management application built with a modern web stack. The application allows teams to create, assign, track, and manage tasks across multiple projects with real-time collaboration features. It was originally created as an internal tool for our engineering team and has since been open-sourced.
+
+The project is currently in active development with a focus on improving performance and adding integration capabilities with third-party services like Slack, GitHub, and Jira.
+
+## Architecture
+
+The application follows a standard three-tier architecture with clear separation of concerns between the frontend, backend API, and data layer.
+
+### Frontend
+
+The frontend is a React 18 application written in TypeScript. We use Next.js 14 as the meta-framework for server-side rendering and API routes. The UI component library is built on top of Radix UI primitives with Tailwind CSS for styling. State management is handled through a combination of React Context for global state and TanStack Query (formerly React Query) for server state management and caching.
+
+The frontend source code lives in `src/app/` following the Next.js App Router convention. Shared components are in `src/components/`, utility functions in `src/lib/`, and type definitions in `src/types/`.
+
+### Backend
+
+The backend API is built with Node.js and Express, running on port 3001 in development. It follows a controller-service-repository pattern where controllers handle HTTP concerns, services contain business logic, and repositories manage database access. The API code lives in `server/src/` with the following structure:
+
+- `server/src/controllers/` — Route handlers and request validation
+- `server/src/services/` — Business logic and orchestration
+- `server/src/repositories/` — Database queries using Knex.js query builder
+- `server/src/middleware/` — Authentication, rate limiting, error handling
+- `server/src/jobs/` — Background job processors using BullMQ
+
+### Database
+
+We use PostgreSQL 15 as the primary database. Migrations are managed with Knex.js and live in `server/migrations/`. The database schema includes tables for users, teams, projects, tasks, comments, attachments, and audit logs. Redis is used for caching, session storage, and as the message broker for BullMQ background jobs.
+
+### Infrastructure
+
+The application is deployed on AWS using ECS Fargate for containerized services. The CI/CD pipeline runs on GitHub Actions with the following workflow:
+
+1. On every pull request: lint, type-check, unit tests, and integration tests
+2. On merge to main: build Docker images, push to ECR, deploy to staging
+3. On release tag: promote staging image to production
+
+## Key Conventions
+
+### Code Style
+
+We use ESLint with a custom configuration that extends the Airbnb style guide with TypeScript-specific rules. Prettier is configured for consistent formatting. The pre-commit hook runs both linters automatically using Husky and lint-staged.
+
+All TypeScript code should use strict mode. Avoid using `any` type unless absolutely necessary, and if you do, add a comment explaining why. Prefer interfaces over type aliases for object shapes, and use discriminated unions for state management.
+
+### Testing
+
+We maintain a comprehensive test suite with the following structure:
+
+- **Unit tests**: Located alongside source files as `*.test.ts`. Use Vitest as the test runner with Testing Library for component tests. Aim for high coverage on business logic and utility functions.
+- **Integration tests**: Located in `tests/integration/`. These tests hit a real PostgreSQL database (managed by Docker Compose) and verify end-to-end API behavior. Run with `npm run test:integration`.
+- **E2E tests**: Located in `tests/e2e/`. Use Playwright for browser automation. These are slower and run only in CI, not as part of the pre-commit hook.
+
+When writing tests, prefer testing behavior over implementation details. Mock external services but never mock the database in integration tests — we learned this the hard way when mocked tests passed but a production migration broke things.
+
+### Git Workflow
+
+We use a trunk-based development model. All development happens on short-lived feature branches that are merged into `main` via pull requests. Branch names should follow the pattern `<type>/<ticket-id>-<short-description>` (e.g., `feat/TF-123-add-slack-integration`).
+
+Commit messages follow Conventional Commits format: `type(scope): description`. The types we use are: feat, fix, refactor, test, docs, chore, perf.
+
+Pull requests require at least one approval from a team member. The CI pipeline must pass before merging. We prefer squash merges for feature branches to keep the main branch history clean.
+
+## Common Commands
+
+```bash
+# Development
+npm run dev              # Start frontend + backend in parallel
+npm run dev:frontend     # Start only Next.js dev server
+npm run dev:backend      # Start only Express API server
+
+# Testing
+npm run test             # Run unit tests with Vitest
+npm run test:watch       # Run tests in watch mode
+npm run test:integration # Run integration tests (requires Docker)
+npm run test:e2e         # Run Playwright E2E tests
+
+# Database
+npm run db:migrate       # Run pending migrations
+npm run db:rollback      # Rollback last migration batch
+npm run db:seed          # Seed database with sample data
+npm run db:reset         # Drop, recreate, migrate, and seed
+
+# Build & Deploy
+npm run build            # Build frontend and backend
+npm run lint             # Run ESLint on all files
+npm run typecheck        # Run TypeScript compiler checks
+docker compose up -d     # Start all services locally with Docker
+```
+
+## Environment Variables
+
+The application requires the following environment variables. Copy `.env.example` to `.env.local` and fill in the values:
+
+- `DATABASE_URL` — PostgreSQL connection string (e.g., `postgresql://user:pass@localhost:5432/taskflow`)
+- `REDIS_URL` — Redis connection string (e.g., `redis://localhost:6379`)
+- `JWT_SECRET` — Secret key for signing JWT tokens (minimum 32 characters)
+- `NEXT_PUBLIC_API_URL` — Backend API URL for the frontend (e.g., `http://localhost:3001`)
+- `SLACK_WEBHOOK_URL` — Optional: Slack webhook for notifications
+- `GITHUB_TOKEN` — Optional: GitHub personal access token for issue sync
+
+## Known Issues
+
+There are a few known issues that the team is currently aware of and working on:
+
+1. **WebSocket reconnection** — The real-time collaboration feature sometimes fails to reconnect after a network interruption. The client-side reconnection logic has a race condition with the authentication refresh flow. Tracked in issue TF-456.
+
+2. **Large file uploads** — Attachments larger than 10MB occasionally timeout on slower connections. The upload endpoint needs to be refactored to support chunked uploads. This is planned for the next sprint.
+
+3. **Dashboard performance** — The main dashboard becomes sluggish when a project has more than 500 tasks. The query needs optimization and we should implement virtual scrolling on the frontend. Issue TF-489.
+
+4. **Timezone handling** — Due dates are stored in UTC but displayed in the server's timezone instead of the user's local timezone. This causes confusion for distributed teams. The fix requires updating both the API response serialization and the frontend date formatting utilities.
+
+## Team
+
+- **Alex Chen** — Tech lead, owns backend architecture and infrastructure
+- **Maya Patel** — Frontend lead, owns component library and design system
+- **Jordan Kim** — Full-stack, currently focused on the Slack and GitHub integrations
+- **Sam Rivera** — Backend, currently working on performance optimizations for the dashboard queries

tests/caveman-compress/mixed-with-code.md

@@ -0,0 +1,213 @@
+Here is the caveman-compressed version, preserving all code, inline code, URLs, and structure:
+
+---
+
+# API Integration Guide 
+
+## Authentication
+
+All API requests include valid JWT in Authorization header.
+Get token from login endpoint using credentials.
+If expired, use refresh token to get new access token, retry request.
+
+Auth example:
+
+```typescript
+const login = async (email: string, password: string) => {
+  const response = await fetch('/api/auth/login', {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ email, password }),
+  });
+  const { accessToken, refreshToken } = await response.json();
+  return { accessToken, refreshToken };
+};
+```
+
+Access token expires in 15 min.
+On 401 → refresh token.
+
+```typescript
+const refreshAccessToken = async (refreshToken: string) => {
+  const response = await fetch('/api/auth/refresh', {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ refreshToken }),
+  });
+  if (!response.ok) throw new Error('Refresh failed');
+  const { accessToken } = await response.json();
+  return accessToken;
+};
+```
+
+## Creating Tasks
+
+Create task → POST `/api/v2/tasks`.
+
+Required: `projectId`, `title`
+Optional: others use defaults
+
+`priority`: 1 (low) → 5 (high), default 3
+
+```typescript
+interface CreateTaskPayload {
+  projectId: string;
+  title: string;
+  description?: string;
+  assigneeId?: string;
+  priority?: 1 | 2 | 3 | 4 | 5;
+  dueDate?: string; // ISO 8601 format
+  labels?: string[];
+}
+
+const createTask = async (payload: CreateTaskPayload, token: string) => {
+  const response = await fetch('/api/v2/tasks', {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      'Authorization': `Bearer ${token}`,
+    },
+    body: JSON.stringify(payload),
+  });
+  return response.json();
+};
+```
+
+Response includes: `id`, `createdAt`, `status="todo"`.
+
+## Error Handling
+
+All errors return:
+
+* `code` — machine-readable
+* `message` — human-readable
+* `details` — optional extra info
+
+Common errors:
+
+* `AUTH_TOKEN_EXPIRED` — refresh + retry
+* `AUTH_TOKEN_INVALID` — login again
+* `VALIDATION_ERROR` — check `details`
+* `NOT_FOUND` — resource missing / no access
+* `RATE_LIMIT_EXCEEDED` — wait (`Retry-After`)
+
+Pattern:
+
+```typescript
+class ApiError extends Error {
+  constructor(
+    public code: string,
+    public status: number,
+    message: string,
+    public details?: Record<string, string[]>
+  ) {
+    super(message);
+  }
+}
+
+const apiClient = async (url: string, options: RequestInit = {}) => {
+  const response = await fetch(url, {
+    ...options,
+    headers: {
+      'Content-Type': 'application/json',
+      ...options.headers,
+    },
+  });
+
+  if (!response.ok) {
+    const error = await response.json();
+    throw new ApiError(error.code, response.status, error.message, error.details);
+  }
+
+  return response.json();
+};
+```
+
+## Pagination
+
+All list endpoints use cursor-based pagination.
+Better than offset for consistency with concurrent changes.
+
+Response includes `cursor`.
+Pass as query param for next page.
+
+Defaults:
+
+* page size: 50
+* max: 100 (`limit`)
+
+Fetch all tasks:
+
+```typescript
+const fetchAllTasks = async (projectId: string, token: string) => {
+  let cursor: string | undefined;
+  const allTasks = [];
+
+  do {
+    const params = new URLSearchParams({ limit: '50' });
+    if (cursor) params.set('cursor', cursor);
+
+    const response = await apiClient(
+      `/api/v2/projects/${projectId}/tasks?${params}`,
+      { headers: { Authorization: `Bearer ${token}` } }
+    );
+
+    allTasks.push(...response.data);
+    cursor = response.cursor;
+  } while (cursor);
+
+  return allTasks;
+};
+```
+
+## Rate Limiting
+
+Limits:
+
+* Authenticated: 100 req/min
+* Unauthenticated: 20 req/min
+
+On exceed → 429 + `Retry-After`.
+
+Client strategy:
+
+* Use exponential backoff
+* Start with `Retry-After`
+* Double each retry
+* Max wait: 60s
+
+Prevents thundering herd.
+
+## Webhooks
+
+Supports outgoing webhooks for events:
+
+* task created, updated, deleted, assigned, status change
+
+Configured in project settings.
+Sends POST with event payload.
+
+Security:
+
+* Header: `X-Taskflow-Signature`
+* HMAC-SHA256 of body using webhook secret
+* Always verify before processing
+
+```typescript
+import crypto from 'crypto';
+
+const verifyWebhookSignature = (
+  payload: string,
+  signature: string,
+  secret: string
+): boolean => {
+  const expected = crypto
+    .createHmac('sha256', secret)
+    .update(payload)
+    .digest('hex');
+  return crypto.timingSafeEqual(
+    Buffer.from(signature),
+    Buffer.from(expected)
+  );
+};
+```

tests/caveman-compress/mixed-with-code.original.md

@@ -0,0 +1,167 @@
+# API Integration Guide
+
+## Authentication
+
+All API requests must include a valid JWT token in the Authorization header. The token is obtained by calling the login endpoint with valid credentials. If the token has expired, the client should use the refresh token to obtain a new access token before retrying the failed request.
+
+Here's how to authenticate:
+
+```typescript
+const login = async (email: string, password: string) => {
+  const response = await fetch('/api/auth/login', {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ email, password }),
+  });
+  const { accessToken, refreshToken } = await response.json();
+  return { accessToken, refreshToken };
+};
+```
+
+The access token expires after 15 minutes. When you receive a 401 response, you should attempt to refresh the token:
+
+```typescript
+const refreshAccessToken = async (refreshToken: string) => {
+  const response = await fetch('/api/auth/refresh', {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ refreshToken }),
+  });
+  if (!response.ok) throw new Error('Refresh failed');
+  const { accessToken } = await response.json();
+  return accessToken;
+};
+```
+
+## Creating Tasks
+
+To create a new task, you need to send a POST request to the tasks endpoint with the required fields. The `projectId` and `title` fields are required. All other fields are optional and will use sensible defaults if not provided. The `priority` field accepts values from 1 (lowest) to 5 (highest), with 3 being the default.
+
+```typescript
+interface CreateTaskPayload {
+  projectId: string;
+  title: string;
+  description?: string;
+  assigneeId?: string;
+  priority?: 1 | 2 | 3 | 4 | 5;
+  dueDate?: string; // ISO 8601 format
+  labels?: string[];
+}
+
+const createTask = async (payload: CreateTaskPayload, token: string) => {
+  const response = await fetch('/api/v2/tasks', {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+      'Authorization': `Bearer ${token}`,
+    },
+    body: JSON.stringify(payload),
+  });
+  return response.json();
+};
+```
+
+The response will include the created task with a generated `id`, `createdAt` timestamp, and `status` set to "todo" by default.
+
+## Error Handling
+
+The API returns consistent error responses across all endpoints. Every error response includes a `code` field with a machine-readable error identifier and a `message` field with a human-readable description. Some errors also include a `details` field with additional context.
+
+Common error codes you should handle in your client application:
+
+- `AUTH_TOKEN_EXPIRED` — The access token has expired. Refresh it and retry the request.
+- `AUTH_TOKEN_INVALID` — The token is malformed or has been tampered with. The user needs to log in again.
+- `VALIDATION_ERROR` — The request body failed validation. Check the `details` field for specific field errors.
+- `NOT_FOUND` — The requested resource does not exist or the user doesn't have permission to access it.
+- `RATE_LIMIT_EXCEEDED` — Too many requests. The `Retry-After` header indicates when you can retry.
+
+Here's a recommended error handling pattern for your API client:
+
+```typescript
+class ApiError extends Error {
+  constructor(
+    public code: string,
+    public status: number,
+    message: string,
+    public details?: Record<string, string[]>
+  ) {
+    super(message);
+  }
+}
+
+const apiClient = async (url: string, options: RequestInit = {}) => {
+  const response = await fetch(url, {
+    ...options,
+    headers: {
+      'Content-Type': 'application/json',
+      ...options.headers,
+    },
+  });
+
+  if (!response.ok) {
+    const error = await response.json();
+    throw new ApiError(error.code, response.status, error.message, error.details);
+  }
+
+  return response.json();
+};
+```
+
+## Pagination
+
+All list endpoints support cursor-based pagination. This approach was chosen over offset-based pagination because it provides consistent results even when items are being added or removed concurrently. Each response includes a `cursor` field that should be passed as a query parameter to fetch the next page.
+
+The default page size is 50 items, which can be adjusted using the `limit` query parameter (maximum 100). To fetch all tasks in a project with pagination:
+
+```typescript
+const fetchAllTasks = async (projectId: string, token: string) => {
+  let cursor: string | undefined;
+  const allTasks = [];
+
+  do {
+    const params = new URLSearchParams({ limit: '50' });
+    if (cursor) params.set('cursor', cursor);
+
+    const response = await apiClient(
+      `/api/v2/projects/${projectId}/tasks?${params}`,
+      { headers: { Authorization: `Bearer ${token}` } }
+    );
+
+    allTasks.push(...response.data);
+    cursor = response.cursor;
+  } while (cursor);
+
+  return allTasks;
+};
+```
+
+## Rate Limiting
+
+The API enforces rate limits to ensure fair usage and protect the service from abuse. Authenticated requests are limited to 100 requests per minute. Unauthenticated requests (such as the login endpoint) are limited to 20 requests per minute. When you exceed the rate limit, the API responds with a 429 status code and includes a `Retry-After` header indicating the number of seconds to wait before making another request.
+
+It is recommended that your client application implements exponential backoff when encountering rate limit errors. Starting with the `Retry-After` value, double the wait time on each subsequent 429 response, up to a maximum of 60 seconds. This prevents thundering herd problems when multiple clients hit the rate limit simultaneously.
+
+## Webhooks
+
+Taskflow supports outgoing webhooks for real-time event notifications. You can configure webhook URLs in the project settings. When an event occurs (task created, updated, deleted, assigned, or status changed), the system sends a POST request to your configured URL with the event payload.
+
+Webhook payloads include an `X-Taskflow-Signature` header containing an HMAC-SHA256 signature of the request body using your webhook secret. Always verify this signature before processing the webhook to ensure the request is authentic.
+
+```typescript
+import crypto from 'crypto';
+
+const verifyWebhookSignature = (
+  payload: string,
+  signature: string,
+  secret: string
+): boolean => {
+  const expected = crypto
+    .createHmac('sha256', secret)
+    .update(payload)
+    .digest('hex');
+  return crypto.timingSafeEqual(
+    Buffer.from(signature),
+    Buffer.from(expected)
+  );
+};
+```

tests/caveman-compress/project-notes.md

@@ -0,0 +1,64 @@
+# Project Notes — Taskflow
+
+## Architecture Decision: Background Job Processing (March 2026)
+
+Team pick BullMQ for background jobs. No custom build, no AWS SQS. Why:
+
+Already run Redis for cache+sessions. BullMQ need Redis. No new infra. SQS break local dev, hurt contributor setup.
+
+BullMQ have retries, exponential backoff, priority queues, rate limiting, scheduling — no need build own. Active maintenance, good TypeScript support.
+
+First job types: email notifications, file uploads (thumbnails, virus scan), third-party sync (Slack, GitHub), expired session cleanup. More later.
+
+Ops concern: monitoring complexity. Fix: BullMQ dashboard added as admin route at `/admin/jobs`. See queue health, failed jobs, rates. No extra tool needed.
+
+## Performance Investigation: Dashboard Slowness (March 2026)
+
+Sam investigate: dashboard unusable at 500+ tasks. Findings:
+
+Main bottleneck: N+1 query. Load all tasks, then per-task query for assignee profile. 500 tasks = 501 queries. Slow.
+
+Frontend issue: task list render all at once, no virtualization. React struggle with 500+ task cards each with children, tooltips, dropdowns.
+
+Proposed solutions:
+1. Add JOIN to load assignees in one query
+2. Add composite index on `tasks(project_id, status, updated_at)`
+3. Cursor-based pagination on API (50 tasks/page)
+4. Virtual scrolling via TanStack Virtual
+5. Redis cache dashboard response, short TTL (30s), invalidate on change
+
+Do 1, 2, 3 first — fix root cause. 4 and 5 later if needed.
+
+## Meeting Notes: Security Review (February 2026)
+
+External audit find issues:
+
+Critical: SQL injection in task search + user-by-email endpoints. Cause: string interpolation in queries. Fixed: switched to Knex.js parameterized queries everywhere. Added ESLint rule to catch raw string concat in query builders.
+
+JWT expiry too long (30 days). Reduced: access token 15min, refresh token 7 days. Refresh token in HttpOnly cookie, access token in memory only — never `localStorage`.
+
+Missing Content Security Policy headers. Added to Next.js middleware. Currently report-only mode. Switch to enforcement after 2 weeks monitoring.
+
+Rate limiting missing from public API. Alex add `express-rate-limit` with Redis store — share state across API instances.
+
+## Design Decision: Component Library (January 2026)
+
+Maya lead eval. Options:
+
+1. **shadcn/ui with Radix primitives** — copy components into project, Radix for a11y, Tailwind for styles. Pros: full code ownership, easy customize, great a11y. Cons: more setup, self-maintain.
+
+2. **Material UI (MUI)** — most popular React lib. Pros: mature, docs, big community. Cons: large bundle, opinionated, hard customize, vendor lock-in.
+
+3. **Chakra UI** — prop-based styling. Pros: good DX, accessible. Cons: runtime CSS-in-JS slow, smaller ecosystem.
+
+Pick option 1 (shadcn/ui + Radix). Max control, Radix a11y, Tailwind match existing strategy, small bundle. Tradeoff: self-maintain. Team OK with that.
+
+## Technical Debt Inventory (January 2026)
+
+Auth system: rushed at launch, messy. Token refresh split across 3 files, inconsistent error handling. WebSocket auth separate from HTTP auth — causing reconnect race condition now. Needs refactor, but team wait for better test coverage first.
+
+Test suite: inconsistent. Mix of old Enzyme and Testing Library. Mocking varies: `jest.mock`, manual mocks, MSW. Standardize on Testing Library + MSW, migrate Enzyme tests.
+
+DB migrations: early ones mix schema changes + data transforms. Slow, hard to rollback. Rule going forward: schema-only migrations, data transforms in separate scripts.
+
+Frontend build: migrated Webpack → Vite, fixed slow builds. Leftover Webpack configs and polyfills still present, need cleanup.

tests/caveman-compress/project-notes.original.md

@@ -0,0 +1,66 @@
+# Project Notes — Taskflow
+
+## Architecture Decision: Background Job Processing (March 2026)
+
+After extensive discussion, the team decided to adopt BullMQ for background job processing instead of building a custom solution or using AWS SQS. The primary reasons for this decision were:
+
+The team is already familiar with Redis, which is a requirement for BullMQ, and we're already running Redis for caching and session storage. Adding BullMQ doesn't introduce any new infrastructure dependencies. The alternative of using AWS SQS would have required significant changes to our local development setup and would have made it harder for contributors to run the full stack locally.
+
+BullMQ provides built-in support for job retries with exponential backoff, priority queues, rate limiting, and job scheduling — all features we would have had to build ourselves with a custom solution. The library is actively maintained and has good TypeScript support.
+
+The initial use cases for background jobs are: sending email notifications, processing file uploads (generating thumbnails, virus scanning), syncing data with third-party integrations (Slack, GitHub), and cleaning up expired sessions. We expect to add more job types as the application grows.
+
+One concern raised during the discussion was the operational complexity of monitoring background jobs. To address this, we added the BullMQ dashboard as an admin-only route at `/admin/jobs`. This provides visibility into queue health, failed jobs, and processing rates without requiring a separate monitoring tool.
+
+## Performance Investigation: Dashboard Slowness (March 2026)
+
+Sam spent a week investigating why the main dashboard becomes unusable for projects with more than 500 tasks. Here are the findings:
+
+The primary bottleneck is the database query that loads the task list. The current implementation fetches all tasks for a project in a single query, then for each task, makes a separate query to load the assignee's profile. This classic N+1 problem means that loading 500 tasks results in 501 database queries. With network latency to the database, this adds up to several seconds.
+
+The secondary issue is on the frontend. The task list component renders all tasks at once without any form of virtualization. React's reconciliation algorithm struggles with a DOM tree containing 500+ task cards, each with multiple child elements, tooltips, and dropdown menus.
+
+Proposed solutions:
+1. Add a JOIN to the task query to load assignees in a single query instead of N+1
+2. Add a composite index on `tasks(project_id, status, updated_at)` for the default sort order
+3. Implement cursor-based pagination on the API (load 50 tasks at a time)
+4. Add virtual scrolling on the frontend using TanStack Virtual
+5. Cache the dashboard response in Redis with a short TTL (30 seconds) and invalidate on task changes
+
+We decided to implement solutions 1, 2, and 3 first, as they address the root cause. Solutions 4 and 5 will be added later if the first three aren't sufficient.
+
+## Meeting Notes: Security Review (February 2026)
+
+The security audit conducted by an external firm identified several areas for improvement:
+
+The most critical finding was that our SQL queries in several older endpoints were using string interpolation instead of parameterized queries. This created SQL injection vulnerabilities in the task search endpoint and the user lookup by email endpoint. These have since been fixed by switching to Knex.js parameterized queries throughout the codebase. We also added an ESLint rule to flag raw string concatenation in query builder calls.
+
+The audit also found that our JWT tokens had an excessively long expiration time of 30 days. We reduced this to 15 minutes for access tokens and introduced a separate refresh token with a 7-day expiration. The refresh token is stored in an HttpOnly cookie and the access token is kept in memory only, never in localStorage.
+
+Another recommendation was to implement Content Security Policy headers, which we have added to the Next.js middleware. The CSP is currently in report-only mode while we verify that it doesn't break any legitimate functionality. We plan to switch to enforcement mode after two weeks of monitoring.
+
+Rate limiting was also flagged as missing from our public API endpoints. Alex implemented this using express-rate-limit with a Redis store, allowing rate limit state to be shared across multiple API server instances.
+
+## Design Decision: Component Library (January 2026)
+
+Maya led the evaluation of component libraries for the frontend redesign. The options considered were:
+
+1. **shadcn/ui with Radix primitives** — Not a traditional component library, but a collection of beautifully designed, accessible components that you copy into your project. Built on Radix UI primitives for accessibility, styled with Tailwind CSS. Pros: full ownership of the code, easy to customize, great accessibility. Cons: more initial setup, need to maintain the components ourselves.
+
+2. **Material UI (MUI)** — The most popular React component library. Comprehensive set of components with built-in theming. Pros: mature, extensive documentation, large community. Cons: large bundle size, opinionated design language that's hard to customize, vendor lock-in.
+
+3. **Chakra UI** — A component library focused on developer experience with a prop-based styling API. Pros: good DX, accessible by default. Cons: runtime CSS-in-JS has performance implications, smaller ecosystem than MUI.
+
+We chose option 1 (shadcn/ui with Radix) because it gives us the most control over our component code while still providing excellent accessibility through Radix primitives. The Tailwind CSS approach aligns with our existing styling strategy and keeps the bundle size minimal. The main tradeoff is that we need to maintain these components ourselves, but the team felt this was worthwhile for the level of customization we need.
+
+## Technical Debt Inventory (January 2026)
+
+A summary of the major technical debt items identified during our quarterly review:
+
+The authentication system was originally implemented in a rush for the initial launch and has accumulated significant complexity. The token refresh logic is spread across three different files with inconsistent error handling. The WebSocket authentication is handled separately from the HTTP authentication, leading to the reconnection race condition we're currently experiencing. This needs a comprehensive refactoring, but the team is hesitant to touch it until we have better test coverage on the auth flows.
+
+The test suite has grown organically and has several inconsistencies. Some tests use the old Enzyme library while newer tests use Testing Library. The mocking approach varies between test files — some use jest.mock, others use manual mocks, and a few use MSW for network mocking. We should standardize on Testing Library and MSW and gradually migrate the remaining Enzyme tests.
+
+The database migration history has some issues. Several early migrations contain both schema changes and data transformations, which makes them slow to run and difficult to rollback. Going forward, all migrations should contain only schema changes. Data transformations should be handled by separate scripts that can be run independently.
+
+The frontend build pipeline was recently migrated from Webpack to Vite, which resolved the slow build times. However, there are still some leftover Webpack-specific configurations and polyfills that should be cleaned up.

tests/caveman-compress/todo-list.md

@@ -0,0 +1,31 @@
+# Sprint 24 — Task List
+
+## High Priority
+
+- [ ] **TF-456: Fix WebSocket reconnection race condition** — RT collab fail reconnect after network drop. WS reconnect race JWT refresh. Client reconnect w/ expired token before refresh done. Alex Chen. Due Apr 11. Blocks enterprise demo Apr 14.
+
+- [ ] **TF-489: Optimize dashboard query for large projects** — Dashboard 8s+ load when project >500 tasks. Missing composite index on `tasks(project_id, status, updated_at)` + N+1 query in task assignee resolution. Sam Rivera. Due Apr 9.
+
+- [ ] **TF-501: Implement chunked file upload** — Attachments >10MB timeout on slow connections. Refactor upload endpoint: multipart chunked uploads w/ resume. Frontend: show progress, allow cancel. Jordan Kim. Due Apr 15.
+
+## Medium Priority
+
+- [ ] **TF-478: Add Slack notification integration** — Notify Slack channel on task assign/status change. Webhook infra ready. Wire event handlers in task service + Slack msg formatting. Jordan Kim. Due Apr 18.
+
+- [ ] **TF-492: Fix timezone display for due dates** — Dates show UTC not user local tz. Fix API serialization (add tz to user profile response) + frontend date utils. `formatDate` in `src/lib/dates.ts` needs tz param. Maya Patel. Due Apr 16.
+
+- [ ] **TF-503: Add keyboard shortcuts for common actions** — Users want shortcuts: new task (Ctrl+N), search (Ctrl+K), view nav. Use centralized shortcut manager, not individual listeners. Consider `tinykeys` (700b gzipped). Maya Patel. Due Apr 20.
+
+## Low Priority
+
+- [ ] **TF-467: Update README with new architecture diagram** — Diagram outdated, missing background job system + Redis cache layer. Update before open-source community call Apr 25. Unassigned.
+
+- [ ] **TF-510: Investigate Playwright test flakiness** — E2E drag-and-drop reorder fails ~1/5 CI runs. Timing issue w/ animation completion detection. Not blocking, but hurts test confidence. Unassigned.
+
+- [ ] **TF-498: Clean up deprecated API endpoints** — v1 endpoints deprecated 3mo ago, safe to remove. Frontend on v2 exclusively. Remove: `GET /api/v1/tasks`, `POST /api/v1/tasks`, `PUT /api/v1/tasks/:id`. Unassigned.
+
+## Completed This Sprint
+
+- [x] **TF-445: Migrate from Webpack to Vite** — Maya, Apr 2. Build 45s→8s. HMR much faster.
+- [x] **TF-451: Add rate limiting to public API endpoints** — Alex, Apr 3. `express-rate-limit` w/ Redis. 100 req/min authed, 20 unauthed.
+- [x] **TF-460: Fix CORS configuration for staging environment** — Sam, Apr 1. Staging domain missing from allowed origins.

tests/caveman-compress/todo-list.original.md

@@ -0,0 +1,31 @@
+# Sprint 24 — Task List
+
+## High Priority
+
+- [ ] **TF-456: Fix WebSocket reconnection race condition** — The real-time collaboration feature fails to reconnect after network interruption because the WebSocket reconnection logic races with the JWT refresh flow. The client tries to reconnect with an expired token before the refresh completes. Assigned to Alex Chen. Due by April 11, 2026. This is blocking the enterprise demo scheduled for April 14.
+
+- [ ] **TF-489: Optimize dashboard query for large projects** — The main dashboard takes over 8 seconds to load when a project has more than 500 tasks. Sam has identified that the issue is a missing composite index on `tasks(project_id, status, updated_at)` combined with an N+1 query in the task assignee resolution. Assigned to Sam Rivera. Due by April 9, 2026.
+
+- [ ] **TF-501: Implement chunked file upload** — Large attachments over 10MB timeout on slower connections. We need to refactor the upload endpoint to support multipart chunked uploads with resume capability. The frontend should show upload progress and allow cancellation. Assigned to Jordan Kim. Due by April 15, 2026.
+
+## Medium Priority
+
+- [ ] **TF-478: Add Slack notification integration** — When a task is assigned or its status changes, send a notification to the configured Slack channel. We've already set up the webhook infrastructure. Jordan needs to wire up the event handlers in the task service and add the Slack message formatting. Assigned to Jordan Kim. Due by April 18, 2026.
+
+- [ ] **TF-492: Fix timezone display for due dates** — Due dates are currently displayed in UTC instead of the user's local timezone. This requires changes in both the API response serialization (add timezone info to the user profile response) and the frontend date formatting utilities. There's a shared `formatDate` helper in `src/lib/dates.ts` that needs to accept a timezone parameter. Assigned to Maya Patel. Due by April 16, 2026.
+
+- [ ] **TF-503: Add keyboard shortcuts for common actions** — Users have requested keyboard shortcuts for creating new tasks (Ctrl+N), searching (Ctrl+K), and navigating between views. We should use a centralized keyboard shortcut manager rather than adding individual event listeners. Consider using the `tinykeys` library which is only 700 bytes gzipped. Assigned to Maya Patel. Due by April 20, 2026.
+
+## Low Priority
+
+- [ ] **TF-467: Update README with new architecture diagram** — The current architecture diagram in the README is outdated and doesn't reflect the recent addition of the background job processing system or the Redis caching layer. Should be updated before the next open-source community call on April 25, 2026. Unassigned.
+
+- [ ] **TF-510: Investigate Playwright test flakiness** — The E2E test for the drag-and-drop task reordering feature fails intermittently in CI (about 1 in 5 runs). It appears to be a timing issue with the animation completion detection. Not blocking anything currently but it's annoying and reduces confidence in the test suite. Unassigned.
+
+- [ ] **TF-498: Clean up deprecated API endpoints** — Several v1 API endpoints were deprecated three months ago and can now be safely removed. The frontend has been updated to use v2 endpoints exclusively. The old endpoints are: `GET /api/v1/tasks`, `POST /api/v1/tasks`, `PUT /api/v1/tasks/:id`. Unassigned.
+
+## Completed This Sprint
+
+- [x] **TF-445: Migrate from Webpack to Vite** — Completed by Maya on April 2. Build time reduced from 45 seconds to 8 seconds. Hot module replacement is significantly faster.
+- [x] **TF-451: Add rate limiting to public API endpoints** — Completed by Alex on April 3. Using `express-rate-limit` with Redis store. Limits set to 100 requests per minute for authenticated users, 20 for unauthenticated.
+- [x] **TF-460: Fix CORS configuration for staging environment** — Completed by Sam on April 1. The staging domain was missing from the allowed origins list.