mirror of
https://github.com/SuperClaude-Org/SuperClaude_Framework.git
synced 2025-12-29 16:16:08 +00:00
refactor: PM Agent complete independence from external MCP servers (#439)
* refactor: PM Agent complete independence from external MCP servers ## Summary Implement graceful degradation to ensure PM Agent operates fully without any MCP server dependencies. MCP servers now serve as optional enhancements rather than required components. ## Changes ### Responsibility Separation (NEW) - **PM Agent**: Development workflow orchestration (PDCA cycle, task management) - **mindbase**: Memory management (long-term, freshness, error learning) - **Built-in memory**: Session-internal context (volatile) ### 3-Layer Memory Architecture with Fallbacks 1. **Built-in Memory** [OPTIONAL]: Session context via MCP memory server 2. **mindbase** [OPTIONAL]: Long-term semantic search via airis-mcp-gateway 3. **Local Files** [ALWAYS]: Core functionality in docs/memory/ ### Graceful Degradation Implementation - All MCP operations marked with [ALWAYS] or [OPTIONAL] - Explicit IF/ELSE fallback logic for every MCP call - Dual storage: Always write to local files + optionally to mindbase - Smart lookup: Semantic search (if available) → Text search (always works) ### Key Fallback Strategies **Session Start**: - mindbase available: search_conversations() for semantic context - mindbase unavailable: Grep docs/memory/*.jsonl for text-based lookup **Error Detection**: - mindbase available: Semantic search for similar past errors - mindbase unavailable: Grep docs/mistakes/ + solutions_learned.jsonl **Knowledge Capture**: - Always: echo >> docs/memory/patterns_learned.jsonl (persistent) - Optional: mindbase.store() for semantic search enhancement ## Benefits - ✅ Zero external dependencies (100% functionality without MCP) - ✅ Enhanced capabilities when MCPs available (semantic search, freshness) - ✅ No functionality loss, only reduced search intelligence - ✅ Transparent degradation (no error messages, automatic fallback) ## Related Research - Serena MCP investigation: Exposes tools (not resources), memory = markdown files - mindbase superiority: PostgreSQL + pgvector > Serena memory features - Best practices alignment: /Users/kazuki/github/airis-mcp-gateway/docs/mcp-best-practices.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: add PR template and pre-commit config - Add structured PR template with Git workflow checklist - Add pre-commit hooks for secret detection and Conventional Commits - Enforce code quality gates (YAML/JSON/Markdown lint, shellcheck) NOTE: Execute pre-commit inside Docker container to avoid host pollution: docker compose exec workspace uv tool install pre-commit docker compose exec workspace pre-commit run --all-files 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: update PM Agent context with token efficiency architecture - Add Layer 0 Bootstrap (150 tokens, 95% reduction) - Document Intent Classification System (5 complexity levels) - Add Progressive Loading strategy (5-layer) - Document mindbase integration incentive (38% savings) - Update with 2025-10-17 redesign details * refactor: PM Agent command with progressive loading - Replace auto-loading with User Request First philosophy - Add 5-layer progressive context loading - Implement intent classification system - Add workflow metrics collection (.jsonl) - Document graceful degradation strategy * fix: installer improvements Update installer logic for better reliability * docs: add comprehensive development documentation - Add architecture overview - Add PM Agent improvements analysis - Add parallel execution architecture - Add CLI install improvements - Add code style guide - Add project overview - Add install process analysis * docs: add research documentation Add LLM agent token efficiency research and analysis * docs: add suggested commands reference * docs: add session logs and testing documentation - Add session analysis logs - Add testing documentation * feat: migrate CLI to typer + rich for modern UX ## What Changed ### New CLI Architecture (typer + rich) - Created `superclaude/cli/` module with modern typer-based CLI - Replaced custom UI utilities with rich native features - Added type-safe command structure with automatic validation ### Commands Implemented - **install**: Interactive installation with rich UI (progress, panels) - **doctor**: System diagnostics with rich table output - **config**: API key management with format validation ### Technical Improvements - Dependencies: Added typer>=0.9.0, rich>=13.0.0, click>=8.0.0 - Entry Point: Updated pyproject.toml to use `superclaude.cli.app:cli_main` - Tests: Added comprehensive smoke tests (11 passed) ### User Experience Enhancements - Rich formatted help messages with panels and tables - Automatic input validation with retry loops - Clear error messages with actionable suggestions - Non-interactive mode support for CI/CD ## Testing ```bash uv run superclaude --help # ✓ Works uv run superclaude doctor # ✓ Rich table output uv run superclaude config show # ✓ API key management pytest tests/test_cli_smoke.py # ✓ 11 passed, 1 skipped ``` ## Migration Path - ✅ P0: Foundation complete (typer + rich + smoke tests) - 🔜 P1: Pydantic validation models (next sprint) - 🔜 P2: Enhanced error messages (next sprint) - 🔜 P3: API key retry loops (next sprint) ## Performance Impact - **Code Reduction**: Prepared for -300 lines (custom UI → rich) - **Type Safety**: Automatic validation from type hints - **Maintainability**: Framework primitives vs custom code 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: consolidate documentation directories Merged claudedocs/ into docs/research/ for consistent documentation structure. Changes: - Moved all claudedocs/*.md files to docs/research/ - Updated all path references in documentation (EN/KR) - Updated RULES.md and research.md command templates - Removed claudedocs/ directory - Removed ClaudeDocs/ from .gitignore Benefits: - Single source of truth for all research reports - PEP8-compliant lowercase directory naming - Clearer documentation organization - Prevents future claudedocs/ directory creation 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * perf: reduce /sc:pm command output from 1652 to 15 lines - Remove 1637 lines of documentation from command file - Keep only minimal bootstrap message - 99% token reduction on command execution - Detailed specs remain in superclaude/agents/pm-agent.md 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * perf: split PM Agent into execution workflows and guide - Reduce pm-agent.md from 735 to 429 lines (42% reduction) - Move philosophy/examples to docs/agents/pm-agent-guide.md - Execution workflows (PDCA, file ops) stay in pm-agent.md - Guide (examples, quality standards) read once when needed Token savings: - Agent loading: ~6K → ~3.5K tokens (42% reduction) - Total with pm.md: 71% overall reduction 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: consolidate PM Agent optimization and pending changes PM Agent optimization (already committed separately): - superclaude/commands/pm.md: 1652→14 lines - superclaude/agents/pm-agent.md: 735→429 lines - docs/agents/pm-agent-guide.md: new guide file Other pending changes: - setup: framework_docs, mcp, logger, remove ui.py - superclaude: __main__, cli/app, cli/commands/install - tests: test_ui updates - scripts: workflow metrics analysis tools - docs/memory: session state updates 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: simplify MCP installer to unified gateway with legacy mode ## Changes ### MCP Component (setup/components/mcp.py) - Simplified to single airis-mcp-gateway by default - Added legacy mode for individual official servers (sequential-thinking, context7, magic, playwright) - Dynamic prerequisites based on mode: - Default: uv + claude CLI only - Legacy: node (18+) + npm + claude CLI - Removed redundant server definitions ### CLI Integration - Added --legacy flag to setup/cli/commands/install.py - Added --legacy flag to superclaude/cli/commands/install.py - Config passes legacy_mode to component installer ## Benefits - ✅ Simpler: 1 gateway vs 9+ individual servers - ✅ Lighter: No Node.js/npm required (default mode) - ✅ Unified: All tools in one gateway (sequential-thinking, context7, magic, playwright, serena, morphllm, tavily, chrome-devtools, git, puppeteer) - ✅ Flexible: --legacy flag for official servers if needed ## Usage ```bash superclaude install # Default: airis-mcp-gateway (推奨) superclaude install --legacy # Legacy: individual official servers ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: rename CoreComponent to FrameworkDocsComponent and add PM token tracking ## Changes ### Component Renaming (setup/components/) - Renamed CoreComponent → FrameworkDocsComponent for clarity - Updated all imports in __init__.py, agents.py, commands.py, mcp_docs.py, modes.py - Better reflects the actual purpose (framework documentation files) ### PM Agent Enhancement (superclaude/commands/pm.md) - Added token usage tracking instructions - PM Agent now reports: 1. Current token usage from system warnings 2. Percentage used (e.g., "27% used" for 54K/200K) 3. Status zone: 🟢 <75% | 🟡 75-85% | 🔴 >85% - Helps prevent token exhaustion during long sessions ### UI Utilities (setup/utils/ui.py) - Added new UI utility module for installer - Provides consistent user interface components ## Benefits - ✅ Clearer component naming (FrameworkDocs vs Core) - ✅ PM Agent token awareness for efficiency - ✅ Better visual feedback with status zones 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor(pm-agent): minimize output verbosity (471→284 lines, 40% reduction) **Problem**: PM Agent generated excessive output with redundant explanations - "System Status Report" with decorative formatting - Repeated "Common Tasks" lists user already knows - Verbose session start/end protocols - Duplicate file operations documentation **Solution**: Compress without losing functionality - Session Start: Reduced to symbol-only status (🟢 branch | nM nD | token%) - Session End: Compressed to essential actions only - File Operations: Consolidated from 2 sections to 1 line reference - Self-Improvement: 5 phases → 1 unified workflow - Output Rules: Explicit constraints to prevent Claude over-explanation **Quality Preservation**: - ✅ All core functions retained (PDCA, memory, patterns, mistakes) - ✅ PARALLEL Read/Write preserved (performance critical) - ✅ Workflow unchanged (session lifecycle intact) - ✅ Added output constraints (prevents verbose generation) **Reduction Method**: - Deleted: Explanatory text, examples, redundant sections - Retained: Action definitions, file paths, core workflows - Added: Explicit output constraints to enforce minimalism **Token Impact**: 40% reduction in agent documentation size **Before**: Verbose multi-section report with task lists **After**: Single line status: 🟢 integration | 15M 17D | 36% 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: consolidate MCP integration to unified gateway **Changes**: - Remove individual MCP server docs (superclaude/mcp/*.md) - Remove MCP server configs (superclaude/mcp/configs/*.json) - Delete MCP docs component (setup/components/mcp_docs.py) - Simplify installer (setup/core/installer.py) - Update components for unified gateway approach **Rationale**: - Unified gateway (airis-mcp-gateway) provides all MCP servers - Individual docs/configs no longer needed (managed centrally) - Reduces maintenance burden and file count - Simplifies installation process **Files Removed**: 17 MCP files (docs + configs) **Installer Changes**: Removed legacy MCP installation logic 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: update version and component metadata - Bump version (pyproject.toml, setup/__init__.py) - Update CLAUDE.md import service references - Reflect component structure changes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: kazuki <kazuki@kazukinoMacBook-Air.local> Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
kazuki nakai
@@ -0,0 +1,833 @@
|
||||
# Research: Python Directory Naming & Automation Tools (2025)
|
||||
|
||||
**Research Date**: 2025-10-14
|
||||
**Research Context**: PEP 8 directory naming compliance, automated linting tools, and Git case-sensitive renaming best practices
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
### Key Findings
|
||||
|
||||
1. **PEP 8 Standard (2024-2025)**:
|
||||
- Packages (directories): **lowercase only**, underscores discouraged but widely used in practice
|
||||
- Modules (files): **lowercase**, underscores allowed and common for readability
|
||||
- Current violations: `Developer-Guide`, `Getting-Started`, `User-Guide`, `Reference`, `Templates` (use hyphens/uppercase)
|
||||
|
||||
2. **Automated Linting Tool**: **Ruff** is the 2025 industry standard
|
||||
- Written in Rust, 10-100x faster than Flake8
|
||||
- 800+ built-in rules, replaces Flake8, Black, isort, pyupgrade, autoflake
|
||||
- Configured via `pyproject.toml`
|
||||
- **BUT**: No built-in rules for directory naming validation
|
||||
|
||||
3. **Git Case-Sensitive Rename**: **Two-step `git mv` method**
|
||||
- macOS APFS is case-insensitive by default
|
||||
- Safest approach: `git mv foo foo-tmp && git mv foo-tmp bar`
|
||||
- Alternative: `git rm --cached` + `git add .` (less reliable)
|
||||
|
||||
4. **Automation Strategy**: Custom pre-commit hooks + manual rename
|
||||
- Use `check-case-conflict` pre-commit hook
|
||||
- Write custom Python validator for directory naming
|
||||
- Integrate with `validate-pyproject` for configuration validation
|
||||
|
||||
5. **Modern Project Structure (uv/2025)**:
|
||||
- src-based layout: `src/package_name/` (recommended)
|
||||
- Configuration: `pyproject.toml` (universal standard)
|
||||
- Lockfile: `uv.lock` (cross-platform, committed to Git)
|
||||
|
||||
---
|
||||
|
||||
## Detailed Findings
|
||||
|
||||
### 1. PEP 8 Directory Naming Conventions
|
||||
|
||||
**Official Standard** (PEP 8 - https://peps.python.org/pep-0008/):
|
||||
> "Python packages should also have short, all-lowercase names, although the use of underscores is discouraged."
|
||||
|
||||
**Practical Reality**:
|
||||
- Underscores are widely used in practice (e.g., `sqlalchemy_searchable`)
|
||||
- Community doesn't consider underscores poor practice
|
||||
- **Hyphens are NOT allowed** in package names (Python import restrictions)
|
||||
- **Camel Case / Title Case = PEP 8 violation**
|
||||
|
||||
**Current SuperClaude Framework Violations**:
|
||||
```yaml
|
||||
# ❌ PEP 8 Violations
|
||||
docs/Developer-Guide/ # Contains hyphen + uppercase
|
||||
docs/Getting-Started/ # Contains hyphen + uppercase
|
||||
docs/User-Guide/ # Contains hyphen + uppercase
|
||||
docs/User-Guide-jp/ # Contains hyphen + uppercase
|
||||
docs/User-Guide-kr/ # Contains hyphen + uppercase
|
||||
docs/User-Guide-zh/ # Contains hyphen + uppercase
|
||||
docs/Reference/ # Contains uppercase
|
||||
docs/Templates/ # Contains uppercase
|
||||
|
||||
# ✅ PEP 8 Compliant (Already Fixed)
|
||||
docs/developer-guide/ # lowercase + hyphen (acceptable for docs)
|
||||
docs/getting-started/ # lowercase + hyphen (acceptable for docs)
|
||||
docs/development/ # lowercase only
|
||||
```
|
||||
|
||||
**Documentation Directories Exception**:
|
||||
- Documentation directories (`docs/`) are NOT Python packages
|
||||
- Hyphens are acceptable in non-package directories
|
||||
- Best practice: Use lowercase + hyphens for readability
|
||||
- Example: `docs/getting-started/`, `docs/user-guide/`
|
||||
|
||||
---
|
||||
|
||||
### 2. Automated Linting Tools (2024-2025)
|
||||
|
||||
#### Ruff - The Modern Standard
|
||||
|
||||
**Overview**:
|
||||
- Released: 2023, rapidly adopted as industry standard by 2024-2025
|
||||
- Speed: 10-100x faster than Flake8 (written in Rust)
|
||||
- Replaces: Flake8, Black, isort, pydocstyle, pyupgrade, autoflake
|
||||
- Rules: 800+ built-in rules
|
||||
- Configuration: `pyproject.toml` or `ruff.toml`
|
||||
|
||||
**Key Features**:
|
||||
```yaml
|
||||
Autofix:
|
||||
- Automatic import sorting
|
||||
- Unused variable removal
|
||||
- Python syntax upgrades
|
||||
- Code formatting
|
||||
|
||||
Per-Directory Configuration:
|
||||
- Different rules for different directories
|
||||
- Per-file-target-version settings
|
||||
- Namespace package support
|
||||
|
||||
Exclusions (default):
|
||||
- .git, .venv, build, dist, node_modules
|
||||
- __pycache__, .pytest_cache, .mypy_cache
|
||||
- Custom patterns via glob
|
||||
```
|
||||
|
||||
**Configuration Example** (`pyproject.toml`):
|
||||
```toml
|
||||
[tool.ruff]
|
||||
line-length = 88
|
||||
target-version = "py38"
|
||||
|
||||
exclude = [
|
||||
".git",
|
||||
".venv",
|
||||
"build",
|
||||
"dist",
|
||||
]
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I", "N"] # N = naming conventions
|
||||
ignore = ["E501"] # Line too long
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"__init__.py" = ["F401"] # Unused imports OK in __init__.py
|
||||
"tests/*" = ["N802"] # Function name conventions relaxed in tests
|
||||
```
|
||||
|
||||
**Naming Convention Rules** (`N` prefix):
|
||||
```yaml
|
||||
N801: Class names should use CapWords convention
|
||||
N802: Function names should be lowercase
|
||||
N803: Argument names should be lowercase
|
||||
N804: First argument of classmethod should be cls
|
||||
N805: First argument of method should be self
|
||||
N806: Variable in function should be lowercase
|
||||
N807: Function name should not start/end with __
|
||||
|
||||
BUT: No rules for directory naming (non-Python file checks)
|
||||
```
|
||||
|
||||
**Limitation**: Ruff validates **Python code**, not directory structure.
|
||||
|
||||
---
|
||||
|
||||
#### validate-pyproject - Configuration Validator
|
||||
|
||||
**Purpose**: Validates `pyproject.toml` compliance with PEP standards
|
||||
|
||||
**Installation**:
|
||||
```bash
|
||||
pip install validate-pyproject
|
||||
# or with pre-commit integration
|
||||
```
|
||||
|
||||
**Usage**:
|
||||
```bash
|
||||
# CLI
|
||||
validate-pyproject pyproject.toml
|
||||
|
||||
# Python API
|
||||
from validate_pyproject import validate
|
||||
validate(data)
|
||||
```
|
||||
|
||||
**Pre-commit Hook**:
|
||||
```yaml
|
||||
# .pre-commit-config.yaml
|
||||
repos:
|
||||
- repo: https://github.com/abravalheri/validate-pyproject
|
||||
rev: v0.16
|
||||
hooks:
|
||||
- id: validate-pyproject
|
||||
```
|
||||
|
||||
**What It Validates**:
|
||||
- PEP 517/518 build system configuration
|
||||
- PEP 621 project metadata
|
||||
- Tool-specific configurations ([tool.ruff], [tool.mypy])
|
||||
- JSON Schema compliance
|
||||
|
||||
**Limitation**: Validates `pyproject.toml` syntax, not directory naming.
|
||||
|
||||
---
|
||||
|
||||
### 3. Git Case-Sensitive Rename Best Practices
|
||||
|
||||
**The Problem**:
|
||||
- macOS APFS: case-insensitive by default
|
||||
- Git: case-sensitive internally
|
||||
- Result: `git mv Foo foo` doesn't work directly
|
||||
- Risk: Breaking changes across systems
|
||||
|
||||
**Best Practice #1: Two-Step git mv (Safest)**
|
||||
|
||||
```bash
|
||||
# Step 1: Rename to temporary name
|
||||
git mv docs/User-Guide docs/user-guide-tmp
|
||||
|
||||
# Step 2: Rename to final name
|
||||
git mv docs/user-guide-tmp docs/user-guide
|
||||
|
||||
# Commit
|
||||
git commit -m "refactor: rename User-Guide to user-guide (PEP 8 compliance)"
|
||||
```
|
||||
|
||||
**Why This Works**:
|
||||
- First rename: Different enough for case-insensitive FS to recognize
|
||||
- Second rename: Achieves desired final name
|
||||
- Git tracks both renames correctly
|
||||
- No data loss risk
|
||||
|
||||
**Best Practice #2: Cache Clearing (Alternative)**
|
||||
|
||||
```bash
|
||||
# Remove from Git index (keeps working tree)
|
||||
git rm -r --cached .
|
||||
|
||||
# Re-add all files (Git detects renames)
|
||||
git add .
|
||||
|
||||
# Commit
|
||||
git commit -m "refactor: fix directory naming case sensitivity"
|
||||
```
|
||||
|
||||
**Why This Works**:
|
||||
- Git re-scans working tree
|
||||
- Detects same content = rename (not delete + add)
|
||||
- Preserves file history
|
||||
|
||||
**What NOT to Do**:
|
||||
|
||||
```bash
|
||||
# ❌ DANGEROUS: Disabling core.ignoreCase
|
||||
git config core.ignoreCase false
|
||||
|
||||
# Risk: Unexpected behavior on case-insensitive filesystems
|
||||
# Official docs warning: "modifying this value may result in unexpected behavior"
|
||||
```
|
||||
|
||||
**Advanced Workaround (Overkill)**:
|
||||
- Create case-sensitive APFS volume via Disk Utility
|
||||
- Clone repository to case-sensitive volume
|
||||
- Perform renames normally
|
||||
- Push to remote
|
||||
|
||||
---
|
||||
|
||||
### 4. Pre-commit Hooks for Structure Validation
|
||||
|
||||
#### Built-in Hooks (check-case-conflict)
|
||||
|
||||
**Official pre-commit-hooks** (https://github.com/pre-commit/pre-commit-hooks):
|
||||
|
||||
```yaml
|
||||
# .pre-commit-config.yaml
|
||||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.5.0
|
||||
hooks:
|
||||
- id: check-case-conflict # Detects case sensitivity issues
|
||||
- id: check-illegal-windows-names # Windows filename validation
|
||||
- id: check-symlinks # Symlink integrity
|
||||
- id: destroyed-symlinks # Broken symlinks detection
|
||||
- id: check-added-large-files # Prevent large file commits
|
||||
- id: check-yaml # YAML syntax validation
|
||||
- id: end-of-file-fixer # Ensure newline at EOF
|
||||
- id: trailing-whitespace # Remove trailing spaces
|
||||
```
|
||||
|
||||
**check-case-conflict Details**:
|
||||
- Detects files that differ only in case
|
||||
- Example: `README.md` vs `readme.md`
|
||||
- Prevents issues on case-insensitive filesystems
|
||||
- Runs before commit, blocks if conflicts found
|
||||
|
||||
**Limitation**: Only detects conflicts, doesn't enforce naming conventions.
|
||||
|
||||
---
|
||||
|
||||
#### Custom Hook: Directory Naming Validator
|
||||
|
||||
**Purpose**: Enforce PEP 8 directory naming conventions
|
||||
|
||||
**Implementation** (`scripts/validate_directory_names.py`):
|
||||
|
||||
```python
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Pre-commit hook to validate directory naming conventions.
|
||||
Enforces PEP 8 compliance for Python packages.
|
||||
"""
|
||||
import sys
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
# PEP 8: Package names should be lowercase, underscores discouraged
|
||||
PACKAGE_NAME_PATTERN = re.compile(r'^[a-z][a-z0-9_]*$')
|
||||
|
||||
# Documentation directories: lowercase + hyphens allowed
|
||||
DOC_NAME_PATTERN = re.compile(r'^[a-z][a-z0-9\-]*$')
|
||||
|
||||
def validate_directory_names(root_dir='.'):
|
||||
"""Validate directory naming conventions."""
|
||||
violations = []
|
||||
|
||||
root = Path(root_dir)
|
||||
|
||||
# Check Python package directories
|
||||
for pydir in root.rglob('__init__.py'):
|
||||
package_dir = pydir.parent
|
||||
package_name = package_dir.name
|
||||
|
||||
if not PACKAGE_NAME_PATTERN.match(package_name):
|
||||
violations.append(
|
||||
f"PEP 8 violation: Package '{package_dir}' should be lowercase "
|
||||
f"(current: '{package_name}')"
|
||||
)
|
||||
|
||||
# Check documentation directories
|
||||
docs_root = root / 'docs'
|
||||
if docs_root.exists():
|
||||
for doc_dir in docs_root.iterdir():
|
||||
if doc_dir.is_dir() and doc_dir.name not in ['.git', '__pycache__']:
|
||||
if not DOC_NAME_PATTERN.match(doc_dir.name):
|
||||
violations.append(
|
||||
f"Documentation naming violation: '{doc_dir}' should be "
|
||||
f"lowercase with hyphens (current: '{doc_dir.name}')"
|
||||
)
|
||||
|
||||
return violations
|
||||
|
||||
def main():
|
||||
violations = validate_directory_names()
|
||||
|
||||
if violations:
|
||||
print("❌ Directory naming convention violations found:\n")
|
||||
for violation in violations:
|
||||
print(f" - {violation}")
|
||||
print("\n" + "="*70)
|
||||
print("Fix: Rename directories to lowercase (hyphens for docs, underscores for packages)")
|
||||
print("="*70)
|
||||
return 1
|
||||
|
||||
print("✅ All directory names comply with PEP 8 conventions")
|
||||
return 0
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
```
|
||||
|
||||
**Pre-commit Configuration**:
|
||||
|
||||
```yaml
|
||||
# .pre-commit-config.yaml
|
||||
repos:
|
||||
# Official hooks
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.5.0
|
||||
hooks:
|
||||
- id: check-case-conflict
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
|
||||
# Ruff linter
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.1.9
|
||||
hooks:
|
||||
- id: ruff
|
||||
args: [--fix, --exit-non-zero-on-fix]
|
||||
- id: ruff-format
|
||||
|
||||
# Custom directory naming validator
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: validate-directory-names
|
||||
name: Validate Directory Naming
|
||||
entry: python scripts/validate_directory_names.py
|
||||
language: system
|
||||
pass_filenames: false
|
||||
always_run: true
|
||||
```
|
||||
|
||||
**Installation**:
|
||||
|
||||
```bash
|
||||
# Install pre-commit
|
||||
pip install pre-commit
|
||||
|
||||
# Install hooks to .git/hooks/
|
||||
pre-commit install
|
||||
|
||||
# Run manually on all files
|
||||
pre-commit run --all-files
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Modern Python Project Structure (uv/2025)
|
||||
|
||||
#### Standard Layout (uv recommended)
|
||||
|
||||
```
|
||||
project-root/
|
||||
├── .git/
|
||||
├── .gitignore
|
||||
├── .python-version # Python version for uv
|
||||
├── pyproject.toml # Project metadata + tool configs
|
||||
├── uv.lock # Cross-platform lockfile (commit this)
|
||||
├── README.md
|
||||
├── LICENSE
|
||||
├── .pre-commit-config.yaml # Pre-commit hooks
|
||||
├── src/ # Source code (src-based layout)
|
||||
│ └── package_name/
|
||||
│ ├── __init__.py
|
||||
│ ├── module1.py
|
||||
│ └── subpackage/
|
||||
│ ├── __init__.py
|
||||
│ └── module2.py
|
||||
├── tests/ # Test files
|
||||
│ ├── __init__.py
|
||||
│ ├── test_module1.py
|
||||
│ └── test_module2.py
|
||||
├── docs/ # Documentation
|
||||
│ ├── getting-started/ # lowercase + hyphens OK
|
||||
│ ├── user-guide/
|
||||
│ └── developer-guide/
|
||||
├── scripts/ # Utility scripts
|
||||
│ └── validate_directory_names.py
|
||||
└── .venv/ # Virtual environment (local to project)
|
||||
```
|
||||
|
||||
**Key Files**:
|
||||
|
||||
**pyproject.toml** (modern standard):
|
||||
```toml
|
||||
[build-system]
|
||||
requires = ["setuptools>=61.0", "wheel"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "package-name" # lowercase, hyphens allowed for non-importable
|
||||
version = "1.0.0"
|
||||
requires-python = ">=3.8"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
include = ["package_name*"] # lowercase_underscore for Python packages
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 88
|
||||
target-version = "py38"
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = ["E", "F", "W", "I", "N"]
|
||||
```
|
||||
|
||||
**uv.lock**:
|
||||
- Cross-platform lockfile
|
||||
- Contains exact resolved versions
|
||||
- **Must be committed to version control**
|
||||
- Ensures reproducible installations
|
||||
|
||||
**.python-version**:
|
||||
```
|
||||
3.12
|
||||
```
|
||||
|
||||
**Benefits of src-based layout**:
|
||||
1. **Namespace isolation**: Prevents import conflicts
|
||||
2. **Testability**: Tests import from installed package, not source
|
||||
3. **Modularity**: Clear separation of application logic
|
||||
4. **Distribution**: Required for PyPI publishing
|
||||
5. **Editor support**: .venv in project root helps IDEs find packages
|
||||
|
||||
---
|
||||
|
||||
## Recommendations for SuperClaude Framework
|
||||
|
||||
### Immediate Actions (Required)
|
||||
|
||||
#### 1. Complete Git Directory Renames
|
||||
|
||||
**Remaining violations** (case-sensitive renames needed):
|
||||
```bash
|
||||
# Still need two-step rename due to macOS case-insensitive FS
|
||||
git mv docs/Reference docs/reference-tmp && git mv docs/reference-tmp docs/reference
|
||||
git mv docs/Templates docs/templates-tmp && git mv docs/templates-tmp docs/templates
|
||||
git mv docs/User-Guide docs/user-guide-tmp && git mv docs/user-guide-tmp docs/user-guide
|
||||
git mv docs/User-Guide-jp docs/user-guide-jp-tmp && git mv docs/user-guide-jp-tmp docs/user-guide-jp
|
||||
git mv docs/User-Guide-kr docs/user-guide-kr-tmp && git mv docs/user-guide-kr-tmp docs/user-guide-kr
|
||||
git mv docs/User-Guide-zh docs/user-guide-zh-tmp && git mv docs/user-guide-zh-tmp docs/user-guide-zh
|
||||
|
||||
# Update MANIFEST.in to reflect new names
|
||||
sed -i '' 's/recursive-include Docs/recursive-include docs/g' MANIFEST.in
|
||||
sed -i '' 's/recursive-include Setup/recursive-include setup/g' MANIFEST.in
|
||||
sed -i '' 's/recursive-include Templates/recursive-include templates/g' MANIFEST.in
|
||||
|
||||
# Verify no uppercase directory references remain
|
||||
grep -r "Docs\|Setup\|Templates\|Reference\|User-Guide" --include="*.md" --include="*.py" --include="*.toml" --include="*.in" . | grep -v ".git"
|
||||
|
||||
# Commit changes
|
||||
git add .
|
||||
git commit -m "refactor: complete PEP 8 directory naming compliance
|
||||
|
||||
- Rename all remaining capitalized directories to lowercase
|
||||
- Update MANIFEST.in with corrected paths
|
||||
- Ensure cross-platform compatibility
|
||||
|
||||
Refs: PEP 8 package naming conventions"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### 2. Install and Configure Ruff
|
||||
|
||||
```bash
|
||||
# Install ruff
|
||||
uv pip install ruff
|
||||
|
||||
# Add to pyproject.toml (already exists, but verify config)
|
||||
```
|
||||
|
||||
**Verify `pyproject.toml` has**:
|
||||
```toml
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=6.0",
|
||||
"pytest-cov>=2.0",
|
||||
"ruff>=0.1.0", # Add if missing
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 88
|
||||
target-version = ["py38", "py39", "py310", "py311", "py312"]
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = [
|
||||
"E", # pycodestyle errors
|
||||
"F", # pyflakes
|
||||
"W", # pycodestyle warnings
|
||||
"I", # isort
|
||||
"N", # pep8-naming
|
||||
]
|
||||
|
||||
[tool.ruff.lint.per-file-ignores]
|
||||
"__init__.py" = ["F401"] # Unused imports OK
|
||||
"tests/*" = ["N802", "N803"] # Relaxed naming in tests
|
||||
```
|
||||
|
||||
**Run ruff**:
|
||||
```bash
|
||||
# Check for issues
|
||||
ruff check .
|
||||
|
||||
# Auto-fix issues
|
||||
ruff check --fix .
|
||||
|
||||
# Format code
|
||||
ruff format .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### 3. Set Up Pre-commit Hooks
|
||||
|
||||
**Create `.pre-commit-config.yaml`**:
|
||||
```yaml
|
||||
repos:
|
||||
# Official pre-commit hooks
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v4.5.0
|
||||
hooks:
|
||||
- id: check-case-conflict
|
||||
- id: check-illegal-windows-names
|
||||
- id: check-yaml
|
||||
- id: check-toml
|
||||
- id: end-of-file-fixer
|
||||
- id: trailing-whitespace
|
||||
- id: check-added-large-files
|
||||
args: ['--maxkb=1000']
|
||||
|
||||
# Ruff linter and formatter
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.1.9
|
||||
hooks:
|
||||
- id: ruff
|
||||
args: [--fix, --exit-non-zero-on-fix]
|
||||
- id: ruff-format
|
||||
|
||||
# pyproject.toml validation
|
||||
- repo: https://github.com/abravalheri/validate-pyproject
|
||||
rev: v0.16
|
||||
hooks:
|
||||
- id: validate-pyproject
|
||||
|
||||
# Custom directory naming validator
|
||||
- repo: local
|
||||
hooks:
|
||||
- id: validate-directory-names
|
||||
name: Validate Directory Naming
|
||||
entry: python scripts/validate_directory_names.py
|
||||
language: system
|
||||
pass_filenames: false
|
||||
always_run: true
|
||||
```
|
||||
|
||||
**Install pre-commit**:
|
||||
```bash
|
||||
# Install pre-commit
|
||||
uv pip install pre-commit
|
||||
|
||||
# Install hooks
|
||||
pre-commit install
|
||||
|
||||
# Run on all files (initial check)
|
||||
pre-commit run --all-files
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
#### 4. Create Custom Directory Validator
|
||||
|
||||
**Create `scripts/validate_directory_names.py`** (see full implementation above)
|
||||
|
||||
**Make executable**:
|
||||
```bash
|
||||
chmod +x scripts/validate_directory_names.py
|
||||
|
||||
# Test manually
|
||||
python scripts/validate_directory_names.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Future Improvements (Optional)
|
||||
|
||||
#### 1. Consider Repository Rename
|
||||
|
||||
**Current**: `SuperClaude_Framework`
|
||||
**PEP 8 Compliant**: `superclaude-framework` or `superclaude_framework`
|
||||
|
||||
**Rationale**:
|
||||
- Package name: `superclaude` (already compliant)
|
||||
- Repository name: Should match package style
|
||||
- GitHub allows repository renaming with automatic redirects
|
||||
|
||||
**Process**:
|
||||
```bash
|
||||
# 1. Rename on GitHub (Settings → Repository name)
|
||||
# 2. Update local remote
|
||||
git remote set-url origin https://github.com/SuperClaude-Org/superclaude-framework.git
|
||||
|
||||
# 3. Update all documentation references
|
||||
grep -rl "SuperClaude_Framework" . | xargs sed -i '' 's/SuperClaude_Framework/superclaude-framework/g'
|
||||
|
||||
# 4. Update pyproject.toml URLs
|
||||
sed -i '' 's|SuperClaude_Framework|superclaude-framework|g' pyproject.toml
|
||||
```
|
||||
|
||||
**GitHub Benefits**:
|
||||
- Old URLs automatically redirect (no broken links)
|
||||
- Clone URLs updated automatically
|
||||
- Issues/PRs remain accessible
|
||||
|
||||
---
|
||||
|
||||
#### 2. Migrate to src-based Layout
|
||||
|
||||
**Current**:
|
||||
```
|
||||
SuperClaude_Framework/
|
||||
├── superclaude/ # Package at root
|
||||
├── setup/ # Package at root
|
||||
```
|
||||
|
||||
**Recommended**:
|
||||
```
|
||||
superclaude-framework/
|
||||
├── src/
|
||||
│ ├── superclaude/ # Main package
|
||||
│ └── setup/ # Setup package
|
||||
```
|
||||
|
||||
**Benefits**:
|
||||
- Prevents accidental imports from source
|
||||
- Tests import from installed package
|
||||
- Clearer separation of concerns
|
||||
- Standard for modern Python projects
|
||||
|
||||
**Migration**:
|
||||
```bash
|
||||
# Create src directory
|
||||
mkdir -p src
|
||||
|
||||
# Move packages
|
||||
git mv superclaude src/superclaude
|
||||
git mv setup src/setup
|
||||
|
||||
# Update pyproject.toml
|
||||
```
|
||||
|
||||
```toml
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
include = ["superclaude*", "setup*"]
|
||||
```
|
||||
|
||||
**Note**: This is a breaking change requiring version bump and migration guide.
|
||||
|
||||
---
|
||||
|
||||
#### 3. Add GitHub Actions for CI/CD
|
||||
|
||||
**Create `.github/workflows/lint.yml`**:
|
||||
```yaml
|
||||
name: Lint
|
||||
|
||||
on: [push, pull_request]
|
||||
|
||||
jobs:
|
||||
lint:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install uv
|
||||
run: curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
- name: Install dependencies
|
||||
run: uv pip install -e ".[dev]"
|
||||
|
||||
- name: Run pre-commit hooks
|
||||
run: |
|
||||
uv pip install pre-commit
|
||||
pre-commit run --all-files
|
||||
|
||||
- name: Run ruff
|
||||
run: |
|
||||
ruff check .
|
||||
ruff format --check .
|
||||
|
||||
- name: Validate directory naming
|
||||
run: python scripts/validate_directory_names.py
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary: Automated vs Manual
|
||||
|
||||
### ✅ Can Be Automated
|
||||
|
||||
1. **Code linting**: Ruff (autofix imports, formatting, naming)
|
||||
2. **Configuration validation**: validate-pyproject (pyproject.toml syntax)
|
||||
3. **Pre-commit checks**: check-case-conflict, trailing-whitespace, etc.
|
||||
4. **Python naming**: Ruff N-rules (class, function, variable names)
|
||||
5. **Custom validators**: Python scripts for directory naming (preventive)
|
||||
|
||||
### ❌ Cannot Be Fully Automated
|
||||
|
||||
1. **Directory renaming**: Requires manual `git mv` (macOS case-insensitive FS)
|
||||
2. **Directory naming enforcement**: No standard linter rules (need custom script)
|
||||
3. **Documentation updates**: Link references require manual review
|
||||
4. **Repository renaming**: Manual GitHub settings change
|
||||
5. **Breaking changes**: Require human judgment and migration planning
|
||||
|
||||
### Hybrid Approach (Best Practice)
|
||||
|
||||
1. **Manual**: Initial directory rename using two-step `git mv`
|
||||
2. **Automated**: Pre-commit hook prevents future violations
|
||||
3. **Continuous**: Ruff + pre-commit in CI/CD pipeline
|
||||
4. **Preventive**: Custom validator blocks non-compliant names
|
||||
|
||||
---
|
||||
|
||||
## Confidence Assessment
|
||||
|
||||
| Finding | Confidence | Source Quality |
|
||||
|---------|-----------|----------------|
|
||||
| PEP 8 naming conventions | 95% | Official PEP documentation |
|
||||
| Ruff as 2025 standard | 90% | GitHub stars, community adoption |
|
||||
| Git two-step rename | 95% | Official docs, Stack Overflow consensus |
|
||||
| No automated directory linter | 85% | Tool documentation review |
|
||||
| Pre-commit best practices | 90% | Official pre-commit docs |
|
||||
| uv project structure | 85% | Official Astral docs, Real Python |
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
1. PEP 8 Official Documentation: https://peps.python.org/pep-0008/
|
||||
2. Ruff Documentation: https://docs.astral.sh/ruff/
|
||||
3. Real Python - Ruff Guide: https://realpython.com/ruff-python/
|
||||
4. Git Case-Sensitive Renaming: Multiple Stack Overflow threads (2022-2024)
|
||||
5. validate-pyproject: https://github.com/abravalheri/validate-pyproject
|
||||
6. Pre-commit Hooks Guide (2025): https://gatlenculp.medium.com/effortless-code-quality-the-ultimate-pre-commit-hooks-guide-for-2025-57ca501d9835
|
||||
7. uv Documentation: https://docs.astral.sh/uv/
|
||||
8. Python Packaging User Guide: https://packaging.python.org/
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
**The Reality**: There is NO fully automated one-click solution for directory renaming to PEP 8 compliance.
|
||||
|
||||
**Best Practice Workflow**:
|
||||
|
||||
1. **Manual Rename**: Use two-step `git mv` for macOS compatibility
|
||||
2. **Automated Prevention**: Pre-commit hooks with custom validator
|
||||
3. **Continuous Enforcement**: Ruff linter + CI/CD pipeline
|
||||
4. **Documentation**: Update all references (semi-automated with sed)
|
||||
|
||||
**For SuperClaude Framework**:
|
||||
- Complete the remaining directory renames manually (6 directories)
|
||||
- Set up pre-commit hooks with custom validator
|
||||
- Configure Ruff for Python code linting
|
||||
- Add CI/CD workflow for continuous validation
|
||||
|
||||
**Total Effort Estimate**:
|
||||
- Manual renaming: 15-30 minutes
|
||||
- Pre-commit setup: 15-20 minutes
|
||||
- Documentation updates: 10-15 minutes
|
||||
- Testing and verification: 20-30 minutes
|
||||
- **Total**: 60-95 minutes for complete PEP 8 compliance
|
||||
|
||||
**Long-term Benefit**: Prevents future violations automatically, ensuring ongoing compliance.
|
||||
Reference in New Issue
Block a user