SuperClaude/docs/research/research_python_directory_naming_automation_2025.md

Research: Python Directory Naming & Automation Tools (2025)

Research Date: 2025-10-14 Research Context: PEP 8 directory naming compliance, automated linting tools, and Git case-sensitive renaming best practices

---

Executive Summary

Key Findings

1. PEP 8 Standard (2024-2025):

   • Packages (directories): lowercase only, underscores discouraged but widely used in practice
   • Modules (files): lowercase, underscores allowed and common for readability
   • Current violations: Developer-Guide, Getting-Started, User-Guide, Reference, Templates (use hyphens/uppercase)

2. Automated Linting Tool: Ruff is the 2025 industry standard

   • Written in Rust, 10-100x faster than Flake8
   • 800+ built-in rules, replaces Flake8, Black, isort, pyupgrade, autoflake
   • Configured via pyproject.toml
   • BUT: No built-in rules for directory naming validation

3. Git Case-Sensitive Rename: Two-step git mv method

   • macOS APFS is case-insensitive by default
   • Safest approach: git mv foo foo-tmp && git mv foo-tmp bar
   • Alternative: git rm --cached + git add . (less reliable)

4. Automation Strategy: Custom pre-commit hooks + manual rename

   • Use check-case-conflict pre-commit hook
   • Write custom Python validator for directory naming
   • Integrate with validate-pyproject for configuration validation

5. Modern Project Structure (uv/2025):

   • src-based layout: src/package_name/ (recommended)
   • Configuration: pyproject.toml (universal standard)
   • Lockfile: uv.lock (cross-platform, committed to Git)

---

Detailed Findings

1. PEP 8 Directory Naming Conventions

Official Standard (PEP 8 - https://peps.python.org/pep-0008/):

"Python packages should also have short, all-lowercase names, although the use of underscores is discouraged."

Practical Reality:

• Underscores are widely used in practice (e.g., sqlalchemy_searchable)
• Community doesn't consider underscores poor practice
• Hyphens are NOT allowed in package names (Python import restrictions)
• Camel Case / Title Case = PEP 8 violation

Current SuperClaude Framework Violations:

# ❌ PEP 8 Violations
docs/Developer-Guide/     # Contains hyphen + uppercase
docs/Getting-Started/     # Contains hyphen + uppercase
docs/User-Guide/          # Contains hyphen + uppercase
docs/User-Guide-jp/       # Contains hyphen + uppercase
docs/User-Guide-kr/       # Contains hyphen + uppercase
docs/User-Guide-zh/       # Contains hyphen + uppercase
docs/Reference/           # Contains uppercase
docs/Templates/           # Contains uppercase

# ✅ PEP 8 Compliant (Already Fixed)
docs/developer-guide/     # lowercase + hyphen (acceptable for docs)
docs/getting-started/     # lowercase + hyphen (acceptable for docs)
docs/development/         # lowercase only

Documentation Directories Exception:

• Documentation directories (docs/) are NOT Python packages
• Hyphens are acceptable in non-package directories
• Best practice: Use lowercase + hyphens for readability
• Example: docs/getting-started/, docs/user-guide/

---

2. Automated Linting Tools (2024-2025)

Ruff - The Modern Standard

Overview:

• Released: 2023, rapidly adopted as industry standard by 2024-2025
• Speed: 10-100x faster than Flake8 (written in Rust)
• Replaces: Flake8, Black, isort, pydocstyle, pyupgrade, autoflake
• Rules: 800+ built-in rules
• Configuration: pyproject.toml or ruff.toml

Key Features:

Autofix:
  - Automatic import sorting
  - Unused variable removal
  - Python syntax upgrades
  - Code formatting

Per-Directory Configuration:
  - Different rules for different directories
  - Per-file-target-version settings
  - Namespace package support

Exclusions (default):
  - .git, .venv, build, dist, node_modules
  - __pycache__, .pytest_cache, .mypy_cache
  - Custom patterns via glob

Configuration Example (pyproject.toml):

[tool.ruff]
line-length = 88
target-version = "py38"

exclude = [
    ".git",
    ".venv",
    "build",
    "dist",
]

[tool.ruff.lint]
select = ["E", "F", "W", "I", "N"]  # N = naming conventions
ignore = ["E501"]  # Line too long

[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401"]  # Unused imports OK in __init__.py
"tests/*" = ["N802"]      # Function name conventions relaxed in tests

Naming Convention Rules (N prefix):

N801: Class names should use CapWords convention
N802: Function names should be lowercase
N803: Argument names should be lowercase
N804: First argument of classmethod should be cls
N805: First argument of method should be self
N806: Variable in function should be lowercase
N807: Function name should not start/end with __

BUT: No rules for directory naming (non-Python file checks)

Limitation: Ruff validates Python code, not directory structure.

---

validate-pyproject - Configuration Validator

Purpose: Validates pyproject.toml compliance with PEP standards

Installation:

pip install validate-pyproject
# or with pre-commit integration

Usage:

# CLI
validate-pyproject pyproject.toml

# Python API
from validate_pyproject import validate
validate(data)

Pre-commit Hook:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/abravalheri/validate-pyproject
    rev: v0.16
    hooks:
      - id: validate-pyproject

What It Validates:

• PEP 517/518 build system configuration
• PEP 621 project metadata
• Tool-specific configurations ([tool.ruff], [tool.mypy])
• JSON Schema compliance

Limitation: Validates pyproject.toml syntax, not directory naming.

---

3. Git Case-Sensitive Rename Best Practices

The Problem:

• macOS APFS: case-insensitive by default
• Git: case-sensitive internally
• Result: git mv Foo foo doesn't work directly
• Risk: Breaking changes across systems

Best Practice #1: Two-Step git mv (Safest)

# Step 1: Rename to temporary name
git mv docs/User-Guide docs/user-guide-tmp

# Step 2: Rename to final name
git mv docs/user-guide-tmp docs/user-guide

# Commit
git commit -m "refactor: rename User-Guide to user-guide (PEP 8 compliance)"

Why This Works:

• First rename: Different enough for case-insensitive FS to recognize
• Second rename: Achieves desired final name
• Git tracks both renames correctly
• No data loss risk

Best Practice #2: Cache Clearing (Alternative)

# Remove from Git index (keeps working tree)
git rm -r --cached .

# Re-add all files (Git detects renames)
git add .

# Commit
git commit -m "refactor: fix directory naming case sensitivity"

Why This Works:

• Git re-scans working tree
• Detects same content = rename (not delete + add)
• Preserves file history

What NOT to Do:

# ❌ DANGEROUS: Disabling core.ignoreCase
git config core.ignoreCase false

# Risk: Unexpected behavior on case-insensitive filesystems
# Official docs warning: "modifying this value may result in unexpected behavior"

Advanced Workaround (Overkill):

• Create case-sensitive APFS volume via Disk Utility
• Clone repository to case-sensitive volume
• Perform renames normally
• Push to remote

---

4. Pre-commit Hooks for Structure Validation

Built-in Hooks (check-case-conflict)

Official pre-commit-hooks (https://github.com/pre-commit/pre-commit-hooks):

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-case-conflict        # Detects case sensitivity issues
      - id: check-illegal-windows-names # Windows filename validation
      - id: check-symlinks             # Symlink integrity
      - id: destroyed-symlinks         # Broken symlinks detection
      - id: check-added-large-files    # Prevent large file commits
      - id: check-yaml                 # YAML syntax validation
      - id: end-of-file-fixer          # Ensure newline at EOF
      - id: trailing-whitespace        # Remove trailing spaces

check-case-conflict Details:

• Detects files that differ only in case
• Example: README.md vs readme.md
• Prevents issues on case-insensitive filesystems
• Runs before commit, blocks if conflicts found

Limitation: Only detects conflicts, doesn't enforce naming conventions.

---

Custom Hook: Directory Naming Validator

Purpose: Enforce PEP 8 directory naming conventions

Implementation (scripts/validate_directory_names.py):

#!/usr/bin/env python3
"""
Pre-commit hook to validate directory naming conventions.
Enforces PEP 8 compliance for Python packages.
"""
import sys
from pathlib import Path
import re

# PEP 8: Package names should be lowercase, underscores discouraged
PACKAGE_NAME_PATTERN = re.compile(r'^[a-z][a-z0-9_]*$')

# Documentation directories: lowercase + hyphens allowed
DOC_NAME_PATTERN = re.compile(r'^[a-z][a-z0-9\-]*$')

def validate_directory_names(root_dir='.'):
    """Validate directory naming conventions."""
    violations = []

    root = Path(root_dir)

    # Check Python package directories
    for pydir in root.rglob('__init__.py'):
        package_dir = pydir.parent
        package_name = package_dir.name

        if not PACKAGE_NAME_PATTERN.match(package_name):
            violations.append(
                f"PEP 8 violation: Package '{package_dir}' should be lowercase "
                f"(current: '{package_name}')"
            )

    # Check documentation directories
    docs_root = root / 'docs'
    if docs_root.exists():
        for doc_dir in docs_root.iterdir():
            if doc_dir.is_dir() and doc_dir.name not in ['.git', '__pycache__']:
                if not DOC_NAME_PATTERN.match(doc_dir.name):
                    violations.append(
                        f"Documentation naming violation: '{doc_dir}' should be "
                        f"lowercase with hyphens (current: '{doc_dir.name}')"
                    )

    return violations

def main():
    violations = validate_directory_names()

    if violations:
        print("❌ Directory naming convention violations found:\n")
        for violation in violations:
            print(f"  - {violation}")
        print("\n" + "="*70)
        print("Fix: Rename directories to lowercase (hyphens for docs, underscores for packages)")
        print("="*70)
        return 1

    print("✅ All directory names comply with PEP 8 conventions")
    return 0

if __name__ == '__main__':
    sys.exit(main())

Pre-commit Configuration:

# .pre-commit-config.yaml
repos:
  # Official hooks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-case-conflict
      - id: trailing-whitespace
      - id: end-of-file-fixer

  # Ruff linter
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.1.9
    hooks:
      - id: ruff
        args: [--fix, --exit-non-zero-on-fix]
      - id: ruff-format

  # Custom directory naming validator
  - repo: local
    hooks:
      - id: validate-directory-names
        name: Validate Directory Naming
        entry: python scripts/validate_directory_names.py
        language: system
        pass_filenames: false
        always_run: true

Installation:

# Install pre-commit
pip install pre-commit

# Install hooks to .git/hooks/
pre-commit install

# Run manually on all files
pre-commit run --all-files

---

5. Modern Python Project Structure (uv/2025)

Standard Layout (uv recommended)

project-root/
├── .git/
├── .gitignore
├── .python-version           # Python version for uv
├── pyproject.toml            # Project metadata + tool configs
├── uv.lock                   # Cross-platform lockfile (commit this)
├── README.md
├── LICENSE
├── .pre-commit-config.yaml   # Pre-commit hooks
├── src/                      # Source code (src-based layout)
│   └── package_name/
│       ├── __init__.py
│       ├── module1.py
│       └── subpackage/
│           ├── __init__.py
│           └── module2.py
├── tests/                    # Test files
│   ├── __init__.py
│   ├── test_module1.py
│   └── test_module2.py
├── docs/                     # Documentation
│   ├── getting-started/      # lowercase + hyphens OK
│   ├── user-guide/
│   └── developer-guide/
├── scripts/                  # Utility scripts
│   └── validate_directory_names.py
└── .venv/                    # Virtual environment (local to project)

Key Files:

pyproject.toml (modern standard):

[build-system]
requires = ["setuptools>=61.0", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "package-name"  # lowercase, hyphens allowed for non-importable
version = "1.0.0"
requires-python = ">=3.8"

[tool.setuptools.packages.find]
where = ["src"]
include = ["package_name*"]  # lowercase_underscore for Python packages

[tool.ruff]
line-length = 88
target-version = "py38"

[tool.ruff.lint]
select = ["E", "F", "W", "I", "N"]

uv.lock:

• Cross-platform lockfile
• Contains exact resolved versions
• Must be committed to version control
• Ensures reproducible installations

.python-version:

3.12

Benefits of src-based layout:

1. Namespace isolation: Prevents import conflicts
2. Testability: Tests import from installed package, not source
3. Modularity: Clear separation of application logic
4. Distribution: Required for PyPI publishing
5. Editor support: .venv in project root helps IDEs find packages

---

Recommendations for SuperClaude Framework

Immediate Actions (Required)

1. Complete Git Directory Renames

Remaining violations (case-sensitive renames needed):

# Still need two-step rename due to macOS case-insensitive FS
git mv docs/Reference docs/reference-tmp && git mv docs/reference-tmp docs/reference
git mv docs/Templates docs/templates-tmp && git mv docs/templates-tmp docs/templates
git mv docs/User-Guide docs/user-guide-tmp && git mv docs/user-guide-tmp docs/user-guide
git mv docs/User-Guide-jp docs/user-guide-jp-tmp && git mv docs/user-guide-jp-tmp docs/user-guide-jp
git mv docs/User-Guide-kr docs/user-guide-kr-tmp && git mv docs/user-guide-kr-tmp docs/user-guide-kr
git mv docs/User-Guide-zh docs/user-guide-zh-tmp && git mv docs/user-guide-zh-tmp docs/user-guide-zh

# Update MANIFEST.in to reflect new names
sed -i '' 's/recursive-include Docs/recursive-include docs/g' MANIFEST.in
sed -i '' 's/recursive-include Setup/recursive-include setup/g' MANIFEST.in
sed -i '' 's/recursive-include Templates/recursive-include templates/g' MANIFEST.in

# Verify no uppercase directory references remain
grep -r "Docs\|Setup\|Templates\|Reference\|User-Guide" --include="*.md" --include="*.py" --include="*.toml" --include="*.in" . | grep -v ".git"

# Commit changes
git add .
git commit -m "refactor: complete PEP 8 directory naming compliance

- Rename all remaining capitalized directories to lowercase
- Update MANIFEST.in with corrected paths
- Ensure cross-platform compatibility

Refs: PEP 8 package naming conventions"

---

2. Install and Configure Ruff

# Install ruff
uv pip install ruff

# Add to pyproject.toml (already exists, but verify config)

Verify pyproject.toml has:

[project.optional-dependencies]
dev = [
    "pytest>=6.0",
    "pytest-cov>=2.0",
    "ruff>=0.1.0",  # Add if missing
]

[tool.ruff]
line-length = 88
target-version = ["py38", "py39", "py310", "py311", "py312"]

[tool.ruff.lint]
select = [
    "E",   # pycodestyle errors
    "F",   # pyflakes
    "W",   # pycodestyle warnings
    "I",   # isort
    "N",   # pep8-naming
]

[tool.ruff.lint.per-file-ignores]
"__init__.py" = ["F401"]  # Unused imports OK
"tests/*" = ["N802", "N803"]  # Relaxed naming in tests

Run ruff:

# Check for issues
ruff check .

# Auto-fix issues
ruff check --fix .

# Format code
ruff format .

---

3. Set Up Pre-commit Hooks

Create .pre-commit-config.yaml:

repos:
  # Official pre-commit hooks
  - repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v4.5.0
    hooks:
      - id: check-case-conflict
      - id: check-illegal-windows-names
      - id: check-yaml
      - id: check-toml
      - id: end-of-file-fixer
      - id: trailing-whitespace
      - id: check-added-large-files
        args: ['--maxkb=1000']

  # Ruff linter and formatter
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.1.9
    hooks:
      - id: ruff
        args: [--fix, --exit-non-zero-on-fix]
      - id: ruff-format

  # pyproject.toml validation
  - repo: https://github.com/abravalheri/validate-pyproject
    rev: v0.16
    hooks:
      - id: validate-pyproject

  # Custom directory naming validator
  - repo: local
    hooks:
      - id: validate-directory-names
        name: Validate Directory Naming
        entry: python scripts/validate_directory_names.py
        language: system
        pass_filenames: false
        always_run: true

Install pre-commit:

# Install pre-commit
uv pip install pre-commit

# Install hooks
pre-commit install

# Run on all files (initial check)
pre-commit run --all-files

---

4. Create Custom Directory Validator

Create scripts/validate_directory_names.py (see full implementation above)

Make executable:

chmod +x scripts/validate_directory_names.py

# Test manually
python scripts/validate_directory_names.py

---

Future Improvements (Optional)

1. Consider Repository Rename

Current: SuperClaude_Framework PEP 8 Compliant: superclaude-framework or superclaude_framework

Rationale:

• Package name: superclaude (already compliant)
• Repository name: Should match package style
• GitHub allows repository renaming with automatic redirects

Process:

# 1. Rename on GitHub (Settings → Repository name)
# 2. Update local remote
git remote set-url origin https://github.com/SuperClaude-Org/superclaude-framework.git

# 3. Update all documentation references
grep -rl "SuperClaude_Framework" . | xargs sed -i '' 's/SuperClaude_Framework/superclaude-framework/g'

# 4. Update pyproject.toml URLs
sed -i '' 's|SuperClaude_Framework|superclaude-framework|g' pyproject.toml

GitHub Benefits:

• Old URLs automatically redirect (no broken links)
• Clone URLs updated automatically
• Issues/PRs remain accessible

---

2. Migrate to src-based Layout

Current:

SuperClaude_Framework/
├── superclaude/          # Package at root
├── setup/                # Package at root

Recommended:

superclaude-framework/
├── src/
│   ├── superclaude/      # Main package
│   └── setup/            # Setup package

Benefits:

• Prevents accidental imports from source
• Tests import from installed package
• Clearer separation of concerns
• Standard for modern Python projects

Migration:

# Create src directory
mkdir -p src

# Move packages
git mv superclaude src/superclaude
git mv setup src/setup

# Update pyproject.toml

[tool.setuptools.packages.find]
where = ["src"]
include = ["superclaude*", "setup*"]

Note: This is a breaking change requiring version bump and migration guide.

---

3. Add GitHub Actions for CI/CD

Create .github/workflows/lint.yml:

name: Lint

on: [push, pull_request]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install uv
        run: curl -LsSf https://astral.sh/uv/install.sh | sh

      - name: Install dependencies
        run: uv pip install -e ".[dev]"

      - name: Run pre-commit hooks
        run: |
          uv pip install pre-commit
          pre-commit run --all-files

      - name: Run ruff
        run: |
          ruff check .
          ruff format --check .

      - name: Validate directory naming
        run: python scripts/validate_directory_names.py

---

Summary: Automated vs Manual

✅ Can Be Automated

1. Code linting: Ruff (autofix imports, formatting, naming)
2. Configuration validation: validate-pyproject (pyproject.toml syntax)
3. Pre-commit checks: check-case-conflict, trailing-whitespace, etc.
4. Python naming: Ruff N-rules (class, function, variable names)
5. Custom validators: Python scripts for directory naming (preventive)

❌ Cannot Be Fully Automated

1. Directory renaming: Requires manual git mv (macOS case-insensitive FS)
2. Directory naming enforcement: No standard linter rules (need custom script)
3. Documentation updates: Link references require manual review
4. Repository renaming: Manual GitHub settings change
5. Breaking changes: Require human judgment and migration planning

Hybrid Approach (Best Practice)

1. Manual: Initial directory rename using two-step git mv
2. Automated: Pre-commit hook prevents future violations
3. Continuous: Ruff + pre-commit in CI/CD pipeline
4. Preventive: Custom validator blocks non-compliant names

---

Confidence Assessment

Finding	Confidence	Source Quality
PEP 8 naming conventions	95%	Official PEP documentation
Ruff as 2025 standard	90%	GitHub stars, community adoption
Git two-step rename	95%	Official docs, Stack Overflow consensus
No automated directory linter	85%	Tool documentation review
Pre-commit best practices	90%	Official pre-commit docs
uv project structure	85%	Official Astral docs, Real Python

---

Sources

1. PEP 8 Official Documentation: https://peps.python.org/pep-0008/
2. Ruff Documentation: https://docs.astral.sh/ruff/
3. Real Python - Ruff Guide: https://realpython.com/ruff-python/
4. Git Case-Sensitive Renaming: Multiple Stack Overflow threads (2022-2024)
5. validate-pyproject: https://github.com/abravalheri/validate-pyproject
6. Pre-commit Hooks Guide (2025): https://gatlenculp.medium.com/effortless-code-quality-the-ultimate-pre-commit-hooks-guide-for-2025-57ca501d9835
7. uv Documentation: https://docs.astral.sh/uv/
8. Python Packaging User Guide: https://packaging.python.org/

---

Conclusion

The Reality: There is NO fully automated one-click solution for directory renaming to PEP 8 compliance.

Best Practice Workflow:

1. Manual Rename: Use two-step git mv for macOS compatibility
2. Automated Prevention: Pre-commit hooks with custom validator
3. Continuous Enforcement: Ruff linter + CI/CD pipeline
4. Documentation: Update all references (semi-automated with sed)

For SuperClaude Framework:

• Complete the remaining directory renames manually (6 directories)
• Set up pre-commit hooks with custom validator
• Configure Ruff for Python code linting
• Add CI/CD workflow for continuous validation

Total Effort Estimate:

• Manual renaming: 15-30 minutes
• Pre-commit setup: 15-20 minutes
• Documentation updates: 10-15 minutes
• Testing and verification: 20-30 minutes
• Total: 60-95 minutes for complete PEP 8 compliance

Long-term Benefit: Prevents future violations automatically, ensuring ongoing compliance.