SuperClaude/superclaude/core/RESEARCH_CONFIG.md
kazuki nakai 050d5ea2ab
refactor: PEP8 compliance - directory rename and code formatting (#425)
* fix(orchestration): add WebFetch auto-trigger for infrastructure configuration

Problem: Infrastructure configuration changes (e.g., Traefik port settings)
were being made based on assumptions without consulting official documentation,
violating the 'Evidence > assumptions' principle in PRINCIPLES.md.

Solution:
- Added Infrastructure Configuration Validation section to MODE_Orchestration.md
- Auto-triggers WebFetch for infrastructure tools (Traefik, nginx, Docker, etc.)
- Enforces MODE_DeepResearch activation for investigation
- BLOCKS assumption-based configuration changes

Testing: Verified WebFetch successfully retrieves Traefik official docs (port 80 default)

This prevents production outages from infrastructure misconfiguration by ensuring
all technical recommendations are backed by official documentation.

* feat: Add PM Agent (Project Manager Agent) for seamless orchestration

Introduces PM Agent as the default orchestration layer that coordinates
all sub-agents and manages workflows automatically.

Key Features:
- Default orchestration: All user interactions handled by PM Agent
- Auto-delegation: Intelligent sub-agent selection based on task analysis
- Docker Gateway integration: Zero-token baseline with dynamic MCP loading
- Self-improvement loop: Automatic documentation of patterns and mistakes
- Optional override: Users can specify sub-agents explicitly if desired

Architecture:
- Agent spec: SuperClaude/Agents/pm-agent.md
- Command: SuperClaude/Commands/pm.md
- Updated docs: README.md (15→16 agents), agents.md (new Orchestration category)

User Experience:
- Default: PM Agent handles everything (seamless, no manual routing)
- Optional: Explicit --agent flag for direct sub-agent access
- Both modes available simultaneously (no user downside)

Implementation Status:
-  Specification complete
-  Documentation complete
-  Prototype implementation needed
-  Docker Gateway integration needed
-  Testing and validation needed

Refs: kazukinakai/docker-mcp-gateway (IRIS MCP Gateway integration)

* feat: Add Agent Orchestration rules for PM Agent default activation

Implements PM Agent as the default orchestration layer in RULES.md.

Key Changes:
- New 'Agent Orchestration' section (CRITICAL priority)
- PM Agent receives ALL user requests by default
- Manual override with @agent-[name] bypasses PM Agent
- Agent Selection Priority clearly defined:
  1. Manual override → Direct routing
  2. Default → PM Agent → Auto-delegation
  3. Delegation based on keywords, file types, complexity, context

User Experience:
- Default: PM Agent handles everything (seamless)
- Override: @agent-[name] for direct specialist access
- Transparent: PM Agent reports delegation decisions

This establishes PM Agent as the orchestration layer while
respecting existing auto-activation patterns and manual overrides.

Next Steps:
- Local testing in agiletec project
- Iteration based on actual behavior
- Documentation updates as needed

* refactor(pm-agent): redesign as self-improvement meta-layer

Problem Resolution:
PM Agent's initial design competed with existing auto-activation for task routing,
creating confusion about orchestration responsibilities and adding unnecessary complexity.

Design Change:
Redefined PM Agent as a meta-layer agent that operates AFTER specialist agents
complete tasks, focusing on:
- Post-implementation documentation and pattern recording
- Immediate mistake analysis with prevention checklists
- Monthly documentation maintenance and noise reduction
- Pattern extraction and knowledge synthesis

Two-Layer Orchestration System:
1. Task Execution Layer: Existing auto-activation handles task routing (unchanged)
2. Self-Improvement Layer: PM Agent meta-layer handles documentation (new)

Files Modified:
- SuperClaude/Agents/pm-agent.md: Complete rewrite with meta-layer design
  - Category: orchestration → meta
  - Triggers: All user interactions → Post-implementation, mistakes, monthly
  - Behavioral Mindset: Continuous learning system
  - Self-Improvement Workflow: BEFORE/DURING/AFTER/MISTAKE RECOVERY/MAINTENANCE

- SuperClaude/Core/RULES.md: Agent Orchestration section updated
  - Split into Task Execution Layer + Self-Improvement Layer
  - Added orchestration flow diagram
  - Clarified PM Agent activates AFTER task completion

- README.md: Updated PM Agent description
  - "orchestrates all interactions" → "ensures continuous learning"

- Docs/User-Guide/agents.md: PM Agent section rewritten
  - Section: Orchestration Agent → Meta-Layer Agent
  - Expertise: Project orchestration → Self-improvement workflow executor
  - Examples: Task coordination → Post-implementation documentation

- PR_DOCUMENTATION.md: Comprehensive PR documentation added
  - Summary, motivation, changes, testing, breaking changes
  - Two-layer orchestration system diagram
  - Verification checklist

Integration Validated:
Tested with agiletec project's self-improvement-workflow.md:
 PM Agent aligns with existing BEFORE/DURING/AFTER/MISTAKE RECOVERY phases
 Complements (not competes with) existing workflow
 agiletec workflow defines WHAT, PM Agent defines WHO executes it

Breaking Changes: None
- Existing auto-activation continues unchanged
- Specialist agents unaffected
- User workflows remain the same
- New capability: Automatic documentation and knowledge maintenance

Value Proposition:
Transforms SuperClaude into a continuously learning system that accumulates
knowledge, prevents recurring mistakes, and maintains fresh documentation
without manual intervention.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* docs: add Claude Code conversation history management research

Research covering .jsonl file structure, performance impact, and retention policies.

Content:
- Claude Code .jsonl file format and message types
- Performance issues from GitHub (memory leaks, conversation compaction)
- Retention policies (consumer vs enterprise)
- Rotation recommendations based on actual data
- File history snapshot tracking mechanics

Source: Moved from agiletec project (research applicable to all Claude Code projects)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: add Development documentation structure

Phase 1: Documentation Structure complete

- Add Docs/Development/ directory for development documentation
- Add ARCHITECTURE.md - System architecture with PM Agent meta-layer
- Add ROADMAP.md - 5-phase development plan with checkboxes
- Add TASKS.md - Daily task tracking with progress indicators
- Add PROJECT_STATUS.md - Current status dashboard and metrics
- Add pm-agent-integration.md - Implementation guide for PM Agent mode

This establishes comprehensive documentation foundation for:
- System architecture understanding
- Development planning and tracking
- Implementation guidance
- Progress visibility

Related: #pm-agent-mode #documentation #phase-1

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: PM Agent session lifecycle and PDCA implementation

Phase 2: PM Agent Mode Integration (Design Phase)

Commands/pm.md updates:
- Add "Always-Active Foundation Layer" concept
- Add Session Lifecycle (Session Start/During Work/Session End)
- Add PDCA Cycle (Plan/Do/Check/Act) automation
- Add Serena MCP Memory Integration (list/read/write_memory)
- Document auto-activation triggers

Agents/pm-agent.md updates:
- Add Session Start Protocol (MANDATORY auto-activation)
- Add During Work PDCA Cycle with example workflows
- Add Session End Protocol with state preservation
- Add PDCA Self-Evaluation Pattern
- Add Documentation Strategy (temp → patterns/mistakes)
- Add Memory Operations Reference

Key Features:
- Session start auto-activation for context restoration
- 30-minute checkpoint saves during work
- Self-evaluation with think_about_* operations
- Systematic documentation lifecycle
- Knowledge evolution to CLAUDE.md

Implementation Status:
-  Design complete (Commands/pm.md, Agents/pm-agent.md)
-  Implementation pending (Core components)
-  Serena MCP integration pending

Salvaged from mistaken development in ~/.claude directory

Related: #pm-agent-mode #session-lifecycle #pdca-cycle #phase-2

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: disable Serena MCP auto-browser launch

Disable web dashboard and GUI log window auto-launch in Serena MCP server
to prevent intrusive browser popups on startup. Users can still manually
access the dashboard at http://localhost:24282/dashboard/ if needed.

Changes:
- Add CLI flags to Serena run command:
  - --enable-web-dashboard false
  - --enable-gui-log-window false
- Ensures Git-tracked configuration (no reliance on ~/.serena/serena_config.yml)
- Aligns with AIRIS MCP Gateway integration approach

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* refactor: rename directories to lowercase for PEP8 compliance

- Rename superclaude/Agents -> superclaude/agents
- Rename superclaude/Commands -> superclaude/commands
- Rename superclaude/Core -> superclaude/core
- Rename superclaude/Examples -> superclaude/examples
- Rename superclaude/MCP -> superclaude/mcp
- Rename superclaude/Modes -> superclaude/modes

This change follows Python PEP8 naming conventions for package directories.

* style: fix PEP8 violations and update package name to lowercase

Changes:
- Format all Python files with black (43 files reformatted)
- Update package name from 'SuperClaude' to 'superclaude' in pyproject.toml
- Fix import statements to use lowercase package name
- Add missing imports (timedelta, __version__)
- Remove old SuperClaude.egg-info directory

PEP8 violations reduced from 2672 to 701 (mostly E501 line length due to black's 88 char vs flake8's 79 char limit).

* docs: add PM Agent development documentation

Add comprehensive PM Agent development documentation:
- PM Agent ideal workflow (7-phase autonomous cycle)
- Project structure understanding (Git vs installed environment)
- Installation flow understanding (CommandsComponent behavior)
- Task management system (current-tasks.md)

Purpose: Eliminate repeated explanations and enable autonomous PDCA cycles

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(pm-agent): add self-correcting execution and warning investigation culture

## Changes

### superclaude/commands/pm.md
- Add "Self-Correcting Execution" section with root cause analysis protocol
- Add "Warning/Error Investigation Culture" section enforcing zero-tolerance for dismissal
- Define error detection protocol: STOP → Investigate → Hypothesis → Different Solution → Execute
- Document anti-patterns (retry without understanding) and correct patterns (research-first)

### docs/Development/hypothesis-pm-autonomous-enhancement-2025-10-14.md
- Add PDCA workflow hypothesis document for PM Agent autonomous enhancement

## Rationale

PM Agent must never retry failed operations without understanding root causes.
All warnings and errors require investigation via context7/WebFetch/documentation
to ensure production-quality code and prevent technical debt accumulation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat(installer): add airis-mcp-gateway MCP server option

## Changes

- Add airis-mcp-gateway to MCP server options in installer
- Configuration: GitHub-based installation via uvx
- Repository: https://github.com/oraios/airis-mcp-gateway
- Purpose: Dynamic MCP Gateway for zero-token baseline and on-demand tool loading

## Implementation

Added to setup/components/mcp.py self.mcp_servers dictionary with:
- install_method: github
- install_command: uvx test installation
- run_command: uvx runtime execution
- required: False (optional server)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: kazuki <kazuki@kazukinoMacBook-Air.local>
Co-authored-by: Claude <noreply@anthropic.com>
2025-10-14 08:47:09 +05:30

9.4 KiB

Deep Research Configuration

Default Settings

research_defaults:
  planning_strategy: unified
  max_hops: 5
  confidence_threshold: 0.7
  memory_enabled: true
  parallelization: true
  parallel_first: true  # MANDATORY DEFAULT
  sequential_override_requires_justification: true  # NEW
  
parallel_execution_rules:
  DEFAULT_MODE: PARALLEL  # EMPHASIZED
  
  mandatory_parallel:
    - "Multiple search queries"
    - "Batch URL extractions"
    - "Independent analyses"
    - "Non-dependent hops"
    - "Result processing"
    - "Information extraction"
    
  sequential_only_with_justification:
    - reason: "Explicit dependency"
      example: "Hop N requires Hop N-1 results"
    - reason: "Resource constraint"
      example: "API rate limit reached"
    - reason: "User requirement"
      example: "User requests sequential for debugging"
    
  parallel_optimization:
    batch_sizes:
      searches: 5
      extractions: 3
      analyses: 2
    intelligent_grouping:
      by_domain: true
      by_complexity: true
      by_resource: true
  
planning_strategies:
  planning_only:
    clarification: false
    user_confirmation: false
    execution: immediate
    
  intent_planning:
    clarification: true
    max_questions: 3
    execution: after_clarification
    
  unified:
    clarification: optional
    plan_presentation: true
    user_feedback: true
    execution: after_confirmation

hop_configuration:
  max_depth: 5
  timeout_per_hop: 60s
  parallel_hops: true
  loop_detection: true
  genealogy_tracking: true

confidence_scoring:
  relevance_weight: 0.5
  completeness_weight: 0.5
  minimum_threshold: 0.6
  target_threshold: 0.8

self_reflection:
  frequency: after_each_hop
  triggers:
    - confidence_below_threshold
    - contradictions_detected
    - time_elapsed_percentage: 80
    - user_intervention
  actions:
    - assess_quality
    - identify_gaps
    - consider_replanning
    - adjust_strategy

memory_management:
  case_based_reasoning: true
  pattern_learning: true
  session_persistence: true
  cross_session_learning: true
  retention_days: 30

tool_coordination:
  discovery_primary: tavily
  extraction_smart_routing: true
  reasoning_engine: sequential
  memory_backend: serena
  parallel_tool_calls: true

quality_gates:
  planning_gate:
    required_elements: [objectives, strategy, success_criteria]
  execution_gate:
    min_confidence: 0.6
  synthesis_gate:
    coherence_required: true
    clarity_required: true

extraction_settings:
  scraping_strategy: selective
  screenshot_capture: contextual
  authentication_handling: ethical
  javascript_rendering: auto_detect
  timeout_per_page: 15s

Performance Optimizations

optimization_strategies:
  caching:
    - Cache Tavily search results: 1 hour
    - Cache Playwright extractions: 24 hours
    - Cache Sequential analysis: 1 hour
    - Reuse case patterns: always

  parallelization:
    - Parallel searches: max 5
    - Parallel extractions: max 3
    - Parallel analysis: max 2
    - Tool call batching: true

  resource_limits:
    - Max time per research: 10 minutes
    - Max search iterations: 10
    - Max hops: 5
    - Max memory per session: 100MB

Strategy Selection Rules

strategy_selection:
  planning_only:
    indicators:
      - Clear, specific query
      - Technical documentation request
      - Well-defined scope
      - No ambiguity detected
    
  intent_planning:
    indicators:
      - Ambiguous terms present
      - Broad topic area
      - Multiple possible interpretations
      - User expertise unknown
    
  unified:
    indicators:
      - Complex multi-faceted query
      - User collaboration beneficial
      - Iterative refinement expected
      - High-stakes research

Source Credibility Matrix

source_credibility:
  tier_1_sources:
    score: 0.9-1.0
    types:
      - Academic journals
      - Government publications
      - Official documentation
      - Peer-reviewed papers
    
  tier_2_sources:
    score: 0.7-0.9
    types:
      - Established media
      - Industry reports
      - Expert blogs
      - Technical forums
    
  tier_3_sources:
    score: 0.5-0.7
    types:
      - Community resources
      - User documentation
      - Social media (verified)
      - Wikipedia
    
  tier_4_sources:
    score: 0.3-0.5
    types:
      - User forums
      - Social media (unverified)
      - Personal blogs
      - Comments sections

Depth Configurations

research_depth_profiles:
  quick:
    max_sources: 10
    max_hops: 1
    iterations: 1
    time_limit: 2 minutes
    confidence_target: 0.6
    extraction: tavily_only
    
  standard:
    max_sources: 20
    max_hops: 3
    iterations: 2
    time_limit: 5 minutes
    confidence_target: 0.7
    extraction: selective
    
  deep:
    max_sources: 40
    max_hops: 4
    iterations: 3
    time_limit: 8 minutes
    confidence_target: 0.8
    extraction: comprehensive
    
  exhaustive:
    max_sources: 50+
    max_hops: 5
    iterations: 5
    time_limit: 10 minutes
    confidence_target: 0.9
    extraction: all_sources

Multi-Hop Patterns

hop_patterns:
  entity_expansion:
    description: "Explore entities found in previous hop"
    example: "Paper → Authors → Other works → Collaborators"
    max_branches: 3
    
  concept_deepening:
    description: "Drill down into concepts"
    example: "Topic → Subtopics → Details → Examples"
    max_depth: 4
    
  temporal_progression:
    description: "Follow chronological development"
    example: "Current → Recent → Historical → Origins"
    direction: backward
    
  causal_chain:
    description: "Trace cause and effect"
    example: "Effect → Immediate cause → Root cause → Prevention"
    validation: required

Extraction Routing Rules

extraction_routing:
  use_tavily:
    conditions:
      - Static HTML content
      - Simple article structure
      - No JavaScript requirement
      - Public access
    
  use_playwright:
    conditions:
      - JavaScript rendering required
      - Dynamic content present
      - Authentication needed
      - Interactive elements
      - Screenshots required
    
  use_context7:
    conditions:
      - Technical documentation
      - API references
      - Framework guides
      - Library documentation
    
  use_native:
    conditions:
      - Local file access
      - Simple explanations
      - Code generation
      - General knowledge

Case-Based Learning Schema

case_schema:
  case_id:
    format: "research_[timestamp]_[topic_hash]"
    
  case_content:
    query: "original research question"
    strategy_used: "planning approach"
    successful_patterns:
      - query_formulations: []
      - extraction_methods: []
      - synthesis_approaches: []
    findings:
      key_discoveries: []
      source_credibility_scores: {}
      confidence_levels: {}
    lessons_learned:
      what_worked: []
      what_failed: []
      optimizations: []
    metrics:
      time_taken: seconds
      sources_processed: count
      hops_executed: count
      confidence_achieved: float

Replanning Thresholds

replanning_triggers:
  confidence_based:
    critical: < 0.4
    low: < 0.6
    acceptable: 0.6-0.7
    good: > 0.7
    
  time_based:
    warning: 70% of limit
    critical: 90% of limit
    
  quality_based:
    insufficient_sources: < 3
    contradictions: > 30%
    gaps_identified: > 50%
    
  user_based:
    explicit_request: immediate
    implicit_dissatisfaction: assess

Output Format Templates

output_formats:
  summary:
    max_length: 500 words
    sections: [key_finding, evidence, sources]
    confidence_display: simple
    
  report:
    sections: [executive_summary, methodology, findings, synthesis, conclusions]
    citations: inline
    confidence_display: detailed
    visuals: included
    
  academic:
    sections: [abstract, introduction, methodology, literature_review, findings, discussion, conclusions]
    citations: academic_format
    confidence_display: statistical
    appendices: true

Error Handling

error_handling:
  tavily_errors:
    api_key_missing: "Check TAVILY_API_KEY environment variable"
    rate_limit: "Wait and retry with exponential backoff"
    no_results: "Expand search terms or try alternatives"
    
  playwright_errors:
    timeout: "Skip source or increase timeout"
    navigation_failed: "Mark as inaccessible, continue"
    screenshot_failed: "Continue without visual"
    
  quality_errors:
    low_confidence: "Trigger replanning"
    contradictions: "Seek additional sources"
    insufficient_data: "Expand search scope"

Integration Points

mcp_integration:
  tavily:
    role: primary_search
    fallback: native_websearch
    
  playwright:
    role: complex_extraction
    fallback: tavily_extraction
    
  sequential:
    role: reasoning_engine
    fallback: native_reasoning
    
  context7:
    role: technical_docs
    fallback: tavily_search
    
  serena:
    role: memory_management
    fallback: session_only

Monitoring Metrics

metrics_tracking:
  performance:
    - search_latency
    - extraction_time
    - synthesis_duration
    - total_research_time
    
  quality:
    - confidence_scores
    - source_diversity
    - coverage_completeness
    - contradiction_rate
    
  efficiency:
    - cache_hit_rate
    - parallel_execution_rate
    - memory_usage
    - api_cost
    
  learning:
    - pattern_reuse_rate
    - strategy_success_rate
    - improvement_trajectory