Implement comprehensive document sharding system across all BMM workflows enabling 90%+ token savings for large multi-epic projects through selective loading optimization.
## Document Sharding System
### Core Features
- **Universal Support**: All 12 BMM workflows (Phase 1-4) handle both whole and sharded documents
- **Dual Loading Strategy**: Full Load (Phase 1-3) vs Selective Load (Phase 4)
- **Automatic Discovery**: Workflows detect format transparently (whole → sharded priority)
- **Efficiency Optimization**: 90%+ token reduction for 10+ epic projects in Phase 4
### Implementation Details
**Phase 1-3 Workflows (7 workflows) - Full Load Strategy:**
- product-brief, prd, gdd, create-ux-design, tech-spec, architecture, solutioning-gate-check
- Load entire sharded documents when present
- Transparent to user experience
- Better organization for large projects
**Phase 4 Workflows (5 workflows) - Selective Load Strategy:**
- sprint-planning (Full Load exception - needs all epics)
- epic-tech-context, create-story, story-context, code-review (Selective Load)
- Load ONLY the specific epic needed (e.g., epic-3.md for Epic 3 stories)
- Massive efficiency: Skip loading 9 other epics in 10-epic project
### Workflow Enhancements
**Added to all workflows:**
- `input_file_patterns` in workflow.yaml with wildcard discovery
- Document Discovery section in instructions.md
- Support for sharded index + section files
- Brownfield `docs/index.md` support
**Pattern standardization:**
```yaml
input_file_patterns:
document:
whole: "{output_folder}/*doc*.md"
sharded: "{output_folder}/*doc*/index.md"
sharded_single: "{output_folder}/*doc*/section-{{id}}.md" # Selective load
```
### Retrospective Workflow Major Overhaul
Transformed retrospective into immersive, interactive team experience:
**Epic Discovery Priority (Fixed):**
- Priority 1: Check sprint-status.yaml for last completed epic
- Priority 2: Ask user directly
- Priority 3: Scan stories folder (last resort)
**New Capabilities:**
- Deep story analysis: Extract dev notes, mistakes, review feedback, lessons learned
- Previous retro integration: Track action items, verify lessons applied
- Significant change detection: Alert when discoveries require epic updates
- Intent-based facilitation: Natural conversation vs scripted phrases
- Party mode protocol: Clear speaker identification (Name (Role): dialogue)
- Team dynamics: Drama, disagreements, diverse perspectives, authentic conflict
**Structure:**
- 12 whole-number steps (no decimals)
- Highly interactive with constant user engagement
- Cross-references previous retro for accountability
- Synthesizes patterns across all stories
- Detects architectural assumption changes
## Documentation
**Created:**
- `docs/document-sharding-guide.md` - Comprehensive 300+ line guide
- What is sharding, when to use it (token thresholds)
- How sharding works (discovery system, loading strategies)
- Using shard-doc tool
- Full Load vs Selective Load patterns
- Complete examples and troubleshooting
- Custom workflow integration patterns
**Updated:**
- `README.md` - Added Document Sharding feature section
- `docs/index.md` - Added under Advanced Topics → Optimization
- `src/modules/bmm/workflows/README.md` - Added sharding section with usage
- `src/modules/bmb/workflows/create-workflow/workflow-creation-guide.md` - Added complete implementation patterns for workflow builders
**Documentation levels:**
1. Overview (README.md) - Quick feature highlight
2. User guide (BMM workflows README) - Practical usage
3. Reference (document-sharding-guide.md) - Complete details
4. Builder guide (workflow-creation-guide.md) - Implementation patterns
## Efficiency Gains
**Example: 10-Epic Project**
Before sharding:
- epic-tech-context for Epic 3: Load all 10 epics (~50k tokens)
- create-story for Epic 3: Load all 10 epics (~50k tokens)
- story-context for Epic 3: Load all 10 epics (~50k tokens)
After sharding with selective load:
- epic-tech-context for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
- create-story for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
- story-context for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
## Breaking Changes
None - fully backward compatible. Workflows work with existing whole documents.
## Files Changed
**Workflows Updated (25 files):**
- 7 Phase 1-3 workflows: Added full load sharding support
- 5 Phase 4 workflows: Added selective load sharding support
- 1 retrospective workflow: Complete overhaul with sharding support
**Documentation (5 files):**
- Created: document-sharding-guide.md
- Updated: README.md, docs/index.md, BMM workflows README, BMB workflow-creation-guide
- Removed: Old conversion report (obsolete)
## Future Extensibility
- BMB workflows now aware of sharding patterns
- Custom modules can easily implement sharding support
- Standard patterns documented for consistency
- No need to explain concept in future development
12 KiB
Implementation Ready Check - Workflow Instructions
The workflow execution engine is governed by: {project-root}/bmad/core/tasks/workflow.xml You MUST have already loaded and processed: {project-root}/bmad/bmm/workflows/3-solutioning/solutioning-gate-check/workflow.yaml Communicate all findings and analysis in {communication_language} throughout the assessment
📚 Input Document Discovery
This workflow validates: PRD, epics/stories, architecture, and may reference UX design, tech specs, or brownfield project documentation.
Discovery Process (execute for each referenced document):
- Search for whole document first - Use fuzzy file matching to find the complete document
- Check for sharded version - If whole document not found, look for
{doc-name}/index.md - If sharded version found:
- Read
index.mdto understand the document structure - Read ALL section files listed in the index
- Treat the combined content as if it were a single document
- Read
- Brownfield projects: The
document-projectworkflow always creates{output_folder}/docs/index.md
Priority: If both whole and sharded versions exist, use the whole document.
Fuzzy matching: Be flexible with document names - users may use variations in naming conventions.
Check if {output_folder}/bmm-workflow-status.yaml exists No workflow status file found. Implementation Ready Check can run standalone or as part of BMM workflow path. **Recommended:** Run `workflow-init` first for project context tracking and workflow sequencing. Continue in standalone mode or exit to run workflow-init? (continue/exit) Set standalone_mode = true Exit workflow Load the FULL file: {output_folder}/bmm-workflow-status.yaml Parse workflow_status section Check status of "solutioning-gate-check" workflow Get project_level from YAML metadata Find first non-completed workflow (next expected workflow)Based on the project_level, understand what artifacts should exist: - Level 0-1: Tech spec and simple stories only (no PRD, minimal solutioning) - Level 2: PRD, tech spec, epics/stories (no separate architecture doc) - Level 3-4: Full suite - PRD, architecture document, epics/stories, possible UX artifacts
⚠️ Gate check already completed: {{solutioning-gate-check status}} Re-running will create a new validation report. Continue? (y/n) Exiting. Use workflow-status to see your next step. Exit workflow ⚠️ Next expected workflow: {{next_workflow}}. Gate check is out of sequence. Continue with gate check anyway? (y/n) Exiting. Run {{next_workflow}} instead. Exit workflowSet standalone_mode = false
The validation approach must adapt to the project level - don't look for documents that shouldn't exist at lower levels
project_context
Search the {output_folder} for relevant planning and solutioning documents based on project level identified in Step 0For Level 0-1 projects, locate:
- Technical specification document(s)
- Story/task lists or simple epic breakdowns
- Any API or interface definitions
For Level 2-4 projects, locate:
- Product Requirements Document (PRD)
- Architecture document (architecture.md) (Level 3-4 only)
- Technical Specification (Level 2 includes architecture within)
- Epic and story breakdowns
- UX artifacts if the active path includes UX workflow
- Any supplementary planning documents
Create an inventory of found documents with:
- Document type and purpose
- File path and last modified date
- Brief description of what each contains
- Any missing expected documents flagged as potential issues
document_inventory
Load and thoroughly analyze each discovered document to extract: - Core requirements and success criteria - Architectural decisions and constraints - Technical implementation approaches - User stories and acceptance criteria - Dependencies and sequencing requirements - Any assumptions or risks documentedFor PRD analysis (Level 2-4), focus on:
- User requirements and use cases
- Functional and non-functional requirements
- Success metrics and acceptance criteria
- Scope boundaries and explicitly excluded items
- Priority levels for different features
For Architecture/Tech Spec analysis, focus on:
- System design decisions and rationale
- Technology stack and framework choices
- Integration points and APIs
- Data models and storage decisions
- Security and performance considerations
- Any architectural constraints that might affect story implementation
For Epic/Story analysis, focus on:
- Coverage of PRD requirements
- Story sequencing and dependencies
- Acceptance criteria completeness
- Technical tasks within stories
- Estimated complexity and effort indicators
document_analysis
Systematically validate alignment between all artifacts, adapting validation based on project levelPRD ↔ Architecture Alignment (Level 3-4):
- Verify every PRD requirement has corresponding architectural support
- Check that architectural decisions don't contradict PRD constraints
- Identify any architectural additions beyond PRD scope (potential gold-plating)
- Ensure non-functional requirements from PRD are addressed in architecture document
- If using new architecture workflow: verify implementation patterns are defined
PRD ↔ Stories Coverage (Level 2-4):
- Map each PRD requirement to implementing stories
- Identify any PRD requirements without story coverage
- Find stories that don't trace back to PRD requirements
- Validate that story acceptance criteria align with PRD success criteria
Architecture ↔ Stories Implementation Check:
- Verify architectural decisions are reflected in relevant stories
- Check that story technical tasks align with architectural approach
- Identify any stories that might violate architectural constraints
- Ensure infrastructure and setup stories exist for architectural components
For Level 0-1 projects (Tech Spec only):
- Validate internal consistency within tech spec
- Check that all specified features have corresponding stories
- Verify story sequencing matches technical dependencies
alignment_validation
Identify and categorize all gaps, risks, and potential issues discovered during validationCheck for Critical Gaps:
- Missing stories for core requirements
- Unaddressed architectural concerns
- Absent infrastructure or setup stories for greenfield projects
- Missing error handling or edge case coverage
- Security or compliance requirements not addressed
Identify Sequencing Issues:
- Dependencies not properly ordered
- Stories that assume components not yet built
- Parallel work that should be sequential
- Missing prerequisite technical tasks
Detect Potential Contradictions:
- Conflicts between PRD and architecture approaches
- Stories with conflicting technical approaches
- Acceptance criteria that contradict requirements
- Resource or technology conflicts
Find Gold-Plating and Scope Creep:
- Features in architecture not required by PRD
- Stories implementing beyond requirements
- Technical complexity beyond project needs
- Over-engineering indicators
gap_risk_analysis
Review UX artifacts and validate integration: - Check that UX requirements are reflected in PRD - Verify stories include UX implementation tasks - Ensure architecture supports UX requirements (performance, responsiveness) - Identify any UX concerns not addressed in storiesValidate accessibility and usability coverage:
- Check for accessibility requirement coverage in stories
- Verify responsive design considerations if applicable
- Ensure user flow completeness across stories
ux_validation
Compile all findings into a structured readiness report with: - Executive summary of readiness status - Project context and validation scope - Document inventory and coverage assessment - Detailed findings organized by severity (Critical, High, Medium, Low) - Specific recommendations for each issue - Overall readiness recommendation (Ready, Ready with Conditions, Not Ready)Provide actionable next steps:
- List any critical issues that must be resolved
- Suggest specific document updates needed
- Recommend additional stories or tasks required
- Propose sequencing adjustments if needed
Include positive findings:
- Highlight well-aligned areas
- Note particularly thorough documentation
- Recognize good architectural decisions
- Commend comprehensive story coverage where found
readiness_assessment
Load the FULL file: {output_folder}/bmm-workflow-status.yaml Find workflow_status key "solutioning-gate-check" ONLY write the file path as the status value - no other text, notes, or metadata Update workflow_status["solutioning-gate-check"] = "{output_folder}/bmm-readiness-assessment-{{date}}.md" Save file, preserving ALL comments and structure including STATUS DEFINITIONSFind first non-completed workflow in workflow_status (next workflow to do) Determine next agent from path file based on next workflow
✅ Implementation Ready Check Complete!
Assessment Report:
- Readiness assessment saved to: {output_folder}/bmm-readiness-assessment-{{date}}.md
{{#if standalone_mode != true}} Status Updated:
- Progress tracking updated: solutioning-gate-check marked complete
- Next workflow: {{next_workflow}} {{else}} Note: Running in standalone mode (no progress tracking) {{/if}}
Next Steps:
{{#if standalone_mode != true}}
- Next workflow: {{next_workflow}} ({{next_agent}} agent)
- Review the assessment report and address any critical issues before proceeding
Check status anytime with: workflow-status
{{else}}
Since no workflow is in progress:
- Refer to the BMM workflow guide if unsure what to do next
- Or run
workflow-initto create a workflow path and get guided next steps {{/if}}
status_update_result