mirror of
https://github.com/bmadcode/BMAD-METHOD.git
synced 2025-12-29 16:14:59 +00:00
feat: add universal document sharding support with dual-strategy loading
Implement comprehensive document sharding system across all BMM workflows enabling 90%+ token savings for large multi-epic projects through selective loading optimization.
## Document Sharding System
### Core Features
- **Universal Support**: All 12 BMM workflows (Phase 1-4) handle both whole and sharded documents
- **Dual Loading Strategy**: Full Load (Phase 1-3) vs Selective Load (Phase 4)
- **Automatic Discovery**: Workflows detect format transparently (whole → sharded priority)
- **Efficiency Optimization**: 90%+ token reduction for 10+ epic projects in Phase 4
### Implementation Details
**Phase 1-3 Workflows (7 workflows) - Full Load Strategy:**
- product-brief, prd, gdd, create-ux-design, tech-spec, architecture, solutioning-gate-check
- Load entire sharded documents when present
- Transparent to user experience
- Better organization for large projects
**Phase 4 Workflows (5 workflows) - Selective Load Strategy:**
- sprint-planning (Full Load exception - needs all epics)
- epic-tech-context, create-story, story-context, code-review (Selective Load)
- Load ONLY the specific epic needed (e.g., epic-3.md for Epic 3 stories)
- Massive efficiency: Skip loading 9 other epics in 10-epic project
### Workflow Enhancements
**Added to all workflows:**
- `input_file_patterns` in workflow.yaml with wildcard discovery
- Document Discovery section in instructions.md
- Support for sharded index + section files
- Brownfield `docs/index.md` support
**Pattern standardization:**
```yaml
input_file_patterns:
document:
whole: "{output_folder}/*doc*.md"
sharded: "{output_folder}/*doc*/index.md"
sharded_single: "{output_folder}/*doc*/section-{{id}}.md" # Selective load
```
### Retrospective Workflow Major Overhaul
Transformed retrospective into immersive, interactive team experience:
**Epic Discovery Priority (Fixed):**
- Priority 1: Check sprint-status.yaml for last completed epic
- Priority 2: Ask user directly
- Priority 3: Scan stories folder (last resort)
**New Capabilities:**
- Deep story analysis: Extract dev notes, mistakes, review feedback, lessons learned
- Previous retro integration: Track action items, verify lessons applied
- Significant change detection: Alert when discoveries require epic updates
- Intent-based facilitation: Natural conversation vs scripted phrases
- Party mode protocol: Clear speaker identification (Name (Role): dialogue)
- Team dynamics: Drama, disagreements, diverse perspectives, authentic conflict
**Structure:**
- 12 whole-number steps (no decimals)
- Highly interactive with constant user engagement
- Cross-references previous retro for accountability
- Synthesizes patterns across all stories
- Detects architectural assumption changes
## Documentation
**Created:**
- `docs/document-sharding-guide.md` - Comprehensive 300+ line guide
- What is sharding, when to use it (token thresholds)
- How sharding works (discovery system, loading strategies)
- Using shard-doc tool
- Full Load vs Selective Load patterns
- Complete examples and troubleshooting
- Custom workflow integration patterns
**Updated:**
- `README.md` - Added Document Sharding feature section
- `docs/index.md` - Added under Advanced Topics → Optimization
- `src/modules/bmm/workflows/README.md` - Added sharding section with usage
- `src/modules/bmb/workflows/create-workflow/workflow-creation-guide.md` - Added complete implementation patterns for workflow builders
**Documentation levels:**
1. Overview (README.md) - Quick feature highlight
2. User guide (BMM workflows README) - Practical usage
3. Reference (document-sharding-guide.md) - Complete details
4. Builder guide (workflow-creation-guide.md) - Implementation patterns
## Efficiency Gains
**Example: 10-Epic Project**
Before sharding:
- epic-tech-context for Epic 3: Load all 10 epics (~50k tokens)
- create-story for Epic 3: Load all 10 epics (~50k tokens)
- story-context for Epic 3: Load all 10 epics (~50k tokens)
After sharding with selective load:
- epic-tech-context for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
- create-story for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
- story-context for Epic 3: Load Epic 3 only (~5k tokens) = 90% reduction
## Breaking Changes
None - fully backward compatible. Workflows work with existing whole documents.
## Files Changed
**Workflows Updated (25 files):**
- 7 Phase 1-3 workflows: Added full load sharding support
- 5 Phase 4 workflows: Added selective load sharding support
- 1 retrospective workflow: Complete overhaul with sharding support
**Documentation (5 files):**
- Created: document-sharding-guide.md
- Updated: README.md, docs/index.md, BMM workflows README, BMB workflow-creation-guide
- Removed: Old conversion report (obsolete)
## Future Extensibility
- BMB workflows now aware of sharding patterns
- Custom modules can easily implement sharding support
- Standard patterns documented for consistency
- No need to explain concept in future development
This commit is contained in:
@@ -10,6 +10,24 @@
|
||||
<critical>Generate all documents in {document_output_language}</critical>
|
||||
<critical>This workflow replaces architecture with a conversation-driven approach</critical>
|
||||
|
||||
## 📚 Input Document Discovery
|
||||
|
||||
This workflow requires: PRD and epics/stories, and may reference UX design specifications or brownfield project documentation.
|
||||
|
||||
**Discovery Process** (execute for each referenced document):
|
||||
|
||||
1. **Search for whole document first** - Use fuzzy file matching to find the complete document
|
||||
2. **Check for sharded version** - If whole document not found, look for `{doc-name}/index.md`
|
||||
3. **If sharded version found**:
|
||||
- Read `index.md` to understand the document structure
|
||||
- Read ALL section files listed in the index
|
||||
- Treat the combined content as if it were a single document
|
||||
4. **Brownfield projects**: The `document-project` workflow always creates `{output_folder}/docs/index.md`
|
||||
|
||||
**Priority**: If both whole and sharded versions exist, use the whole document.
|
||||
|
||||
**Fuzzy matching**: Be flexible with document names - users may use variations in naming conventions.
|
||||
|
||||
<step n="0" goal="Validate workflow readiness" tag="workflow-status">
|
||||
<action>Check if {output_folder}/bmm-workflow-status.yaml exists</action>
|
||||
|
||||
|
||||
@@ -18,10 +18,23 @@ recommended_inputs:
|
||||
- epics: "Epic definitions with user stories and acceptance criteria"
|
||||
- ux_spec: "UX specification with interface designs and interaction patterns (optional)"
|
||||
|
||||
# Input file references (fuzzy matched from output folder)
|
||||
prd_file: "{output_folder}/*PRD*, PRD.md, or prd/index.md + files"
|
||||
epics_file: "{output_folder}/bmm-epics.md or epics.md or user-stories.md"
|
||||
ux_spec_file: "{output_folder}/ux-spec.md or ux-specification.md or user-experience.md"
|
||||
# Smart input file references - handles both whole docs and sharded docs
|
||||
# Priority: Whole document first, then sharded version
|
||||
input_file_patterns:
|
||||
prd:
|
||||
whole: "{output_folder}/*prd*.md"
|
||||
sharded: "{output_folder}/*prd*/index.md"
|
||||
|
||||
epics:
|
||||
whole: "{output_folder}/*epic*.md"
|
||||
sharded: "{output_folder}/*epic*/index.md"
|
||||
|
||||
ux_design:
|
||||
whole: "{output_folder}/*ux*.md"
|
||||
sharded: "{output_folder}/*ux*/index.md"
|
||||
|
||||
document_project:
|
||||
sharded: "{output_folder}/docs/index.md"
|
||||
|
||||
# Module path and component files
|
||||
installed_path: "{project-root}/bmad/bmm/workflows/3-solutioning/architecture"
|
||||
|
||||
@@ -4,6 +4,24 @@
|
||||
<critical>You MUST have already loaded and processed: {project-root}/bmad/bmm/workflows/3-solutioning/solutioning-gate-check/workflow.yaml</critical>
|
||||
<critical>Communicate all findings and analysis in {communication_language} throughout the assessment</critical>
|
||||
|
||||
## 📚 Input Document Discovery
|
||||
|
||||
This workflow validates: PRD, epics/stories, architecture, and may reference UX design, tech specs, or brownfield project documentation.
|
||||
|
||||
**Discovery Process** (execute for each referenced document):
|
||||
|
||||
1. **Search for whole document first** - Use fuzzy file matching to find the complete document
|
||||
2. **Check for sharded version** - If whole document not found, look for `{doc-name}/index.md`
|
||||
3. **If sharded version found**:
|
||||
- Read `index.md` to understand the document structure
|
||||
- Read ALL section files listed in the index
|
||||
- Treat the combined content as if it were a single document
|
||||
4. **Brownfield projects**: The `document-project` workflow always creates `{output_folder}/docs/index.md`
|
||||
|
||||
**Priority**: If both whole and sharded versions exist, use the whole document.
|
||||
|
||||
**Fuzzy matching**: Be flexible with document names - users may use variations in naming conventions.
|
||||
|
||||
<workflow>
|
||||
|
||||
<step n="0" goal="Validate workflow readiness" tag="workflow-status">
|
||||
|
||||
@@ -32,6 +32,32 @@ recommended_inputs:
|
||||
- epics_stories: "{output_folder}/epic*.md"
|
||||
- ux_artifacts: "{output_folder}/ux*.md"
|
||||
|
||||
# Smart input file references - handles both whole docs and sharded docs
|
||||
# Priority: Whole document first, then sharded version
|
||||
input_file_patterns:
|
||||
prd:
|
||||
whole: "{output_folder}/*prd*.md"
|
||||
sharded: "{output_folder}/*prd*/index.md"
|
||||
|
||||
epics:
|
||||
whole: "{output_folder}/*epic*.md"
|
||||
sharded: "{output_folder}/*epic*/index.md"
|
||||
|
||||
architecture:
|
||||
whole: "{output_folder}/*architecture*.md"
|
||||
sharded: "{output_folder}/*architecture*/index.md"
|
||||
|
||||
ux_design:
|
||||
whole: "{output_folder}/*ux*.md"
|
||||
sharded: "{output_folder}/*ux*/index.md"
|
||||
|
||||
tech_spec:
|
||||
whole: "{output_folder}/*tech-spec*.md"
|
||||
sharded: "{output_folder}/*tech-spec*/index.md"
|
||||
|
||||
document_project:
|
||||
sharded: "{output_folder}/docs/index.md"
|
||||
|
||||
# Validation criteria data
|
||||
validation_criteria: "{installed_path}/validation-criteria.yaml"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user