This document describes the internal architecture of the workflow engine — how definitions are parsed, steps are dispatched, state is persisted, and catalogs are resolved.
For usage instructions, see [README.md](README.md).
## Execution Model
When `specify workflow run` is invoked, the engine loads a YAML definition, resolves inputs, and dispatches steps sequentially through the step registry:
```mermaid
flowchart TD
A["specify workflow run my-workflow"] --> B["WorkflowEngine.load_workflow()"]
B --> C["WorkflowDefinition.from_yaml()"]
C --> D["_resolve_inputs()"]
D --> E["validate_workflow()"]
E --> F["RunState.create()"]
F --> G["_execute_steps()"]
G --> H{Step type?}
H -- command --> I["CommandStep.execute()"]
H -- shell --> J["ShellStep.execute()"]
H -- gate --> K["GateStep.execute()"]
H -- "if" --> L["IfThenStep.execute()"]
H -- switch --> M["SwitchStep.execute()"]
H -- "while/do-while" --> N["Loop steps"]
H -- "fan-out/fan-in" --> O["Fan-out/fan-in"]
I --> P{Result status?}
J --> P
K --> P
L --> P
M --> P
N --> P
O --> P
P -- COMPLETED --> Q{Has next_steps?}
P -- PAUSED --> R["Save state → exit"]
P -- FAILED --> S["Log error → exit"]
Q -- Yes --> G
Q -- No --> T{More steps?}
T -- Yes --> G
T -- No --> U["Status = COMPLETED"]
style R fill:#ff9800,color:#fff
style S fill:#f44336,color:#fff
style U fill:#4caf50,color:#fff
```
### Sequential Execution
Steps execute sequentially. Each step receives a `StepContext` containing resolved inputs, accumulated step results, and workflow-level defaults. After execution, the step's output is stored in `context.steps[step_id]` and made available to subsequent steps via expressions like `{{ steps.specify.output.file }}`.
### Nested Steps (Control Flow)
Steps like `if`, `switch`, `while`, and `do-while` return `next_steps` — inline step definitions that the engine executes recursively via `_execute_steps()`. Nested steps share the same `StepContext` and `RunState`, so their outputs are visible to later top-level steps.
### State Persistence and Resume
The engine saves `RunState` to disk after each step, enabling resume from the exact point of interruption:
```mermaid
flowchart LR
A["CREATED"] --> B["RUNNING"]
B --> C["COMPLETED"]
B --> D["PAUSED"]
B --> E["FAILED"]
B --> F["ABORTED"]
D -- "resume()" --> B
E -- "resume()" --> B
```
When a `gate` step pauses execution, the engine persists `current_step_index` and all accumulated `step_results`. On `specify workflow resume <run_id>`, the engine restores the context and continues from the paused step.
> **Note:** Resume tracking is at the top-level step index only. If a
| `while` | `WhileStep` | Loop while condition is truthy | Yes (if true) |
| `do-while` | `DoWhileStep` | Loop, always runs body at least once | Yes (always) |
| `fan-out` | `FanOutStep` | Dispatch per item over a collection | No (engine expands) |
| `fan-in` | `FanInStep` | Aggregate results from fan-out | No |
## Step Registry
All step types register into `STEP_REGISTRY` via `_register_builtin_steps()` in `src/specify_cli/workflows/__init__.py`. The registry maps `type_key` strings to step instances:
Registration is explicit — each step class is imported and instantiated. New step types follow the same pattern: subclass `StepBase`, set `type_key`, implement `execute()` and optionally `validate()`.
## Expression Engine
Workflow definitions use Jinja2-like `{{ expression }}` syntax for dynamic values. The expression engine in `src/specify_cli/workflows/expressions.py` supports:
C -- No --> E{.specify/workflow-catalogs.yml exists?}
E -- Yes --> F["Project-level catalog stack"]
E -- No --> G{"~/.specify/workflow-catalogs.yml exists?"}
G -- Yes --> H["User-level catalog stack"]
G -- No --> I["Built-in defaults"]
I --> J["default (install allowed)"]
I --> K["community (discovery only)"]
style D fill:#ff9800,color:#fff
style F fill:#2196f3,color:#fff
style H fill:#2196f3,color:#fff
style J fill:#4caf50,color:#fff
style K fill:#9e9e9e,color:#fff
```
Catalogs are fetched with a 1-hour cache (per-URL, SHA256-hashed cache files in `.specify/workflows/.cache/`). Each catalog entry has a `priority` (for merge ordering) and `install_allowed` flag.
When `specify workflow add <id>` installs from catalog, it downloads the workflow YAML from the catalog entry's `url` field into `.specify/workflows/<id>/workflow.yml`.
