diff --git a/README.md b/README.md index 55519e2..673b0c8 100644 --- a/README.md +++ b/README.md @@ -164,6 +164,8 @@ Developing the dashboard itself needs node: `make web-dev` runs a Vite dev serve - [examples/ops-agent](examples/ops-agent): a complete agent folder with connections, modules, a command, and an archived module - [docs/design.md](docs/design.md): why gcontext is built this way, decision by decision - [docs/modules.md](docs/modules.md): writing portable, shareable modules +- [docs/workflows.md](docs/workflows.md): the workflow template standard, the contract for distributable context-based workflows +- [docs/share-workflow.md](docs/share-workflow.md): instructions an author's agent follows to turn a lived workflow into a shareable template ## Scope diff --git a/docs/share-workflow.md b/docs/share-workflow.md new file mode 100644 index 0000000..3270ecc --- /dev/null +++ b/docs/share-workflow.md @@ -0,0 +1,66 @@ +# share-workflow + +Instructions for an AI agent that turns a private, lived workflow into a marketplace template. You are the agent; the human in the conversation is the author. The input is their workflow module, personal state included. The output is a template folder that follows the workflow template standard (docs/workflows.md) and contains zero personal data. + +Work through the phases in order. Propose, let the author confirm, then act. Do not skip a phase. + +## Phase 0: load + +1. Ask the author which workflow module to share, if not already stated. Read it completely: `index.md`, `steps/`, `functions/` and `commands/` if present, and the run folders in `runs/`. +2. Read the template spec (`docs/workflows.md`, or fetch it from the gcontext repo if it is not in reach). Do not work from memory of the spec; the spec is the contract the output must pass. + +## Phase 1: eligibility + +Check the workflow against the five tests for shareable workflows and report the result per test: + +1. **Procedure generic, state personal.** The steps contain a loop that is not tied to the author's life; the personal content sits in state files that can start empty for someone else. +2. **The AI is needed every run.** Each run requires interpretation and judgment. If a cron job or a static script could do it, it is not a workflow. +3. **The state feeds judgment.** Later runs read the accumulated state to act better. Knowledge qualifies; telemetry nobody consults does not. +4. **Lived first.** At least one real, completed run exists in `runs/`. The shapes must have been discovered by use, not designed on paper. +5. **Low trust barrier.** Few parameter slots, few secrets. Every credential a stranger must grant raises the install cost. + +If any test fails, tell the author which and why, and continue only after their explicit confirmation. The marketplace is open; you warn, the author decides. + +## Phase 2: extract the slots + +Walk `index.md`, every file in `steps/`, and `functions/` if present. Collect every author-specific element: personal and company names, domains, URLs, account identifiers, email addresses, concrete product names, file paths outside the module, and data values from the author's own work. + +Classify each element as exactly one of: + +- **(a) Parameter slot**: a value a new user supplies at setup or per run. Becomes a `parameters` entry in the manifest. +- **(b) Connection requirement**: a service capability the workflow needs. Becomes a structured `connections` entry (`kind` + `description`), described generically ("the hosting panel API", not "Coolify"). +- **(c) Personal state**: files or content that must not ship (playbooks learned from the author's systems, configs, credentials references, logs). Excluded from the template; the setup command will regenerate the empty shapes. +- **(d) Generic rewrite**: a concrete-service mention inside a step that stays in the text but must be reworded to the capability kind. + +Present the full classification as one list and get the author's confirmation before rewriting anything. This is the author's main control point; the leak scan in phase 5 is the second net. + +## Phase 3: generate the template + +Create the template folder next to the source module (for example `-template/`). Build: + +- **`index.md`**: the frontmatter manifest per the spec: `id` (url-safe slug), `name`, `description`, `parameters` (name, description, required), `connections` (kind, description), `tags`. Then the body, rewritten clean: the objective in the first paragraph, what each parameter means in practice, the workflow's run naming scheme, and the general cross-step context. +- **`steps/`**: the same files as the source, with the classified specifics replaced by parameter references and generic capability wording. Keep the structure untouched: the shapes were proven by use; you strip, you do not redesign. Every step file must state Purpose, Input, Output (with schema when tabular), How to execute, and Done when; if a source step lacks one of these, derive it from what the lived runs show and confirm with the author. +- **`functions/`**: same treatment, only if the source has it. +- **`commands/setup.md`**: generate it from the slots, following the setup contract in the spec: read index.md and steps/index.md first; bind every setup-time parameter; map each connection requirement to a real service in the user's environment; generate the personal state (list in the command exactly what it creates); smoke-test the critical path; never edit steps/. Give it command frontmatter (`description`, optional `parameters`) and a self-contained prose body that assumes only file access, so it works in gcontext as an MCP prompt and standalone in any agent. + +## Phase 4: fabricate the example run + +Build `runs/example/` inside the template, in the exact runs/ shape: `index.md` (map and status), `0-parameters.*`, one folder per step with results, `done/info.md`. + +- Default: start from the author's most representative real run and replace every real value with a coherent fake: invented names, plausible numbers, same schemas, same story arc. +- Fallback: if the author's runs are too sensitive to anonymize confidently, fabricate the example fully from the step definitions. Say so to the author. +- Keep the fake data internally consistent: the same invented name must flow through all steps, so a site visitor can follow one item from parameters to done. This example is what the marketplace renders on the workflow's page; it is the template's sales pitch. + +## Phase 5: verify + +Run three checks and show the results: + +1. **Spec compliance**: every required file exists (index.md with parseable frontmatter carrying all fields, steps/ with index and numbered files, commands/setup.md, runs/example/ complete with done/); every step states Purpose, Input, Output, How, Done when. +2. **Leak scan**: search the entire template, example run included, for every author-specific string collected in phase 2, plus generic patterns: email addresses, things shaped like API keys or tokens, the author's domains. Present every hit. The template passes only with zero unexplained hits. +3. **Cold read**: in a fresh context (a subagent or a new session) that sees only the template folder, have the agent explain back what the workflow does, what it needs, and what a run produces. If the explanation is wrong or incomplete, the template is not self-contained; fix and repeat. + +## Phase 6: hand off + +The finished template is a local folder. Submission: the marketplace accepts templates through its API with a review step (submitted entries stay pending until approved). If the submission endpoint is not yet available, tell the author the template is ready and where it lives, and stop there. + +Never submit without the author's explicit go-ahead, and never include the source module or any personal state in what is submitted. diff --git a/docs/workflows.md b/docs/workflows.md new file mode 100644 index 0000000..8406d51 --- /dev/null +++ b/docs/workflows.md @@ -0,0 +1,167 @@ +# Workflows + +A context-based workflow is a module with a fixed shape. It is a series of steps the agent executes with judgment, where every run leaves a persistent trace on disk. The workflow remembers what happened last run, accumulates knowledge, and gets better over time. A skill or prompt runs and forgets; a workflow holds state. + +This document is the template spec: the contract a workflow folder must follow to be distributable. The CLI (`gcontext add`), the site directory, and the authoring tooling all build against it. It is one standard for all workflows; there are no per-domain variants. + +## Folder anatomy + +``` +/ + index.md # required: frontmatter manifest + objective, parameters, context + steps/ # required: the procedure + index.md # map: one line per step + 0-preflight.md # numbered step files, executed in order + 1-init.md + ... + commands/ # required: entry points + setup.md # required: the install interview (see setup contract) + start-workflow.md # optional: the run driver, added when it earns its place + functions/ # optional: per-step helper library + 2-transform/ + index.md # which helper applies to which case (the switch) + from-x.md + from-y.md + runs/ # one folder per run + example/ # ships with the template: a fabricated run (see example run) + 2026-08-07/ # the user's own runs, generated locally, never shared +``` + +The only code-enforced requirement is `index.md` with valid frontmatter. Everything else is convention: step files state what they need from previous steps in prose; nothing checks it mechanically. The process is AI-driven, so requirements live in the text the agent reads, not in validation code. + +## Manifest: frontmatter in index.md + +The manifest is YAML frontmatter at the top of the workflow's `index.md`. There is no separate manifest file, and no version field: installs are snapshots, and a version mechanism comes only when updates exist. + +```yaml +--- +id: coolify-ops # unique, url-safe slug; the argument to `gcontext add` + # and the site path /workflows/ +name: Coolify Ops # human display name, shown in the directory +description: > # one or two sentences; the directory card text + Mirror of a Coolify instance with operational playbooks that + accumulate as incidents are resolved. +parameters: # what a run starts with; bound at setup or per run + - name: instance-url + description: Base URL of the instance to operate + required: true + - name: scope + description: Limit operations to one project + required: false +connections: # service capabilities the workflow needs + - kind: http-api # generic kind, not a product name + description: The hosting panel API (Coolify, Dokploy, or similar) +tags: [ops, infrastructure] # directory filtering +--- +``` + +Field notes: + +- `id` is the identity everywhere: the install argument, the folder name, the site slug. Lowercase letters, digits, hyphens. +- `name` is the display name for the directory and the workflow page; the id stays the machine identity. +- `parameters` are slots, not values. The setup interview or the run start binds them. Never ship bound values. +- `connections` entries are structured (`kind` plus `description`) so the site can render them as requirement badges. They name capability kinds, not products. The body of `index.md` may mention concrete services as examples; the steps must not depend on one (see docs/modules.md on connection-agnostic modules). The agent maps kinds to its own `connections/` at run time. +- `tags` is a flat list for the directory. Keep it short. + +After the frontmatter, the body of `index.md` carries: the objective in the first paragraph, what each parameter means in practice, the workflow's run naming scheme (see runs/), and the general context the agent needs across all steps. Context specific to one step belongs in that step's file. + +## steps/ + +`steps/index.md` is the map: one line per step, in order. Each step is one numbered file (`0-preflight.md`, `1-init.md`, ...). Number from 0 when there is a gate or check before real work starts. + +Each step file states: + +- **Purpose**: what this step achieves and why it exists. +- **Input**: what it needs, and from where (parameters, a previous step's results file, a connection). +- **Output**: what it writes into the run folder, with the schema when the output is tabular (column list for a CSV, field list for JSON). +- **How to execute**: the procedure, in enough detail that an agent without prior context can do it. Include known blockers and how to classify or route them. +- **Done when**: the condition that closes the step. + +Steps that pause for the user (approval, manual action, batching) say so explicitly: what to present, what to wait for. + +## runs/ + +Every execution of the workflow is one folder in `runs/`. + +**The run folder name is workflow-defined.** Each workflow states its own run naming scheme in its `index.md`: whatever identifies one run in that domain. A gym migration names runs by gym id, an invoicing workflow by plant and period, a research pipeline by batch name. The ISO date (`2026-08-07`; second run the same day `2026-08-07-b`) is only the default for workflows with no better key. The run name should carry meaning; the date is the fallback. + +Inside a run folder: + +``` +runs// + index.md # map and status: scope of the run, per-step status table + 0-parameters.csv # the parameters this run started with (.csv, .json, or .md) + 1-init/ + results.csv # the step's output, schema per the step file + script.py # optional: a generated script saved to avoid regenerating it + 2-transform/ + results.json + done/ + info.md # written when the run closes: what was achieved, what was learned +``` + +Conventions: + +- `0-parameters.*` records what the run started with, always, even when trivial. It is what makes a run reproducible and auditable. +- One folder per executed step, named like the step file without the extension. Its main artifact is `results.*`. When the agent generated code worth keeping, it saves the script next to the results. +- The run's `index.md` is the resume point: a session picking up a half-finished run reads it and continues from the first step that is not done. +- `done/` closes the run: `info.md` summarizes what was achieved and anything learned that should change the steps, plus any final deliverable files. A run without `done/` is open. +- Learnings that outlive the run (a new blocker type, a better procedure) get folded back into the step files or `functions/`. That is how the workflow improves with use. + +## functions/ (optional) + +Some steps do the same transformation from ever-varying inputs. `functions/` is a mini library the agent picks from, organized per step: `functions//index.md` describes the cases (the switch), one file per case describes the procedure or code for it. The step file points to its functions folder. Add `functions/` only when a step has proven to need it; most workflows ship without it. + +## What ships vs what is generated + +A workflow is distributed as a template. The template is the procedure plus one fabricated demonstration; the state is born empty on the user's machine. + +Ships in the template: + +- `index.md` with the frontmatter manifest +- `steps/` +- `commands/setup.md` (and any other generic commands) +- `functions/` when the workflow has them +- `runs/example/`: the example run + +Never ships, generated locally at setup and use: + +- the user's own run folders in `runs/` +- every personalized file: configs, credentials references, scripts bound to the user's systems, playbooks learned from the user's own work + +Installs are snapshots. The user's copy is theirs: personalized, growing, never overwritten by an update. `gcontext add` on an existing module warns and stops instead of overwriting. + +## The example run + +Every template ships one fabricated run at `runs/example/`, in the exact `runs/` shape: `index.md`, `0-parameters.*`, one folder per step with plausible results, `done/info.md`. All names and data in it are fake, made up by the author; it must contain nothing personal. + +The example run has two jobs: + +- On the site, it is the centerpiece of the workflow's page: the visitor browses it file by file and sees exactly what each step produces before installing anything. +- In the installed folder, it is the reference: the agent reads it to see what a correct run looks like before executing its first real one. + +The folder name `example` (instead of a run key) is what marks it fabricated. The setup command leaves it in place. + +## The setup command contract + +`commands/setup.md` is the bridge from template to personal instance. It is an interview: the agent asks, the user answers in plain words, the agent builds and confirms. The contract: + +1. **Read first**: the command starts by instructing the agent to read the workflow's `index.md` and `steps/index.md` so the interview is informed. +2. **Bind every parameter slot**: ask for each manifest parameter that is bound at setup time (per-run parameters are only explained, not bound). +3. **Map connections**: for each `connections` entry, find a matching service in the agent's environment or help the user create one. In gcontext that means `connections/`; standalone it means whatever access the user's agent has. +4. **Generate the personal state**: create the files this workflow needs locally (config, scripts against the user's systems, an empty runs/ besides the example). What gets generated is listed in the setup command itself. +5. **Smoke test**: verify the critical path (a read against the user's system, a dry run of the first step) before declaring setup done. +6. **Never rewrite the procedure**: setup personalizes state; it does not edit `steps/`. + +The same file must work on both install paths: + +- **In gcontext**: `commands/setup.md` carries the standard command frontmatter (`description`, optional `parameters`), so the server exposes it as an MCP prompt and the user runs it as a slash command. +- **Standalone**: the user downloads the plain folder, opens any agent in it, and says "run the setup in commands/setup.md". The agent reads the file and executes the same interview. Therefore the body must be self-contained prose that assumes only file access, not gcontext tools. + +## Sharing a workflow + +Authors turn a lived workflow into a template with the share-workflow instructions: docs/share-workflow.md. It strips the personal specifics into parameter slots and connection requirements, generates the setup command, fabricates the example run, and verifies the result against this spec. + +## Relation to modules + +A workflow is a module (see docs/modules.md): installed into `modules/`, connection-agnostic, growing with use. The workflow spec adds the fixed shape on top: manifest frontmatter, `steps/`, `runs/`, the setup contract, the example run. Everything modules.md says about growth and portability applies unchanged.