SnapOtter/RELEASE.md

Releasing SnapOtter

SnapOtter ships a multi-arch container image to Docker Hub (snapotter/snapotter) and GHCR (ghcr.io/snapotter-hq/snapotter). Releases are cut by hand, and the image publish is gated. The build and scan run automatically, but the public latest and version tags only go live after a maintainer approves the run.

How the pipeline is wired

.github/workflows/release.yml runs on workflow_dispatch only. Nothing publishes when a PR merges to main. One dispatch runs these jobs in order:

1. release: semantic-release computes the next version from the Conventional Commits since the last tag, tags it, and creates the GitHub release with notes.
2. prebuilt: builds the source archives (amd64, arm64) and attaches them to the GitHub release.
3. docker: builds both arches and pushes them to Docker Hub + GHCR by digest only. These pushes are unnamed and are not pullable by any tag.
4. scan: Trivy scans the built image and fails the run on unignored CRITICAL/HIGH vulnerabilities.
5. sbom: generates CycloneDX + SPDX SBOMs and attaches them to the GitHub release.
6. manifest: gated. Creates the public latest, vX.Y.Z, X.Y, and X tags on both registries. This is the moment the image becomes pullable by tag.

The manifest job targets the publish-images GitHub Environment, which requires a maintainer to approve the run before it proceeds. If Trivy fails at step 4, manifest never runs and no tags are published.

One-time setup: RELEASE_TOKEN

Step 1 (release) pushes a version-bump commit and tag to main, which is protected. The default GitHub Actions token cannot push past the required status checks, so the workflow authenticates that push with RELEASE_TOKEN: a fine-grained PAT owned by an admin. Because enforce_admins is off on main, an admin identity bypasses the checks.

Create it once, and rotate it when it expires:

1. GitHub → Settings → Developer settings → Fine-grained personal access tokens → Generate new token.
2. Resource owner: snapotter-hq. Repository access: Only select repositories → snapotter-hq/SnapOtter.
3. Repository permissions: Contents read+write, Issues read+write, Pull requests read+write. semantic-release commits and tags (Contents) and comments on released issues/PRs (Issues, Pull requests). Metadata read is added automatically.
4. Expiration: your call, up to a year. Set a reminder to rotate before it lapses.
5. Store it as a repo secret:

   gh secret set RELEASE_TOKEN --repo snapotter-hq/SnapOtter
   

   Paste the token when prompted.

If RELEASE_TOKEN is missing, the workflow falls back to the default token, the push to main is rejected by branch protection, and the release job fails at the semantic-release step. So set the secret before cutting a release. A classic PAT with the repo scope also works and is simpler to configure, but it can reach every repo you have access to, so the fine-grained token is preferred.

Cut a release

0. Confirm the GPU runner is online. The ai-bundles job signs the OCR runtime index only after verify-ocr-nvidia passes, and that job needs the self-hosted box. The gated manifest job lists ai-bundles in its needs, so an offline runner means no public image tags at all. A queued self-hosted job waits up to 24 hours rather than failing, so check before you dispatch:

   gh api repos/snapotter-hq/SnapOtter/actions/runners --jq '
     [.runners[] | select(.status=="online")
      | {name, labels: [.labels[].name | ascii_downcase]}
      | select(.labels | (index("self-hosted") and index("linux")
                          and index("x64") and index("snapotter-nvidia")))
      | .name] as $ok
     | if ($ok|length)>0 then "ready: \($ok|join(", "))"
       else "NO ONLINE snapotter-nvidia RUNNER, do not dispatch" end'
   

   This has to be run by a maintainer, not by the workflow: listing self-hosted runners requires Administration:read, which is not an available GITHUB_TOKEN permission. If it reports no runner, start the service on the GPU box (systemctl --user start github-runner) and re-check. The workflow's own preflight-gpu-runner job claims the same labels before anything expensive runs, so a missed check shows up as a run parked on the first job instead of a late, unexplained stall.

1. Make sure main is green and everything you want in the release is merged.

2. Optional: to override the auto-generated release notes, add a .release-notes.md on main. If present, it replaces the GitHub release body and seeds the docs changelog.

3. Dispatch the workflow:

   gh workflow run release.yml --repo snapotter-hq/SnapOtter
   

   Or from the Actions tab: Release → Run workflow.

4. Watch the run. Jobs release through sbom run without intervention.

Approve the publish

When the run reaches manifest it pauses with a "Review deployments" prompt, and you get a GitHub notification.

Before approving, check:

• release produced the version you expected and the GitHub release looks right.
• scan (Trivy) passed. Review the report attached to the GitHub release if anything is unclear.
• The build checked out the right tag (the docker job uses vX.Y.Z).
• Optional: smoke-test the digest before it becomes latest. The digest is in the docker job log under "Build and push by digest": bash docker run --rm -p 1349:1349 ghcr.io/snapotter-hq/snapotter@sha256:<digest>

Then approve: open the run, click Review deployments, select publish-images, and Approve and deploy. The manifest job runs and the tags go live.

To abort, Reject the deployment instead. The version tag and the GitHub release still exist, so delete them by hand if you want to fully unwind. No image tags get published.

After the tags are live

• Confirm the tags resolve: bash docker pull snapotter/snapotter:latest docker pull ghcr.io/snapotter-hq/snapotter:vX.Y.Z
• Optional: attest provenance by dispatching Attest Provenance (attest.yml) with the image digest.
• The Docker Hub description syncs from DOCKERHUB.md through its own workflow. No action needed unless you changed it.

Notes

• The docker job uploads layers by digest before the gate. They are untagged, invisible on the registry tag lists, and exist so Trivy can scan the real image before you approve. Registry garbage collection reclaims unreferenced digests over time. If you want nothing at all pushed before approval, move the environment: publish-images gate from manifest up to the docker job, at the cost of approving before the scan runs.
• To change who can approve, edit Settings → Environments → publish-images → Required reviewers.
• AI feature bundles are built and published by the release's ai-bundles job, which calls the reusable ai-bundles.yml to build, verify, sign, and upload them to HuggingFace. The gated manifest job lists ai-bundles in its needs, so a version's public image tags go live only once that version's bundles are published. Publishing is idempotent: if the bundle release already exists it is verified, not re-uploaded.