From 3ddff71205efd26d5bf6dbe6c2e6c9107a71dceb Mon Sep 17 00:00:00 2001 From: lefarcen <935902669@qq.com> Date: Tue, 8 Sep 2026 11:14:16 +0000 Subject: [PATCH] feat(ci): publish the What's New card from the repository (#7892) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(ci): publish the What's New card from the repository Move the post-update highlight document into the repo and publish it from GitHub Actions, so changing the card is a pull request instead of a task that needs personal Cloudflare credentials. - docs/whats-new.json is the source of truth, next to its spec doc. - pnpm guard validates it through the daemon's own parser on every PR, so a document that would silently show no card cannot merge. - whats-new-publish.yml uploads on merge to main (and on manual dispatch, with a read-only dry-run mode) and verifies the object by reading it back from the origin the daemon fetches, past the edge cache. * docs: name the CI job that actually runs the guard check * fix(ci): gate the What's New publish behind a main-only environment A workflow_dispatch runs the workflow file from the ref it is dispatched against, so the in-workflow main-only assertion was editable by whoever triggered it: anyone with write access could publish unreviewed content to every installed client from their own branch. Split the workflow so credentials live in one place and that place is gated by GitHub rather than by YAML. `validate` holds no secrets and runs on any ref; `publish` declares the `whats-new-publish` environment, whose deployment-branch policy allows main only and which owns the R2 secrets. Dropping the environment declaration to escape the branch policy also drops the secrets. Also points the card at the new Arena URLs supplied by product. * fix(ci): pin the What's New publish trust boundary in a check that runs The missing-credentials error still told operators the R2 keys are repository secrets. Following that string is the exact setup the previous revision closed: repository secrets are readable from any job on any branch, so a modified workflow dispatched from an unreviewed ref could publish to every installed client — or just take the keys. Point the error at the `whats-new-publish` environment and refuse the repository path in the same breath. The boundary itself had no coverage: nothing read the workflow, so moving `environment:` or a `secrets.*` expression back onto the arbitrary-ref job would stay green while still publishing correctly from main. That coverage cannot be lane-routed. Running the planner over `.github/workflows/whats-new-publish.yml` selects neither `web_tests_required` nor `ui_p0_validation_required`, so the `e2e_vitest` workload does not run for a workflow-only edit — an assertion living only there would be skipped for exactly the change class it guards. The check therefore lives in `pnpm guard`, which CI invokes from the unconditional preflight job, same reasoning as the document guard next to it. It asserts that the credential-bearing job declares the environment by its exact name, needs `validate` and is gated on its output with no `always()`/`cancelled()` escape, reads all four bucket bindings, and that no other job attaches an environment or reads any secret at all. The e2e suite drives it over the real workflow mutated into each of those regressions in turn, so every assertion is proven able to fail; gutting the checker turns all eleven red. * chore(whats-new): seed the document from the live card Bringing the workflow in with new copy attached ties the infrastructure to a content decision. The Arena CTA this document pointed at is not live — `/llm-arena-for-design/` and `/zh/llm-arena/llm-arena-for-design/` both 404 — and the card renders regardless, so merging would have put a dead button on every client's Home. Seed the repository copy from the object that is already serving instead: byte-identical to `https://whatsnew.open-design.ai/whats-new.json`, id `0.21.0`. That puts the current state under version control and makes the first automatic publish idempotent — it uploads exactly what is already there, so the merge exercises the whole chain (validate, upload, read-back against the origin the daemon fetches) without changing anything a user sees. The Arena copy is not lost; it moves to the PR body as a follow-up for the release owner, who can now change the card through this workflow without holding personal Cloudflare credentials. * fix(ci): detect secrets by context, not by access shape The guard added last round matched `${{ secrets.NAME }}` with a regex over the raw workflow text. A guard whose whole job is to stop a secret reaching the arbitrary-ref job had a blind spot for most of the ways to read one. Verified against the previous implementation, each returning zero violations while a secret reached `validate`: `secrets['NAME']`, `secrets["NAME"]`, `secrets [ 'NAME' ]`, `secrets . NAME`, `toJSON(secrets)`, a bare `secrets`, and `fromJSON(toJSON(secrets)).NAME`. Extending the regex would have repeated the mistake that caused it. One bypass proves why: a double-quoted YAML scalar may carry a backslash line continuation, so STOLEN: "${{ sec\ rets.NAME }}" parses to a live secret read while the source text contains no such substring. No text scanner can see that, however many shapes it lists. So the workflow is read through a YAML parser, and detection is a single property instead of an enumeration: GitHub's expression language has no string-to-context evaluation, so a secret cannot be reached without naming the `secrets` context as an identifier. Finding that identifier inside any `${{ … }}` expression covers every access form, including ones nobody has thought of. Parsing also retires the hand-rolled job splitter and its own failure modes, and lets the check distinguish "no violations" from "nothing was examined" — an unparseable document, a flow-style jobs mapping, or a renamed publish job now fail closed rather than passing quietly. Two holes the shape list never covered are closed as a side effect: a job-level `secrets:` key passes credentials to a called workflow with no expression at all, and workflow-level `env:` is inherited by every job. `yaml@2.9.0` becomes a root devDependency. It was already in the lockfile via `tools/release` and pinned repository-wide by the existing pnpm override, so the lockfile gains only the root importer entry. Test coverage moves with it: 26 of 46 cases now go red when the checker is gutted, up from 11. --------- Co-authored-by: lefarcen --- .github/scripts/publish_whats_new.py | 322 +++++++++++++++ .github/workflows/whats-new-publish.yml | 167 ++++++++ docs/whats-new.json | 13 + docs/whats-new.md | 101 ++++- e2e/tests/scripts/whats-new-publish.test.ts | 398 +++++++++++++++++++ package.json | 3 +- pnpm-lock.yaml | 3 + scripts/check-whats-new-document.ts | 193 +++++++++ scripts/check-whats-new-publish-workflow.ts | 413 ++++++++++++++++++++ scripts/guard.ts | 4 + tools/pack/AGENTS.md | 2 +- 11 files changed, 1599 insertions(+), 20 deletions(-) create mode 100644 .github/scripts/publish_whats_new.py create mode 100644 .github/workflows/whats-new-publish.yml create mode 100644 docs/whats-new.json create mode 100644 e2e/tests/scripts/whats-new-publish.test.ts create mode 100644 scripts/check-whats-new-document.ts create mode 100644 scripts/check-whats-new-publish-workflow.ts diff --git a/.github/scripts/publish_whats_new.py b/.github/scripts/publish_whats_new.py new file mode 100644 index 0000000000..70a157fd27 --- /dev/null +++ b/.github/scripts/publish_whats_new.py @@ -0,0 +1,322 @@ +#!/usr/bin/env python3 +"""Publish the repository's post-update "What's New" document to R2. + +The daemon reads one hosted JSON document and resolves anything malformed to +"no highlight" instead of an error (apps/daemon/src/services/whats-new.ts). +A successful upload therefore proves nothing on its own: a broken document +uploads just as cleanly as a good one and simply removes the card. This +publisher is built around that asymmetry: + + * the bytes are parsed before they are sent, so a broken document cannot + reach the bucket at all; + * the object is read back from the public origin the daemon actually + fetches, with the edge cache bypassed, and the round-tripped bytes must + equal what was sent. + +Structural validation of the document against the shipping parser lives in +`scripts/check-whats-new-document.ts` (part of `pnpm guard`); the workflow +runs it before this script so there is exactly one implementation of the +content contract. + +Environment: + WHATS_NEW_DOCUMENT path to the source document + WHATS_NEW_OBJECT_KEY object key in the bucket + WHATS_NEW_PUBLIC_URL public URL the daemon reads + WHATS_NEW_STORAGE_ENDPOINT R2 S3 endpoint origin + WHATS_NEW_STORAGE_BUCKET bucket name + WHATS_NEW_STORAGE_ACCESS_KEY_ID / WHATS_NEW_STORAGE_SECRET_ACCESS_KEY + WHATS_NEW_DRY_RUN "true" to validate and diff without uploading +""" + +from __future__ import annotations + +import json +import os +import sys +import tempfile +import urllib.error +import urllib.request +import uuid +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parent)) + +from lib.github import append_summary # noqa: E402 +from lib.r2 import R2Client, R2Credentials, R2Error # noqa: E402 + +# Matches the `Cache-Control` the object is already served with. Short enough +# that an edit reaches users promptly, long enough that Home activations do +# not hammer the origin. The daemon adds its own ~10 minute process cache. +CACHE_CONTROL = "public, max-age=300" +CONTENT_TYPE = "application/json" +READ_BACK_TIMEOUT = 20.0 +# The public origin sits behind Cloudflare, which answers the stdlib default +# `Python-urllib/x.y` User-Agent with 403. Without an explicit one, read-back +# verification fails on every run and the publisher can never confirm anything. +READ_HEADERS = { + "accept": "application/json", + "user-agent": "open-design-whats-new-publisher/1 (+https://github.com/nexu-io/open-design)", +} + +REQUIRED_STORAGE_VARS = ( + "WHATS_NEW_STORAGE_ENDPOINT", + "WHATS_NEW_STORAGE_BUCKET", + "WHATS_NEW_STORAGE_ACCESS_KEY_ID", + "WHATS_NEW_STORAGE_SECRET_ACCESS_KEY", +) + +# The GitHub environment that holds the R2 credentials, named here because the +# missing-credentials error is the one place an operator reads when the publish +# fails — and following it to the wrong place reopens the trust boundary. +# Repository secrets are readable from any job on any branch, so storing them +# there would let a modified workflow dispatched from an unreviewed ref publish +# to every installed client. The environment's deployment-branch policy allows +# `main` only; see docs/whats-new.md. +PUBLISH_ENVIRONMENT = "whats-new-publish" + + +class PublishError(RuntimeError): + pass + + +def _env(name: str, default: str | None = None) -> str: + value = os.environ.get(name, default if default is not None else "").strip() + if not value: + raise PublishError(f"{name} is required") + return value + + +def _highlight_id(payload: object) -> str | None: + if not isinstance(payload, dict): + return None + value = payload.get("id") + return value.strip() if isinstance(value, str) and value.strip() else None + + +def read_document(path: Path) -> tuple[bytes, object]: + """Read the source document, refusing anything that is not valid JSON.""" + try: + body = path.read_bytes() + except OSError as error: + raise PublishError(f"cannot read {path}: {error}") from error + if not body.strip(): + raise PublishError(f"{path} is empty; publish `{{}}` to retire the card instead") + try: + payload = json.loads(body) + except json.JSONDecodeError as error: + raise PublishError(f"{path} is not valid JSON: {error}") from error + if not isinstance(payload, dict): + raise PublishError(f"{path} must contain a JSON object") + return body, payload + + +def fetch_live(url: str) -> tuple[bytes | None, object | None]: + """Fetch the currently published document. Absence is not an error.""" + try: + request = urllib.request.Request(url, headers=READ_HEADERS) + with urllib.request.urlopen(request, timeout=READ_BACK_TIMEOUT) as response: + body = response.read() + except (urllib.error.URLError, TimeoutError) as error: + print(f"note: could not read the live document ({error}); continuing") + return None, None + try: + return body, json.loads(body) + except json.JSONDecodeError: + print("note: the live document is not valid JSON; continuing") + return body, None + + +def read_back(url: str, expected: bytes) -> None: + """Verify the object from the origin the daemon reads, past the edge cache. + + The object is served with a 300s `Cache-Control`, so a plain GET right + after the upload can legitimately return the previous document. A unique + query string produces a distinct cache key, which forces the edge to go + to the bucket and makes this a real check on what was stored rather than + a check on what happens to be cached. + """ + separator = "&" if "?" in url else "?" + probe = f"{url}{separator}publish-verify={uuid.uuid4().hex}" + try: + request = urllib.request.Request(probe, headers=READ_HEADERS) + with urllib.request.urlopen(request, timeout=READ_BACK_TIMEOUT) as response: + status = response.status + body = response.read() + content_type = response.headers.get("content-type", "") + cache_control = response.headers.get("cache-control", "") + except (urllib.error.URLError, TimeoutError) as error: + raise PublishError(f"read-back of {url} failed: {error}") from error + + if status != 200: + raise PublishError(f"read-back of {url} returned HTTP {status}") + if body != expected: + raise PublishError( + f"read-back of {url} returned {len(body)} bytes that differ from the {len(expected)} bytes uploaded" + ) + if "json" not in content_type.lower(): + raise PublishError(f"read-back of {url} returned content-type {content_type!r}, expected JSON") + print(f"read-back verified: {url} serves the uploaded bytes (cache-control: {cache_control or 'unset'})") + + +def main() -> int: + document_path = Path(_env("WHATS_NEW_DOCUMENT", "docs/whats-new.json")) + object_key = _env("WHATS_NEW_OBJECT_KEY", "whats-new.json") + public_url = _env("WHATS_NEW_PUBLIC_URL", "https://whatsnew.open-design.ai/whats-new.json") + dry_run = os.environ.get("WHATS_NEW_DRY_RUN", "").strip().lower() == "true" + + body, payload = read_document(document_path) + new_id = _highlight_id(payload) + print(f"source: {document_path} ({len(body)} bytes), highlight id: {new_id or '(none — card retired)'}") + + _, live_payload = fetch_live(public_url) + live_id = _highlight_id(live_payload) + print(f"live: {public_url}, highlight id: {live_id or '(none)'}") + + notes: list[str] = [] + if new_id is not None and new_id == live_id: + # The client stores the last id it showed and only re-opens the card + # when the id changes, so republishing under the same id updates the + # document for new profiles but shows nothing to anyone who already + # dismissed it. That is a legitimate copy-fix flow, not an error. + notes.append( + f"`id` is unchanged (`{new_id}`), so users who already saw this card will not see it again. " + "Change `id` when the card should re-appear." + ) + if new_id is None: + notes.append("This document resolves to no highlight, which takes the card down.") + + for note in notes: + print(f"note: {note}") + + if dry_run: + print("dry run: skipping upload") + append_summary( + "\n".join( + [ + "### What's New — dry run", + "", + f"- source: `{document_path}` → `{object_key}`", + f"- would publish id: `{new_id or '(none)'}` (live: `{live_id or '(none)'}`)", + *[f"- note: {note}" for note in notes], + ] + ) + ) + return 0 + + missing = [name for name in REQUIRED_STORAGE_VARS if not os.environ.get(name, "").strip()] + if missing: + raise PublishError( + "missing R2 credentials: " + + ", ".join(missing) + + f". These must be environment secrets on the `{PUBLISH_ENVIRONMENT}` GitHub" + " environment, which is restricted to `main`. Do NOT add them as repository" + " secrets: those are readable from any job on any branch and would let an" + " unreviewed ref publish to every installed client. See docs/whats-new.md." + ) + + client = R2Client( + endpoint=_env("WHATS_NEW_STORAGE_ENDPOINT"), + bucket=_env("WHATS_NEW_STORAGE_BUCKET"), + credentials=R2Credentials( + _env("WHATS_NEW_STORAGE_ACCESS_KEY_ID"), + _env("WHATS_NEW_STORAGE_SECRET_ACCESS_KEY"), + ), + ) + # Unlike release artifacts, this object is deliberately mutable: the whole + # point of the file is that operators overwrite it. + client.put_bytes( + key=object_key, + body=body, + content_type=CONTENT_TYPE, + cache_control=CACHE_CONTROL, + if_none_match=False, + ) + print(f"uploaded {object_key} ({len(body)} bytes)") + + read_back(public_url, body) + + append_summary( + "\n".join( + [ + "### What's New published", + "", + f"- source: `{document_path}` → `{object_key}`", + f"- highlight id: `{new_id or '(none)'}` (was `{live_id or '(none)'}`)", + f"- verified at {public_url}", + *[f"- note: {note}" for note in notes], + "", + "Edge cache and the daemon's own cache mean users see this within ~15 minutes.", + ] + ) + ) + return 0 + + +def self_check() -> None: + """Prove the read-back gate can fail before trusting it to pass. + + Read-back is the only thing standing between "the upload command exited + zero" and "the card users fetch is the one we meant to publish", so a + green read-back is worthless unless a wrong body actually turns it red. + """ + import http.server + import threading + + served: dict[str, bytes] = {"body": b'{"id":"served"}'} + requested: list[str] = [] + + class Handler(http.server.BaseHTTPRequestHandler): + def do_GET(self) -> None: # noqa: N802 - stdlib naming + requested.append(self.path) + body = served["body"] + self.send_response(200) + self.send_header("content-type", "application/json") + self.send_header("cache-control", CACHE_CONTROL) + self.send_header("content-length", str(len(body))) + self.end_headers() + self.wfile.write(body) + + def log_message(self, *_args: object) -> None: + return + + server = http.server.HTTPServer(("127.0.0.1", 0), Handler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + url = f"http://127.0.0.1:{server.server_address[1]}/whats-new.json" + try: + read_back(url, served["body"]) + if len(requested) != 1 or "publish-verify=" not in requested[0]: + raise PublishError("read-back must bypass the edge cache with a unique query string") + + served["body"] = b'{"id":"stale"}' + try: + read_back(url, b'{"id":"fresh"}') + except PublishError: + pass + else: + raise PublishError("read-back accepted a body that differs from what was uploaded") + + with tempfile.TemporaryDirectory() as scratch: + path = Path(scratch) / "whats-new.json" + for bad in (b"", b"{ not json }", b"[]"): + path.write_bytes(bad) + try: + read_document(path) + except PublishError: + continue + raise PublishError(f"read_document accepted an invalid document: {bad!r}") + finally: + server.shutdown() + server.server_close() + print("publish_whats_new self-check passed") + + +if __name__ == "__main__": + try: + if len(sys.argv) > 1 and sys.argv[1] == "self-check": + self_check() + raise SystemExit(0) + raise SystemExit(main()) + except (PublishError, R2Error) as error: + print(f"error: {error}", file=sys.stderr) + raise SystemExit(1) from error diff --git a/.github/workflows/whats-new-publish.yml b/.github/workflows/whats-new-publish.yml new file mode 100644 index 0000000000..0c66d3772d --- /dev/null +++ b/.github/workflows/whats-new-publish.yml @@ -0,0 +1,167 @@ +name: whats-new-publish + +# Publish the post-update "What's New" card from the repository copy of the +# document (docs/whats-new.json) to the R2 object the daemon reads. Editing +# the card is a copy change, so it goes through the normal PR review path and +# publishes on merge; nobody needs local Cloudflare credentials. +# +# Standalone (not part of ci.yml): this workflow performs a trusted write and +# must stay outside the low-privilege PR gate. +# +# TRUST BOUNDARY — the card is visible to every installed client the moment it +# lands, so "published" must imply "reviewed". `workflow_dispatch` can be run +# from any ref, and a dispatch runs the workflow file *from that ref*, so any +# check written in this file is editable by the person triggering it. Nothing +# here can be the control. +# +# The control is the `whats-new-publish` environment: its deployment-branch +# policy allows `main` only, and the R2 credentials are environment secrets +# scoped to it. A job that declares the environment cannot start on another +# ref; a job that drops the declaration to get around that no longer receives +# the secrets. The publish job is the only job that declares it, and it is the +# only job that touches credentials. The `main`-only assertion below is +# defense in depth and a clear error message — not the boundary. +# +# Reachability — `validate` has no `needs` and no `if`, so it always runs. +# `publish` is skipped when `validate` fails or is cancelled, which is the +# intended fail-closed behavior: `always() && !cancelled()` is deliberately +# NOT used here, because "publish anyway" is exactly what must not happen. +# Within `validate`, a condition that would otherwise skip silently is a step +# that exits 1 instead — a publish that quietly does nothing looks green and +# leaves the old card up. + +on: + push: + branches: [main] + paths: + - "docs/whats-new.json" + - ".github/scripts/publish_whats_new.py" + - ".github/scripts/lib/r2.py" + - ".github/workflows/whats-new-publish.yml" + workflow_dispatch: + inputs: + dry_run: + description: Validate and diff against the live document without uploading + type: boolean + default: false + +permissions: + contents: read + +concurrency: + # Never publish two documents at once; the object is last-write-wins. + group: whats-new-publish + cancel-in-progress: false + +env: + WHATS_NEW_DOCUMENT: docs/whats-new.json + WHATS_NEW_OBJECT_KEY: whats-new.json + # The exact URL the daemon fetches (DEFAULT_WHATS_NEW_URL in + # apps/daemon/src/services/whats-new.ts). Read-back verification hits this, + # not a configurable mirror, so a green run means the card users will see. + WHATS_NEW_PUBLIC_URL: https://whatsnew.open-design.ai/whats-new.json + +jobs: + # No environment, no secrets. Safe to run from any ref, which is what makes + # `dry_run` useful for previewing a copy change before it merges. + validate: + name: validate document + runs-on: ubuntu-24.04 + timeout-minutes: 10 + outputs: + publish: ${{ steps.mode.outputs.publish }} + steps: + - name: Checkout + uses: actions/checkout@v6.0.2 + with: + # Pin both jobs to one commit so the document that was validated is + # the document that gets published, even if the ref moves in between. + ref: ${{ github.sha }} + + - name: Resolve run mode + id: mode + env: + REPOSITORY: ${{ github.repository }} + EVENT_NAME: ${{ github.event_name }} + REF: ${{ github.ref }} + DRY_RUN: ${{ inputs.dry_run }} + run: | + set -euo pipefail + if [ "$REPOSITORY" != "nexu-io/open-design" ]; then + echo "This workflow publishes the nexu-io/open-design card only; got $REPOSITORY" >&2 + exit 1 + fi + publish=true + if [ "$EVENT_NAME" = "workflow_dispatch" ] && [ "$DRY_RUN" = "true" ]; then + publish=false + fi + # Fail rather than silently downgrading to a dry run, so a mistaken + # branch dispatch is a red run with an explanation instead of a + # green one that published nothing. + if [ "$publish" = "true" ] && [ "$REF" != "refs/heads/main" ]; then + echo "Publishing requires refs/heads/main; got $REF." >&2 + echo "Re-run with dry_run to preview a branch, or dispatch against main." >&2 + exit 1 + fi + echo "publish=$publish" >> "$GITHUB_OUTPUT" + echo "Mode: $([ "$publish" = "true" ] && echo 'publish' || echo 'dry run') from $REF @ ${{ github.sha }}" + + - name: Setup Node.js + uses: actions/setup-node@v6 + with: + node-version: 24 + + # The content contract has exactly one implementation: the guard check + # that `pnpm guard` runs on every PR, which drives the document through + # the daemon's shipping parser. Re-running it here covers the cases the + # PR gate cannot: a manual dispatch on an arbitrary ref, and any direct + # write to main. + - name: Validate document against the shipping parser + run: npx --yes tsx@4.22.3 scripts/check-whats-new-document.ts + + - name: Report live vs proposed + env: + WHATS_NEW_DRY_RUN: "true" + run: python3 .github/scripts/publish_whats_new.py + + publish: + name: publish to R2 + needs: validate + if: needs.validate.outputs.publish == 'true' + runs-on: ubuntu-24.04 + timeout-minutes: 10 + # The trust boundary. Deployment-branch policy: `main` only. The R2 + # credentials are secrets ON THIS ENVIRONMENT, not repository secrets — + # repository secrets would be readable from any job on any branch and + # would defeat the whole control. + environment: whats-new-publish + steps: + - name: Checkout + uses: actions/checkout@v6.0.2 + with: + ref: ${{ github.sha }} + + - name: Assert reviewed ref + env: + REF: ${{ github.ref }} + run: | + set -euo pipefail + # Defense in depth. The environment's branch policy is what actually + # stops a non-main run; this step makes the intent legible and gives + # a readable failure if the policy is ever loosened. + if [ "$REF" != "refs/heads/main" ]; then + echo "Refusing to publish from $REF; only reviewed content on main may reach users." >&2 + exit 1 + fi + + - name: Publish + env: + # Environment secrets on `whats-new-publish`, following the naming + # used by the releases / repository-assets / workload-results + # buckets. Absent or repo-level-only => the publisher fails loudly + # with the exact names. + WHATS_NEW_STORAGE_ACCESS_KEY_ID: ${{ secrets.CLOUDFLARE_R2_WHATS_NEW_AK }} + WHATS_NEW_STORAGE_SECRET_ACCESS_KEY: ${{ secrets.CLOUDFLARE_R2_WHATS_NEW_SK }} + WHATS_NEW_STORAGE_ENDPOINT: ${{ secrets.CLOUDFLARE_R2_WHATS_NEW_URL }} + WHATS_NEW_STORAGE_BUCKET: ${{ secrets.CLOUDFLARE_R2_WHATS_NEW_BUCKET }} + run: python3 .github/scripts/publish_whats_new.py diff --git a/docs/whats-new.json b/docs/whats-new.json new file mode 100644 index 0000000000..d35d530943 --- /dev/null +++ b/docs/whats-new.json @@ -0,0 +1,13 @@ +{ + "id": "0.21.0", + "title": "Reliable, Start to Finish", + "body": "HTML previews stay visible as you switch projects, files, or between Code and Preview.\nAgent connections are more reliable, with improved support for current DeepSeek Harness and Kimi releases.\nRelaunching Open Design now recovers more reliably after an unexpected shutdown.", + "imageUrl": "https://whatsnew.open-design.ai/0.18.0.webp", + "linkUrl": "https://github.com/nexu-io/open-design/releases/tag/open-design-v0.21.0", + "locales": { + "zh-CN": { + "title": "从启动到预览，始终可靠", + "body": "切换项目、文件，或在「代码」与「预览」之间切换时，HTML 预览保持正常显示。\n提升 Agent 连接稳定性，并完善对新版 DeepSeek Harness 与 Kimi 的兼容。\n意外退出后重新启动时，Open Design 能够更可靠地恢复运行。" + } + } +} diff --git a/docs/whats-new.md b/docs/whats-new.md index e3bc36fa59..7a6005ec81 100644 --- a/docs/whats-new.md +++ b/docs/whats-new.md @@ -7,15 +7,24 @@ if the source is unreachable or empty, no card shows and Home is unaffected. ## Where the content lives -The card content is a single hand-curated JSON document on a dedicated R2 -bucket: +The card content is a single hand-curated JSON document, kept in this +repository at: + +``` +docs/whats-new.json +``` + +`.github/workflows/whats-new-publish.yml` publishes that file to the object the +daemon reads: ``` https://whatsnew.open-design.ai/whats-new.json ``` -There is **no per-release publish tooling** and the content is **not** carried -in release `metadata.json`. To change what users see, edit that one file. +Changing the card is therefore an ordinary pull request — no local Cloudflare +credentials, no wrangler, no per-person bottleneck. The content is **not** +carried in release `metadata.json`, and there is no per-release publish +tooling: one file, edited when the copy should change. - The daemon proxies it at `GET /api/whats-new` (also `od whats-new [--json]`), so the web UI and CLI read the exact same payload. @@ -40,8 +49,11 @@ card when the current `id` differs. So: document is deliberately curated, so surfacing the current highlight to a new user once is intended. -To retire the card entirely, publish an empty object (`{}`) or a document -without a valid `id`/`title`/`body`; the daemon then resolves to "no highlight". +To retire the card entirely, publish an empty object (`{}`); the daemon then +resolves to "no highlight". Any *other* incomplete document also resolves to +"no highlight", but that is the accident case, not the intended one — the guard +accepts only a complete highlight or the empty document, so taking the card +down is an explicit act rather than something a typo can do for you. ## Document schema @@ -62,8 +74,9 @@ without a valid `id`/`title`/`body`; the daemon then resolves to "no highlight". } ``` -Field rules (anything missing or malformed makes the card silently not show, so -validate before uploading): +Field rules — anything missing or malformed makes the card silently not show, +which is why `pnpm guard` checks the repository document against the shipping +parser rather than trusting review: - `id` — **required**, non-empty string. The show-once key. - `title`, `body` — **required**, non-empty strings. @@ -74,16 +87,68 @@ validate before uploading): `zh-CN`, …); each may override `title`/`body`/`linkUrl`. An exact locale wins, then the bare language (`zh` for `zh-TW`), then the base fields. -## Updating the file (S3 API) +## Updating the card -The bucket is S3-compatible. With an R2 token scoped to the bucket: +1. Edit `docs/whats-new.json` and open a pull request. +2. `pnpm guard` validates the document on every PR + (`scripts/check-whats-new-document.ts`). It runs the document through the + daemon's own parser and fails if the card would not show, if an optional + field would be silently dropped, or if a field name is misspelled. This + matters because the runtime is fail-safe: a malformed document does not + error, it just makes the card disappear. +3. On merge to `main`, `whats-new-publish.yml` uploads the file + (`application/json`, `cache-control: public, max-age=300`) and then reads it + back from `https://whatsnew.open-design.ai/whats-new.json` with the edge + cache bypassed. The job fails unless the bytes served match the bytes + uploaded — an exit code from the upload alone is not treated as proof. -```bash -AWS_ACCESS_KEY_ID=… AWS_SECRET_ACCESS_KEY=… \ -aws s3 cp whats-new.json s3:///whats-new.json \ - --endpoint-url https://.r2.cloudflarestorage.com \ - --content-type application/json --cache-control 'public, max-age=300' -``` +Republishing the current `main` content without a code change: run the +**whats-new-publish** workflow manually (`workflow_dispatch`) against `main`. +Its `dry_run` input validates the document and reports the live-vs-proposed +`id` without uploading; a dry run needs no credentials and works from any +branch, so it is a safe way to preview a copy change before it merges. A real +publish only runs on `main` — see the trust boundary below. -Keep `Cache-Control` modest so an edit reaches users promptly; the daemon also -caches the document for ~10 minutes. +Propagation takes up to ~15 minutes: the object's own `max-age=300` plus the +daemon's ~10 minute in-process cache. + +### Credentials and the trust boundary + +The card is visible to every installed client as soon as it lands, so +"published" has to imply "reviewed". The control that guarantees it is the +**`whats-new-publish` GitHub environment**: + +- its deployment-branch policy allows **`main` only**, so a job that declares + the environment cannot start on any other ref; +- the R2 credentials are **environment secrets on that environment** — + `CLOUDFLARE_R2_WHATS_NEW_AK`, `CLOUDFLARE_R2_WHATS_NEW_SK`, + `CLOUDFLARE_R2_WHATS_NEW_URL`, `CLOUDFLARE_R2_WHATS_NEW_BUCKET`. + +Both halves are load-bearing. `workflow_dispatch` runs the workflow file from +the ref it is dispatched against, so every check written inside the workflow is +editable by whoever triggers it — including the `main`-only assertion. What is +not editable is where the secrets live: dropping the `environment:` declaration +to escape the branch policy also drops access to the secrets, and the publisher +then fails naming the variables it is missing. + +**Do not add these as repository secrets.** Repository secrets are readable +from any job on any branch, which would let anyone with write access publish +unreviewed content by dispatching a modified workflow from their own branch. + +That shape is pinned by the `what's new publish workflow` check in `pnpm guard` +(`scripts/check-whats-new-publish-workflow.ts`): the credential-bearing job is +the one that declares the environment, it cannot start without a green +`validate`, and no other job may hold an environment or read a secret. The +check runs from CI's unconditional preflight job rather than a path-routed +test lane, because an edit to the workflow alone selects no test workload — +a lane-routed assertion would be skipped for exactly the edit it guards. + +It reads the workflow through a YAML parser rather than matching text, and +looks for the `secrets` **context identifier** inside any `${{ … }}` +expression rather than for particular access shapes. Both choices are +load-bearing: `secrets['NAME']`, `toJSON(secrets)` and a bare `secrets` are all +secret reads, and a double-quoted YAML scalar can hide `secrets` from raw text +entirely by splitting it across a backslash line continuation. + +The repository-wide `CLOUDFLARE_API_TOKEN` is a Pages-scoped token and cannot +reach R2 at all; do not route this publish through it. diff --git a/e2e/tests/scripts/whats-new-publish.test.ts b/e2e/tests/scripts/whats-new-publish.test.ts new file mode 100644 index 0000000000..fbb5862398 --- /dev/null +++ b/e2e/tests/scripts/whats-new-publish.test.ts @@ -0,0 +1,398 @@ +import { execFileSync, spawnSync } from "node:child_process"; +import { mkdtemp, mkdir, readFile, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { afterEach, beforeEach, describe, expect, test, vi } from "vitest"; + +import { + WHATS_NEW_DOCUMENT_PATH, + checkWhatsNewDocument, +} from "../../../scripts/check-whats-new-document.ts"; +import { + WHATS_NEW_WORKFLOW_PATH, + findWhatsNewWorkflowViolations, + parseWorkflowJobs, + workflowJobNames, +} from "../../../scripts/check-whats-new-publish-workflow.ts"; + +const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../../.."); + +// The daemon resolves a malformed What's New document to "no highlight" +// instead of an error, so every failure mode below is invisible at publish +// time: the upload succeeds and the card simply never appears. These cases +// exist to prove the guard can actually see each of them — a guard that only +// ever runs against the good document is an untested guard. +async function documentRoot(document: string): Promise { + const root = await mkdtemp(path.join(tmpdir(), "whats-new-guard-")); + await mkdir(path.join(root, path.dirname(WHATS_NEW_DOCUMENT_PATH)), { recursive: true }); + await writeFile(path.join(root, WHATS_NEW_DOCUMENT_PATH), document, "utf8"); + return root; +} + +const validDocument = { + id: "1.2.3", + title: "Headline", + body: "First bullet.\nSecond bullet.", + imageUrl: "https://whatsnew.open-design.ai/cover.webp", + linkUrl: "https://open-design.ai/release/", + locales: { + "zh-CN": { title: "标题", body: "第一条。\n第二条。", linkUrl: "https://open-design.ai/zh/release/" }, + }, +}; + +async function check(document: unknown): Promise { + return checkWhatsNewDocument(await documentRoot(JSON.stringify(document, null, 2))); +} + +describe("what's new document guard", () => { + let consoleError: ReturnType; + let consoleLog: ReturnType; + + beforeEach(() => { + consoleError = vi.spyOn(console, "error").mockImplementation(() => {}); + consoleLog = vi.spyOn(console, "log").mockImplementation(() => {}); + }); + + afterEach(() => { + consoleError.mockRestore(); + consoleLog.mockRestore(); + }); + + test("the repository document is publishable", async () => { + await expect(checkWhatsNewDocument(repoRoot)).resolves.toBe(true); + }); + + test("a well-formed document passes", async () => { + await expect(check(validDocument)).resolves.toBe(true); + }); + + test("the empty retirement document passes", async () => { + await expect(check({})).resolves.toBe(true); + }); + + test("invalid JSON fails", async () => { + await expect(checkWhatsNewDocument(await documentRoot("{ not json }"))).resolves.toBe(false); + }); + + test("a missing document fails", async () => { + await expect(checkWhatsNewDocument(await mkdtemp(path.join(tmpdir(), "whats-new-guard-")))).resolves.toBe(false); + }); + + test.each(["id", "title", "body"] as const)("a missing %s fails", async (field) => { + const { [field]: _dropped, ...rest } = validDocument; + await expect(check(rest)).resolves.toBe(false); + }); + + test("an empty required string fails", async () => { + await expect(check({ ...validDocument, title: " " })).resolves.toBe(false); + }); + + test("a misspelled field fails instead of being silently ignored", async () => { + const { imageUrl: cover, ...rest } = validDocument; + await expect(check({ ...rest, imageURL: cover })).resolves.toBe(false); + }); + + test("a non-https imageUrl fails", async () => { + await expect(check({ ...validDocument, imageUrl: "http://whatsnew.open-design.ai/cover.webp" })).resolves.toBe( + false, + ); + }); + + test("a non-https locale linkUrl fails", async () => { + await expect( + check({ + ...validDocument, + locales: { "zh-CN": { ...validDocument.locales["zh-CN"], linkUrl: "open-design.ai/zh/" } }, + }), + ).resolves.toBe(false); + }); + + test("a locale whose overrides are all invalid fails", async () => { + await expect(check({ ...validDocument, locales: { "zh-CN": { title: "" } } })).resolves.toBe(false); + }); + + test("a blank bullet line fails", async () => { + await expect(check({ ...validDocument, body: "First bullet.\n\nSecond bullet." })).resolves.toBe(false); + }); + + test("the check is registered in the public guard entrypoint", () => { + const names = execFileSync("pnpm", ["--silent", "guard", "--list-checks"], { + cwd: repoRoot, + encoding: "utf8", + }) + .trim() + .split("\n"); + expect(names).toContain("what's new document"); + }); +}); + +describe("what's new publisher", () => { + // The publisher's own gate: a green upload means nothing unless the object + // is read back from the origin the daemon fetches, and that read-back is + // only evidence if a wrong body turns it red. The self-check asserts both + // halves against a local origin. + test("the read-back gate and the JSON gate can both fail", () => { + const result = spawnSync("python3", [path.join(repoRoot, ".github/scripts/publish_whats_new.py"), "self-check"], { + cwd: repoRoot, + encoding: "utf8", + }); + expect(result.stderr ?? "").toBe(""); + expect(result.status).toBe(0); + expect(result.stdout).toContain("self-check passed"); + }); + + // A malformed document must be rejected before anything else happens, so a + // bad edit cannot reach the bucket even if the credentials are present. + test("an invalid document is rejected before credentials are considered", async () => { + const root = await documentRoot("{ not json }"); + const result = spawnSync("python3", [path.join(repoRoot, ".github/scripts/publish_whats_new.py")], { + cwd: root, + encoding: "utf8", + env: { + ...process.env, + WHATS_NEW_DOCUMENT: WHATS_NEW_DOCUMENT_PATH, + WHATS_NEW_DRY_RUN: "false", + WHATS_NEW_STORAGE_ENDPOINT: "", + WHATS_NEW_STORAGE_BUCKET: "", + WHATS_NEW_STORAGE_ACCESS_KEY_ID: "", + WHATS_NEW_STORAGE_SECRET_ACCESS_KEY: "", + }, + }); + expect(result.status).toBe(1); + expect(result.stderr).toContain("not valid JSON"); + }); +}); + +// The card reaches every installed client the moment it is published, so +// "published" must imply "reviewed". Nothing written inside the workflow file +// can enforce that on its own: `workflow_dispatch` runs the workflow from the +// ref it is dispatched against, so every check in it is editable by whoever +// triggers it. The control is the `whats-new-publish` environment — main-only +// branch policy, R2 credentials as secrets on it — and that control only holds +// while the workflow keeps a specific shape: the job that can reach the +// credentials is the job that declares the environment, and it cannot start +// unless validation succeeded. +// +// Moving `environment:` or a `secrets.*` expression onto the job that runs +// from arbitrary refs reopens the hole with no visible symptom — the workflow +// still publishes correctly from `main`. Each case below mutates the real +// workflow into exactly one of those regressions and requires the checker to +// name it, so the assertions cannot pass by never looking at anything. +describe("what's new publish workflow trust boundary", () => { + let workflow: string; + + beforeEach(async () => { + workflow = await readFile(path.join(repoRoot, WHATS_NEW_WORKFLOW_PATH), "utf8"); + }); + + /** + * Apply one textual regression. Asserting the edit landed is the point: a + * mutation that silently no-ops would leave the "this goes red" case green + * for the wrong reason. + */ + function mutate(source: string, find: string, replace: string): string { + expect(source).toContain(find); + const mutated = source.replace(find, replace); + expect(mutated).not.toBe(source); + return mutated; + } + + test("the shipped workflow satisfies the boundary", () => { + expect(findWhatsNewWorkflowViolations(workflow)).toEqual([]); + }); + + test("the parser sees the two real jobs, so the assertions are not vacuous", () => { + expect(workflowJobNames(workflow)).toEqual(["validate", "publish"]); + const parsed = parseWorkflowJobs(workflow); + expect(parsed).not.toBeNull(); + expect(JSON.stringify(parsed?.jobs.get("validate"))).toContain("scripts/check-whats-new-document.ts"); + expect(JSON.stringify(parsed?.jobs.get("publish"))).toContain("publish_whats_new.py"); + }); + + // The regression the reviewer named: the environment expression drifts off + // the credential-bearing job and onto the one that any ref can dispatch. + test("moving the environment onto the arbitrary-ref job is rejected", () => { + const lifted = mutate(workflow, " environment: whats-new-publish\n", ""); + const onValidate = mutate( + lifted, + " validate:\n name: validate document\n", + " validate:\n name: validate document\n environment: whats-new-publish\n", + ); + const reported = findWhatsNewWorkflowViolations(onValidate).join("\n"); + expect(reported).toContain("job `validate` declares environment `whats-new-publish`"); + expect(reported).toContain("job `publish` declares environment (none)"); + }); + + test("a mistyped environment name is rejected", () => { + const mutated = mutate(workflow, "environment: whats-new-publish", "environment: whats-new-publish-2"); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("it must be exactly `whats-new-publish`"); + }); + + test("dropping the environment entirely is rejected", () => { + const mutated = mutate(workflow, " environment: whats-new-publish\n", ""); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("declares environment (none)"); + }); + + test("unhooking publish from validate is rejected", () => { + const mutated = mutate(workflow, " needs: validate\n", ""); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("must declare `needs: validate`"); + }); + + test("publishing past a failed validate is rejected", () => { + const mutated = mutate( + workflow, + " if: needs.validate.outputs.publish == 'true'", + " if: always() && needs.validate.outputs.publish == 'true'", + ); + const reported = findWhatsNewWorkflowViolations(mutated).join("\n"); + expect(reported).toContain("uses `always()`"); + }); + + test("a dry run that can still publish is rejected", () => { + const mutated = mutate(workflow, " if: needs.validate.outputs.publish == 'true'\n", ""); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain( + "must be gated on `if: needs.validate.outputs.publish == 'true'`", + ); + }); + + test.each(["CLOUDFLARE_R2_WHATS_NEW_AK", "CLOUDFLARE_R2_WHATS_NEW_BUCKET"] as const)( + "reading %s from the arbitrary-ref job is rejected", + (secret) => { + const mutated = mutate( + workflow, + " - name: Setup Node.js", + ` - name: Leak\n env:\n STOLEN: \${{ secrets.${secret} }}\n run: env\n\n - name: Setup Node.js`, + ); + const reported = findWhatsNewWorkflowViolations(mutated).join("\n"); + expect(reported).toContain("job `validate` reads the `secrets` context"); + expect(reported).toContain(secret); + }, + ); + + /** A step that exfiltrates whatever `expression` resolves to. */ + function leakStep(expression: string): string { + return [" - name: Leak", " env:", ` STOLEN: ${expression}`, " run: env", ""].join( + "\n", + ); + } + + /** Splice a step into `validate`, the job any ref can dispatch. */ + function intoValidate(source: string, step: string): string { + return mutate(source, " - name: Setup Node.js", `${step}\n - name: Setup Node.js`); + } + + // GitHub's expression language reaches a secret in more shapes than + // `secrets.NAME`, and a detector that enumerates shapes is only ever as good + // as the list someone thought of. Each of these was verified to slip past + // dot-notation matching, returning an empty violation list — in the guard + // that exists specifically to stop a secret reaching the arbitrary-ref job. + test.each([ + ["single-quoted bracket", "${{ secrets['UNRELATED_SECRET'] }}"], + ["double-quoted bracket", '${{ secrets["UNRELATED_SECRET"] }}'], + ["bracket with inner whitespace", "${{ secrets [ 'UNRELATED_SECRET' ] }}"], + ["dot with surrounding whitespace", "${{ secrets . UNRELATED_SECRET }}"], + ["the whole context serialized", "${{ toJSON(secrets) }}"], + ["the whole context bare", "${{ secrets }}"], + ["the whole context round-tripped", "${{ fromJSON(toJSON(secrets)).UNRELATED_SECRET }}"], + ["an expression folded across lines", "${{\n secrets.UNRELATED_SECRET\n }}"], + ] as const)("a secret reaching the arbitrary-ref job as %s is rejected", (_shape, expression) => { + const mutated = intoValidate(workflow, leakStep(expression)); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("job `validate` reads"); + }); + + // The one shape no raw-text scanner can see: a double-quoted YAML scalar + // with a backslash line continuation. The source never contains the string + // `secrets`; the parsed value does. Only reading the document as YAML — not + // as text — closes this. + test("a secret hidden by a YAML line continuation is rejected", () => { + const hidden = [' STOLEN: "${{ sec\\', " rets.UNRELATED_SECRET }}\""].join("\n"); + const step = [" - name: Leak", " env:", hidden, " run: env", ""].join("\n"); + const mutated = intoValidate(workflow, step); + // Precondition: the giveaway substring is genuinely absent from the source, + // so a passing assertion below cannot be text matching by accident. + expect(mutated).not.toContain("secrets.UNRELATED_SECRET"); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("job `validate` reads"); + }); + + test.each([ + ["a run: block", " - name: Leak\n run: echo ${{ secrets['UNRELATED_SECRET'] }}\n"], + [ + "a with: input", + " - name: Leak\n uses: actions/github-script@v8\n with:\n github-token: ${{ secrets['UNRELATED_SECRET'] }}\n", + ], + ] as const)("a secret reaching the arbitrary-ref job through %s is rejected", (_where, step) => { + const mutated = intoValidate(workflow, step); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("job `validate` reads"); + }); + + // `secrets:` as a job key passes credentials into a called workflow without + // any `${{ }}` expression at all, so expression scanning alone cannot see it. + test("passing secrets into a called workflow is rejected", () => { + // No `${{ }}` anywhere, so expression scanning alone cannot see this one. + const mutated = mutate( + workflow, + " validate:\n name: validate document\n", + " validate:\n name: validate document\n secrets:\n PASSED: literal\n", + ); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("job `validate` declares a `secrets:` key"); + }); + + // A document this guard cannot understand must fail, never pass quietly: + // "no violations found" and "nothing was examined" have to be distinct. + test.each([ + ["unparseable YAML", (source: string) => `${source}\n : : :\n`], + ["a jobs mapping written in flow style", () => "name: x\njobs: {validate: {runs-on: a}, publish: {runs-on: b}}\n"], + ["a renamed publish job", (source: string) => source.replace(" publish:\n", " upload:\n")], + ] as const)("%s fails closed", (_shape, transform) => { + expect(findWhatsNewWorkflowViolations(transform(workflow)).length).toBeGreaterThan(0); + }); + + test("any secret at all on the arbitrary-ref job is rejected, not just the R2 four", () => { + const mutated = mutate( + workflow, + " - name: Setup Node.js", + " - name: Leak\n env:\n STOLEN: ${{ secrets.GITHUB_TOKEN }}\n run: env\n\n - name: Setup Node.js", + ); + const reported = findWhatsNewWorkflowViolations(mutated).join("\n"); + expect(reported).toContain("job `validate` reads the `secrets` context"); + expect(reported).toContain("GITHUB_TOKEN"); + }); + + test("dropping one of the four credential bindings is rejected", () => { + const mutated = mutate( + workflow, + " WHATS_NEW_STORAGE_BUCKET: ${{ secrets.CLOUDFLARE_R2_WHATS_NEW_BUCKET }}\n", + "", + ); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain( + "no longer reads `secrets.CLOUDFLARE_R2_WHATS_NEW_BUCKET`", + ); + }); + + test("handing the whole secret store to a called workflow is rejected", () => { + const mutated = mutate( + workflow, + " validate:\n name: validate document\n", + " validate:\n name: validate document\n secrets: inherit\n", + ); + expect(findWhatsNewWorkflowViolations(mutated).join("\n")).toContain("job `validate` declares a `secrets:` key"); + }); + + // The boundary has to be checked by something that actually runs. An edit to + // `.github/workflows/whats-new-publish.yml` selects neither + // `web_tests_required` nor `ui_p0_validation_required`, so the `e2e_vitest` + // workload holding this file does NOT run for it. `pnpm guard` does, from + // the unconditional preflight job — which is why the assertions above live + // in a guard check rather than only here. + test("the check is registered in the public guard entrypoint", () => { + const names = execFileSync("pnpm", ["--silent", "guard", "--list-checks"], { + cwd: repoRoot, + encoding: "utf8", + }) + .trim() + .split("\n"); + expect(names).toContain("what's new publish workflow"); + }); +}); diff --git a/package.json b/package.json index b1f6d9a365..3d0d16afcb 100644 --- a/package.json +++ b/package.json @@ -36,7 +36,8 @@ "@open-design/tools-serve": "workspace:*", "@types/node": "20.19.39", "tsx": "4.22.3", - "typescript": "5.9.3" + "typescript": "5.9.3", + "yaml": "2.9.0" }, "engines": { "node": "~24", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 74984f8189..4cf2684be3 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -50,6 +50,9 @@ importers: typescript: specifier: 5.9.3 version: 5.9.3 + yaml: + specifier: 2.9.0 + version: 2.9.0 apps/closure: dependencies: diff --git a/scripts/check-whats-new-document.ts b/scripts/check-whats-new-document.ts new file mode 100644 index 0000000000..195bf50203 --- /dev/null +++ b/scripts/check-whats-new-document.ts @@ -0,0 +1,193 @@ +/* ─────────────────────────────────────────────────────────────────── + * Guard: the repository copy of the post-update "What's New" card. + * + * `docs/whats-new.json` is the source of truth that + * `.github/workflows/whats-new-publish.yml` uploads to the R2 object the + * daemon reads (`DEFAULT_WHATS_NEW_URL`). The daemon parses that document + * fail-safe: anything missing or malformed resolves to "no highlight" + * rather than an error, so a typo does not fail a request — it silently + * removes the card from every user's Home. A successful upload therefore + * proves nothing about whether the card will appear. + * + * This guard closes that gap before merge by running the shipping parser + * (`parseWhatsNewDocument`) over the repository document and asserting the + * invariant the fail-safe hides: + * + * the document is either a complete, parser-valid highlight, or the + * explicit empty retirement document `{}` — never something in between. + * + * It runs in `pnpm guard`, which CI invokes from the `preflight` job. That + * job is declared `inputs: ["*"]` / `reusable: false` in + * `.github/config/convergence.json` and its guard step carries no `if`, so it + * runs on every pull request. That matters here: the document lives under + * `docs/`, which `.github/config/scopes.json` treats as `certain-exempt` — a + * change to it selects no scoped validation workload at all, so a lane-routed + * test would simply never run for the one edit it exists to check. + * + * Run standalone: `pnpm exec tsx scripts/check-whats-new-document.ts` + * Or as part of `pnpm guard` (registered in scripts/guard.ts). + * ─────────────────────────────────────────────────────────────────── */ + +import { readFile } from "node:fs/promises"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { parseWhatsNewDocument } from "../apps/daemon/src/services/whats-new.ts"; + +const repoRoot = path.resolve(import.meta.dirname, ".."); + +/** Kept in sync with the workflow's `WHATS_NEW_DOCUMENT` input. */ +export const WHATS_NEW_DOCUMENT_PATH = "docs/whats-new.json"; + +// Field names the daemon parser reads. Anything else in the document is a +// typo (`imageURL`, `link_url`, `locale`) that the parser would drop without +// complaint, which is exactly the silent-no-card failure this guard exists +// to catch. +const KNOWN_TOP_LEVEL_KEYS = new Set(["id", "title", "body", "imageUrl", "linkUrl", "locales"]); +const KNOWN_LOCALE_KEYS = new Set(["title", "body", "linkUrl"]); + +function isObject(value: unknown): value is Record { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +export async function checkWhatsNewDocument(root: string = repoRoot): Promise { + const documentPath = path.join(root, WHATS_NEW_DOCUMENT_PATH); + const violations: string[] = []; + + let raw: string; + try { + raw = await readFile(documentPath, "utf8"); + } catch (error) { + console.error(`What's New document check failed: cannot read ${WHATS_NEW_DOCUMENT_PATH}`); + console.error(error); + return false; + } + + let payload: unknown; + try { + payload = JSON.parse(raw) as unknown; + } catch (error) { + console.error(`What's New document check failed: ${WHATS_NEW_DOCUMENT_PATH} is not valid JSON.`); + console.error(error); + return false; + } + + if (!isObject(payload)) { + console.error(`What's New document check failed: ${WHATS_NEW_DOCUMENT_PATH} must be a JSON object.`); + return false; + } + + // Retiring the card is a real operation the daemon supports: an empty + // document resolves to "no highlight" on purpose. Accept it explicitly so + // the completeness rules below cannot be read as "the card can never be + // taken down". + if (Object.keys(payload).length === 0) { + console.log( + `What's New document check passed: ${WHATS_NEW_DOCUMENT_PATH} is the empty retirement document, so no card will show.`, + ); + return true; + } + + for (const key of Object.keys(payload)) { + if (!KNOWN_TOP_LEVEL_KEYS.has(key)) { + violations.push(`unknown top-level field ${JSON.stringify(key)}; the daemon parser ignores it`); + } + } + + const parsed = parseWhatsNewDocument(payload); + if (parsed.id == null || parsed.content == null) { + violations.push( + "the shipping parser resolves this document to no highlight; `id`, `title`, and `body` must all be present, non-empty strings", + ); + } + + const content = parsed.content; + if (content != null) { + // A dropped optional field is the silent half of the failure mode: the + // card still shows, just without the cover art or with the wrong link. + if (payload.imageUrl !== undefined && content.imageUrl == null) { + violations.push("`imageUrl` is present but the parser dropped it; it must be a non-empty https: URL"); + } + if (payload.linkUrl !== undefined && content.linkUrl == null) { + violations.push("`linkUrl` is present but the parser dropped it; it must be a non-empty https: URL"); + } + + const rawLocales = payload.locales; + if (rawLocales !== undefined) { + if (!isObject(rawLocales)) { + violations.push("`locales` must be an object keyed by app locale id (for example `zh-CN`)"); + } else { + for (const [locale, entry] of Object.entries(rawLocales)) { + if (!isObject(entry)) { + violations.push(`locale ${JSON.stringify(locale)} must be an object of title/body/linkUrl overrides`); + continue; + } + for (const key of Object.keys(entry)) { + if (!KNOWN_LOCALE_KEYS.has(key)) { + violations.push(`locale ${JSON.stringify(locale)} has unknown override ${JSON.stringify(key)}`); + } + } + const overrides = content.locales?.[locale]; + if (overrides == null) { + violations.push(`locale ${JSON.stringify(locale)} was dropped by the parser; every override it declares is invalid`); + continue; + } + for (const key of Object.keys(entry)) { + if (!KNOWN_LOCALE_KEYS.has(key)) continue; + if (overrides[key as "title" | "body" | "linkUrl"] == null) { + violations.push( + `locale ${JSON.stringify(locale)} declares ${JSON.stringify(key)} but the parser dropped it; strings must be non-empty and linkUrl must be https:`, + ); + } + } + } + } + } + + for (const [label, text] of [["body", content.body] as const, ...localeBodies(content.locales)]) { + // Each line renders as one bullet, so a blank line renders as an + // empty bullet. + const lines = text.split("\n"); + if (lines.some((line) => line.trim().length === 0)) { + violations.push(`${label} has a blank line; every line renders as one bullet`); + } + } + } + + if (violations.length > 0) { + console.error(`What's New document check failed for ${WHATS_NEW_DOCUMENT_PATH}:`); + for (const violation of violations) console.error(`- ${violation}`); + console.error( + "The daemon resolves a malformed document to 'no highlight' instead of an error, so publishing this file would silently remove the card rather than fail. See docs/whats-new.md.", + ); + return false; + } + + const localeIds = Object.keys(content?.locales ?? {}); + const localeLabel = localeIds.length > 0 ? ` plus ${localeIds.join(", ")} override${localeIds.length === 1 ? "" : "s"}` : ""; + console.log( + `What's New document check passed: ${WHATS_NEW_DOCUMENT_PATH} resolves to highlight id ${parsed.id}${localeLabel}.`, + ); + return true; +} + +function localeBodies( + locales: Record | undefined, +): ReadonlyArray { + if (locales == null) return []; + const entries: Array = []; + for (const [locale, override] of Object.entries(locales)) { + if (typeof override.body === "string") entries.push([`locale ${locale} body`, override.body] as const); + } + return entries; +} + +// ─── Standalone entrypoint ─────────────────────────────────────────── + +const isInvokedDirectly = + process.argv[1] != null && fileURLToPath(import.meta.url) === path.resolve(process.argv[1]); + +if (isInvokedDirectly) { + const passed = await checkWhatsNewDocument(); + if (!passed) process.exitCode = 1; +} diff --git a/scripts/check-whats-new-publish-workflow.ts b/scripts/check-whats-new-publish-workflow.ts new file mode 100644 index 0000000000..220577e939 --- /dev/null +++ b/scripts/check-whats-new-publish-workflow.ts @@ -0,0 +1,413 @@ +/* ─────────────────────────────────────────────────────────────────── + * Guard: the trust boundary of the "What's New" publish workflow. + * + * The card is visible to every installed client the moment it lands, so + * "published" has to imply "reviewed". Nothing written inside + * `.github/workflows/whats-new-publish.yml` can enforce that on its own: + * `workflow_dispatch` runs the workflow file *from the ref it is dispatched + * against*, so every check in that file is editable by whoever triggers it. + * + * The control is the `whats-new-publish` GitHub environment — its + * deployment-branch policy allows `main` only, and the R2 credentials are + * secrets on it. That control only holds while the workflow keeps a specific + * shape: + * + * the job that can reach the credentials is the job that declares the + * environment, and it cannot start unless validation succeeded first. + * + * Move `environment:` or a secret onto the job that runs from arbitrary refs + * and the boundary is gone, with no failing test and no visible symptom — the + * workflow still publishes correctly from `main`. This guard is what makes + * that edit red. + * + * It runs in `pnpm guard`, which CI invokes from the `preflight` job. That job + * is enabled unconditionally in `.github/scripts/scopes.py` and its guard step + * carries no `if`, so it runs on every pull request. That is load-bearing here + * for the same reason it is for `check-whats-new-document.ts`: an edit to + * `.github/workflows/whats-new-publish.yml` selects neither `web_tests_required` + * nor `ui_p0_validation_required`, so the `e2e_vitest` workload does not run + * for it. A lane-routed assertion would be skipped for exactly the change class + * it exists to catch. + * + * ── Why this reads the document as YAML, not as text ────────────────── + * + * The first version matched `${{ secrets.NAME }}` with a regex over the raw + * file. That is a detector built from a list of shapes someone thought of, and + * it is only as good as the list. Verified bypasses of it, all of which reached + * a secret into the arbitrary-ref job while the check returned no violations: + * `secrets['NAME']`, `secrets["NAME"]`, `secrets [ 'NAME' ]`, + * `secrets . NAME`, `toJSON(secrets)`, a bare `secrets`, and + * `fromJSON(toJSON(secrets)).NAME`. + * + * One of them cannot be fixed by a better regex at all. A double-quoted YAML + * scalar may carry a backslash line continuation, so + * + * STOLEN: "${{ sec\ + * rets.NAME }}" + * + * parses to `${{ secrets.NAME }}` while the *source text* contains no such + * substring. Any raw-text scanner is blind to it by construction. Reading the + * document through a YAML parser is what closes that class, and it removes the + * hand-rolled job splitter along with its own failure modes (flow-style + * mappings, quoted job keys, unusual indentation). + * + * Detection is then one rule rather than a shape list: GitHub's expression + * language has no string-to-context evaluation, so a secret cannot be reached + * without naming the `secrets` context as an identifier. Finding that + * identifier inside any `${{ … }}` expression catches every access form, + * including ones nobody has thought of yet. + * + * Run standalone: `pnpm exec tsx scripts/check-whats-new-publish-workflow.ts` + * Or as part of `pnpm guard` (registered in scripts/guard.ts). + * ─────────────────────────────────────────────────────────────────── */ + +import { readFile } from "node:fs/promises"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +import { parse } from "yaml"; + +const repoRoot = path.resolve(import.meta.dirname, ".."); + +export const WHATS_NEW_WORKFLOW_PATH = ".github/workflows/whats-new-publish.yml"; + +/** The protected environment. A typo here silently creates an unprotected one. */ +export const WHATS_NEW_PUBLISH_ENVIRONMENT = "whats-new-publish"; + +/** The job allowed to hold the environment and the credentials. */ +export const WHATS_NEW_PUBLISH_JOB = "publish"; + +/** The job whose output gates publication. */ +export const WHATS_NEW_VALIDATE_JOB = "validate"; + +/** The gate that keeps a dry run from publishing. */ +const PUBLISH_CONDITION = "needs.validate.outputs.publish == 'true'"; + +/** + * Status functions that would let publication run past a failed `validate`, or + * past a failed guard step inside `publish`. + */ +const CONDITION_ESCAPES = ["always", "cancelled", "failure"] as const; + +/** + * Secrets on the `whats-new-publish` environment. Kept here rather than read + * out of the workflow so that *dropping* a binding is a violation too — a + * publish job missing one of these fails at runtime with a credentials error, + * which reads like a configuration problem rather than a workflow edit. + */ +export const WHATS_NEW_R2_SECRETS = [ + "CLOUDFLARE_R2_WHATS_NEW_AK", + "CLOUDFLARE_R2_WHATS_NEW_SK", + "CLOUDFLARE_R2_WHATS_NEW_URL", + "CLOUDFLARE_R2_WHATS_NEW_BUCKET", +] as const; + +// ─── GitHub Actions expression scanning ────────────────────────────── + +/** + * The `${{ … }}` expressions in one scalar, in source order. + * + * Single-quoted string literals are skipped while scanning for the closing + * `}}` (GitHub escapes a quote by doubling it). Without that, `format('}}')` + * would end the span early and hide whatever follows it in the same + * expression. + */ +function expressionSpans(scalar: string): string[] { + const spans: string[] = []; + let cursor = 0; + + while (cursor < scalar.length) { + const start = scalar.indexOf("${{", cursor); + if (start < 0) break; + + let index = start + 3; + let quoted = false; + while (index < scalar.length) { + const character = scalar[index]; + if (quoted) { + if (character === "'") { + if (scalar[index + 1] === "'") { + index += 2; + continue; + } + quoted = false; + } + index += 1; + continue; + } + if (character === "'") { + quoted = true; + index += 1; + continue; + } + if (character === "}" && scalar[index + 1] === "}") break; + index += 1; + } + + spans.push(scalar.slice(start + 3, Math.min(index, scalar.length))); + cursor = index + 2; + } + + return spans; +} + +/** Blank out string literals so their contents cannot read as identifiers. */ +function withoutStringLiterals(expression: string): string { + return expression.replace(/'(?:''|[^'])*'/g, " "); +} + +/** + * `secrets` used as a context identifier, not as a property name on something + * else (`inputs.secrets`) and not as part of a longer word. + */ +const SECRETS_CONTEXT = /(? SECRETS_CONTEXT.test(withoutStringLiterals(expression))); +} + +/** Secret names read by dot or bracket access, for binding completeness. */ +function referencedSecretNames(scalar: string): string[] { + const names: string[] = []; + for (const expression of expressionSpans(scalar)) { + for (const match of expression.matchAll(/\bsecrets\s*\.\s*([A-Za-z0-9_]+)/g)) names.push(match[1] as string); + for (const match of expression.matchAll(/\bsecrets\s*\[\s*'([^']+)'\s*\]/g)) names.push(match[1] as string); + for (const match of expression.matchAll(/\bsecrets\s*\[\s*"([^"]+)"\s*\]/g)) names.push(match[1] as string); + } + return names; +} + +// ─── Parsed-document traversal ─────────────────────────────────────── + +function isMapping(value: unknown): value is Record { + return value !== null && typeof value === "object" && !Array.isArray(value); +} + +/** Every string in a subtree, keys included — nothing is out of scope. */ +function scalarStrings(value: unknown, collected: string[] = []): string[] { + if (typeof value === "string") collected.push(value); + else if (Array.isArray(value)) for (const item of value) scalarStrings(item, collected); + else if (isMapping(value)) { + for (const [key, child] of Object.entries(value)) { + collected.push(key); + scalarStrings(child, collected); + } + } + return collected; +} + +/** Every `if:` expression in a subtree — job-level and step-level alike. */ +function conditionals(value: unknown, collected: string[] = []): string[] { + if (Array.isArray(value)) for (const item of value) conditionals(item, collected); + else if (isMapping(value)) { + for (const [key, child] of Object.entries(value)) { + if (key === "if" && typeof child === "string") collected.push(child); + conditionals(child, collected); + } + } + return collected; +} + +/** Job-level `environment:`, which may be a name or a `{ name, url }` mapping. */ +function declaredEnvironment(job: unknown): string | null { + if (!isMapping(job)) return null; + const environment = job.environment; + if (typeof environment === "string") return environment; + if (isMapping(environment) && typeof environment.name === "string") return environment.name; + return environment === undefined ? null : "(unrecognized)"; +} + +/** Job-level `needs:`, normalized to a list. */ +function declaredNeeds(job: unknown): string[] | null { + if (!isMapping(job) || job.needs === undefined) return null; + if (typeof job.needs === "string") return [job.needs]; + if (Array.isArray(job.needs) && job.needs.every((item) => typeof item === "string")) return job.needs as string[]; + return []; +} + +export interface WorkflowJobs { + readonly jobs: ReadonlyMap; + /** Everything outside `jobs:` — workflow-level `env:` reaches every job. */ + readonly workflowLevel: unknown; +} + +/** + * Parse the workflow into its jobs. Returns null when the document cannot be + * understood, which callers must treat as a violation rather than as "nothing + * found" — "no problems" and "nothing was examined" have to stay distinct. + */ +export function parseWorkflowJobs(workflow: string): WorkflowJobs | null { + let document: unknown; + try { + document = parse(workflow); + } catch { + return null; + } + if (!isMapping(document) || !isMapping(document.jobs)) return null; + + const { jobs, ...workflowLevel } = document; + return { jobs: new Map(Object.entries(jobs)), workflowLevel }; +} + +/** Job ids in declaration order. Exported for coverage of the parse step. */ +export function workflowJobNames(workflow: string): string[] { + return [...(parseWorkflowJobs(workflow)?.jobs.keys() ?? [])]; +} + +// ─── The boundary ──────────────────────────────────────────────────── + +/** + * The properties that make the environment an actual control rather than a + * label. Returns one human-readable violation per broken property. + */ +export function findWhatsNewWorkflowViolations(workflow: string): string[] { + const parsed = parseWorkflowJobs(workflow); + if (parsed == null) { + return [ + `${WHATS_NEW_WORKFLOW_PATH} could not be read as a workflow document with a \`jobs:\` mapping; this guard cannot vouch for a file it cannot parse`, + ]; + } + + const violations: string[] = []; + const { jobs, workflowLevel } = parsed; + const publish = jobs.get(WHATS_NEW_PUBLISH_JOB); + + if (!jobs.has(WHATS_NEW_VALIDATE_JOB)) { + violations.push(`no \`${WHATS_NEW_VALIDATE_JOB}\` job; publication must be gated on a validation job`); + } + + if (publish === undefined) { + violations.push( + `no \`${WHATS_NEW_PUBLISH_JOB}\` job; the credential-bearing job must be named \`${WHATS_NEW_PUBLISH_JOB}\` so this guard can hold it to the boundary`, + ); + return violations; + } + + // 1. The credential-bearing job is bound to the protected environment. + const environment = declaredEnvironment(publish); + if (environment !== WHATS_NEW_PUBLISH_ENVIRONMENT) { + violations.push( + `job \`${WHATS_NEW_PUBLISH_JOB}\` declares environment ${environment == null ? "(none)" : `\`${environment}\``}; it must be exactly \`${WHATS_NEW_PUBLISH_ENVIRONMENT}\`, whose deployment-branch policy is what restricts publication to main`, + ); + } + + // 2. It cannot start unless validation actually succeeded. `needs:` is the + // skip-propagating half; the `if:` is the "validated but deliberately a + // dry run" half. + const needs = declaredNeeds(publish); + if (needs == null || needs.length !== 1 || needs[0] !== WHATS_NEW_VALIDATE_JOB) { + violations.push( + `job \`${WHATS_NEW_PUBLISH_JOB}\` must declare \`needs: ${WHATS_NEW_VALIDATE_JOB}\` so a failed validation skips it`, + ); + } + const jobCondition = isMapping(publish) && typeof publish.if === "string" ? publish.if.trim() : null; + if (jobCondition !== PUBLISH_CONDITION) { + violations.push( + `job \`${WHATS_NEW_PUBLISH_JOB}\` must be gated on \`if: ${PUBLISH_CONDITION}\` so a dry run cannot publish`, + ); + } + // A status function anywhere in the job — its own `if` or a step's — would + // let work continue past a failure that was supposed to stop it. + for (const condition of conditionals(publish)) { + for (const escape of CONDITION_ESCAPES) { + if (new RegExp(`\\b${escape}\\s*\\(`).test(condition)) { + violations.push( + `job \`${WHATS_NEW_PUBLISH_JOB}\` uses \`${escape}()\` in \`${condition}\`; that lets work continue past a failure that must stop it`, + ); + } + } + } + + // 3. The four bucket credentials are read by that job. + const publishSecretNames = new Set(scalarStrings(publish).flatMap(referencedSecretNames)); + for (const secret of WHATS_NEW_R2_SECRETS) { + if (!publishSecretNames.has(secret)) { + violations.push( + `job \`${WHATS_NEW_PUBLISH_JOB}\` no longer reads \`secrets.${secret}\`; the publisher needs all four bindings`, + ); + } + } + + // 4. Nothing else may reach a secret. Every other job runs from arbitrary + // refs, and workflow-level `env:` is inherited by all of them. + const elsewhere: Array<[string, unknown]> = [ + ["the workflow level", workflowLevel], + ...[...jobs].filter(([name]) => name !== WHATS_NEW_PUBLISH_JOB).map(([name, job]): [string, unknown] => [ + `job \`${name}\``, + job, + ]), + ]; + + for (const [label, subtree] of elsewhere) { + if (subtree !== workflowLevel) { + const otherEnvironment = declaredEnvironment(subtree); + if (otherEnvironment != null) { + violations.push( + `${label} declares environment \`${otherEnvironment}\`; only \`${WHATS_NEW_PUBLISH_JOB}\` may attach an environment, because any other job can be dispatched from an unreviewed ref`, + ); + } + // `secrets:` as a job key passes credentials into a called workflow with + // no `${{ }}` expression at all, so expression scanning cannot see it. + if (isMapping(subtree) && subtree.secrets !== undefined) { + violations.push( + `${label} declares a \`secrets:\` key; that hands credentials to a called workflow, routing around the environment boundary`, + ); + } + } + + for (const scalar of scalarStrings(subtree)) { + if (!usesSecretsContext(scalar)) continue; + violations.push( + `${label} reads the \`secrets\` context in \`${scalar.replace(/\s+/g, " ").trim().slice(0, 120)}\`; anything that runs from arbitrary refs must hold no secrets`, + ); + } + } + + return violations; +} + +export async function checkWhatsNewPublishWorkflow(root: string = repoRoot): Promise { + const workflowPath = path.join(root, WHATS_NEW_WORKFLOW_PATH); + + let workflow: string; + try { + workflow = await readFile(workflowPath, "utf8"); + } catch (error) { + console.error(`What's New publish workflow check failed: cannot read ${WHATS_NEW_WORKFLOW_PATH}`); + console.error(error); + return false; + } + + const violations = findWhatsNewWorkflowViolations(workflow); + if (violations.length > 0) { + console.error(`What's New publish workflow check failed for ${WHATS_NEW_WORKFLOW_PATH}:`); + for (const violation of violations) console.error(`- ${violation}`); + console.error( + "The card reaches every installed client on publish, so publication must stay bound to the `whats-new-publish` environment (main-only) and unreachable without a green validate. See docs/whats-new.md.", + ); + return false; + } + + console.log( + `What's New publish workflow check passed: only \`${WHATS_NEW_PUBLISH_JOB}\` holds the \`${WHATS_NEW_PUBLISH_ENVIRONMENT}\` environment and its ${WHATS_NEW_R2_SECRETS.length} R2 secrets, gated on \`${WHATS_NEW_VALIDATE_JOB}\`.`, + ); + return true; +} + +// ─── Standalone entrypoint ─────────────────────────────────────────── + +const isInvokedDirectly = + process.argv[1] != null && fileURLToPath(import.meta.url) === path.resolve(process.argv[1]); + +if (isInvokedDirectly) { + const passed = await checkWhatsNewPublishWorkflow(); + if (!passed) process.exitCode = 1; +} diff --git a/scripts/guard.ts b/scripts/guard.ts index 52bcf3285a..5c58911250 100644 --- a/scripts/guard.ts +++ b/scripts/guard.ts @@ -21,6 +21,8 @@ import { checkDesignSystemUnknownTokens, } from "./check-tokens-fixture-sync.ts"; import { checkCraftReferences } from "./lint-craft-references.ts"; +import { checkWhatsNewDocument } from "./check-whats-new-document.ts"; +import { checkWhatsNewPublishWorkflow } from "./check-whats-new-publish-workflow.ts"; import { collectCssHardcodedColorMatches, cssWideAndSpecialColorKeywords, realNamedColors } from "./style-policy.ts"; import { checkScriptsLibraryArchitecture } from "./lib/guard/architecture.ts"; import { runGuardChecks, type GuardCheck, type GuardContext } from "./lib/guard/core.ts"; @@ -1527,6 +1529,8 @@ const checks: GuardCheck[] = [ { name: "tools layout", run: checkToolsLayout }, { name: "style policy", run: checkStylePolicy }, { name: "craft references", run: checkCraftReferences }, + { name: "what's new document", run: ({ repoRoot: root }) => checkWhatsNewDocument(root) }, + { name: "what's new publish workflow", run: ({ repoRoot: root }) => checkWhatsNewPublishWorkflow(root) }, { name: "HTML plugin preview contracts", run: ({ repoRoot: root }) => checkHtmlPluginPreviewContracts(root) }, { name: "plugin preview manifest", run: checkPluginPreviewManifest }, { name: "design system manifests", run: checkDesignSystemManifests }, diff --git a/tools/pack/AGENTS.md b/tools/pack/AGENTS.md index 46e0fb1c11..d830571233 100644 --- a/tools/pack/AGENTS.md +++ b/tools/pack/AGENTS.md @@ -65,7 +65,7 @@ The runtime updater reads `https://releases.open-design.ai//latest/meta - Release metadata may include `control.launcher.version.{min, url}` — the installer-reinstall floor. The updater compares `min` against the **physically installed outer package version** (read at check time from the outer bundle's `open-design-config.json` via the launcher launch path; `OD_UPDATE_INSTALLED_VERSION` overrides it for tests), NOT the running payload version — payload updates never touch the outer bundle, so a broken outer generation must reach the installer path even when its payload is current. When the floor trips (or `min` is set but the outer version is unreadable — conservative), the updater selects the installer artifact and, when no newer release exists, offers a same-version installer reinstall; snapshot field `reinstall` carries `{reason, installedVersion, minVersion, url}` and the web UI presents the optional operator `url` as a jump link with default i18n copy as fallback. Publication: channel policy is managed as one repo-vars pair per channel — `RELEASE_LAUNCHER_VERSION_MIN_` + `RELEASE_LAUNCHER_VERSION_MIN_URL_` — passed through workflows verbatim (no YAML fallback expressions) and resolved by the shared resolver in `tools/release/src/storage/launcher-version-floor.ts`: a non-stable channel whose own pair is unset falls back to the STABLE pair as a unit, and format/https/floor validation is applied at that single point. `publish-metadata` hard-fails when `min` exceeds the release version — a floor this release cannot satisfy would make the same-version reinstall offer nag forever. `verify-metadata` re-resolves the same channel policy and checks the published block against it; `summary-metadata` surfaces the floor in the step summary. - The updater exposes a manual disaster-recovery `clear-cache` action (`od:update:clear-cache` IPC, sidecar action `clear-cache`, Settings → About "Clear update cache" row with a two-stage inline confirm). It resets one-shot update state (downloaded release, install freeze) back to `idle`, purges `releases/`, `staging/`, `downloads/`, and `.back/`, removes a stale launcher `attempt.json` plus any non-`confirmed` desktop-handoff journal, and deletes non-retained launcher payload versions. Runtime `active`/`lastSuccessful` versions, explicit `retained` cleanup entries, `install.json`, and a `confirmed` handoff journal are never touched; locked files defer through the existing cleanup.json retry machinery. An installer helper already spawned by a prior install action is not cancelled. - Release-note source lives at `docs/CHANGELOG/v/.md`. All channels use the same publication pipeline, while stable additionally requires `en` and `zh-CN` before platform builds proceed. -- Post-update "what's new" highlights are NOT carried in release `metadata.json`. The daemon's `/api/whats-new` fetches a single hand-curated document on a dedicated R2 bucket (`https://whatsnew.open-design.ai/whats-new.json`, overridable with `OD_WHATS_NEW_URL`); the web home surface shows a one-time card driven by that document's `id`, not the running version. Operators edit that one file after a release — there is no per-version publish tooling. +- Post-update "what's new" highlights are NOT carried in release `metadata.json`. The daemon's `/api/whats-new` fetches a single hand-curated document on a dedicated R2 bucket (`https://whatsnew.open-design.ai/whats-new.json`, overridable with `OD_WHATS_NEW_URL`); the web home surface shows a one-time card driven by that document's `id`, not the running version. The document is maintained in this repository at `docs/whats-new.json` and published by `.github/workflows/whats-new-publish.yml`; operators edit that one file after a release — there is no per-version publish tooling. See `docs/whats-new.md`. ### Channel identity rules