mirror of
https://github.com/nexu-io/open-design.git
synced 2026-10-27 09:51:12 +01:00
Marc ChanandGitHub
2ba458503f
* fix(pack): notarize macOS builds only once * perf(pack): separate type validation and defer bounded Windows caches * test(e2e): bound BuildKit cleanup job away from pack cache save The deferred Windows pack-cache job checkouts the repo, so slicing clean-closed-pr-buildkit to EOF made the no-checkout assertion fail. Generated-By: looper 0.16.1 (runner=fixer, agent=omp) * test(ci): bound cache cleanup assertions to their job * fix(ci): preserve cache provenance when the release branch advances * perf(ci): reduce compression work in Windows cache handoff * fix(ci): align Windows cache restore and persistence paths * perf(ci): retain latest complete Windows artifacts for repeat builds * fix(ci): keep newer pack caches when ancestor saves finish later Prefix pruning in cache-maintenance used cache creation time, so an older ancestor finishing after a newer same-ref release deleted that newer cache. Prune only when the validated source SHA is the current commit. Generated-By: looper 0.16.1 (runner=fixer, agent=omp)
227 lines
15 KiB
Markdown
227 lines
15 KiB
Markdown
# tools/pack
|
|
|
|
Local packaging control plane for Open Design.
|
|
|
|
`tools-pack` is the cross-platform packaging and smoke-lifecycle control plane. The macOS commands include:
|
|
|
|
- `tools-pack mac build --to all`
|
|
- `tools-pack mac build --to app|dmg|zip`
|
|
- `tools-pack mac build --to all --signed`
|
|
- `tools-pack mac build --to all --portable` for release artifacts that must not bake local tools-pack runtime paths
|
|
- `tools-pack mac install`
|
|
- `tools-pack mac start`
|
|
- `tools-pack mac stop`
|
|
- `tools-pack mac logs`
|
|
- `tools-pack mac uninstall`
|
|
- `tools-pack mac cleanup`
|
|
|
|
Build artifacts are namespace-scoped under `.tmp/tools-pack/out/mac/namespaces/<namespace>/`.
|
|
Public release bundles keep channel-distinct identities: `Open Design.app`, `Open Design Beta.app`,
|
|
`Open Design Prerelease.app`, or `Open Design Preview.app`. Local `tools-pack install` adds the developer
|
|
namespace so installs can coexist without affecting runtime data/log/cache paths.
|
|
|
|
Packaged runtime state is namespace-scoped under `.tmp/tools-pack/runtime/mac/namespaces/<namespace>/`:
|
|
|
|
- Packaged daemon storage is governed only by the root `AGENTS.md` section
|
|
**Daemon data directory contract**. Before changing or documenting packaged
|
|
storage propagation, you MUST read that section; this README MUST NOT
|
|
restate it.
|
|
- `logs/` contains packaged process logs for `desktop`, `web`, and `daemon`.
|
|
- `runtime/` is the sidecar runtime base used by the packaged desktop/web/daemon process group.
|
|
- `cache/` is reserved for namespace-local packaged cache state.
|
|
- `user-data/` is the Electron/Chromium `userData` root, with `user-data/session/` used for `sessionData`.
|
|
|
|
Every packaged root registers the exact five-field sidecar stamp
|
|
(`channel / namespace / source / mode / app`) in its own argv. Direct OS
|
|
launches register during packaged startup; tools-pack launches receive the same
|
|
stamp through the sidecar launch atomic. No identity file participates in
|
|
discovery or ownership.
|
|
|
|
`tools-pack mac stop` scans exact argv stamps for desktop, Web, and daemon across
|
|
both tools-pack and direct packaged sources, requests private graceful teardown,
|
|
waits for every captured generation to exit, and then force-stops only the
|
|
still-matching process trees. Cleanup and uninstall refuse to remove artifacts
|
|
while any captured generation remains alive.
|
|
|
|
This keeps `stop` from killing processes outside the current namespace.
|
|
|
|
Packaged desktop also writes main-process lifecycle logs to `logs/desktop/latest.log` so Finder/manual launches are
|
|
diagnosable. This log is intentionally scoped to packaged desktop startup/shutdown/process errors and does not capture
|
|
web/renderer console output.
|
|
|
|
The packaged daemon path contract lives only in the root `AGENTS.md` section
|
|
**Daemon data directory contract**. Before changing or documenting packaged
|
|
path propagation, you MUST read that section; this README MUST NOT restate it.
|
|
|
|
Packaged desktop checks release metadata, verifies the downloaded artifact, and exposes update actions through desktop
|
|
IPC. Launcher-based builds prefer verified payload activation followed by relaunch; installer replacement remains the
|
|
fallback for artifact types and older builds that cannot apply a payload in place.
|
|
|
|
Electron-builder resources live under `tools/pack/resources/mac/`. The current logo is staged there as the mac icon/DMG
|
|
placeholder so future design-provided assets can replace the resource files without changing packaging code.
|
|
|
|
Local developer artifacts bake the tools-pack namespace runtime root so `tools-pack mac start/stop/logs/cleanup` can manage
|
|
them from the repo. Release artifacts use `--portable` so the installed app resolves namespace data/log/runtime/user-data
|
|
from the user's Electron `userData` root instead of the build machine's `.tmp` path.
|
|
|
|
### macOS compatibility notes
|
|
|
|
- `tools-pack mac build --portable --to zip` is the safest manual-install artifact for Intel Macs. This path was smoke-tested on macOS 12.7.6 Monterey on a 2015 Intel iMac and the app launched successfully from `/Applications`.
|
|
- Finder/manual launches on macOS may not inherit your shell-managed `PATH`. If packaged Open Design cannot detect agent CLIs that work in Terminal, expose those binaries to the GUI login environment or launch the packaged app from a shell session that already sees them.
|
|
|
|
## Windows
|
|
|
|
Local lifecycle commands:
|
|
|
|
- `tools-pack win build --to dir` for fast unpacked smoke builds.
|
|
- `tools-pack win build --to nsis` for installer builds.
|
|
- `tools-pack win build --to all` for both outputs.
|
|
- `tools-pack win install`
|
|
- `tools-pack win start`
|
|
- `tools-pack win inspect --expr "document.title"`
|
|
- `tools-pack win logs`
|
|
- `tools-pack win stop`
|
|
- `tools-pack win cleanup`
|
|
- `tools-pack win list`
|
|
- `tools-pack win reset`
|
|
|
|
Build artifacts are namespace-scoped under `.tmp/tools-pack/out/win/namespaces/<namespace>/`.
|
|
Packaged runtime state is namespace-scoped under `.tmp/tools-pack/runtime/win/namespaces/<namespace>/`.
|
|
`--to dir` may point `built-app.json` at an immutable cached `win-unpacked` executable while keeping
|
|
namespace-local config and runtime paths outside that cache entry.
|
|
|
|
## Linux
|
|
|
|
Local lifecycle commands:
|
|
|
|
- `tools-pack linux build --to all` (default; produces AppImage)
|
|
- `tools-pack linux build --to appimage` (explicit AppImage)
|
|
- `tools-pack linux build --to dir` (unpacked output for fast iteration)
|
|
- `tools-pack linux build --containerized` (run electron-builder inside `electronuserland/builder:base` Docker for a wider glibc compatibility target — requires Docker)
|
|
- `tools-pack linux build --to all --portable` (release artifacts that must not bake local tools-pack runtime paths)
|
|
- `tools-pack linux install`
|
|
- `tools-pack linux install --headless` (install the headless launcher script instead of the AppImage)
|
|
- `tools-pack linux start`
|
|
- `tools-pack linux start --headless` (start the headless entry — daemon + web, no Electron)
|
|
- `tools-pack linux stop`
|
|
- `tools-pack linux stop --headless` (stop a running headless process)
|
|
- `tools-pack linux inspect` (desktop status, eval, and screenshot for AppImage mode)
|
|
- `tools-pack linux inspect --headless` (status only)
|
|
- `tools-pack linux logs`
|
|
- `tools-pack linux uninstall`
|
|
- `tools-pack linux uninstall --headless`
|
|
- `tools-pack linux cleanup`
|
|
- `tools-pack linux cleanup --headless`
|
|
|
|
Build artifacts are namespace-scoped under `.tmp/tools-pack/out/linux/namespaces/<namespace>/`. Packaged logs, sidecar runtime, cache, and Electron user-data are namespace-scoped under the tools-pack runtime root. Daemon storage follows only the root `AGENTS.md` **Daemon data directory contract**. Containerized build cache lives under `.tmp/tools-pack/.docker-cache/{electron,electron-builder}/`.
|
|
|
|
Local installs use XDG paths:
|
|
|
|
- AppImage: `~/.local/bin/Open-Design.<namespace>.AppImage`
|
|
- Menu entry: `~/.local/share/applications/open-design-<namespace>.desktop`
|
|
- Icon: `~/.local/share/icons/hicolor/512x512/apps/open-design-<namespace>.png`
|
|
|
|
The `<namespace>` suffix is unconditional so multiple developer namespaces can coexist on the same desktop. The `.desktop` file registers the `od://` scheme via `MimeType=x-scheme-handler/od;` and pre-sets `OD_PACKAGED_NAMESPACE` on the `Exec=` line so menu launches identify the correct namespace.
|
|
|
|
### Headless mode (`--headless`)
|
|
|
|
Headless mode targets environments without a display (WSL2, headless servers, CI) where Electron can't run. If you have a desktop, use the AppImage; if you're SSH'd into a machine or in WSL, use headless.
|
|
|
|
`--headless` makes `install`, `start`, `stop`, `uninstall`, and `cleanup` operate on the headless entry (`@open-design/packaged/dist/headless.mjs`) instead of the AppImage. Headless mode runs daemon + web without Electron.
|
|
|
|
- `install --headless` writes a shell launcher at `~/.local/bin/open-design-headless-<namespace>` that bakes in the namespace and resource paths. The launcher is self-contained, but the assembled app directory at those paths must remain in place — don't move it after install.
|
|
- `start --headless` launches through the sidecar atomic and waits up to 95s for private status readiness.
|
|
- `stop --headless` terminally stops the exact `mode=headless` argv stamp. The mode field keeps it disjoint from the AppImage desktop without identity files or public IPC paths.
|
|
- `inspect --headless` returns status only. Eval and screenshot require AppImage mode because there is no Electron renderer in headless mode.
|
|
- `uninstall --headless` removes the headless launcher after a safe stop.
|
|
- `cleanup --headless` stops the headless process before removing namespace output/runtime roots.
|
|
|
|
`logs` always reads `logs/desktop/latest.log` regardless of mode, so headless output is visible via `tools-pack linux logs`.
|
|
|
|
### AppImage launch mode (FUSE caveat)
|
|
|
|
`tools-pack linux start` always spawns the AppImage with `--appimage-extract-and-run`. Smoke testing on Ubuntu 24.04 and Arch Linux showed that direct FUSE-mounted AppImage launches make Node module loads (Express, better-sqlite3, etc.) slow enough that the daemon sidecar consistently failed to clear `apps/packaged`'s 35-second startup timeout. Extract-and-run unpacks the AppImage into `/tmp/appimage_extracted_<hex>/` and exec's the inner Electron from there, bypassing FUSE and getting daemon boot in under 5 seconds — roughly an order-of-magnitude improvement.
|
|
|
|
**Implication for end-users:** if launching the installed AppImage manually (not via `tools-pack linux start`), pass `--appimage-extract-and-run` yourself, or rely on a desktop launcher / `appimage-launcher` daemon that handles extract-and-run automatically.
|
|
|
|
### Optional system tools
|
|
|
|
`tools-pack linux install` and `tools-pack linux uninstall` invoke `update-desktop-database` and `gtk-update-icon-cache` as best-effort post-hooks. Either tool being absent (`iconCache: "missing"` in the output) is harmless — the icon and menu entry still work, the cache just isn't refreshed. Install via your distro:
|
|
|
|
- Arch / CachyOS: `sudo pacman -S desktop-file-utils gtk-update-icon-cache`
|
|
- Debian / Ubuntu: `sudo apt install desktop-file-utils gtk-update-icon-cache`
|
|
- Fedora: `sudo dnf install desktop-file-utils gtk-update-icon-cache`
|
|
|
|
`libfuse2` is needed for FUSE-mounted AppImage launch (the default mode when running an AppImage directly without `--appimage-extract-and-run`). `tools-pack linux start` always uses extract-and-run and bypasses FUSE entirely, so it does not need `libfuse2`. Most modern distros ship `libfuse2` by default; older Ubuntu LTS hosts may need `sudo apt install libfuse2t64` (or `libfuse2` on pre-24.04).
|
|
|
|
### Sandbox / chrome-sandbox
|
|
|
|
Electron 41 on Linux requires `kernel.unprivileged_userns_clone=1` (default on Arch, Ubuntu 24+, Debian 12+) or AppImage's `--no-sandbox` fallback. Most modern distros need no extra setup.
|
|
|
|
### Distro compatibility target
|
|
|
|
AppImages built natively on a rolling distro (e.g., Arch / CachyOS) link against recent glibc and may not run on stable distros (Ubuntu 22.04, Debian 12). Use `--containerized` to build against the `electronuserland/builder:base` baseline (Ubuntu 18.04 / glibc 2.27), which is the compatibility target for release AppImages rather than a guarantee for every Linux distribution.
|
|
|
|
Verified smoke coverage in this repository currently includes:
|
|
|
|
- Main PR CI: packaged Linux smoke is intentionally outside the main CI gate.
|
|
- Release lane: Ubuntu GitHub-hosted runner, containerized AppImage build plus Xvfb AppImage runtime smoke when the Linux release lane is enabled.
|
|
- Manual AppImage behavior used to choose `--appimage-extract-and-run`: Ubuntu 24.04 and Arch Linux.
|
|
|
|
### Format choice: why AppImage first
|
|
|
|
Linux desktop apps in this space split across formats: VS Code ships `.deb` + `.rpm` + Snap; Discord ships AppImage + `.deb`; Slack ships `.deb` + `.rpm`; Cursor and Obsidian ship AppImage. We start with AppImage because one artifact can cover the widest glibc-compatible target without distro repositories, store packaging, signing infrastructure, or per-format install scripts, and it integrates cleanly with the namespace-scoped install layout. `.deb` / `.rpm` / Snap / Flatpak can land incrementally when user demand justifies the extra release ownership.
|
|
|
|
### Out of scope (later phases)
|
|
|
|
- AppImage signing (`--signed`) — deferred pending a GPG key infrastructure decision and a user-facing verification flow design (no ETA).
|
|
- AppImage auto-update feed (`latest-linux.yml`) — the linux electron-builder config has no `publish` block wired, so a generated feed would point users at a feed that never updates. Tracked alongside signing.
|
|
- Additional package formats: `.deb`, `.rpm`, Snap, Flatpak — deferred until there is demand and an owner for per-distro metadata, signing/store/repository plumbing, install/remove hooks, and release validation.
|
|
- Full Linux AppImage and headless packaged smoke remain outside the main PR gate; run the applicable tools-pack validation manually or through a release lane when Linux packaging changes.
|
|
|
|
`--to dmg` is manual-install DMG output only. Any builder-generated updater metadata such as `latest-mac.yml` or
|
|
`.blockmap` files is treated as scratch and cleaned from the builder directory; release-beta generates the authoritative
|
|
`latest-mac.yml` feed during release asset preparation, pointing at the update ZIP.
|
|
|
|
## Branch build timing experiments
|
|
|
|
`release-beta.yml` supports branch/SHA builds with `publish=false`. For a full
|
|
macOS arm64, Intel and Windows packaging comparison, dispatch it from the branch
|
|
carrying the workflow and pin `ref` to the commit under test:
|
|
|
|
```bash
|
|
gh workflow run release-beta.yml --ref <branch> \
|
|
-f ref=<commit-sha> -f publish=false \
|
|
-f enable_mac_arm64=true -f enable_mac_x64=true -f enable_win_x64=true \
|
|
-f enable_linux_x64=false \
|
|
-f mac_arm64_target=all -f mac_x64_target=all -f win_x64_target=all \
|
|
-f mac_arm64_sign_mode=notarize -f mac_x64_sign_mode=notarize \
|
|
-f mac_arm64_smoke_mode=core -f mac_x64_smoke_mode=core -f win_x64_smoke_mode=core
|
|
```
|
|
|
|
This creates test artifacts and isolated dogfood downloads without changing any
|
|
channel's latest metadata. Windows signing remains off by default. `core`
|
|
checks installation/startup; it does not exercise the full updater lifecycle.
|
|
If the branch's packaged version is behind the current stable/beta feed, add
|
|
`-f force=true` while retaining `-f publish=false` to permit that test build.
|
|
Web source, tests and generated route types are validated in an independent
|
|
same-commit job. Stable retains its existing verification gate; prerelease
|
|
retains its independently dispatched, advisory verification.
|
|
|
|
Compare build step times separately from queueing, dependency installation,
|
|
Apple notarization/retries, artifact availability, smoke, and cache transfer.
|
|
The Windows outer cache keeps at most the newest complete entry per build layer
|
|
within a 6 GiB payload budget. Final installers, portable ZIPs and unpacked trees
|
|
remain reusable: dropping them made same-source repeat builds slower in the
|
|
timing experiment. Historical copies are discarded. The cache snapshot's `sizes.json` reports
|
|
retained/removed bytes by node. Snapshot compression/upload still costs time
|
|
in the build job; saving the GitHub cache and pruning old outer caches run in
|
|
`cache-maintenance.yml`, outside release concurrency. Cache maintenance failure
|
|
does not change the build's artifact validity.
|
|
|
|
A changed outer cache prefix starts cold. Measure a second run at the same
|
|
commit to assess the warm-cache tradeoff, and compare equivalent runner images,
|
|
artifact targets and signing modes. A single run is observational evidence,
|
|
not a stable percentage saving. `publish=false` also skips PostHog sourcemap
|
|
upload, so its total duration is not directly comparable with a publishing run.
|