5b6fbb55ea
* feat(analytics): define client metrics schema and unify telemetry metadata * feat(desktop): measure process startup phases through business readiness Add launch_attempt, business_ready, and launch_result timing with monotonic phase offsets, per-attempt correlation, shared app session identity, explicit process/activation measurement boundaries, and idempotent terminal results. Preserve packaged sidecar phases across the desktop handoff and report bounded startup failures without blocking successful startup on telemetry. Window mount/reveal alone does not declare business Ready. Preload/web owners must wire od:startup:context and od:startup:business-ready with matching launch_attempt_id/app_session_id plus required_data_ready/actions_ready=true. Align with the shared event registry/schema and envelope fields (device_id, event_id, app_session_id, client_type, capture_source, env, channel, app_version, os_version, arch, occurred_at, reported_at, capability), plus startup_type, launch_attempt_id, started_at, duration_ms, outcome, reason, failure_stage, stage_offsets_ms, stage_durations_ms, measurement_start, start_time_source, and os_launch_time_available. The existing daemon safety bridge overwrites some envelope fields; shared-envelope alignment stays with that owner. Coverage limits: no OS icon-click timestamp or separate launcher-process span; no new packaged launch report without real installation identity; failures before packaged runtime context is armed and forced process exits can be unobserved. Cold means a fresh desktop process; warm means hidden-window restore; reopen means desktop restored from a headless owner. Measurement acceptance and preload ownership need maintainer confirmation; native QA/publication remain. Validation (local Node 24 / pnpm 10.33.2): - Red specs captured before implementation. - desktop focused tests: 7 files, 47 passed. - packaged focused tests: 6 files, 87 passed. - pnpm --filter @open-design/desktop typecheck: passed. - pnpm --filter @open-design/packaged typecheck: passed. - pnpm guard: passed. - pnpm typecheck: passed after building missing local workspace artifacts. - git diff --cached --check: passed. Local commit only. No web/memory-sampling changes or external operations. * feat(web): measure project navigation intent to actionable ready * feat(observability): add bounded periodic process memory sampling * feat(telemetry): integrate launch readiness and bounded process memory metrics Unify metric envelopes, classify process launches with a persisted heuristic, wire preload and renderer readiness, cancel abandoned navigation, and bind consent-aware memory sampling to the Electron lifecycle. Document the metric definitions and stabilize validation fixtures that inherited host configuration or raced signal-handler readiness. * docs(telemetry): translate stability drafts to English Rewrite the historical project-entry draft in English and neutralize the stability-metrics intro so odcrew-facing docs stay English-only. * fix(telemetry): distinguish Linux Node installs from packaged runtimes * fix(telemetry): wait for committed conversation navigation before readiness * fix(telemetry): exclude deferred headless restores from launch timing * fix(telemetry): require current Home project and agent reads to succeed * fix(telemetry): fail closed on damaged memory sampling consent * docs(telemetry): note catalog read failures cannot block Home readiness * feat(telemetry): prepare update stage and desktop restart observations * feat(telemetry): prepare responsiveness collectors pending transport integration * fix(desktop): cancel consent lookup and anchor memory sampling to ready * fix(telemetry): classify and correlate crash and exit reports * feat(telemetry): measure consented visible preview attempts * feat(telemetry): wire page switch intent and readiness timing * fix(telemetry): validate responsiveness metrics and fence revoked buffers * fix(telemetry): require successful scoped Home resources before readiness * fix(telemetry): clear chat health collection on consent withdrawal * docs(telemetry): record integrated responsiveness validation and budgets * fix(telemetry): admit update lifecycle stage metrics * test(contracts): restore merged client metric schema cases * fix(telemetry): route native stability metrics through safety capture Remove metrics-consent checks from startup, memory, exit and update-stage reporting while preserving product analytics consent and privacy projection. * docs(telemetry): align stability metrics with safety-channel policy Document consent-independent stability reporting and unchanged product/content consent, privacy projection, budgets and remaining native QA. * fix(telemetry): observe runtime daemon exits and correlate memory samples Capture parent-observed daemon death through the safety channel, preserve stronger evidence across restart, and join exits to real packaged memory sampling with opaque process IDs and nullable V8 heap limits. * fix(telemetry): send web stability metrics on the safety channel Drop the metrics-consent gate from web responsiveness, preview and chat health collectors to match the native safety-channel routing, and remove test hooks for the deleted consent API. * test(web): record an open intent before asserting a trusted first paint * test(web): align component tests with commit-driven navigation and visible-preview gating ProjectView switches conversations in the router onCommit callback, preview diagnostics require a visible frame, and catalog/page-switch reads changed their call shape; update mocks and expectations accordingly. * fix(telemetry): require successful catalogs before reporting page navigation ready A failed design-system/skill/template read cleared loading without marking the catalog ready, so page switches still reported ready. Mirror the startup gate. * fix(desktop): prune expired stage-only update check flows Every scheduled check opens a metric flow directory. Remove stage-only flows past the replay TTL during the existing startup scan; installer flows stay. * fix(telemetry): isolate reporting from navigation and catalog fallback * fix(telemetry): preserve business behavior with passive observation * fix(telemetry): classify daemon channel by service packaged flag On Linux CI the Node binary lives under /opt/, so execPath heuristics marked unpackaged runs as packaged and reported channel=stable. Without an explicit or version-inferred channel, derive it from the service's packagedRuntime flag instead. Also restore the desktop SHOW-hook boundary assertion to match activeDesktop.show(). * fix(telemetry): keep killed daemons and fresh-install launch attempts observable OPEND-4221: the sidecar supervisor exited 0 whenever its target died from a signal, so an external SIGKILL of the daemon reached packaged as code=0/signal=null and was reported as clean-exit. The supervisor now exits 0 only for stops it forwarded itself and encodes any other terminating signal as 128 + N; observeDaemonExit decodes that back to reason=killed with the signal. OPEND-4220: packaged flushed client_launch_attempt directly at daemon-ready and dropped it when installation.json did not exist yet, which is always the case on a fresh install. Launch events now stay buffered until desktop main arms the daemon-backed reporter, and the daemon retains launch metrics (bounded, in order) until the installation identity exists, replaying them before later events. Explicit opt-out still delivers immediately on the safety channel. * fix(telemetry): exclude url_load previews from the preview denominator trackVisiblePreview probes the frame with od:preview-metric-probe and, after 15s without od:preview-metric-result, finishes the attempt as error / probe_unverified. URL-loaded artifacts never receive the srcdoc observability bridge, so they cannot answer; every visible, successful URL preview was therefore counted as an error. Frames that cannot answer the probe are now excluded from the denominator, matching how the PR scopes Live Artifact HTML. * fix(telemetry): move renderer safety identity to the installation id on fresh install OPEND-4220 retest: on a fresh install the renderer safety channel bootstraps before the installation id is minted, falls back to the anonymous id, and kept it for the whole session, so renderer stability events carried a different device_id than native launch/memory events. setIdentity now also patches the safety context, and a late bootstrap keeps the installation id. --------- Co-authored-by: heggria <bshengtao@gmail.com>
11 KiB
Update stage telemetry (OPEND-3719)
Delivery status
This candidate adds stage journals and next-launch attribution.
client_update_stage_result is registered additively in CLIENT_METRIC_SCHEMAS
in packages/contracts/src/analytics/client-metrics.ts, so the common parser
admits validated stage records and strips undeclared fields before submission.
The contract admission and front-stage replay regressions cover this integration.
No release, real installation, remote ingestion, QA acceptance or seven-day
outcome is established by these local changes.
Contract and compatibility
The new client_update_stage_result metric uses the common client telemetry
builder and parser: event schema 4, metric schema 1 (a new event, not a revision
of existing event meanings). update_install_result, update_lifecycle_observed
and update_apply_observed retain their meanings and formats. Installer apply
summaries retain schema 1 and their existing flow IDs. New stage files are
independent of those summaries, so failures before installation can be replayed.
A new check allocates an opaque flow ID before fetching metadata. The subsequent download, validation, install handoff and next-launch result share it. Periodic checks have their own check denominator; when they rediscover the same available or downloaded release, the original release flow is restored for installation. Explicit retries after a failed stage allocate a new flow. Starting installation from a restored cache in a new process can have no corresponding check/download records: those stages are unknown for that flow, not invented successes.
Each stage has separate immutable started and terminal result records.
Terminal values are success, failed, cancelled, unknown, and (restart
only) not_applied. Cancellation is emitted only for an actual AbortError;
there is no new user cancellation capability. Missing terminal records remain
unknown at the reporting cutoff; killing a process cannot guarantee a final event.
occurred_at is the observation time, preserved in replay. reported_at is the
receiver's report time. Durations are null: this work does not claim pure hash,
installer execution, or restart latency measurements. Unknown versions, session
identity and OS capabilities are null. Installation identity is not a user ID.
| Stage | Attempt boundary | Result boundary |
|---|---|---|
| check | Supported check requested | Metadata/candidate processing completes or fails; no available update is a successful check |
| download | Candidate transfer/preparation requested | Verified artifact is prepared/promoted, or the download/preparation operation fails |
| verify | Checksum resolution and managed verified-download transaction requested | Managed checksum plus final updater hash passes; checksum mismatch fails; incomplete transfer gives unknown |
| install_verify | Existing artifact rehash requested before opening/activation | Rehash passes, bytes are unavailable, or checksum differs |
| install_open | Validated artifact activation/helper launch requested | Installer helper armed or payload activation/relaunch helper armed, or synchronous failure |
| restart | Successful install handoff recorded | A later desktop session reaches its main-window-ready boundary and observes the running version |
The managed-download library combines transfer and verification. Consequently
verify.started is the start of the verification pipeline, which includes
prerequisite transfer, not the instant hashing begins. A network error is
download.failed and verify.unknown/download_incomplete; a checksum mismatch
is verify.failed/checksum_mismatch and a failed verified-download operation.
Those two stage counts overlap and must not be added as independent incidents.
Post-download preparation failures can fail download while verification succeeds.
Pre-install checksum errors fail install_verify; they do not fabricate an
install_open attempt. Reason codes are a closed allowlist, never raw errors.
Denominators and deduplication
For a stage S, the denominator is distinct (flow_id, S) with a valid started
record in the eligible version/channel/path/trigger cohort. The observed failure
fraction is terminal failed joined to those starts divided by that denominator.
Report success, cancellation, unknown and unresolved counts beside it. This is
an attempt metric, not a user rate. A zero or absent denominator is not
calculable; do not display zero percent. Orphan results have no denominator.
Use a fixed reporting cutoff and allow for late next-launch/replay delivery. Missing results, disk failures, old clients and clients that never restart are unknown or absent, not successes. Old clients with only legacy observations have unknown stage capability. Their legacy version-match results must not be counted in the new complete-attempt cohort.
Restart success requires the target version to be running in the matching
channel/namespace. The previous version still running is not_applied, never
installation failure: the installer may have been cancelled or never completed.
Another version/identity is unknown. Same-version reinstall is unknown because
version equality cannot prove replacement. Installer execution failures after a
helper has armed remain unobservable; a synchronous open/activation error is a
separate install_open.failed. The new result requires a different desktop session ID at main-window readiness.
A daemon-only restart, repeated window-ready callback or periodic scan cannot
close the new restart metric. Legacy apply classification remains compatible.
Local exclusive file creation retains only the first record for each flow,
stage and phase. Receipts suppress repeated scans after local queue acceptance;
retries use the same insert ID update_stage:<flow>:<stage>:<phase>. A crash
between enqueue and receipt can resubmit that ID, requiring remote insert-ID
deduplication. Local enqueue, legacy submitted, and HTTP 200 do not prove
remote durable storage. Receipts are not delivery guarantees and do not imply
retry after the in-memory analytics queue loses an accepted event.
Consent, privacy and cost
Unsupported configurations and dry runs do not create new metric records. Requests rejected by restart-safety preflight before reaching the updater are outside these stage denominators; they remain a coverage residual.
稳定性埋点沿用安全通道，不受匿名指标开关控制（telemetry.metrics）。
New stage records do not read consent. The desktop injects the existing startup
session ID and product OS version; neither is guessed. The daemon replays
client_update_stage_result through captureSafety, including after opt-out or
unavailable consent. Old consent/identity-skip receipts are reconsidered with the
original insert ID; accepted safety receipts still deduplicate.
Operational legacy product observations retain their existing
consent policy. No paths, URLs, prompts, contents or raw PIDs are transmitted.
The local namespace routes the journal and is stripped from the new metric.
The updater never awaits the new telemetry queue. It allows at most 32 pending writes and at most 12 phase files per flow. The existing ten-second daemon timer replays at most 64 flow directories per tick for the new metric, rotating across the inventory (at most 768 fixed-name file reads). The incoming desktop scanner runs asynchronously once at main-window readiness, reads one restart-start record per stored flow and records at most 64 outcomes. Observation eligibility lasts seven days. Old operational journals and the existing legacy scanner retain their previous storage/scanning policy; this change does not claim bounded total directory size or implement journal garbage collection. Native CPU/IO measurements and retention policy remain follow-up work.
Local coverage and remaining acceptance
Local regressions cover common-schema admission and privacy projection, failed
enqueue retry with stable insert IDs, safety replay after metrics opt-out, and
private-field stripping. Cancellation fixtures abort metadata fetching or
checksum resolution; they do not claim a user-facing transfer-cancel control.
The new contract admission regression fails on aa88f4387 and on the previous
candidate before registration, and passes with registration in place.
| Path / trigger | Local evidence | Remaining |
|---|---|---|
| Installer / manual | Check, transfer, checksum, open success/failure and shared flow fixtures | Real DMG/NSIS installation and post-install process/version evidence |
| Installer / silent | Configured auto-open fixture; no default automatic installer policy is introduced | Validate supported release configuration; installer UI still requires OS/user interaction |
| Payload / manual | macOS prepared-payload activation fixture with shared stage/legacy flow | Native macOS and Windows full launcher lifecycle |
| Payload / silent | Same payload fixture with silent attribution; scheduler verifies silent arguments | Real startup preference, shutdown, exact executable identity, rollback and installed-outer cold start |
| Both paths / both triggers | Next-launch success, unchanged-version and unrelated-version fixtures | Actual restart and telemetry ingestion on supported release artifacts |
QA should use the repository's updater lifecycle harness and an isolated local
namespace; follow the authoritative daemon-data contract in AGENTS.md rather
than inventing data locations. For each applicable path/trigger:
- Start an older supported package. Force metadata offline:
one check start/result must share a flow and report
check_failed. - Retry with metadata reachable and artifact transfer disconnected: check succeeds, download fails and verification is unknown. Retry is a different flow.
- Serve mismatched checksum bytes: verify fails with
checksum_mismatch; no install-open attempt appears. Also tamper with the downloaded file before install:install_verifyfails and the installer is not launched. - Make helper opening fail: install-open has one start/failed pair. It must not create restart success or relabel the error as a version-unchanged outcome.
- Apply successfully and restart: verify target running version and original flow correlation. Repeat scans/restart and verify stable insert IDs/receipts.
- Reopen the old version without completing installation: restart is
not_applied. Same-version reinstall and legacy journals must not imply new-stage success. - Opt out, corrupt/unreadably configure consent, then repeat: new safety metrics are still journaled/sent with the same schema and privacy projection. Product observations must continue to honor metrics consent.
- Inspect remote persisted rows using an authorized read-only workflow; explicitly distinguish those rows from local capture mocks, queue receipts and HTTP responses.
No QA, live ingestion or seven-day observation is performed by this job. Owner: QA/release owner for native execution and rollout, telemetry owner for persisted rows and cohort queries; assignments and dates require confirmation. For a release on 2026-10-22, the earliest complete seven-day observation is 2026-10-29. None of these observations establishes a performance improvement or full ticket acceptance.