External/open-design

Fork 0

mirror of https://github.com/nexu-io/open-design.git synced 2026-10-27 09:51:12 +01:00

Files

T

5b6fbb55ea [OPEND-3712] feat(telemetry): stability metrics for launch, project entry, and memory (#8805 )

* feat(analytics): define client metrics schema and unify telemetry metadata

* feat(desktop): measure process startup phases through business readiness

Add launch_attempt, business_ready, and launch_result timing with monotonic
phase offsets, per-attempt correlation, shared app session identity, explicit
process/activation measurement boundaries, and idempotent terminal results.
Preserve packaged sidecar phases across the desktop handoff and report bounded
startup failures without blocking successful startup on telemetry.

Window mount/reveal alone does not declare business Ready. Preload/web owners
must wire od:startup:context and od:startup:business-ready with matching
launch_attempt_id/app_session_id plus required_data_ready/actions_ready=true.

Align with the shared event registry/schema and envelope fields (device_id,
event_id, app_session_id, client_type, capture_source, env, channel,
app_version, os_version, arch, occurred_at, reported_at, capability), plus
startup_type, launch_attempt_id, started_at, duration_ms, outcome, reason,
failure_stage, stage_offsets_ms, stage_durations_ms, measurement_start,
start_time_source, and os_launch_time_available. The existing daemon safety
bridge overwrites some envelope fields; shared-envelope alignment stays with
that owner.

Coverage limits: no OS icon-click timestamp or separate launcher-process span;
no new packaged launch report without real installation identity; failures
before packaged runtime context is armed and forced process exits can be
unobserved. Cold means a fresh desktop process; warm means hidden-window
restore; reopen means desktop restored from a headless owner. Measurement
acceptance and preload ownership need maintainer confirmation; native
QA/publication remain.

Validation (local Node 24 / pnpm 10.33.2):
- Red specs captured before implementation.
- desktop focused tests: 7 files, 47 passed.
- packaged focused tests: 6 files, 87 passed.
- pnpm --filter @open-design/desktop typecheck: passed.
- pnpm --filter @open-design/packaged typecheck: passed.
- pnpm guard: passed.
- pnpm typecheck: passed after building missing local workspace artifacts.
- git diff --cached --check: passed.

Local commit only. No web/memory-sampling changes or external operations.

* feat(web): measure project navigation intent to actionable ready

* feat(observability): add bounded periodic process memory sampling

* feat(telemetry): integrate launch readiness and bounded process memory metrics

Unify metric envelopes, classify process launches with a persisted heuristic, wire preload and renderer readiness, cancel abandoned navigation, and bind consent-aware memory sampling to the Electron lifecycle. Document the metric definitions and stabilize validation fixtures that inherited host configuration or raced signal-handler readiness.

* docs(telemetry): translate stability drafts to English

Rewrite the historical project-entry draft in English and neutralize the
stability-metrics intro so odcrew-facing docs stay English-only.

* fix(telemetry): distinguish Linux Node installs from packaged runtimes

* fix(telemetry): wait for committed conversation navigation before readiness

* fix(telemetry): exclude deferred headless restores from launch timing

* fix(telemetry): require current Home project and agent reads to succeed

* fix(telemetry): fail closed on damaged memory sampling consent

* docs(telemetry): note catalog read failures cannot block Home readiness

* feat(telemetry): prepare update stage and desktop restart observations

* feat(telemetry): prepare responsiveness collectors pending transport integration

* fix(desktop): cancel consent lookup and anchor memory sampling to ready

* fix(telemetry): classify and correlate crash and exit reports

* feat(telemetry): measure consented visible preview attempts

* feat(telemetry): wire page switch intent and readiness timing

* fix(telemetry): validate responsiveness metrics and fence revoked buffers

* fix(telemetry): require successful scoped Home resources before readiness

* fix(telemetry): clear chat health collection on consent withdrawal

* docs(telemetry): record integrated responsiveness validation and budgets

* fix(telemetry): admit update lifecycle stage metrics

* test(contracts): restore merged client metric schema cases

* fix(telemetry): route native stability metrics through safety capture

Remove metrics-consent checks from startup, memory, exit and update-stage
reporting while preserving product analytics consent and privacy projection.

* docs(telemetry): align stability metrics with safety-channel policy

Document consent-independent stability reporting and unchanged product/content
consent, privacy projection, budgets and remaining native QA.

* fix(telemetry): observe runtime daemon exits and correlate memory samples

Capture parent-observed daemon death through the safety channel, preserve stronger evidence across restart, and join exits to real packaged memory sampling with opaque process IDs and nullable V8 heap limits.

* fix(telemetry): send web stability metrics on the safety channel

Drop the metrics-consent gate from web responsiveness, preview and chat
health collectors to match the native safety-channel routing, and remove
test hooks for the deleted consent API.

* test(web): record an open intent before asserting a trusted first paint

* test(web): align component tests with commit-driven navigation and visible-preview gating

ProjectView switches conversations in the router onCommit callback, preview
diagnostics require a visible frame, and catalog/page-switch reads changed
their call shape; update mocks and expectations accordingly.

* fix(telemetry): require successful catalogs before reporting page navigation ready

A failed design-system/skill/template read cleared loading without marking the
catalog ready, so page switches still reported ready. Mirror the startup gate.

* fix(desktop): prune expired stage-only update check flows

Every scheduled check opens a metric flow directory. Remove stage-only flows
past the replay TTL during the existing startup scan; installer flows stay.

* fix(telemetry): isolate reporting from navigation and catalog fallback

* fix(telemetry): preserve business behavior with passive observation

* fix(telemetry): classify daemon channel by service packaged flag

On Linux CI the Node binary lives under /opt/, so execPath heuristics
marked unpackaged runs as packaged and reported channel=stable. Without
an explicit or version-inferred channel, derive it from the service's
packagedRuntime flag instead. Also restore the desktop SHOW-hook boundary
assertion to match activeDesktop.show().

* fix(telemetry): keep killed daemons and fresh-install launch attempts observable

OPEND-4221: the sidecar supervisor exited 0 whenever its target died from a
signal, so an external SIGKILL of the daemon reached packaged as
code=0/signal=null and was reported as clean-exit. The supervisor now exits 0
only for stops it forwarded itself and encodes any other terminating signal as
128 + N; observeDaemonExit decodes that back to reason=killed with the signal.

OPEND-4220: packaged flushed client_launch_attempt directly at daemon-ready and
dropped it when installation.json did not exist yet, which is always the case
on a fresh install. Launch events now stay buffered until desktop main arms the
daemon-backed reporter, and the daemon retains launch metrics (bounded, in
order) until the installation identity exists, replaying them before later
events. Explicit opt-out still delivers immediately on the safety channel.

* fix(telemetry): exclude url_load previews from the preview denominator

trackVisiblePreview probes the frame with od:preview-metric-probe and, after
15s without od:preview-metric-result, finishes the attempt as error /
probe_unverified. URL-loaded artifacts never receive the srcdoc observability
bridge, so they cannot answer; every visible, successful URL preview was
therefore counted as an error. Frames that cannot answer the probe are now
excluded from the denominator, matching how the PR scopes Live Artifact HTML.

* fix(telemetry): move renderer safety identity to the installation id on fresh install

OPEND-4220 retest: on a fresh install the renderer safety channel bootstraps
before the installation id is minted, falls back to the anonymous id, and kept
it for the whole session, so renderer stability events carried a different
device_id than native launch/memory events. setIdentity now also patches the
safety context, and a late bootstrap keeps the installation id.

---------

Co-authored-by: heggria <bshengtao@gmail.com>

2026-10-22 05:55:52 +00:00

5.4 KiB

Raw Permalink Blame History

Historical draft. The integrated definitions, wiring, and current limitations are in stability-metrics.md; that document supersedes this draft.

Project entry → Ready (historical draft, not reviewed / not published)

This file records a candidate measurement definition for this branch. It is not a public contract, QA gate, or production acceptance criteria.

Measurement boundary and Ready

Path: shared project open handler → workspace/project reads before navigation → route commit → ProjectView business data and authority ready → next foreground animation frame while still actionable → Ready.
Start: synchronous enter of the project-open handler, before the first await, using performance.now() — not component mount and not URL update.
Ready: project detail and scope loaded; conversation and current-authority messages loaded successfully; messages belong to the current conversation; existing send / new-conversation gates allow action. Background tabs do not report Ready; re-check after returning to the foreground. The animation frame is a post-commit actionable check; it does not claim pixels painted or iframe preview completion.
Explicit errors (open, detail/scope, conversation/message reads) report failed. Read-only, materialization waits, and billing-identity-not-ready states must not be reported as Ready. There is no fixed timeout; unfinished started attempts must be counted as missing measurements, not dropped or treated as success.
Read-only access does not meet this “actionable” definition and is excluded from Ready quantiles unless a separate readable-Ready group is approved later; do not fill with 0 ms.
Cancel: a newer click supersedes an older one (including the same project), leaving the source page / loading project, or page exit. Late requests, late frames, and duplicate effects must not emit Ready for a stale attempt.
Same-project conversation selection is tagged conversation_switch. Programmatic conversation restore must not invent a user choice. Ordinary file-tab URL sync does not create a project-entry attempt. First direct load / refresh has no proven click start and does not emit project entry; launch Ready is defined separately in the launch metrics.

Local candidate event (needs shared schema alignment)

Event client_navigation_result: one started and at most one terminal ready / cancelled / failed, sharing navigation_id. duration_ms uses the same monotonic origin; started is 0.

Field	Local meaning
navigation_id	New per real intent; terminal association / dedupe key
navigation_type	`project_entry` / `conversation_switch` (query separately)
source_type / target_type	Fixed page kinds (home subpages fold into source_type); target project / conversation
status	started / ready / cancelled / failed
duration_ms	Intent to this status, ms; never treat cancelled as a fast success
reason	Fixed reason enum; no exception text / paths / prompts

Target project/conversation IDs are process-local matching only; do not add new identity fields to the report. Shared identity, version, env, schema, sampling, and launch attempt linkage come from the shared telemetry envelope work — this branch does not hand-copy those public fields.

Statistics sketch

Unit: observable navigation attempts; dedupe (navigation_id, status); at most one terminal per id. Keep project entry and conversation switch separate.
Success latency: only terminal ready duration_ms (P50/P90 and success N); also report distinct started N, failed N, cancelled N, unterminated N, and coverage.
Success-rate denominator should list all started; optionally also report success excluding explicit user cancel, always with cancel rate. Bucket by started time across windows and leave a settle window so terminal-only queries do not drop hanging attempts.
Account / install / version / platform / channel splits depend on shared public fields; do not invent SQL or claim live values before alignment.
Live values, published builds, and warehouse evidence: not measured here. Local mock unit tests do not prove production wiring.

Validation and remaining integration

Write navigation-timing.test.ts (module-missing red) before implementation. Focused wiring tests cover cancel, navigate-away, re-entry, late async completion, message load/fail, materialization block, conversation select, background→foreground, StrictMode replay, and Ready dedupe. Full web/daemon suites and publication are out of scope for this draft.

Needs separate owners for: shared event union / field names / schema / env identity / dedupe; launch attempt correlation and independent launch business Ready; router guard cancel callback when a navigation is rejected or superseded (unterminated started must not count as success until wired). Ordinary page switches, first create/import/duplicate, and external deep links that do not use this click entry are not claimed as covered.

Open product questions (for maintainer review, not assumed decided): whether strict actionable Ready is the final business definition; whether read-only or intentionally restricted “running but cannot create” states get a separate group or a readable-Ready metric. Packaged macOS/Windows QA on slow networks, failures, and cancels, plus post-publish warehouse evidence, remain future work.

5.4 KiB Raw Permalink Blame History