- chatExtractionEnabled now defaults OFF (explicit opt-in only); it gates
the regex pack, chat-form profile capture, and LLM extractor together
while leaving memory injection untouched
- add a one-time boot migration (memory-cleanup.ts) that deletes regex-pack
artifacts by name+template fingerprint, prunes user_profile to the
canonical field set the settings editor writes, and forces the legacy
config flag off once; marker-gated so a conscious re-opt-in sticks
- persist writer provenance (frontmatter source:) on every new entry so
future audits can tell auto-extracted facts from hand-written ones
- LLM-extracted entries are deliberately kept: on-disk they are
indistinguishable from hand-written prose, and they are the least
harmful of the retired pipelines' output
Co-authored-by: NJUHua <113895241+NJUHua@users.noreply.github.com>
* fix(daemon): stop the memory extractor re-mining turns and mining raw transport
The background memory extractor is fired from a run's child-close hook, so a
turn that is retried or re-ground during a long autonomous build re-enters
extraction with the same user message + reply. During an out-of-credits build
one turn was re-analyzed ~33 times, each paying for a full provider call to
re-confirm "nothing new" (~3.4M tokens wasted over Jul 2-4). Separately, the
"## Assistant reply" the extractor mined was built from the child's raw stdout
(JSONL transport: system:init, stream_event, hook_started/hook_response) rather
than the model's rendered reply, so every extraction mined the wrong text.
- Add a duplicate-turn gate keyed by (conversationId, user message, reply) that
early-returns before any provider call when a turn was already extracted. A
failed / out-of-credits attempt yields an empty reply, so the whole re-fire
storm shares one signature and collapses to a single pass. The signature is
recorded only AFTER a successful provider call, so a failed attempt never
permanently skips the turn, and it is scoped per conversation so an identical
turn in another conversation is still examined. Not gated on retry count: a
turn that only succeeds on retry produces its reply on that attempt and must
still be mined.
- Capture the rendered reply through the send() choke point from both the
structured-agent text_delta channel and the plain/BYOK stdout channel, so
every agent family feeds the extractor real reply text and never raw
transport frames.
Adds apps/daemon/tests/memory-llm-dedupe.test.ts covering the dedup gate,
record-after-success, per-conversation scope, and reply serialization.
* fix(daemon): scope the memory duplicate-turn gate to a real conversation id
Review found that the `''` fallback conversation key made the new de-dup gate
global for any extractWithLLM() caller that doesn't thread a conversationId.
The BYOK/API-mode `/api/memory/extract` post-turn path calls extractWithLLM()
without one, so two unrelated conversations sharing the same (userMessage,
assistantMessage) pair would collide there and the second extraction would be
wrongly skipped.
Gate the de-dup on a real (non-empty) conversation id: id-less callers now get
no de-dup at all, which is the safe pre-existing behavior for the explicit HTTP
extraction path. The chat close hook that caused the re-fire storm always passes
run.conversationId, so its protection is preserved.
Adds a regression test for the id-less HTTP path: two identical turns with no
conversationId must both reach the provider.
