fix: split the memory pipeline's internal LLM prompts into system and user messages#938
Draft
gorkem2020 wants to merge 12 commits into
Draft
fix: split the memory pipeline's internal LLM prompts into system and user messages#938gorkem2020 wants to merge 12 commits into
gorkem2020 wants to merge 12 commits into
Conversation
gorkem2020
force-pushed
the
fix/prompt-architecture
branch
3 times, most recently
from
July 15, 2026 18:08
2b42580 to
e13f4b0
Compare
gorkem2020
added a commit
to gorkem2020/memory-lancedb-pro
that referenced
this pull request
Jul 15, 2026
Checkpoint recut of deploy/fleet-2026-07-15b (21 PRs) surfaced several merge-time regressions and stale test fixtures once the full test/tsc/build gate ran end to end: - index.ts: a plain (non-conflicted) auto-merge reintroduced CRLF into the modelRun:true block inside generateReflectionTextUnbounded; restored to LF. - index.ts: restored PR CortexReach#934's "regex fallback skipped" debug log + return in the below-threshold defer branch, dropped when PR CortexReach#951's merge took the persistence-based rollback wholesale instead of unioning both changes. - index.ts: removed a duplicate minMessages declaration left over from reconciling PR CortexReach#939's assistant-context accumulation block with PR CortexReach#951's own minMessages usage earlier in the same function. - src/admission-control.ts: hoisted the constructed-durable short-circuit from evaluateWithUtility() into evaluate() itself, so a constructed durable candidate is rejected before scoreUtility() spends an LLM call, not just before its score is blended in. - src/admission-control.ts: restored the Grounding/Conversation register fields in buildUtilityPrompt's user template (dropped during the PR CortexReach#938 split-prompt merge), since PR CortexReach#927's own admission tests depend on the judge seeing them inline. - Updated several PR CortexReach#927/CortexReach#938/CortexReach#939 test assertions that predate the split-prompt {system, user} return shape to destructure the right half instead of matching the whole object, and updated stale "## Recent Conversation" / "Evaluate whether this candidate" mock-routing strings to the current prompt text. - test/assistant-context-capture-counting-symmetry.test.mjs: gave each captureAssistant/payloadShape scenario its own directory so the new watermark sidecar file (keyed by dirname(dbPath) + sessionKey) doesn't bleed state between scenarios that previously only shared in-memory state; updated the delta-only sanity check to reflect the documented rollback characteristic where per-turn deltas below minMessages never accumulate. - test/smart-extractor-branches.mjs: updated a fixture comment/payload that had already predicted and documented this exact combination, switching to full accumulated history per turn as prescribed. Full suite (minus the environment-only Ollama port-conflict test), tsc, and build all green after these fixes.
gorkem2020
force-pushed
the
fix/prompt-architecture
branch
2 times, most recently
from
July 17, 2026 21:50
71765a9 to
45d012f
Compare
All four internal calls (extract-candidates, admission-utility,
dedup-decision, merge-memory) packed instructions, criteria, identity, and
the output-format contract into a single user-role message with no system
prompt. Restructure each builder to return {system, user}: static
instructions/criteria/format go in system, the per-call conversation
excerpt / candidate rows / neighbor rows go in user.
Give each stage an honest, distinct identity (extraction agent, dedup
decider, merge writer, admission judge) instead of every call introducing
itself with the same generic extraction-assistant wording.
Rebuild the merge prompt template cleanly: the old version rendered
broken bold markers ("** Category **:"), progressively indented list
items, and split-hyphen tokenization artifacts ("up - to - date").
Thread an optional sourceKind through the admission prompt builder so a
reflection-sourced excerpt is honestly framed as "Source document (agent
reflection)" rather than "Conversation excerpt" (default unchanged). The
actual reflection call site lives on a not-yet-merged branch and will
pass sourceKind: "reflection" when that branch is integrated.
LlmClient.completeJson() gains an optional systemPrompt third argument.
The api-key path threads it into the messages array's system role; the
OAuth path threads it into the Responses API's existing `instructions`
field (which already carried system-prompt semantics, so no prepend
fallback was needed). Both default to the historical generic system text
when omitted, so memory-upgrader.ts's unrelated call site is unaffected.
Update three existing test files whose mock LLM servers routed responses
by matching prompt-opener phrases in the user message; those phrases now
live in the system message, so the mocks combine both messages before
matching. No scoring weights, thresholds, or decision protocol changed.
…uations - buildDedupPrompt and buildMergePrompt: separate each Abstract/Overview/ Content field with a blank line so markdown embedded in Overview/Content doesn't visually collapse into the label line above it. - admission-control's buildUtilityPrompt: indent continuation lines of a multi-line Overview/Content so they stay nested under their bullet instead of landing flush-left mid-list. - smart-extractor's dedup candidate listing: extract formatExistingMemoryForDedupPrompt so the same continuation-line indentation applies to the numbered existing-memories list, and the formatting is directly unit-testable.
…ission few-shot example buildUtilityPrompt's Overview/Content bullets were glued to Category/ Abstract with a single newline; give them their own blank-line-separated block like the dedup/merge prompts already do. The system prompt had no few-shot example at all, so add one wrapped in explicit "--- EXAMPLE (not part of the live data) ---" / "--- END EXAMPLE ---" markers and a clearly labeled "Example response:" line, matching the fencing convention already established for the batch-utility prompt on a sibling branch. No batch variant of buildUtilityPrompt exists on this branch (or on master) to formatting-match against -- it lives on the separate, unmerged feat/admission-batch-utility branch, out of scope here.
buildReflectionPrompt returned one flat string with the distiller's
identity, heading contract, hard rules, section rules, governance format,
and output template all glued to the per-call tool-error-signal and
conversation payload. Split it into {system, user} like the other four
LLM call sites: system carries every static instruction (now opening
with an explicit "You are a memory reflection distiller." identity
line, matching the extraction/dedup/merge/admission stage-identity
convention), user carries only the per-call tool error signals and the
fenced conversation INPUT block.
generateReflectionTextUnbounded rejoins system + "\n\n" + user before
handing the result to the embedded-agent runner and the CLI fallback,
since neither transport (runEmbeddedPiAgent's params bag, the CLI's
--message arg) has a distinct system-prompt field to split into -- same
mechanical-merge posture already used on the sibling admission-batch-
utility branch for its own not-yet-split call site. A test asserts the
rejoined text is byte-identical to the prior single-string prompt, so
this is a pure prompt-architecture/formatting change with no behavior
delta: same headings, same rules, same output-format contract.
index.ts carries this repo's known mixed CRLF/LF encoding; both the
function body and its call site were rewritten with a latin1 Buffer
splice (never Edit) to avoid normalizing the file, per this fork's
CLAUDE.md. Diff is 10 lines.
Durable/recurring state and habit changes (e.g. "switched my commute to the M4", "Spanish lesson before breakfast") were getting classified as "events" by the extraction LLM and then hugging the admission threshold instead of persisting as the user's new ongoing state. Add a compact rubric bullet to the extraction prompt's Common Confusion section: recurring/durable state and habit changes route to preferences or patterns; events stays reserved for genuinely one-off occurrences.
Wire test/extraction-category-rubric.test.mjs into package.json's local test chain and scripts/ci-test-manifest.mjs's llm-clients-and-auth group, alongside its sibling prompt-architecture.test.mjs.
gorkem2020
force-pushed
the
fix/prompt-architecture
branch
from
July 18, 2026 15:47
45d012f to
035d6a2
Compare
…tities, markdown payload shape, raw-JSON contracts
Fleet-proven prompt-architecture foundation, ported from the deployed
build:
- src/prompt-blocks.ts single-sources the shared static prompt content:
agent identities (every prompt opens 'You are a memory ...'), the
six-category taxonomy line, the scoring rubric, bare JSON shape
rendering (jsonShape - examples are shown exactly as the model must
emit them, unfenced; fenced examples taught fence-mimicking models to
fence their output), and the markdown candidate/memory formatters.
- Candidate and existing-memory payloads render as markdown blocks
('## Candidates' / '### 1. <category>') through one shared formatter.
- Every output contract reads 'Return JSON only (the raw object, no
markdown code fences)' and shows the shape bare.
- The admission judge is transcript-free: it scores the candidate's own
content plus store-derived features, never a conversation excerpt.
- The dedup decider carries the anti-category-wall doctrine (category
labels never decide a verdict by themselves) and the extraction
transcript rides a fenced block like the distiller's INPUT.
- Distiller opener names the domain and forbids fenced output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The memory pipeline makes four internal LLM calls: extraction, admission scoring, dedup decision, and merge. All four packed instructions, criteria, output-format contract, and the per-call data into a single user-role message, with no system prompt beyond a generic one-liner shared by every call ("You are a memory extraction assistant..."). This meant the admission judge, the dedup decider, and the merge writer all introduced themselves with extraction-assistant wording, and every call mixed static instructions in with the untrusted conversation excerpt in the same message.
What changed
Each of the four prompt builders (
buildExtractionPrompt,buildDedupPrompt,buildMergePromptinsrc/extraction-prompts.ts, andbuildUtilityPromptinsrc/admission-control.ts) now returns{system, user}: static instructions, criteria, identity, and the output-format contract live insystem; the per-call conversation excerpt, candidate rows, or neighbor rows live inuser.Each stage gets an honest, distinct identity instead of sharing generic wording:
LlmClient.completeJson()gains an optional third argument,systemPrompt. The api-key path threads it into themessagesarray's system role. The OAuth path threads it into the Responses API's existinginstructionsfield; that field already carried system-prompt semantics (it previously held the same generic string), so no degrade-to-prepend fallback was needed; the request shape stays exactly as it was, just with a per-call value instead of one fixed string. Both paths default to the historical generic text whensystemPromptis omitted, so the one caller outside this pipeline (memory-upgrader.ts's one-off migration prompt) is unaffected.The merge prompt template had a live-observed formatting bug: broken bold markers rendered as
** Category **:, progressively indented list items, and a tokenization artifact ("up - to - date" from a space-hyphen-space pattern in the source). Rebuilt the template cleanly; added a snapshot-style test asserting the assembled prompt contains none of those artifacts.Reflection-sourced admission gating previously reused the extraction path's "Conversation excerpt:" framing even when the excerpt was actually a reflection document.
buildUtilityPromptandAdmissionController.evaluate()now take an optionalsourceKind: "conversation" | "reflection"(default"conversation"), and the admission judge sees "Source document (agent reflection):" for reflection-sourced rows instead. The admission judge only ever scores the excerpt as given; it never rewrites candidates.No scoring weights, thresholds, or the decision protocol changed. Same decisions in and out; better-formed prompts.
Test plan
LlmClienttests confirming the api-key path's system message and the OAuth path'sinstructionsfield both honor an explicitsystemPromptand fall back to the historical default when omitted.tscbuild green.