feat(memory): long-term memory as a personal wiki that lights up the knowledge base - #55
Draft
lyingbug wants to merge 39 commits into
Draft
feat(memory): long-term memory as a personal wiki that lights up the knowledge base#55lyingbug wants to merge 39 commits into
lyingbug wants to merge 39 commits into
Conversation
Plans a per-user long-term memory subsystem that reuses the wiki page / folder / link-graph / revision machinery, and introduces memory anchors as a bipartite bridge between a user's memory pages and the knowledge base wiki so the KB graph can be rendered "lit up" per user. Covers the three-layer memory model (episodic notes / semantic pages / graph), principal-scoped memory spaces, the read path (no extra LLM calls) and the debounced async write path, illumination + coverage metrics, the reverse loop that feeds anchor signals back into wiki issues, API and frontend surface, four-level config gating, privacy and memory-borne prompt-injection defenses, cost budgets, phased delivery, evaluation thresholds and open questions. Also records why the previous Neo4j conversation memory (removed in 380e371) failed and how each failure mode is avoided. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…-memory-wiki-e57e Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…pling Rebased the plan onto v0.7.2 (18739bb) and reworked it around two hard constraints the design must now satisfy. Zero Neo4j: drops the "Neo4j as an optional accelerator" escape hatch and the open question about Cypher-based multi-hop recall. Graph relationships are expressed with relational tables plus JSON link arrays in the main database, and all graph math runs in Go. The glossary keeps the Neo4j document entity graph only as a boundary contrast, with an explicit note that the memory subsystem neither reads nor writes it. Lite as a first-class target: adds a runtime-form contract section. The capability test is DB_DRIVER=sqlite plus an empty REDIS_ADDR, not handler.Edition, and six binding constraints follow from it -- dual migrations with a portable type mapping, no dialect-specific SQL, heat and coverage computed in Go rather than SQL, async work routed through interfaces.TaskEnqueuer and registered in both the asynq and SyncTaskExecutor paths, Redis-optional locking with a per-space in-process fallback, and short transactions to respect SQLite's single writer. Also records a pre-existing gap: task_pending_ops and task_dead_letters are missing from the sqlite migrations, which leaves Lite wiki ingest incomplete; fixing it becomes a Phase 0 prerequisite. Reworks configuration into a per-feature settings design: ~35 settings across eight groups, five override levels with per-category merge rules, safety settings that can only be tightened, effective values that report their source and lock state, and explicit capability declarations so unavailable features grey out instead of silently failing. Deployment form now only shifts defaults and capabilities, never code paths -- notably, Lite is not assumed to run small models, since it can use remote models just like standard. Refreshes six assumptions against upstream: query_understand no longer yields a rewrite on unparsable output, SearchResult carries ContentRevision / ContentRewritten for merge trust, wiki stats exclude archived pages, ingest prompts resolve language via ResolveLanguageName, memory migrations move to 000081-000083, and knowledge auto-tagging is noted as a sibling async pattern. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Introduces the storage layer, the layered settings framework and the core services for per-user long-term memory. Data model: memory_spaces keyed by (tenant, principal type, principal id) so one store follows a person across web, OIDC, IM, API and embed rather than being tied to sessions.user_id, whose format varies per channel. Memory itself is wiki-shaped -- slugs, markdown bodies, [[links]], revisions -- so people read and edit it the way they already read a wiki. Notes are append-only with the message ids that justify them; pages are the deduplicated, injectable unit; anchors bridge a memory to a knowledge-base wiki page. Migrations are written for both PostgreSQL and SQLite, and the SQLite chain also picks up task_pending_ops and task_dead_letters, which had only ever existed on the Postgres side and left Lite's wiki ingest without its durable queue. Settings: a single descriptor table declares all 39 keys once -- type, default, bounds, which layers may set them and how layers combine -- and the resolver is driven entirely from it. Merge behaviour is per-key rather than global: enable flags AND together so any layer can veto, budgets take the stricter value so a narrow layer cannot inflate cost, and privacy keys can only tighten. Resolved values carry the layer that decided them and the layer that locks them, so a UI can explain why a control is read-only. Services: recall assembles resident and query-relevant memories under a token budget with no model call, scoring lexically in Go with CJK bigrams rather than through a vector index -- a memory space holds hundreds of short strings, so this needs no embedding model, no per-user hidden knowledge base and behaves identically on both databases. The write path is gated by rules before it costs anything, debounced per session, and produces reviewable notes; consolidation supersedes contradicting memories instead of overwriting them, and a daily sweep decays and archives rather than deletes. Illumination and coverage are pure functions over anchors so both engines produce identical numbers. Safety: extraction reads only user-authored turns, so a poisoned document cannot implant a durable instruction; preferences take effect solely through whitelisted structured fields; free text is sanitised and injected as labelled data; instruction-shaped and credential-shaped candidates are rejected at write time rather than at read time. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…graph Connects the subsystem built in the previous commit to the places it has to be useful. Chat: a MEMORY_RECALL stage runs after query understanding and before retrieval, so the anchor hints it produces reach reranking and the memory block is ready when the prompt is assembled. The block is prepended as labelled background data, once, guarded by a flag because the pure-chat path builds its prompt before the pipeline runs while the RAG path rebuilds it midway. After the answer, cited wiki pages become asked_about anchors and the turn is offered to the write-path gate; both are best-effort. Agent: four tools -- search, read, remember, forget. Deliberately no tool for structured preferences or arbitrary field edits, since those steer generation and belong to the user. None accepts a space identifier, so an agent cannot be argued into reading someone else's memory. Tasks: extract, consolidate and decay register on both the asynq mux and the SyncTaskExecutor. Registering only the former is how a background feature ends up silently dead on Lite, so the two lists are kept adjacent and commented as a pair. The decay sweep runs from a ticker rather than the asynq scheduler, because Lite has no scheduler and a memory store that only forgets in one deployment form would grow without bound in the other. Wiki graph: ?overlay=memory decorates nodes with the caller's heat and state. Additive and opt-in, so a client that never asks sees an unchanged response. Tests pin the parts where a mistake is invisible: the settings merge rules, the illumination maths on both the decay and exemption paths, coverage arithmetic, relevance scoring including CJK, k-anonymity suppression, and the injection defences. Three defects surfaced this way and are fixed here -- built-in defaults were being folded as though they were a layer, which made boolean switches unturnable in the restrictive direction; the de-duplication hash did not trim after collapsing punctuation, so 'X.' and 'X' were stored twice; and a single stopword in common was enough to surface an unrelated memory, now guarded by a query-coverage floor. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…illumination The memory centre puts everything a person can do with their own memory on one page, because the feature only earns trust if 'what does it know about me, and how do I change it' has a single obvious answer. Four tabs in the order people ask the questions: what is remembered, what is waiting for approval, how it all connects, and what it is allowed to do. The review inbox lets a proposed memory be edited in place before it is accepted. Forcing a correction through a second dialog is what makes people reject instead of fix, and a rejected-but-true memory is a worse outcome than a reworded one. The settings panel is generated entirely from the descriptor catalogue the API returns, so a setting added on the backend appears without a frontend change and can never drift from what is actually enforced. Each row shows where its value came from and, when a wider layer has pinned it, says so and goes read-only rather than accepting a click that would do nothing. A basic/advanced split keeps the first screen to the six questions most people have. The same panel renders the workspace policy under system settings at the tenant layer. Knowledge-graph illumination is an opt-in toggle on the existing wiki graph: nodes are shaded by how much of each page the viewer has actually engaged with, disputed pages are outlined in the warning colour regardless of age, and a coverage bar shows how much of the knowledge base is lit. Additive throughout — the request only asks for the overlay when the toggle is on. GraphCanvas is a new self-contained force layout used by the memory graph. The wiki browser's own renderer is left in place and extended instead: extracting 575 lines of entangled simulation, pan/zoom, bloom and drawer state out of a 6.5k-line component, with no visual regression harness available, would be the riskiest change here for no user-visible gain. Unifying the two renderers is follow-up work. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…e schema drift Three fixes, all found by running the thing rather than reading it. Backlink maintenance no longer bumps a memory's version. A page gaining an inbound link because another memory started pointing at it is bookkeeping, not a revision: counting it as one showed the user a version jump they did not make and handed an open editor a spurious optimistic-lock conflict. The repository now has a separate UpdateLinks that writes only the link arrays. Validation failures return the status that describes them. A stale version is 409 so the editor can reload and show the newer text; asserting a system-derived anchor relation, or an unknown type or scope, is 400. All three were 500, which reads as 'the server is broken' for what are ordinary disagreements about a request. The SQLite migration chain is brought back in line with PostgreSQL. Comparing the two schemas column by column, built for real rather than read from migrations, turned up nine missing columns and three missing tables — and one of them, tenants.api_principal_config, is on a struct GORM writes on every insert, so a fresh Lite database could not create a workspace at all. Memory needed a bootable Lite instance to be verifiable, and the plan already carried the sqlite task-queue tables as a prerequisite, so the rest of the drift is closed here too. Deliberately left alone: the pgvector embeddings table, a dead column whose Go field was removed, and a one-off migration artefact. Also defers the memory decay sweep's startup Invoke to the end of container construction. Providers are lazy, but that one eager resolution pulled the whole service graph up before the data-source scheduler was registered, and the process died on boot. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Marks the plan as implemented through Phase 4 and records the two places the implementation deliberately diverged, with the reasoning, rather than quietly leaving the document describing something else. Recall no longer uses a hidden vector knowledge base per memory space. The original reasoning about the vector abstraction was correct but answered the wrong question: a memory space holds hundreds of short strings, not millions of documents, so scoring them in Go costs microseconds and the problem the hidden KB solved does not arise — while its costs (a KB row per user to filter out of every user-facing query, a hard dependency on a configured embedding model, a path to validate across ten drivers) are real. The setting is renamed to memory.recall.relevance_enabled to match. The wiki graph renderer was extended in place rather than extracted. Pulling 575 lines of entangled simulation, pan/zoom and drawer state out of a 6.5k-line component with no visual regression harness would have been the highest-risk change here for no user-visible gain. Two force layouts coexist for now; the document says so. The Lite prerequisite section is widened to what building both schemas and diffing them actually found: three tables and nine columns, one of which meant a fresh Lite database could not create a workspace at all. Appendix 2 now separates design verification from implementation verification and lists the eight defects the latter surfaced, including the settings merge treating built-in defaults as a layer, backlinks bumping the page version, and frontend query parameters never reaching the wire. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
cursor
Bot
changed the title
…dit found A review question -- why is there a lone /platform/memory page owning the settings? -- turned out to be right, and auditing the whole plan against the implementation turned up more of the same. Settings now live in the three places the design named, and only there. Personal preferences under Settings > Account, workspace policy under Settings > Workspace, per-agent overrides in the agent editor's conversation section. The memory centre keeps only a link. Previously the full user-level surface was a tab on that page and the design document had been quietly rewritten to match, which is the mistake behind the mistake: the plan stopped being a plan and became a description of whatever had been built. The workspace entry was worse than misplaced -- it was unreachable. It sat in navItems but in no nav group, and the sidebar renders from groups, so the panel could never be opened. The memory centre now matches its sibling pages. It was a centred 1240px column with an h1, a row of filled buttons and four bespoke stat cards, none of which appears anywhere else in the app; it now fills the content area like AgentList and KnowledgeBaseList, with a heading row of text icon buttons and counts as a quiet line. Four settings are deleted rather than kept: show_used_memories, cite_memories, member_coverage_visible and auto_file_wiki_issues had no behaviour behind them. A control that does nothing is worse than a missing one, because the user turns it on, believes something changed, and has no way to find out otherwise. Three that deserved implementing instead of deleting are now real. Retention purge honours retention.days and purge_archived_after_days -- the only path that deletes rather than archives, and only when an operator asks for it. Anchor resolution turns the entity names an extraction proposed into real wiki anchors by exact title, alias or slug match, which is what makes the memory-to-knowledge link automatic; the conversation's knowledge-base scope now travels with the extraction so there is something to resolve against. And the agent system prompt carries a short memory brief: agent mode had memory tools but no way to know it should reach for them before it knew anything about the person. Adds the cross-space and cross-workspace isolation tests that Phase 0 claimed were green but were never written -- seven assertions covering reads, writes, deletes, hit counting, de-duplication, the insights aggregate and space lookup. This is the property where a bug is a privacy incident rather than a defect. The plan document now opens with a per-item coverage table separating what is implemented, what diverged and why, and what the design promised but the code does not do -- including the chat-side surface, the wiki page memory sidebar and audit logging, which are honestly listed as missing rather than papered over. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Nothing was ever written to memory in any default configuration. Three faults stacked up, and each one hid the next. The write mode defaults to explicit-only, which declines the automatic extractor and defers to a direct request instead — but no direct request could ever arrive. RememberExplicit had no caller and the trigger's Explicit flag was never set by anything, so the mode meant "never write". Asking to be remembered now works: ConsiderSession recognises an imperative in the user's own words and stores the statement verbatim, with no model call, which is what makes explicit-only free as designed. Both word orders occur naturally, so the statement may sit on either side of the imperative. Switching to gated-auto did not help, because extraction refused to run without a dedicated extraction model while that setting's own description promises the conversation's model is used when it is left blank. The turn's model now travels with the trigger and is the fallback, so the documented behaviour is the real one. Agent mode reached the write path not at all — it only received a brief — leaving memory capture dependent on the model choosing to call a tool. The same gate now runs after an agent turn, sharing one settings resolution with the brief. Declining to write is no longer silent. The reason is logged, because the first symptom of all of this was a log with no trace of memory in it. Widen the injection guard while here: RememberExplicit leans on it, and a pronoun between the verb and its object was enough to slip past. Cover the phrasing family rather than one phrasing, and pin down that preferences, which read as directives by nature, are still storable. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The write path was unreachable in every default configuration and the tests did not notice, because they exercised the memory API directly and so entered below the gate that was refusing everything. These enter at the gate, with real repositories on SQLite, and assert on what is in the database afterwards. Both fail against the previous behaviour. Pinned alongside the fix: the message that exposed it is a statement, not a request, so the default mode declining it is correct. The obvious over-correction is to store every sentence containing "I am", and that is worth a test of its own. "Off" still means off, direct request or not. In the UI the same question had no answer at all — an empty memory list looks identical whether nothing was said worth keeping or nothing could ever be kept. The memory centre now reads the effective write mode and says which of the two it is, with a link to the setting. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Every control in the memory settings panel rendered as its raw key — "memory.write.mode" where "写入方式" belonged — in all four languages. The lookup builds "memory.settings.keys.<key>.label", and i18n reads a dot as a path separator, so it descended into memory → write → mode instead of finding the single entry named "memory.write.mode". The group headings were fine, which is why this survived earlier review: they are single words. The setting keys come from the backend and keep their dots, so the translations drop them and the component substitutes underscores. Guarded by tests over the locale files, since the failure is invisible in a type-check and easy to reintroduce by adding a setting: no translation key may contain a dot, every one carries a non-empty label, and the four locales describe the same set. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The Memory Centre was unreachable by clicking. Its menu entry existed in the store, but the sidebar renders from a hand-written list of paths that did not mention it, and the complementary list — which would have caught the omission by rendering whatever the first one skipped — is computed and never used. So the page could only be opened by typing its URL, which is how it went unnoticed: every walkthrough so far navigated directly. Derive both lists from one array, so an entry the store adds can be absent from the sidebar only on purpose. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The plan described the write path as if it worked, which is how the gap survived: the explicit-only mode had no implementation behind it and the document did not distinguish "designed" from "reachable" on this point. Section 6.2 now opens with the direct-request path that mode depends on, states where the line falls between saying something and asking for it to be remembered — the distinction the default mode turns on — and records both faults that made the whole path inert, so the next reader does not have to rediscover them. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
A queued task runs without the request that queued it, and everything the request had on its context has to be put back deliberately. Two things were not, and each failed in its own quiet way. Settings were resolved from the tenant and space layers only, so a write mode chosen in personal settings was invisible out here. Picking "always automatic" for yourself queued an extraction that then read the built-in default, concluded automatic writes were not allowed, and reported success having done nothing — 10ms and an empty result, which reads exactly like "the conversation held nothing worth remembering". The user layer is recoverable because the space records its owner; the agent layer is not, so the agent id now travels in the payload alongside the model id. Past that gate, the first tenant-scoped query failed outright: every repository reads the workspace id from the context and the task had none. The restoration goes inside the writer's entry points rather than the task handler, because the decay sweep arrives from a ticker and never passes through a handler. Both faults also applied to the decay sweep, which was therefore ageing memories against default retention rather than the owner's. While here, stamp the conversation's traceparent onto the extract and consolidate payloads. The payloads already embedded TracingContext but nothing ever filled it, so each task opened an orphan trace named after itself instead of appearing under the turn that caused it. Every other task type in the project already does this. Declining queued work is now logged with the mode that declined it, since the symptom of all of the above was a task that succeeded and did nothing. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Memory had three front doors — a sidebar entry, a page of its own, and two settings panels that rendered the same generated form — and no way to tell which was in charge. It now has one: an account-level settings section holding what is remembered, what is waiting to be confirmed, how it connects, and the handful of choices that govern any of it. The sidebar entry, the route, the page shell and its icons are gone. Memory is personal data one reviews occasionally, like message history, which this product keeps in settings; it did not earn a slot in the main navigation. A shortcut sits in the avatar menu alongside the other personal entries, so it is reachable without occupying one. The two settings screens looked interchangeable for a concrete reason: the personal layer can only set seven keys, and the "basics" of both screens were exactly the five they share. The workspace screen is now framed as what it is — a ceiling for everyone, with policy questions first and the operational knobs behind a link — and says so above the form. Platform invariants, which belong to no layer, appear only there; on a personal screen they were rows nobody could act on. Styling follows the dialog it now lives in rather than TDesign's defaults: the same row shape, type scale and control column as the other sections, the quiet standing note used elsewhere instead of a coloured alert, and the card treatment the model list uses. Settings save as they are changed, like every other section here — a separate save step reads as safer and is not, since it adds a state where the form and the effective value disagree with nothing to say which is which. Also drops a `confirm` field the frontend sent on "forget everything" that no backend field ever matched, and the strings left behind by all of the above. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
They were confusing for a concrete reason: the personal layer can set only seven keys, so the two screens' primary sets were almost exactly the five they share, rendered from one catalogue with identical wording. Same labels, same controls, no way to tell which was in charge. A setting key can now carry level-specific wording, and the five shared keys use it. "写入方式" on a personal screen is a choice; on the workspace screen it is "允许的最大写入方式", with help that says what it forbids members from doing. The distinction is in the words, where the user is looking, rather than only in the page title. Memory cards expose their actions through the shared card menu every other card in this product uses, instead of three small icon buttons of their own, and the disclosure link that reveals the remaining settings now looks like one. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
A pass over the whole feature looking for the fault that has bitten it five times — declared but never wired — found three more instances. The space settings layer was the widest: three keys advertised that they could be set per memory space, and `UpdateSpaceConfig` implemented it, but nothing anywhere called it and no screen offered it. The layer is now honestly described as reserved for shared spaces, which are a later phase, and the method that could never be called is gone. Six API client functions had no caller: search and stats duplicate what the list and space endpoints already return, and anchor management and insights have no UI at all. They are removed rather than left as an invitation to assume the feature exists; the plan claimed manual anchoring was possible in the memory centre, and it was not. Seven response interfaces existed and described the API accurately while no function or call site used them, so every response was `any` all the way to the template. The ones with an endpoint behind them now type their function, and the call sites drop the annotation that was discarding it — which is the kind of check that would have caught the `confirm` field the frontend was sending to a struct that never had one. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Memory labels rendered as raw keys twice, for two different reasons — a key absent from a locale, and a key shaped so i18n could never resolve it. Neither shows up in a type-check or a build; the string just comes out wrong at runtime, in one language, on one screen. This walks the feature's sources for literal translation references and resolves each against all four locale files. Verified to fail when a referenced key is removed. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…the dialog The card menu never opened: the popup's `visible` was controlled while its own click trigger was also enabled, so the two fought and the click did nothing. The popup owns its state now and is only observed for styling the trigger. Settings also saved silently and one write per keystroke. The convention in this dialog is a debounced save with a toast, which matters most on the multi-select rows where choosing three types was three requests. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The menu did open; it opened underneath. Card menus are pinned to z-index 99 because that is right on ordinary pages, but the settings dialog sits at 1100, so from inside it the menu rendered behind the overlay and the button read as dead. Two earlier attempts at this chased the click handling instead, which was never the problem. Card menus opened from the dialog now carry a modifier that lifts them to 3500, the same level the anchored form popups shown from that dialog already use. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Section 10 described a standalone page on the main navigation and a file list that no longer exists. It now describes the single settings section, says why a browser and a settings form share one scrollable surface, and states the cost of that plainly rather than only its benefit. Section 11.5 explains why the two settings screens looked like one form — the personal layer can set seven keys and five of them are also workspace keys, so the overlap is structural, not a wording slip — and how per-level wording resolves it. Also corrects a false claim: the document said anchors could be created by hand in the memory centre. They cannot, anywhere. They only ever arise automatically. New: a section listing all seven "declared but never wired" defects with their symptoms, what they had in common, and what verification now has to traverse. The pattern was not carelessness; it was tests that entered below the seam that was broken. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
**Decay compounded, so memories vanished in weeks.** Strength was the previous
strength multiplied by a factor computed from the memory's total age, so each
daily sweep re-applied the whole decay. With the shipped half-lives a profile
crossed the archive threshold on day 45 rather than the ~999 days its half-life
describes — measured, not estimated. Strength is now a function of elapsed time
since the last use, which also makes the sweep idempotent, and the reference
falls back to created_at rather than updated_at, which the sweep itself moves.
Negligible changes no longer cost a write, which is what the daily cadence was
chosen to avoid in the first place.
**Every content rule on the write path was bypassable.** Instruction detection,
deny patterns and PII handling lived in the two callers that thought of them,
so the agent's memory_remember tool and the memory editor went straight to the
store. An agent could persist "ignore your system prompt" as a preference,
which is then prepended to every later turn, and a credential was stored
verbatim whatever memory.privacy.blocked_patterns said. The checks now sit on
writePageInScope, the one path every write shares.
**PII redaction left the tail of an 18-digit national ID in the text.** The
pattern was `\b\d{15}|\d{17}[\dXx]\b`; alternation binds looser than
concatenation, so neither branch carried both boundaries and the 15-digit arm
always won: `...231X` became `[id]31X`, and digits inside a longer order number
matched too. Grouped.
**Export always failed.** It opened the API path in a new tab, and
authentication is a Bearer header the axios interceptor attaches — a plain
navigation carries none of it, so the button produced a 401 in a blank tab. It
now downloads through axios like every other download here.
**Pinning an archived memory un-archived it, and pinning twice conflicted.**
The server reads an absent status as "active", and neither the list nor the
editor sent one; the list also kept its old version instead of the one in the
response.
**task_pending_ops grew forever.** A row was written per session per debounce
window and never read — only wiki task types are peeked or claimed — while the
coalescing its comment claimed came entirely from the asynq task id. Removed,
with the one thing it could have bought written down.
Tests pin the three backend rules, each of which was invisible to the existing
suite: the shared write path refuses instructions and credentials and redacts
identifiers, whole identifiers are masked, and decay matches its half-life and
does nothing on a second same-day sweep.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
**The ranking boost could never match wiki content.** A wiki anchor stores the page slug; every candidate the matcher compared against was an id or a title. Wiki pages are what anchors were designed around, so enabling the boost changed nothing for them. **The write gate stopped at the first direct request.** "Remember I use Go — I also moved to the platform team" stored the first clause and dropped the rest of the turn. It now falls through to the automatic gate. **"Remember" matched anywhere in a sentence.** "I don't remember my password being changed" was stored as a fact about the user, as was "I can never remember which flag enables it". The marker now has to open a clause — the position that makes it an imperative rather than a mention — allowing a leading word of politeness. Stepping back a byte at a time also landed inside the multi-byte Chinese clause marks, so a trailing "，记住" stopped working; that steps by rune now. Dropped "note that" as too weak to carry the reading on its own. **Illumination vanished on drill-down.** The overview graph passed overlay=memory and the three neighbourhood loads did not, so mastery shading disappeared the moment a user clicked into a node — the view where "what do I already know here" matters most. Also: the relation and extraction-source multi-selects showed raw tokens (asked_about, user_message) because only setting keys had been translated, and the memory search box fired a request per keystroke. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Eleven of them, ordered by impact, each with what it costs a user and why it is not addressed in this branch. Extraction is dead for the IM, API-key and embed channels; supersede is documented and does not happen; insights have no UI and a k-anonymity gate measured against readers rather than dissenters; retrieval anchors do not reach the bridged graph; relation weights are ignored in ranking; consolidation's lock is process-local; the retention purge orphans rows; the SQLite column add is not idempotent; and a few interaction and arithmetic slips. Also lists the unreferenced fields and constants left over. None of them change behaviour, and every one is a candidate for the next "declared but never wired". Recorded here rather than carried in my head, since that is precisely what went wrong the last seven times. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The earlier guard covered the setting keys subtree only, which is how the relation and extraction-source multi-selects came to render raw tokens in all four languages while every check passed. Writing it immediately turned up one more: memory.privacy.pii_redaction offers an option no locale labelled. It also caught my own first attempt, which asserted against a hardcoded list I had guessed rather than read — so the list is now the values the descriptors actually declare, and a parity test sits behind it that fails whenever the locales stop offering the same set, whatever that set becomes. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Adding im, api or embed to memory.channels bought nothing. Extraction read the transcript through the owner-scoped service helper, which decides from the caller's principal and tenant role — a background task has neither. For web sessions it fell through; for every channel-managed session it returned "session not found", retried three times and was dropped, leaving a log line as the only symptom. Three of the four supported channels could never write a memory. Reconstructing a caller identity in the worker would be the wrong repair: the mapping from a principal to its session scope differs per channel and, for tenant API keys, depends on the key id. So the scope travels with the work. It is captured in the request, where it is knowable, and the worker reads the session under exactly that scope and no other — which also removes the previous reliance on an empty owner matching by the legacy rule. Tests cover the channel case that was dead, a session that is not the owner's, and a trigger that could not establish a scope at all. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
**Anchoring needed no read access to the knowledge base it wrote into.** The route cannot check it — the id arrives in the body, so KBAccessRead has nothing to bind to — so anyone could anchor into any knowledge base in the workspace and colour its coverage and insight aggregates. The handler now verifies the caller can read it, using the knowledge-base service it was already holding and never consulting. AddAnchor also validates that a knowledge base and a target were actually supplied, and no longer answers 200 with a null body when its own read-back finds nothing. **The insights k-anonymity gate protected the wrong people.** It measured the widest population for a target, which asked_about dominates because it is created automatically for every cited page. Ten readers and one objection published a contested insight reporting ten people — making the one objector identifiable in a small workspace. It now counts and reports only those who corrected or disagreed. The untouched-pages list is also capped, with the remainder counted, so a large unread wiki no longer answers with one entry per page. **Reverting silently won over a concurrent edit.** The write passed no expected version, which disables the optimistic check. It now guards with the page's own version, closing the read-modify-write race, and accepts the caller's last-seen version so an editor cannot revert over a change it never saw; the dialog sends it and reports a conflict the way saving already did. **A retention purge left orphans.** Hard-deleting a page left its revisions, anchors and the notes merged into it pointing at an id that no longer resolves — which is how a purged memory keeps colouring the illumination overlay. All of it goes in one transaction now, with the observations marked no longer merged rather than deleted so the evidence trail survives. Writing the test caught that revisions key on page_id while anchors key on memory_page_id, so the first version of this fix deleted nothing. Also: bridged graph mode reported "showing 40 of 25" because Returned counted satellites while Total counted memories; and three stored-but-never-read dependencies are gone, with the unused VectorKBID column now described as reserved rather than as a feature that exists. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Eleven findings became eight fixed and a shorter list of what is left, each with the cost of leaving it and the reason. Two entries changed character rather than disappearing: supersede is now described as the behaviour that exists rather than the one the design promised, and the SQLite column add is recorded as consistent with every other migration in that directory rather than as a defect to patch in one place. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Asking against a normal (non-wiki) knowledge base records asked_about anchors — target kind "knowledge", target ref the knowledge id — and the space stat counts them. Nothing read them. All four consumers excluded them: the bridged graph skips anything that is not a wiki page and anything without a memory page id, illumination is an overlay on the wiki graph, insights filter to wiki pages, and the ranking boost loaded hints only for anchors tied to a recalled memory page, which retrieval anchors never have. Four rows written per turn that no code path could reach. Ranking is the consumer they belong to: asked_about describes a person's engagement with the knowledge base, which is exactly what personalised ranking needs. Hints are now loaded for the space and the knowledge bases in play rather than through the recalled pages, so an ordinary knowledge base matches on its knowledge id and a wiki on its slug. The relation weight each hint already carried is applied instead of discarded, so a page someone corrected outranks one they merely mentioned — which is what memory.overlay.relation_weights claims to control. The bridged graph still shows nothing for an ordinary knowledge base, and correctly so: it draws memory-to-wiki-page edges and there are no wiki pages. It says that now instead of presenting an empty canvas. Also stops two node captions rendering as one unreadable line: repulsion balances gravity at about a hundred pixels, which is about how wide a dozen Chinese characters render. A positional separation pass keeps captions apart without raising repulsion and blowing dense graphs apart, and the full title is available on hover now that the visible one is truncated. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Two deferred entries resolved. Retrieval anchors staying out of the bridged graph is now recorded as correct behaviour rather than a gap — they belong to no memory, so there is no edge to draw — and the remaining hole is stated plainly: an ordinary knowledge base has no illumination view at all, because illumination is an overlay on the wiki graph and insights filter to wiki pages. Its anchors do affect ranking now, which is the part that was dead. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Illumination only ever existed for wiki knowledge bases: the overlay was an overlay on the wiki graph, coverage counted wiki pages, and insights filtered to them. An ordinary knowledge base recorded engagement and had nothing to show for it. Nothing about the maths was wiki-specific. The repository projection already took a target kind, and the heat, state and coverage functions are pure over plain slices, so the change is to stop hardcoding the kind and to project the other unit: a document, keyed by the id a retrieval anchor already records, and bucketed by the first segment of its folder path exactly as a wiki page is bucketed by its first breadcrumb. A wiki graph carries its overlay inside the graph response because it is drawing nodes anyway. A document list has no such carrier, so there is now an endpoint for it. Insights work on documents too, with the stored file size standing in for a wiki page's body length. It is coarser — a scanned PDF is large and may still say little — but the signal is "asked about a great deal, and barely anything here", and size carries it. Tests pin that the two unit kinds do not leak into each other's reports, and that documents bucket by folder the way pages bucket by breadcrumb. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
The lens the wiki graph applies to its nodes, applied to a document list: which of these has this reader actually engaged with, and how much of the knowledge base that adds up to. A dot beside each document title carries the state — touched, familiar, mastered, or flagged for something disputed. Nothing is drawn for a document nobody has touched, which is the common case: a column of grey dots would be noise that also drowns the few marks that mean something. The tooltip says it in words, with how many interactions and how long ago. The vocabulary is the wiki legend's, deliberately, because it is one idea seen from two angles and a reader should not have to learn two sets of colours for it. One component renders the dot for both the grid and the list, so the two cannot drift into different vocabularies. The toggle sits beside the view switch and borrows its shape, since both are lenses over the same list, and the choice is remembered — someone who wants this usually wants it every time. Turning it on reveals a coverage bar reading "lit of total". The whole control is hidden, not disabled, when memory or illumination is off: a switch that cannot do anything is worse than no switch. Verified end to end against a real knowledge base with three documents: two anchored and lit with their states and heat, the third correctly dark, coverage at two of three, and the insights report naming the unread one. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Chapter 7 described illumination as something that happens to a wiki. Nothing about the maths was wiki-specific — heat, state and coverage are pure functions over a slice of anchors, and the repository projection already took a target kind — so it now covers both, and says where the two differ: the carrier. A wiki graph folds its overlay into the graph response because it is drawing nodes anyway; a document list asks for it directly. Moves "an ordinary knowledge base has no illumination view" out of the known issues and into the fixed list. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…bset An ordinary knowledge base lit up in the document list while the memory graph stayed empty beside a header counting its anchors. Both filters in the satellite pass excluded exactly those anchors: anything that is not a wiki page, and anything without a memory page id — which every retrieval anchor lacks, and retrieval is the only way an ordinary knowledge base produces one. The premise of the bridged view is where a person's understanding meets the organisation's, and that includes what they have engaged with even when no one memory is responsible for it. So satellites now cover both kinds, drawn with an edge when a specific memory is anchored to them and without one otherwise. The legend and the hint say that, instead of promising wiki pages. Documents are anchored by id, because that is what a retrieval result carries, and an id is not a label. The graph service has no knowledge service and should not grow one for a display concern, so the handler substitutes titles after the fact; a document since deleted keeps its id, which at least says something is there. Unattached satellites are capped at thirty, most-engaged first. They are the numerous kind — one per cited item per turn — and past a few dozen they stop being a picture of what someone works with. Verified against real data: two ordinary documents appear under their own titles as unattached satellites while four wiki anchors keep their edges. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Retrieval anchors staying out of the bridged graph was recorded as correct behaviour last round, on the reasoning that they belong to no memory and so have no edge. That reasoning held for the edge and not for the node: they are still something this person engaged with, and leaving them out meant an ordinary knowledge base could light up its document list and show an empty canvas beside a header counting its anchors. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
- Updated GraphCanvas to support double-clicking nodes to center them in the viewport. - Added a new `panToNode` function to facilitate panning to specific nodes. - Enhanced the MemoryEditorDrawer with additional fields for memory preferences and improved layout. - Introduced MemoryPersonalSettingsDrawer for user-specific memory settings, including export and forget options. - Improved UI components for MemoryGraphPanel and MemoryInbox, including better loading states and user interactions. - Updated localization files for English and Chinese to reflect new features and UI changes. These changes aim to improve user experience and functionality in managing and visualizing memory data.
- Introduced a new document outlining the long-term memory restructuring plan, detailing the transition from a complex Memory Wiki to a more user-friendly model focused on "Saved Memory" and "Chat History Memory." - Updated the MemoryPage and MemoryNote interfaces to include new fields: `saved` and `memory_key`, allowing for better differentiation between user-saved memories and automatically inferred memories. - Enhanced the memory retrieval system to prioritize saved memories during recall, ensuring they are always considered regardless of type. - Modified the frontend components to reflect the new memory structure, including updates to the MemoryList and MemorySettings views for improved user interaction. - Updated localization files for English and Chinese to accommodate new terminology and features related to memory management. These changes aim to improve user trust and clarity in memory management, ensuring users can easily understand and control what is remembered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Implements per-user long-term memory, end to end: storage, layered settings, recall in chat, agent tools and brief, background extraction and consolidation, the illumination overlay on the knowledge graph, and the memory UI.
docs/长期记忆功能规划.mdcarries the design, opens with a per-item coverage table, and closes with the defects that were fixed and the ones that remain.The idea is to stop treating memory as one more retrieval source and build it out of the wiki machinery this repo already has.
Memory is wiki-shaped. Memories live in
memory_pages— slug, markdown body,[[links]], aliases, revisions — so people read and edit them the way they already read a wiki. Three layers: append-onlymemory_notescarrying the message ids that justify them, deduplicatedmemory_pagesas the only thing that reaches a prompt, and a graph made of link arrays plus anchors.Anchors make the knowledge base personal.
memory_anchorsbridges a memory to a knowledge-base wiki page, filled in from retrieval citations (free) and by resolving the entity names an extraction proposed. From that one table: the wiki graph shaded by how much of each page you have actually engaged with (unlit→touched→familiar→mastered, plusflaggedfor pages you disputed), a bridged graph showing where your understanding attaches to the organisation's, a per-folder mastery metric, and anonymised aggregates over which pages are asked about but thin.Two hard constraints, both honoured. Zero Neo4j: graph edges are JSON arrays in the main database and all graph maths runs in Go. Lite is a first-class target: the capability test is
DB_DRIVER=sqliteplus an emptyREDIS_ADDR, never the product edition, and the whole feature is verified on both databases.How a memory gets written
The write mode ships as
explicit_only: saying "记住……" in a conversation stores that statement verbatim, with no model call at all. Above that,gated_autoruns a pure-rule gate, debounces into a single background extraction per session, then consolidates — deduplicating by normalised hash and merging near-duplicates.The line between those modes is whether the user said something or asked for it to be remembered. "我是wizard，一个程序工程师" is durable context for the extractor to weigh; "记住我是wizard" is an instruction to store it. The marker has to open a clause, so "I don't remember my password" is nobody's fact.
Every write — extraction, the agent's
memory_remember, the memory editor, the API — passes the same content rules on the one path they all share: an instruction is refused, a credential is refused, and direct identifiers follow the workspace PII policy.Background work carries what the request knew: the session owner scope it was permitted to read under, the agent whose overrides shaped the decision, and the model the turn used. None of that is recoverable in a worker, and each absence had made the feature silently do nothing.
One home, in the product's own style
Memory has a single surface: Settings → Account → 长期记忆, holding what is remembered, what is waiting to be confirmed, how it connects, and the choices that govern any of it. There is no sidebar entry and no page of its own — memory is personal data one reviews occasionally, like message history, which this product keeps in settings. A shortcut sits in the avatar menu beside the other personal entries.
memory_one_home_settings_and_workspace_policy.mp4
The workspace screen is a ceiling for everyone and now reads as one. The two screens looked interchangeable for a structural reason: the personal layer can set seven keys and five of those are also workspace keys, so the "basics" of both were the same five rows. A setting key can now carry per-level wording — personal 写入方式 is workspace 允许的最大写入方式, with help that says what it forbids members from doing.
Workspace memory policy, with rows worded as limits and a note saying this sets a ceiling
Styling follows the dialog rather than TDesign's defaults: the same row shape, type scale and control column as the other sections, the quiet standing note instead of a coloured alert, the card treatment the model list uses, the shared card menu,
t-emptyfor empty states, the product's green in the graph rather than TDesign's blue, and debounced save-on-change with a toast exactly asChatHistorySettingsdoes it.A memory card
Other design decisions worth review
Recall costs no model call and no new infrastructure. Resident memories come from an indexed query; relevant ones are scored lexically in Go with CJK bigrams. The plan called for a hidden vector KB per memory space; implementation showed that answered the wrong question — a space holds hundreds of short strings, so scoring them in Go takes microseconds, while a hidden KB per user costs a row to filter out of every user-facing query and a hard dependency on a configured embedding model. Section 6.4 records the reversal.
Illumination is pure functions. Heat, state and coverage are computed over plain slices. Expressing them as SQL (
power(),ln(),GROUP BY ROLLUP) works on Postgres and fails outright on Lite, so the repository only runs portable SELECTs. Both engines produce identical numbers, measured rather than assumed.Memory-borne prompt injection is treated as a first-class threat. Extraction reads only
role=userturns — never documents, wiki pages, search results or tool output — so a poisoned document cannot implant an instruction that survives every future session. Preferences take effect solely through whitelisted structured fields. Free text is sanitised and injected as labelled data.Forgetting is load-bearing. Typed half-lives, decay exemptions, archival rather than deletion, per-space caps, an opt-in retention purge that also removes what pointed at the page, and a sweep on a ticker so it behaves the same with and without Redis. Strength is a function of elapsed time since last use, which makes the sweep idempotent.
Nothing here can break a conversation. Recall runs under a deadline and degrades to "no memory this turn"; anchor recording, extraction and the agent brief are best-effort; a failure anywhere in the subsystem loses memory, never the answer.
Scope
The plan's coverage table is authoritative, and a "known issues" section lists what is not fixed, with the cost of each. The largest is that the insights half of the feature has no UI; the rest are a semantic question about which anchors belong in the bridged graph, relation weights not reaching the ranking boost, a process-local consolidation lock, and two interaction slips.
Not implemented and listed as such: the chat-side surface (
@记忆, a "remember this" button, the used-memories drawer, memory citations), new-session continuation suggestions, the wiki page "your memory" sidebar, manual anchoring UI, an insights view, audit logging,POST /memory/import, and all of Phase 5.Settings and code with no behaviour behind them are deleted rather than shipped dark: four dead switches, the unreachable space settings layer, six API client functions whose endpoints have no UI, a
task_pending_opsrow written on every extraction that nothing ever read, and three stored-but-never-consulted dependencies.Also fixes a pre-existing blocker found on the way: comparing the PostgreSQL and SQLite schemas column by column turned up three missing tables and nine missing columns, one of them
tenants.api_principal_config, which GORM writes on every insert — meaning a fresh Lite database could not create a workspace at all.Type of Change
Related Issue
Implements the
知识库形态 → 探索知识库与 Memory 结合的应用场景item indocs/ROADMAP.md.Testing
This feature was unusable seven times, always the same way: something declared but never wired, with the tests entering below the seam that was broken. The plan lists all of them. That pattern shaped the tests:
write_path_test.godrivesConsiderSessionagainst real repositories on SQLite and asserts what is in the database afterwards. The earlier end-to-end tests called the memory API directly and so entered below the gate that was refusing everything.background_context_test.gopins the workspace on the context, the space owner's user layer and the payload's agent layer in the settings resolution, and that the transcript is read under the requester's session scope — including the channel case that was dead, a session that is not the owner's, and a trigger with no scope at all.write_rules_test.goasserts instructions and credentials are refused and identifiers masked, that whole identifiers are masked, and that decay matches its half-life and does nothing on a second same-day sweep.page_idwhile anchors key onmemory_page_id, so the first version of that fix deleted nothing.memory_scope_test.goprove cross-space and cross-workspace reads, writes, deletes, hit counting, de-duplication, space lookup and the insights aggregate stay confined.go test ./...is green across all packages;vue-tsc --build,vite buildand the locale tests pass.End-to-end on both databases. A 28-assertion HTTP suite covering settings resolution, lazy space creation, writing, links and backlinks, revisions and revert, optimistic locking, the graph, anchors, layered locks and clamping, memory switched off, export and forget passes identically on Lite (SQLite, no Redis) and PostgreSQL.
Verified through the product, not the API. A Lite instance ran with a local stand-in chat provider so the full conversation path executes, including the schema-constrained extraction call. Confirmed live, against the running deployment after the final changes: an instruction refused through the manual write API, an 18-digit identifier stored as
[id], an anchor into an unreadable knowledge base answered 404 and one missing its target 400, and a real chat turn producingextracted 1 candidate notesand a consolidatedpipeline-sourced memory. Earlier rounds confirmed a direct request recalled in a brand-new conversation, and the review inbox path.A fresh conversation recalling the memory stored in an earlier one
An automatically extracted candidate waiting in the review inbox
The bridged graph is the feature's central idea made visible: solid nodes are the person's own memories, dashed satellites are the knowledge-base pages their understanding is anchored to.
Bridged memory graph with dashed knowledge-base satellites anchored to memory nodes
Defects found and fixed on this branch
Measured, not estimated, where a number appears.
gated_autorefused to run without a dedicated extraction model while that setting promises the conversation's model when blank; agent mode reached the write path not at all.memory.channelswas configured.task_pending_opsgrew forever.anyto the template — including aconfirmfield no Go struct had. The write gate stopped at the first direct request. "Remember" matched anywhere in a sentence. Search fired per keystroke. Bridged mode could report "showing 40 of 25". Plus four dead settings, backlink maintenance bumping the page version, a startupInvokethat killed the process on boot, one shared stopword surfacing unrelated memories, and an injection guard matching one phrasing rather than the family.Not verified, and stated as such in the plan: extraction quality and cost against a real model on real conversations, read-path latency under load, long-run convergence of a memory space. One open question is recorded rather than settled: settings and data management share one scrollable dialog section, which two reviewers independently called heavy; the alternative reintroduces a page with no navigation entry.
Checklist
git diff --check origin/main...HEADpassesgofmt)go test ./..., frontend type-check, build and locale tests greenScreenshots / Recordings
Included above.
To show artifacts inline, enable in settings.