Conversation
Adds benchmarks/common/goodmemory_client.py - an async client for the GoodMemory HTTP bridge (npm i -g goodmemory; goodmemory-http-bridge) exposing the same add/search/delete_user interface as Mem0Client - and wires --backend goodmemory (+ --goodmemory-host) into the LOCOMO, LongMemEval, and BEAM runners. Notes: - Bearer-token auth (GOODMEMORY_HTTP_BRIDGE_TOKEN) and the bridge's per-request caller header are handled automatically. - Benchmark seeding writes every turn deterministically (verified fact annotations + rules-only extraction), mirroring how GoodMemory's own benchmark harnesses ingest conversations; assistant turns are stored as user-scoped facts with the original role preserved in the content, since assistant-authored writes are blocked by the product write policy. - Runs are isolated by scope (user ids embed the project name); the bridge has no bulk delete endpoint, so delete_user relies on fresh project names. - Smoke-tested end to end against a live bridge (add -> search round trip).
…tack The bridge coerces any non-hybrid recall strategy (including auto) to a rules-only lexical floor, which cannot surface semantically-relevant facts. Default recall_strategy to hybrid so representative recall works out of the box (override via GOODMEMORY_RECALL_STRATEGY), and document the bridge config it needs: an embedding endpoint, the recommended retrieval preset (semantic candidate union), and in-memory storage. Deterministic verbatim seeding is unchanged (no LLM extractor required). Validated 4/4 on cross-topic recall probes (the query-relevant fact ranks first).
|
Refreshed the GoodMemory adapter in commit Validation:
The PR remains ready for maintainer review. |
|
Refreshed this integration for the published GoodMemory Validation is current: 5/5 unit tests, all three runners compile, |
|
Current verification update: commit Validation on Python 3.12.13, Node v22.14.0, Bun 1.3.14, and npm 10.9.2:
This is integration/runtime evidence only. No benchmark answerer, judge, paid model, or score was run. The PR is mergeable and still awaiting maintainer review. |
Adds GoodMemory
v0.7.5as a local HTTP backend next to Mem0 Cloud/OSS.What is included
benchmarks/common/goodmemory_client.py: an async adapter around the publishedgoodmemory-client==0.1.0package. The official client owns the HTTP wire contract, caller/scope headers, bearer auth, retries, and routing metadata.--backend goodmemoryplus--goodmemory-hostin the LoCoMo, LongMemEval, and BEAM runners.goodmemory-http-bridge --recommended; embeddings are optional.Contract and evidence boundaries
metadataPatch.attributes.sourceRole.--project-name.autoby default and reports strategy fallback instead of hiding a retrieval downgrade.recall-contextcontract returns at most 12 selected items and has no caller-controlled item limit. The documented comparable configuration therefore uses--top-k 10 --top-k-cutoffs 10; requests above 12 emit an explicit warning.0.0instead of synthesizing a number that could influence the retrieval judge.Validation
Environment: Python 3.12.13, Node v22.14.0, Bun 1.3.14, npm 10.9.2.
python -m unittest discover -s tests -v— 6/6 passed.python -m py_compilefor the adapter and all three runners.--helppaths load and expose the GoodMemory backend under Python 3.12.python -m pip check— no broken requirements.git diff --check.goodmemory@0.7.5package with bearer auth and an in-memory recommended bridge: two timestamped fixture facts accepted, a same-scope query recalled the relevant deployment-region fact, and a different scope returned no results. The bridge reportedretrievalTier: preset-recommended; embeddings were intentionally disabled for this contract smoke.No benchmark answerer, judge, paid model, or score was run as part of this validation.