docs: surface Jina task hints (and the re-embed they require)#953
docs: surface Jina task hints (and the re-embed they require)#953DanielMao1 wants to merge 1 commit into
Conversation
91b4971 to
f530388
Compare
app3apps
left a comment
There was a problem hiding this comment.
Reviewed head f530388. Surfacing Jina query/passage task hints is valuable, but the documented migration is unsafe as written.
- The READMEs recommend
reembed --source-db <your dbPath> --force. That path callsimportEntry(), which usestable.add()rather than replacing the existing ID. An exact-head reproduction starting with one row completed with two rows sharing that ID: the old vector remained beside the new one. Retries compound the duplicates. - The prescribed backup is not a whole-store backup:
--scope globalexcludes agent/custom scopes, andmemory-pro exporthard-limits the result to 1000 rows, while re-embed reads the whole table. - The statement that the same procedure applies to
embedding.dimensionsis incorrect. Opening the old table as the newly configured target fails vector-dimension validation; a changed dimension requires a fresh target database.
Please replace the in-place --force guidance with a safe cutover using a new target dbPath and the old path only as --source-db, including a complete verified backup, dry run, row/ID-count verification, and gateway stop/restart steps. If in-place migration is intended, the command itself must replace records transactionally and be safe to retry before the README recommends it.
Requesting changes.
The recommended-config prompt says "Use Jina for embedding" but never mentions task hints. Since that prompt is what generates the user's openclaw.json, every user who follows this README silently runs at degraded recall. Per the jina-embeddings-v5-text paper (arXiv:2602.15547 §3, §4.2.1), the model distinguishes queries from documents by pre-pending a "Query:" or "Document:" prefix, and is trained around a dedicated asymmetric retrieval adapter: "encoding queries and documents differently can yield large improvements in retrieval." Without `task`, that prefix is never applied. Measured on a real 4k-memory Chinese store (208 docs: 12 with a known gold answer + 196 distractors, 12 real user queries): without task MRR 0.690 Top-1 7/12 (58%) mean margin +0.049 with task MRR 0.854 Top-1 9/12 (75%) mean margin +0.143 mean margin = cos(gold) - cos(best distractor). Without task hints the gold answer loses the top slot 42% of the time. Second half: task hints only affect vectors written after they are set, so enabling them over an existing store leaves queries and passages in different spaces with no warning. Re-embedding identical text with vs without a task hint gives cosine 0.53-0.70 (mean 0.63) between the two vectors — same model, same text. `reembed` already handles this correctly (cli.ts:1900 -> embedBatchPassage), it just isn't mentioned anywhere near the task-hint config. The Requirements line goes to all 11 README translations (that block is English everywhere). The re-embed section is README.md + README_CN.md only, since that section is localized.
f530388 to
34842dc
Compare
|
Thanks — all three are real and I've reworked the section (force-pushed, still one commit
|
Problem
The recommended-config prompt says "Use Jina for embedding" but never mentions task hints.
jina-embeddings-v5is an asymmetric retrieval model — withouttask, the retrieval adapter is never activated. Since that prompt is what generates the user'sopenclaw.json, every user who follows this README silently runs at degraded recall.The knob already exists (
embedding.taskQuery/embedding.taskPassage, documented inconfigSchema) — it's just not surfaced anywhere a new user would look.Evidence
Real production store (~4k Chinese memories, ~6 months of use). Retrieval set: 208 docs (12 with a known gold answer + 196 distractors from the same store) × 12 real user queries.
mean margin = cos(gold) − cos(best distractor). Without task hints the gold answer loses the top slot 42% of the time.Why the second commit
Task hints only affect vectors written after they are set. Turning them on over an existing store leaves queries in one space and passages in another — and nothing warns about it.
Measured on the same store: re-embedding identical text with vs without a task hint gives cosine 0.53–0.70 (mean 0.63) between the two vectors. Same model, same text. Enabling the config without re-embedding captures only about half the gain.
reembedalready does the right thing (cli.ts:1900→embedBatchPassage), it just isn't mentioned near the task-hint config.A note on "asymmetric"
The gain comes from having a task tag at all, not from pairing query/passage correctly. On a 6-doc probe, tagging both sides with
retrieval.queryscored about the same as the official pairing (+0.314 vs +0.321 — noise-level). Tagging only one side captures roughly half. So the README line matters more than getting the pairing exactly right.Changes
README.md+README_CN.mdonly; other translations are localized and belong to your translation flowDocs only, no code changes.