Skip to content

docs: surface Jina task hints (and the re-embed they require)#953

Open
DanielMao1 wants to merge 1 commit into
CortexReach:masterfrom
DanielMao1:docs/jina-task-hints
Open

docs: surface Jina task hints (and the re-embed they require)#953
DanielMao1 wants to merge 1 commit into
CortexReach:masterfrom
DanielMao1:docs/jina-task-hints

Conversation

@DanielMao1

Copy link
Copy Markdown
Contributor

Problem

The recommended-config prompt says "Use Jina for embedding" but never mentions task hints. jina-embeddings-v5 is an asymmetric retrieval model — without task, the retrieval adapter is never activated. Since that prompt is what generates the user's openclaw.json, every user who follows this README silently runs at degraded recall.

The knob already exists (embedding.taskQuery / embedding.taskPassage, documented in configSchema) — it's just not surfaced anywhere a new user would look.

Evidence

Real production store (~4k Chinese memories, ~6 months of use). Retrieval set: 208 docs (12 with a known gold answer + 196 distractors from the same store) × 12 real user queries.

MRR Top-1 mean margin
without task 0.690 7/12 (58%) +0.049
with task 0.854 9/12 (75%) +0.143

mean margin = cos(gold) − cos(best distractor). Without task hints the gold answer loses the top slot 42% of the time.

Why the second commit

Task hints only affect vectors written after they are set. Turning them on over an existing store leaves queries in one space and passages in another — and nothing warns about it.

Measured on the same store: re-embedding identical text with vs without a task hint gives cosine 0.53–0.70 (mean 0.63) between the two vectors. Same model, same text. Enabling the config without re-embedding captures only about half the gain.

reembed already does the right thing (cli.ts:1900embedBatchPassage), it just isn't mentioned near the task-hint config.

A note on "asymmetric"

The gain comes from having a task tag at all, not from pairing query/passage correctly. On a 6-doc probe, tagging both sides with retrieval.query scored about the same as the official pairing (+0.314 vs +0.321 — noise-level). Tagging only one side captures roughly half. So the README line matters more than getting the pairing exactly right.

Changes

  • 1/2: one line in the Requirements list → applied to all 11 README translations (that block is English everywhere, keeping it in sync)
  • 2/2: a re-embed note in the existing-users/upgrade section → README.md + README_CN.md only; other translations are localized and belong to your translation flow

Docs only, no code changes.

@DanielMao1
DanielMao1 force-pushed the docs/jina-task-hints branch 3 times, most recently from 91b4971 to f530388 Compare July 17, 2026 08:17

@app3apps app3apps left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed head f530388. Surfacing Jina query/passage task hints is valuable, but the documented migration is unsafe as written.

  1. The READMEs recommend reembed --source-db <your dbPath> --force. That path calls importEntry(), which uses table.add() rather than replacing the existing ID. An exact-head reproduction starting with one row completed with two rows sharing that ID: the old vector remained beside the new one. Retries compound the duplicates.
  2. The prescribed backup is not a whole-store backup: --scope global excludes agent/custom scopes, and memory-pro export hard-limits the result to 1000 rows, while re-embed reads the whole table.
  3. The statement that the same procedure applies to embedding.dimensions is incorrect. Opening the old table as the newly configured target fails vector-dimension validation; a changed dimension requires a fresh target database.

Please replace the in-place --force guidance with a safe cutover using a new target dbPath and the old path only as --source-db, including a complete verified backup, dry run, row/ID-count verification, and gateway stop/restart steps. If in-place migration is intended, the command itself must replace records transactionally and be safe to retry before the README recommends it.

Requesting changes.

The recommended-config prompt says "Use Jina for embedding" but never
mentions task hints. Since that prompt is what generates the user's
openclaw.json, every user who follows this README silently runs at
degraded recall.

Per the jina-embeddings-v5-text paper (arXiv:2602.15547 §3, §4.2.1), the
model distinguishes queries from documents by pre-pending a "Query:" or
"Document:" prefix, and is trained around a dedicated asymmetric retrieval
adapter: "encoding queries and documents differently can yield large
improvements in retrieval." Without `task`, that prefix is never applied.

Measured on a real 4k-memory Chinese store (208 docs: 12 with a known gold
answer + 196 distractors, 12 real user queries):

  without task    MRR 0.690   Top-1  7/12 (58%)   mean margin +0.049
  with task       MRR 0.854   Top-1  9/12 (75%)   mean margin +0.143

mean margin = cos(gold) - cos(best distractor). Without task hints the gold
answer loses the top slot 42% of the time.

Second half: task hints only affect vectors written after they are set, so
enabling them over an existing store leaves queries and passages in
different spaces with no warning. Re-embedding identical text with vs
without a task hint gives cosine 0.53-0.70 (mean 0.63) between the two
vectors — same model, same text. `reembed` already handles this correctly
(cli.ts:1900 -> embedBatchPassage), it just isn't mentioned anywhere near
the task-hint config.

The Requirements line goes to all 11 README translations (that block is
English everywhere). The re-embed section is README.md + README_CN.md only,
since that section is localized.
@DanielMao1

DanielMao1 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — all three are real and I've reworked the section (force-pushed, still one commit 34842dc).

  1. In-place reembed doubles rows. Confirmed: importEntry() writes with table.add(), and the command's own guard already refuses same-path runs unless --force. The doc now drops --force entirely and prescribes a cutover into a fresh target dbPath, with the old path passed only as --source-db — source ≠ target, so each id lands once.

  2. export is not a whole-store backup. Right — it defaults to --limit 1000 and a single --scope, so it silently drops rows past 1000 and every non-global scope. Backup is now a filesystem copy of the LanceDB directory (cp -r), with a note on why export isn't safe here. I also opened docs: use a whole-store filesystem backup before upgrade #957 to fix the same export-as-backup pattern in the pre-existing "Upgrading from pre-v1.1.0?" section.

  3. embedding.dimensions can't reuse the old table. Corrected — a new vector width only goes into a new table, so the doc folds dimension/model changes into the same fresh-dbPath cutover rather than claiming in-place works.

@DanielMao1
DanielMao1 requested a review from app3apps July 20, 2026 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants