feat: enhance augment prompt to detect disguised identifiers

[asteier2026]

Summary

• Adds an Additional extraction requirements block to the LLM entity augmentation prompt
• Requires verbatim span extraction (no normalization, reconstruction, or reformatting)
• Adds detection examples for identifiers written as digit words (phone/SSN/credit card) and letter-by-letter name spellings

Motivation

The augmenter was missing entities that were obfuscated or spoken aloud, e.g.:

• "nine o two, five five five, one two three four" (phone number)
• "J-O-H-N" (name spelled out with hyphens)

[greptile-apps]

Greptile Summary

Adds an Additional extraction requirements block to the LLM augment prompt in _get_augment_prompt, reinforcing verbatim span extraction and introducing detection hints for obfuscated identifiers (digit-word phone/SSN/credit-card numbers and letter-by-letter name spellings).

• The verbatim-extraction requirement is correctly motivated: the downstream anonymizer must locate and replace the exact text that appears in the document, so normalizing "nine o two…" to "902-…" would break replacement.
• The new detection examples show input patterns only; no corresponding expected output (label + reason) is included, leaving the LLM without a worked instance for the most novel cases.
• The requirements block is positioned after <<EXAMPLE_BLOCK>>, so the existing few-shot example does not demonstrate the new disguised-identifier behavior.

Confidence Score: 4/5

The change is prompt-only with no surrounding code logic altered; the verbatim extraction intent is sound and the new detection examples are well-chosen.

The prompt addition is logically coherent and solves a real gap. Two structural weaknesses — missing labeled output examples for the new cases and the post-example placement of the requirements — could cause inconsistent labeling or reduced LLM adherence across model versions, but neither causes an immediate breakage in the current workflow.

src/anonymizer/engine/detection/detection_workflow.py — specifically the ordering of the new requirements block relative to <<EXAMPLE_BLOCK>> and the absence of worked output examples for digit-word and letter-spelled identifiers.

Important Files Changed

Filename	Overview
src/anonymizer/engine/detection/detection_workflow.py	Adds an "Additional extraction requirements" block to the augment prompt in _get_augment_prompt, reinforcing verbatim span extraction and adding detection hints for disguised identifiers (digit-word phone/SSN/CC numbers, letter-spelled names). The block is placed after <<EXAMPLE_BLOCK>> and before the input separator. No code logic is changed, only the static prompt text.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Input text + Seed entities] --> B[_get_augment_prompt]
    B --> C{strict_labels?}
    C -- Yes --> D[label_block: ONLY allowed labels]
    C -- No --> E[label_block: prefer known labels, allow new]
    D --> F[example_block substitution]
    E --> F
    F --> G[Prompt: Task + LABEL_BLOCK + Rules + EXAMPLE_BLOCK]
    G --> H[NEW: Additional extraction requirements\n- Verbatim span extraction\n- No normalization or reconstruction\n- Detect disguised identifiers\n  e.g. digit-word phone numbers\n  e.g. letter-by-letter names]
    H --> I[--- Input text + Seed entities ---]
    I --> J[LLM augmentation inference]
    J --> K[New detected entities JSON]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Input text + Seed entities] --> B[_get_augment_prompt]
    B --> C{strict_labels?}
    C -- Yes --> D[label_block: ONLY allowed labels]
    C -- No --> E[label_block: prefer known labels, allow new]
    D --> F[example_block substitution]
    E --> F
    F --> G[Prompt: Task + LABEL_BLOCK + Rules + EXAMPLE_BLOCK]
    G --> H[NEW: Additional extraction requirements\n- Verbatim span extraction\n- No normalization or reconstruction\n- Detect disguised identifiers\n  e.g. digit-word phone numbers\n  e.g. letter-by-letter names]
    H --> I[--- Input text + Seed entities ---]
    I --> J[LLM augmentation inference]
    J --> K[New detected entities JSON]

Loading

_{Reviews (1): Last reviewed commit: "feat: enhance augment prompt to handle d..." | Re-trigger Greptile}

[greptile-apps]

Extraction requirements placed after the example block

Standard few-shot prompting places all instructions before the demonstrations so the example can illustrate those instructions. Here, "Additional extraction requirements" appears after <<EXAMPLE_BLOCK>>, so the concrete example does not show verbatim extraction of a disguised identifier — leaving the LLM without a worked instance of the most novel new behavior. Moving these requirements above <<EXAMPLE_BLOCK>> (or adding a second worked example that demonstrates digit-word extraction) would make the prompt self-consistent.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[asteier2026]

I think it's good the way it is.