feat: apply direct identifier replacements before rewrite LLM call

[asteier2026]

Summary

• Adds COL_PREREPLACE_TEXT and COL_PREREPLACE_TAGGED_TEXT constants
• Adds _apply_direct_replacements generator that programmatically substitutes direct identifiers from
  the replacement map before the rewrite LLM call, using a single-pass regex to prevent cascade
  replacements (e.g. Alice→Bob→Carlos)
• Removes <replacement_map> block from the rewrite prompt — the LLM no longer needs to apply
  replacements
• Updates repair.py to use pre-replaced text as the repair baseline
• Removes COL_REPLACEMENT_MAP_FOR_PROMPT (no longer needed)
• Adds whitespace-normalized fallback matching in both _filter_replacement_map_to_input_entities and
  _get_replace_pairs to handle LLM-normalised Unicode whitespace (e.g. U+202F → U+0020) in entity values
• _apply_direct_replacements falls back to passing text through unchanged on error rather than
  dropping the record

Motivation

The rewrite LLM was inconsistently applying replacement map entries, especially for entities that
appear multiple times. Programmatic pre-replacement guarantees all occurrences are substituted before
the LLM sees the text. The whitespace fixes handle cases where the LLM normalises unusual Unicode
whitespace in entity values, which caused map lookups to silently miss.

[greptile-apps]

Greptile Summary

This PR moves the application of replace-method entity substitutions from the LLM rewrite prompt to a deterministic, single-pass regex step (_apply_direct_replacements) that runs before the LLM call, fixing inconsistent multi-occurrence replacement. The repair baseline in repair.py is updated accordingly to use the pre-replaced text.

• New _apply_direct_replacements generator: builds a combined regex from all (original → synthetic) pairs and applies it in one pass to both plain and tagged text, writing results into COL_PREREPLACE_TEXT / COL_PREREPLACE_TAGGED_TEXT. A longest-first sort and re.escape guard against both substring shadowing and regex injection, and a test confirms there is no Alice→Bob→Carlos cascade.
• Prompt simplification: _format_rewrite_disposition_block now also filters out replace entities, and the <replacement_map> block is removed from the rewrite prompt entirely — the LLM only handles generalize, remove, and suppress_inference methods.
• Whitespace-normalization fallback added in both _get_replace_pairs and llm_replace_workflow.py to handle LLM-normalized Unicode whitespace variants (e.g. U+202F → U+0020) that would otherwise miss a match.

Confidence Score: 4/5

Safe to merge with one fix: the silent exception fallback in _apply_direct_replacements can leave replace-method entities completely unprotected in the LLM output.

The programmatic pre-replacement approach is solid and the single-pass regex correctly avoids cascade. However, if _apply_direct_replacements throws, the fallback passes unmodified PII-containing text to the LLM while _format_rewrite_disposition_block has already stripped replace entities from the disposition block and the prompt no longer carries a replacement map — meaning those entities reach the final output with no anonymization applied at all.

src/anonymizer/engine/rewrite/rewrite_generation.py — specifically the exception handling in _apply_direct_replacements (lines 228-231).

Important Files Changed

Filename	Overview
src/anonymizer/engine/rewrite/rewrite_generation.py	Core change: adds _apply_direct_replacements (single-pass regex) and _get_replace_pairs; removes LLM replacement map from prompt. Silent exception fallback in _apply_direct_replacements leaves replace-method entities unprotected because _format_rewrite_disposition_block already excluded them from the LLM's disposition block.
src/anonymizer/engine/rewrite/repair.py	Swaps COL_TEXT for COL_PREREPLACE_TEXT as the repair baseline; the repair LLM now compares against pre-replaced text rather than raw PII, which is correct behavior.
src/anonymizer/engine/constants.py	Replaces COL_REPLACEMENT_MAP_FOR_PROMPT with COL_PREREPLACE_TEXT and COL_PREREPLACE_TAGGED_TEXT constants; straightforward rename.
src/anonymizer/engine/replace/llm_replace_workflow.py	Adds a whitespace-normalized fallback lookup so replacement map entries that differ only in Unicode whitespace from detected entity values are still matched; clean and well-scoped.
tests/engine/test_rewrite_generation.py	Tests updated and expanded: covers passthrough, multi-occurrence, no-cascade (Alice→Bob→Carlos), unicode-whitespace fallback, and unmatched-entity warning. Good coverage of the new logic.

_{Reviews (4): Last reviewed commit: "fix: normalize Unicode whitespace when f..." | Re-trigger Greptile}

[greptile-apps]

Silent exception leaves replace entities completely unprotected

When the try block throws, plain_text and tagged_text fall back to the originals (with PII), but _format_rewrite_disposition_block has already run and excluded all replace entities from COL_REWRITE_DISPOSITION_BLOCK. The LLM prompt also no longer contains a replacement map. The net effect is that on any exception the LLM sees PII-containing text with zero instructions to anonymize the affected entities — they flow through to COL_REWRITTEN_TEXT completely unprotected.

In the old code the LLM had the replacement map as a fallback; that safety net no longer exists. Consider either re-raising (let the record fail explicitly) or, at minimum, logging at ERROR level and setting a failure flag so downstream callers can skip or quarantine the row rather than emit unprotected PII.