fix: keep rewrite final judge in evaluate

[fallintoplace]

What changed

RewriteWorkflow.run() no longer tries to execute the final judge. The non-critical final judge path stays in RewriteWorkflow.evaluate(), which is the only place that has the right model-selection contract and is already documented as the place where judge_evaluation is added.

I also updated the affected rewrite workflow tests, the rewrite LLM call estimate, and the repo-level workflow notes so they match the actual behavior.

Why

The current run() path silently degrades before it ever reaches the adapter:

• it passes an unsupported evaluation= keyword into FinalJudgeWorkflow.columns()
• even after removing that keyword, run() still only has RewriteModelSelection, while the final judge needs the evaluate-side rewrite_judge alias

Because _run_final_judge() catches broadly, the failure gets hidden and looks like normal fallback behavior.

Impact

• run() now stays aligned with the public interface contract: rewrite judge output is added later via evaluate().
• rewrite runs stop carrying a dead final-judge path that can only fail and get swallowed.
• evaluate-side final judge fallback now only defaults judge_evaluation to None and preserves the needs_human_review value produced by the evaluation/repair flow.
• rewrite measurement no longer overcounts a final-judge LLM call during run().

Validation

• uv run --group dev pytest tests/engine/test_rewrite_workflow.py tests/interface/test_anonymizer_interface.py tests/test_measurement.py tests/engine/test_final_judge.py -q
• uv run --group dev ruff check src/anonymizer/engine/rewrite/rewrite_workflow.py src/anonymizer/measurement/metrics/llm_calls.py tests/engine/test_rewrite_workflow.py tests/test_measurement.py

[github-actions]

All contributors have signed the DCO ✍️ ✅
_{Posted by the DCO Assistant Lite bot.}

[greptile-apps]

Greptile Summary

This PR removes the broken final-judge execution path from RewriteWorkflow.run() and consolidates it exclusively in RewriteWorkflow.evaluate(). The old run() path silently swallowed the failure inside a broad except clause.

• rewrite_workflow.py: Removes _run_final_judge() from run(), updates its signature to EvaluateModelSelection, and drops the incorrect COL_NEEDS_HUMAN_REVIEW = True default in the exception handler.
• llm_calls.py and tests: LLM-call estimate for rewrite_final_judge corrected to 0; run()-path tests drop the now-removed judge mock returns.

Confidence Score: 5/5

The change removes a code path guaranteed to fail silently; the working judge path in evaluate() is untouched.

Every changed file is consistent: the dead run()-path judge call is gone, _run_final_judge() now accepts the correct model-selection type, the exception handler no longer incorrectly sets needs_human_review, and all tests and the LLM-call estimate are updated to match.

No files require special attention.

Important Files Changed

Filename	Overview
src/anonymizer/engine/rewrite/rewrite_workflow.py	Core fix: removes dead final-judge path from run(), updates _run_final_judge() signature to EvaluateModelSelection, drops incorrect COL_NEEDS_HUMAN_REVIEW default.
src/anonymizer/measurement/metrics/llm_calls.py	rewrite_final_judge estimate corrected to unconditional 0.
tests/engine/test_rewrite_workflow.py	Removes judge-step mock returns from all run()-path tests.
tests/test_measurement.py	Updates expected rewrite_final_judge estimate to 0.
AGENTS.md	Documentation updated to reflect FinalJudgeWorkflow runs only in evaluate().

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant Caller
    participant run as RewriteWorkflow.run()
    participant evaluate as RewriteWorkflow.evaluate()
    participant judge as _run_final_judge()

    Note over run: RewriteModelSelection (no rewrite_judge alias)
    Caller->>run: "run(df, selected_models=RewriteModelSelection)"
    run-->>Caller: RewriteResult (no judge columns)

    Note over evaluate: EvaluateModelSelection (has rewrite_judge alias)
    Caller->>evaluate: "evaluate(df, selected_models=EvaluateModelSelection)"
    evaluate->>judge: _run_final_judge(EvaluateModelSelection)
    judge-->>evaluate: judge_evaluation column
    evaluate-->>Caller: RewriteResult (with judge_evaluation)

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant Caller
    participant run as RewriteWorkflow.run()
    participant evaluate as RewriteWorkflow.evaluate()
    participant judge as _run_final_judge()

    Note over run: RewriteModelSelection (no rewrite_judge alias)
    Caller->>run: "run(df, selected_models=RewriteModelSelection)"
    run-->>Caller: RewriteResult (no judge columns)

    Note over evaluate: EvaluateModelSelection (has rewrite_judge alias)
    Caller->>evaluate: "evaluate(df, selected_models=EvaluateModelSelection)"
    evaluate->>judge: _run_final_judge(EvaluateModelSelection)
    judge-->>evaluate: judge_evaluation column
    evaluate-->>Caller: RewriteResult (with judge_evaluation)

Loading

_{Reviews (1): Last reviewed commit: "fix: keep rewrite final judge in evaluat..." | Re-trigger Greptile}

[fallintoplace]

I have read the DCO document and I hereby sign the DCO.

[memadi-nv]

looks great! Thanks for the fix. Just left a small nit comment.

[binaryaaron]

Reviewed the final-judge execution change and its interaction with evaluate(), needs_human_review, preview sizing, failed records, and LLM-call accounting.

Moving the holistic judge out of RewriteWorkflow.run() fixes the model-selection mismatch and keeps judge output informational, while evaluate() remains the explicit path for detection and final judging. The updated measurement estimate correctly reflects that anonymization no longer invokes the final judge.

Validation: the prospective merge onto current main passes 1,081 applicable tests, and Ruff is clean. No blocking findings. LGTM.