refactor(sdk): migrate background-knowledge-demands and meaning-directness onto the contract factory

[adnanrhussain]

Both contracts are structurally identical to organizational-structure and purpose-clarity, which already run on the factory: text + grade_level in, one Google step, one flesch_kincaid_grade preprocessing entry. The two bespoke files restated all of that in code. 451 lines deleted, 72 added; each evaluator goes from 235 lines to 66.

Closes 2 of the 3 PREPROCESSING_GAPS entries. vocabulary-complexity stays, because it is multi-step and cannot move until step condition semantics are settled.

Three observable changes

Everything else — model, temperature, grades, prompts, error wrapping, telemetry shape — the factory already derives from the same contract, so it is unchanged.

1. fk_score now uses the declared implementation. These two called the hand-rolled calculateFleschKincaidGrade; the contract declares text-readability.fleschKincaidGrade rounded to 2dp, which is what the factory runs and what their four siblings already used. For the test text in meaning-directness.test.ts that is 6.8 rather than 6.14.

   Neither implementation matches Python textstat, and the declared one is the further out of the two. That is not made worse here — it is consolidated: all five single-step FK evaluators now share one code path, so the fix in feat(sdk): add a textstat-exact Flesch-Kincaid and a benchmark report #236 becomes a single change in preprocessing.ts instead of five migrations. feat(sdk): add a textstat-exact Flesch-Kincaid and a benchmark report #236 is parked on legal review.

2. Telemetry stage names now match the contract. smk_evaluation → evaluate_background_knowledge_demands, and conventionality_evaluation → evaluate_meaning_directness. Both old names came from earlier evaluator names and appeared in no contract. (sentence-structure still has this defect and is not touched here.)

3. supportedGrades is derived rather than restated. Both files hardcoded ['3'…'12']; the schemas declare exactly that, so the values do not change.

A vacuous test, fixed

meaning-directness.test.ts asserted the prompt contained an FK score with toMatch(/\d+(\.\d+)?/). The grade level in the same prompt satisfies that, so it passed with preprocessing switched off entirely. It now pins the exact value the declared implementation produces, and that the placeholder was substituted at all.

Confirmed load-bearing: forcing the old hand-rolled value through runPreprocessingStep fails it.

Validation

• typecheck, lint, test:unit (1080), build, scripts/check.py
• Live: 20 tests pass across both integration suites with the new fk_score, so the change did not move the declared score expectations
• The gap allowlist self-cleaned as designed: with the migration in place but the entries still listed, the inverted assertion failed both evaluators — they were removed only after that
• Verified no tests were lost, per file, against main (1080 both sides)
• Deleted src/prompts/background-knowledge-demands/ and src/prompts/meaning-directness/, whose only importers were the two rewritten files
• Mutation testing on single-step.ts (72.6%) and preprocessing.ts (93.1%) — the code now carrying these two evaluators. Every surviving mutant is a log message, an error-message separator, or a fallback literal on a path no contract reaches; no behavioural mutant survived. Asserting log wording would be brittle and protect nothing.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Copilot]

Pull request overview

Refactors the TypeScript SDK’s background-knowledge-demands and meaning-directness evaluators to use the shared single-step contract factory, aligning preprocessing (FK), telemetry stage naming, and supported grade derivation with what the contracts declare.

Changes:

• Migrated both evaluators to defineSingleStepEvaluator, removing bespoke prompt/flow code and relying on contract-derived preprocessing and metadata.
• Tightened the meaning-directness unit test to assert the exact {fk_score} substitution result and ensure the placeholder is replaced.
• Updated the registry conformance gap list to remove these evaluators from PREPROCESSING_GAPS (leaving only vocabulary-complexity).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sdks/typescript/src/evaluators/student-facing-text/ela-reading/meaning-directness.ts	Refactored evaluator to single-step factory with contract-based prompts/preprocessing.
sdks/typescript/src/evaluators/student-facing-text/ela-reading/background-knowledge-demands.ts	Refactored evaluator to single-step factory with contract-based prompts/preprocessing.
sdks/typescript/src/prompts/meaning-directness/index.ts	Removed bespoke prompt wrapper module (now using factory-rendered templates).
sdks/typescript/src/prompts/background-knowledge-demands/index.ts	Removed bespoke prompt wrapper module (now using factory-rendered templates).
sdks/typescript/tests/unit/evaluators/meaning-directness.test.ts	Strengthened assertion for FK score substitution to avoid a vacuous regex match.
sdks/typescript/tests/unit/registry-conformance.test.ts	Updated PREPROCESSING_GAPS list/comments to reflect the migration.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.