fix(ts-sdk)!: bound text at 1-10,000 chars measured as the caller sent it - #202
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR brings the TypeScript SDK’s text-length handling into §4.1 conformance by (1) capping validated text at the spec default of 10,000 characters and (2) ensuring telemetry/log-reported text_length_chars reflects the same trimmed text length that validation actually bounds.
Changes:
- Reduced
VALIDATION_LIMITS.MAX_TEXT_LENGTHfrom 100,000 to 10,000 and kept validation based ontrim()ed text. - Introduced
textLengthChars(text)and migrated evaluators to use it so logging/telemetry consistently report trimmed length. - Added unit tests covering trimmed-length reporting (success and error paths) and the 10,000-char cap behavior.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| sdks/typescript/src/evaluators/base.ts | Lowers max text length to 10,000, adds textLengthChars(), and uses it in validation logging; validation continues to use trimmed length. |
| sdks/typescript/src/evaluators/conventionality.ts | Uses textLengthChars() for start/success/error telemetry/log length reporting. |
| sdks/typescript/src/evaluators/vocabulary.ts | Uses textLengthChars() for consistent telemetry/log length reporting across paths. |
| sdks/typescript/src/evaluators/text-complexity.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/smk.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/sentence-structure.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/purpose.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/organizational-structure.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/intertextuality.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/grade-level-appropriateness.ts | Uses textLengthChars() for consistent telemetry/log length reporting. |
| sdks/typescript/src/evaluators/math/standards-alignment.ts | Uses textLengthChars() for consistent telemetry/log length reporting of question. |
| sdks/typescript/tests/unit/evaluators/text-length-reporting.test.ts | Adds tests verifying trimmed-length telemetry reporting and enforcing the new 10,000-char max constraint. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
adnanrhussain
force-pushed
the
ahussain/sdk-spec-text-limits
branch
from
August 27, 2026 02:23
66e86ab to
d68f2d0
Compare
adnanrhussain
force-pushed
the
ahussain/sdk-spec-text-limits
branch
from
August 27, 2026 02:35
d68f2d0 to
8e9cad5
Compare
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings §4.1 bounds to their spec defaults and removes input normalization.
Limits
min10 → 1,max100,000 → 10,000. The SDK defaults carry no product opinion: the minimum excludes empty input and nothing else. Meaningful minimums belong to each evaluator's input schema — whichmath-question-alignmentalready does, readingmaxLengthstraight from itsinput_schema.json.One length, measured on the caller's text
The SDK previously validated the trimmed length while sending the original text to the model, so the bound enforced described a string the model never saw. Telemetry reported a third thing, agreeing with neither.
text_length_charstrimsurvives only as a validity test — whitespace-only input is rejected outright, never repaired. Padding is the caller's to remove; absorbing it silently means never telling them their data is malformed, and billing them for characters we pretended weren't there.Breaking
InputValidationError, down from 100,000.received 10,004 characters). A trailing newline on an otherwise-maximal text is enough — deliberately, since that text is what reaches the model.min: 1means anything non-empty passes; evaluators that need a real floor must declare it.Nothing in the repo is affected: the largest fixture input is 2,910 characters.
Known dead branch
With
min: 1the minimum check is unreachable —text.length < 1implies the empty string, which the whitespace-only check rejects first. Mutation testing confirms it:if (false)survives and the block reports as uncovered.Left in place because it goes live the moment an evaluator declares a higher minimum, and because deleting it would mean re-deriving the bound later. Flagged here so the permanent coverage gap is not mistaken for a missing test — it should not be papered over with one.
Spec follow-up
§4.1 in #162 says "Trim … then validate", "Trimmed length < min", "Trimmed length > max", and "
text_length_chars… is the trimmed length". This PR contradicts all four. §4.1 also never says whether the trimmed text is the input or only the measurement — the ambiguity that let the SDK validate one string and send another. Both need fixing before #162 merges.Tests
12 in a new file, plus 6 existing suites updated where they asserted the old
min: 10.Verified load-bearing by mutation, not just by passing: reintroducing the trim in
validateTextfails 1 test, and in an evaluator's telemetry call fails 3 — so the validation and reporting paths are pinned independently. One test asserts the reported length equals the text the prompt actually carried, which is the invariant that was broken.668/668 pass ·
tsc --noEmitclean · lint 0 errors.