Repository navigation
fix: Report one finding per defect in schema validation. - #73
Merged
Merged
Conversation
A document that failed schema validation was reported with the actual defect plus the consequences of the branch the validation failed on. A bounded context missing `language` came with two unknown-field findings for valid fields; a rejection carrying a removed field came with the findings of the events alternative it never intended. Two mechanisms produced this, and both are filtered out now: - Closing a document with `unevaluatedProperties` turns every field of a failed `if`/`then` branch into an unknown field, because a failed branch drops its annotations. Such a field is reported only when no failing branch declares it, which keeps genuine typos and drops the fallout. - `oneOf` reports the causes of every alternative. The branch the document aims at is identified by how much of it the document exhibits: required properties that are present, and constants it matches. Only that branch is reported; when the document resembles the branches equally, all of them still are. Both decisions need the schema itself, not just the error, so the decoded schema document is kept next to every compiled validator and addressed by the JSON pointer inside each error's schema URL. A mutation sweep over the kinds whose schemas combine branches guards the direction that matters: filtering must never silence a defect outright. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GYE7CQJ77q3Ne4QMgUFT1
Two schema keywords fail without nested causes, and the translation only ever walked causes, so both produced no diagnostic at all. `not` is how the core schema forbids a field depending on the value of a sibling: an Actor of type `human` has no `backedBy`. Such a document was reported with three unknown-field findings for valid fields before follow-on errors were filtered out, and with nothing at all after. It now yields one finding that names the field and the condition, both read back out of the schema because the failure itself carries neither. A `oneOf` whose branches match more than once fails the same way, since there is no failing branch to point at. No embedded schema can produce it today - every `oneOf` in them separates its branches by a contradicting constant or by a closed object - but the failure mode is silence, so it is covered against a future schema change. Beyond those two, a wrapper error that produced nothing now reports itself. A document the schema rejected must not pass in silence, whatever the reason the translation had nothing to say. The finding about a forbidden field points at the key rather than at its value, which for a list starts on the following line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GYE7CQJ77q3Ne4QMgUFT1
Two ways were left for one defect to draw more than one finding. Alternatives that fail on the same field reported it once each. An Event whose `scope` misspells `aggregate` matches neither alternative equally well, so both are reported, and both name the misspelled field. The same sentence at the same position twice tells the reader nothing the first one did not, so a repeated diagnostic is dropped. An alternative that is itself a set of alternatives states its requirements one level down, where the comparison against its siblings could not see them. It scored nothing, lost to any flat sibling, and left the comparison tied, which reports everything. A Process Manager whose reaction misspells `event` drew six findings that way. Affinity now reaches through such a branch and takes the score of the variant the instance comes closest to, which leaves the two findings that describe the defect. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GYE7CQJ77q3Ne4QMgUFT1
The package comment described the branch comparison and the filtering as they were two commits ago. It now covers scoring a branch that is itself a set of alternatives, dropping a repeated diagnostic, and reporting the failures that carry no nested cause. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011GYE7CQJ77q3Ne4QMgUFT1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #72
What changed
A document that failed schema validation was reported with the actual defect plus the consequences of the branch the validation failed on. A bounded context declaring a
ubiquitousLanguagewithoutlanguageproduced three findings, two of which named valid fields; a Given-When-Then rejection carrying the removedreasonfield produced four, of which only the last described the document. A reader who followed the first finding was sent in the wrong direction, and with the 0.15.0 migration those two shapes are exactly what upgrading users meet.One defect now yields one finding. Never more, and never none.
Fewer findings: dropping the follow-on errors
unevaluatedPropertiesfallout. The core schema dispatches kinds throughallOfwithif/thenand closes the document withunevaluatedProperties: false. A failed branch drops its annotations, so every field that branch would have evaluated counts as unevaluated and reads as an unknown field. Anunknown fieldfinding is now dropped when some failing branch below the same schema position declares that property, and kept otherwise. A genuine typo appears in no branch and survives: a bounded context missinglanguagethat also carriesbogusFieldstill reports both, and an Event whosedatais misspelled asdatreports the missingdatanext to the unknowndat.oneOf. The validator reports the causes of every alternative, so the findings of branches the document never intended stand next to the real one. The branch the document aims at is identified by how much of it the document already exhibits: one point per required property that is present, one more per property whoseconstit matches. The highest unique score wins and only that branch is reported. A tie reports every branch, because then the document genuinely identifies none, which is the accurate answer for something likethen: {}.Both decisions need the schema itself, not just the error. The library exposes no
ErrorKindforunevaluatedPropertiesand no lookup of a subschema by pointer, so the decoded schema document is kept next to every compiled validator and addressed by the JSON pointer inside each error'sSchemaURL.More findings: breaking the silence
Two keywords fail without nested causes, and a translation that only walks causes turns such a failure into no diagnostic at all.
notis how the schema forbids a field depending on the value of a sibling: an Actor of typehumanhas nobackedBy. That document was reported with three unknown-field findings for valid fields before the filters above, and with nothing at all after them, so this is a regression the first commit would have introduced on its own. It now yields one finding naming the field and the condition, both read back out of the schema because the failure carries neither:A
oneOfwhose branches match more than once fails the same way, since there is no failing branch to point at. All 23oneOfoccurrences across the three schemas were checked pairwise: every pair is separated by a contradictingconstor by a closed object, so no embedded schema can produce this today. It is covered because the failure mode is silence and a schema change could introduce it unnoticed.Beyond those two, a wrapper error that produced nothing now reports itself, whatever the reason the translation had nothing to say.
Both phrasings and the choice of
constraint-violationover a new rule ID were signed off before implementing. The existing entry for that ID already covers them word for word: "a combination of fields the schema forbids."Fewer findings, again
Two ways were left for one defect to draw more than one finding.
Alternatives failing on the same field reported it once each. An Event whose
scopemisspellsaggregatefits neither alternative better, so both are reported, and both name the misspelled field. A diagnostic that repeats one already in the list is now dropped.An alternative that is itself a set of alternatives states its requirements one level down, where the comparison against its siblings could not see them. It scored nothing, lost to any flat sibling, and left the comparison tied, which reports everything. A Process Manager whose reaction misspells
eventdrew six findings that way. Affinity now reaches through such a branch and takes the score of the variant the instance comes closest to.Why it is safe
Filtering findings has one dangerous direction: silencing a defect outright. A mutation sweep guards it. It renames one field at a time across the kinds whose schemas combine branches and insists that a finding naming that field survives. Run over a wider corpus of 18 documents covering every such kind, all 435 mutations that make a document invalid still yield a finding that names the mutated field, none is silenced and none is repeated. The finding count over those 435 drops from 2647 to 803. Both filters were verified by sabotage: forcing either one to over-suppress fails the sweep.
The sweep did not catch the
notcase, because renaming a field never adds a forbidden one. That case has its own test, and the wrapper fallback now rules the failure mode out structurally.Open for review
The Linter Rules entries for
unknown-fieldandmissing-required-fieldwere checked, as the issue asked. Both still describe what the finding means, so they are unchanged. What changed is how many findings one defect produces, which belongs to neither entry, so one sentence in the introduction of the Pipeline Diagnostics section states it instead. That sentence is the only unprompted user-facing wording in this PR and is easy to drop.🤖 Generated with Claude Code
https://claude.ai/code/session_011GYE7CQJ77q3Ne4QMgUFT1