Bump sqd-query-engine to 6df565a, version 2.16.0-rc.1 - #76
Open
define-null wants to merge 3 commits into
Open
Conversation
Move the experimental query engine from 54b9020 (pinned since the initial integration in July) to the current upstream master tip, and start the 2.16.0 release candidate line. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The engine pin moves to the current master tip, which groups a table's keys into `request`, `output` and storage blocks and closes both public surfaces: a column is filterable only if `request.filters` names it and selectable only if `output.fields` does. The fixtures still spoke the flat layout — `query_name`, `field_name`, `json_encoding` — and the loader rejects an unknown key rather than ignoring it, so every test that parsed one failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MEHFzAGrwxmu429ck2oDk9
A parse fault was a verdict on the whole staged bundle, so one stale catalog stopped the worker serving every kind — including the kinds it holds chunks of and could answer. A schema is load-bearing only for the chunks that name it, and `DatasetsIndex` already refuses an assignment reaching for an id the bundle does not offer, so dropping the schema puts the refusal where the dependency actually is. The store path had this right already: `read_store` skips a file it cannot read and lets a later bundle reinstall the id. A bundle in which nothing parses is still permanent — there is no query it could serve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MEHFzAGrwxmu429ck2oDk9
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bump
sqd-query-engine(the experimental dynamic query engine) from54b9020to6df565a, the current upstream master tip, and set the worker version to2.16.0-rc.1. The previous pin dated from the initial integration in July and had been orphaned upstream (not reachable from master).Upstream range: 52 commits. Full list: subsquid/sqd-query-engine@54b9020...6df565a
The catalog format changed
Two of those commits reshape the dataset description the engine loads, and
DatasetDescriptionisdeny_unknown_fields, so a stale catalog is a load error rather than a silently ignored key:c31a4a0makesfiltersa required, closed list — a table that accepts none saysfilters: [].b72c404groups a table's keys intorequest(what a client may send),output(what a client may see) and the storage keys.query_name→request.name,field_name→output.name,json_encoding: hex→encoding: hex_bytes;stats,dictionary,childrenandparent_keyare gone.Both surfaces are now closed: a column is filterable only if
request.filtersnames it, and selectable only ifoutput.fieldsdoes.The four inlined test catalogs still spoke the flat layout, which is what the first CI run caught — 23 failures, all of them
missing field 'filters'or an unknown key. They are rewritten here; no production code needed changes.Note for whoever ships this: the catalogs the worker fetches at runtime have to be republished in the new layout too, or the worker hits the same load error outside the tests.
One bad schema no longer takes the bundle down
Separate commit. Parsing a staged schema failed the whole bundle, which is how a stale catalog for
a kind this worker holds no chunk of would stop it serving the kinds it does hold.
load_dirnowdrops the schema it cannot parse, warns, and counts it (
schema_bundle_schemas_skipped), leavingthe rest installed.
That is not a loosening: the id is left out of
bundle_ids, andDatasetsIndexalready refuses anassignment whose chunks reach for an id the bundle does not offer — so the refusal lands on the
assignment that actually depends on the schema rather than on every assignment. A bundle in which
nothing parses stays a permanent fault, since no query could be served from it. The store path had
this shape already:
read_storeskips a file it cannot read and lets a later bundle reinstall theid.
Test plan
cargo check --all-targets --lockedcargo test --all-features --lib --bins --test state_pbt --test state_regressione2e,query_surface,query_concurrency), default and--all-features🤖 Generated with Claude Code
https://claude.ai/code/session_01MEHFzAGrwxmu429ck2oDk9