Skip to content

Bump sqd-query-engine to 6df565a, version 2.16.0-rc.1 - #76

Open
define-null wants to merge 3 commits into
masterfrom
bump-experimental-query-engine
Open

Bump sqd-query-engine to 6df565a, version 2.16.0-rc.1#76
define-null wants to merge 3 commits into
masterfrom
bump-experimental-query-engine

Conversation

@define-null

@define-null define-null commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Summary

Bump sqd-query-engine (the experimental dynamic query engine) from 54b9020 to 6df565a, the current upstream master tip, and set the worker version to 2.16.0-rc.1. The previous pin dated from the initial integration in July and had been orphaned upstream (not reachable from master).

Upstream range: 52 commits. Full list: subsquid/sqd-query-engine@54b9020...6df565a

The catalog format changed

Two of those commits reshape the dataset description the engine loads, and DatasetDescription is deny_unknown_fields, so a stale catalog is a load error rather than a silently ignored key:

  • c31a4a0 makes filters a required, closed list — a table that accepts none says filters: [].
  • b72c404 groups a table's keys into request (what a client may send), output (what a client may see) and the storage keys. query_namerequest.name, field_nameoutput.name, json_encoding: hexencoding: hex_bytes; stats, dictionary, children and parent_key are gone.

Both surfaces are now closed: a column is filterable only if request.filters names it, and selectable only if output.fields does.

The four inlined test catalogs still spoke the flat layout, which is what the first CI run caught — 23 failures, all of them missing field 'filters' or an unknown key. They are rewritten here; no production code needed changes.

Note for whoever ships this: the catalogs the worker fetches at runtime have to be republished in the new layout too, or the worker hits the same load error outside the tests.

One bad schema no longer takes the bundle down

Separate commit. Parsing a staged schema failed the whole bundle, which is how a stale catalog for
a kind this worker holds no chunk of would stop it serving the kinds it does hold. load_dir now
drops the schema it cannot parse, warns, and counts it (schema_bundle_schemas_skipped), leaving
the rest installed.

That is not a loosening: the id is left out of bundle_ids, and DatasetsIndex already refuses an
assignment whose chunks reach for an id the bundle does not offer — so the refusal lands on the
assignment that actually depends on the schema rather than on every assignment. A bundle in which
nothing parses stays a permanent fault, since no query could be served from it. The store path had
this shape already: read_store skips a file it cannot read and lets a later bundle reinstall the
id.

Test plan

  • cargo check --all-targets --locked
  • cargo test --all-features --lib --bins --test state_pbt --test state_regression
  • Conformance tier (e2e, query_surface, query_concurrency), default and --all-features
  • New: a bundle carrying one unparseable schema installs the other two and offers only their ids
  • New: a bundle in which nothing parses is still refused as permanent, with nothing left in the store

🤖 Generated with Claude Code

https://claude.ai/code/session_01MEHFzAGrwxmu429ck2oDk9

define-null and others added 2 commits September 4, 2026 15:45
Move the experimental query engine from 54b9020 (pinned since the
initial integration in July) to the current upstream master tip, and
start the 2.16.0 release candidate line.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The engine pin moves to the current master tip, which groups a table's
keys into `request`, `output` and storage blocks and closes both public
surfaces: a column is filterable only if `request.filters` names it and
selectable only if `output.fields` does. The fixtures still spoke the
flat layout — `query_name`, `field_name`, `json_encoding` — and the
loader rejects an unknown key rather than ignoring it, so every test that
parsed one failed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MEHFzAGrwxmu429ck2oDk9
@mo4islona mo4islona changed the title Bump sqd-query-engine to 6e3f691, version 2.16.0-rc.1 Bump sqd-query-engine to 6df565a, version 2.16.0-rc.1 Sep 5, 2026
A parse fault was a verdict on the whole staged bundle, so one stale
catalog stopped the worker serving every kind — including the kinds it
holds chunks of and could answer. A schema is load-bearing only for the
chunks that name it, and `DatasetsIndex` already refuses an assignment
reaching for an id the bundle does not offer, so dropping the schema puts
the refusal where the dependency actually is.

The store path had this right already: `read_store` skips a file it cannot
read and lets a later bundle reinstall the id. A bundle in which nothing
parses is still permanent — there is no query it could serve.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MEHFzAGrwxmu429ck2oDk9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants