Skip to content

feat(service-analytics): serve a $field RLS rule by declining native SQL (#7598) - #7916

Merged
huangyiirene merged 3 commits into
mainfrom
claude/issue-7598-analytics-field-decline
Aug 12, 2026
Merged

feat(service-analytics): serve a $field RLS rule by declining native SQL (#7598)#7916
huangyiirene merged 3 commits into
mainfrom
claude/issue-7598-analytics-field-decline

Conversation

@huangyiirene

@huangyiirene huangyiirene commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Closes #7598

Implements the maintainer ruling of 2026-08-12 (comment 5261751132): Q1 = B, Q2 = A. Nothing here re-opens either.

Did the ruling's premises hold under measurement?

Yes — both of them, and one of them only after a fix.

premise verdict
canHandle can see the where and the read scope at that point ✅ Held. ctx.getReadScope and ctx.getCube are both on StrategyContext and both already used one method away in applyReadScope. The where is read through lowerAnalyticsWhere, so the authored array sugar (['amount','=',{$field:'budget'}]) is seen after parseFilterAST has lowered it — scanning the raw where would have missed it and sent that spelling to native SQL.
declining routes to a path that enforces the four #5222 rulings ✅ Held end to end — but not out of the box. Two defects sat on the road between the decline and the driver, and the ruling would have shipped a broken capability without them. Both are fixed here; see below.

Neither is grounds to send the card back. Recording them because the ruling rests on the earlier report's measurements and these two were not in it.

The two things that had to be fixed for B to actually work

  1. ObjectQLStrategy.convertFilter lowered equality BARE. case 'equals': return v0 is right for a literal ({amount: 5} is implicit equality, every backend reads it) and produces { amount: { $field: 'budget' } } for a reference — a field spec whose only key is $field, which no backend reads as an equality. That is [spec] parseFilterAST lowers ['a', '=', { $field: 'b' }] to a bare field spec no backend reads as equality #7597's defect at a different door, and it would have broken the capability on $eq — the single spelling compileCelToFilter emits most, while its five siblings worked. Fixed by branching on the comparand, exactly as [spec] parseFilterAST lowers ['a', '=', { $field: 'b' }] to a bare field spec no backend reads as equality #7597's fix did.
  2. The where door's NULL-safe $ne guard (非否定路径上的 $ne / $nin / $notContains:driver-sql 排除 NULL 行,driver-memory / formula 返回它们(#5146 只裁定了 $not) #5298) widens a cross-field comparison. { amount: { $ne: {$field:'budget'} } } lowered to {$or:[{amount:null},{amount:{$ne:ref}}]}, which admits the both-NULL row — the row the corpus, both SQL drivers and @objectstack/formula all EXCLUDE, because it satisfies the inner $eq. The guard is right for a literal (a NULL column must satisfy $ne: 5) and wrong for a reference, whose NULL semantics are decided by the referent too and are already written total by applyCrossFieldComparison. Six corpus cases turned on it. Widening a $ne whose producer is an RLS rule is the direction that matters, so this is stated as a measurement rather than a tidy-up.

Which refusal arm is now dead, and which is still someone's backstop

The card asked for this explicitly. The conclusion:

DEAD — removed. filter-normalizer.ts's scalar-comparand arm (#7694's interim). It is not merely unreachable, it is incompatible with the ruling: the gate sits inside fieldLeaves, the one leaf producer for all three consumers of the normalized tree — NativeSQLStrategy, the /analytics/sql echo, and ObjectQLStrategy.convertFilter, the engine path. Refusing there refuses the very execution B routes to. Reverse-verified below.

STILL LIVE — kept, and now load-bearing rather than inherited. filter-normalizer.ts's $between-endpoint arm. This door lowers $between into a gte leaf and an lte leaf, so a routed endpoint reference would reach driver-sql wearing an operator #5222 compiles. { amount: { $between: [{$field:'budget'}, 100] } } would then quietly succeed on the analytics face while CROSS_FIELD_REFUSALS pins it refused on both SQL drivers and #7596 has removed the position from FieldReferenceSchema entirely. It gets its own wording now (fieldReferenceBetweenBoundMessage), because its repair differs from the read-scope one — #5240 in the direction that separates.

STILL LIVE — kept whole. read-scope-sql.ts's arm. applyReadScope no longer reaches it (the decline runs first), but ObjectQLStrategy.generateSql does — that is the /analytics/sql echo, so this refusal is the echo's decline for the scope half. It is also a public export of the package. Envelope untouched per Q2 = A.

NEW, and deliberately unreachable — named rather than hidden. NativeSQLStrategy.assertNoCrossFieldComparison, a one-line fail-closed backstop at the door that binds, using the same walk over the same two inputs as the decline, so the two cannot drift. Kept because the failure mode if routing ever regresses is not an error but a silent bindtoSqlBindValue JSON-stringifies the reference and the statement compiles perfectly. It throws a bare Error (undeclared 500), not INVALID_FILTER/400, following buildFilterClauseSql's #5333 precedent for the same class: an arrival is drift between our own gate and our own emitter, not a caller-shaped mistake.

What was measured

  • Files/lines: 9 files inside packages/services/service-analytics, plus the changeset and the lockfile.

    • strategies/native-sql-strategy.ts — the canHandle ruling comment + decline, crossFieldComparisonIn, the backstop
    • strategies/filter-normalizer.ts — scalar arm removed, $between arm kept and re-worded, operatorIsNullTotal reference arm, invalidFilterError exported
    • strategies/objectql-strategy.ts — echo decline, equals comparand branch
    • comparand-shape.tsfindCrossFieldComparand, the two refusal sentences
    • read-scope-sql.ts — docs + $between wording only; no envelope, code, or status change
    • analytics-service.ts — a diagnostic only, see the second commit
    • package.json (devDependency), and the two test files
  • Suite: pnpm --filter @objectstack/service-analytics test73 files / 1655 tests passed (1558 before; +97). Blast radius, all against built deps: @objectstack/rest 1477/1477, @objectstack/runtime 2096/2096 (this includes analytics-query-read-scope-withhold.test.ts, the analytics dataset 路由的 message 正则兜底没有退休时间表:六族拒收仍靠措辞分类,改一个字就换一个 HTTP 码 #5367 end-to-end pin, and analytics-read-scope-refusal-envelope.test.ts), @objectstack/driver-sql 1399 passed / 50 skipped, @objectstack/driver-sqlite-wasm 374/374.

  • ESLint --no-inline-config over all changed files → no output. Local gates: check-engine-double-contractOK — 164 pinned, 133 DEBT, 2 exempt; check-changeset-fixed → in sync with 69 packages; check-nul-bytes → OK, 7238 files; check-adr-0087-registration --base origin/main → no declared-breaking changeset.

  • Reverse verification — all three directions predicted IN WRITING first, in the suite header, before any of them was run:

    1. Restore the scalar arm → predicted: kills the unit lowering assertions and takes the end-to-end suite with them. Measured: 75 failed (44 in cross-field-reference-refusal, 31 in cross-field-engine-fallback). That coupling is the whole argument for removing it.
    2. Remove the $between arm → predicted: only $between cells, and the end-to-end ones red by resolving. Measured: 6 failed (4 unit + 2 end-to-end), and the two end-to-end ones read expected the analytics face to refuse this query, but it returned [] — the laundering demonstrated rather than argued.
    3. Remove the operatorIsNullTotal reference arm → predicted: exactly the three $ne class pairs, the self-$ne control and the two $not-of-$eq cases, plus one unit cell. Measured: 7 failed, exactly those.

    All three restored by inverse edit inside the worktree (never git stash); grep confirms zero markers remain and the suite is green again.

Acceptance criteria

  1. cross-field-engine-fallback.test.ts drives the whole CROSS_FIELD_CASES corpus through AnalyticsService against a real SqliteWasmDriver, once as a where and once as a read scope, asserting the corpus's own declared id lists — rows, not routing. executeRawSql is supplied and nativeSql is true, so native SQL wins every query unless it declines; asserting it was never called is the decline's measurement, and the literal-comparand controls (which do reach it) stop that being vacuous.
  2. CROSS_FIELD_REFUSALS driven whole through the same face. Each case asserted refused with INVALID_FILTER/400, labelled by who refuses — routed-to-the-driver for the scalar positions (the four rulings) or refused at the analytics door for the rest. A guard asserts all four rulings are actually represented among the routed cases, so the loop can't go vacuously green if the corpus drifts.
  3. /analytics/sql declines, with tests for both halves (whereINVALID_FILTER/400; read scope → READ_SCOPE_COMPILE_FAILED/500), that the refusal is total (no SELECT half-rendered), and the paired control that /analytics/query still serves the same query and returns rows.
  4. ✅ The canHandle comment records the ruling verbatim, including the acknowledged new behaviour.
  5. ⚠️ No change needed, and this is a correction to the card's premise. comparand-shape-refusal.test.ts's "converging with driver-sql" comment is not stale: its pinned case is {name: {$contains: {$field: 'status'}}}, a LIKE-family operator [spec] SqlDriver 将 $field 编译为列对列比较(cross-field comparison push-down) #5222 left in its own refusal arm (a column-side LIKE pattern cannot be metacharacter-escaped portably). fix(analytics): refuse a { $field } comparand on both SQL-lowering doors instead of binding it (#7598) #7694's seat already re-measured exactly this and wrote the finding into the comment, which now states it at length. It still converges after this PR, so the test and its comment are untouched. I did not change what it asserts.
  6. ✅ N/A — the new engine double implements executeAggregate only (a read). check:engine-double-contract green.
  7. ✅ Every refusal assertion checks code and status. The one exception is the new unreachable backstop, which is deliberately an undeclared 500 per §"NEW, deliberately unreachable" above.
  8. packages/spec untouched — the ruling forbids it and nothing here needed it.

Deliberately not done

The second commit

The decline creates exactly one configuration that previously (wrongly) compiled: a host advertising nativeSql without an aggregate bridge now reaches resolveStrategy's dead end for a $field-carrying filter, and the bare "no strategy can handle this cube" reads like a broken driver when every other query on that cube still works. Measured not reachable from AnalyticsServicePlugin — its default queryCapabilities derives both flags from the bridges it wired and auto-wires the aggregate bridge from the engine — so only a host that overrides queryCapabilities by hand can hit it. Named anyway, with a narrowness control (a literal filter on the same deployment still runs).

Notes for the reviewer

  • @objectstack/driver-sqlite-wasm added as a devDependency of service-analytics so the fallback suite can end at a real SQL engine in-process. Test-only, no runtime edge; @objectstack/driver-sql was already there on the same precedent (fix(analytics): refuse a { $field } comparand on both SQL-lowering doors instead of binding it (#7598) #7694 / service-automation). Validate Package Dependencies is green.
  • The local branch in my worktree is wt/issue-7598-analytics-field-decline, pushed to claude/issue-7598-analytics-field-decline — the primary checkout already held the designated branch name, and detaching it was not permitted in this environment. The remote branch is the designated one.

🤖 Generated with Claude Code

https://claude.ai/code/session_01EWcRLiMFvDoQV3zS2LgEHH

…e SQL (#7598)

Implements the maintainer ruling of 2026-08-12 (Q1 = B).
`NativeSQLStrategy.canHandle` now declines a query whose `where` or read
scope carries a `{ $field }` reference in a scalar comparand position, so
the query routes to the ObjectQL/engine path and `driver-sql` compiles the
comparison under the four #5222 rulings with the metadata it owns. The
capability becomes available on the analytics face and the security rules
stay in exactly one place — no `StrategyContext` hook, no `packages/spec`
change, no second copy of the rulings here.

The ruling site carries the ruling as a comment, as it required.

Refusal arms, after the routing:

- `filter-normalizer`'s scalar arm (#7694's interim) is REMOVED. It sat in
  `fieldLeaves`, the one leaf producer for all three consumers of the
  tree — including the engine path — so it refused the execution the
  routing exists to reach. Reverse-verified: restoring it turns both
  suites red (75 cells).
- its `$between`-endpoint arm STAYS, and is now load-bearing rather than
  inherited: this door splits `$between` into `gte` / `lte`, so a routed
  endpoint reference would reach the driver under an operator #5222
  compiles and succeed here alone. It gets its own wording.
- `read-scope-sql`'s arm STAYS, whole. Its remaining caller is
  `ObjectQLStrategy.generateSql` — the `/analytics/sql` echo, which also
  declines a `$field`-carrying `where` rather than half-rendering it.
- the envelope is untouched: `READ_SCOPE_COMPILE_FAILED` / 500 with the
  message withheld, per #5367 as re-affirmed by Q2 = A.

Two fixes the routing needed, both found by measurement:

- `ObjectQLStrategy.convertFilter` lowered an equality comparand bare,
  producing `{ amount: { $field: … } }` — a field spec no backend reads as
  an equality (#7597's defect, other door). It now branches on the
  comparand.
- a reference comparand no longer takes the NULL-safe `$ne` guard (#5298),
  which is right for a literal and wrong for a reference: measured, it
  admitted the both-NULL row that the shared corpus, both SQL drivers and
  the in-memory evaluator exclude.

Tests: the shared `CROSS_FIELD_*` corpus driven end-to-end through
`AnalyticsService` against a real SQLite engine — right rows for a
`$field` `where` AND a `$field` read scope, the four rulings still biting
on the fallback path, and the echo declining.

Closes #7598

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWcRLiMFvDoQV3zS2LgEHH
@vercel

vercel Bot commented Aug 12, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 12, 2026 4:29am

Request Review

@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/service-analytics.

5 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/api/data-api.mdx (via @objectstack/service-analytics)
  • content/docs/api/index.mdx (via @objectstack/service-analytics)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/service-analytics)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/service-analytics)
  • content/docs/plugins/packages.mdx (via @objectstack/service-analytics)

3 release-owned page(s) also reference the affected code. These are read-only:

  • content/docs/releases/implementation-status.mdx (via @objectstack/service-analytics)
  • content/docs/releases/v17.mdx (via @objectstack/service-analytics)
  • content/docs/releases/v9.mdx (via @objectstack/service-analytics)

content/docs/releases/ is RELEASE-OWNED (AGENTS.md "Documentation Guardrails"): release
notes are written centrally at release time, and a code PR that edits them is the exact PR
that guardrail exists to stop. They are still audited — read-only. If one of them is actually
wrong, file an issue or open a dedicated docs-only PR; do not edit it here.

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@github-actions github-actions Bot added documentation Improvements or additions to documentation dependencies Pull requests that update a dependency file tests tooling labels Aug 12, 2026
claude added 2 commits August 12, 2026 04:21
…ath exists (#7598)

The decline creates one configuration that previously (wrongly) compiled:
a host advertising `nativeSql` WITHOUT an aggregate bridge now reaches
`resolveStrategy`'s dead end for a `$field`-carrying filter, and the bare
"no strategy can handle this cube" reads like a broken driver when every
other query on that cube still works.

Measured: not reachable from `AnalyticsServicePlugin`, whose default
`queryCapabilities` derives both flags from the bridges it wired and
auto-wires the aggregate bridge from the engine — so only a host that
overrides `queryCapabilities` by hand can hit it. Named anyway, with the
narrowness control (a literal filter on the same deployment still runs).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWcRLiMFvDoQV3zS2LgEHH
… it away (#7598)

The engine-fallback suite's `executeAggregate` bridge passed its query as
`as any` to work around one key: the analytics contract types a measure's
`method` as a plain string while `AggregationNode.function` is the closed
`AggregationFunction` enum. That erased the whole options object and moved
`check:query-options-erasure`'s test surface 242 -> 243.

The query is now typed `DriverQuery` and only `method` is cast, to the one
type it has to be. Every other key stays checked — which is the point of
the ratchet, and better for a bridge whose whole job is to stand in for
the real one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EWcRLiMFvDoQV3zS2LgEHH
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation size/xl tests tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[spec] service-analytics' read-scope / Cube filter compilers still refuse $field, so a CEL field-to-field RLS rule 400s on those faces

2 participants