fix(objectql,driver-sql,driver-memory,driver-mongodb)!: FilterArray 在 engine 门下沉,四驱动数组方言删除 (#5158 拍板 C 第 2 步) - #5329
Conversation
…rray at the engine door, delete the drivers' array dialect (#5158) Maintainer ruling C, step 2. `FilterArray` is INPUT-ONLY authoring sugar and the spec has said so since #5285; this makes the runtime agree. Door 2 — a direct in-process engine call — used to pass a `where` array through verbatim, so four drivers grew a second filter compiler to meet it, including an INFIX dialect (`[condA, 'or', condB]`) no schema declared and `parseFilterAST` cannot express. Cloud's `RemoteTransport.buildWhereSQL` has refused that same input since cloud#1075: one query, two answers, decided by whether the caller went over the wire. All six engine entry points (`find`/`findOne`/`count`/`aggregate`/`update`/ `delete`) now run `isFilterAST` -> `parseFilterAST`, the same sink the protocol face has always used, and refuse an array that cannot be lowered. `[]` keeps its meaning everywhere: no filter. The drivers' array compilers are deleted and replaced by a loud `INVALID_FILTER` / 400 refusal that names the lowering — converging driver-sql, driver-sqlite-wasm, driver-memory and driver-mongodb with cloud's transport. Deliberately retired: `engine-findone-contract`'s pin that a non-object `where` "is the driver's to interpret". That sentence was the engine's blessing of the second dialect, and it cost `findOne({ where: [] })` walking past #4419's own guard to return an arbitrary row. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
📓 Docs Drift CheckThis PR changes 4 package(s): 17 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
…er-filter-array-at-engine-door
… local mongod skip hid (#5158) CI caught what the sandbox could not run. `mongodb-driver.test.ts` is gated on `describe.skipIf(!sharedMongod)`; `mongodb-memory-server` downloads its binary from fastdl.mongodb.org, which this environment cannot reach, so the suite skipped locally and ran on CI — where `should filter with legacy array style` asserted that the driver COMPILES `where: [['age','>=',30], ['role','=','user']]`. That case tests the dialect itself, so it gets the same treatment as driver-sql's two compile-asserting cases: a REFUSAL pin, plus a counterpart proving the identical authored shape still returns the same rows once lowered through `parseFilterAST`. Migrating it instead would have quietly changed what the case measures. The filter document is provably unchanged: the deleted `translateArrayFilter` folded an all-AND group to `{$and:[{age:{$gte:30}},{role:'user'}]}`, and `translateFilter(parseFilterAST(<same array>))` produces that document byte-for-byte — verified locally against `translateFilter`, the only function `MongoDBDriver.find` uses to build a filter. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbu27iNUfQCHeuS551Rqo7
CI 补漏:
|
| 类别 | 处数 | 说明 |
|---|---|---|
spec schema 测试 filters:(已退役的 legacy 键) |
12 | QuerySchema.parse,不碰驱动 |
view metadata filter:({field,operator,value} 视图筛选契约) |
33 | 另一形状,与 FilterArray 无关 |
better-auth adapter where: [{field,value}] |
5 | 另一契约(plugin-auth) |
queryast 测试的 filters:(「legacy 键被忽略」用例) |
6 | 驱动只读 query.where,该键从不进 applyFilters |
| analytics / realtime / lint / 协议 metadata | 10 | 与驱动过滤无关 |
| 本 PR 新增的 engine 门用例 | 19 | 断言下沉与拒收,应当是数组 |
| 本 PR 的驱动拒收 pin | 2 | 同上 |
第 2 轮 —— 「本地跳过、CI 才跑」的盲区清点(这条比修那一行重要)。先按 describe/it.skipIf|runIf、process.env.*(PG|POSTGRES|MYSQL|MONGO|REDIS|TURSO|LIBSQL)*、OS_TEST_* 扫出 17 个带条件跳过的测试文件,再对其中所有会跑真驱动的 96 个文件做数组清点:
| 盲区 | 文件数 | CI 里由谁跑 | 数组调用点 |
|---|---|---|---|
driver-mongodb(skipIf(!sharedMongod),需下载 mongod) |
6 | Test Core 分片 + Temporal Conformance |
2(均在 mongodb-driver.test.ts,即本次修的两条) |
driver-sql live PG/MySQL(skipIf(!OS_TEST_POSTGRES_URL/…MYSQL_URL)) |
4 | Temporal Conformance(带 services: 起真库) |
0 |
packages/qa/dogfood(skipIf(!organizationsAvailable) 等) |
96 | Dogfood Regression Gate ×3 |
0 |
packages/lint(skipIf(!existsSync(dist/…))) |
2 | Test Core |
0(与过滤无关) |
即:除已修的这两条外,盲区里没有第二处。第二片、第三片不会再红在同一根因上。
我如何核对那些本地跑不起来的套件(不假装跑过)
沙箱内既无 mongod 二进制(find / -name mongod 无结果、~/.cache/mongodb-binaries 不存在)也无外网,driver-mongodb 的 5 个套件确实无法真跑。核对方式两条,都说清覆盖面:
- 静态穷尽清点 —— 上表第 2 轮,模式与结果如上,覆盖 6 个 mongodb 文件 + 4 个 live-dialect 文件 + 96 个 dogfood 文件。
- 功能等价验证 ——
MongoDBDriver的每一个读写入口(mongodb-driver.ts的 240/252/370/441/459/546 行)都只用translateFilter(query.where, …)构造过滤器,而translateFilter本地可跑。于是写了一个临时 probe(跑完即删,未进提交),把这两个盲区文件里我改动过的原样表达式喂给translateFilter:
✓ mongodb-datetime-storage.test.ts:144 — prefix AND group lowers to both bounds
✓ mongodb-driver.test.ts — the retired dialect case: raw array refused
✓ mongodb-driver.test.ts — its counterpart: same authored shape, lowered
Test Files 1 passed (1) Tests 3 passed (3)
其中第 1 条同时补上了另一个盲区:mongodb-datetime-storage.test.ts 也在那 5 个 skip 套件里,我在首轮迁移过它却从未在本地执行过;probe 证明它下沉后得到 {$and:[{created_at:{$gte:'2026-04-29'}},{created_at:{$lt:'2026-07-29'}}]}(上界按 #4042 整日半开),与该用例上方 $between 条产出同一组行。
这两条是间接证据(证的是过滤文档,不是 mongod 上的真实行),我不声称等同于跑过那些套件 —— 留给 CI 判定。
重跑
| 命令 | 结果 |
|---|---|
pnpm --filter @objectstack/driver-mongodb exec vitest run |
Tests 139 passed | 121 skipped,Test Files 7 passed | 5 skipped(skip 数 120→121:一条用例拆成两条) |
pnpm turbo run typecheck --filter=@objectstack/driver-mongodb |
Tasks: 4 successful, 4 total |
pnpm lint(全仓) |
无输出(通过) |
复现 Temporal Conformance 的 non-SQL 腿:TZ=America/New_York pnpm --filter core --filter formula --filter driver-memory --filter driver-mongodb --filter service-analytics test |
core 429、formula 357、driver-memory 324、driver-mongodb 139 passed | 121 skipped、service-analytics 591 —— 全绿 |
同 job 的 SQL 腿(无 live URL,故 4 个套件仍 skip):TZ=America/New_York pnpm --filter @objectstack/driver-sql test |
Tests 794 passed | 44 skipped |
另:本分支已 merge origin/main(f1cc3a316,两笔均只动 packages/spec / plugin-email,与本 PR 零重叠),merge 后重建 spec 并跑 check:generated → ✓ All 9 generated artifacts are up to date.
保持 draft,分支未变(claude/issue-5158-lower-filter-array-at-engine-door),未开新 PR。
Generated by Claude Code
PM 复核(会话
|
where 也 NULL-safe —— $not 下推守卫、{$not:{}} 为零行、{} 析取项吸收 $or (#5325)
#5335
…ity-batch Textual conflict: packages/plugins/driver-mongodb/src/mongodb-filter.ts — both sides rewrote translateFilter/translateCondition (#5239 reduction vs #5329 array-dialect deletion + #5368 $null gate/path threading). Resolution keeps both: main's array refusal and path threading, this branch's three-valued reduction and shape gates; the three helpers both sides defined (unsupportedFilterError, describeFilterOperand, safeShapePreview) are de-duplicated onto main's copies. Semantic reconciliation the textual merge could not see (AGENTS.md s10): main's #5347 $null comparand gate sat in the emitter, and this branch's reduction makes emitters skippable by a boolean identity — { $or: [ {}, { stage: { $null: 'yes' } } ] } would have translated to match-all while driver-sql refuses it. The gate's load-bearing copy moved onto the validating walk (reduceFilterKey), mirroring driver-sql's #5368 placement; the emitter arm keeps its local check. Pinned in mongodb-null-comparand-refusal.test.ts (three identity-sibling fixtures). Fixture triage: the 'legacy array dialect is untouched' pin in mongodb-filter-boolean-identity.test.ts pinned a dialect #5329 deleted — replaced wholesale with the surviving boundary ([] = absent filter = match-all, non-empty array refused before the reduction runs). The reduceFilterKey field-arm comment on { field: {} } is recalibrated to current main: #5327 gated the shape on the other four backends; this driver remaining the one still answering it is now tracked by #5376. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB
Fixes #5158
拍板 C 的第 2 步(engine 半边)。第 1 步(spec 声明
FilterArray为仅输入授权糖)已由 spec 车道以 #5285 / PR #5306 落地在b49ccfdfe;本单只消费那份声明,packages/spec/**一行未改。1. 现场核对(行号已漂,以现场为准)
issue 与派发评论引用的行号今天都动过,逐条重核如下:
0f1711470的实际位置engine.ts:4309engine.ts:4429-4443(requireFindOnePredicate的文档注释 +Array.isArray(where)判定)protocol.ts:4533protocol.ts:4558-4562(isFilterAST→parseFilterAST,否则malformedFilterArrayError)SqlDriver.applyFilters数组分支sql-driver.ts:6033-6112(改前)reduceFilterNode、#5296nullSafeNegationOperand、#5327reduceFilterKey的{field:{}}闸门)不重叠:那三处都在对象路径(applyFilterCondition/ 归约遍历),本 PR 只删数组遍历engine.ts同日两次改动previous另外核到一个 issue 正文未写、但决定了实现形状的事实:
FilterArray的中缀连接形式([condA, 'or', condB])parseFilterAST根本表达不了(isFilterAST对它返回false,parseFilterAST返回undefined),FilterArraySchema也不接受它。也就是说驱动编译的那套方言里,有一支从来不是FilterArray的任何声明形状 —— 它是纯粹的驱动私产。这决定了 Door 2 必须像 Door 1 一样「先isFilterAST闸门、再下沉、否则响亮拒收」,而不是无条件parseFilterAST(后者对拼错的操作符有$${op}宽松兜底,会把作者的错误变成一个没人执行的$sounds_like条件)。2. 下沉点的选择与理由
新增模块级函数
lowerWhereFilterArray(object, operation, bag)(packages/objectql/src/engine.ts),在六个入口点各调用一次,紧跟已有的foldEngineOptionAliases/rejectUnknownEngineOptions:find/findOne/count/aggregate/update/deletefoldEngineOptionAliases([P1]filterfolds towhereinengine.findonly —findOne/count/update/deletesilently match EVERY row, and the hook docs teach the broken call #4346 的别名折叠)已经站的位置。别名折叠与形状下沉是同一件事的两半 —— 先把filter折成where,再把where的值下沉成FilterCondition—— 放在一起,后面所有代码(token 解析、by-id 快路、middleware、hook)读到的where恒为对象。foldEngineOptionAliases内部:那个函数按 slot 表折键,并且刻意在无别名时返回同一引用(withResolvedWhere的零分配纪律);值形状下沉是另一件事,显式的第二次调用更可 grep,也与rejectUnknownEngineOptions的既有风格一致。update/delete的顺序:下沉必须在 by-id 提取之前 —— 未下沉的数组上where.id恒为undefined,所以update({ where: [['id','=',x]] })过去一律路由到多行路径。三种到达,三个答案,与 Door 1 逐条对齐:
[][]在where上继续下传,而 engine 在其下游,所以最终一致isFilterAST为真parseFilterAST下沉400 INVALID_FILTER;engine 直调没有 HTTP 状态,措辞对齐3. 不可达证明(先证再删)
结论:生产路径结构性不可达;类型层不可达不成立**,如实报告。**
3.1 类型层:证不成立
QueryAST.where是FilterCondition,而FilterCondition的非递归半边是{ [key: string]: any }索引签名 —— TypeScript 里数组对索引签名类型是可赋值的。实测:仓内 15 个驱动测试文件里,where: [['category','=','Electronics']]这类写法不带任何 cast 也能通过tsc --noEmit(本 PR 前后pnpm typecheck全绿 124/124 都证实了这点)。所以「类型挡住了数组」是错的,这也正是运行时必须响亮拒收、而不是靠类型的原因。这一点已写进
sql-driver-filter-no-silent-drop.test.ts的注释,免得下一个人再推一次。3.2 调用点清点:生产调用方 0
即:没有任何生产调用方以数组形式到达
SqlDriver.applyFilters—— 与侦察清点的读数一致,本 PR 独立复核后确认。3.3 探针实测:全仓测试穿过拒收
删除后的驱动分支本身就是探针 —— 任何经 engine/协议到达驱动的数组都会抛
INVALID_FILTER。以此跑全仓pnpm turbo run test:@objectstack/objectql1908、@objectstack/runtime1333、@objectstack/metadata-protocol388、@objectstack/client222 全绿 —— 没有一条经 engine 或协议的路径产生数组。唯一的到达者是驱动测试自己的直调(15 个文件、23 条 driver-sql 用例 + memory/mongo 若干),它们都是
driver.find(...)直接喂数组。这类调用是存在的,所以删除对直调驱动的消费方不是零行为变化;对产品路径是零行为变化。两句话都写在 changeset 的「可观察的行为变更」里,不含糊。4. 四驱动删除清单
driver-sqlapplyFilters的数组遍历分支 + 其比较发射器protected applyAstComparison(方言私有,唯一调用点在被删分支);canonicalAstOperator导入随之移除driver-memoryconvertToMongoQuery的 legacy array 分支driver-mongodbtranslateArrayFilter+translateComparison+mapFieldNamedriver-sqlite-wasmextends SqlDriver随之变更三处统一替换为
filterArrayReachedDriverError—— 逐字相同的措辞,ADR-0112 信封(INVALID_FILTER/ 400),消息指向parseFilterAST而不是 SQL builder,并明说中缀形式没有下沉、要写前缀。[]在三个驱动里仍然提前返回(「无过滤」)。顺带消失的一条:
driver-mongodb的createdAt→created_at字段别名只存在于数组路径(mapFieldName仅被已删的translateComparison调用),对象路径从未应用过。按 PD #12「消费端别名是债务不是模式」不再补回,已在测试里钉住现状。5. Pin 退役说明(刻意,不是顺手删测试)
5.1
engine-findone-contract.test.ts—— 派发点名的那一条退役:
这是拍板 C 的直接后果,不是碍事的测试。 那句话是 engine 对第二套方言的明文祝福 —— 产品里唯一让
FilterArray不经下沉抵达驱动的门。C 把门关上后,「归 driver 解释」对任何where都不再成立。而且这条祝福是有代价的,退役同时补上了代价对应的 pin:
findOne({ where: [] })过去被requireFindOnePredicate当作「表达式树 = 有谓词」放行,于是limit: 1落在整张表上返回任意一行 —— #4419 要挡的缺陷,活在 #4419 自己的守卫里面。两条替代用例:an array where is LOWERED before the guard, and still selects the record(断言驱动收到{ name: 'Two' },不是数组)`where: []` no longer walks past the guard as "an expression tree"engine.ts里那段祝福的注释原文也一并改写,并说明保留的Array.isArray分支从此只是纵深防御,不再是契约。5.2
sql-driver-filter-no-silent-drop.test.ts—— 第二处退役,一并说明派发只点名了 5.1。但删掉驱动方言必然动到这个 #3948 的 pin 文件,故在此显式报告:
compiles a nested condition array、compiles an infix logical join)—— 它们断言的正是方言本身 —— 改为断言拒收,并新增「同一授权形状经parseFilterAST下沉后照常编译、返回同样的行」的对照用例。leaves an empty filter alone原样保留。5.3 其余 13 个驱动测试文件:迁移而非删除
统一手法:把原本喂给驱动的裸数组改成
parseFilterAST(<同一个数组>)—— 即显式走声明的那条路。作者写的形状一字不改,覆盖面不减,而且这些用例现在顺带证明了下沉链路本身。涉及#2704空值算子、#3777/#4042日历日上界、#3912文本存储、#4436拒收信封、#5041跨字段引用、ADR-0015 §18 columnMap 等既有 pin,逐条保持原断言。6. 测试
pnpm turbo run test(全仓)pnpm --filter @objectstack/objectql exec vitest runTests 1908 passed,Test Files 118 passedpnpm --filter @objectstack/driver-sql exec vitest runTests 794 passed | 44 skipped,Test Files 61 passed | 4 skippedpnpm --filter @objectstack/driver-memory exec vitest runTests 324 passed,Test Files 12 passedpnpm --filter @objectstack/driver-mongodb exec vitest runTests 139 passed | 120 skipped(5 个套件需下载 mongod,沙箱无网,预先跳过)pnpm --filter @objectstack/driver-sqlite-wasm exec vitest runTests 237 passed,Test Files 17 passedpnpm --filter @objectstack/client exec vitest runTests 222 passed,Test Files 17 passedpnpm turbo run typecheck(全仓)pnpm lint(全仓 eslint)pnpm --filter @objectstack/spec check:generated✓ All 9 generated artifacts are up to date.check:driver-conformance/check:engine-double-contract/check:adr-anchors/check:route-envelope/check:durability-log-level/check:startup-registry-verdict/check:error-code-casing/check:nul-bytes新增
packages/objectql/src/engine-filter-array-lowering.test.ts(25 条)。断言落在驱动入参,不是结果行 —— 结果行相同正是本单的目标,只测结果证明不了下沉发生了。测试用的记录型驱动故意只认对象形式,一旦收到数组就抛;仓内四个驱动过去两种都认,这正是没人发现自己拿到了哪一种的原因。反向验证(把改动 stash 掉,新用例必须失败)
25 条里唯一仍通过的是
a FilterCondition object passes through byte-for-byte—— 它本来就不该受影响。派发点名的四条语义,逐条核对
FilterCondition而非数组it.each)FilterBuilder元组 + showcase 5 处形状不回归FilterBuilder全部 7 种输出形状逐条钉住;{current_user_id}token 那处形状单独一条;wire 面(Door 1)未改动,metadata-protocol388 条全绿[]仍是「无过滤」find/count语义逐字不变(engine 删键、驱动提前返回)。唯一例外findOne([]),单独报告见下7. 需要维护者过目的两处语义变化(未擅自扩大,如实列出)
findOne({ where: [] }):放行任意行 → 抛错。[]的含义没变(仍是「无过滤」),变的是findOne终于看得见它。这正是 ObjectQL silently drops unsupported predicate keys;findOnethen returns the first row #4419 守卫的目的,视为修复而非回归;但它确实是行为变化,已在 changeset 和 5.1 里点名。driver.find(o, { where: ['close_date','before','2024-01-01'] })仍然抛(消息从「Unrecognized filter operator "close_date"」变为「A filter ARRAY reached the driver」)—— 不变。engine.find(o, { where: ['stage','=','won'] })(格式良好的裸三元组)过去由驱动抛错,现在被下沉为{stage:'won'}并正常执行。这是有意的:FilterArrayComparison就是 spec 第 1 步声明的合法输入形状之一(filter-array-declaration.test.ts的PRODUCED表里就有),Door 1 也一直这样接受它。engine 门若在这里比协议门严,就是又造了一个方言 —— 恰是 C 要消灭的东西。操作符不在词表里的裸三元组(before/sounds_like,A filter with an operator outside VALID_AST_OPERATORS is silently dropped, not rejected — single-condition views return unfiltered results #3948 的原始 case)仍然抛错,只是改由 engine 门抛。8. 与 cloud
RemoteTransport的合流关系(本 PR 不碰 cloud 仓)TursoDriver extends SqlDriver(cloud/packages/driver-turso/src/turso-driver.ts:166),原样继承方言;而同仓的RemoteTransport.buildWhereSQL自 cloud#1075 起对同一输入响亮拒收(isFilterNode(filters)为假即uncompilableWhere)—— 同一产品内部两个答案,两侧零测试。本 PR 删除方言后,TursoDriver继承到的是同一个INVALID_FILTER/ 400 拒收,分叉自然消灭,cloud 侧无需任何跟进改动。一处残留差异,留给 cloud 车道知悉(本 PR 不动):
[]的读法两侧仍不同 —— framework 三个驱动读作「无过滤」,cloud 的isFilterNode([])为假故拒收。因为 engine 门现在会把[]删键,[]已经不可能经 engine 到达任一驱动,该差异对产品路径不可观察;如果 cloud 车道希望彻底统一,那是一次独立的取舍(framework 侧的「[]= 无过滤」有sql-driver-filter-no-silent-drop的明文 pin,不该顺手改)。9. 范围外发现(已按 PD #10 立单/评论,本 PR 一律未修)
搜索去重后:
$between静默不发谓词(匹配 0 行),driver-sql 同一过滤器抛错 —— 一个过滤器两个答案 #5328 ——driver-memory对形状错误的$between(comparand 非二元数组)静默不发谓词,find返回[];driver-sql同一输入抛INVALID_FILTER。机制:normalizeFilterCondition的$betweenarm 是有条件写入,形状不对时整条约束消失。是先前存在的:原来接住这个输入的是数组路径的守卫,而数组路径正是本 PR 删掉的那条;对象路径从未有过。相关用例已按现状钉住(resolves.toEqual([]))并在注释里指向 driver-memory 对形状错误的$between静默不发谓词(匹配 0 行),driver-sql 同一过滤器抛错 —— 一个过滤器两个答案 #5328,好让分叉可见而非口口相传。$not—— mingo 抛无 code 的 MingoError,CEL!expr降下来的 RLS scope 在该驱动上直接 500 #5324(未开新单,避免造孪生)—— 该 issue 记的是driver-memory实时路径$not抛无信封MingoError;实测发现机制更宽:normalizeFilterCondition的default: result[op] = val是通用透传,任何未知$op都一样逃出 ADR-0112 信封(实测{name:{$sounds_like:'alpha'}}→unknown query operator $sounds_like,code/status皆undefined;driver-sql 同输入是INVALID_FILTER/ 400)。已在评论里建议该 issue 的「响亮拒收」方案按未知$op通用来写,而不是只加一条$not分支。SqlDriver.applyFilters对真值标量where('active'/42)不发谓词也不报错。仅 cast 可达(协议门unusableFilterError拒收,FilterCondition不描述该形状),且与本单的数组轴无关;已在代码里留注释说明「本次刻意不动」。若 PM 认为该独立成单,请示下。@objectstack/lint的runtime-lazy-deps.test.ts在高并发下偶发失败:已在git stash -u(即干净origin/main)上复现同一失败,与本 PR 无关;单独重跑与最终全仓跑均通过。仅记录,未立单(疑似沙箱资源相关的 flake,而非代码缺陷)。10. 硬禁区自查
packages/spec/**—— 0 处改动(git diff --stat可验)content/docs/releases/**—— 0 处改动;变更记录只走.changeset/filter-array-lowered-at-engine-door.mdTursoDriver/RemoteTransport用于第 8 节)check:generated9/9 up to dateGenerated by Claude Code