Skip to content

fix(spec): check:liveness 的 stale-evidence 判红,摘要行的数与词对齐 (#5623) - #6057

Merged
os-zhuang merged 3 commits into
mainfrom
claude/issue-5623-liveness-stale-evidence-gate
Aug 7, 2026
Merged

fix(spec): check:liveness 的 stale-evidence 判红,摘要行的数与词对齐 (#5623)#6057
os-zhuang merged 3 commits into
mainfrom
claude/issue-5623-liveness-stale-evidence-gate

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Fixes #5623

前提复现(先证伪,再动手)

origin/main(739f496)上把 packages/spec/liveness/query.json 的 5 条 evidence 路径故意改坏(指回迁移前的 packages/plugins/driver-sql/...),跑真 gate:

evidence paths: 330 resolved against this checkout, 101 attributed to another repo (…)

⚠ 5 'live' entr(ies) cite a missing file:
    query/fields → packages/plugins/driver-sql/src/sql-driver.ts
    query/where → packages/plugins/driver-sql/src/sql-driver.ts
    query/orderBy → packages/plugins/driver-sql/src/sql-driver.ts
    query/limit → packages/plugins/driver-sql/src/sql-driver.ts
    query/offset → packages/plugins/driver-sql/src/sql-driver.ts

EXIT=0。issue 的两条实测逐条成立:五条断链全被点名却不判红,摘要行的 330 在改坏五条之后一动没动。代码位置:check-liveness.mtsfailed 表达式里没有 report.staleEvidence,而摘要行印的是 report.evidenceLocal(local 路径总数),头顶的注释还写着 "actually resolved against this checkout" —— 注释本身就是这个 bug。

main 当前 330 条本仓路径全部解析成功,所以本 PR 落地即绿,不需要修任何 ledger 数据(scope guardrail 里那条例外没有触发)。

「注意的反面」:⚠/✗ 分级的原始意图查证结果

issue 要求先核对分级是不是有意设计。查证结论:这里的 ⚠ 不是对本仓路径的有意宽容,是解析器时代的遗留。四条证据:

  1. evidence.mts 的文件头记录了它自己的来历:fix(spec): liveness stale-evidence check was ~100% false positives — and was burying a real one #3857 之前的检查是 evidence.split(':')[0],227 条里报 48 条、全是误报(prose 解析产物或有意的跨仓指针)。那种噪声比下,它当然不能判红。同一段还写明代价:被那 48 条埋掉的唯一一条真腐烂 object.enable.clone(消费者从 @objectstack/objectql 搬到了 @objectstack/metadata-protocol)一直没人看见。
  2. .changeset/liveness-register-orphan-proofs.md 的收尾句:"the orphan list joins the stale-evidence list at empty, so both mean something again" —— 解析修好之后,这份清单的语义就是「有命中即信号」。
  3. 同族旁证:同目录的 check-empty-state.mtsrotted-evidence 一直是 + exit 1;liveness/README.md 说它的执行点路径 "resolves like evidence above, so a pointer that rots is reported rather than trusted"。同一族里,腐烂指针的既定处置就是判红。
  4. 反向对照:这个 gate 里有意的宽容每一条都在代码里明说了理由 —— verifiedAt 年龄("Age never fails CI — re-verification is a worklist")、undrilled 容器计数("a worklist, not a failure")、PENDING_GOVERNANCE("a worklist, not a merge gate"),连另一条 ⚠(orphan proof tags)都写了 "flag it (warning)"。唯独 stale-evidence 那一段,一个解释都没有。刻意的宽容在这个文件里是会写下来的;这条没写。

所以按 issue 给的边界收紧,没有需要升级到 needs_decision 的发现。

改动

1. live 条目引用本仓缺失文件 → 判红

report.staleEvidence.length > 0 进入 failed,输出块从 改为 并附三条修法(仓内搬家 → 改指针并顺手补 verifiedAt;搬去别的仓 → 加 realm marker;消费者真没了 → 按 ADR-0049 重新判定,而不是随手指向一个看着像的幸存者)。

边界:cross-repo attribution 完全不受影响。 这不是靠约定守住的,是结构性的 —— checkEvidence 只对 local 桶做存在性检查,foreign 桶(realm marker objectui: / cloud: / ee:,以及 packages/services/service-ai/… 前缀)从来不进 missing。当前 101 条跨仓归属一条都不会判红,自测里有三个用例钉死这一点(含一个「同一条 evidence 里 objectui: 子句 + 仓内路径」的混合用例 —— 如果 realm 作用域不在子句边界结束,一个 marker 就能整条豁免 gate)。

2. 摘要行:选「两个数都印」,而不是二选一

issue 给了两个选项(改成真实解析计数 / 把 "resolved" 改成 "declared")。两个都印才是对的,理由是这两个数各自有用途,丢掉任何一个都损失信息:

绿的时候两者相等 —— 这恰恰是当初只印前一个会被读成「通过」的原因:

evidence paths: 330 repo-local path(s) declared by 'live' entries, 330 resolved against this checkout; 101 attributed to another repo (objectui / cloud — not resolvable here).

有断链时行尾追加 , N MISSING,数与词一一对应。JSON 报告同步新增 evidenceMissing,evidenceLocal 的注释从 "actually resolved" 改成 "DECLARED, not yet proven"。

3. 新增 --ledger-root= 路径参数

让 gate 读 packages/spec/liveness 的一份副本。它是为了让「判红」这件事本身可证明:判红的 gate 只值它那份「它确实会红」的证明,而这需要一条真的腐烂指针来红。往 shipped ledger 里提交一条不是选项(那会让其他每一次运行都红),测试中途改动被跟踪文件则会在崩溃时留下脏 worktree。所以自测把真 ledger 拷进临时目录、只坏一条指针、再跑这个脚本本身 —— CI 跑的同一条代码路径,仓库状态零改动;其余全部仍对真 repoRoot 解析,所以那个 exit 1 只有一个成因。

反向验证(方向先判,再跑)

预判:(标准方向)—— 同一批断链在改前 exit 0、改后 exit 1,且摘要行的数应当移动

origin/main 代码 本 PR 代码
5 条本仓路径断链 ⚠ 5 … / EXIT=0 / 330 resolved ✗ 5 … / EXIT=1 / 330 declared, 325 resolved, 5 MISSING
干净 ledger EXIT=0 EXIT=0,330 declared, 330 resolved

改前那一跑就是「新测试若在 main 上运行会红」的证明:测试断言 exit 1,main 给的是 exit 0

测试

新增 packages/spec/scripts/liveness/check-liveness.test.ts(8 例),spawn 真脚本断言退出码,precedent 是 scripts/check-generated-ledger.test.ts。这一层是必需的:这个 bug 从来不是「检查看不见」—— 它把五条全点名了还是 exit 0,所以只测 checkEvidenceevidence.test.ts 全程是绿的,再加一个同层单测也不会红。

✓ is green against a verbatim copy of the shipped ledgers 1341ms      ← 控制组
✓ FAILS when a `live` entry cites a repo-local file that is gone 1274ms
✓ names EVERY rotted pointer, not just the first 1406ms
✓ stays green when the missing path is attributed to ANOTHER repo 1335ms   ← 边界
✓ still fails a local path that shares a string with a foreign clause 1328ms
✓ prints declared and resolved as separate numbers, equal on a green run 1417ms
✓ MOVES the resolved count when a pointer rots — the mis-labelled count of #5623 2862ms
✓ reports foreign attributions separately and never as missing 1378ms
Test Files 1 passed (1) / Tests 8 passed (8)

全量:

pnpm --filter @objectstack/spec test        → Test Files 326 passed (326), Tests 8316 passed (8316)
pnpm --filter @objectstack/spec typecheck   → tsc --noEmit OK; check:test-typecheck OK(债务账本未增)
pnpm --filter @objectstack/spec check:liveness      → EXIT 0(330 declared / 330 resolved / 101 foreign)
pnpm --filter @objectstack/spec check:empty-state   → ✓ all classified
pnpm check:type-check-coverage / check:release-notes / check:objectui-changeset /
  check:published-files / check:nul-bytes / spec check:generated --reconcile-only /
  spec check:spec-changes / scripts/check-changeset-no-major.mjs   → 全部 PASS
npx eslint(改动的两个文件)                        → 0 problems

Changeset:为什么是 @objectstack/spec: patch,不是空 frontmatter,也不是 skip-changeset

派单给的默认是「dev-scripts/CI-only → 空 frontmatter」。实测后改了,因为这个 PR 不满足那个前提:

  • packages/spec/package.jsonfiles 里有 liveness,npm pack --dry-run 实测 29 个文件入包,含 liveness/README.md —— 本 PR 改的那段 README 是发布内容scripts/ 入包 0 个文件,那部分确实是 dev-only。
  • 所以它不是 "releases nothing",skip-changeset(route 2)和空 changeset(route 3)都不适用,pr-automation.yml 的 route 1 才是:it releases something → 具名包
  • 而且 pr-automation.yml 已经把空 frontmatter 明确降级为 LAST RESORT:它是 changesets/action 的真实输入,全空集合会走 "All changesets are empty; not creating PR" 分支,发布静默地绿着卡住 —— 空 changeset 会静默卡死已 version 的发布:Release run 全绿,但 npm 和 Docker 什么都没发(17.0.0-rc.2 现在就卡着) #4898 卡住 17.0.0-rc.2 的正是这个。

因此本 PR 不需要也不应该带 skip-changeset 标签:它写了一个具名包的 changeset,Check Changeset 走的是「有 changeset」那条路。

一个新开始判红的 gate 必须同步它面向作者的文档,否则下一位作者只能靠撞红的 CI 才知道规则变了 —— README 那一段(新的失败语义 + 跨仓边界)因此是这次改动的一部分,而不是搭车。

#5475 的关系(packages/spec/scripts/** 的 tsconfig 覆盖)

实测答案:对错误数 无影响,对文件数轻微增加(而新增文件是干净的)。

tsconfig.test.json 的文件头已经写明 scripts/ "is in no tsconfig at all",实测 16 files / 33 errors。我用一个把 scripts/liveness/** 纳入的探针 program 量了本 PR 之后的这个子目录:

scripts/liveness/check-liveness.mts(686,24): error TS7006: Parameter 's' implicitly has an 'any' type.

唯一 1 条,且在我没碰过的行(v.stale.forEach((s) => …),verifiedAt worklist 那段)。新增的 check-liveness.test.ts 贡献 0 条错误。所以:#5475 的错误清单不因本 PR 变长(不会更难),但它的文件分母 +1(16 → 17)。既没变简单也没变得不必要 —— --ledger-root 那点解析逻辑也没有引入新的类型面。

#5837 的碰撞面

无。本 PR 只碰 scripts/liveness/check-liveness.mts + 新增同目录自测 + liveness/README.md + changeset;不读也不写 authorable-surface.json / json-schema.manifest.json / api-surface.json,不碰 build-schemas.ts / build-docs.ts / .gitattributes

`live` 判定的语义就是它的 evidence 指针。指针指向本仓已不存在的文件时,
这条声明不再可证伪 —— declared 有,enforced 无 —— 而一次目录重组或改名
就足以造成它。实测(origin/main,故意改坏 query.json 的 5 条路径):逐条
点名了,退出码仍是 0;摘要行的 330 也一动没动,因为它数的是 local 路径
总数,不是解析成功数。

- `live` 条目引用本仓缺失文件 → `✗` 判红(退出码 1),并给出三条修法。
  边界不变:cross-repo attribution(objectui / cloud / service-ai,当前
  101 条)只计数不解析,永远不判红 —— checkEvidence 只对 local 桶做存在
  性检查,这是结构性的分界。
- 摘要行改成两个数:declared 与 resolved,绿的时候相等;有断链时追加
  `, N MISSING`。保留 declared 是因为它是 #3857 留下的解析健康度信号。
- 新增 `--ledger-root=<dir>`,自测据此把真 gate 跑在只坏了一条指针的
  ledger 副本上,不改动仓库任何文件。

以前是 ⚠ 不是对本仓路径的有意宽容,是解析器时代的遗留:#3857 之前
`evidence.split(':')[0]` 在 227 条里报 48 条、全是误报。同族的
check-empty-state 对 rotted-evidence 一直是 ✗ + exit 1。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wsZeReNTqiceBfLb5Pyf5
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 7, 2026 2:30am

Request Review

`packages/spec/liveness/**` 是发布内容(package.json 的 `files` 含
`liveness`;`npm pack --dry-run` 实测 29 个文件入包,含该目录的
README.md),所以本 PR 并非「releases nothing」,route 2 的
`skip-changeset` 与 route 3 的空 changeset 都不适用。

pr-automation.yml 也把空 frontmatter 明确降级为 LAST RESORT:它是
changesets/action 的真实输入,全空集合会走
"All changesets are empty; not creating PR" 分支,发布静默绿着卡住
(#4898 卡住 17.0.0-rc.2)。脚本本身(scripts/)不入包,是 dev-only。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014wsZeReNTqiceBfLb5Pyf5
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

112 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via @objectstack/spec)
  • content/docs/automation/connectors.mdx (via @objectstack/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via @objectstack/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/cli.mdx (via @objectstack/spec)
  • content/docs/deployment/tenancy-modes.mdx (via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/deployment/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via @objectstack/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/data-service.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/examples.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via @objectstack/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via @objectstack/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/spec)
  • content/docs/kernel/services.mdx (via @objectstack/spec)
  • content/docs/permissions/authorization.mdx (via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/http-protocol.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v16.mdx (via @objectstack/spec)
  • content/docs/releases/v17.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/apps.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/field-grouping-and-order.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang added this pull request to the merge queue Aug 7, 2026
Merged via the queue into main with commit 502564d Aug 7, 2026
25 checks passed
@os-zhuang
os-zhuang deleted the claude/issue-5623-liveness-stale-evidence-gate branch August 7, 2026 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size/m tests tooling

Projects

None yet

2 participants