Skip to content

pnpm test:e2e fails against a long-lived local dev server once demo_bootstrap has claimed the seeds #665

Closed

Description

@xuyushun441-sys

A local-DX footgun found while browser-verifying #663. CI is unaffected and green — this only bites a local run.

What happens

playwright.config.ts reuses an already-running dev server when one is up (reuseExistingServer defaults on outside CI). Point pnpm test:e2e at a dev server that has been running for more than ten minutes and 11 of 16 specs fail, all cascading from:

Error: no seeded accounts returned
  expect(records.length, 'no seeded accounts returned').toBeGreaterThan(0)
  at e2e/smoke.spec.ts:55

Error: no seeded crm_account — the demo seed did not load
  at seededAccountId (e2e/fixtures.ts:85)

The message is misleading: the seed loaded fine. The rows are simply not visible to the user the suite runs as.

Why

e2e/global-setup.ts signs up e2e-admin@hotcrm.test. That lands as a plain org member that owns nothing and holds no sharing grant. Under sharingModel: 'private' it can only read records it owns or has been shared.

On a cold CI database that is invisible, because nothing has claimed the seeds yet — every seeded row is ownerless, so the tolerant read succeeds and the suite passes. On a local dev server the demo_bootstrap schedule (*/10 * * * *) eventually fires and claims all ~115 seeded rows for the first user, and from that moment the e2e user reads 0 rows. pnpm demo:staff has the same effect for the same reason.

So the suite's green depends on a race it does not declare: the seeds must still be unowned when it runs.

Reproduce

  1. pnpm demo:reset && pnpm dev
  2. pnpm test:e2e immediately → passes
  3. Wait for demo_bootstrap to fire (≤ 10 min), or run pnpm demo:staff
  4. pnpm test:e2e again → 11 failures

Confirmed identical on 17.0.0-rc.1 (origin/main worktree) and 17.0.0-rc.2, so it is not an upgrade regression — it is the suite's own environmental assumption.

Options (needs a deliberate call, hence an issue rather than a drive-by fix)

Each changes what the suite actually proves, so this is worth deciding rather than patching:

  1. Authenticate as the seeded dev admin (admin@objectos.ai, already E2E_ADMIN_EMAIL/E2E_ADMIN_PASSWORD-overridable). Simplest, but the dev admin is documented as seeded on an empty DB in dev — needs checking that objectstack start, which is what CI's webServer runs, seeds it at all. If it does not, this trades a local failure for a CI one.
  2. Grant the e2e user access after sign-up (share, permission set, or make it the owner of what it creates and stop depending on seeded rows). Most faithful to "a normal user drives the app", and would make the specs independent of seed ownership entirely — but it is the largest change.
  3. Assert the precondition loudly. Keep the current model but fail in global-setup.ts with "the seeded records are already owned; run pnpm demo:reset first" instead of letting 11 specs fail on a misleading message. Cheapest, and turns a confusing failure into an instruction — but leaves the race in place.

My inclination is (3) now and (2) when the specs are next touched, but the call is the maintainer's.

Activity

  1. os-zhuang commented on Aug 4, 2026

    @os-zhuang
    Contributor

    分诊 + PM 裁定(PM 循环第 1 轮扫检,会话 session_01Rvtsew6XgsSjxVa59HRPRK):已入队 pm:queue。

    三个选项按验证策略裁定(这是工作范围问题,不上升维护者):现在做方案 3(在 e2e/global-setup.ts 里响亮断言前置条件 —— 种子已被认领时给出「先跑 pnpm demo:reset」的明确指令,而不是让 11 个 spec 带着误导性信息级联失败)。理由:

    • 方案 1(改用 seeded dev admin)依赖「CI 的 objectstack start 是否播种 dev admin」这个未经证实的前提,验证成本高且可能把本地失败换成 CI 失败;
    • 方案 2(给 e2e 用户授权/自持数据)最忠实于「普通用户驱动应用」,但改动面最大,应该在下次系统性触碰 e2e spec 时做 —— 派发时会要求 dev 把方案 2 作为后续 issue 单独立案(而不是靠记忆);
    • 方案 3 成本最低,把一个环境竞态从「11 个误导性失败」变成「一条可执行的指令」,且不改变套件当前证明的东西。

    文件面:e2e/global-setup.ts(+ 必要时 e2e/fixtures.ts 的报错措辞)。待当前批次有空位后派发。


    Generated by Claude Code

  2. self-assigned this
    on Aug 4, 2026
  3. os-zhuang commented on Aug 4, 2026

    @os-zhuang
    Contributor

    认领:PM 循环第 2 轮(接替已完成的 #648 批位)
    会话:session_01Rvtsew6XgsSjxVa59HRPRK
    分支:claude/issue-665-e2e-seed-precondition
    Worktree:hotcrm-issue-665
    文件面:e2e/global-setup.ts(+ 必要时 e2e/fixtures.ts 报错措辞、test/ 若需要)(越界即停,报告说明)

    按上方 PM 裁定执行方案 3;方案 2 由 dev 作为后续 issue 单独立案,不在本 PR 实现。


    Generated by Claude Code

  4. os-zhuang commented on Aug 4, 2026

    @os-zhuang
    Contributor

    验收(PM 循环第 2 轮,会话 session_01Rvtsew6XgsSjxVa59HRPRK):ACCEPT —— PR #668,已转 ready 并入合并队列。

    复核要点:

    1. 按裁定只做了方案 3,且做得比裁定要求更扎实:双探针(crm_account private+被 sweep vs crm_product public_read+不被 sweep)把「种子被认领」与「根本没播种」两个同样读到 0 行、补救相反的状态区分开,各给一条含成因与补救的错误;守卫的元数据前提(哪个对象会变暗、哪个不会)被 test/e2e-seed-precondition.test.ts 直接钉住 —— 把 crm_product 加进 CLAIMED_OBJECTS 或改 sharingModel 都会红。
    2. 「CI 为什么一直绿」从运气变成了被测量的机制:objectstack start 不播种 dev admin ⇒ e2e 账号是组织第一个用户 ⇒ sweep 把种子认领给套件自己。这个解释写进了守卫和 e2e: make the specs independent of who owns the seeded records (option 2 from #665) #669。
    3. 中途 sweep 仍可达的两处断言改挂真实成因(SEEDS_UNREADABLE_MID_RUN),不再指控种子加载器。
    4. 套件证明的东西未变(无授权变更、未切 dev admin);方案 2 已立案 e2e: make the specs independent of who owns the seeded records (option 2 from #665) #669(未指派,验收标准明确)。
    5. CI 9/9 全绿,含 Playwright 16 passed;文件面与认领声明一致,无越界。

    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingpm:dispatchedDispatched to a dev agent by /pm-dispatchpm:queueReady for the PM dispatch looptesting

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions