Skip to content

[Desktop E2E] WorkHub correction test flakes while automatic session title is pending #3762

Description

@sunrioa

What happened

WorkHub handles a first natural-language correction intermittently fails because it assumes the first Session has received its automatically generated title as soon as the first assistant response becomes visible.

Session title generation runs asynchronously. The first response is therefore not a synchronization point for the title update. If WorkHub is enabled before that update finishes, the source Session may still be named New Chat.

The WorkHub submission is routed to the correct source Session, but the assertion checks its display name:

  • Expected: 检查支付回调重复投递时的幂等性
  • Received: New Chat

In a focused 10-run sample, the test passed 7 times and failed 3 times with the same mismatch.

Expected behavior: the E2E should establish deterministic preconditions before testing WorkHub routing and natural-language correction. It should not intermittently fail based on automatic title-generation timing.

简体中文

WorkHub handles a first natural-language correction 测试会偶发失败,因为它在首个模型回复显示后,就假定第一个 Session 已经获得自动生成的标题。

Session 标题是在后台异步生成的,因此首个回复出现并不代表标题更新也已完成。如果此时启用 WorkHub,源 Session 的名称仍可能是 New Chat

WorkHub 实际上已经将请求路由到了正确的源 Session,但测试断言检查的是它的显示名称:

  • 预期:检查支付回调重复投递时的幂等性
  • 实际:New Chat

在一次定向重复运行 10 次的测试中,7 次通过、3 次因为同样的名称不一致而失败。

预期行为:E2E 应在验证 WorkHub 路由和自然语言纠错前建立确定的前置状态,不应因为自动标题生成的时序而偶发失败。

How to reproduce

  1. Check out the current main.

  2. Install dependencies and build Maka:

    npm install
    npm run build
  3. Run the focused E2E repeatedly:

    cd apps/desktop
    npx playwright test \
      --config=e2e/playwright.config.ts \
      e2e/workhub-reconstruction.spec.ts \
      --grep "WorkHub handles a first natural-language correction" \
      --repeat-each=10 \
      --workers=1 \
      --reporter=line
  4. Observe the assertion for .workhub-submitted-session strong.

The problem is timing-dependent and does not fail on every execution. In the observed 10-run sample, 3 runs received New Chat instead of the expected source Session name.

简体中文
  1. 检出当前 main

  2. 安装依赖并构建 Maka:

    npm install
    npm run build
  3. 重复运行定向 E2E:

    cd apps/desktop
    npx playwright test \
      --config=e2e/playwright.config.ts \
      e2e/workhub-reconstruction.spec.ts \
      --grep "WorkHub handles a first natural-language correction" \
      --repeat-each=10 \
      --workers=1 \
      --reporter=line
  4. 查看 .workhub-submitted-session strong 的断言结果。

该问题与执行时序有关,并非每次都会失败。在已观察的 10 次运行中,有 3 次得到 New Chat,而不是预期的源 Session 名称。

Environment

  • Current Maka main: 81a811975
  • Observed pre-fix head: 4a9d62b307
    • The additional commit only changed transcript subscription recovery.
    • The affected WorkHub and title-generation code is unchanged on current main.
  • OS: macOS 26.5.2 (arm64)
  • Surface: Desktop Electron E2E
  • Node.js: v22.23.1
  • Electron: 43.4.1
  • Playwright: ^1.62.1
简体中文
  • 当前 Maka main81a811975
  • 实际复现提交:4a9d62b307
    • 该提交只修改了 transcript subscription 恢复;
    • 受影响的 WorkHub 和标题生成代码在当前 main 中没有变化。
  • 操作系统:macOS 26.5.2(arm64)
  • 测试范围:Desktop Electron E2E
  • Node.js:v22.23.1
  • Electron:43.4.1
  • Playwright:^1.62.1

Logs, screenshots, or additional context

Observed assertion failure:

Expected string: "检查支付回调重复投递时的幂等性"
Received string: "New Chat"
Timeout: 10000ms

Automatic Session title generation runs in the background and may wait longer than this assertion. Waiting for the first assistant response therefore does not guarantee that the title has been committed.

The failing trace shows that the submission reached the correct source Session. Only its display name was still New Chat, so this is an E2E synchronization problem rather than evidence of incorrect WorkHub routing.

简体中文

观察到的断言错误:

预期字符串:"检查支付回调重复投递时的幂等性"
实际字符串:"New Chat"
超时时间:10000ms

Session 自动标题在后台异步生成,其执行时间可能超过该断言的等待时间。因此,等待首个模型回复并不能保证标题已经写入。

失败记录表明请求已经进入正确的源 Session,只有显示名称仍为 New Chat。所以这是 E2E 同步问题,并非 WorkHub 生产路由错误。

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions