Skip to content

feat(test): add scenario integration tests and self-dogfood e2e workflow#120

Merged
wadackel merged 1 commit into
mainfrom
feat/issue-118
May 16, 2026
Merged

feat(test): add scenario integration tests and self-dogfood e2e workflow#120
wadackel merged 1 commit into
mainfrom
feat/issue-118

Conversation

@wadackel
Copy link
Copy Markdown
Collaborator

Summary

Adds two complementary test-harness layers on top of the existing unit tests, as scoped by #118.

  • src/main.integration.test.ts — a vitest integration suite that drives main.ts through the full eventName x allowed_contexts x parse outcome matrix (13 table-driven scenarios), including the discussion_comment regression guard that asserts issue_number is never emitted.
  • .github/workflows/e2e.yaml — a self-dogfooding workflow that runs the built action against real issue_comment / discussion_comment events in this repo, gated by a protected environment: e2e (required-reviewer approval per run). The Verify step extracts an expectation directive embedded in the triggering comment under the reserved _expect_ params key and fails the job on mismatch.
  • Verifier split — pure functions live in src/e2e-verify.ts (unit-tested via src/e2e-verify.test.ts); a CLI shim in scripts/e2e-verify.ts is invoked from the workflow.
  • docs/e2e.md — comment convention, security model, required repo settings, operational tradeoffs, and smoke-run procedure. README.md gains a one-line link.

Notable design choices

  • Directive carrier: the original issue proposal embedded # expect: <json> on a separate line of the comment body. The action's parser rejects any trailing non-params text, so the directive is instead carried as a single-quoted JSON string under a reserved _expect_ params key (e.g. .test foo=bar, _expect_='{"continue":"true","command":"test","params":{"foo":"bar"},"context":"issue"}'). The verifier strips _expect_ from the actual params before deep-equal so directive-bearing comments do not pollute the expected.params shape. Production code (src/main.ts, src/parse.ts, action.yaml) is untouched.
  • Negative-case limitation: the directive can only be carried on comments that produce continue=true (the action emits params only on that path). Filtered / unknown-command / malformed-params cases fall back to the smoke check — documented in docs/e2e.md.
  • Two-job workflow: registered runs on pull_request / push:main for visibility (echo only, no environment gate); dogfood runs only on comment events, behind environment: e2e. dist/ drift is gated by the existing ci.yaml check rather than duplicated here.
  • Typo guards: the verifier reports unknown directive keys and non-boolean failed values as mismatches.

Not part of this PR

  • Repo admin tasks documented in docs/e2e.md (enable Discussions, create the e2e environment with required reviewers).
  • Smoke runs across all three event kinds (operational, post-merge — see docs/e2e.md "Smoke run" section).
  • e2e.yaml is intentionally NOT wired into publish.needs of ci.yaml; semantic-release continues to ship from main independently.

Test plan

  • pnpm typecheck — exit 0
  • pnpm test — 95 tests across 5 files passing (existing 58 + 13 integration + 20 e2e-verify unit + 4 utils)
  • pnpm lint — 0 warnings, 0 errors on 15 files
  • pnpm format:check — clean
  • pnpm build && git diff --exit-code dist/ — no drift (production code unchanged)
  • actionlint .github/workflows/e2e.yaml — exit 0
  • CLI shim smoke tests: matching directive → exit 0, deliberate mismatch → exit 1 with clear message, no directive → smoke-pass when action did not crash
  • (post-merge) Configure e2e environment + enable Discussions per docs/e2e.md
  • (post-merge) Run the 6 dogfood smoke-run scenarios (3 event kinds x positive/negative) per docs/e2e.md

References

Adds two complementary test-harness layers on top of the existing unit
tests:

- `src/main.integration.test.ts` — a `vitest` integration suite that
  drives `main.ts` through the full `eventName x allowed_contexts x parse
  outcome` matrix (13 table-driven scenarios), including the
  `discussion_comment` regression guard that asserts `issue_number` is
  never emitted.
- `.github/workflows/e2e.yaml` — a self-dogfooding workflow that runs
  the built action against real `issue_comment` / `discussion_comment`
  events in this repo, gated by a protected `environment: e2e`
  (required-reviewer approval per run). The Verify step extracts an
  expectation directive embedded in the triggering comment under the
  reserved `_expect_` params key (see `docs/e2e.md` for the comment
  convention) and fails the job on mismatch.

The verifier is split into pure functions in `src/e2e-verify.ts`
(unit-tested via `src/e2e-verify.test.ts`) and a CLI shim in
`scripts/e2e-verify.ts` invoked from the workflow. `docs/e2e.md`
covers the comment convention, security model, required repo settings,
operational tradeoffs, and smoke-run procedure. `README.md` gains a
one-line link to the new doc.

The e2e workflow is intentionally NOT wired into `publish.needs` of
`ci.yaml`; `semantic-release` continues to ship from `main`
independently.

Closes #118

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@wadackel wadackel merged commit efe79f6 into main May 16, 2026
6 checks passed
@wadackel wadackel deleted the feat/issue-118 branch May 16, 2026 12:01
@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version 2.3.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expand the test harness: scenario-level integration tests + self-dogfooding e2e workflow

1 participant