Skip to content

ci: Try to auto-fix flaky test issues#20793

Draft
mydea wants to merge 3 commits into
developfrom
fn/auto-fix-issue
Draft

ci: Try to auto-fix flaky test issues#20793
mydea wants to merge 3 commits into
developfrom
fn/auto-fix-issue

Conversation

@mydea
Copy link
Copy Markdown
Member

@mydea mydea commented May 11, 2026

This adds a workflow that tries to auto-fix a given issue. It is auto-run for flaky test issues.

@mydea mydea self-assigned this May 11, 2026
Comment thread .github/workflows/auto-fix-issue.yml
@mydea mydea force-pushed the fn/auto-fix-issue branch from be4e7e3 to 86f6509 Compare May 11, 2026 10:37
@mydea
Copy link
Copy Markdown
Member Author

mydea commented May 12, 2026

adjusted this a bit and introduced prompt injection checker from triage-issue here as well.

id: triage
uses: anthropics/claude-code-action@v1
with:
anthropic_api_key: ${{ secrets.ANTHROPIC_API_KEY }}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutable third-party action ref runs with secrets and write token

The workflow pins anthropics/claude-code-action@v1 (a moving tag) in a job that exposes ANTHROPIC_API_KEY, a pull-requests: write GITHUB_TOKEN, and id-token: write. A compromise or retag of the upstream v1 tag would let attacker-controlled action code exfiltrate the Anthropic API key, open or modify pull requests under the repo's identity, and mint OIDC tokens. Pin third-party actions to a full 40-character commit SHA to remove the upstream tag-rewrite supply-chain path.

Verification

Read the workflow hunk; confirmed permissions block grants pull-requests: write and id-token: write and the step passes ANTHROPIC_API_KEY and GITHUB_TOKEN into a third-party action referenced by floating tag @v1. Checked references/github-workflows.md mutable-action table: third-party mutable ref with secrets/OIDC/non-trivial write token => medium.

Identified by Warden security-review · SZ3-4PD

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 72358be. Configure here.

github_token: ${{ secrets.GITHUB_TOKEN }}
allowed_non_write_users: '*'
prompt: |
Fix the issue in getsentry/sentry-javascript with number #{{ steps.parse-issue.outputs.issue_number }}.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing $ in GitHub Actions expression interpolation

High Severity

The prompt uses {{ steps.parse-issue.outputs.issue_number }} instead of ${{ steps.parse-issue.outputs.issue_number }}. Without the $ prefix, GitHub Actions won't interpolate the expression, and Claude will receive the literal string {{ steps.parse-issue.outputs.issue_number }} instead of the actual issue number. This means Claude will never know which issue to fix. Line 58 of the same file correctly uses the ${{ }} syntax for the same output.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 72358be. Configure here.

---
title: '[Flaky CI]: {{ env.JOB_NAME }} - {{ env.TEST_NAME }}'
labels: Tests, Bug
labels: Tests, Bug, "Flaky Test"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Template label change is a no-op for issue creation

Medium Severity

Adding "Flaky Test" to the template's frontmatter labels has no effect because report-ci-failures.mjs hard-codes labels as ['Tests', 'Bug'] when creating issues (ignoring the template's labels field). This means auto-created flaky test issues will never receive the Flaky Test label, and the auto-fix workflow's condition contains(github.event.issue.labels.*.name, 'Flaky Test') will never match for those issues when the issues trigger is eventually enabled.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 72358be. Configure here.

@github-actions
Copy link
Copy Markdown
Contributor

size-limit report 📦

Path Size % Change Change
@sentry/browser 26.84 kB - -
@sentry/browser - with treeshaking flags 25.28 kB - -
@sentry/browser (incl. Tracing) 44.73 kB - -
@sentry/browser (incl. Tracing + Span Streaming) 46.73 kB - -
@sentry/browser (incl. Tracing, Profiling) 49.73 kB - -
@sentry/browser (incl. Tracing, Replay) 84.38 kB - -
@sentry/browser (incl. Tracing, Replay) - with treeshaking flags 73.81 kB - -
@sentry/browser (incl. Tracing, Replay with Canvas) 89.08 kB - -
@sentry/browser (incl. Tracing, Replay, Feedback) 101.71 kB - -
@sentry/browser (incl. Feedback) 44.03 kB - -
@sentry/browser (incl. sendFeedback) 31.66 kB - -
@sentry/browser (incl. FeedbackAsync) 36.77 kB - -
@sentry/browser (incl. Metrics) 27.93 kB - -
@sentry/browser (incl. Logs) 28.08 kB - -
@sentry/browser (incl. Metrics & Logs) 28.75 kB - -
@sentry/react 28.59 kB - -
@sentry/react (incl. Tracing) 47.01 kB - -
@sentry/vue 31.75 kB - -
@sentry/vue (incl. Tracing) 46.59 kB - -
@sentry/svelte 26.86 kB - -
CDN Bundle 29.24 kB - -
CDN Bundle (incl. Tracing) 47.16 kB - -
CDN Bundle (incl. Logs, Metrics) 30.61 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) 48.28 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) 69.94 kB - -
CDN Bundle (incl. Tracing, Replay) 84.55 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) 85.62 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) 90.36 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) 91.46 kB - -
CDN Bundle - uncompressed 85.99 kB - -
CDN Bundle (incl. Tracing) - uncompressed 141.47 kB - -
CDN Bundle (incl. Logs, Metrics) - uncompressed 90.18 kB - -
CDN Bundle (incl. Tracing, Logs, Metrics) - uncompressed 144.93 kB - -
CDN Bundle (incl. Replay, Logs, Metrics) - uncompressed 215.01 kB - -
CDN Bundle (incl. Tracing, Replay) - uncompressed 260.18 kB - -
CDN Bundle (incl. Tracing, Replay, Logs, Metrics) - uncompressed 263.62 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback) - uncompressed 273.87 kB - -
CDN Bundle (incl. Tracing, Replay, Feedback, Logs, Metrics) - uncompressed 277.31 kB - -
@sentry/nextjs (client) 49.52 kB - -
@sentry/sveltekit (client) 45.23 kB - -
@sentry/node-core 60.84 kB +0.02% +11 B 🔺
@sentry/node 165.98 kB +0.01% +10 B 🔺
@sentry/node - without tracing 73.96 kB +0.01% +5 B 🔺
@sentry/aws-serverless 108.07 kB +0.01% +5 B 🔺
@sentry/cloudflare (withSentry) - minified 170.66 kB - -
@sentry/cloudflare (withSentry) 430.47 kB - -

View base workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant