Skip to content

fix(tests): repair Playwright suite against the compose stack - #6958

Open
jonpspri wants to merge 1 commit into
mainfrom
playwright-compose-repair
Open

jonpspri wants to merge 1 commit into
mainfrom
playwright-compose-repair

Conversation

@jonpspri

Copy link
Copy Markdown
Collaborator

Repairs the Playwright UI suite so it runs green against the standard compose testing stack.

What was broken

  • ~150 tests failed 401: the runner minted JWTs with the config-default changeme secret while the compose gateway validates with the compose secret.
  • admin_api fixture fell back silently: its JWT path called an unimported helper; the swallowed NameError produced an empty token.
  • test_edit_modal_auth_type_oauth_fields flaked: it assumed table row 0 is non-OAuth; leftover OAuth gateways from sibling suites broke that.
  • test_select_all_prompts_survives_search timed out: a DB holding only team-scoped prompts left the modal list empty.
  • test_developer_rpc_tools_call_not_denied hit httpbin.org: degraded externals returned retryable 503s, and the gateway backoff pushed past the client timeout.

Fixes

  • run_playwright_test exports the compose-matching JWT_SECRET_KEY and PLATFORM_ADMIN_PASSWORD.
  • The admin_api fallback uses make_test_jwt with the same claims as the working api_request_context fixture.
  • The OAuth-fields test picks a non-OAuth gateway from the API and opens it by name (GatewaysPage.open_edit_modal_by_name), avoiding hidden DOM rows left by client-side search.
  • The prompts test creates and cleans up its own prompt.
  • The RPC tool test uses a local fast-404 endpoint instead of httpbin.org.

Signed-off-by: Jonathan Springer jps@s390x.com

- Makefile: export compose-matching JWT_SECRET_KEY and PLATFORM_ADMIN_PASSWORD in run_playwright_test — test-minted JWTs were signed with the config-default 'changeme' secret while the gateway validates with the compose secret, so ~150 tests failed with 401 'Invalid authentication credentials'
- conftest: admin_api fixture's JWT fallback called unimported _create_jwt_token (NameError swallowed by 'except Exception: pass'), producing an empty token; use make_test_jwt with the same claims as the working api_request_context fixture
- test_edit_modal_auth_type_oauth_fields: no longer assumes table row 0 is non-OAuth (leftover OAuth test gateways from other suites legitimately render OAuth fields on open); picks a non-OAuth gateway from the API and opens it by name via new GatewaysPage.open_edit_modal_by_name (client-side search filters leave hidden rows in the DOM that index-based selection trips on)
- test_select_all_prompts_survives_search: create and clean up a prompt so the modal's prompt list is never empty (a DB holding only team-scoped prompts from other suites timed out the wait)
- test_developer_rpc_tools_call_not_denied: replace external httpbin.org tool URL with a local fast-404 endpoint — degraded httpbin returned retryable 503s, and the gateway's exponential backoff pushed the call past the client timeout

Signed-off-by: Jonathan Springer <jps@s390x.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant