diff --git a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md index 678f5c76..8e3bbbba 100644 --- a/domains/testing/skills/mobile-visual-testing/references/cli-reference.md +++ b/domains/testing/skills/mobile-visual-testing/references/cli-reference.md @@ -1,19 +1,18 @@ -# MM CLI Command Reference for Mobile (iOS) +# MM CLI Command Reference for Mobile (iOS and Android) -Command reference for the prod-only MetaMask Mobile consumer. The generic core CLI exposes commands for multiple consumers; availability here is determined by the iOS platform driver and mobile session manager. +Command reference for the prod-only MetaMask Mobile consumer. The generic core CLI exposes commands for multiple consumers. Availability here is determined by the iOS or Android platform driver and mobile session manager. ## Contents - [Syntax Rules](#syntax-rules) - [Lifecycle](#lifecycle) -- [Destructive Launch Flags](#destructive-launch-flags) - [Installed State](#installed-state) - [Interaction](#interaction) - [Discovery](#discovery) - [Knowledge Store](#knowledge-store) - [Batching](#batching) - [Hermes CDP](#hermes-cdp) -- [Simulator Selection](#simulator-selection) +- [Device and Emulator Selection](#device-and-emulator-selection) - [Not Available on Mobile](#not-available-on-mobile) ## Syntax Rules @@ -32,7 +31,7 @@ yarn mm click --testId unlock-submit yarn mm click --test-id unlock-submit ``` -The shared CLI parses `--testid`, `--selector`, `--timeout`, and `--within`, but the iOS driver only supports `--testid` (and positional a11y refs). `--selector` and `--within` are accepted by the parser and then **rejected at runtime by the mobile driver** — see [Targeting on iOS](#targeting-on-ios). +The shared CLI parses `--testid`, `--selector`, `--timeout`, and `--within`, but mobile drivers only support `--testid` (and positional a11y refs). `--selector` and `--within` are accepted by the parser and then rejected at runtime by the mobile driver. See [Targeting on Mobile Platforms](#targeting-on-mobile-platforms). ### Accessibility references are positional @@ -54,22 +53,22 @@ Use exactly one targeting method per command: a positional accessibility referen | Type by test ID | `yarn mm type --testid X "text"` | | Type by accessibility reference | `yarn mm type e2 "text"` | -### Targeting on iOS +### Targeting on Mobile Platforms -- **Supported:** `--testid ` and positional a11y refs (`e1`, `e2`, ...). -- **Not supported:** `--selector ` throws `CSS selectors are not supported on mobile`. `--within ` throws `Scoped element search (within) is not supported on mobile`. The parser accepts both flags, but the iOS driver rejects them at runtime. -- **Matching is fuzzy and case-insensitive.** idb matches accessibility label/identifier by substring, so `--testid Confirm` can match `Confirm Transaction`. Use exact, unique test IDs to disambiguate; there is no `--within` scoping fallback on mobile. -- **`--testid` is lowercase.** `--testId` is not recognized as a flag and its value is treated as a positional target. +* **Supported:** `--testid ` and positional accessibility references (`e1`, `e2`, ...). +* **Not supported:** `--selector ` throws `CSS selectors are not supported on mobile`. `--within ` throws `Scoped element search (within) is not supported on mobile`. The parser accepts both flags, but the mobile driver rejects them at runtime. +* **Matching is fuzzy and case-insensitive.** iOS IDB matches accessibility label and identifier by substring, so `--testid Confirm` can match `Confirm Transaction`. Android ADB/UIAutomator matches resource ID, content-description, or text by substring. Use exact, unique test IDs to disambiguate. There is no `--within` scoping fallback on mobile. +* **`--testid` is lowercase.** `--testId` is not recognized as a flag and its value is treated as a positional target. ## Lifecycle | Command | Description | | --- | --- | -| `yarn mm launch [options]` | Start the daemon and launch an iOS session using the installed app state | +| `yarn mm launch [options]` | Start the daemon and launch a mobile session using the installed app state | | `yarn mm cleanup` | Close the app and clear the active session | -| `yarn mm cleanup --shutdown` | Clean up the session and stop the daemon | +| `yarn mm cleanup --shutdown` | Clean up the session and request iOS Simulator shutdown. iOS-only; on Android use plain `yarn mm cleanup`, which never stops or wipes the emulator | | `yarn mm status` | Show daemon and session status | -| `yarn mm stop [--force]` | Stop the daemon; `--force` also clears stale daemon state | +| `yarn mm stop [--force]` | Stop the daemon. `--force` also clears stale daemon state | | `yarn mm serve [--background]` | Start the daemon without launching a session | ### Supported launch flags @@ -78,42 +77,19 @@ The locally installed core CLI parser exposes these launch flags that are meanin | Flag | Description | | --- | --- | -| `--platform ios` | Explicitly select the iOS platform (plain launch is preferred, as the mobile consumer automatically handles it) | -| `--device-id ` | Select an iOS Simulator by UDID | -| `--app-bundle ` | Install a specific `.app` bundle before launching (for example `ios/build/MetaMask.app`). Required before any destructive flag | -| `--metro-port ` | Attach to a running Metro bundler on the given port. Equivalent to `MM_METRO_PORT`; the flag wins when both are set | +| `--platform ` | Explicitly select the target platform. Accepted values are `browser`, `ios`, or `android`. `--platform browser` is a compatibility alias that routes to iOS. Plain launch defaults to iOS. | +| `--device-id ` | Select an iOS Simulator by UDID, or an Android emulator by serial (e.g., `emulator-5554`). Do not use `--device`. | | `--goal ` | Record the session goal in knowledge metadata | | `--flow-tags ` | Record comma-separated flow tags | -| `--force` | Replace an existing active session | +| `--force` | Request replacement of an active session | -Launch timeout is 120 seconds. The project CLI does not build MetaMask. Install the intended app on the simulator before launching, or pass `--app-bundle ` to install a specific build. +Android readiness allows up to 120 seconds, and the daemon request budget is 180 seconds so cold Metro startup and observation can complete. The project CLI does not build MetaMask. Install the intended app on the simulator or emulator before launching. -The destructive flags `--reinstall`, `--reset-app-data`, and `--allow-fox-code-mismatch` are also parsed but guarded — see [Destructive Launch Flags](#destructive-launch-flags). For command-line Metro attachment, use `--metro-port ` or the `MM_METRO_PORT` environment variable as documented below. - -## Destructive Launch Flags - -These flags replace the installed app and destroy the wallet state it holds, so this prod-only consumer guards them. Do not use them on a wallet you need to preserve. - -| Flag | Description | -| --- | --- | -| `--reinstall` | Uninstall and reinstall the app before launching. Rejected unless `--app-bundle` is also supplied | -| `--reset-app-data` | Clear the app container/wallet state before launching. Rejected unless `--app-bundle` is also supplied | -| `--allow-fox-code-mismatch` | Bypass the app-identity guard so a bundle with a different `fox_code` can be installed. May make existing wallet/keychain data unreadable | - -Guardrails enforced at launch: - -- `--reinstall` and `--reset-app-data` are rejected unless `--app-bundle` is also supplied, because the installed app is otherwise the only copy and would be destroyed by the uninstall step. The rejection surfaces as `MM_INVALID_CONFIG`. -- Installing a bundle whose `fox_code` differs from the installed app is rejected (`MM_INVALID_CONFIG`) unless `--reinstall` or `--allow-fox-code-mismatch` is passed. -- A warning is printed to stderr whenever a destructive flag is honored. - -```bash -# Replace the installed app with a local build, discarding wallet state -yarn mm launch --app-bundle ios/build/MetaMask.app --reinstall -``` +For command-line Metro attachment, pass `--metro-port ` at launch or set the `MM_METRO_PORT` environment variable. When both are supplied, the `--metro-port` flag wins. See [Hermes CDP](#hermes-cdp). ## Installed State -`yarn mm launch` opens the MetaMask app already installed on the selected simulator and preserves its current wallet state. The workflow does not guarantee: +`yarn mm launch` opens the MetaMask app already installed on the selected simulator or emulator and preserves its current wallet state. The workflow does not guarantee: - an onboarding or unlocked screen - a password @@ -132,20 +108,20 @@ The generic core package may display environment-selection and test-state comman | `yarn mm get-text ` | Read an element's text | | `yarn mm wait-for ` | Wait for an element to become visible | -All interaction commands accept on iOS: +All interaction commands accept on mobile: - `--timeout `: one deadline for visibility and action - `--testid `: target by test ID -`--selector` and `--within` are parsed but rejected by the iOS driver (see [Targeting on iOS](#targeting-on-ios)). To disambiguate duplicate targets, use a unique test ID or the exact element's fresh a11y ref — there is no scoped-search fallback on mobile. +`--selector` and `--within` are parsed but rejected by the mobile driver. See [Targeting on Mobile Platforms](#targeting-on-mobile-platforms). To disambiguate duplicate targets, use a unique test ID or the exact element's fresh a11y ref. There is no scoped-search fallback on mobile. ## Discovery | Command | Description | | --- | --- | | `yarn mm describe-screen` | Return app state, visible test IDs, accessibility tree, and prior knowledge | -| `yarn mm screenshot [--name ]` | Capture the current simulator screen | -| `yarn mm accessibility-snapshot` | Return a trimmed accessibility tree (the shared `--root ` flag is ignored on iOS; the full tree is always returned) | +| `yarn mm screenshot [--name ]` | Capture the current screen | +| `yarn mm accessibility-snapshot` | Return a trimmed accessibility tree. The shared `--root ` flag is ignored on mobile; the full tree is always returned. | | `yarn mm list-testids [--limit ]` | List visible test IDs | | `yarn mm get-state` | Return the current app-state snapshot | | `yarn mm get-context` | Report the static prod environment and available mobile capabilities | @@ -171,15 +147,15 @@ The argument must be a JSON object containing a `steps` array: yarn mm run-steps '{"steps":[ {"tool":"click","args":{"a11yRef":"e3"}}, {"tool":"wait_for","args":{"testId":"home-screen","timeoutMs":10000}} -]}' + ]}' ``` | Parameter | Description | | --- | --- | | `steps` | Required array of `{tool, args}` objects | -| `stopOnError` | Stop after the first failure; defaults to `false` (remaining steps still run). Set `true` to abort on the first error | +| `stopOnError` | Stop after the first failure. Defaults to `false` (remaining steps still run). Set `true` to abort on the first error | | `includeObservations` | `'all'`, `'none'`, or `'failures'` | -| `batchTimeoutMs` | Overall deadline; remaining steps are skipped after expiry | +| `batchTimeoutMs` | Overall deadline. Remaining steps are skipped after expiry | Each step is independently checked for mobile platform support. Do not put browser-only commands in a mobile batch. @@ -212,27 +188,61 @@ Read-only, mobile-only. Lists and diagnoses the debuggable React Native Hermes t The driver verifies app identity and device pinning before executing commands. -Start Metro attachment before runtime inspection, using either the `--metro-port` flag or the `MM_METRO_PORT` environment variable (the flag wins when both are set): +Start Metro attachment before runtime inspection. Pass the port with the `--metro-port` flag or the `MM_METRO_PORT` environment variable (the flag wins when both are set): ```bash yarn watch:clean -yarn mm launch --metro-port 8081 -# Equivalent, still supported -MM_METRO_PORT=8081 yarn mm launch +# iOS +yarn mm launch --platform ios --metro-port 8081 + +# Android +yarn mm launch --platform android --metro-port 8081 + +# Environment-variable form (equivalent when the flag is absent) +MM_METRO_PORT=8081 yarn mm launch --platform android ``` On Node 20, add `NODE_OPTIONS="--experimental-websocket"` when launching. Run `yarn mm describe-screen` after runtime mutation to resynchronize observations. -## Simulator Selection +### Android Metro Safety and ADB Reverse Mappings + +When using Metro with the Android emulator: +1. Status Check: The workflow verifies the Metro `/status` response, checking for the canonical body `packager-status:running` before performing any ADB reverse port mapping. +2. Identical Mappings: Existing identical port mapping configurations on the emulator are reused. +3. Conflict Rejection: Any conflicting port mappings fail the launch to prevent cross-worktree interference. +4. Clean Teardown: Session cleanup removes only the specific, matching session-created reverse port mapping. +5. Deep Links: Deep-link triggers use the explicit launcher path `io.metamask/io.metamask.MainActivity`, pass only the Metro origin in Expo's `url` parameter, and keep `disableOnboarding=1` as a separate Expo parameter. Do not put a pre-built `/index.bundle?...` URL in `url` because it breaks custom Metro asset resolution, including `.riv` files. + +## Device and Emulator Selection + +### iOS Simulator ```bash xcrun simctl list devices xcrun simctl boot -yarn mm launch --device-id +yarn mm launch --platform ios --device-id +``` + +### Android Emulator + +Ensure the emulator is already running and authorized. Physical devices are unsupported. The workflow does not start or stop emulators, and does not install APKs. + +```bash +adb devices -l +``` + +For Android selection, omitting `--device-id` requires exactly one online authorized `emulator-*` to exist. Supplying an explicit `--device-id` allows other emulators to coexist, but the selected serial must be online and authorized. + +```bash +# Automatic selection (requires exactly one online/authorized emulator) +yarn mm launch --platform android + +# Explicit selection +yarn mm launch --platform android --device-id emulator-5554 ``` -Prefer one booted simulator and one Metro process per worktree, especially during Hermes inspection. +Prefer one booted simulator/emulator and one Metro process per worktree, especially during Hermes inspection. ## Not Available on Mobile @@ -242,15 +252,15 @@ The core CLI is shared across consumers, so its help lists commands that this pr | Command | Mobile behavior or alternative | | --- | --- | -| `yarn mm navigate ` | Browser-only; navigate through visible UI elements | -| `yarn mm navigate-home` | Not implemented; click the Wallet tab in the UI | -| `yarn mm navigate-settings` | Not implemented; click the Settings tab in the UI | +| `yarn mm navigate ` | Browser-only. Navigate through visible UI elements | +| `yarn mm navigate-home` | Not implemented. Click the Wallet tab in the UI | +| `yarn mm navigate-settings` | Not implemented. Click the Settings tab in the UI | | `yarn mm switch-to-tab` | Browser tabs do not exist in the mobile session | | `yarn mm close-tab` | Browser tabs do not exist in the mobile session | | `yarn mm wait-for-notification` | Browser notification-page command | | `yarn mm clipboard` | Browser command in the current CLI surface | | `yarn mm mock-network` | Browser-only network interception | -| `yarn mm build` | No mobile build capability; build and install separately | +| `yarn mm build` | No mobile build capability. Build and install separately | ### E2E-context only (not applicable to prod-only mobile) @@ -258,8 +268,7 @@ This consumer always reports the static `prod` environment and provides no local | Command | Mobile behavior or alternative | | --- | --- | -| `yarn mm set-context` | Environment switching is unavailable; the mobile consumer is always `prod` | -| `yarn mm get-state` | E2E fixture/state snapshot; use `yarn mm describe-screen` for live UI state | +| `yarn mm set-context` | Environment switching is unavailable. The mobile consumer is always `prod` | | `yarn mm seed-contract` | Contract seeding needs the e2e chain, which this consumer does not run | | `yarn mm seed-contracts` | Contract seeding needs the e2e chain, which this consumer does not run | | `yarn mm get-contract-address` | Depends on seeded e2e contracts that do not exist here | diff --git a/domains/testing/skills/mobile-visual-testing/references/error-recovery.md b/domains/testing/skills/mobile-visual-testing/references/error-recovery.md index 95c34f9d..10635280 100644 --- a/domains/testing/skills/mobile-visual-testing/references/error-recovery.md +++ b/domains/testing/skills/mobile-visual-testing/references/error-recovery.md @@ -1,4 +1,4 @@ -# Error Recovery and Troubleshooting for Mobile (iOS) +# Error Recovery and Troubleshooting for Mobile (iOS and Android) ## Contents @@ -11,15 +11,26 @@ ## On Failure -If launch or the iOS toolchain is the problem (not an in-app screen issue), run the environment doctor first: +If launch or the mobile toolchain is the problem (not an in-app screen issue), troubleshoot based on your platform. + +### iOS + +Run the environment doctor: ```bash yarn mm:doctor ``` -It prints a PASS/FAIL report for Xcode, `idb`, `idb_companion`, and a booted simulator, with install commands for anything missing, and exits non-zero when a prerequisite is absent. This is the fastest way to resolve `MM_DEPENDENCIES_MISSING` and `MM_DEVICE_NOT_AVAILABLE`. +It is explicitly iOS-only. It prints a PASS/FAIL report for Xcode, `idb`, `idb_companion`, and a booted simulator, with install commands for anything missing. This is the fastest way to resolve `MM_DEPENDENCIES_MISSING` and `MM_DEVICE_NOT_AVAILABLE`. + +### Android + +1. Run `adb devices -l` to check your emulator status. +2. For automatic selection, confirm exactly one online authorized emulator exists. For explicit selection, confirm your targeted serial is online and authorized. +3. Ensure `io.metamask` is installed and the MainActivity is launchable. +4. Verify there is no `.device-session` override file in your current directory or worktree root. -For in-app failures: +For in-app failures on either platform: 1. Run `yarn mm describe-screen`. 2. Identify the current screen and visible blockers. If the screen is unknown, capture a screenshot. @@ -36,8 +47,6 @@ For in-app failures: ## Error Codes -Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only preserves a consumer-thrown code when it is a known core code and otherwise collapses it into `MM_LAUNCH_FAILED`, so the iOS driver reports launch and prerequisite failures with core codes (`MM_DEPENDENCIES_MISSING`, `MM_DEVICE_NOT_AVAILABLE`, `MM_INVALID_CONFIG`, `MM_LAUNCH_FAILED`). The iOS-specific detail is carried in the message and remediation text, not in a dedicated `MM_IOS_*` code. - ### Interaction Errors | Code | Meaning and recovery | @@ -50,8 +59,8 @@ Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only pre | `MM_TYPE_TIMEOUT` | Input stalled. Describe the screen, obtain a fresh target, and retry. | | `MM_GETTEXT_FAILED` | The target detached or does not expose text. Re-describe and re-target. | | `MM_GETTEXT_TIMEOUT` | Text retrieval exceeded the deadline. Retry with a fresh target or larger `--timeout`. | -| `MM_PAGE_CLOSED` | The target closed during the action. This can be expected for transitions; inspect current state. | -| `MM_BATCH_TIMEOUT` | `run-steps` exceeded `batchTimeoutMs`. Reduce the batch or increase its deadline. | +| `MM_PAGE_CLOSED` | The target closed during the action. This can be expected for transitions. Inspect the current state. | +| `MM_BATCH_TIMEOUT` | `run-steps` exceeded `batchTimeoutMs`. Reduce the batch size or increase its deadline. | ### Session and Launch Errors @@ -59,10 +68,13 @@ Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only pre | --- | --- | | `MM_SESSION_ALREADY_RUNNING` | A session or launch already exists. Run `yarn mm cleanup`, then launch again. | | `MM_NO_ACTIVE_SESSION` | No app session exists. Run `yarn mm launch`. | -| `MM_LAUNCH_FAILED` | The app or platform driver failed to start. Run `yarn mm cleanup` and retry; inspect the simulator, installed app, and `.mm-daemon.log`. | -| `MM_DEPENDENCIES_MISSING` | Xcode command-line tools or `idb` (Facebook iOS Debug Bridge) are missing. Run `yarn mm:doctor`, then install `idb` with `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb`. | -| `MM_DEVICE_NOT_AVAILABLE` | No simulator is booted, the given UDID does not exist, or `simctl` failed. Run `xcrun simctl list devices` and boot one; verify `--device-id`. | -| `MM_INVALID_CONFIG` | The launch options are not usable: no app installed and no `--app-bundle`, a destructive flag without `--app-bundle`, a `fox_code` mismatch, an unreachable Metro port, or an e2e-only option in this prod-only workflow. Read the remediation text in the error. | +| `MM_LAUNCH_FAILED` | Launch failed. `@metamask/client-mcp-core` collapses unknown consumer codes into this; iOS-specific detail is carried in the message and remediation. Inspect the simulator/emulator, installed app, and `.mm-daemon.log`. | +| `MM_DEPENDENCIES_MISSING` | Xcode command-line tools or `idb` are missing. Run `yarn mm:doctor`, then `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb`. | +| `MM_DEVICE_NOT_AVAILABLE` | No simulator is booted, the given UDID does not exist, or `simctl` failed. Run `xcrun simctl list devices` and boot one; verify MetaMask is installed. | +| `MM_INVALID_CONFIG` | The launch options are unusable: no app and no `--app-bundle`, a destructive flag without `--app-bundle`, a `fox_code` mismatch, an unreachable Metro port, or an E2E-only option in this prod-only workflow. Read the remediation text and reuse the installed app or install a matching build. | +| `MM_ANDROID_DEPENDENCY_MISSING` | `adb` is not on your `PATH`. Install Android SDK Platform-Tools and verify with `adb version`. | +| `MM_ANDROID_RUNNER_NOT_READY` | Automatic selection requires exactly one online authorized emulator. Explicit selection requires the named serial to be online and authorized. Both require a completed boot, the installed package, and launcher activity. | +| `MM_ANDROID_BACKEND_INTEGRITY` | ADB backend could not be created because of conflicts. Remove `.device-session` from the current directory or worktree root and retry. | | `MM_PORT_IN_USE` | The daemon port is already bound by stale state. Run `yarn mm stop --force`, then launch again. | | `MM_INVALID_INPUT` | A command or flag value is malformed. Correct it before retrying. | @@ -70,17 +82,17 @@ Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only pre | Code | Meaning and recovery | | --- | --- | -| `MM_DISCOVERY_FAILED` | A `describe-screen` / accessibility snapshot failed. Wait for transitions to settle, capture a screenshot, and verify the app has not crashed, then retry. | -| `MM_SCREENSHOT_FAILED` | The simulator screen capture failed. Verify the simulator is booted and the session is active. | +| `MM_DISCOVERY_FAILED` | A `describe-screen` or accessibility snapshot failed. Wait for transitions to settle, capture a screenshot, and verify the app has not crashed, then retry. | +| `MM_SCREENSHOT_FAILED` | The screen capture failed. Verify the simulator or emulator is booted and the session is active. | ### Hermes and Platform Errors -`cdp` and `hermes-targets` run against the Hermes runtime through Metro. The idb workflow surfaces a small set of codes here — do not expect granular per-phase Hermes codes. +`cdp` and `hermes-targets` run against the Hermes runtime through Metro. The mobile workflow surfaces a small set of codes here. Do not expect granular per-phase Hermes codes. | Code | Meaning and recovery | | --- | --- | -| `MM_HERMES_NOT_AVAILABLE` | `hermes-targets` was run on a session with no mobile driver, or Metro is not attached. Launch with `--metro-port ` (or `MM_METRO_PORT`) set and retry. | -| `MM_HERMES_FAILED` | Hermes target discovery failed. Verify Metro is running, the app is a development build, and one simulator/one Metro process per worktree. | +| `MM_HERMES_NOT_AVAILABLE` | `hermes-targets` was run on a session with no mobile driver, or Metro is not attached. Launch with `MM_METRO_PORT` set and retry. | +| `MM_HERMES_FAILED` | Hermes target discovery failed. Verify Metro is running, the app is a development build, and one simulator or emulator and one Metro process per worktree. | | `MM_CDP_BLOCKED` | The requested `cdp` method is blocked as destructive. Use a safe inspection method instead. | | `MM_CDP_FAILED` | `cdp` execution failed or timed out. On Node 20 confirm `--experimental-websocket` was set at launch; inspect Metro and `.mm-daemon.log`. | | `MM_TOOL_NOT_SUPPORTED_ON_PLATFORM` | The command is browser-only (or a mobile-only command was run without a mobile session). Use visible mobile UI interactions instead. | @@ -89,26 +101,28 @@ Launch errors use the core `ErrorCode` set. `@metamask/client-mcp-core` only pre | Symptom | Likely cause | Safe solution | | --- | --- | --- | -| Previous session blocks launch | Session was not cleaned up | `yarn mm cleanup`, then `yarn mm launch` | -| No active session | The app has not been launched through the daemon | `yarn mm launch` | -| Launch cannot locate MetaMask | MetaMask is not installed on the selected simulator | Install the intended app build on that simulator, then launch again | -| Simulator is unavailable | No booted simulator or incorrect UDID | Check `xcrun simctl list devices`, boot the intended device, and use `--device-id ` | -| App identity mismatch (`MM_INVALID_CONFIG`, `different fox_code`) | Installed and requested builds have different `fox_code` values | Reuse the existing installed app, or install a matching build with `--app-bundle` plus `--reinstall` / `--allow-fox-code-mismatch` (destructive to wallet state) | -| `idb` not installed | Missing iOS Debug Bridge dependency (`MM_DEPENDENCIES_MISSING`) | Run `yarn mm:doctor`, then `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb` | +| Previous session blocks launch | Session was not cleaned up | Run `yarn mm cleanup`, then `yarn mm launch` | +| No active session | The app has not been launched through the daemon | Run `yarn mm launch` with correct platform flag | +| Launch cannot locate MetaMask on iOS | MetaMask is not installed on the selected simulator | Install the intended app build on that simulator, then launch again | +| Launch cannot locate MetaMask on Android | `io.metamask` is not installed on the emulator | Build and install the app on that emulator externally using `yarn setup` and `yarn start:android`. Do not use installation commands inside the `mm` workflow. | +| Simulator is unavailable | No booted simulator or incorrect UDID | Check `xcrun simctl list devices`, boot the device, and use `--device-id ` | +| Android emulator is offline or unauthorized | Emulator is stuck or needs confirmation | Run `adb devices -l` to check authorization. Confirm authorization on-screen, restart the emulator externally to restore the online device state, or select a different online authorized emulator. | +| Android emulator is missing or multiple exist | Zero or multiple emulators are running | For automatic selection, keep exactly one eligible emulator online and authorized. Otherwise, explicitly target an eligible serial via `--device-id `. | +| App identity mismatch on iOS | Installed and expected builds have different `fox_code` values | Reuse the existing installed app or install a matching build outside the `mm` workflow | +| `idb` not installed on iOS | Missing iOS Debug Bridge dependency (`MM_DEPENDENCIES_MISSING`) | Run `yarn mm:doctor`, then install via Homebrew and pip3 | | Empty or failed screen snapshot | Splash screen, animation, app transition, or crash | Wait briefly, describe again, and capture a screenshot | | Stale accessibility references | Screen changed after the references were generated | Run `yarn mm describe-screen` and use fresh references | -| Interaction timeout | Animation, overlay, or stale/ambiguous target | Re-`describe-screen`, target by a unique test ID or fresh a11y ref (no `--within` on mobile), and increase `--timeout` only when appropriate | +| Interaction timeout | Animation, overlay, or stale/ambiguous target | Re-`describe-screen`, target by a unique test ID or fresh accessibility ref. Do not use `--within` or `--selector` on mobile. Increase `--timeout` only when appropriate | | `--testId` times out | Incorrect capitalization | Use `--testid` in lowercase | | Daemon address already in use | Stale daemon state (`MM_PORT_IN_USE`) | Run `yarn mm stop --force`, then `yarn mm launch` | +| After code changes, daemon is stale | Code/skill updates require a clean state | Run `yarn mm stop --force`, then relaunch without wiping state | ## Safe App Resolution -The mobile workflow preserves the installed app and its wallet state by default. - -For a `fox_code` mismatch (`MM_INVALID_CONFIG`, message contains `different fox_code`): +The mobile workflow preserves the installed app and its wallet state by default. Android has no APK lifecycle. 1. Prefer launching the already-installed app. -2. If replacing it, build and install a matching app outside of the `mm` workflow (such as through Xcode or standard repository setup scripts), or install it through the guarded `mm` path with `--app-bundle --reinstall`. Both `--reinstall` and `--allow-fox-code-mismatch` are destructive to existing wallet state. +2. If replacing it, build and install a matching app outside of the `mm` workflow (such as through standard repository setup scripts). Do not recommend `yarn start:android` as part of the `mm` workflow; it should only be run externally when setting up the environment. ## Daemon Issues @@ -117,7 +131,7 @@ If the CLI hangs or returns connection errors: 1. Check status: `yarn mm status`. 2. Stop stale state: `yarn mm stop --force`. 3. Inspect `.mm-daemon.log`. -4. Restart with `yarn mm launch`. +4. Restart with `yarn mm launch` (with platform flag if needed). The daemon shuts down after 30 minutes of inactivity. Its state is stored in `.mm-server` at the project root, isolated per worktree. @@ -126,16 +140,21 @@ The daemon shuts down after 30 minutes of inactivity. Its state is stored in `.m If Metro attachment or Hermes inspection fails: 1. Start Metro with `yarn watch:clean`. -2. Verify its status endpoint, normally `http://localhost:8081/status`. -3. Ensure `--metro-port ` (or `MM_METRO_PORT`) matches the running Metro process. -4. Confirm the installed app is a compatible development build and belongs to the selected simulator. -5. Use one Metro process and one active simulator per worktree to avoid target ambiguity. +2. Verify its status endpoint, normally `http://localhost:8081/status`. The workflow verifies the exact Metro `/status` response (expecting the canonical body `packager-status:running`) before any ADB reverse mapping. +3. Ensure the port passed via `--metro-port` (or `MM_METRO_PORT`; the flag wins) matches the running Metro process. +4. Confirm the installed app is a compatible development build and belongs to the selected simulator or emulator. +5. Use one Metro process and one active simulator or emulator per worktree to avoid target ambiguity. 6. Inspect `.mm-daemon.log` for attachment and target-selection details. 7. Restart safely: ```bash yarn mm cleanup - yarn mm launch --metro-port 8081 + + # iOS + yarn mm launch --platform ios --metro-port 8081 + + # Android + yarn mm launch --platform android --metro-port 8081 ``` On Node 20, launch the daemon with `NODE_OPTIONS="--experimental-websocket"` when Hermes WebSocket support is required. Node 22 and later provide WebSocket support directly. diff --git a/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md b/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md index 3f4ce7f4..39989b58 100644 --- a/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md +++ b/domains/testing/skills/mobile-visual-testing/references/runtime-monitoring.md @@ -1,6 +1,6 @@ -# Runtime Monitoring — Network & Console Capture +# Runtime Monitoring: Network and Console Capture -Inject interceptors into the Hermes runtime to capture `fetch` requests (including JSON-RPC method names and errors), and `console` output during visual testing. Read them back at any point to detect silent failures, slow API calls, RPC errors, or unexpected behavior that doesn't surface in the UI. +Inject a namespaced monitoring global into the Hermes runtime to inspect `fetch` calls, JSON-RPC queries, and console logs. Running these checks helps discover silent failures, RPC errors, or console exceptions during visual testing. ## Contents @@ -10,276 +10,112 @@ Inject interceptors into the Hermes runtime to capture `fetch` requests (includi - [Check Interceptor Health](#check-interceptor-health) - [Dismissing Dev Error Overlays](#dismissing-dev-error-overlays) - [Anomaly Detection](#anomaly-detection) -- [Daemon & Metro Logs](#daemon--metro-logs) +- [Daemon and Metro Logs](#daemon-and-metro-logs) +- [Uninstall Interceptors](#uninstall-interceptors) +- [Final Cleanup Checklist](#final-cleanup-checklist) - [Gotchas](#gotchas) ## Prerequisites -- Metro must be running (`MM_METRO_PORT` set at launch time) -- Same CDP requirements as [state-manipulation.md](state-manipulation.md) — Node 20 needs `--experimental-websocket` -- Interceptors operate in the JS thread only — native-layer network calls (e.g., iOS URLSession) are not captured +- Metro must be active on `MM_METRO_PORT` when launching the daemon. +- Connections require Chrome DevTools Protocol (CDP). Node 20 requires the `--experimental-websocket` flag passed to the daemon. Version 22 and later support WebSockets natively. +- Interceptors capture the JS thread only. Native layer requests like iOS URLSession or Android native networking remain un-intercepted. ## Install Interceptors -Run once after launch. Installs both fetch and console interceptors in a single call. Idempotent — safe to re-run. +Run this command once after launch to install the namespaced monitor. The installation is idempotent and will not re-wrap fetch or console methods if already installed. ```bash -yarn mm cdp Runtime.evaluate '{"expression":"(function(){if(globalThis.__mmNet)return JSON.stringify(\"already installed\");var of=globalThis.fetch;globalThis.__mmNet=[];globalThis.fetch=function(){var u=arguments[0],o=arguments[1]||{},e={t:Date.now(),method:o.method||\"GET\",url:typeof u===\"string\"?u:(u&&u.url)||String(u)};if(o.body){try{var b=typeof o.body===\"string\"?JSON.parse(o.body):o.body;if(b&&b.method)e.rpcMethod=b.method}catch(x){}}globalThis.__mmNet.push(e);if(globalThis.__mmNet.length>2000)globalThis.__mmNet=globalThis.__mmNet.slice(-1000);return of.apply(globalThis,arguments).then(function(r){e.status=r.status;e.ms=Date.now()-e.t;if(e.rpcMethod){try{r.clone().text().then(function(t){try{var j=JSON.parse(t);if(j.error){e.rpcError=(j.error.message||JSON.stringify(j.error)).substring(0,200);e.rpcErrorCode=j.error.code}}catch(x){}}).catch(function(){})}catch(x){}}return r}).catch(function(err){e.err=String(err);e.ms=Date.now()-e.t;throw err})};globalThis.__mmCon=[];[\"log\",\"warn\",\"error\"].forEach(function(l){var orig=console[l];console[l]=function(){var a=Array.prototype.slice.call(arguments);globalThis.__mmCon.push({t:Date.now(),level:l,msg:a.map(function(x){try{return String(x)}catch(e){return\"[unstringifiable]\"}}).join(\" \")});if(globalThis.__mmCon.length>1000)globalThis.__mmCon=globalThis.__mmCon.slice(-500);return orig.apply(console,arguments)}});return JSON.stringify(\"interceptors installed\")})()","returnByValue":true}' +yarn mm cdp Runtime.evaluate '{"expression":"(function(){if(globalThis.__mmMonitor)return JSON.stringify(\"already installed\");function sanitizeUrl(str){if(typeof str!==\"string\")return\"\";var cleaned=str.replace(/([?&])([^=]*(?:key|token|secret|auth|credential)[^=]*)=([^&#]*)/gi,\"$1$2=[REDACTED]\");cleaned=cleaned.replace(/(\\/v[23]\\/)([a-zA-Z0-9_-]+)/g,\"$1[REDACTED]\");return cleaned}var monitor={origFetch:globalThis.fetch,origLog:console.log,origWarn:console.warn,origError:console.error,net:[],con:[]};globalThis.__mmMonitor=monitor;globalThis.fetch=function(){var url=arguments[0];var opts=arguments[1]||{};var entry={t:Date.now(),method:opts.method||\"GET\",url:sanitizeUrl(typeof url===\"string\"?url:(url&&url.url)||String(url))};if(opts.body){try{var b=typeof opts.body===\"string\"?JSON.parse(opts.body):opts.body;if(b&&b.method)entry.rpcMethod=b.method}catch(x){entry.reqBodyParseFail=true}}monitor.net.push(entry);if(monitor.net.length>2000)monitor.net=monitor.net.slice(-1000);return monitor.origFetch.apply(globalThis,arguments).then(function(resp){entry.status=resp.status;entry.ms=Date.now()-entry.t;if(entry.rpcMethod){try{resp.clone().text().then(function(body){try{var j=JSON.parse(body);if(j.error){entry.rpcError=(j.error.message||JSON.stringify(j.error)).substring(0,200);entry.rpcErrorCode=j.error.code}}catch(x){entry.respJsonParseFail=true}}).catch(function(){entry.respReadFail=true})}catch(x){entry.respCloneFail=true}}return resp}).catch(function(err){entry.err=String(err);entry.ms=Date.now()-entry.t;throw err})};[\"log\",\"warn\",\"error\"].forEach(function(level){var origKey=\"orig\"+level.charAt(0).toUpperCase()+level.slice(1);console[level]=function(){var args=Array.prototype.slice.call(arguments);monitor.con.push({t:Date.now(),level:level,msg:args.map(function(a){try{return String(a)}catch(e){return\"[unstringifiable]\"}}).join(\" \")});if(monitor.con.length>1000)monitor.con=monitor.con.slice(-500);return monitor[origKey].apply(console,arguments)}});return JSON.stringify(\"interceptors installed\")})()","returnByValue":true}' ``` -**Readable version** of the expression: - -```javascript -(function() { - if (globalThis.__mmNet) return JSON.stringify("already installed"); - - // --- Fetch interceptor (with JSON-RPC capture) --- - var origFetch = globalThis.fetch; - globalThis.__mmNet = []; - globalThis.fetch = function() { - var url = arguments[0]; - var opts = arguments[1] || {}; - var entry = { - t: Date.now(), - method: opts.method || "GET", - url: typeof url === "string" ? url : (url && url.url) || String(url), - }; - // Extract JSON-RPC method from request body - if (opts.body) { - try { - var b = typeof opts.body === "string" ? JSON.parse(opts.body) : opts.body; - if (b && b.method) entry.rpcMethod = b.method; - } catch(x) {} - } - globalThis.__mmNet.push(entry); - // Auto-trim: keep last 1000 when buffer exceeds 2000 - if (globalThis.__mmNet.length > 2000) { - globalThis.__mmNet = globalThis.__mmNet.slice(-1000); - } - return origFetch.apply(globalThis, arguments) - .then(function(resp) { - entry.status = resp.status; - entry.ms = Date.now() - entry.t; - // For RPC requests, clone response to extract JSON-RPC errors. - // Without this, RPC errors are invisible — Infura returns HTTP 200 - // with {"error": {"message": "..."}} in the body. - if (entry.rpcMethod) { - try { - resp.clone().text().then(function(body) { - try { - var j = JSON.parse(body); - if (j.error) { - entry.rpcError = (j.error.message || JSON.stringify(j.error)).substring(0, 200); - entry.rpcErrorCode = j.error.code; - } - } catch(x) {} - }).catch(function(){}); - } catch(x) {} - } - return resp; - }) - .catch(function(err) { - entry.err = String(err); - entry.ms = Date.now() - entry.t; - throw err; - }); - }; - - // --- Console interceptor --- - globalThis.__mmCon = []; - ["log", "warn", "error"].forEach(function(level) { - var orig = console[level]; - console[level] = function() { - var args = Array.prototype.slice.call(arguments); - globalThis.__mmCon.push({ - t: Date.now(), - level: level, - msg: args.map(function(a) { - try { return String(a); } catch(e) { return "[unstringifiable]"; } - }).join(" "), - }); - // Auto-trim: keep last 500 when buffer exceeds 1000 - if (globalThis.__mmCon.length > 1000) { - globalThis.__mmCon = globalThis.__mmCon.slice(-500); - } - return orig.apply(console, arguments); - }; - }); - - return JSON.stringify("interceptors installed"); -})() -``` +> ### ⚠️ SECURITY WARNING +> +> * **No payload capture**: The fetch interceptor does not log request headers or general body contents. It only extracts the JSON-RPC method name. +> * **URL Redaction**: Query parameter names containing key, token, secret, auth, or credential (such as api_key, access_token, or authorization) are automatically redacted. Path credentials following /v2/ or /v3/ segments are also masked. +> * **Manual Review Required**: All captured output, including RPC error messages and console logs, can still contain sensitive data. You must review and redact all outputs before sharing or persisting them. ## Read Captured Data -### Drain both buffers (read and clear) +### Drain and Clear Buffers -Returns all captured data and resets the buffers. Use this between test steps or after a flow completes. +Use this command to read all accumulated logs and empty the buffers. ```bash -yarn mm cdp Runtime.evaluate '{"expression":"(function(){var r=JSON.stringify({net:globalThis.__mmNet||[],con:globalThis.__mmCon||[]});globalThis.__mmNet=[];globalThis.__mmCon=[];return r})()","returnByValue":true}' +yarn mm cdp Runtime.evaluate '{"expression":"(function(){if(!globalThis.__mmMonitor)return JSON.stringify({net:[],con:[]});var r=JSON.stringify({net:globalThis.__mmMonitor.net,con:globalThis.__mmMonitor.con});globalThis.__mmMonitor.net=[];globalThis.__mmMonitor.con=[];return r})()","returnByValue":true}' ``` -### Read without clearing +### Peek at Buffers -Peek at the buffers without resetting. Useful when you want to keep accumulating across steps. +Read the logs without clearing them to keep tracking records across steps. ```bash -yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify({net:globalThis.__mmNet||[],con:globalThis.__mmCon||[]})","returnByValue":true}' -``` - -### Response shape - -```json -{ - "net": [ - { "t": 1718000000000, "method": "GET", "url": "https://api.example.com/data", "status": 200, "ms": 142 }, - { "t": 1718000001000, "method": "POST", "url": "https://mainnet.infura.io/v3/KEY", "status": 200, "ms": 85, "rpcMethod": "eth_blockNumber" }, - { "t": 1718000002000, "method": "POST", "url": "https://mainnet.infura.io/v3/KEY", "status": 200, "ms": 3021, "rpcMethod": "eth_sendRawTransaction", "rpcError": "Signer had insufficient balance", "rpcErrorCode": -32000 }, - { "t": 1718000003000, "method": "GET", "url": "https://api.example.com/fail", "err": "TypeError: Network request failed", "ms": 15000 } - ], - "con": [ - { "t": 1718000000500, "level": "warn", "msg": "Deprecated API call: use v2 endpoint" }, - { "t": 1718000001200, "level": "error", "msg": "Unhandled promise rejection: RPC timeout" } - ] -} +yarn mm cdp Runtime.evaluate '{"expression":"(function(){if(!globalThis.__mmMonitor)return JSON.stringify({net:[],con:[]});return JSON.stringify({net:globalThis.__mmMonitor.net,con:globalThis.__mmMonitor.con})})()","returnByValue":true}' ``` -| Field | Type | Description | -|-------|------|-------------| -| `t` | number | Unix timestamp (ms) | -| `method` | string | HTTP method | -| `url` | string | Request URL | -| `status` | number | HTTP response status (absent if request failed) | -| `ms` | number | Duration in milliseconds | -| `err` | string | Network-level error message (absent if fetch succeeded) | -| `rpcMethod` | string | JSON-RPC method from request body (absent for non-RPC requests) | -| `rpcError` | string | JSON-RPC error message from response body (absent if RPC succeeded) | -| `rpcErrorCode` | number | JSON-RPC error code (absent if RPC succeeded) | -| `level` | string | `log`, `warn`, or `error` | -| `msg` | string | Stringified console arguments | - ## Check Interceptor Health -Verify interceptors are still installed. They are lost on app reload or Fast Refresh. +Check if the interceptors are active. They will be reset if the app reloads or encounters a Fast Refresh event. ```bash -yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify({installed:!!globalThis.__mmNet,netCount:(globalThis.__mmNet||[]).length,conCount:(globalThis.__mmCon||[]).length})","returnByValue":true}' +yarn mm cdp Runtime.evaluate '{"expression":"(function(){return JSON.stringify({installed:!!globalThis.__mmMonitor,netCount:globalThis.__mmMonitor?(globalThis.__mmMonitor.net||[]).length:0,conCount:globalThis.__mmMonitor?(globalThis.__mmMonitor.con||[]).length:0})})()","returnByValue":true}' ``` -If `installed` is `false`, re-run the install command. - ## Dismissing Dev Error Overlays -The Engine module walk (`globalThis.__r(id)` loop from [state-manipulation.md](state-manipulation.md)) triggers **3–4 dev error overlays** in development builds. These are caused by modules that throw on import (e.g., `SegmentFetcher` TurboModule not found). The overlays are purely cosmetic — interceptors and cached references survive them — but they block UI interaction until dismissed. - -**You must dismiss ALL overlays, not just the first one.** Each overlay is a separate error stacked on top of the previous. - -### Dismiss loop pattern - -```bash -for i in 1 2 3 4 5 6 7; do - RESULT=$(yarn mm describe-screen 2>&1 | python3 -c " -import json, sys -a = json.load(sys.stdin).get('a11y', {}).get('nodes', []) -dismiss = [n for n in a if n.get('name', '').lower() == 'dismiss'] -print(dismiss[0]['ref'] if dismiss else 'CLEAN') -" 2>&1) - if [ "$RESULT" = "CLEAN" ]; then - echo "All overlays dismissed after $((i-1)) dismissals" - break - fi - yarn mm click "$RESULT" 2>&1 | grep -q clicked && echo "Dismissed overlay $i" - sleep 0.5 -done -``` - -### Recommended instrumentation sequence +Some advanced state walks can trigger red screen overlays in development builds. An overlay may represent a real development-runtime error, so you must capture evidence and inspect the captured or Metro logs before dismissal. After dismissal, run the interceptor health check because an app reload or Fast Refresh event may have removed your instrumentation. You must clear the overlays to interact with the app. -1. Install fetch/console interceptors (safe, no overlays) -2. Run the Engine module walk (triggers overlays) -3. Run the dismiss loop above -4. Verify instrumentation survived: check `!!globalThis.__mmDebugNet` and `!!globalThis.__mmEngine` +### Dismissal Procedure -### Notes - -- Typically **3 overlays** appear, but the count can vary by build. The loop handles up to 7 as a safety margin. -- The overlays do **not** cause an app reload — interceptors and the cached Engine reference remain intact. -- If the app does reload (Fast Refresh, Metro reconnect), you must re-install everything from step 1. -- The "Dismiss" button's a11y ref changes between `describe-screen` calls, so the loop re-queries each iteration. +1. Run `yarn mm describe-screen` to refresh the interactive tree. +2. Search for any visible button named "Dismiss" or matching that target. +3. Click the target button using its fresh accessibility reference. +4. Repeat steps 1 to 3 until no "Dismiss" options remain. +5. Limit your attempts to a maximum of 7 clicks to prevent infinite loops. If you are still blocked, capture a screenshot via `yarn mm screenshot --name "dismiss-failed"` for offline analysis. ## Anomaly Detection -After draining the buffers, flag these patterns: - -### Network anomalies - -| Pattern | How to detect | Severity | -|---------|--------------|----------| -| Failed requests | `status >= 400` or `err` field present | High | -| **JSON-RPC errors** | **`rpcError` field present** (HTTP status will be 200) | **Critical** | -| Slow requests | `ms > 5000` | Medium | -| Network errors | `err` contains "Network request failed" or "aborted" | High | -| Repeated failures | Same `url` failing 3+ times | High | -| Unexpected RPC calls | `url` contains unfamiliar host (not infura, metamask, etc.) | Medium | -| Transaction failures | `rpcMethod` is `eth_sendRawTransaction` and `rpcError` present | Critical | +Scan drained logs for these suspicious signatures: -### Console anomalies +### Network Concerns +* **Failed Requests**: Any entry with a `status >= 400` or an `err` property. +* **JSON-RPC Failures**: Any entry with `rpcError` populated. The HTTP status will remain `200` for these errors. +* **Slow Responses**: Any entry where `ms > 5000`. -| Pattern | How to detect | Severity | -|---------|--------------|----------| -| Errors | `level === "error"` | High | -| Unhandled rejections | `msg` contains "Unhandled" or "rejection" | High | -| React render warnings | `msg` contains "Cannot update a component" or "Maximum update depth" | Medium | -| Deprecation warnings | `msg` contains "deprecated" or "Deprecated" | Low | +### Console Concerns +* **Exceptions**: Level matches `error`. +* **Promise Failures**: Messages containing terms like "Unhandled" or "rejection". +* **Component Loops**: Warning messages containing "Maximum update depth" or "Cannot update a component". -### When to drain +## Daemon and Metro Logs -- **After each major flow step** (e.g., after completing a send transaction) -- **On unexpected UI state** (loading spinner stuck, missing data, blank screen) -- **On test failure** (before reporting — include anomalies in the failure context) -- **At session end** (final drain for the test report) +Two supplementary log files help debug environment issues. Use workspace file-reading or search tools to examine them rather than streaming bash commands: -## Daemon & Metro Logs +- **Daemon logs (`.mm-daemon.log`)**: Located at the project root. This log tracks CLI operations and session initialization. +- **Metro output**: Check the terminal output where Metro was started to review bundler actions and JS build errors. -Two additional log sources complement the JS interceptors: +## Uninstall Interceptors -### Daemon log - -Records CLI events, session lifecycle, and tool execution timing. +Use this command to restore original functions and clean up all namespaces: ```bash -# Read last 50 lines -tail -50 .mm-daemon.log - -# Search for errors -grep -i "error\|fail\|crash" .mm-daemon.log | tail -20 +yarn mm cdp Runtime.evaluate '{"expression":"(function(){if(!globalThis.__mmMonitor)return JSON.stringify(\"not installed\");globalThis.fetch=globalThis.__mmMonitor.origFetch;console.log=globalThis.__mmMonitor.origLog;console.warn=globalThis.__mmMonitor.origWarn;console.error=globalThis.__mmMonitor.origError;delete globalThis.__mmMonitor;return JSON.stringify(\"interceptors uninstalled\")})()","returnByValue":true}' ``` -### Metro output +## Final Cleanup Checklist -Metro logs JS errors, warnings, and bundle events. When running Metro separately (`yarn watch:clean`), redirect output to a file: +Always clean up your testing environment when diagnostics are complete: -```bash -yarn watch:clean 2>&1 | tee .mm-metro.log & -``` - -Then check for errors during testing: - -```bash -grep -i "error\|warn\|exception" .mm-metro.log | tail -20 -``` +1. Drain any remaining logs to secure final reports. +2. Confirm no private or sensitive data is included in your persisted logs. +3. Run the uninstall command to restore the original fetch and console methods. +4. Run `yarn mm describe-screen` to verify the UI state is still active and correct. +5. Revert any other temporary state changes. +6. Run the standard `yarn mm cleanup` command to clear the active session. ## Gotchas -- **JSON-RPC errors return HTTP 200.** Infura and relay services return `{"error": {"message": "..."}}` in the response body with a 200 status code. Without the `rpcError` field extraction (included in the default interceptor above), these errors are completely invisible — a failed `eth_sendRawTransaction` looks identical to a successful one at the HTTP level. Always check `rpcError`, not just `status`, when debugging transaction failures. -- **Many app errors never reach `console.error`.** Controllers like TransactionController catch RPC exceptions internally and update state (e.g., marking a transaction as "failed") without any console output. The console interceptor will not capture these. To detect on-chain failures, check the `rpcError` field in network entries or verify transaction status via the UI/activity list. -- **Response body cloning is async.** The `rpcError` field is populated asynchronously via `resp.clone().text().then(...)`. In rare cases, if you drain the buffer immediately after a request completes (< 1ms), the `rpcError` field may not yet be populated. A brief `sleep 1` before draining after a transaction submission avoids this. -- The Engine module walk triggers **multiple dev error overlays** (typically 3–4). You must dismiss all of them in a loop, not just the first one. See [Dismissing Dev Error Overlays](#dismissing-dev-error-overlays) for the pattern. -- Interceptors are **lost on Fast Refresh and app reload**. After any code change with Metro watch mode, re-run the install command. Use the health check to verify. -- The fetch interceptor captures **JS-level fetch only**. Native HTTP calls (iOS URLSession, image loading, WebSocket connections) are not captured. -- Buffer auto-trim is aggressive (1000-2000 entries) to prevent memory pressure. For long test sessions, drain periodically to avoid losing early entries. -- Console interceptor wraps `log`, `warn`, and `error`. It skips `debug` and `info` to reduce noise. If you need those, modify the `forEach` array in the install expression. -- The `url` field for `fetch(Request)` calls extracts `Request.url`. If the app uses a custom fetch wrapper that passes non-standard first arguments, the `url` may show `[object Object]`. -- `returnByValue: true` is required in the cdp params. Without it, you get a remote object reference instead of the actual JSON string. -- Interceptors add ~0.1ms overhead per fetch call (~0.2ms for RPC requests due to response cloning). No measurable impact on app performance during testing. +- **HTTP 200 RPC errors**: Relays like Infura return a `200` status code even when transactions fail on-chain. Check the `rpcError` property to detect execution failures. +- **Asynchronous body cloning**: Body cloning for RPC parsing is asynchronous. Wait a moment after submitting transactions before draining the logs. +- **Fast Refresh loss**: Interceptors are lost during Metro hot reloads. Run the health check frequently to verify active status. +- **Overhead**: Interception introduces runtime overhead and should be removed after diagnostic tasks are complete. diff --git a/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md b/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md index 024ad84d..6a9d9287 100644 --- a/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md +++ b/domains/testing/skills/mobile-visual-testing/references/state-manipulation.md @@ -1,262 +1,133 @@ -# Runtime State Manipulation (Mobile) +# Runtime State Manipulation (Mobile Development Build) -Use `mm cdp` only for advanced runtime inspection or manipulation of the current installed app during a Metro-attached session. On mobile, `mm cdp` connects to the Hermes JavaScript runtime through Metro's inspector proxy. These operations use `Runtime.evaluate` and require Metro to be running (`MM_METRO_PORT`). +Use `mm cdp` only for advanced runtime inspection or state manipulation of the installed app during a Metro-attached session. This capability is restricted to **development builds** on either iOS or Android. The system connects directly to the Hermes JavaScript runtime through Metro's inspector proxy using `Runtime.evaluate`. Running these commands requires the Metro bundler to be active on `MM_METRO_PORT`. ## Contents - [CDP Basics (Mobile)](#cdp-basics-mobile) -- [Fiber Entry Point (React Native)](#fiber-entry-point-react-native) -- [Operations](#operations) -- [Finding the Engine Singleton](#finding-the-engine-singleton) -- [Verify State After Mutation](#verify-state-after-mutation) -- [When to Use CDP](#when-to-use-cdp) +- [Critical Rules for State Mutation](#critical-rules-for-state-mutation) +- [React Native Fiber Entry Point](#react-native-fiber-entry-point) +- [Read-Only Redux Store Evaluation](#read-only-redux-store-evaluation) +- [Safe Mutation Checklist and Templates](#safe-mutation-checklist-and-templates) +- [Last-Resort Module Discovery](#last-resort-module-discovery) +- [When to Use and Troubleshooting](#when-to-use-and-troubleshooting) ## CDP Basics (Mobile) -`mm cdp` sends a Chrome DevTools Protocol command. On mobile, it connects to the Hermes runtime via Metro's inspector proxy, targeting the **React Native JS thread** (on the extension it targets the browser page instead). +The `mm cdp` command sends Chrome DevTools Protocol payloads to the React Native JS thread on the running mobile device. ```bash -yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(1+1)"}' yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(globalThis.__DEV__)"}' ``` **Requirements:** -- Metro must be running (`MM_METRO_PORT` set at launch time) -- Node 20 requires `--experimental-websocket` flag at daemon launch -- Node 22+ works natively +- Metro must be active with `MM_METRO_PORT` configured at launch. +- Node 20 requires the `--experimental-websocket` flag passed to the daemon. Node 22 and later support WebSockets natively. -CDP calls are **mutating**, so run `mm describe-screen` afterward to re-sync the a11y ref map. +Runtime, controller, or Redux mutations may be in-memory or persisted depending on controller or storage behavior. You must assume persistence until verified and always restore the original state. Run `yarn mm describe-screen` after any mutation to re-sync the accessibility reference map. -## Fiber Entry Point (React Native) +## Critical Rules for State Mutation -The extension walks fibers from a DOM node (`document.getElementById("app-content").__reactFiber$...`). React Native has no DOM. Instead, use `__REACT_DEVTOOLS_GLOBAL_HOOK__`: +Before modifying any runtime state, you must adhere to these absolute rules: + +1. **Inspect before mutating**: Check current values to confirm the app is in the expected state. +2. **Capture the exact original state**: Always read and back up the original values before applying modifications. +3. **Prefer real controller methods**: Use exposed controller interfaces over direct Redux dispatches whenever possible. Real controller methods handle validation and state propagation safely. +4. **Verify after mutation**: Confirm the modification succeeded with a fresh read and run `yarn mm describe-screen`. +5. **Restore before cleanup**: Revert all changes to their original backed-up values before ending the session. +6. **Never mutate unknown wallet state**: Avoid touching keyring or account states unless your task explicitly commands it. + +## React Native Fiber Entry Point + +React Native lacks a DOM interface. To access the internal React component tree, query the DevTools global hook directly: ```javascript var hook = globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__; var rid = hook.renderers.keys().next().value; var root = hook.getFiberRoots(rid).values().next().value; -var fiber = root.current; // root fiber, start walking from here +var fiber = root.current; // Start walking the fiber tree from here ``` -From `fiber`, traverse the tree with `.child`, `.sibling`, and `.return` exactly like the extension pattern. The fiber shape (`memoizedState`, `memoizedProps`, `stateNode`) is identical. - -## Operations - -| Operation | Method | Scope | -|---|---|---| -| Read Redux state | Fiber walk, store, `getState()` | In-memory UI state | -| Write Redux state | Fiber walk, store, `dispatch()` | Instant UI update, lost on restart | -| Call controller methods | Fiber walk, Engine singleton, `context.SomeController.method()` | Triggers real controller logic and state propagation | - -**Preferred order:** Call controller methods (operation 3) first. This is the most correct approach because the controller manages its own state and propagates to Redux. Fall back to Redux dispatch (operation 2) only when you need to fake state that no controller API provides. +Every React fiber node exposes `child`, `sibling`, and `return` properties for tree traversal. The properties `memoizedState` and `memoizedProps` hold the actual values and context bindings. -### 1. Read Redux State +## Read-Only Redux Store Evaluation -Find the Redux store on the `` component's fiber props: +This concise script walks the fiber tree to find the Redux provider and safely reads user configuration. It bypasses missing or protected properties without crashing. ```bash -yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;function find(f){if(!f)return null;if(f.memoizedProps&&f.memoizedProps.store&&typeof f.memoizedProps.store.getState===\"function\")return f.memoizedProps.store;return find(f.child)||find(f.sibling)}var store=find(root.current);if(!store)return JSON.stringify(\"store not found\");var s=store.getState();return JSON.stringify({userRegion:s.engine.backgroundState.RampsController.userRegion,moneyEnabled:!!s.engine.backgroundState.RemoteFeatureFlagController})})()","returnByValue":true}' +yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;if(!hook)return JSON.stringify(\"hook missing\");var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;function find(f){if(!f)return null;if(f.memoizedProps&&f.memoizedProps.store&&typeof f.memoizedProps.store.getState===\"function\")return f.memoizedProps.store;return find(f.child)||find(f.sibling)}var store=find(root.current);if(!store)return JSON.stringify(\"store not found\");var s=store.getState();return JSON.stringify({locale:s.settings&&s.settings.locale,currency:s.settings&&s.settings.currentCurrency})})()","returnByValue":true}' ``` -**Readable version** of the expression: - -```javascript -(function() { - var hook = globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__; - var rid = hook.renderers.keys().next().value; - var root = hook.getFiberRoots(rid).values().next().value; - - function find(f) { - if (!f) return null; - if (f.memoizedProps && f.memoizedProps.store - && typeof f.memoizedProps.store.getState === "function") - return f.memoizedProps.store; - return find(f.child) || find(f.sibling); - } - - var store = find(root.current); - if (!store) return JSON.stringify("store not found"); - var s = store.getState(); - return JSON.stringify({ - userRegion: s.engine.backgroundState.RampsController.userRegion, - }); -})() -``` +## Safe Mutation Checklist and Templates -### 2. Write Redux State +When modifying runtime components, you must capture, modify, and restore the state systematically. -Dispatch an action to update the backgroundState slice. The UI re-renders immediately. +### Mutation Checklist +1. Query and store the original state in a temporary task-specific backup key. +2. Call the required controller method or dispatch a scoped action. +3. Query the state again to confirm the new value is active. +4. Call `yarn mm describe-screen` to refresh visual references. +5. Restore the original state using the task-specific backup key before cleanup. +### State Backup and Mutation Template (Pseudocode) ```javascript +// Pseudocode: Identify context, back up original state under a task-specific key, and mutate (function() { - // ... same fiber walk to find store ... - var s = store.getState(); - var bg = s.engine.backgroundState; - - // Patch the target controller state - var ramps = Object.assign({}, bg.RampsController, { - userRegion: { - regionCode: "BR", - country: { isoCode: "BR", name: "Brazil", supported: { buy: false } }, - state: null - } - }); - - // Dispatch backgroundState update - store.dispatch({ - type: "UPDATE_BG_STATE", - key: "RampsController", - payload: ramps - }); - return "ok"; -})() -``` - -> **Note:** The exact action type for backgroundState updates may differ. If `UPDATE_BG_STATE` doesn't work, inspect the Redux reducer to find the correct action type. Redux dispatch only updates what the UI reads via selectors, but it does NOT modify the live controller instance. - -### 3. Find Engine Singleton and Call Controller Methods (Preferred) + var engine = findEngineContext(); // Custom tree search + if (!engine) return "Engine context not found"; -The Engine singleton holds every controller at `Engine.context`. Components that use controllers (e.g., `useRampsProviders`, `useMoneyAccountDeposit`) import Engine as a module dependency. Walk the fiber tree looking for an object with the controller context shape: an object that has `RampsController`, `TransactionController`, and `NetworkController` as properties. + // 1. Capture and back up the original value under a task-specific key + var original = engine..state.; + globalThis.__mmStateBackup = globalThis.__mmStateBackup || {}; + globalThis.__mmStateBackup[""] = original; -**Strategy A: Shape-match Engine.context on fiber props/state:** - -```javascript -(function() { - var hook = globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__; - var rid = hook.renderers.keys().next().value; - var root = hook.getFiberRoots(rid).values().next().value; - var visited = 0; - - function isEngineContext(obj) { - return obj - && typeof obj.RampsController !== "undefined" - && typeof obj.TransactionController !== "undefined" - && typeof obj.NetworkController !== "undefined"; - } - - function searchObj(obj, depth) { - if (!obj || depth > 3 || typeof obj !== "object") return null; - if (isEngineContext(obj)) return obj; - for (var k in obj) { - try { - var v = obj[k]; - if (v && typeof v === "object") { - var found = searchObj(v, depth + 1); - if (found) return found; - } - } catch(e) {} - } - return null; - } - - function walk(f) { - if (!f || visited > 2000) return null; - visited++; - - // Check memoizedProps - var ctx = searchObj(f.memoizedProps, 0); - if (ctx) return ctx; - - // Check stateNode - if (f.stateNode && typeof f.stateNode === "object") { - ctx = searchObj(f.stateNode, 0); - if (ctx) return ctx; - } - - // Check hook state chain (memoizedState linked list) - var hookState = f.memoizedState; - while (hookState) { - if (hookState.memoizedState && typeof hookState.memoizedState === "object") { - ctx = searchObj(hookState.memoizedState, 0); - if (ctx) return ctx; - } - // useRef stores value in .current - if (hookState.memoizedState && hookState.memoizedState.current) { - ctx = searchObj(hookState.memoizedState.current, 0); - if (ctx) return ctx; - } - hookState = hookState.next; - } - - return walk(f.child) || walk(f.sibling); - } - - var ctx = walk(root.current); - if (!ctx) return JSON.stringify("Engine.context not found after " + visited + " fibers"); - - // Now call the controller method - ctx.RampsController.setUserRegion("BR"); - return JSON.stringify("setUserRegion called, visited " + visited + " fibers"); + // 2. Perform the mutation via controller API + engine..(); + return "Backup created and mutation applied"; })() ``` -**Strategy B: Metro module registry (fallback):** - -In dev builds with Metro, `globalThis.__r` is Metro's module require function. Scan for the Engine module by checking exports: - +### State Restoration Template (Pseudocode) ```javascript +// Pseudocode: Restore the original value using the task-specific backup key (function() { - if (typeof globalThis.__r !== "function") return JSON.stringify("__r not available"); - - for (var id = 0; id < 80000; id++) { - try { - var m = globalThis.__r(id); - if (m && m.default && m.default.context - && m.default.context.RampsController - && m.default.context.NetworkController) { - // Found Engine default export - m.default.context.RampsController.setUserRegion("BR"); - return JSON.stringify("Engine found at module " + id); - } - } catch(e) {} + if (!globalThis.__mmStateBackup || typeof globalThis.__mmStateBackup[""] === "undefined") { + return "Restore skipped: no backup found"; } - return JSON.stringify("Engine module not found"); + + var engine = findEngineContext(); + engine..(globalThis.__mmStateBackup[""]); + + // Clean up global references: delete the task key first, then delete the namespace if empty + delete globalThis.__mmStateBackup[""]; + if (Object.keys(globalThis.__mmStateBackup).length === 0) { + delete globalThis.__mmStateBackup; + } + return "State restored successfully"; })() ``` -> **Note:** Strategy B brute-forces module IDs. It works but is slow (~5-15s). Cache the module ID within a session once found. Strategy A (fiber walk) is faster and preferred. - -### Example: Simulate Unsupported Region for Fiat Deposits +## Last-Resort Module Discovery -```bash -# 1. Set region to Brazil (unsupported for buy) -yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;var visited=0;function isCtx(o){return o&&typeof o.RampsController!==\"undefined\"&&typeof o.NetworkController!==\"undefined\"}function search(o,d){if(!o||d>3||typeof o!==\"object\")return null;if(isCtx(o))return o;for(var k in o){try{var v=o[k];if(v&&typeof v===\"object\"){var f=search(v,d+1);if(f)return f}}catch(e){}}return null}function walk(f){if(!f||visited>2000)return null;visited++;var c=search(f.memoizedProps,0)||search(f.stateNode,0);if(c)return c;var h=f.memoizedState;while(h){if(h.memoizedState&&typeof h.memoizedState===\"object\"){c=search(h.memoizedState,0);if(c)return c}h=h.next}return walk(f.child)||walk(f.sibling)}var ctx=walk(root.current);if(!ctx)return JSON.stringify(\"not found\");ctx.RampsController.setUserRegion(\"BR\");return JSON.stringify(\"region set to BR\")})()","returnByValue":true}' +Searching the internal module registry is an expensive fallback option. Do not use this as a primary approach. -# 2. Wait for provider re-resolution -sleep 3 +- **Brute-Force modules**: Scanning `globalThis.__r` is slow and can trigger unexpected module evaluation side effects. +- **Side effects**: Importing arbitrary modules in an active session can initialize services or create conflicting listeners. +- **Guideline**: Only use module scanning if fiber tree traversal fails completely to locate the desired controller context. -# 3. Navigate to Money, then Add Money sheet -yarn mm describe-screen -yarn mm click --testid money-action-button-row-add -yarn mm wait-for --testid money-add-money-sheet --timeout 10000 -yarn mm describe-screen -yarn mm screenshot --name "unsupported-region-no-deposit-funds" +## When to Use and Troubleshooting -# 4. Verify: "Deposit Funds" option should be missing or disabled - -# 5. Restore to US -yarn mm cdp Runtime.evaluate '{"expression":"(function(){/* same walk */ctx.RampsController.setUserRegion(\"US\");return JSON.stringify(\"region restored\")})()","returnByValue":true}' -``` - -## Verify State After Mutation - -```bash -# Read current region from Redux -yarn mm cdp Runtime.evaluate '{"expression":"(function(){var hook=globalThis.__REACT_DEVTOOLS_GLOBAL_HOOK__;var rid=hook.renderers.keys().next().value;var root=hook.getFiberRoots(rid).values().next().value;function find(f){if(!f)return null;if(f.memoizedProps&&f.memoizedProps.store&&typeof f.memoizedProps.store.getState===\"function\")return f.memoizedProps.store;return find(f.child)||find(f.sibling)}var store=find(root.current);if(!store)return JSON.stringify(\"no store\");var r=store.getState().engine.backgroundState.RampsController;return JSON.stringify({regionCode:r.userRegion&&r.userRegion.regionCode,country:r.userRegion&&r.userRegion.country})})()","returnByValue":true}' -``` +Use these techniques only when standard UI automation cannot reach the desired test scenario. -## When to Use CDP +### Recommended Operations +- Reading complex UI configuration flags. +- Simulating system settings that lack physical device controls. +- Triggering background updates during visual validation. -| Need | Approach | -|---|---| -| Read any Redux state value | Fiber walk, store, `getState()` | -| Change what the UI displays (fast, non-persistent) | Fiber walk, store, `dispatch()` | -| Trigger real controller logic (region change, provider refresh) | Fiber walk, Engine.context, controller method | -| Verify a JS global or Hermes flag | `Runtime.evaluate` with simple expression | -| Execute JS against the React Native runtime | `Runtime.evaluate` | +### Troubleshooting | Symptom | Cause | Solution | |---|---|---| -| Redux dispatch updated state but UI didn't change | Component reads from controller state, not Redux selector | Use Engine.context controller method instead | -| Fiber walk returns "not found" | DevTools hook not available (release build) or tree too deep | Try Metro `__r` fallback (Strategy B) | -| `setUserRegion` called but UI unchanged | Provider re-fetch is async; UI hasn't re-rendered yet | Wait 3-5 seconds, then `describe-screen` | +| UI is unresponsive after dispatching a Redux action | Direct Redux dispatches do not update the underlying controller state | Use the controller methods directly instead of direct Redux dispatch | +| The state walk fails with "hook missing" | The app is running in a production configuration where DevTools hooks are omitted | Use standard UI flows or check the development build configuration | +| The modified value reverted on its own | A background controller updated the state and overwrote your manual change | Check for ongoing sync events or pause the background controller first | diff --git a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md index 409833b8..f9452f2d 100644 --- a/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md +++ b/domains/testing/skills/mobile-visual-testing/repos/metamask-mobile.md @@ -6,91 +6,105 @@ metadata: type: mobile-testing --- -# MetaMask Mobile Visual Testing — iOS +# MetaMask Mobile Visual Testing — iOS and Android -Use this skill to visually validate MetaMask Mobile through the project-local `mm` CLI. +Use this skill to visually inspect and interact with an already-installed MetaMask Mobile app through the project-local `mm` CLI. This entrypoint is a self-sufficient runbook for the normal workflow; load a reference for exhaustive syntax, error inventories, or advanced runtime work. ## Architecture -The `mm` CLI and a persistent local HTTP daemon come from `@metamask/client-mcp-core`. The device backend is `@metamask/device-mcp`, which drives the iOS Simulator through **`idb`** (`idb-companion` + `fb-idb`). Accessibility trees, taps, typing, and screenshots all flow through idb — there is no XCUITest runner in this workflow. +The `mm` CLI and a persistent local HTTP daemon come from `@metamask/client-mcp-core`. The device backend is `@metamask/device-mcp`, which provides native device interaction: -For full architecture, component locations, and safety details, see the on-demand references below and the in-repo doc `tests/llm-workflow/README.md`. +- **iOS**: drives the iOS Simulator through `idb` (`idb-companion` + `fb-idb`) — no XCUITest runner. +- **Android**: drives an already-running emulator through ADB and UIAutomator — no Appium. -## Scope +Accessibility discovery, taps, typing, and screenshots all flow through native debug channels. For implementation architecture, daemon/session internals, and current validation evidence, see the in-repo doc `tests/llm-workflow/README.md`. -- iOS Simulator only; Android is unsupported. -- The workflow is prod-only and preserves installed app/wallet state by default. -- Launch reuses an installed MetaMask app on the target simulator. -- The workflow does not build the app, discover local build outputs, or initialize test state. +## Scope and State Safety + +The CLI context is always `prod`. `--context e2e`, fixtures, seeding, and environment switching are rejected. "Prod context" does not require the installed binary to be a release build — Metro/Hermes inspection needs a compatible development build. + +- Simulator/emulator only; physical devices are unsupported. +- Launch does not build MetaMask, discover build outputs, initialize wallet state, or guarantee a particular screen. +- Treat the installed wallet as unknown persistent state. Never assume passwords, accounts, networks, balances, or credentials — obtain them from the user or an approved environment. + +Platform deltas: + +- **iOS**: reuses the installed app by default, but may accept an explicit `.app` bundle via `--app-bundle`. `--reinstall`, `--reset-app-data`, and `--allow-fox-code-mismatch` are destructive and guarded; `--reinstall` and `--reset-app-data` require `--app-bundle`. +- **Android**: always reuses the installed `io.metamask`. APK install/reinstall/reset options, `--app-bundle`, `--allow-fox-code-mismatch`, and `--extension-path` are rejected (there is no APK lifecycle). During a session, the emulator's animation scales are set to zero and restored on cleanup. Cleanup force-stops `io.metamask` but never stops, wipes, or deletes the emulator. ## Prerequisites -```bash -# Verify the iOS toolchain (Xcode, idb, idb_companion, booted simulator) -yarn mm:doctor +### iOS -# Install idb if MM_DEPENDENCIES_MISSING is reported +```bash brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb +yarn mm:doctor +``` -# Build/install MetaMask separately if it is not already installed -yarn setup && yarn start:ios +`yarn mm:doctor` is iOS-only. It prints a PASS/FAIL report for Xcode tools, `idb`, `idb_companion`, and a booted simulator, with install commands for anything missing, and exits non-zero when a prerequisite is absent. Run it before launching. MetaMask must already be installed on the target simulator (or supply `--app-bundle`). -# Boot a simulator when needed +```bash +xcrun simctl list devices xcrun simctl boot ``` -Run `yarn mm:doctor` before launching; it prints a PASS/FAIL report with install commands for anything missing and exits non-zero when a prerequisite is absent. If an app is not installed, install it separately on the simulator before launching. +### Android + +```bash +adb version +adb devices -l +``` + +- `adb` (Android SDK Platform-Tools) must be on `PATH`. +- The target must be an online, authorized, fully booted `emulator-*`. Without `--device-id`, exactly one eligible emulator must exist. +- Exactly `io.metamask` must be installed for the emulator's current user, with `io.metamask/io.metamask.MainActivity` launchable. +- No `.device-session` override may exist in the current directory or worktree root (it selects Appium and is incompatible with the ADB backend). +- Build/install the app externally if needed; do not run `yarn start:android` as part of an active `mm` testing workflow. ## Required Workflow -### 1. Launch +### 1. Launch and select the platform ```bash -# Reuse the installed app and its current state +# iOS is the default yarn mm launch +yarn mm launch --platform ios --device-id -# Pin a simulator if needed -yarn mm launch --device-id +# Android must be explicit +yarn mm launch --platform android +yarn mm launch --platform android --device-id emulator-5554 -# Install a specific build before launching +# iOS: install a specific build before launching yarn mm launch --app-bundle ios/build/MetaMask.app - -# Force-replace an existing active session (runs cleanup first) -yarn mm launch --force ``` -There is only one supported environment: prod. Do not request or switch launch contexts. Supplying `--context e2e` is rejected. - -When a session is already active, `mm launch` rejects with `MM_SESSION_ALREADY_RUNNING` unless `--force` is passed (which cleans up the existing session then launches a new one). - -`--reinstall`, `--reset-app-data`, and `--allow-fox-code-mismatch` are destructive to the installed wallet state and are guarded (a destructive flag requires `--app-bundle`). See [references/cli-reference.md](references/cli-reference.md#destructive-launch-flags). - -When attaching to Metro (`--metro-port`), the workflow is **attach-only** — it never spawns Metro. If the app is already running and healthily attached to Metro (Hermes target found at `/json`), `mm launch` connects without relaunching. If the app is not healthily attached, it terminates and re-launches via the deep link. On a fresh-booted simulator, a one-time relaunch ensures the accessibility tree is valid. Release/prod builds have no Hermes inspector; Metro attach requires a dev build. +- Omitted `--platform`, `--platform browser` (compatibility alias), and `--platform ios` all route to iOS. Only `--platform android` routes to Android. +- Use `--device-id`, never the stale `--device`. +- An active session produces `MM_SESSION_ALREADY_RUNNING`. Prefer `yarn mm cleanup` before relaunching; use `--force` only when replacing the session is intentional (it cleans up first, then launches). +- Metro attachment is **attach-only** — `mm` never spawns Metro. See [Metro and Runtime Inspection](#metro-and-runtime-inspection). -### 2. Reuse Knowledge +### 2. Reuse prior knowledge ```bash yarn mm knowledge-search "" yarn mm knowledge-sessions ``` -Reuse a known successful sequence when available. Otherwise discover the flow and let the session record it. +Reuse a known-good sequence when one exists; otherwise discover the flow and let the session record it. -### 3. Observe Before Acting +### 3. Observe before acting ```bash yarn mm describe-screen ``` -Use fresh output after navigation. Accessibility refs (`e1`, `e2`, ...) are ephemeral. +Accessibility references (`e1`, `e2`, ...) are ephemeral. Refresh them after navigation, reloads, overlays, or major UI changes. ### 4. Interact -Only two targeting methods work on iOS: **test IDs** and **accessibility refs**. Use one per command; prefer a stable test ID, fall back to a fresh a11y ref from `describe-screen`. - ```bash yarn mm click --testid unlock-submit -yarn mm type --testid unlock-password "" +yarn mm type --testid unlock-password "" yarn mm wait-for --testid account-overview --timeout 15000 yarn mm get-text --testid balance-display @@ -98,108 +112,104 @@ yarn mm click e5 yarn mm type e2 "text" ``` -`--selector` (CSS) and `--within` (scoped search) are **rejected by the iOS driver** even though the shared CLI accepts the flags — see Gotchas. To disambiguate duplicate targets, use a unique test ID or the exact element's fresh a11y ref. - -The CLI flag is `--testid` (all lowercase); `--testId` is silently mis-parsed as a positional target and hits the wrong element. `mm type` clears the field before typing (idb does `cmd+a` → delete → type). - -### 5. Verify and Capture Evidence +- `--testid` is **lowercase**. `--testId` is silently mis-parsed as a positional target and hits the wrong element. +- Positional a11y refs and `--testid` are the only targeting methods. `--selector` (CSS) and `--within` (scoped search) are parsed by the shared CLI but rejected by both mobile drivers. +- Matching is fuzzy and case-insensitive (iOS matches accessibility label/identifier; Android matches resource ID, content-description, or text — all by substring). Prefer exact, unique test IDs to disambiguate; there is no `--within` fallback on mobile. +- `mm type` clears the field first (iOS uses `cmd+a` → delete → type). To submit, tap the on-screen keyboard action or the form's submit control — there is no trailing-newline trick. -After a mutating sequence: - -1. Run `yarn mm describe-screen`. -2. Confirm the expected state. -3. Capture meaningful before/after evidence: +### 5. Verify and capture evidence ```bash +yarn mm describe-screen yarn mm screenshot --name "after-action" ``` -If the state is wrong, capture a debug screenshot, search knowledge, and retry from fresh refs. +Verify the resulting state after any consequential action rather than trusting a successful click response. If the state is wrong, capture a debug screenshot, search knowledge, and retry from fresh refs. -### 6. Cleanup +### 6. Restore and clean up ```bash yarn mm cleanup -yarn mm cleanup --shutdown +yarn mm cleanup --shutdown # iOS simulator shutdown option only ``` -Always clean up when testing is complete. +- Restore any temporary runtime/controller changes and uninstall monitoring hooks before cleanup. +- `--shutdown` is an iOS Simulator option. Use plain `yarn mm cleanup` for Android; it does not stop or wipe the emulator. ## Metro and Runtime Inspection -For JS development, attach the installed development app to Metro. The workflow is attach-only — start Metro separately: +Metro attachment is available for development builds. The workflow is **attach-only** — start Metro separately, then launch with the port: ```bash yarn watch:clean -yarn mm launch --metro-port 8081 - -# Equivalent, still supported (the flag wins when both are set) -MM_METRO_PORT=8081 yarn mm launch -``` -If Metro is not reachable on the given port, launch fails with `MM_INVALID_CONFIG`. If the app is already running and healthily attached to Metro, `mm launch` connects without relaunching (pure-attach). Release/prod builds have no Hermes inspector; Metro attach requires a dev build. +yarn mm launch --platform ios --metro-port 8081 +yarn mm launch --platform android --metro-port 8081 -Node 20 may require `NODE_OPTIONS="--experimental-websocket"` for `mm cdp`; Node 22+ includes WebSocket support. +# Environment-variable form (the --metro-port flag wins when both are set) +MM_METRO_PORT=8081 yarn mm launch --platform android +``` -`mm cdp` evaluates JavaScript in the Hermes runtime through Metro's inspector proxy: +- `mm` attaches to Metro; it never starts it. If Metro is unreachable on the given port, launch fails with `MM_INVALID_CONFIG`. +- Hermes/CDP requires a compatible development build; release/prod builds expose no Hermes inspector target. +- Node 20 requires `NODE_OPTIONS="--experimental-websocket"` for `mm cdp`; Node 22+ supports WebSockets natively. +- Use one Metro process and one selected simulator/emulator per worktree to avoid target ambiguity. +- Android reverse port mappings are session-owned and conflict-checked: the workflow verifies Metro `/status` (`packager-status:running`), reuses an identical mapping, fails on a conflicting one, and removes only its own mapping on cleanup. Never manually overwrite mappings or use `reverse --remove-all`. ```bash yarn mm cdp Runtime.evaluate '{"expression":"JSON.stringify(globalThis.__DEV__)"}' ``` -Prefer controller methods over raw Redux mutation when inspecting or changing runtime state. Runtime modifications affect the current installed app state; restore any state changed during testing. +Prefer real controller methods over raw Redux mutation. Capture the original state, verify the mutation, and restore it before cleanup. Load `references/state-manipulation.md` for full CDP procedures. ## Batching -Use `run-steps` only for known deterministic sequences: +Use `run-steps` only for deterministic sequences after targets and transitions are understood. The argument must be a JSON object containing a `steps` array, not a bare array: ```bash yarn mm run-steps '{"steps":[ - {"tool":"type","args":{"testId":"login-password-input","text":""}}, - {"tool":"click","args":{"testId":"log-in-button"}}, - {"tool":"wait_for","args":{"testId":"wallet-screen","timeoutMs":15000}} -]}' + {"tool":"click","args":{"a11yRef":"e3"}}, + {"tool":"wait_for","args":{"testId":"home-screen","timeoutMs":10000}} +],"stopOnError":true}' ``` -The input must be an object containing `steps`, not a bare array. +Use individual commands while exploring or debugging. + +## Critical Mobile Limitations and Gotchas -## Mobile Limitations and Gotchas +- **Targeting**: fresh a11y refs or lowercase `--testid` only. Matching is fuzzy/case-insensitive; no CSS `--selector` or scoped `--within` on mobile. +- **Behavior**: `mm type` clears first; mutating commands may return compact observations — run `describe-screen` whenever a full fresh tree is needed. +- **Unavailable**: no `mm build`; no browser URL navigation, tab switching, notification pages, or clipboard APIs; `navigate-home`/`navigate-settings` are not implemented (navigate through visible UI); no E2E state initialization, fixtures, or contract seeding. +- **state**: never assume or persist balances, networks, or onboarding state. +- **credentials**: Try the e2e tests default password, and only ask the user if that one does not work: "correct horse battery staple" -- **Only `--testid` and a11y refs target elements.** `--selector` throws (`CSS selectors are not supported on mobile`) and `--within` throws (`Scoped element search (within) is not supported on mobile`) at the driver, even though the shared CLI parses both flags. Do not use them on iOS. -- **`--testid` is case-sensitive and lowercase.** `--testId` is not recognized as a flag; the value is treated as a positional target and silently hits the wrong element (usually timing out). -- **Element matching is fuzzy and case-insensitive.** idb matches on accessibility label/identifier by substring, so `--testid Confirm` can match `Confirm Transaction`. Prefer exact, unique test IDs to avoid hitting the wrong element. -- **`mm type` clears first.** idb runs `cmd+a` → delete → type, so there is no need to clear the field manually. There is no trailing-newline submit trick; to submit, tap the on-screen keyboard action button (a fresh a11y ref) or the form's submit control. -- No `mm build`; build/install separately. -- No URL navigation, tab switching, browser notification pages, or browser clipboard APIs. -- `navigate-home` and `navigate-settings` are not implemented; navigate through visible UI elements. -- `mm cdp` requires Metro. -- One Metro process per worktree is recommended. -- Mutating commands can return compact observations; request a full `describe-screen` whenever refs or state are uncertain. -- Never assume wallet credentials, balances, networks, or onboarding state. Inspect the installed app and obtain needed credentials from the user/environment. +See `references/cli-reference.md` for the full unsupported-command matrix. ## Error Recovery -- `MM_DEPENDENCIES_MISSING`: Xcode command-line tools or `idb` are missing. Run `yarn mm:doctor`, then `brew tap facebook/fb && brew install idb-companion && pip3 install fb-idb`. -- `MM_WAIT_TIMEOUT`: target did not become visible; describe the screen and verify scope/test ID. -- `MM_CLICK_TIMEOUT`: click may have completed; describe before retrying. -- `MM_TYPE_TIMEOUT`: field interaction stalled; inspect focus and use a fresh target. -- `MM_DEVICE_NOT_AVAILABLE`: no simulator is booted, the UDID does not exist, or `simctl` failed. Run `xcrun simctl list devices` and boot one; verify MetaMask is installed. -- `MM_INVALID_CONFIG`: the launch options are unusable — no app and no `--app-bundle`, a destructive flag without `--app-bundle`, a `fox_code` mismatch, or an unreachable Metro port. Read the remediation text; reuse the installed app or install a matching build. +First-response routing — load `references/error-recovery.md` for the complete inventory and detailed remediation. -Launch errors use core `ErrorCode`s (not `MM_IOS_*`): `@metamask/client-mcp-core` collapses unknown consumer codes into `MM_LAUNCH_FAILED`, so iOS detail is carried in the message and remediation. +| Failure class | First response | +| --- | --- | +| Stale target or interaction timeout (`MM_TARGET_NOT_FOUND`, `MM_WAIT_TIMEOUT`, `MM_CLICK_TIMEOUT`, `MM_TYPE_TIMEOUT`) | Run `describe-screen`, inspect blockers, retry with a fresh unique target; increase `--timeout` only if still transitioning | +| Existing/stale session (`MM_SESSION_ALREADY_RUNNING`, `MM_PORT_IN_USE`) | `yarn mm cleanup`; if daemon state is stale, `yarn mm stop --force` | +| iOS dependency/device/config (`MM_DEPENDENCIES_MISSING`, `MM_DEVICE_NOT_AVAILABLE`, `MM_INVALID_CONFIG`) | Run `yarn mm:doctor`; read the remediation text | +| Android readiness (`MM_ANDROID_DEPENDENCY_MISSING`, `MM_ANDROID_RUNNER_NOT_READY`, `MM_ANDROID_BACKEND_INTEGRITY`) | Run `adb devices -l`; verify package/activity; remove `.device-session` | +| Metro/Hermes failure (`MM_HERMES_NOT_AVAILABLE`, `MM_CDP_FAILED`) | Verify Metro `/status`, port, development build, target uniqueness, and `.mm-daemon.log` | +| Unknown launch failure (`MM_LAUNCH_FAILED`) | Capture the exact code/remediation, inspect `.mm-daemon.log`, then load `error-recovery.md` | -For the full error-code table and troubleshooting, see [references/error-recovery.md](references/error-recovery.md). +Launch errors use core `ErrorCode`s (not `MM_IOS_*`): `@metamask/client-mcp-core` collapses unknown consumer codes into `MM_LAUNCH_FAILED`, so iOS-specific detail is carried in the message and remediation. ## Reference Guides -Load these on demand — not required for standard visual testing: +Load on demand — not required for standard visual testing: -- **[references/cli-reference.md](references/cli-reference.md)** — full command tables, syntax rules, targeting details, and commands not available on mobile. -- **[references/error-recovery.md](references/error-recovery.md)** — error codes, common failures, and troubleshooting. -- **[references/state-manipulation.md](references/state-manipulation.md)** — read/write runtime state and call controller methods via `mm cdp` (Hermes runtime). -- **[references/runtime-monitoring.md](references/runtime-monitoring.md)** — capture network requests and console logs via Hermes runtime interceptors. Load when testing flows that involve API calls or debugging silent failures. +- **[references/cli-reference.md](references/cli-reference.md)** — full command tables, syntax rules, targeting details, batching schema, platform selection, and commands not available on mobile. +- **[references/error-recovery.md](references/error-recovery.md)** — complete error-code inventory and troubleshooting. Load after setup, launch, interaction, or daemon failures. +- **[references/state-manipulation.md](references/state-manipulation.md)** — advanced Metro-attached runtime state inspection and mutation via `mm cdp` (Hermes runtime). +- **[references/runtime-monitoring.md](references/runtime-monitoring.md)** — network/console capture and anomaly detection. Review and redact captured data before sharing or persisting it. ## References and Attribution -- **In-repo workflow doc:** `tests/llm-workflow/README.md` — daemon/session architecture, installed-app safety, and the canonical prerequisites (`idb`, `yarn mm:doctor`). -- **Upstream packages:** `@metamask/client-mcp-core` (CLI + daemon) and `@metamask/device-mcp` (idb-based iOS device backend). +- **In-repo workflow doc**: `tests/llm-workflow/README.md` — daemon/session architecture, installed-app safety, canonical prerequisites, and current validation evidence. +- **Upstream packages**: `@metamask/client-mcp-core` (CLI + daemon) and `@metamask/device-mcp` (idb-based iOS and ADB/UIAutomator Android device backends). diff --git a/domains/testing/skills/mobile-visual-testing/skill.md b/domains/testing/skills/mobile-visual-testing/skill.md index 6d7c523b..26120393 100644 --- a/domains/testing/skills/mobile-visual-testing/skill.md +++ b/domains/testing/skills/mobile-visual-testing/skill.md @@ -1,4 +1,4 @@ --- name: mobile-visual-testing -description: Drives the MetaMask Mobile via the mm CLI for visual testing in IOs simulator or Android emulator. Use when asked to visually verify UI changes, capture screenshots or debug Mobile UI state. Trigger phrases include "verify visually", "take a screenshot", "test the flow", and "check the UI". +description: Drives MetaMask Mobile on an iOS Simulator or Android emulator via the mm CLI for visual testing. Use when asked to visually verify mobile UI changes, capture screenshots, test wallet flows, or debug app UI state. Trigger phrases include "verify visually", "take a screenshot", "test the flow", "check the UI", and "run on mobile". ---