E2E Tests

Choose Secrets → E2E test variables, select an environment, and use Add variable to save test-only values under Global or your profile’s environment. Use the exact variable names from your test files; no Maestro-specific prefix is required. Newly saved variables are used automatically, with no portal profile mapping or enablement form. Managed iOS runners need MACH CLI 2.13.0 or later for private test output handling.

Precedence is explicit --maestro-env values, then the selected profile’s environment secrets, then Global E2E secrets. The CLI resolves inheritance and platform-specific environment overrides from mach.config.json; when environment is omitted, it uses the same profile-name scope fallback as builds. Dashboard runs use the environment recorded with the build. Older builds can use their existing legacy mapping or be tested from the linked CLI project. Scoped secrets are never resolved by guessing when context is missing. Requested variable names are checked before runner launch or billing. Automatic attempts keep one encrypted snapshot; a newly queued rerun resolves current values.

Test values are encrypted and isolated from application build variables, mach env pull and environment exports. Developers see resolved names and their environment; only administrators can change values. There is no reveal or export action. Change history records actors, names and times without values. Existing legacy configurations and explicit pauses remain unchanged until an administrator saves variables with the new workflow or chooses Use automatically. Queued and running tests are unchanged.

Updated runners scrub known values and common encodings from streamed logs, errors and textual reports. An incompatible runner cannot retrieve central values. Screenshots and recordings cannot be reliably redacted; authenticated-run evidence requires developer or higher project access and uses expiring file links. Use sample test accounts and avoid logging secrets or placing them in screenshots. Local --local runs continue to use local variables and do not download stored test credentials. No new scheduler or publishing permission is created by this feature.

E2E Tests brings together mobile and web test runs. Managed Maestro executions validate a mobile build on a Mach runner. Local Playwright executions validate a web app in the browser on your machine or existing CI runner, then save a tenant-scoped outcome summary in the same list.

Both are useful for smoke tests, checkout flows, login flows, onboarding, and other critical UI journeys. They do not replace unit tests, API tests, backend checks, or pixel-perfect visual regression testing.

Open an active test from E2E Tests and choose Cancel test. Android Maestro runners are terminated, and iOS runners are cancelled through their managed GitLab pipeline. A local Playwright command checks Mach every two seconds and stops its own browser process tree when it sees the cancellation request. Queued runs have no charge; a managed run that already started is billed only through its stop time.

Add Maestro flows under `.maestro`, then ask Mach to queue tests after a successful build. Android emulator runs need an APK that includes emulator support.

If your simulator app stores authentication tokens or encryption keys in Keychain, see Configuration → Simulator Keychain and encrypted sessions for the optional ios.simulatorSigning profile setting. Rebuild before testing. After login, stop and relaunch without clearing app data and assert the authenticated screen remains visible before any flow logs in again. This verifies session restoration instead of hiding a failure with a second login.

Use `--wait` when a pipeline should fail immediately if the UI smoke test fails. Without `--wait`, Mach queues the run and the dashboard continues updating as the runner reports progress.

`mach web test` automatically provisions the project's local Playwright dependency and Chromium when needed. It creates a test run in E2E Tests, but the browser, code, test output, screenshots, videos, traces, reports, and environment values never leave the local runner. Use `--no-setup` on a CI image that already has Playwright and its browser installed.