Mobile testing

Mobile tests use the same plain-English steps as web tests. agent-qa drives the app through Appium, with UiAutomator2 on Android and XCUITest on iOS, on a device on this machine or on a remote one.

Set up agent-qa to test this project's Android app on a local emulator or device, and run one test. Read https://vostride.com/docs/agent-qa/guides/mobile-testing.md first.

1. Check for the Android SDK (ANDROID_HOME or ANDROID_SDK_ROOT) and an emulator or a connected device. Tell me if either is missing; do not install Android Studio yourself.
2. Install Appium in the project, then the Android driver with npx agent-qa install-mobile-drivers --android.
3. Run npx agent-qa devices to see the phones, emulators and AVDs this machine has. Add a device under registry.devices in agent-qa.config.yaml with platform: android and a model it lists: the model a phone reports, or an emulator's AVD name or device profile. Add os, quoted, only to pin a version. Never create agent-qa.local.yaml; agent-qa no longer reads it.
4. Add an Android target with the app's appPackage and appActivity, app.path when a debug build is available, and device set to that device's name.
5. Run npx agent-qa doctor and fix the mobile checks it reports.
6. Write a short test for the app's first screen, and run it. It runs on the target's device.

Report the files you changed, the doctor results, and the run's verdict.

What you needCopy Anchor Link

For devices on this machine:

• Appium. agent-qa does not bundle it. Install it in the project, or globally:

  npm install -D appium

• Drivers. Install them through agent-qa:

  npx agent-qa install-mobile-drivers --android

  Use --ios or --all for iOS. --update updates installed drivers.

• Android: the Android SDK, with ANDROID_HOME or ANDROID_SDK_ROOT set, and an emulator or a connected phone.

• iOS: a Mac with Xcode and a simulator.

A device on a remote Appium server or on BrowserStack needs none of these here. npx agent-qa doctor checks what your devices need, and skips the local tools when every device is remote. See doctor's rows.

Set up the projectCopy Anchor Link

npx agent-qa@latest init --platform android

Use ios, or a list such as web,android or web,ios, as needed. A mobile project starts with an example-android or example-ios target, a commented device example and no example tests, because there is no shared app to test.

Describe the appCopy Anchor Link

The target holds the app's IDs and builds, and names the device its tests use:

registry:
  targets:
    shop-android:
      platform: android
      appPackage: com.example.shop
      appActivity: .MainActivity
      device: pixel
      app:
        path: builds/shop-debug.apk

Describe the deviceCopy Anchor Link

A device is described by what people call it. agent-qa finds it on this machine when a test needs it:

registry:
  devices:
    pixel:
      platform: android
      model: Pixel 8
    iphone:
      platform: ios
      model: iPhone 16
      os: "18.4"

App IDs belong on the target, never on the device. Everything about a device lives in agent-qa.config.yaml, so every machine and CI job reads the same description.

How a device is found hereCopy Anchor Link

• iOS: a simulator whose name is the model, on the os when one is set. A plugged-in iPhone is never used.
• Android: a phone by the model it reports, as adb devices -l prints it (model:SM_S921B), or an emulator or AVD by its AVD name or its device profile.
• Case does not matter, and spaces, _ and - count as the same. os: "18" matches 18.2.
• When several match, the newest OS wins, then a running one. If that still leaves more than one, the test is refused and id picks one.
• With no model, the one running device of that platform is used.
• A matching simulator or AVD that is not running is started. agent-qa never creates one.
• When nothing matches, the test is refused before it starts, with a line listing what this machine has.

npx agent-qa devices shows where each device would run and what this machine has, so you can copy the model to write:

$ npx agent-qa devices
  pixel        here: Pixel_8_API_35 emulator emulator-5554
  iphone       not here: wants an iPhone 16 simulator on iOS 18.4
  galaxy-farm  remote farm (appium https://farm.example.com:443)
  Attached:   none
  Emulators:  Pixel_8_API_35 (Pixel 8, Android 15, running emulator-5554)
  Simulators: iPhone 16 Pro (18.4), iPhone 16e (18.4)

npx agent-qa devices list --json prints the same as JSON.

Choose the deviceCopy Anchor Link

A test runs on --device when you pass it, else the device in its own use, else its target's device:

use:
  device: pixel

npx agent-qa run tests/mobile/sign-in.yaml --device pixel

• A suite runs on one device: --device, else the device in the suite's use, else the one device its tests agree on. Tests that disagree are refused before the suite starts.
• device goes in a suite's or test's use, or on the target, never in the config's use.
• --device picks devices on this machine only, and refuses a device with a remote. To send a test to a remote device, name it in the suite's or test's use, or on the target.

Choose the app stateCopy Anchor Link

use.mobile.appState decides whether each session starts fresh:

• preserve keeps the app's data between sessions, so a signed-in user stays signed in;
• reset starts from a clean install state.

A native mobile app target needs it, and init writes preserve for Android and iOS projects.

Write the testCopy Anchor Link

test-id: t_mint-river-glow-salt-cove-dune-fern-harp-lake-moss
name: Sign in on Android
target: shop-android
steps:
  - Tap Sign in.
  - Enter email {{env:TEST_EMAIL}} and password {{secret:TEST_PASSWORD}}, then submit.
  - Verify the home screen greets the user by name.

The test runs on the target's device. A step can also run an Appium command directly, such as {{appium: mobile: pressKey, keycode: 4}}.

CapabilitiesCopy Anchor Link

A device's capabilities are merged after agent-qa's own and after the remote's, key by key, and null removes a key. A key carries a prefix such as appium: unless it is a W3C one such as platformName or browserName:

registry:
  devices:
    pixel-chrome:
      platform: android
      model: Pixel 8
      capabilities:
        browserName: Chrome
        appium:automationName: UiAutomator2

• A capability that contradicts the device or its target is refused before the session opens, with one line naming both places: for example appium:deviceName against model, appium:appPackage against the target, or appium:noReset against use.mobile.appState.
• Write every credential as a quoted {{secret:NAME}} reference. validate refuses a literal under a capability name that holds a credential word, at any depth. appium:keystorePath, appium:keyAlias and appium:sendKeyStrategy are settings, not credentials; appium:keystorePassword and appium:keyPassword are credentials.
• Secrets in a device's capabilities or a target's app.ref are used only to connect to the server. Steps, expected values, target URLs and hooks cannot use them.

Appium on this machineCopy Anchor Link

For a device on this machine, agent-qa uses the dashboard's Appium when the dashboard runs one, or starts its own on port 4723. To use an Appium server you run yourself on this machine, set AGENT_QA_APPIUM_URL. It must point at this machine; list any other server under registry.remotes instead.

Remote devicesCopy Anchor Link

A device with a remote runs on that remote, and agent-qa starts no Appium for it:

• An Appium server you or your team run, listed with protocol: appium. The server picks the device from model and os, and installs app.ref. See Remote browsers and devices.
• BrowserStack's real devices, through the BrowserStack plugin, which also uploads app.path for you.

plugins:
  - name: browserstack
    package: "@vostride/agent-qa-plugin-browserstack"
    options:
      username: "{{secret:BROWSERSTACK_USERNAME}}"
      accessKey: "{{secret:BROWSERSTACK_ACCESS_KEY}}"

registry:
  remotes:
    - name: browserstack-devices
      plugin: browserstack
  devices:
    bs-pixel:
      platform: android
      remote: browserstack-devices
      model: Google Pixel 9
      os: "15"

The run page names where each mobile test ran: this machine, or the remote's server, Appium version and device type.

Good to knowCopy Anchor Link

• Android runs record a video of the session. iOS runs do not.
• Saved logins are web only. On mobile, appState: preserve keeps a signed-in session.
• Android emulators and phones, and iOS simulators, are covered by agent-qa's own tests. Real iPhones behind a remote server work through capabilities, but agent-qa's tests do not cover them.
• agent-qa.local.yaml is no longer read, and a leftover one stops every command that loads the config. See v0.2.x to v0.3.x.