Suites

A suite runs multiple test files as one workflow. Use suites for smoke checks, release gates, and multi-step regression flows where the tests should share setup, cleanup, target selection, or run defaults.

For the complete schema reference, see Suite.

Create a suite file

suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis
name: Issue tracker release smoke
target: issue-tracker-web
context: |
  Run against the browser target configured as issue-tracker-web.
  The QA user is already signed in to the Acme workspace.
setup:
  - h_seed-web-workspace-calm-cedar-lamp-river-field
teardown:
  - h_update-borg-artha-any-packet-derive-torch-front-plied-bed
use:
  browser:
    name: chromium
    headless: true
    viewport:
      width: 1280
      height: 720
  timeout:
    navigation: 30s
    step: 90s
    test: 20m
  cache: false
tests:
  - test: tests/task-create.yaml
    id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila
  - test: tests/task-complete.yaml
    id: t_rag-ember-rye-lamp-calm-track-sage-fair-yam-nix

suite-id is optional but useful for stable run identity. Generate one with:

>npx agent-qa ids generate suite

What the suite owns

The suite schema supports:

• suite-id: optional generated suite ID.
• name: human-readable suite name.
• target: target name from registry.targets.
• context: shared background passed to each child test.
• setup: hook IDs that run before suite tests.
• teardown: hook IDs that run after suite execution.
• use: suite-level run overrides.
• tests: ordered child test entries.

The tests list is required and must contain at least one entry.

Tests list and identity

Each child entry has a path and the expected test ID inside that file.

tests:
  - test: tests/task-create.yaml
    id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila

The runner loads each file, parses the test definition, and checks that the file's test-id matches the suite entry id. A mismatch fails the suite before that child test runs.

Tests run in the order listed. If a child test fails, later children are marked skipped for the suite run.

Target and overrides

When target is set on the suite, that target overrides each child test target for the suite run. This lets the same test files run against a shared web, Android, or iOS target in different suites.

target: issue-tracker-web

Suite use values act as shared defaults. Child test use values can still override them.

use:
  browser:
    name: chromium
    headless: true

There is no separate field named overrides in suite YAML. Overrides are expressed through target and use.

Precedence

Config files are loaded before runtime overrides.

config file -> env overrides -> CLI flags

Within test execution, use values are layered from broadest to most specific:

global -> suite -> test -> CLI flags

Nested objects are deep-merged. For example, a global viewport can remain in effect while a suite changes only browser.name, and a child test changes only browser.headless.

# agent-qa.config.yaml
use:
  browser:
    name: chromium
    headless: true
    viewport:
      width: 1280
      height: 720

# suite
use:
  browser:
    name: firefox

# child test
use:
  browser:
    headless: false

The child test keeps the inherited viewport, uses firefox, and runs with a visible browser.

For keys that have CLI flags, flags win for that run. For example, --device selects the mobile device for the suite run.

Setup and teardown

Suite setup hooks run before the platform adapter is set up and before any child test starts. Variables exported by successful suite setup hooks are available to the suite target URL interpolation and to every child test.

Per-test setup hooks run before that child test's steps. Per-test teardown hooks run after that child test finishes. Suite teardown hooks run after suite execution and receive the accumulated suite variables.

setup:
  - h_seed-web-workspace-calm-cedar-lamp-river-field
teardown:
  - h_update-borg-artha-any-packet-derive-torch-front-plied-bed

Use suite hooks for shared records, login state, or cleanup that belongs to the whole group. Use test hooks when each child needs its own data.

Mobile suites

Mobile suites use the same suite shape. The suite target can be android or ios, and use.mobile.appState must resolve to preserve or reset.

target: issue-tracker-android
use:
  device: android-local
  mobile:
    appState: reset
tests:
  - test: tests/mobile-inbox.yaml
    id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila

If the suite does not set use.device, every child test must resolve to one device. Child tests in the same mobile suite cannot mix multiple device names unless the suite itself selects the device.

Run a suite

Run one suite file directly:

>npx agent-qa run suites/release-smoke.suite.yaml

Or run discovered suites from your workspace config:

>npx agent-qa run --suite