Suite

A suite YAML file runs multiple tests as one workflow. Use suites for release smoke checks, regression bundles, and end-to-end web journeys where several tests should share a target, context, or setup hook.

Example web suite

suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis
name: "Issue tracker: web task lifecycle"
target: issue-tracker-web
context: |
  Run this suite against the browser target configured as issue-tracker-web.
  The QA user is already signed in.
setup:
  - h_seed-web-workspace-calm-cedar-lamp-river-field
teardown:
  - h_update-borg-artha-any-packet-derive-torch-front-plied-bed
use:
  browser:
    name: chromium
    headless: true
    viewport:
      width: 1280
      height: 720
  cache: false
tests:
  - test: tests/task-create-and-complete.yaml
    id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila

suite-id

suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis

Description: Optional generated suite ID.

Possible values: canonical suite IDs that start with s_ and contain 10 id-agent words.

Required: no.

Default: not set. Generate it with agent-qa ids generate suite or the dashboard.

name

name: "Issue tracker: web task lifecycle"

Description: Human-readable suite name.

Possible values: any string.

Required: yes.

Default: none.

target

target: issue-tracker-web

Description: Target name from registry.targets. Every listed test runs against this target unless runtime preparation resolves otherwise.

Possible values: any configured target name. For web suites, use a target whose platform is web.

Required: yes.

Default: none.

context

context: |
  Run this suite against the browser target configured as issue-tracker-web.
  The QA user is already signed in.

Description: Shared background passed to every test in the suite.

Possible values: any string, usually a multi-line block.

Required: no.

Default: not set.

setup

setup:
  - h_seed-web-workspace-calm-cedar-lamp-river-field

Description: Hook IDs that run before suite tests.

Possible values: hook IDs registered in hooks.yaml.

Required: no.

Default: no setup hooks.

teardown

teardown:
  - h_update-borg-artha-any-packet-derive-torch-front-plied-bed

Description: Hook IDs that run after suite execution.

Possible values: hook IDs registered in hooks.yaml.

Required: no.

Default: no teardown hooks.

use

use:
  browser:
    name: chromium
    headless: true
  cache: false

Description: Suite-level run overrides. Use authState here when every child web test should share one selected authenticated browser context.

Possible values: the same override shape supported by test use: browser, timeout, healing, planner, logCapture, cache, authState, mobile, llm, parallel, and device.

Required: no.

Default: inherited from global config and CLI flags.

use.browser

use:
  browser:
    name: chromium
    headless: true
    viewport:
      width: 1280
      height: 720

Description: Browser defaults for every web test in the suite.

Possible values: name, headless, and optional viewport.width / viewport.height.

Default: inherited.

use.timeout

use:
  timeout:
    step: 3m
    test: 40m
    navigation: 90s

Description: Timeout defaults for every test in the suite.

Possible values: duration strings.

Default: inherited.

use.healing

use:
  healing:
    maxAttempts: 2

Description: Self-healing retry budget for suite tests.

Possible values: number.

Default: inherited.

use.planner

use:
  planner:
    maxSubActions: 10
    previousStepCount: 5

Description: Planner defaults for suite tests.

Possible values: numbers for maxSubActions and previousStepCount.

Default: inherited.

use.logCapture

use:
  logCapture:
    console: true
    network: true

Description: Browser console and network log capture defaults.

Possible values: booleans for console and network.

Default: inherited.

use.cache

use:
  cache: false

Description: Enables or disables action cache for suite tests.

Possible values: true or false.

Default: inherited.

use.authState

use:
  authState: qa-admin

Description: Named web auth state to load for the suite's shared browser context. Child tests inherit the suite auth state when they omit use.authState. Repeating the same child auth-state name is allowed; selecting a different child auth state is rejected.

Possible values: a configured lowercase auth-state name.

Default: not set.

Only one primary auth state is supported per suite run. See Auth state.

use.llm

use:
  llm: codex

Description: LLM config name for suite tests.

Possible values: a name from registry.llms.

Default: inherited.

use.parallel

use:
  parallel: false

Description: Allows parallel execution where supported.

Possible values: true or false.

Default: inherited.

use.device

use:
  device: android-local

Description: Mobile device profile override. Web suites usually do not set this.

Possible values: a name from registry.devices.

Default: inherited or CLI-selected.

use.mobile

use:
  mobile:
    appState: preserve

Description: Native mobile app-state override. Web suites do not use this value.

Possible values: preserve or reset.

Default: inherited.

tests

tests:
  - test: tests/task-create-and-complete.yaml
    id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila

Description: Ordered list of test entries. Each entry links a file path to the expected test ID inside that file.

Possible values: one or more objects with test and id.

Required: yes.

Default: none.

tests.test

tests:
  - test: tests/task-create-and-complete.yaml

Description: Workspace-relative path to a test YAML file.

Possible values: path string.

Required: yes for each test entry.

tests.id

tests:
  - id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila

Description: Generated test-id inside the referenced test file. This keeps identity stable even when filenames change.

Possible values: canonical test IDs that start with t_ and contain 10 id-agent words.

Required: yes for each test entry.

Hook variable flow

Suite hooks run sequentially. Variables exported by a successful hook are passed to later hooks and to suite tests. Per-test setup hooks can add or override variables for that test. Suite teardown hooks receive accumulated suite variables.

This makes suites a good place to create shared web records, authenticate a browser session, or clean up server state after all tests complete.