Write your first test

Create a source-grounded agent-qa test file with a target, context, optional hooks, per-test overrides, and natural-language steps.

A test file describes one journey through one configured target. The required shape is small: test-id, name, target, and steps. Everything else is optional and should be added only when the run needs it.

Use the test configuration reference when you need every field. This page focuses on the first useful file.

Create a target

Tests point at a named target from agent-qa.config.yaml. A web target uses platform: web and must include url.

registry:
  targets:
    issue-tracker-web:
      platform: web
      url: http://localhost:3000

Add a test file

Create a YAML file under your test directory. The default workspace created by agent-qa init uses tests/**/*.yaml.

test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila
name: Find a task by title
target: issue-tracker-web
context: |
  The QA user is already signed in.
  The Tasks page contains project work items for the current workspace.
use:
  browser:
    name: chromium
    headless: true
    viewport:
      width: 1280
      height: 720
  timeout:
    step: 90s
    test: 10m
  logCapture:
    console: true
    network: true
meta:
  timeout: 10m
  retries: 1
  record: true
steps:
  - Open the Tasks page.
  - Search for "billing review".
  - Verify the first matching task title is "Billing review".

test-id is a stable generated ID. Generate one with the dashboard or the CLI:

>npx agent-qa ids generate test

Add only the context the agent needs

context is passed to the agent before the first step. Keep it about product state, user role, environment assumptions, and names the agent should recognize.

context: |
  The QA user is an admin in the Acme workspace.
  The product calls tasks "work items" in the sidebar.

Do not put credentials in context. Use env files, secrets, or hooks for runtime values that should not be written into prompts or artifacts.

Use hooks when the run needs setup or cleanup

setup and teardown contain hook IDs from your hook registry. Setup hooks run before steps. Teardown hooks run after the test finishes.

setup:
  - h_seed-task-workspace-bird-lake-slate-palm-cloud-frost
teardown:
  - h_update-borg-artha-any-packet-derive-torch-front-plied-bed

Add hooks for data seeding, API assertions, or cleanup. Leave them out for a first browser-only smoke test.

Write steps in plain language

Steps can be plain strings:

steps:
  - Open the Tasks page.
  - Verify the Create task button is visible.

Use structured step objects when one instruction needs its own timeout, retry behavior, screenshot request, max-attempts budget, or capture rule.

steps:
  - step: Search for "billing review" and open the first matching task.
    timeout: 90s
    retries: 1
    screenshot: true
    maxAttempts: 2
  - step: Copy the task key from the task detail header.
    capture:
      variable: TASK_KEY
      method: regex
      pattern: "TASK-[0-9]+"
  - Verify the activity feed mentions "{{env:TASK_KEY}}".

capture writes a runtime variable that later steps can reference with {{env:NAME}}. Supported capture methods are regex, selector, and ai.

steps:
  - step: Read the task status badge.
    capture:
      variable: TASK_STATUS
      method: selector
      selector: "[data-testid='task-status']"
  - step: Identify the account name shown in the page header.
    capture:
      variable: ACCOUNT_NAME
      method: ai
      description: The visible account or workspace name in the header.

Run it

Run a single test from the CLI:

>npx agent-qa run tests/find-task.yaml

The dashboard and CLI read the same file-backed definitions. Use the dashboard when you want the run timeline, screenshots, and agent reasoning while you tune the first test.