---
title: "How a run works"
description: "How an agent-qa run works: observe, plan, act and verify for each plain-English step, the verdicts and exit codes, suites and parallel runs, and post-run learning."
canonical_url: "https://vostride.com/docs/agent-qa/how-it-works"
md_url: "https://vostride.com/docs/agent-qa/how-it-works.md"
last_updated: "2026-10-24T01:12:08+05:30"
---

> [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt)
> Follow the Markdown links to read individual pages; the indexes list the available documentation.

# How a run works

> How an agent-qa run works: observe, plan, act and verify for each plain-English step, the verdicts and exit codes, suites and parallel runs, and post-run learning.

A test is a target and a list of plain-English steps. A run takes one or more tests, drives the app step by step, and ends with a verdict for every test and one for the whole run.

## One step at a time
For each step, agent-qa repeats a short loop until the step is done:

### Observe
It reads the current screen: the accessibility tree and page structure, plus a screenshot when the model can see images.

### Recall
It reads the [knowledge brief](https://vostride.com/docs/agent-qa/memory/serving.md) for this test: your policies, cautions about known failure modes, and facts learned on earlier runs.

### Plan and act
The model picks the next action, such as a click, a typed value or a navigation, and agent-qa performs it on the real app.

### Verify
It checks the result against what the step asked for. A step that says "Verify the total is $36.00" passes only when the screen proves it.

A step may take several actions. Each step is capped at 30 actions and 30 model calls by default (`use.planner.maxActions` and `use.planner.maxPlanningTurns`), and stops with the limit as its reason when it needs more.

Two things can skip the model:

- **The action cache.** When the same step meets the same screen, agent-qa replays the action that completed it last time. See [Caching](https://vostride.com/docs/agent-qa/caching.md).
- **Procedures.** A flow that memidex has seen pass several times can replay without planning. See [How knowledge reaches a run](https://vostride.com/docs/agent-qa/memory/serving.md).

When an action fails or the check rejects its result, the failure goes back to the model, which plans a recovery. A test that passes after such a recovery is `healed`. See [Self-healing](https://vostride.com/docs/agent-qa/self-improvement/healing.md).

## Verdicts
Each test ends with one of these:

| Verdict | Meaning |
| --- | --- |
| `passed` | Every step passed. |
| `healed` | Every step passed, and at least one needed a planned recovery. Counts as a pass. |
| `failed` | A step failed, or a setup hook failed. |
| `timeout` | The test ran longer than `use.timeout.test`. |
| `cancelled` | The run was stopped, for example with `Ctrl`+`C`. |
| `skipped` | An earlier test in the same suite failed, so this one did not start. |

The run's verdict is `failed` if any test failed or timed out, `cancelled` if the run was stopped, `healed` if any test healed, and `passed` otherwise. A [quarantined](https://vostride.com/docs/agent-qa/self-improvement/quarantine.md) test's failure is reported but does not fail the run.

## Exit codes
| Code | Meaning |
| --- | --- |
| `0` | The run passed or healed. |
| `1` | The run failed, or a test timed out. A missing test file, `.env`, hooks file or secrets file also exits 1. |
| `2` | agent-qa could not run the tests: an invalid config, a missing model credential, a pattern that matched nothing, or a refused run. |
| `130` | The run was cancelled. |

## Suites and parallel runs
A [suite](https://vostride.com/docs/agent-qa/guides/suites.md) runs its tests one after another on one browser or device. When one fails, the rest of that suite is skipped.

Tests named directly run one at a time by default. With `use.parallel: true` or `--parallel`, up to `use.workers` tests run at once (3 by default).

Every test in one run must belong to the same product. A target's product is its `product` field, or the target's own name. Tests on different products need separate `agent-qa run` invocations.

## After the verdict
The verdict and the exit code are final the moment the run ends. agent-qa then spends up to 60 seconds on post-run work:

1. **Analysis.** It classifies each failure, for example as an app bug, a test bug, a flaky failure or an environment problem.
2. **Curation.** It turns what the run proved into knowledge records. See [How knowledge is learned](https://vostride.com/docs/agent-qa/memory/learning.md).
3. **Proposals.** Extra checks on new knowledge, when an [eval corpus](https://vostride.com/docs/agent-qa/self-improvement/evals.md) exists.

A line on stderr reports each stage as `pending`, `running`, `complete`, `failed` or `skipped`. Work that does not finish in time stays queued, and the next local run finishes it. `agent-qa run --detach` skips the wait entirely. [Daemon and background work](https://vostride.com/docs/agent-qa/daemon.md) has the details.

## Where a run lives
Each run gets an ID such as `r_` followed by ten words, and a folder under `.agent-qa/runs/`. Read runs through agent-qa rather than from the folder:

```bash
npx agent-qa runs list
npx agent-qa runs overview <runId>
npx agent-qa digest <runId>
```

[Review a run](https://vostride.com/docs/agent-qa/guides/review-a-run.md) walks through the evidence, and the [dashboard](https://vostride.com/docs/agent-qa/dashboard.md) shows the same runs in a browser.

## The model
One model entry, `use.llm`, plans and checks every step of a run. A test or suite can name a different entry in its own `use.llm`. The same model writes up analysis and knowledge after the run.

Models that accept images get a screenshot with each decision. Set `vision: false` on a model entry to run text-only: agent-qa then works from the accessibility tree and page structure, and cannot check purely visual details.

## Documentation navigation

- [All documentation](https://vostride.com/docs.md)
- Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md)
- Previous: [Coding agents](https://vostride.com/docs/agent-qa/coding-agents.md)
- Next: [Caching](https://vostride.com/docs/agent-qa/caching.md)