Configuration
The files that configure agent-qa, how their settings combine, and how to check them.
agent-qa is configured with plain files. The ones you commit describe the project, including where tests run; the local ones hold secrets.
| File | What it holds | Commit it |
|---|---|---|
agent-qa.config.yaml | Targets, devices, remotes, plugins, models, settings and services. | Yes |
Test files, tests/**/*.yaml | One journey each. | Yes |
Suite files, suites/**/*.suite.yaml | Tests that run together. | Yes |
hooks.yaml | Scripts tests can run. | Yes |
agent-qa-knowledge/policies/ | Your QA rules. | Yes |
.env | Variables for tests. | No |
.env.secrets.local | Secrets for tests, remotes and plugins, masked everywhere. | No |
~/.agent-qa/auth.json | Model credentials, outside the project. | No |
Devices, remotes and plugins are described in agent-qa.config.yaml, so every machine and CI job reads the same description; only their credentials stay in the secrets file. agent-qa.local.yaml is no longer read. See v0.2.x to v0.3.x.
How settings combine
Settings under use cascade. A suite overrides the project, a test overrides its suite, and command-line flags override everything:
agent-qa.config.yaml → suite use → test use → CLI flagsA few environment variables override the config file and lose to flags: AGENT_QA_HEADLESS, AGENT_QA_LOG_LEVEL, AGENT_QA_DASHBOARD_PORT, AGENT_QA_MCP_PORT and AGENT_QA_CACHE_DIR. No flag or environment variable moves a test to a remote; the files decide where tests run.
To see where each setting of a test came from:
npx agent-qa run tests/checkout.yaml --dry-run --config-debugRules every file follows
- Unknown keys are errors in the config, test and suite files. A typo fails validation instead of being ignored.
hooks.yamlignores keys it does not know. - Paths are relative to the config file and must stay inside its folder.
- Durations use a number and a unit:
30s,5m,1h,7d. - Sizes use a number and a unit:
256k,1m,5mb. - IDs are a prefix and ten lowercase words joined by hyphens:
t_for tests,s_for suites,h_for hooks.
Check your configuration
| Command | Checks |
|---|---|
agent-qa validate | The config and every test and suite, against their schemas, including each plugin's options and settings. |
agent-qa doctor | The whole setup: Node.js, agent-qa's versions and whether a newer one is out, files, credentials, the model, browsers, devices, remotes, plugins, the knowledge store, Docker, each coding agent's skills and MCP entry, and the AGENTS.md block. See doctor's rows. |
agent-qa config show | The values in your config file, with secrets masked. |
agent-qa plugins list | Each plugins entry and whether it is installed at the right version. |
agent-qa devices | Where each device runs, and what this machine has. |
A key that agent-qa removed fails validation with a message that says what replaced it. See removed keys. A config from agent-qa v0.1 fails with a message that lists the upgrade steps; see v0.1.x to v0.2.x.