agent-qa.config.yaml

agent-qa.config.yaml sits at the root of your project. init writes a complete one, and everything below is relative to the folder it lives in.

Example

workspace:  testMatch: [tests/**/*.yaml]  suiteMatch: [suites/**/*.suite.yaml]  hooksFile: hooks.yaml  envFile: .env  secretsFile: .env.secrets.localservices:  dashboard:    port: 3470  cache:    dir: .agent-qa/cache  vision:    screenshotSize: 256k    effectiveResolution: 768  accessibility:    enabled: true    runAfter: test-endregistry:  llms:    - name: claude-code      provider: anthropic-subscription      model: claude-sonnet-5-5  targets:    komazon:      platform: web      url: https://playground.vostride.dev/shop/plugins:  - name: subscription    package: "@vostride/agent-qa-subscription-auth"use:  llm: claude-code  browser:    name: chromium    headless: true    viewport: { width: 1280, height: 720 }  timeout:    step: 20m    test: 90m    navigation: 2m  planner:    maxActions: 30    maxPlanningTurns: 30    previousStepCount: 5

workspaceWhere tests, suites, hooks, variables and secrets live. Required.

The smallest valid config is the workspace block. Running tests also needs a model entry and a target, and a native mobile app target needs use.mobile.appState.

Top level

Key	Required	Holds
`workspace`	Yes	Where project files live.
`services`	No	Local services and their settings.
`knowledge`	No	Where memidex keeps its records.
`registry`	No	Models, remotes, targets and devices.
`use`	No	Run settings.
`plugins`	No	Plugin entries: browser and device services, and subscription sign-in.
`analytics`	No	Usage analytics and the dashboard's pass rate.

workspace

Key	Type	Meaning
`testMatch`	list of globs, required	Where test files are. `init` writes `tests/*/.yaml`.
`suiteMatch`	list of globs, required	Where suite files are. `init` writes `suites/*/.suite.yaml`.
`testPathIgnore`	list of globs	Files to leave out of discovery.
`hooksFile`	path, required	The hooks file. It must exist.
`envFile`	path, required	Variables for tests. It must exist and may be empty.
`secretsFile`	path, required	Secrets for tests, remotes and plugins, masked everywhere. It must exist and may be empty.

services

Key	Type	Default	Meaning
`dashboard.port`	number	3470	Dashboard port.
`dashboard.dbPath`	path	`.agent-qa/runs.db`	The run store.
`dashboard.artifactsDir`	path	`.agent-qa/artifacts`	Run artifacts.
`dashboard.baseUrl`	URL	none	Your dashboard's address, used for links in MCP results.
`dashboard.host`	host	`127.0.0.1`	Only loopback addresses are accepted.
`mcp.enabled`	boolean	`true` when the block exists	Declaring `services.mcp` makes the dashboard serve MCP too.
`mcp.port`	number	3471	Port for the dashboard's MCP endpoint.
`cache.dir`	path	`.agent-qa/cache`	Where the action cache lives. Required in the block.
`authState.dir`	path	`.agent-qa/auth-states`	Where saved logins live.
`logging.level`	`silent`, `error`, `warn`, `info`, `debug`	`warn` from `init`	Log level. `--log-level` overrides it.
`recording.enabled`	boolean	`true`	Record a video of each session.
`recording.keepFailuresOnly`	boolean	`false`	Keep videos of failed tests only.
`vision.screenshotSize`	size	none	Largest screenshot sent to the model. `init` writes `256k`. Needs `effectiveResolution`.
`vision.effectiveResolution`	number	none	Longest edge, in pixels, of screenshots sent to the model. `init` writes 768.
`accessibility.enabled`	boolean	none	Run accessibility checks on web. Required in the block.
`accessibility.standard`	`wcag2a`, `wcag2aa`, `wcag2aaa`	AA rules	Which rules to check.
`accessibility.runAfter`	`test-end`, `every-step`	none	When to check.
`accessibility.failOnViolation`	boolean	`false`	Fail the step on critical or serious violations.
`accessibility.disableRules`	list	none	axe rule IDs to skip.
`accessibility.exclude`	list	none	CSS selectors to leave out.

Without services.vision, screenshots are sent to the model at full size.

knowledge

Key	Type	Default	Meaning
`dir`	path inside the project	`agent-qa-knowledge`	Where memidex keeps records and policies.

registry

llms

A list of named model entries. Tests use the one use.llm names. Each entry has name, provider, model, and for some providers baseURL. See Models and credentials.

A list of places tests run other than this machine: a Playwright server, a CDP endpoint, an Appium server, or a service a plugin serves. Each entry has a protocol and url, or a plugin, never both.

registry:
  remotes:
    - name: grid
      protocol: playwright
      url: wss://grid.example.com/playwright
      query:
        token: "{{secret:GRID_TOKEN}}"
    - name: kernel
      plugin: kernel
      settings:
        headless: true

Key	Type	Meaning
`name`	lowercase slug, required	The remote's name, used by `use.browser.remote` and a device's `remote`. `local` is reserved for this machine.
`protocol`	`playwright`, `cdp`, `appium`	What the URL speaks. URL remotes only.
`url`	URL	The endpoint. URL remotes only.
`query`	map	Parameters added to the URL. `playwright` and `cdp` only.
`headers`	map	Headers sent when agent-qa connects. URL remotes only.
`basicAuth`	`user`, `password`	Basic authentication. URL remotes only.
`plugin`	`plugins` entry name	The entry that serves it. Plugin remotes only.
`settings`	map	The plugin's settings for this place. Plugin remotes only.
`connectTimeout`	duration, default `5m`	How long the connection may take.
`uploadTimeout`	duration, default `10m`	How long a plugin's app upload may take. Plugin remotes only.
`context`	`new`, `reuse`	Whether a web test opens its own browser context or uses the browser's first one. `cdp` and plugin remotes.
`capabilities`	map	Appium capabilities for every device on the remote. `appium` and plugin remotes.

Every credential is a quoted {{secret:NAME}} reference to a name in workspace.secretsFile. See Remote browsers and devices and Plugins.

targets

Key	Type	Meaning
`<name>`	lowercase letters, digits, hyphens	The target's name, used by tests and suites.
`platform`	`web`, `android`, `ios`, required	What the target is.
`url`	URL	Where web tests start. Required for web.
`product`	text	The product the target belongs to. It partitions knowledge and policies. Defaults to the target's name.
`environment`	`local`, `shared`, `production`	A label for the target's environment. agent-qa runs every target the same way.
`appPackage`, `appActivity`	text	The Android app.
`bundleId`	text	The iOS app.
`device`	device name	The device this mobile target's tests run on, unless a suite or test picks another.
`app.path`	relative path	A build installed on a device on this machine before a mobile test.
`app.ref`	text	What a remote server installs: an `https` URL or the server's own reference.

devices

A map of named devices for mobile tests. See Mobile testing.

Key	Type	Meaning
`<name>`	text	The device's name, used by a target's `device`, `use.device` in a suite or test, and `--device`.
`platform`	`android`, `ios`, required	The device's platform.
`model`	text	What the device is called, such as `Pixel 8` or `iPhone 16`.
`os`	quoted text	The Android or iOS version, such as `"18.4"`. A number is refused.
`id`	text	An Android serial or a simulator UDID, to pick one of two identical devices.
`remote`	remote name	An `appium` or plugin remote the device runs on. Without it, the device is on this machine.
`capabilities`	map	Appium capabilities, merged after agent-qa's own and the remote's.

use

These are project-wide run settings. Suites and tests can override each one in their own use.

Key	Type	Default	Meaning
`llm`	model entry name	none	The model that runs tests and learns from them.
`browser.name`	`chromium`, `firefox`, `webkit`	none	Browser for web tests. `name` and `headless` are both required when `browser` is set.
`browser.headless`	boolean	`true`	Hide the browser window.
`browser.viewport`	`width` and `height`	none	Window size.
`browser.remote`	remote name, or `local`	`local`	Where web tests run. See Remote browsers and devices.
`mobile.appState`	`preserve`, `reset`	none	Keep or clear app data between mobile sessions. Required to run a native mobile app; `init` writes `preserve` for Android and iOS projects.
`timeout.step`, `timeout.test`, `timeout.navigation`	durations	none	All three are required when `timeout` is set. `init` writes 20m, 90m and 2m.
`planner.maxActions`	number	30	Most actions in one step.
`planner.maxPlanningTurns`	number	30	Most model calls in one step.
`planner.previousStepCount`	number	5	How many earlier steps the model sees. Required when `planner` is set.
`logCapture.console`, `logCapture.network`	booleans	none	Record console and network logs. Both are required when `logCapture` is set.
`cache`	boolean	`true`	`false` turns off the action cache.
`parallel`	boolean	`false`	Run tests at the same time.
`workers`	number	3	How many tests run at once when parallel.
`buildTag`	text	none	Names the app build under test. `--build-tag` and `AGENT_QA_BUILD_TAG` override it.

device and authState belong in a suite or test, not here.

plugins

A list of plugin entries. Each one names a plugin package and holds its account's options, and remotes name the entry.

plugins:
  - name: subscription
    package: "@vostride/agent-qa-subscription-auth"
  - name: kernel
    package: "@vostride/agent-qa-plugin-kernel"
    options:
      apiKey: "{{secret:KERNEL_API_KEY}}"

Key	Type	Meaning
`name`	lowercase slug, required	The entry's name, unique in the list. Remotes name it with `plugin`.
`package`	npm package name	The plugin's package, exactly as installed.
`path`	folder inside the project	A plugin folder, instead of `package`, while you develop one.
`options`	map	The account's values, checked against the plugin's own schema. Credentials are `{{secret:NAME}}` references.

See Plugins.

analytics

Key	Meaning
`privacy: true`	Turns off usage analytics and error reporting. See Safety and privacy.
`passRateScope.attributes`	Counts only runs with these run attributes in the dashboard's pass rate. Each value is exact text or `{ regex: "<pattern>" }`.

Removed keys

These keys are gone. validate and every command that loads the config refuse each one with a message that says what to do.

Key	Instead
`allowProduction`	Delete it. Production targets run like any other target, and `environment` is a label.
`eval`	Delete it. `agent-qa eval replay` takes its settings as flags.
`services.cache.ttl`	Delete it. Cached plans do not expire by age.
`services.mcp.host`, `path`, `transport`	Delete them. `agent-qa mcp` listens on `127.0.0.1`, answers on any path, and uses stdio or `--http`.
`services.accessibility.runAfter: navigation`	Use `test-end` or `every-step`.
`knowledge.contextBudgetTokens`, `knowledge.commitArtifacts`	Delete them. agent-qa sizes each run's knowledge itself, and knowledge bundles never commit images.
`registry.providers`	List remote browsers and Appium servers under `registry.remotes`, with credentials in the secrets file.
`registry.devices.<name>.match`	Describe the device with `model` and `os`, and `id` for one unit. Move app IDs to the target and other Appium settings to `capabilities`.
`registry.devices.<name>.transport`	List the Appium server under `registry.remotes` and set the device's `remote`.
`registry.targets.<name>.app.browserstack`	Set `app.ref`.
`plugins.auth`, `plugins.providers`	Make `plugins` a list of named entries.

agent-qa.local.yaml is no longer read either. See v0.2.x to v0.3.x.

agent-qa.config.yaml

ExampleCopy Anchor Link

Top levelCopy Anchor Link

workspaceCopy Anchor Link

servicesCopy Anchor Link

knowledgeCopy Anchor Link

registryCopy Anchor Link

llmsCopy Anchor Link

remotesCopy Anchor Link

targetsCopy Anchor Link

devicesCopy Anchor Link

useCopy Anchor Link

pluginsCopy Anchor Link

analyticsCopy Anchor Link

Removed keysCopy Anchor Link

On this page

Example

Top level

workspace

services

knowledge

registry

llms

remotes

targets

devices

use

plugins

analytics

Removed keys