CLI
Every agent-qa command, grouped by what you are doing, with the options that matter.
Run agent-qa from the folder that holds agent-qa.config.yaml, usually with npx agent-qa. Every command prints its options with --help:
npx agent-qa run --helpGlobal options
| Option | Meaning |
|---|---|
--config <path> | Config file to use. Default agent-qa.config.yaml in the current folder. Put it before the command: agent-qa --config ci.yaml run. |
--log-level <level> | silent, error, warn, info or debug. |
--verbose, --quiet | Shorthand for debug and silent. |
-V, --version | Prints the version. |
Commands that read runs or knowledge take --json. It prints exactly what the matching MCP tool returns, with secrets masked.
Set up
| Command | What it does |
|---|---|
init | Sets up a project in the folder it runs in, and on a set-up project offers what to repair or upgrade. Flags: --platform, --llm, --model, --base-url, --agents, --no-browsers, --no-dashboard, --force, --setup, --pm, --yes, --json, and --capabilities to print the capability manifest without writing anything. See Setup and doctor and Agent quickstart. |
install-browsers | Installs browsers: --chromium, --firefox, --webkit or --all. --with-deps adds Linux system libraries; --force reinstalls. |
install-mobile-drivers | Installs Appium drivers: --android, --ios or --all. --update updates them. |
doctor | Checks Node.js, the config, the knowledge store, agent-qa's versions, each coding agent's skills and MCP entry, the AGENTS.md block, the secrets file, workspace files, Docker, credentials, the model connection, the browser, each remote and plugin entry, Appium, the Android SDK, Xcode, whether a newer agent-qa is out, and test discovery. It skips the local browser when every web test runs on a remote, and the local mobile tools when every device is remote. --remote <name> shows one remote's rows; --json prints the checks as JSON. Exits 3 when a row fails, 0 otherwise, and 2 for an unknown --remote. See doctor. |
validate [files...] | Checks the config and every test and suite, or the files you name, without running anything. |
config show | Prints the config file's values with secrets masked. |
config set <key> [value] | Writes one value into the config, keeping comments. It writes the value as text, so edit numbers and booleans in the YAML directly. |
skills | Lists the packaged skills and where they are. |
skills uninstall | Removes what init installed for coding agents: the skills, the MCP entries and the AGENTS.md block its manifest lists. --agents takes claude-code, codex or all, separated by commas, default all; --dry-run prints what would be removed and changes nothing; --json prints the result as JSON. |
Credentials
| Command | What it does |
|---|---|
auth login --config <name> | Signs in a subscription entry (Claude or Codex) in the browser. |
auth set --config <name> --type api-key | Saves an API key, asking for it at a hidden prompt. --type bearer-token is for Anthropic-compatible endpoints. The key may be passed as the last argument instead, for CI. |
auth status | Shows each model entry and whether it has a credential. |
auth test [--config <name>] | Sends one short request and reports the response time. |
auth logout [--config <name>] | Removes a stored credential. |
Here --config names a model entry from registry.llms, not a file. To use another config file, put the global option first: agent-qa --config other.yaml auth status.
Saved logins and devices
| Command | What it does |
|---|---|
auth-state capture --target <t> --name <n> | Opens a browser, lets you sign in, and saves the session. |
auth-state list [--target <t>] | Lists saved logins. |
auth-state remove --target <t> [--name <n>] | Removes one saved login, or all for the target. |
devices, devices list | Lists each device in registry.devices with the phone, emulator or simulator it finds on this machine, or the remote it runs on, then everything this machine has. --json. See Mobile testing. |
devices init was removed. Devices are found by model and os, so nothing needs binding.
Plugins
| Command | What it does |
|---|---|
plugins add <package> | Declares the package in package.json at an exact version, installs it with the project's package manager and adds its plugins entry. The package may carry an exact version, as <package>@<version>. --pm <name> picks the package manager, default detected from the project; --name <name> names the entry, default taken from the package name. |
plugins list | Lists each plugins entry with its package, origin, version, protocol, the remotes that use it and its state: ok, not_installed, version_mismatch, protocol_mismatch, bad_manifest or unused. It reads manifests only and runs no plugin code. --json. |
plugins describe <name> | Shows what an entry's plugin serves and every option and setting it takes, from its shipped schemas. --json. |
plugin validate <target> | For plugin authors. Checks a plugin folder (./path) or an installed package: its manifest, schemas, credential marks and skill, then a quick conformance pass. --json. |
See Plugins and Write a plugin.
Run tests
npx agent-qa run [patterns...] [options]With no patterns, run runs every test that workspace.testMatch finds.
| Option | Meaning |
|---|---|
--suite | Run every suite that workspace.suiteMatch finds. |
--all | Run every suite, then every test. |
--headless, --no-headless | Override use.browser.headless. |
--browser <name> | Run web tests in chromium, firefox or webkit, overriding use.browser.name. Where they run still comes from the files. |
--device <name> | Run mobile tests on this device from registry.devices, instead of the one the test, suite or target names. It must be a device on this machine; a device with a remote is refused. |
--var KEY=VALUE | Set a variable, read in steps as {{env:KEY}}. Repeatable. Put test paths before it. |
--run-attr KEY=VALUE | Attach a label to the run, for filtering in the dashboard. Repeatable. |
--build-tag <tag> | Name the build of your app being tested. Also AGENT_QA_BUILD_TAG or use.buildTag. |
--parallel, --no-parallel | Run tests at the same time, or one by one. |
--workers <n> | How many tests run at once with --parallel. Default 3. |
--disable-cache | Neither read nor write the action cache. |
--save-cache | Write cache entries even on your default branch. |
--record, --no-record | Turn session video on or off for this run. |
--screenshot-mode <mode> | failure, the default, or every-step, which also keeps a screenshot after every step. |
--reporter <names...> | console (default) or junit. Naming reporters replaces the default, so list console too to keep terminal output. |
--junit-output <path> | Write a JUnit report. |
--dry-run | Parse and list the tests, without running them. |
--list-tests | List discovered test files and exit. |
--config-debug | Print each test's resolved settings and where each came from. |
--detach | Exit right after the verdict, leaving post-run analysis and curation queued. |
No flag or environment variable moves a test to a remote; the files decide. --bail, --screenshot-dir and --max-ablations are gone, and run refuses them as unknown options.
Exit codes: 0 passed or healed, 1 failed, 2 could not run, 130 cancelled. See How a run works.
| Command | What it does |
|---|---|
cache purge --all | Clears every cached action. |
Review runs
| Command | What it does |
|---|---|
runs list | Lists finished runs, newest first. Filters: --verdict, --test, --since, --limit. |
runs overview <runId> | Maps a run: tests, steps, evidence, screenshots and knowledge. |
runs evidence <runId> | Reads one step's --kind console, network, a11y, aria or variables, or exports a file with --screenshot <sha> --out <file>. |
runs knowledge <runId> | Shows what the run was given and what it changed in memidex. |
runs mark <runId> --kind <kind> | Marks a verdict as false-fail, false-pass or confirmed-bug. See Mark a run. |
digest [runId] | Summarizes a run: verdicts by cause, post-run status and marks. Defaults to the latest run. |
Knowledge
| Command | What it does |
|---|---|
knowledge bundles | Lists every record with its trust and status. |
knowledge inspect | Filters records by --product, --scope, --kind, --status, --anchor, --record, --as-of-build or --unresolved. |
knowledge show <recordId> | Shows one record and where it came from. |
knowledge diff <ref> | Lists changes since a run count, or a build tag with --product. |
knowledge gates <recordId> | Shows the marks and approvals behind a record, its promotions, and whether it would promote now. |
knowledge brief-why <runId> --step <n> | Explains what a step's brief said, and why. |
knowledge rollback | Undoes a curation job (--job) or a campaign (--from-build, --to-build, --anchor-cluster, --origin-class). |
knowledge purge <recordId> | Deletes a record from this machine for good. |
knowledge rebuild-index | Rebuilds the search index from the records. |
knowledge drain | Finishes queued curation now. --status only reports. |
entities resolve <ref> | Finds a record from a short ID such as MEM-001, an agentqa:// URI or free text. |
entities get <id> | Shows the card for an exact short ID or URI. |
See Inspect and correct.
Ask, chat and functions
| Command | What it does |
|---|---|
ask <message> | Does one thing you ask for in plain words. --session <id> continues a conversation. |
chat | Opens a conversation. --continue, --session <id>, --list. |
fn [id] | Calls one function by ID with --input '<json>', or lists them all with --list. |
See Ask and chat.
Services
| Command | What it does |
|---|---|
dashboard | Starts the dashboard. --port <n>, --db <path>, --open. |
serve | Starts the dashboard with your config's settings, with no flags. |
mcp | Starts the MCP server over stdio. --http [port] serves it on 127.0.0.1 instead, port 3471 by default. |
daemon | Runs the background host in the foreground. daemon status and daemon stop manage it. |
queue list, queue cancel <runId> | Shows or cancels runs queued in a running dashboard. Pass its address with --server <url>. |
agent-jobs list | Lists background jobs. pump, cancel and requeue manage them. |
See Dashboard, MCP server and Daemon and background work.
Evals
agent-qa eval replays recorded runs offline: eval extract turns recorded runs into staged recordings, eval replay asks the current model again and scores the answers, and eval report builds one report. eval replay takes --judge <name>, repeated for up to three, --k <n>, --seed <n> and --judge-allow-same-family. See Offline replay.
Environment variables
| Variable | Effect |
|---|---|
AGENT_QA_HEADLESS | true or false, overrides use.browser.headless. |
AGENT_QA_LOG_LEVEL | Overrides services.logging.level. |
AGENT_QA_DASHBOARD_PORT | Overrides services.dashboard.port. |
AGENT_QA_CACHE_DIR | Overrides services.cache.dir. |
AGENT_QA_MCP_PORT | Overrides services.mcp.port. |
AGENT_QA_BUILD_TAG | Names the app build, like --build-tag. |
AGENT_QA_APPIUM_URL | An Appium server on this machine to use instead of starting one. Another machine's Appium goes under registry.remotes. |
AGENT_QA_DASHBOARD_URL | The dashboard address for queue commands. |
AGENT_QA_LIVE_EVENTS | true prints every run event as a JSON line on stdout. |
AGENT_QA_MODEL_CACHE_DIR | Where to look for the local embedding model. |
AGENT_QA_DEBUG_LLM | Logs the shape of model requests and provider errors. |