CLI

The agent-qa CLI is the local control surface for initializing projects, running tests and suites, opening the dashboard, validating files, managing credentials, and exposing AI-native integrations.

Global options

Use these options before the command name.

• --config <path>: load a config file. The default path is agent-qa.config.yaml.
• --log-level <level>: set verbosity to silent, error, warn, info, or debug.
• --verbose: shorthand for --log-level debug.
• --quiet: shorthand for --log-level silent.

>agent-qa --config ./agent-qa.config.yaml --log-level debug doctor

Setup

init

Purpose: create the initial agent-qa workspace files.

Usage: agent-qa init

Important options: --dir <path>, --platform <type>, --skip-install, --force.

Example: agent-qa init --platform web --dir .

install-browsers

Purpose: install Playwright-managed browser support for web tests.

Usage: agent-qa install-browsers

Important options: --all, --chromium, --firefox, --webkit, --with-deps, --force.

Example: agent-qa install-browsers --chromium

install-mobile-drivers

Purpose: install Appium drivers for Android and iOS test execution.

Usage: agent-qa install-mobile-drivers

Important options: --all, --android, --ios, --update, --unsafe.

Example: agent-qa install-mobile-drivers --all

doctor

Purpose: validate environment dependencies and local setup.

Usage: agent-qa doctor

Important options: use the global logging and config options when you need source attribution or quieter output.

Example: agent-qa doctor --verbose

Running and inspecting

run

Purpose: discover and execute test files or suites.

Usage: agent-qa run [patterns...]

Important options: --browser <name>, --platform <name>, --headless, --no-headless, --no-cache, --no-memory, --bail, --dry-run, --list-tests, --junit-output <path>, --screenshot-dir <dir>, --screenshot-mode <mode>, --reporter <names...>, --record, --config-debug, --test, --suite, --all, --device <name>, --var <kv...>, --run-attr <kv...>.

>agent-qa run tests/login.yaml --browser chromium --reporter console dashboard

dashboard

Purpose: start the dashboard web server.

Usage: agent-qa dashboard

Important options: --port <number>, --db <path>, --open.

Example: agent-qa dashboard --port 3470 --open

serve

Purpose: start configured local agent-qa services using the dashboard-backed service starter.

Usage: agent-qa serve

Important options: use global --config <path> when the service configuration lives outside the default path.

Example: agent-qa --config ./agent-qa.config.yaml serve

queue list

Purpose: show pending and running jobs in the dashboard execution queue.

Usage: agent-qa queue list

Important options: --json, --all, --server <url>.

Example: agent-qa queue list --all --server http://localhost:3470

queue cancel

Purpose: cancel a pending or running queue job by run id.

Usage: agent-qa queue cancel <runId>

Important options: --server <url>.

Example: agent-qa queue cancel run_123 --server http://localhost:3470

Writing and validation

validate

Purpose: validate config, test files, and suite references.

Usage: agent-qa validate [files...]

Important options: pass one or more files to validate a focused change; omit files to auto-discover the workspace.

Example: agent-qa validate tests/login.yaml suites/smoke.yaml

create-test

Purpose: scaffold a test YAML file with an auto-generated canonical test id.

Usage: agent-qa create-test <path>

Important options: the command fails if the output file already exists.

Example: agent-qa create-test tests/new-login.yaml

create-suite

Purpose: scaffold a suite YAML file with an auto-generated canonical suite id.

Usage: agent-qa create-suite <path>

Important options: the command fails if the output file already exists.

Example: agent-qa create-suite suites/smoke.yaml

ids generate

Purpose: generate a canonical agent-qa entity id with id-agent.

Usage: agent-qa ids generate <type>

Important options: --json.

Example: agent-qa ids generate test --json

ids validate

Purpose: validate that an id matches the canonical contract for an entity type.

Usage: agent-qa ids validate <type> <id>

Important options: --json.

Example: agent-qa ids validate test t_example-id --json

Configuration and auth

config set

Purpose: set a config value.

Usage: agent-qa config set <key> [value]

Important options: use global --config <path> to choose the config file.

Example: agent-qa config set services.dashboard.port 3470

config show

Purpose: show resolved config with source attribution.

Usage: agent-qa config show

Important options: use global --config <path> to inspect a non-default config file.

Example: agent-qa --config ./agent-qa.config.yaml config show

auth login

Purpose: authenticate a named subscription LLM config.

Usage: agent-qa auth login --config <name>

Important options: required --config <name>.

Example: agent-qa auth login --config claude

auth set

Purpose: save an API key or bearer token credential for a named LLM config.

Usage: agent-qa auth set --config <name> --type <type> [secret]

Important options: required --config <name> and --type <type> where type is api-key or bearer-token.

Example: agent-qa auth set --config openai --type api-key

auth status

Purpose: show credential status for configured LLMs.

Usage: agent-qa auth status

Important options: use global --config <path> to inspect credentials for a specific workspace config.

Example: agent-qa auth status

auth logout

Purpose: remove stored credentials for a named config.

Usage: agent-qa auth logout

Important options: --config <name>.

Example: agent-qa auth logout --config openai

auth-state capture

Purpose: open a headed browser for a web target, let you sign in manually, then save a named auth state after terminal confirmation.

Usage: agent-qa auth-state capture --target <target> --name <name>

Example: agent-qa auth-state capture --target app-staging --name qa-admin

auth test

Purpose: test an LLM connection using named config credentials.

Usage: agent-qa auth test

Important options: --config <name>, --provider <name>, --model <name>.

Example: agent-qa auth test --config openai

devices list

Purpose: show configured devices with the merged shared and local view.

Usage: agent-qa devices list

Important options: use global --config <path> for a non-default workspace config.

Example: agent-qa devices list

devices init

Purpose: scan connected devices and generate agent-qa.local.yaml.

Usage: agent-qa devices init

Important options: use global --config <path> to align generated local bindings with a specific workspace.

Example: agent-qa devices init

Maintenance

cache purge

Purpose: clear cached action plans.

Usage: agent-qa cache purge

Important options: --test <path>, --all, --force.

Example: agent-qa cache purge --all --force

clean-memory

Purpose: remove orphaned memory observation directories.

Usage: agent-qa clean-memory

Important options: -y, --yes.

Example: agent-qa clean-memory --yes

AI-native integrations

mcp

Purpose: start the MCP server over stdio for agent workflows.

Usage: agent-qa mcp

Important options: use global --config <path> to load workspace-specific analytics and service settings.

Example: agent-qa --config ./agent-qa.config.yaml mcp

skills

Purpose: list packaged agent-qa skills.

Usage: agent-qa skills

Important options: --json.

Example: agent-qa skills --json