Quickstart

agent-qa ships as an npm package. Add it to an existing codebase when you already have an app repository, or start a small JavaScript workspace when you want to try agent-qa beside a non-JavaScript project first.

Install agent-qa

Install the CLI as a dev dependency so every teammate and CI job can run the same version.

>npm install -D agent-qa

If you want to use Codex or Claude Code subscription auth instead of provider API keys, install the optional subscription auth package as well.

>npm install -D @vostride/agent-qa-subscription-auth

Set up the testing environment

agent-qa runs web flows through Playwright-managed browsers, mobile flows through Appium, and hook-backed workflows through Docker. Install only the runtime support your project needs.

Web browsers

Install Chromium for the fastest first run. Use --all instead of --chromium when you want Chromium, Firefox, and WebKit.

>npx agent-qa install-browsers --chromium

Mobile drivers

For Android or iOS runs, install Appium first:

>npm install -g appium>appium --version

Then install the agent-qa Appium drivers:

>npx agent-qa install-mobile-drivers --all

Android also needs Android Studio, a reachable emulator or device, and the usual Android SDK environment variables. iOS needs Xcode and a simulator or connected device.

Hook runtime

Hooks run in an isolated Docker environment. Install Docker from Docker's Get Started page, start Docker Desktop or the Docker daemon, then confirm the CLI can reach it:

>docker --version>docker info

You only need Docker for tests or suites that use hooks. If your first run does not use hooks, you can set this up later.

Verify the environment

Run the doctor command after installing browser, mobile, or hook runtime support. It checks the local runtime pieces before your first test run.

>npx agent-qa doctor

Initialize the config

Create the config, example test, local folders, and ignore rules with the init command.

>npx agent-qa init

The init flow creates the files you will usually keep in source control:

agent-qa.config.yaml
tests/example-pass.yaml

It may also create local runtime folders under .agent-qa/. Keep generated run artifacts out of commits unless your team intentionally stores them.

Open the dashboard

Start the local dashboard from the project root:

>npx agent-qa dashboard

Open the printed local URL, then configure the model agent-qa should use:

1. Go to Config.
2. Add an LLM.
3. Choose the provider or subscription auth mode.
4. Test the connection.
5. Update the execution details to use the newly created LLM.
6. Save the config.

The dashboard writes back to your local config files, so review the diff the same way you would review any other project configuration change.

Run your first test from the dashboard

Use the generated passing example before writing a custom flow.

1. Open Tests.
2. Select Example passing test.
3. Click Run.

By default, runs execute in headless mode. Disable headless mode in the execution settings when you want to watch the browser or mobile session directly.

While the test is running, the live view shows the active execution. After the run completes, open the run view and inspect the full timeline:

• what the agent observed before each step
• how it planned the next action
• what it actually executed, such as clicking a button or filling an input
• how it verified the result
• how each assertion was evaluated, including the reasoning behind the pass or failure

This view is the fastest way to learn whether a failure came from product behavior, test wording, environment setup, or model interpretation.

Run your first test from the CLI

agent-qa is designed to work with teams at scale. Run the same tests from CI, release jobs, or post-deploy checks to catch regressions before they reach users.

For CI, start with a narrow command that targets the tests you trust, then expand to suites as coverage grows:

>npx agent-qa run tests/example-pass.yaml

Use the dashboard for rich local debugging and the CLI for repeatable automation. They read the same source-controlled config and test files.