Quickstart

This page takes you from an empty folder to a first run. The examples test Komazon, a small shop at https://playground.vostride.dev/shop/ that exists for this purpose. One of its pages has a deliberate bug, and one of the examples catches it.

Working with a coding agent? The agent quickstart hands this whole page to it with one prompt.

Before you startCopy Anchor Link

• Node.js 24 or newer.
• A model: a Claude or ChatGPT subscription, an Anthropic, OpenAI or Gemini API key, or an OpenAI-compatible or Anthropic-compatible endpoint. Local models work through the compatible endpoints.
• Network access to playground.vostride.dev and to your model provider.
• Docker, but only for tests that use hooks.

1. Set up agent-qaCopy Anchor Link

Run this in the folder that will hold your tests. In an existing app, that is usually the repository root.

>npx agent-qa@latest init

The command is the same for npm, pnpm, yarn and bun: init finds the project's package manager and installs with it.

In a terminal, init asks three questions:

• What platform will you test? Web is selected already. Mobile covers Android and iOS.
• Which model should agent-qa use?
• Which coding agents should agent-qa set up? Claude Code and Codex, with the ones it detects already selected.

Every question has a flag, and --yes takes every default. Setup and doctor lists the flags. The model you choose decides the model entry init writes:

--model <id> replaces the default model, and --base-url <url> points the model at another endpoint, such as a local model server. For example:

npx agent-qa@latest init --llm openai-compatible --model my-model --base-url http://127.0.0.1:1234/v1

Then init works through its steps, in this order:

1. Creates package.json, or updates the one that is there.
2. Writes the scaffold: the config, the example tests and suite, and the knowledge folder.
3. Installs agent-qa at an exact version with the project's package manager, and the subscription sign-in plugin when you chose a subscription.
4. Downloads Chromium for web targets. agent-qa uses its own copy, which leaves your everyday browsers alone. --no-browsers skips this step.
5. Installs the four skills, the MCP entries and the AGENTS.md block for the coding agents you chose. Coding agents shows what it writes.
6. Runs validate on the config and the tests.
7. In a terminal, starts the dashboard. While the model still needs a credential, the dashboard opens on its model page. --no-dashboard skips this step.

It ends with a short list of next steps that holds only what is left to do.

On Linux, Chromium also needs some system libraries. npx agent-qa install-browsers --chromium --with-deps installs them.

init writes:

• package.json, and a managed block in .gitignore;
• agent-qa.config.yaml;
• .env, which starts empty, and .env.secrets.local;
• hooks.yaml and scripts/examples/make-shipping-address.mjs;
• three example tests under tests/examples/, and suites/examples.suite.yaml, which runs two of them;
• the agent-qa-knowledge/ folder;
• for each coding agent you chose, its skills folder and its MCP entry: .claude/skills/ and .mcp.json for Claude Code, .agents/skills/ and .codex/config.toml for Codex.

A mobile-only project gets no example tests, suite or hook script.

If the folder is not a git repository yet, make it one and commit. Later you will read what agent-qa learned as a git diff against this commit.

git init
git add .
git commit -m "Add agent-qa"

The .gitignore block that init wrote keeps .agent-qa/, .env, .env.secrets.local and node_modules/ out of the commit.

2. Connect the modelCopy Anchor Link

init signs no model in, so connect yours before the first run.

With a subscription, sign in. Name the model entry init wrote:

npx agent-qa auth login --config claude-code

It prints a sign-in URL and opens it in your browser. For Claude, paste the code the page shows back into the terminal. Use --config codex for a ChatGPT subscription.

With an API key, add the key on the dashboard's model page, which init opens while the model still needs one. Or save it from the terminal, naming the entry init wrote, anthropic, openai or gemini:

npx agent-qa auth set --config anthropic --type api-key

auth set asks for the key at a hidden prompt, so it never lands in your shell history. agent-qa keeps credentials in ~/.agent-qa/auth.json, outside the project. Models and credentials covers every provider.

Then check the whole setup:

npx agent-qa doctor

doctor looks at Node.js, the config, the browser, Docker, your coding agents and your credentials, and sends one short request to your model. Setup and doctor lists every check.

3. Run the examplesCopy Anchor Link

>npx agent-qa run suites/examples.suite.yaml

Expect one pass, one failure and exit code 1:

• The checkout journey signs in, buys a mug and checks the order. It passes.
• The cart-total test catches the playground's deliberate bug. Two mugs should cost $53.00, and the bag shows $29.00.

The summary's first line reads Tests: 1 of 2 passed. After the verdict, agent-qa spends up to a minute analyzing the run and recording what it learned. That work never changes the exit code.

4. Look aroundCopy Anchor Link

Open the dashboardCopy Anchor Link

>npx agent-qa dashboard --open

It prints Dashboard running at and a URL. Each run shows its tests, steps and evidence. Accessibility results appear here too: open a test's last step and its A11y tab. Stop the dashboard with Ctrl+C.

Read what agent-qa learnedCopy Anchor Link

Runs write what they learn into agent-qa-knowledge/. After the first runs every record is a new file, so ask git to include new files in the diff:

git add --intent-to-add agent-qa-knowledge
git diff -- agent-qa-knowledge

Commit the records you want to keep. memidex explains what each file means.

See the action cacheCopy Anchor Link

agent-qa remembers the actions a passing step took and replays them next time without asking the model to plan, as long as the screen looks the same. The step's check still runs. On your default branch it only saves when asked, so the first of these two runs passes --save-cache:

npx agent-qa run tests/examples/komazon-checkout.yaml --save-cache
npx agent-qa run tests/examples/komazon-checkout.yaml

Steps that type a secret are never cached, so in this journey every step except the sign-in can replay its actions. A step whose check reads a value from the screen still reads it live, and the Cache: line in the summary counts that step as a miss. Caching has the details.

Ask your coding agentCopy Anchor Link

init already set up the coding agents you chose, so ask yours to run the examples. Coding agents shows what init wrote for each one.

When something is missingCopy Anchor Link