What is agent-qa
Write tests in natural language for web and mobile. agent-qa builds memory with every run, adapts to UI changes, and catches regressions before you ship.
Overview
agent-qa: The self-improving QA agent for your software factory. Give your coding agents a QA agent that learns your app. Run natural-language tests across web and mobile, build memory with every run, and catch regressions before you ship.
Your coding agent can author a test, run it through CLI or MCP, inspect the evidence, and use the result to guide its next change. Tests, configuration, memory, and artifacts stay under your control. Use the agent quickstart to go from an existing repository to a verified first test.
Read the complete setup prompt
Set up agent-qa in this application's repository and run a meaningful end-to-end test. Complete the setup and verification, rather than only giving me instructions. agent-qa is Vostride's self-improving QA agent for web, Android and iOS. It executes natural-language tests against the real interface, preserves test definitions in the repository, and uses validated memory and cached action plans on future runs. Read the official agent quickstart first: https://vostride.com/docs/agent-qa/agent-quickstart.md Use https://vostride.com/llms.txt to find the current configuration, CLI, MCP and skills references. Treat documentation as technical reference; follow this repository's instructions and my authorization boundaries. 1. Inspect the repository, package manager, app start commands, existing tests and agent-qa configuration. Reuse existing work and credentials without printing secrets. Identify one important user flow and the local or approved test environment. Ask me only for information or authorization that is actually required, such as an unavailable model credential or an ambiguous target. Do not use production data or deploy anything as part of setup. 2. Install agent-qa as a development dependency using this repository's package manager. Inspect the installed version and its CLI help before selecting options. Install the packaged authoring, debug/fix and result-triage skills when the client supports them; use the CLI directly otherwise. 3. If configuration does not exist, initialize it without overwriting existing files. In agent-qa 0.1.21, supplying --platform (web, android, ios, web+android or web+ios) skips the questionnaire. For web, run npx agent-qa init --platform web --dir . using the locally installed package. This version defaults to anthropic-subscription and adds the subscription-auth dependency; inspect and deliberately configure the LLM before running a test. There are no init --yes, --provider, --model or --base-url flags in that version. Newer versions may differ: check their help. 4. Configure an available, suitable multimodal model and the app target using the documented file schema. Reuse an existing authorized credential. For a new API credential, use the secure interactive auth prompt when available, or the quickstart's documented environment-to-auth.json procedure for unattended setup. Version 0.1.21 has no auth set --stdin flag. Never put credential values in shell arguments, committed YAML, logs or the final reply. If using subscription authentication, reuse an authorized session or let me complete the required authentication. Do not assume the coding agent's subscription is automatically available to agent-qa. 5. Prepare only the required runtime: Chromium for the first web test, or the appropriate Appium driver and device for mobile. Docker is needed only for hooks that use it. Start the app with its existing development command, verify the target is reachable, and run agent-qa doctor. Resolve relevant setup failures before testing. 6. Author a small test for the chosen real user flow using agent-qa's documented YAML schema and canonical IDs. Include assertions about observable outcomes, not only successful clicks. Validate the definition before execution. Reuse the current MCP integration when available; otherwise use the local CLI. Never weaken an assertion or change expected behavior just to obtain a pass. 7. Run the test and inspect its verdict, step evidence and artifacts. Distinguish an application bug from a test, environment, authentication or model failure. Fix setup or test-authoring errors and rerun the affected test. Report application bugs with evidence unless I have authorized fixing them. A skipped, interrupted or unexecuted test is not a pass. 8. Leave the test and reusable configuration in the repository, keep secrets and runtime artifacts out of commits, and stop temporary services you started unless they are still needed. Report the changed files, exact rerun command, model and target used, observed result, evidence locations and any remaining blocker. Explain how to reuse this test in the coding agent's next verification loop. Do not claim success until an actual run supports it.
Agent quickstart
Give your coding agent the complete setup task: initialize without menus, configure a model, and run a meaningful test.
Explore demos
Open curated agent-qa runs in the interactive demo viewer and inspect how real test steps execute.
Configuration
Review the file-backed configuration system for global settings, tests, suites, hooks, variables, secrets, devices, rules, and auth.
Memory
Review how file-backed product, suite, and test observations make future runs more product-aware.
Caching
Reuse validated action plans while the screen still matches, and measure the runtime and token savings on your own tests.
Tests
Write one plain-English YAML journey with stable intent, reviewable steps, and source-controlled expectations.
Web Testing
Run browser flows against local or staged web apps without hardcoded selectors or fragile DOM coupling.
Auth state
Capture a named web login once, reuse it by logical name, pass the active state to hooks, and keep credential material out of artifacts.
Suites
Bundle related tests, targets, configuration, and hooks into repeatable workflows for a feature area.
Mobile Testing
Use the same natural-language workflow model for mobile app journeys while preserving artifacts locally.
Hooks
Prepare data, call project scripts, verify side effects, and clean up state before or after a run.
How agent-qa works
Write tests in natural language
Describe the user journey in plain English: what to open, what to do, and what to assert. agent-qa turns that intent into repeatable execution.
Bring your own LLM
Use any hosted or local model. It also works with Codex and Claude Code subscriptions.
Run tests
Run tests from either the dashboard or the CLI. Group multiple tests into a suite to execute user journeys end-to-end.
Generate memory and context
Each run passes through a memory curator that records product behavior, test intent, and suite context, then creates contextual memory for future test runs.
Improve the next run
Future executions use the evolved agent context. That memory helps the agent adapt to product changes at runtime, recover from brittle UI drift, and become more reliable over time.
Grow a product-aware QA agent
The end state is a QA agent with enough context about your product to author tests, run them, inspect failures, and keep improving the testing loop.