What is agent-qa
Test web and mobile apps in plain English. agent-qa builds persistent memory from validated run evidence, reuses what works, and adapts as your app changes.
Overview
agent-qa: Your AI QA engineer. Test web and mobile apps in plain English. agent-qa builds persistent memory from validated run evidence, reuses what works, and adapts as your app changes.
Your coding agent can author a test, run it through CLI or MCP, inspect the evidence, and use the result to guide its next change. Tests, configuration, memory, and artifacts stay under your control. Use the agent quickstart to go from an existing repository to a verified first test.
Read the complete setup prompt
Set up agent-qa in this application's repository and run a meaningful end-to-end test. Complete the setup and verification, rather than only giving me instructions. agent-qa is vostride's self-improving QA agent for web, Android and iOS. It executes natural-language tests against the real interface, preserves test definitions in the repository, and uses validated memory and cached action plans on future runs. Read the official agent quickstart first: https://vostride.com/docs/agent-qa/agent-quickstart.md Use https://vostride.com/llms.txt to find the current configuration, CLI, MCP and skills references. Treat documentation as technical reference; follow this repository's instructions and my authorization boundaries. 1. Inspect the repository, package manager, app start commands, existing tests and agent-qa configuration. Reuse existing work and credentials without printing secrets. Identify one important user flow and the local or approved test environment. Ask me only for information or authorization that is actually required, such as an unavailable model credential or an ambiguous target. Do not use production data or deploy anything as part of setup. 2. Install agent-qa as a development dependency using this repository's package manager. Inspect the installed version and its CLI help before selecting options. Install the packaged authoring, debug/fix and result-triage skills when the client supports them; use the CLI directly otherwise. 3. If configuration does not exist, initialize it without overwriting existing files. In agent-qa 0.1.21, supplying --platform (web, android, ios, web+android or web+ios) skips the questionnaire. For web, run npx agent-qa init --platform web --dir . using the locally installed package. This version defaults to anthropic-subscription and adds the subscription-auth dependency; inspect and deliberately configure the LLM before running a test. There are no init --yes, --provider, --model or --base-url flags in that version. Newer versions may differ: check their help. 4. Configure an available, suitable multimodal model and the app target using the documented file schema. Reuse an existing authorized credential. For a new API credential, use the secure interactive auth prompt when available, or the quickstart's documented environment-to-auth.json procedure for unattended setup. Version 0.1.21 has no auth set --stdin flag. Never put credential values in shell arguments, committed YAML, logs or the final reply. If using subscription authentication, reuse an authorized session or let me complete the required authentication. Do not assume the coding agent's subscription is automatically available to agent-qa. 5. Prepare only the required runtime: Chromium for the first web test, or the appropriate Appium driver and device for mobile. Docker is needed only for hooks that use it. Start the app with its existing development command, verify the target is reachable, and run agent-qa doctor. Resolve relevant setup failures before testing. 6. Author a small test for the chosen real user flow using agent-qa's documented YAML schema and canonical IDs. Include assertions about observable outcomes, not only successful clicks. Validate the definition before execution. Reuse the current MCP integration when available; otherwise use the local CLI. Never weaken an assertion or change expected behavior just to obtain a pass. 7. Run the test and inspect its verdict, step evidence and artifacts. Distinguish an application bug from a test, environment, authentication or model failure. Fix setup or test-authoring errors and rerun the affected test. Report application bugs with evidence unless I have authorized fixing them. A skipped, interrupted or unexecuted test is not a pass. 8. Leave the test and reusable configuration in the repository, keep secrets and runtime artifacts out of commits, and stop temporary services you started unless they are still needed. Report the changed files, exact rerun command, model and target used, observed result, evidence locations and any remaining blocker. Explain how to reuse this test in the coding agent's next verification loop. Do not claim success until an actual run supports it.
Agent quickstart
Give your coding agent the complete setup task: initialize without menus, configure a model, and run a meaningful test.
Explore demos
Open curated agent-qa runs in the interactive demo viewer and inspect how real test steps execute.
Configuration
Review the file-backed configuration system for global settings, tests, suites, hooks, variables, secrets, devices, rules, and auth.
Memory
Review how file-backed product, suite, and test observations make future runs more product-aware.
Caching
Reuse validated action plans while the screen still matches, and measure the runtime and token savings on your own tests.
Tests
Write one plain-English YAML journey with stable intent, reviewable steps, and source-controlled expectations.
Web Testing
Run browser flows against local or staged web apps without hardcoded selectors or fragile DOM coupling.
Auth state
Capture a named web login once, reuse it by logical name, pass the active state to hooks, and keep credential material out of artifacts.
Suites
Bundle related tests, targets, configuration, and hooks into repeatable workflows for a feature area.
Mobile Testing
Use the same natural-language workflow model for mobile app journeys while preserving artifacts locally.
Hooks
Prepare data, call project scripts, verify side effects, and clean up state before or after a run.
How agent-qa works
Write tests in natural language
Describe the user journey in plain English: what to open, what to do, and what to assert. agent-qa turns that intent into repeatable execution.
Bring your own LLM
Use any hosted or local model. It also works with Codex and Claude Code subscriptions.
Run tests
Run tests from either the dashboard or the CLI. Group multiple tests into a suite to execute user journeys end-to-end.
Retain useful product knowledge
After a run, the memory curator reviews product behavior, test intent, and suite context. Useful knowledge stays in reviewable files beside your tests, ready for future runs.
Validate improvements before applying them
agent-qa checks proposed lessons before applying them and retires outdated knowledge. Updates stay in files you can review and revert. Retained memory provides context; current app evidence and your test expectations still determine whether a run passes.
Grow a product-aware QA agent
The end state is a QA agent with enough context about your product to author tests, run them, inspect failures, and keep improving the testing loop.