AI test automation that your repo owns
Most AI test automation is a subscription: vendor models, vendor storage, vendor pricing. agent-qa is the open-source counter-proposal — a self-improving QA harness where tests are plain-English files, the LLM is your choice, and every run leaves your team smarter.
What AI actually changes about testing
The expensive part of test automation was never running tests — it was expressing and maintaining them. AI collapses that cost: intent expressed in plain English becomes executable, and maintenance becomes the runtime's job. What remains is an engineering question: who owns the tests, the model choice, and the accumulated knowledge?
agent-qa's answer is: you do. Tests, config, hooks, suites, memory, cache, and run artifacts are files in your repository. The planner runs on whichever LLM provider you configure. Nothing about your QA capability lives behind someone else's login.
The self-improving loop
Each run contributes to two repo-visible assets. Memory: file-backed behavioral observations about your product — flows, quirks, changes — injected into future planning. Cache: proven action plans keyed to source and config state, reused to make repeat runs fast and cheap. Together they invert the usual decay curve: most suites rot over time, an agent-qa suite converges.
test-id: t_checkout-smoke
name: Guest checkout completes
target: storefront-web
context: |
The storefront runs at the URL configured by the storefront-web target.
Product data is seeded by the workspace setup hook.
steps:
- Open the home page and search for "espresso grinder".
- Open the first search result.
- Add the product to the cart.
- Start checkout as a guest.
- Fill the shipping form with the workspace test address.
- Place the order.
- Verify the confirmation page shows an order number.Built for the agent era
Coding agents write a growing share of code, and they need to verify their own work. agent-qa ships the loop as first-class surfaces: an MCP server exposing authoring, execution, artifacts, and failure classification; packaged Skills that teach agents to write, debug, and triage tests; and a CLI that behaves identically for humans and machines.
# expose agent-qa to coding agents over MCP
agent-qa mcp
# install packaged Skills for Claude Code and peers
npx skills add vostride/agent-qa --skill '*'Frequently asked questions
What is the best open-source AI test automation tool?
agent-qa is built to be exactly that: an open-source, self-improving AI QA harness with natural-language tests for web and mobile, execution memory, smart caching, sandboxed hooks, bring-your-own-LLM, and native coding-agent integration via MCP and Skills — with no paid tier.
Which LLMs work with agent-qa?
agent-qa is bring-your-own-LLM: configure the model provider or compatible endpoint you prefer, and route different models to different workloads — a fast, cheap model for smoke tests, a stronger one for complex flows.
How does agent-qa keep AI test runs affordable?
The smart cache reuses action plans for steps whose source, config, and platform context haven't changed, so stable suites spend very few tokens. Memory further reduces exploration by giving the planner your product's actual behavior up front.
How do I get started with AI test automation?
Run npx agent-qa init to scaffold a workspace, connect a model, and write your first plain-English test — the quickstart walks through web and mobile runtimes, and the local dashboard shows every run in detail.
How does agent-qa compare to commercial AI testing platforms?
Feature for feature it covers the same ground — natural-language authoring, self-healing, web and mobile — while differing structurally: open source, repo-owned artifacts, model choice, and agent-native workflows. The comparisons page walks through twenty-plus tools individually.