End-to-end tests your coding agent can write, run, and read back
A coding agent changes more code in an hour than a person reads in an afternoon, so the slow step is no longer writing the feature. It is checking it. agent-qa hands that check to the agent: packaged Skills that teach it to author, debug, and triage a test, and an MCP server that gives it the calls to validate the file, queue the run, and read the artifacts back.
The bottleneck moved to verification
Writing code got cheap. Reading it did not. A branch arrives with twelve files touched, the diff looks reasonable, and nobody has exercised the feature. Approving it is a guess, and reviewers guess faster than they verify, which is how a plausible diff becomes a broken checkout.
Unit tests do not settle the question. They pass on code that compiles and returns the right shape while the third screen of the flow is dead. What settles it is a run against the real product, and it has to happen while the agent still has the change in hand, not in a nightly job three merges later.
An agent cannot click through your QA tool
Most testing products are a web app. Sign in, press record, drive a browser, save the test in the vendor's project. A coding agent has none of that: no session in someone else's dashboard, no pointer, and no way to sit through a recording flow. A QA product whose only surface is a UI is invisible to the thing now writing most of the code.
agent-qa's surface is a package and a server instead. npx skills add installs the Skills into Claude Code, Codex, or OpenCode: one for authoring tests, suites, and hooks, one for debugging a failed run, one for evidence-backed triage. agent-qa mcp starts the MCP server, and that is where the calls live. The agent generates a canonical ID, validates the definition before it saves anything, queues a test or suite run, pulls back steps, logs, and artifacts, and asks for a failure classification.
# expose agent-qa to coding agents over MCP
agent-qa mcp
# install packaged Skills for Claude Code and peers
npx skills add vostride/agent-qa --skill '*'It already has the context, and it hands back a diff
A recorder watching a browser learns whatever the person driving it happened to click. Your agent holds more than that: the PRD that says what the banner should read, the route in the code, the support ticket about the promo code customers keep hitting. It writes the test out of context it already has, so the case is one a real customer ran into and the expected result is a product decision rather than the agent's opinion of one.
The output is a file. Same schema a product manager writes by hand, same tests directory, same pull request. What a run learns about your app is committed too, in a visible directory beside the tests rather than a database you cannot read, so a new fact arrives as a diff carrying the evidence that justified it. Whoever typed it, a human reviews it the same way.
test-id: t_checkout-smoke
name: Guest checkout completes
target: storefront-web
context: |
The storefront runs at the URL configured by the storefront-web target.
Product data is seeded by the workspace setup hook.
steps:
- Open the home page and search for "espresso grinder".
- Open the first search result.
- Add the product to the cart.
- Start checkout as a guest.
- Fill the shipping form with the workspace test address.
- Place the order.
- Verify the confirmation page shows an order number.Frequently asked questions
How does a coding agent run an end-to-end test with agent-qa?
Through MCP, locally. Run agent-qa mcp for a stdio server the client launches itself, or start the dashboard and point the client at its MCP endpoint on loopback. Claude Code, Codex CLI, and OpenCode each register it in one line. The agent then calls the tools directly: read the config, generate an ID, validate the test, enqueue the run, and fetch the artifacts when it finishes.
What are agent-qa Skills, and which ones ship?
Skills are packaged instructions an agent loads the way it loads any other skill. Three ship today: agent-qa-authoring for creating and validating tests, suites, and hooks, agent-qa-debug-fix for patching after a failed run, and agent-qa-result-triage for evidence-backed classification. Install them per project or globally with npx skills add vostride/agent-qa.
Can the agent tell whether its own change caused the failure?
That is what the classifier is for. It reads run detail, steps, artifacts, logs, and execution logs, then returns a category with its confidence, the evidence behind it, the likely fix area, and a next action. The categories are fixed, from element_not_found and assertion_failure to hook_failure and infrastructure. The triage Skill treats the result as a hypothesis and checks the local files before anything is patched.
What stops an agent from writing a test that passes without checking anything?
Review, which is why the output is a file and not a row in a vendor's database. Validation catches a malformed definition before it is saved. A weak assertion is caught by a person reading the YAML diff in the pull request, and reading it takes no framework knowledge, because the step says 'Verify the confirmation page shows an order number'.
What does giving an agent MCP access to agent-qa actually grant?
Workspace access, so treat it as such. An enabled agent can read config, inspect artifacts, enqueue runs, mutate tests and suites through the dashboard APIs, and classify failures from local evidence. Keep the endpoint on a loopback host, prefer project-local client config so each repository declares the endpoint it expects, and review your client's auto-approval settings before turning the mutation tools on.