AI test automation that leaves you no script to maintain
Most AI testing means a model wrote a script once and you maintain that script forever. agent-qa generates no script at any point. It works the flow out from what is on screen at each run, and what it proves along the way is committed beside your tests as records you can read.
Generating a script is not the same as testing
The common shape of AI testing is generation. A model reads your app, emits Playwright or Selenium code, and hands it over. The intelligence was present for one afternoon and is gone by the first run. What you own afterwards is an ordinary script with ordinary selectors, and the maintenance bill arrives on the ordinary schedule.
agent-qa generates nothing. There is no compile step and nothing to check in but the file below. At each run the agent reads the current screen, decides the next action, and hands that one action to an execution kernel to perform. The kernels underneath, Playwright and Appium, are never given a program to replay.
test-id: t_checkout-smoke
name: Guest checkout completes
target: storefront-web
context: |
The storefront runs at the URL configured by the storefront-web target.
Product data is seeded by the workspace setup hook.
steps:
- Open the home page and search for "espresso grinder".
- Open the first search result.
- Add the product to the cart.
- Start checkout as a guest.
- Fill the shipping form with the workspace test address.
- Place the order.
- Verify the confirmation page shows an order number.A passing run is evidence, not a green tick
Most harnesses record a pass and discard everything that produced it. agent-qa keeps it: the elements that resolved, the flow that worked end to end, the timings that are real. That is the material a suite can actually compound on.
The curator distills that evidence into durable records. Facts about your product, procedures that replay a proved flow, cautions about paths that misbehave, measurements taken from real runs. Each one lands as a reviewable bundle in a committed directory beside your tests, carrying the evidence that justified it. A record arrives as a diff in a pull request, not as a row in a database you cannot open.
The model proposes and deterministic code applies. agent-qa never edits its own prompts and never edits its own code. The only thing that improves is knowledge, and knowledge is a file you can read, argue with, and revert.
Nothing new is trusted on day one
A record is born a candidate, and a candidate steers nothing. It is first replayed offline against the screens it came from, where it has to fix the decision it targets and flip nothing else. Then it has to earn attributed evidence from live runs: a later run that actually exercised it and did better for it. Only then does it go active and reach the next run.
Failures are classified before anything is learned from them. A flake, a broken environment, a config error: no record is minted and no trust moves, and the test is flagged for a human instead. Trust moves only on failures attributed to wrong knowledge. That is what stops a bad afternoon from becoming a bad habit. Nor does it grade its own homework: the judge on replay is a separate model where one exists, and a deterministic diff where none does.
The payoff is measurable. On a matched rerun a validated action is reused while the screen still matches, so the run spends its time acting rather than deciding: roughly 5x faster, 42s down to 8s, on about 60% fewer tokens. The moment the screen stops matching, the step is worked out again. This is reuse, never blind replay.
# initialize a workspace
npx agent-qa init
# run a test
npx agent-qa run tests/checkout-smoke.yaml
# inspect runs in the local dashboard
agent-qa dashboard --port 3470 --openFrequently asked questions
What is AI test automation?
AI test automation puts a model in charge of how a test is carried out instead of encoding that in advance. What separates the tools is when the model runs. Most run it once, at authoring time, and hand you a script to own. agent-qa runs it at execution time, so each step is worked out against the screen in front of it and no generated code sits between the test and the app.
Does agent-qa generate test code I have to maintain?
No. Nothing is compiled and no script is emitted. The only artifact you keep is the plain-English test file. The execution kernel underneath is handed one decided action at a time, which is why there are no selectors in your repository waiting to go stale.
How does agent-qa avoid learning the wrong thing?
Everything it learns starts as a candidate, and a candidate steers no run. It has to survive an offline replay against the screens it came from, fixing the decision it targets and flipping nothing else, then earn attributed evidence from live runs before it is promoted. Flakes, broken environments and config errors are diagnosed and set aside rather than learned from, and the model that made a decision is never the one that judges it.
Can agent-qa rewrite its own prompts or code?
No, and that boundary is deliberate. The curator emits typed proposals, deterministic code applies them, and the only thing that changes is knowledge. Those records are committed files beside your tests, so an improvement you disagree with is a diff you can review and revert like any other.
If nothing is scripted, why do repeat runs get faster?
Because a validated action is kept and reused the next time the same screen appears. On a matched rerun that is roughly 5x faster, 42 seconds down to 8, on about 60% fewer tokens, since a reused action never reaches the model. As soon as the screen stops matching, the step is worked out from scratch again.