Self-healing

Apps change between runs: a button moves, a dialog appears, a list renders twice. agent-qa recovers from most of this inside the step, without you rewriting the test.

In this illustrative demo, a late notice blocks a click, so the agent closes it and runs all twelve tests. Then four tests are disabled, and the agent fails the step instead of running eight.

Two ways to recoverCopy Anchor Link

Finding the element again. When the page re-renders and the element agent-qa meant to use is gone from its old place, it looks at the screen again and finds the same element by its role, its exact name and the text around it, then retries the action once. This needs no model call and is not reported as a heal.

Planning again. When an action fails, or the check rejects what it did, the failure goes back to the model, which plans another way to do the step. If the step then passes, the test is healed. The step's evidence keeps both decisions: the one that failed and the one that recovered.

Recovery never lowers the bar. The step still has to pass its check, and every step stays within its action and planning limits (use.planner.maxActions and use.planner.maxPlanningTurns, 30 each by default).

The healed verdictCopy Anchor Link

A test is healed when every step passed and at least one needed a planned recovery. A run is healed when at least one test healed and none failed.

• A healed run counts as a pass. It exits with code 0, and JUnit reports no failure.
• The terminal marks a healed test with a yellow check.
• npx agent-qa runs list --verdict healed finds healed runs.

A heal is worth a look. It usually means the test's wording or the app changed in a way the next run should not have to rediscover.

What a heal teachesCopy Anchor Link

• The element the recovery actually used becomes a fact candidate, so the next run starts from it.
• The recovering action is cached like any other action that completed a step.
• Each test's history counts its heals alongside its passes and failures.

RetriesCopy Anchor Link

A test can ask to be run again after a failure:

meta:
  retries: 2

Each retry runs the whole test again with a note telling the model that the previous attempt failed, and where. Only the last attempt counts toward the verdict and the JUnit report. A test that passes only after a retry is classified as flaky by analysis, which can lead to quarantine if it keeps happening.

There is no separate healing setting in v0.2. Recovery is part of every step, within its limits.