---
title: "Self-improvement"
description: "How agent-qa improves with every run: diagnosis first, lessons checked before use, applied locally and proposed in CI, steered by your feedback and fully reversible."
canonical_url: "https://vostride.com/docs/agent-qa/self-improvement"
md_url: "https://vostride.com/docs/agent-qa/self-improvement.md"
last_updated: "2026-10-24T01:12:08+05:30"
---

> [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt)
> Follow the Markdown links to read individual pages; the indexes list the available documentation.

# Self-improvement

> How agent-qa improves with every run: diagnosis first, lessons checked before use, applied locally and proposed in CI, steered by your feedback and fully reversible.

agent-qa gets better at testing your app with every run. It does not rewrite its own prompts or code. It improves what it knows about your app, what it can replay without asking the model, which tests it can trust, and, when you tell it a verdict was wrong, the knowledge or the test behind that verdict.

## The loop
### Run
A test runs and every step records its evidence. The verdict and the exit code are final before any learning starts.

### Diagnose
Analysis gives every failure a root cause. A flaky failure or a broken environment is treated as such, not as a lesson about your app.

### Learn
What the run proved becomes candidate records in [memidex](https://vostride.com/docs/agent-qa/memory.md): facts, cautions and procedures. The model words them; code decides what is allowed.

### Check
A candidate is promoted only with confirmed evidence from a verified step. With an [eval corpus](https://vostride.com/docs/agent-qa/self-improvement/evals.md), it must also pass a replay check and a live canary check.

### Use
The next run gets the relevant records in its brief. Each later run adds evidence for or against them, so trust rises and falls with what the app actually does.

### Correct
When a verdict is wrong, [mark the run](https://vostride.com/docs/agent-qa/self-improvement/feedback.md). agent-qa fixes the record or the test behind it in the same call.

## What improves
| What | How it improves | Where it lives |
| --- | --- | --- |
| Knowledge about your app | Facts, cautions and procedures learned from runs and marks. | `agent-qa-knowledge/memory/` |
| Speed and cost | Actions that completed a step replay without the model on the same screen. | `.agent-qa/cache/` |
| Trust in your suite | Tests that keep failing for flaky or environmental reasons are [quarantined](https://vostride.com/docs/agent-qa/self-improvement/quarantine.md). | `agent-qa-knowledge/quarantine.jsonl` |
| Your tests | Changed only when a mark points at the test itself, or when you ask agent-qa to write or edit one. | `tests/` |
| New tests it writes | Your best-running tests become examples for the next test agent-qa writes. | Derived on your machine |

## What it never does
- Change a verdict or exit code after the run ends.
- Edit its own prompts or code.
- Edit your [policies](https://vostride.com/docs/agent-qa/memory/policies.md). Only you do that.
- Make git commits. Every change waits for your review.
- Promote anything learned in CI.

## Local runs and CI
Local runs apply what they learn: candidates that pass the gate are promoted at once. CI runs only propose, so a CI job never changes what your team's runs trust. Corrections you make yourself, such as marks, apply in both. [How knowledge is learned](https://vostride.com/docs/agent-qa/memory/learning.md#local-runs-and-ci) has the details.

## You stay in control
| You want to | Do this |
| --- | --- |
| See what changed | Review `git diff -- agent-qa-knowledge`, or run `agent-qa knowledge diff <ref>`. |
| Fix a wrong verdict | `agent-qa runs mark <runId> --kind false-fail`, `false-pass` or `confirmed-bug` |
| Drop a record | Reject or forget it with `agent_qa_change_knowledge` |
| Undo a batch | `agent-qa knowledge rollback --job <jobId>` |
| Release a quarantined test | `agent-qa fn quarantine.release` |

## What it costs
Learning uses your configured model, after the verdict and within a minute:

- Analysis calls the model only for failures the rules cannot classify.
- The curator makes one call per candidate, and a run yields at most 50.
- A mark may make one short call to word its fix.

With an eval corpus, replay and canary checks make more calls. [Eval corpus](https://vostride.com/docs/agent-qa/self-improvement/evals.md) explains when and how many.

## In this section
- [Mark a run](https://vostride.com/docs/agent-qa/self-improvement/feedback.md): Tell agent-qa a verdict was wrong, and what happens next.

- [Self-healing](https://vostride.com/docs/agent-qa/self-improvement/healing.md): How a run recovers from a failed action, and what healed means.

- [Flaky tests and quarantine](https://vostride.com/docs/agent-qa/self-improvement/quarantine.md): How repeat flaky failures stop failing your run.

- [Eval corpus](https://vostride.com/docs/agent-qa/self-improvement/evals.md): Replay and canary checks for teams that measure the agent itself.

## Documentation navigation

- [All documentation](https://vostride.com/docs.md)
- Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md)
- Previous: [Policies](https://vostride.com/docs/agent-qa/memory/policies.md)
- Next: [Mark a run](https://vostride.com/docs/agent-qa/self-improvement/feedback.md)