Mark a run

When agent-qa gets a verdict wrong, tell it. A mark is one command, and the correction applies immediately. There is no inbox to approve and nothing to sync.

npx agent-qa runs mark <runId> --kind false-fail --step 3 --note "The banner is optional on this page"

A recent agent-qa run got its verdict wrong. Find it, confirm it from the evidence, and mark it so agent-qa corrects itself. Read https://vostride.com/docs/agent-qa/self-improvement/feedback.md first.

1. Find the run with agent_qa_list_runs or npx agent-qa runs list, and the step where the verdict went wrong with agent_qa_get_run_overview.
2. Read that step's evidence and confirm the verdict really is wrong: false-fail if the app was right, false-pass if the app was wrong, confirmed-bug if the failure is a real app bug.
3. Preview the mark with agent_qa_preview_run_mark and tell me what it will change.
4. Once I agree, apply it with agent_qa_mark_run, or npx agent-qa runs mark <runId> --kind <kind> --step <n> --note "<why>".

Report the run ID, the step, the kind, and what the mark changed.

The three kindsCopy Anchor Link

OptionsCopy Anchor Link

The command exits 0 when the mark is recorded, including when the same mark was already there, and 1 otherwise.

What happensCopy Anchor Link

1. Repeat check. The same kind with the same corrections on the same step returns the earlier receipt. A different mark on that step replaces the earlier one.

2. Diagnosis. If the test's root cause was a flaky failure, a timeout or the environment, nothing is learned from the mark. The receipt says quarantined with the reason. The test itself is not quarantined.

3. Example kept. The marked decision is saved as a labelled example in agent-qa-eval/, which is created the first time you mark a run. See Eval corpus.

4. Evidence. Every record the marked step was given gets a confirmation from you. A false-pass also cancels the positive evidence this run gave to any record.

5. The fix. agent-qa picks one fix from what the step was given, by fixed rules:

   Situation	Fix
   False-fail on a step that was given a learned fact	A new fact from you supersedes it.
   False-pass on a step that was given any learned record	A caution from you is added.
   No record, but the test used a procedure at or before this step	A navigation hint is added.
   No record, and the step's own check looks wrong	The test file is edited to match.
   Anything else	Nothing changes, and the receipt lists what was checked.

   A confirmed-bug mark changes nothing. It confirms the records the step used, because they described the app correctly.

6. Apply. The fix is written at once and promoted with your authority, the strongest path there is.

The fix text is fixed wording plus your note. When your model is available, one short call may reword it, never changing what it says. --corrected-expectation is kept with the mark and its example; it does not change the fix text.

ReceiptsCopy Anchor Link

Every mark returns a receipt with one of these outcomes:

Marks are kept per run in .agent-qa/runs/, on your machine.

Before and afterCopy Anchor Link

To see what a mark would do without writing anything, preview it:

npx agent-qa fn runs.mark.preview --input '{"runId":"<runId>","kind":"false-fail","stepIndex":3}'

agent-qa fn runs.marks.history --input '{"runId":"<runId>"}' lists every mark on a run and what each changed. Your coding agent has the same three operations as MCP tools: agent_qa_mark_run, agent_qa_preview_run_mark and agent_qa_get_run_marks.

Changing your mindCopy Anchor Link

A later mark on the same step replaces the earlier mark, but it does not undo the fix the earlier mark applied. To undo a fix, revert it in git, or reject or forget the record it created with agent_qa_change_knowledge.

In the dashboardCopy Anchor Link

Open a run and use the feedback panel. It shows a preview of the fix before you send it.