Coding-agent verification loop
Turn a code change into a tested user journey, an evidence-backed diagnosis, and a focused rerun.
Your coding agent already has the change, the repository and the product context. agent-qa gives it a repeatable way to check the resulting application: a reviewable test contract, live execution, step evidence and application memory. Complete the agent quickstart first if the runtime is not configured.
Verify a change
Start with the behavior the change is meant to deliver. Select an existing test when it covers that behavior; otherwise author the smallest new test that demonstrates it. Include important side effects and permissions when they are part of the requirement. A sequence of successful clicks can still end in the wrong outcome.
Give your agent this task:
Verify my current change with agent-qa. Read the repository instructions and
the diff, identify the affected user behavior, and reuse or extend the smallest
relevant test. Work against the local or approved test environment. Validate
the test definition, execute it, and inspect the completed run and step evidence.
Preserve the expected behavior from the requirements. If the run fails, identify
whether the problem is in the app, test, environment or model. Fix only issues
within the scope I authorized and rerun the affected test. Report the actual
outcome, evidence paths and exact rerun command. Do not call a dry run, queued
job, interrupted run or missing credential a pass.CLI path
Run the project-local CLI through the repository's package manager. Target files explicitly, so an agent verifying one change does not launch every suite:
npx agent-qa validate tests/app-smoke.yaml
npx agent-qa run tests/app-smoke.yaml --headless --junit-output .agent-qa/app-smoke.junit.xmlUse --dry-run only to inspect which tests would execute. It does not drive the app or verify assertions. --junit-output supplies a standard result artifact for CI. Inspect the recorded outcome as well as the process exit status, especially when diagnosing setup or authentication failures.
MCP path
The public https://vostride.com/mcp service searches documentation and supplies the setup prompt. To run tests, connect to the local agent-qa MCP described in the MCP reference. Start the local dashboard-backed services when the tool needs execution state; opening their browser UI is optional.
| Stage | Tools and evidence |
|---|---|
| Inspect the workspace | agent_qa_discover, agent_qa_get_config; reuse configured targets, models and paths. |
| Author and validate | Generate canonical IDs, use the documented create/update tools, and validate the test or suite before running it. |
| Enqueue execution | Use the installed server's execution tool and retain the returned job/run identifier. |
| Wait for completion | Poll that same run while it is pending or running. A polling timeout does not prove the run stopped. |
| Read the result | agent_qa_get_run, agent_qa_get_run_steps, agent_qa_get_run_artifact and logs. |
| Diagnose | agent_qa_classify_failure; verify its hypothesis against artifacts and the app. |
Discover the installed tool schemas rather than inventing parameters or assuming every release has identical tools. A classifier's confidence is supporting information, not proof of root cause.
Diagnose and fix a failure
Investigate the failed agent-qa run I identify. Retrieve its completed run detail,
failed step, screenshots and logs. Find the expected outcome and the earliest
evidence explaining the failure. Classify it as an application, test-authoring,
environment, authentication or model issue, and explain your evidence. If a fix
is within my authorized scope, make the smallest relevant change, validate any
changed test, and rerun it. Preserve the original failure and the new result.
Do not delete assertions, substitute easier expectations or mark a skipped test
as passing. If you cannot reproduce or execute the test, report that limitation.The packaged agent-qa-debug-fix and agent-qa-result-triage skills contain the detailed workflow for these tasks.
Keep verification meaningful
agent-qa separates the execution runtime from the coding agent, but this alone does not enforce independent test authorship. A client granted test-mutation tools can edit tests. Review the test contract against product requirements, protect release acceptance tests where appropriate, and choose the client permissions your workflow requires.
Self-healing finds another route to the same requested outcome. It does not authorize changing that outcome. Memory and caching reuse prior evidence and actions; the run still has to observe the app and check the current result.
Reuse the next run
Keep stable tests, configuration and useful application memory under the repository's normal review process. Preserve original and rerun identifiers in the agent's report. Record the app revision, environment and model so another teammate or agent can reproduce the result.
Use evaluation experiments to measure whether memory and caching help your own workload. Faster execution is useful only when the assertions continue to catch regressions.