MCP

Use agent-qa MCP tools to discover project context, author tests and suites, enqueue runs, inspect artifacts, and classify failures from an AI agent.

The agent-qa MCP server exposes source-backed tools for AI coding agents. Start the stdio server from the CLI:

>agent-qa mcp

The CLI attempts to load agent-qa config so analytics can use workspace settings. If config loading fails, the MCP startup path can still continue and start the server without loaded analytics config.

Choose a transport

Use the dashboard-backed HTTP endpoint for day-to-day agent work. It starts with the dashboard, binds to loopback by default, and pre-wires dashboard context for run, artifact, queue, and authoring tools.

>agent-qa dashboard --port 3470 --open

With the default service config, the MCP endpoint is:

http://127.0.0.1:3471/mcp

Use stdio when your client only supports local subprocess MCP servers, when you want a lightweight schema/discovery surface, or when you are integrating agent-qa into a client config that launches tools on demand. Stdio clients must pass dashboardUrl to dashboard-backed tools.

When the stdio server connects, agent-qa writes a startup diagnostic to stderr: agent-qa MCP server running over stdio. Waiting for MCP client messages on stdin. Stdout is reserved for MCP protocol traffic. Keep stdout reserved for MCP protocol messages; route human-readable startup logs and client diagnostics through stderr or the client UI.

services:
  mcp:
    enabled: true
    transport: http
    host: 127.0.0.1
    port: 3471
    path: /mcp

Dashboard URL requirement

Some MCP tools are local schema or discovery tools. Test, suite, hook, run, artifact, and triage tools are dashboard-backed and need a dashboard API base URL. Provide dashboardUrl to the tool call, or start MCP through a dashboard-backed path that supplies the dashboard URL.

The server error is explicit: dashboardUrl is required for authoring tools. Start MCP via agent-qa dashboard or pass dashboardUrl.

Use the Dashboard when an agent needs live run state, file-manager APIs, artifacts, logs, or queue cancellation.

Client setup

Install the agent-qa package in the project first, or make sure the agent-qa binary is available on your PATH.

Codex CLI

Use the dashboard HTTP endpoint when the dashboard is running:

codex mcp add agent-qa --url http://127.0.0.1:3471/mcp
codex mcp list

Use stdio when Codex should launch the MCP server itself:

codex mcp add agent-qa -- agent-qa mcp
codex mcp list

Claude Code

Use the dashboard HTTP endpoint for the full dashboard-backed tool surface:

claude mcp add --transport http agent-qa http://127.0.0.1:3471/mcp
claude
/mcp

Use stdio when Claude Code should launch agent-qa mcp as a local subprocess:

claude mcp add agent-qa -- agent-qa mcp
claude
/mcp

OpenCode

OpenCode can add MCP servers interactively:

opencode mcp add
opencode mcp list

For a project-local dashboard HTTP endpoint, add this to opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "agent-qa": {
      "type": "remote",
      "url": "http://127.0.0.1:3471/mcp",
      "enabled": true
    }
  }
}

For stdio, use a local MCP server entry:

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "agent-qa": {
      "type": "local",
      "command": ["agent-qa", "mcp"],
      "enabled": true
    }
  }
}

Other MCP clients

For clients that accept a Streamable HTTP URL, point them at http://127.0.0.1:3471/mcp after starting the dashboard. For clients that only accept stdio servers, configure command agent-qa with args ["mcp"].

If your client uses an mcpServers object, the stdio shape is usually:

{
  "mcpServers": {
    "agent-qa": {
      "command": "agent-qa",
      "args": ["mcp"]
    }
  }
}

Always check the client docs for the exact HTTP field name. Some clients use url, some use serverUrl, and some require a helper process for remote endpoints.

Verify the connection

After registering the server, ask your agent to call agent_qa_discover. A healthy response should include tool names, schema resources, prompt metadata, endpoint metadata, and config path metadata.

Then run a narrow dashboard-backed check:

Start agent-qa dashboard --port 3470.
Call agent_qa_get_config.
If using stdio, pass dashboardUrl: "http://localhost:3470" when calling authoring, run, artifact, or triage tools.
If using the dashboard HTTP MCP endpoint, the MCP server supplies dashboard context automatically.

Discovery, config, schema, and IDs

These tools do not require dashboard file APIs:

agent_qa_discover: returns MCP tools, resources, prompts, endpoint, dashboard, and config path metadata.
agent_qa_get_config: reads active config and returns masked raw config or a targets/devices/providers summary.
agent_qa_schema_reference: returns structured references for config, test YAML, suite YAML, hooks, or canonical IDs.
agent_qa_validate_definition: validates parsed config, test, suite, or hooks objects with agent-qa schemas.
agent_qa_generate_id: generates canonical IDs for test, suite, hook, run, or observation entities.
agent_qa_validate_id: validates canonical IDs and returns the expected id contract.

Test, suite, and hook tools

Use MCP authoring tools before editing YAML directly. They route through the dashboard workspace-safe APIs and still require dashboardUrl.

Tests:

agent_qa_list_tests
agent_qa_read_test
agent_qa_validate_test
agent_qa_create_test
agent_qa_update_test
agent_qa_delete_test

Suites:

agent_qa_list_suites
agent_qa_read_suite
agent_qa_validate_suite
agent_qa_create_suite
agent_qa_update_suite
agent_qa_delete_suite

Hooks:

agent_qa_list_hooks
agent_qa_read_hook
agent_qa_create_hook
agent_qa_update_hook
agent_qa_delete_hook
agent_qa_run_hook

Run and triage tools

Use these dashboard-backed tools to enqueue work, inspect run evidence, and classify failures:

agent_qa_enqueue_test_run: queues a test run through /api/runs/trigger.
agent_qa_enqueue_suite_run: queues a suite run through /api/runs/trigger.
agent_qa_get_run: returns run detail, steps, attempts, and suite child context.
agent_qa_get_run_steps: returns run steps.
agent_qa_get_run_logs: returns run log rows with optional step, level, source, limit, and offset filters.
agent_qa_get_run_execution_logs: returns structured execution logs.
agent_qa_get_run_artifact: returns sanitized run artifacts, child artifacts, and missing artifact sections.
agent_qa_cancel_run: cancels a pending or running run through the dashboard queue.
agent_qa_classify_failure: classifies a failed run using run detail, artifacts, logs, execution logs, and recent related runs.

Resources and prompt

Schema resources are available at:

agent-qa://schema/{schema}

Current schema names are config, test, suite, hooks, and ids.

The MCP prompt agent_qa_authoring_context gives agents the expected authoring sequence: inspect config, generate canonical IDs, validate definitions, and prefer MCP tools before file edits.

Practical workflow

Call agent_qa_discover.
Call agent_qa_get_config to inspect targets, devices, providers, and dashboard settings.
Use agent_qa_generate_id for new IDs.
Validate definitions with agent_qa_validate_definition, agent_qa_validate_test, or agent_qa_validate_suite.
Create or update through MCP authoring tools when dashboardUrl is available.
Enqueue a run with agent_qa_enqueue_test_run or agent_qa_enqueue_suite_run.
Inspect evidence with run, logs, execution logs, and artifact tools.
Classify failures with agent_qa_classify_failure before deciding whether to edit tests, hooks, app code, or infrastructure.

Security notes

Keep local MCP endpoints on loopback hosts: 127.0.0.1, localhost, or ::1. Treat MCP access as workspace access: an enabled agent can read config, inspect artifacts, enqueue runs, mutate tests and suites through dashboard APIs, and classify failures using local evidence.

Review any client-level auto-approval settings before enabling mutation tools. For shared projects, prefer project-local client config so each repository declares the exact agent-qa endpoint it expects.

On this page