agent-qa

Register sandboxed hooks in hooks.yaml and implement scripts that prepare web test data, verify API side effects, and export runtime variables.

Hooks are project scripts that agent-qa runs in Docker. They are useful for preparing data, calling product APIs, verifying web UI side effects, and cleaning up records that a browser test created.

There are two parts:

  1. The hook config file, usually hooks.yaml.
  2. The hook implementation file, such as scripts/verify-task.mjs.

Hook config file

workspace.hooksFile in agent-qa.config.yaml points to the hook registry.

workspace:
  hooksFile: hooks.yaml

The registry schema has one root key: hooks.

hooks:
  - id: h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira
    name: Verify web task through API
    runtime: node
    file: scripts/verify-task.mjs
    deps:
      - scripts/api-client.mjs
    timeout: 45s
    network: true

hooks

hooks:
  - id: h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira
    name: Verify web task through API
    runtime: node
    file: scripts/verify-task.mjs
    timeout: 45s

Description: The list of hook definitions available to tests and suites.

Possible values: zero or more hook objects.

Required: yes in the hook registry file.

Default: none.

hooks.id

hooks:
  - id: h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira

Description: Generated stable hook ID.

Possible values: canonical hook IDs that start with h_ and contain 10 id-agent words.

Required: yes.

Default: none. Generate it with agent-qa ids generate hook or the dashboard.

hooks.name

hooks:
  - name: Verify web task through API

Description: Unique human-readable hook name.

Possible values: any non-empty string.

Required: yes.

Default: none.

hooks.runtime

hooks:
  - runtime: node

Description: Runtime used inside the Docker hook runner.

Possible values: node, bun, python, or bash.

Required: yes.

Default: none.

hooks.file

hooks:
  - file: scripts/verify-task.mjs

Description: Hook entry file. agent-qa copies this file into the sandbox workspace before execution.

Possible values: non-empty path string.

Required: yes.

Default: none.

hooks.deps

hooks:
  - file: scripts/verify-task.mjs
    deps:
      - scripts/api-client.mjs

Description: Additional files copied beside the hook entry file.

Possible values: zero or more path strings.

Required: no.

Default: empty list.

hooks.packageFile

hooks:
  - file: scripts/verify-task.mjs
    packageFile: package.json

Description: Optional package file copied beside the hook entry file.

Possible values: path string.

Required: no.

Default: not set.

hooks.timeout

hooks:
  - timeout: 45s

Description: Hook execution timeout.

Possible values: duration strings such as 30s, 2m, and 1h.

Required: yes.

Default: none.

hooks.network

hooks:
  - network: true

Description: Allows network access in the hook container. Set this to true for API verification hooks that call your local or staging web API.

Possible values: true or false.

Required: no.

Default: true.

Referencing hooks

Use setup and teardown arrays when a hook should run before or after a test or suite:

setup:
  - h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira

Use inline runHook syntax when a hook should run at a specific point in the step list:

steps:
  - Run the API verification hook {{runHook:"h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira"}}.
  - Verify TASK_FOUND is exactly "true"; the current value is "{{env:TASK_FOUND}}".

Inline hooks run before variable interpolation for that step. Variables they export are available to later steps.

Hook implementation

Hook scripts receive environment variables from .env, CLI --var, previously successful hooks, and secrets from the configured secrets file. Secrets are available as environment variables inside the hook container.

To return variables to agent-qa, write a dotenv file to /tmp/agent-qa.env. agent-qa reads that file after the container exits and merges the variables into the run.

// scripts/verify-task.mjs
import { writeFile } from "node:fs/promises"

function requiredEnv(name) {
  const value = process.env[name]
  if (!value || !value.trim()) {
    throw new Error(`Missing required environment variable: ${name}`)
  }
  return value.trim()
}

function escapeEnv(value) {
  return String(value ?? "")
    .replace(/\r?\n/g, " ")
    .replace(/\\/g, "\\\\")
    .replace(/"/g, '\\"')
}

async function writeHookEnv(values) {
  const body = Object.entries(values)
    .map(([key, value]) => `${key}="${escapeEnv(value)}"`)
    .concat("")
    .join("\n")

  await writeFile("/tmp/agent-qa.env", body, "utf-8")
}

const apiUrl = requiredEnv("TASK_API_URL")
const apiKey = requiredEnv("TASK_API_KEY")
const title = requiredEnv("TASK_TITLE")

const response = await fetch(`${apiUrl}/tasks/search`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${apiKey}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ title }),
})

if (!response.ok) {
  throw new Error(`Task lookup failed with HTTP ${response.status}`)
}

const payload = await response.json()
const task = payload.tasks?.find((item) => item.title === title)

if (!task) {
  throw new Error(`Task "${title}" was not found`)
}

await writeHookEnv({
  TASK_FOUND: "true",
  TASK_ID: task.id,
  TASK_STATUS: task.status,
})

console.log(`Verified task ${task.id}: ${task.title}`)

/tmp/agent-qa.env

TASK_FOUND="true"
TASK_ID="task_123"
TASK_STATUS="In Progress"

Description: Dotenv file that a hook writes when it needs to export variables back into the run.

Required: only when the hook needs to return variables.

Default: no variables are exported if the file is not written.

Sandbox behavior

agent-qa copies the hook entry file, dependency files, and optional package file into a temporary Docker workspace. The container runs read-only, mounts a writable /tmp, applies CPU and memory limits, and removes the temporary workspace after execution.

node runtime

Docker image: vostride/agent-qa-hook-runner-node.

Use it for JavaScript and TypeScript-adjacent web API checks.

bun runtime

Docker image: vostride/agent-qa-hook-runner-bun.

Use it when your hook relies on Bun-compatible scripts.

python runtime

Docker image: vostride/agent-qa-hook-runner-python.

Use it for Python data setup or API verification scripts.

bash runtime

Docker image: vostride/agent-qa-hook-runner-bash.

Use it for shell scripts that do not need a larger runtime.

Hooks run sequentially. If one setup hook fails, later hooks in that hook group are skipped. A later successful hook can override a variable exported by an earlier hook.

Secrets and output

Known secret values are redacted from hook stdout, stderr, and error text. Variables written to /tmp/agent-qa.env are also filtered when their value exactly equals a known secret. Do not intentionally export secrets from hooks; export derived IDs, status flags, and other non-secret runtime data instead.