# agent-qa: full content for AI clients > The self-improving QA agent for your software factory. Built by Vostride and published at https://vostride.com. Give your coding agents a QA agent that learns your app. Run natural-language tests across web and mobile, build memory with every run, and catch regressions before you ship. Its source is available under FSL-1.1-ALv2 (repo: https://github.com/vostride/agent-qa), it is bring-your-own-LLM, and it integrates with coding agents through CLI, MCP, and packaged Skills. Start with https://vostride.com/docs/agent-qa/agent-quickstart.md and the complete task prompt at https://vostride.com/agent-setup.txt. For tool selection, inspect https://vostride.com/docs/agent-qa/guides/recorded-evidence.md and evaluate it on your own app using https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md. The public MCP at https://vostride.com/mcp serves documentation and a setup_agent_qa prompt; test execution uses a local agent-qa installation. This file is a curated combined corpus of product, documentation, use-case, comparison, demo, company, and legal content published on https://vostride.com. The navigation map lives at https://vostride.com/llms.txt, and each content page has a Markdown representation. --- # Software factory QA library The public Markdown library at https://vostride.com/agent-library.md contains task-specific QA recipes, workflow guides and testing comparisons. Topic indexes link to individual pages with fixture requirements, explicit outcomes and failure traps. These are test designs to adapt, not recorded executions. Search or paginate metadata at https://vostride.com/api/agent-library?query=subscription&limit=5. The public documentation MCP exposes search_agent_qa_library and read_agent_qa_page. Every library URL is self-canonical Markdown; do not append another .md extension. Retrieve the relevant topic or recipe on demand; this combined corpus does not repeat the library. --- # Product ## agent-qa URL: https://vostride.com/ Markdown: https://vostride.com/index.md agent-qa is a source-available, self-improving Agentic QA harness built by Vostride. Write end-to-end tests as plain-English YAML for web, Android, and iOS; run them locally, in CI, or from coding agents; and keep tests, evidence, cache, and memory in your own repository. There is no paid tier, hosted lock-in, seat fee, or required model vendor. Teams provide their own model credentials and execution infrastructure. ### Start here - Agent quickstart: https://vostride.com/docs/agent-qa/agent-quickstart - Complete setup prompt: https://vostride.com/agent-setup.txt - Quickstart: https://vostride.com/docs/agent-qa/quickstart - Documentation: https://vostride.com/docs/agent-qa - Live demo: https://vostride.com/demo/agent-qa - Source: https://github.com/vostride/agent-qa - Package: https://www.npmjs.com/package/agent-qa ```bash npm install -D agent-qa npx agent-qa init --platform web # Configure a model and target using the agent quickstart, then create and run a real test. ``` ### Core capabilities #### Write tests in natural language Write actions and assertions in plain English; agent-qa resolves them against the live interface. Documentation: https://vostride.com/docs/agent-qa/guides/first-test #### Stops rediscovering your app Turn successful runs into reviewable, evidence-backed memory that makes every future run faster. Documentation: https://vostride.com/docs/agent-qa/memory #### Test web, Android, and iOS with one flow Run the same natural-language flow across web, Android, and iOS with Playwright or Appium. Documentation: https://vostride.com/docs/agent-qa/guides/mobile-testing #### Built for humans and agents Humans and agents author the same reviewable YAML, backed by your repository, skills, and MCP. Documentation: https://vostride.com/docs/agent-qa/mcp #### Improves itself, one proved run at a time agent-qa diagnoses each run, learns only from valid evidence, and proves every lesson before using it. Documentation: https://vostride.com/docs/agent-qa/memory/curator #### Self-healing test execution When an action fails, agent-qa re-observes the screen and finds another route to the same outcome. Documentation: https://vostride.com/docs/agent-qa/dashboard #### Work it out once, not every run Reuse validated actions while the screen still matches, cutting tokens and runtime without generating scripts. Documentation: https://vostride.com/docs/agent-qa/caching #### Everything it knows is a file in your repository Tests, knowledge, and rules stay as reviewable files shared by teammates, agents, and CI. Documentation: https://vostride.com/docs/agent-qa/configuration #### Run sandboxed hooks during tests Run Node, Bun, Python, or Bash hooks in isolated containers and pass structured results back into the test. Documentation: https://vostride.com/docs/agent-qa/guides/hooks #### Bring your own LLM Switch providers, endpoints, or models in config without rewriting a single test. Documentation: https://vostride.com/docs/agent-qa/configuration/global-config --- # Documentation ## Overview - [What is agent-qa](https://vostride.com/docs/agent-qa.md): Write tests in natural language for web and mobile. agent-qa builds memory with every run, adapts to UI changes, and catches regressions before you ship. - [Agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md): Give your coding agent a complete path from an existing repository to a verified end-to-end test. - [Quickstart](https://vostride.com/docs/agent-qa/quickstart.md): Install agent-qa, prepare web and mobile runtimes, connect an LLM, and inspect your first run from the dashboard or CLI. - [Memory overview](https://vostride.com/docs/agent-qa/memory.md): Understand agent-qa memory as file-backed behavioral observations that make future runs more product-aware without replacing live evidence. - [Caching](https://vostride.com/docs/agent-qa/caching.md): Understand how agent-qa caches action plans, builds cache keys, invalidates stale entries, and can make similar subsequent runs 5x faster with 3x fewer planner tokens. - [Configuration](https://vostride.com/docs/agent-qa/configuration.md): Understand the file-backed configuration system that powers agent-qa projects. - [CLI](https://vostride.com/docs/agent-qa/cli.md): Source-backed reference for the agent-qa command line, grouped by setup, execution, authoring, configuration, maintenance, and AI-native workflows. - [MCP](https://vostride.com/docs/agent-qa/mcp.md): Use agent-qa MCP tools to discover project context, author tests and suites, enqueue runs, inspect artifacts, and classify failures from an AI agent. - [Skills](https://vostride.com/docs/agent-qa/skills.md): Practical guide to packaged agent-qa skills and the CLI command that lists them for AI agents. ## Guides - [Coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md): Turn a code change into a tested user journey, an evidence-backed diagnosis, and a focused rerun. - [Evaluate agent-qa on your app](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md): Measure useful bug detection, repeated-run cost, and agent setup success with a reproducible experiment. - [Recorded test evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md): Inspect the recorded web and Android test results behind the agent-qa demo. - [Write your first test](https://vostride.com/docs/agent-qa/guides/first-test.md): Create a source-grounded agent-qa test file with a target, context, optional hooks, per-test overrides, and natural-language steps. - [Web testing](https://vostride.com/docs/agent-qa/guides/web-testing.md): Configure web targets and browser overrides for agent-qa tests that run through Playwright-managed Chromium, Firefox, or WebKit. - [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md): Capture a named web login once, reuse it across tests and suites, and pass the active Playwright storage-state JSON to hooks without exposing credential material elsewhere. - [Mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md): Configure native Android and iOS targets, local or BrowserStack devices, app installs, and required mobile app-state behavior. - [Suites](https://vostride.com/docs/agent-qa/guides/suites.md): Combine multiple agent-qa tests into one ordered suite with shared target, context, hooks, and run overrides. - [Hooks](https://vostride.com/docs/agent-qa/guides/hooks.md): Write sandboxed agent-qa hooks that prepare data, verify side effects, and export runtime variables back to tests and suites. ## Dashboard - [Dashboard](https://vostride.com/docs/agent-qa/dashboard.md): Source-backed tour of the agent-qa dashboard routes for runs, tests, hooks, suites, memory, insights, config, and queue-aware workflows. ## Memory - [Curator](https://vostride.com/docs/agent-qa/memory/curator.md): Learn how the memory curator turns run evidence into added, confirmed, deprecated, deleted, or unchanged observations. - [Behavioral Memory Schema](https://vostride.com/docs/agent-qa/memory/behavioral-memory-schema.md): Read and review the markdown observation files that back agent-qa behavioral memory. - [Runtime Memory Injection](https://vostride.com/docs/agent-qa/memory/runtime-injection.md): Understand how agent-qa indexes file-backed observations and injects matching memory into individual steps. - [Memory Reliability and Maintenance](https://vostride.com/docs/agent-qa/memory/reliability-maintenance.md): Configure memory trust, injection limits, curator locking, ablation, circuit breaker behavior, security scanning, and stale suite cleanup. ## Configuration - [Global Config](https://vostride.com/docs/agent-qa/configuration/global-config.md): Configure workspace discovery, services, registries, plugins, analytics, and default run behavior in agent-qa.config.yaml. - [Test](https://vostride.com/docs/agent-qa/configuration/test.md): Structure a single natural-language web test definition with target, hooks, variables, steps, and per-test run overrides. - [Suite](https://vostride.com/docs/agent-qa/configuration/suite.md): Group multiple web tests into an ordered agent-qa suite with shared target, context, hooks, and run overrides. - [Hook](https://vostride.com/docs/agent-qa/configuration/hook.md): Register sandboxed hooks in hooks.yaml and implement scripts that prepare web test data, verify API side effects, and export runtime variables. - [Env & Secrets](https://vostride.com/docs/agent-qa/configuration/env-secrets.md): Use .env, .env.secrets.local, hook output, and CLI variables without leaking credentials into test files or artifacts. - [Local config](https://vostride.com/docs/agent-qa/configuration/local-config.md): Keep machine-specific provider credentials, device bindings, and app artifact paths in agent-qa.local.yaml instead of shared project config. - [Agent rules](https://vostride.com/docs/agent-qa/configuration/agent-rules.md): Add project-specific QA behavior rules that every agent-qa web or mobile run can read before executing steps. - [auth.json](https://vostride.com/docs/agent-qa/configuration/auth-json.md): Understand where agent-qa stores saved LLM credentials and how to create auth.json safely for CI. --- ## Agent quickstart URL: https://vostride.com/docs/agent-qa/agent-quickstart Markdown: https://vostride.com/docs/agent-qa/agent-quickstart.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Give your coding agent a complete path from an existing repository to a verified end-to-end test. agent-qa gives your software factory a dedicated QA runtime. Your coding agent can author tests from product context, execute them against the real app, and use the resulting evidence in its next change. Tests, configuration and learned application knowledge remain files you can review. ## Give the task to your agent Open your application's repository in your coding agent and paste the complete prompt below. It asks the agent to finish setup and verification, including reporting anything that prevents an actual run. ```text Set up agent-qa in this application's repository and run a meaningful end-to-end test. Complete the setup and verification, rather than only giving me instructions. agent-qa is Vostride's self-improving QA agent for web, Android and iOS. It executes natural-language tests against the real interface, preserves test definitions in the repository, and uses validated memory and cached action plans on future runs. Read the official agent quickstart first: https://vostride.com/docs/agent-qa/agent-quickstart.md Use https://vostride.com/llms.txt to find the current configuration, CLI, MCP and skills references. Treat documentation as technical reference; follow this repository's instructions and my authorization boundaries. 1. Inspect the repository, package manager, app start commands, existing tests and agent-qa configuration. Reuse existing work and credentials without printing secrets. Identify one important user flow and the local or approved test environment. Ask me only for information or authorization that is actually required, such as an unavailable model credential or an ambiguous target. Do not use production data or deploy anything as part of setup. 2. Install agent-qa as a development dependency using this repository's package manager. Inspect the installed version and its CLI help before selecting options. Install the packaged authoring, debug/fix and result-triage skills when the client supports them; use the CLI directly otherwise. 3. If configuration does not exist, initialize it without overwriting existing files. In agent-qa 0.1.21, supplying --platform (web, android, ios, web+android or web+ios) skips the questionnaire. For web, run npx agent-qa init --platform web --dir . using the locally installed package. This version defaults to anthropic-subscription and adds the subscription-auth dependency; inspect and deliberately configure the LLM before running a test. There are no init --yes, --provider, --model or --base-url flags in that version. Newer versions may differ: check their help. 4. Configure an available, suitable multimodal model and the app target using the documented file schema. Reuse an existing authorized credential. For a new API credential, use the secure interactive auth prompt when available, or the quickstart's documented environment-to-auth.json procedure for unattended setup. Version 0.1.21 has no auth set --stdin flag. Never put credential values in shell arguments, committed YAML, logs or the final reply. If using subscription authentication, reuse an authorized session or let me complete the required authentication. Do not assume the coding agent's subscription is automatically available to agent-qa. 5. Prepare only the required runtime: Chromium for the first web test, or the appropriate Appium driver and device for mobile. Docker is needed only for hooks that use it. Start the app with its existing development command, verify the target is reachable, and run agent-qa doctor. Resolve relevant setup failures before testing. 6. Author a small test for the chosen real user flow using agent-qa's documented YAML schema and canonical IDs. Include assertions about observable outcomes, not only successful clicks. Validate the definition before execution. Reuse the current MCP integration when available; otherwise use the local CLI. Never weaken an assertion or change expected behavior just to obtain a pass. 7. Run the test and inspect its verdict, step evidence and artifacts. Distinguish an application bug from a test, environment, authentication or model failure. Fix setup or test-authoring errors and rerun the affected test. Report application bugs with evidence unless I have authorized fixing them. A skipped, interrupted or unexecuted test is not a pass. 8. Leave the test and reusable configuration in the repository, keep secrets and runtime artifacts out of commits, and stop temporary services you started unless they are still needed. Report the changed files, exact rerun command, model and target used, observed result, evidence locations and any remaining blocker. Explain how to reuse this test in the coding agent's next verification loop. Do not claim success until an actual run supports it. ``` The same prompt is available as [plain text](/agent-setup.txt). The public documentation MCP at `https://vostride.com/mcp` can retrieve it with the `setup_agent_qa` prompt. This hosted service reads documentation; the testing runtime executes in your own environment. ## Choose the shortest setup path | Starting point | Next action | | --- | --- | | agent-qa already works in this repository | Reuse its model, targets and tests. Follow the [coding-agent workflow](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). | | New web project with a model API credential | Follow the unattended setup below. | | Subscription authentication | Reuse an authorized session or complete `auth login` when the configured plugin supports it. A person may need to authorize the session. | | Android or iOS | Use the relevant platform during init, then configure the device, app and app-state behavior in the [mobile guide](https://vostride.com/docs/agent-qa/guides/mobile-testing.md). | An agent needs a shell that can run the app and the test runtime. Interactive terminal support lets it answer arrow-key, checkbox and text prompts, but is unnecessary for initialization when `--platform` is supplied. ## Install and inspect the installed version Use the repository's existing package manager. These examples use npm and a project-local installation: ```bash npm install -D agent-qa npx agent-qa --version npx agent-qa init --help npx agent-qa auth set --help ``` Install the existing [workflow skills](https://vostride.com/docs/agent-qa/skills.md) if your client supports them: ```bash npx skills add vostride/agent-qa --skill agent-qa-authoring --skill agent-qa-debug-fix --skill agent-qa-result-triage --yes ``` Skills teach the workflow; the package supplies the executable. Installing skills alone does not install or configure the testing runtime. ## Initialize without the questionnaire For a new web setup: ```bash npx agent-qa init --platform web --dir . ``` In **0.1.21**, specifying `--platform` skips every init question. Supported values are `web`, `android`, `ios`, `web+android` and `web+ios`. The resulting model configuration defaults to `anthropic-subscription`, and init adds `@vostride/agent-qa-subscription-auth` to `package.json`. It does not authenticate or install that dependency for you. There are no `init --yes`, `--provider`, `--model` or `--base-url` options in this version. `--skip-install` is a deprecated no-op. `--force` overwrites files and is unnecessary for a fresh setup. Reuse existing configuration rather than running init over it. Check the installed version's help when using a newer release. ## Configure the model and application For an API-based setup, edit the generated YAML using the [configuration schema](https://vostride.com/docs/agent-qa/configuration/global-config.md). Merge the following fields into the existing document; retain the generated workspace, services and other settings. Replace the uppercase model and endpoint values with a vision-capable model and endpoint you can actually use: ```yaml registry: llms: - name: qa provider: openai-compatible model: YOUR_VISION_MODEL baseURL: https://YOUR_MODEL_ENDPOINT/v1 targets: app: platform: web url: http://127.0.0.1:3000 use: llm: qa browser: name: chromium headless: true ``` Use the app's actual local port. `anthropic-compatible` and `gemini` are also supported; follow their provider-specific configuration. A text-only model is insufficient for screenshot-based visual testing. If replacing the generated subscription setup with API authentication, remove its unused `plugins.auth` entry and its newly added subscription-auth dependency, preserving any entries used elsewhere. Review the diff before installing dependencies. Keep subscription configuration when that is the provider you intend to use. ## Supply credentials without putting them in a command Existing credentials are indexed by the LLM **config name**, such as `qa`, in the [auth store](https://vostride.com/docs/agent-qa/configuration/auth-json.md). Start with `npx agent-qa auth status` to see whether the selected configuration is already authenticated. For an interactive terminal, this command asks for a masked secret: ```bash npx agent-qa auth set --config qa --type api-key ``` In 0.1.21, `auth set` has no `--stdin` option. Do not pass a secret as its positional argument in an agent-generated shell command. For unattended API setup, make `AGENT_QA_API_KEY` available through your existing secret manager or CI environment, then merge it into the documented store without printing it: ```bash node --input-type=module <<'NODE' import { mkdirSync, readFileSync, writeFileSync, chmodSync } from 'node:fs'; import { homedir } from 'node:os'; import { join } from 'node:path'; const secret = process.env.AGENT_QA_API_KEY; if (!secret?.trim()) throw new Error('AGENT_QA_API_KEY is required'); const directory = process.env.XDG_DATA_HOME ? join(process.env.XDG_DATA_HOME, 'agent-qa') : join(homedir(), '.agent-qa'); const file = join(directory, 'auth.json'); mkdirSync(directory, { recursive: true, mode: 0o700 }); let credentials = {}; try { credentials = JSON.parse(readFileSync(file, 'utf8')); } catch (error) { if (error.code !== 'ENOENT') { throw new Error('Could not read or parse existing auth.json; repair it before continuing'); } } if (!credentials || typeof credentials !== 'object' || Array.isArray(credentials)) { throw new Error('Existing auth.json is not an object'); } credentials.qa = { type: 'api', provider: 'openai-compatible', key: secret.trim() }; writeFileSync(file, JSON.stringify(credentials, null, 2), { mode: 0o600 }); chmodSync(file, 0o600); console.log('Saved the qa credential without displaying its value.'); NODE ``` Match the config name and provider to your YAML. This preserves other named credentials. Use an isolated `XDG_DATA_HOME` in CI when jobs should not share credentials. Subscription credentials require the supported authentication flow; copying the coding agent's own session files is not a setup procedure. ```bash npx agent-qa auth status npx agent-qa auth test --config qa ``` `auth test` makes a real model request. A successful auth check proves model access, not that an application test passed. ## Prepare the runtime and first test Install Chromium for the first web test, start the app using its existing development command, and check the environment: ```bash npx agent-qa install-browsers --chromium npx agent-qa doctor npx agent-qa create-test tests/app-smoke.yaml ``` `create-test` supplies a canonical test ID and refuses to overwrite an existing file. Keep that ID, set `target: app`, and replace the sample steps with a small real user flow. For example, in a task application: create a uniquely named task, verify it appears in the list, open it, and verify the saved details. For a read-only product, choose meaningful navigation and content assertions. Match assertions to the actual product requirements. Docker is needed for sandboxed hooks; a simple test without hooks can start without Docker. Native mobile tests require their Appium driver and a usable device or emulator. Browser installation alone does not prepare mobile testing. ```bash npx agent-qa validate tests/app-smoke.yaml npx agent-qa run tests/app-smoke.yaml --headless --junit-output .agent-qa/app-smoke.junit.xml ``` Validation checks definitions. Only execution against the app supplies test evidence. Inspect the terminal verdict, JUnit result and available step artifacts. In MCP workflows, retrieve run detail, steps and artifacts rather than treating a queued job as a completed test. ## Finish with evidence Report the test's actual outcome, target, model, changed files, evidence paths and exact rerun command. A timeout, missing credential, cancelled job or unexecuted test is a blocker or failure, never a pass. Preserve failures that reveal an application bug; changing the expected result is not a fix. Commit reviewable tests and configuration according to the repository's workflow. Keep credentials and runtime artifacts out of commits. Continue with [verify, diagnose and rerun](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), or [evaluate agent-qa on your app](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md). ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Next: [Quickstart](https://vostride.com/docs/agent-qa/quickstart.md) --- ## Caching URL: https://vostride.com/docs/agent-qa/caching Markdown: https://vostride.com/docs/agent-qa/caching.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Understand how agent-qa caches action plans, builds cache keys, invalidates stale entries, and can make similar subsequent runs 5x faster with 3x fewer planner tokens. agent-qa caches action plans for step sub-actions. A repeated run still observes the live page, checks the current screen state, executes actions, records artifacts, and verifies outcomes. The cache only skips redundant planning work when the same step is running in the same source, config, suite, platform, and step-position context. The default cache directory is `.agent-qa/cache`. ## Why caching speeds up repeated runs Most test runs spend time asking the planner what action should happen next: click this button, fill this field, wait for this state, or assert this visible result. When a previous run already planned a sub-action for the same step context, agent-qa can reuse that cached action plan instead of asking the LLM to plan it again. That means caching helps most when you rerun the same test or suite after a small product change, while the test YAML, suite YAML, config, platform, and step order remain stable. The landing-page cache feature shows the intended impact for similar subsequent runs: `5x` faster execution, from `42s -> 8s`, and `3x` fewer planner tokens. Those gains come from cached action plans skipping redundant planner work when the flow and screen state still match. Caching does not turn a run into a replay script. The agent still re-observes the page before each sub-action, and a cached action can be rejected, invalidated, or replaced when the run no longer matches the cached path. ## Configure cache storage Cache storage is configured in `agent-qa.config.yaml` under `services.cache`: ```yaml services: cache: dir: .agent-qa/cache ttl: 7d ``` `services.cache.dir` chooses where cache files are written. `services.cache.ttl` controls how long entries stay fresh. You can override these from the environment: ```bash AGENT_QA_CACHE_DIR=.agent-qa/cache AGENT_QA_CACHE_TTL=24h agent-qa run tests/login.yaml ``` Per-run and per-test behavior is controlled through `use.cache`: ```yaml use: cache: true ``` Set `use.cache: false` in a test or suite override when that workflow should always plan fresh actions. ## Run without cache Use `--no-cache` when you want a single run to bypass action-cache reads and writes from the normal run path: ```bash agent-qa run tests/login.yaml --no-cache ``` The dashboard uses the same runtime behavior. The run options menu exposes **Use cache**; turning it off sends a no-cache run request and the underlying command runs with `--no-cache`. ## Cache keys agent-qa builds a step hash from the execution context, then stores sub-actions under that hash. The step hash uses the first 16 hex characters of a SHA-256 digest over: - config file content - suite file content - suite test index - test file content - step instruction - platform - step index The current sub-action cache files use this shape: ```txt .agent-qa/cache//sub-.json ``` This is why small source changes can intentionally create a new cache context. Changing the config file, moving a test within a suite, editing the test file, changing the step text, switching platform, or moving the step to a different index can all change the key. ## Hits, misses, and invalidation A cache hit returns the stored action plan for a sub-action index. A cache miss asks the planner for a fresh action plan. Entries are treated as misses when: - the cache file is missing or invalid JSON - `CACHE_SCHEMA_VERSION` does not match the current cache schema - `services.cache.ttl` has expired - a run misses at sub-action index N, which invalidates cached sub-actions from that index forward - a cached action fails and the run needs to replan from that point - a cached action contains an old redacted secret marker such as `[secret:loginPassword]` for a step that uses runtime `{{secret:...}}` templates That prefix invalidation keeps a partially stale path from continuing after the first mismatch. The run can still write fresh cache entries for the actions it replans. ## Purge cached plans Purge cache entries for one test file when you know its action path should be rebuilt: ```bash agent-qa cache purge --test tests/login.yaml ``` Purge all cached plans when you want a clean cache directory: ```bash agent-qa cache purge --all --force ``` Without `--force`, the all-cache purge asks for confirmation. Dashboard pages that run tests can also expose cache purge actions. Run detail cache markers show when a step was fully or partially cached, with labels such as `All actions cached` or `2 of 5 actions cached`. ## Cache, memory, and configuration Cache is execution-level reuse. It stores action plans that let repeated runs avoid redundant planning work. Memory is product context. It stores file-backed observations about products, suites, and tests so future runs can reason with prior evidence. Configuration is runtime control. It defines targets, browsers, devices, services, hooks, variables, auth, and default run behavior. Use cache when repeated runs should be faster. Use memory when the agent should remember product behavior. Use configuration when the team needs reviewable defaults and explicit overrides. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Memory overview](https://vostride.com/docs/agent-qa/memory.md) - Next: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) --- ## CLI URL: https://vostride.com/docs/agent-qa/cli Markdown: https://vostride.com/docs/agent-qa/cli.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Source-backed reference for the agent-qa command line, grouped by setup, execution, authoring, configuration, maintenance, and AI-native workflows. The `agent-qa` CLI is the local control surface for initializing projects, running tests and suites, opening the dashboard, validating files, managing credentials, and exposing AI-native integrations. ## Global options Use these options before the command name. - `--config `: load a config file. The default path is `agent-qa.config.yaml`. - `--log-level `: set verbosity to `silent`, `error`, `warn`, `info`, or `debug`. - `--verbose`: shorthand for `--log-level debug`. - `--quiet`: shorthand for `--log-level silent`. ```bash agent-qa --config ./agent-qa.config.yaml --log-level debug doctor ``` ## Setup ### `init` Purpose: create the initial agent-qa workspace files. Usage: `agent-qa init` Important options: `--dir `, `--platform `, `--skip-install`, `--force`. Example: `agent-qa init --platform web --dir .` In 0.1.21, `--platform` skips the entire questionnaire and defaults the generated model to `anthropic-subscription`. Init also adds the subscription-auth dependency; it does not install or authenticate it. Configure the intended model before running a test. There are no init `--yes`, `--provider`, `--model` or `--base-url` flags in this version. `--skip-install` is a deprecated no-op. See the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) for a complete unattended setup using the existing CLI and file schema. ### `install-browsers` Purpose: install Playwright-managed browser support for web tests. Usage: `agent-qa install-browsers` Important options: `--all`, `--chromium`, `--firefox`, `--webkit`, `--with-deps`, `--force`. Example: `agent-qa install-browsers --chromium` ### `install-mobile-drivers` Purpose: install Appium drivers for Android and iOS test execution. Usage: `agent-qa install-mobile-drivers` Important options: `--all`, `--android`, `--ios`, `--update`, `--unsafe`. Example: `agent-qa install-mobile-drivers --all` ### `doctor` Purpose: validate environment dependencies and local setup. Usage: `agent-qa doctor` Important options: use the global logging and config options when you need source attribution or quieter output. Example: `agent-qa doctor --verbose` ## Running and inspecting ### `run` Purpose: discover and execute test files or suites. Usage: `agent-qa run [patterns...]` Important options: `--browser `, `--platform `, `--headless`, `--no-headless`, `--no-cache`, `--no-memory`, `--bail`, `--dry-run`, `--list-tests`, `--junit-output `, `--screenshot-dir `, `--screenshot-mode `, `--reporter `, `--record`, `--config-debug`, `--test`, `--suite`, `--all`, `--device `, `--var `, `--run-attr `. ```bash agent-qa run tests/login.yaml --browser chromium --reporter console dashboard ``` ### `dashboard` Purpose: start the dashboard web server. Usage: `agent-qa dashboard` Important options: `--port `, `--db `, `--open`. Example: `agent-qa dashboard --port 3470 --open` ### `serve` Purpose: start configured local agent-qa services using the dashboard-backed service starter. Usage: `agent-qa serve` Important options: use global `--config ` when the service configuration lives outside the default path. Example: `agent-qa --config ./agent-qa.config.yaml serve` ### `queue list` Purpose: show pending and running jobs in the dashboard execution queue. Usage: `agent-qa queue list` Important options: `--json`, `--all`, `--server `. Example: `agent-qa queue list --all --server http://localhost:3470` ### `queue cancel` Purpose: cancel a pending or running queue job by run id. Usage: `agent-qa queue cancel ` Important options: `--server `. Example: `agent-qa queue cancel run_123 --server http://localhost:3470` ## Writing and validation ### `validate` Purpose: validate config, test files, and suite references. Usage: `agent-qa validate [files...]` Important options: pass one or more files to validate a focused change; omit files to auto-discover the workspace. Example: `agent-qa validate tests/login.yaml suites/smoke.yaml` ### `create-test` Purpose: scaffold a test YAML file with an auto-generated canonical test id. Usage: `agent-qa create-test ` Important options: the command fails if the output file already exists. Example: `agent-qa create-test tests/new-login.yaml` ### `create-suite` Purpose: scaffold a suite YAML file with an auto-generated canonical suite id. Usage: `agent-qa create-suite ` Important options: the command fails if the output file already exists. Example: `agent-qa create-suite suites/smoke.yaml` ### `ids generate` Purpose: generate a canonical agent-qa entity id with `id-agent`. Usage: `agent-qa ids generate ` Important options: `--json`. Example: `agent-qa ids generate test --json` ### `ids validate` Purpose: validate that an id matches the canonical contract for an entity type. Usage: `agent-qa ids validate ` Important options: `--json`. Example: `agent-qa ids validate test t_example-id --json` ## Configuration and auth ### `config set` Purpose: set a config value. Usage: `agent-qa config set [value]` Important options: use global `--config ` to choose the config file. Example: `agent-qa config set services.dashboard.port 3470` ### `config show` Purpose: show resolved config with source attribution. Usage: `agent-qa config show` Important options: use global `--config ` to inspect a non-default config file. Example: `agent-qa --config ./agent-qa.config.yaml config show` ### `auth login` Purpose: authenticate a named subscription LLM config. Usage: `agent-qa auth login --config ` Important options: required `--config `. Example: `agent-qa auth login --config claude` ### `auth set` Purpose: save an API key or bearer token credential for a named LLM config. Usage: `agent-qa auth set --config --type [secret]` Important options: required `--config ` and `--type ` where type is `api-key` or `bearer-token`. Example: `agent-qa auth set --config openai --type api-key` Omitting the secret opens a masked interactive prompt. In 0.1.21 this command has no `--stdin` option. For unattended authentication, use the [environment-to-auth-store procedure](https://vostride.com/docs/agent-qa/agent-quickstart.md#supply-credentials-without-putting-them-in-a-command), preserving existing entries. Avoid passing credentials in shell arguments. ### `auth status` Purpose: show credential status for configured LLMs. Usage: `agent-qa auth status` Important options: use global `--config ` to inspect credentials for a specific workspace config. Example: `agent-qa auth status` ### `auth logout` Purpose: remove stored credentials for a named config. Usage: `agent-qa auth logout` Important options: `--config `. Example: `agent-qa auth logout --config openai` ### `auth-state capture` Purpose: open a headed browser for a web target, let you sign in manually, then save a named auth state after terminal confirmation. Usage: `agent-qa auth-state capture --target --name ` Example: `agent-qa auth-state capture --target app-staging --name qa-admin` ### `auth test` Purpose: test an LLM connection using named config credentials. Usage: `agent-qa auth test` Important options: `--config `, `--provider `, `--model `. Example: `agent-qa auth test --config openai` ### `devices list` Purpose: show configured devices with the merged shared and local view. Usage: `agent-qa devices list` Important options: use global `--config ` for a non-default workspace config. Example: `agent-qa devices list` ### `devices init` Purpose: scan connected devices and generate `agent-qa.local.yaml`. Usage: `agent-qa devices init` Important options: use global `--config ` to align generated local bindings with a specific workspace. Example: `agent-qa devices init` ## Maintenance ### `cache purge` Purpose: clear cached action plans. Usage: `agent-qa cache purge` Important options: `--test `, `--all`, `--force`. Example: `agent-qa cache purge --all --force` ### `clean-memory` Purpose: remove orphaned memory observation directories. Usage: `agent-qa clean-memory` Important options: `-y`, `--yes`. Example: `agent-qa clean-memory --yes` ## AI-native integrations ### `mcp` Purpose: start the MCP server over stdio for agent workflows. Usage: `agent-qa mcp` Important options: use global `--config ` to load workspace-specific analytics and service settings. Example: `agent-qa --config ./agent-qa.config.yaml mcp` ### `skills` Purpose: list packaged agent-qa skills. Usage: `agent-qa skills` Important options: `--json`. Example: `agent-qa skills --json` ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Next: [MCP](https://vostride.com/docs/agent-qa/mcp.md) --- ## Agent rules URL: https://vostride.com/docs/agent-qa/configuration/agent-rules Markdown: https://vostride.com/docs/agent-qa/configuration/agent-rules.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Add project-specific QA behavior rules that every agent-qa web or mobile run can read before executing steps. Agent rules are project-specific instructions for the QA agent. Use them for product language, stable selectors, boundaries, accessibility expectations, and verification habits that should apply across tests. The global config points to the file: ```yaml workspace: agentRules: ./agent-rules.md ``` ## workspace.agentRules ```yaml workspace: agentRules: ./agent-rules.md ``` Description: Path to the Markdown file containing shared QA rules. Possible values: any non-empty workspace-relative path. Required: yes in `agent-qa.config.yaml`. Default: none. ## Example agent-rules.md ```md # agent-qa rules ## Product language - The product calls work items "tasks", not tickets. - The left navigation item for active work is "Assigned". - The completed status is "Done". ## Web selector preferences - Prefer visible labels and roles before CSS selectors. - Prefer data-testid attributes only when labels are ambiguous. - Do not click destructive actions unless the test step explicitly asks for it. ## Verification style - Verify state in the UI before moving to the next step. - When a step creates a task, confirm the task title and status are visible. - If a hook exports TASK_ID, use it only as supporting context; the UI remains the primary assertion for web flows. ``` ## Product language rules ```md ## Product language - The product calls work items "tasks", not tickets. - The completed status is "Done". ``` Description: Terminology the agent should use when interpreting steps and visible UI. Use this when: the product has domain-specific names or labels that are easy to confuse. Default: no product language rules. ## Selector rules ```md ## Web selector preferences - Prefer visible labels and roles before CSS selectors. - Prefer data-testid attributes only when labels are ambiguous. ``` Description: Guidance for how the agent should interact with web UI. Use this when: your app has stable labels, roles, or test IDs that should guide actions. Default: the agent chooses from observed UI context. ## Boundary rules ```md ## Boundaries - Do not delete production-like data. - Do not change billing settings. - Do not invite real users. ``` Description: Actions the QA agent should avoid unless a test explicitly requests them. Use this when: the app includes destructive or externally visible operations. Default: no project-specific boundaries. ## Verification rules ```md ## Verification style - Verify state in the UI before moving to the next step. - Confirm task title and status after creating or editing a task. ``` Description: Shared expectations for assertions and step completion. Use this when: your team wants consistent evidence quality across tests. Default: no extra verification rules. ## Hook and variable rules ```md ## Hooks and variables - Use TASK_ID from hooks as supporting context, not as the only proof that the UI is correct. - Never expose secret values in screenshots, step text, or console output. ``` Description: Guidance for values produced by hooks, captures, and secret interpolation. Use this when: tests combine browser workflows with API verification hooks. Default: no hook-specific rules. ## Keep rules reviewable Treat `agent-rules.md` like code: - Keep rules specific to observable product behavior. - Avoid long prompt essays that duplicate individual tests. - Review changes in pull requests. - Remove stale labels and selectors when the app changes. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Local config](https://vostride.com/docs/agent-qa/configuration/local-config.md) - Next: [auth.json](https://vostride.com/docs/agent-qa/configuration/auth-json.md) --- ## auth.json URL: https://vostride.com/docs/agent-qa/configuration/auth-json Markdown: https://vostride.com/docs/agent-qa/configuration/auth-json.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Understand where agent-qa stores saved LLM credentials and how to create auth.json safely for CI. `auth.json` stores saved LLM credentials for named configs in `registry.llms`. By default, agent-qa writes it outside the project: ```txt ~/.agent-qa/auth.json ``` If `XDG_DATA_HOME` is set, the path changes to: ```txt $XDG_DATA_HOME/agent-qa/auth.json ``` The file is written with mode `0600` when agent-qa saves credentials. ## Credential keys Entries are keyed by the LLM config name, not by the provider name. ```yaml registry: llms: - name: codex provider: openai-subscription model: gpt-5.5 - name: remote-openai provider: openai-compatible model: gpt-5.1 baseURL: https://api.openai.com/v1 use: llm: codex ``` The matching `auth.json` keys are `codex` and `remote-openai`. ## File shape ```json { "remote-openai": { "type": "api", "provider": "openai-compatible", "key": "sk-project-restricted-token" }, "remote-anthropic": { "type": "bearer", "provider": "anthropic-compatible", "token": "restricted-bearer-token" }, "codex": { "type": "oauth", "provider": "openai-subscription", "tokens": { "access": "access-token", "refresh": "refresh-token", "expires": 1799999999999, "accountId": "acct_123" } } } ``` Description: JSON object keyed by LLM config name. Possible values: `api`, `bearer`, and `oauth` credential objects. Required: only for LLM configs that need saved credentials. Default: no saved credentials. ## type api ```json { "remote-openai": { "type": "api", "provider": "openai-compatible", "key": "sk-project-restricted-token" } } ``` Description: API key credential. Provider constraints: supported for `openai-compatible`, `anthropic-compatible`, and `gemini`. Required fields: `type`, `provider`, and `key`. Default: none. ## type bearer ```json { "remote-anthropic": { "type": "bearer", "provider": "anthropic-compatible", "token": "restricted-bearer-token" } } ``` Description: Bearer token credential. Provider constraints: supported for `anthropic-compatible`. Required fields: `type`, `provider`, and `token`. Default: none. ## type oauth ```json { "codex": { "type": "oauth", "provider": "openai-subscription", "tokens": { "access": "access-token", "refresh": "refresh-token", "expires": 1799999999999, "accountId": "acct_123" } } } ``` Description: OAuth tokens from an auth plugin. Provider constraints: used by subscription auth plugins such as `openai-subscription` and `anthropic-subscription`. Required fields: `type`, `provider`, and `tokens`. Default: none. ## tokens.access ```json { "tokens": { "access": "access-token" } } ``` Description: Access token used by the plugin-backed fetch wrapper. Required: yes for OAuth credentials. Default: none. ## tokens.refresh ```json { "tokens": { "refresh": "refresh-token" } } ``` Description: Refresh token used when the plugin refreshes credentials. Required: yes for OAuth credentials. Default: none. ## tokens.expires ```json { "tokens": { "expires": 1799999999999 } } ``` Description: Expiration timestamp in milliseconds. Required: yes for OAuth credentials. Default: none. ## tokens.accountId ```json { "tokens": { "accountId": "acct_123" } } ``` Description: Optional account identifier. Required: no. Default: not set. ## Managing auth locally Use the CLI or dashboard rather than editing `auth.json` by hand when you are working locally. ```bash npx agent-qa auth set --config remote-openai --type api-key ``` For Anthropic-compatible bearer tokens: ```bash npx agent-qa auth set --config remote-anthropic --type bearer-token ``` For subscription providers, declare the auth plugin in `agent-qa.config.yaml`, install it, then authenticate from the dashboard or use the plugin-backed login flow when supported. ## CI runtime auth For CI, create `auth.json` at runtime from CI secrets. Do not commit it. ```bash mkdir -p "$HOME/.agent-qa" chmod 700 "$HOME/.agent-qa" node -e ' const { writeFileSync } = require("node:fs"); const path = `${process.env.HOME}/.agent-qa/auth.json`; const auth = { "remote-openai": { type: "api", provider: "openai-compatible", key: process.env.AGENT_QA_OPENAI_API_KEY } }; writeFileSync(path, JSON.stringify(auth, null, 2), { mode: 0o600 }); ' npx agent-qa run --suite suites/web-release.suite.yaml ``` When CI uses an alternate data directory, set `XDG_DATA_HOME` and write to `$XDG_DATA_HOME/agent-qa/auth.json`. Use CI secret values with the same discipline as test secrets: restricted scopes, disposable credentials when possible, and rotation-ready setup. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Agent rules](https://vostride.com/docs/agent-qa/configuration/agent-rules.md) --- ## Env & Secrets URL: https://vostride.com/docs/agent-qa/configuration/env-secrets Markdown: https://vostride.com/docs/agent-qa/configuration/env-secrets.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Use .env, .env.secrets.local, hook output, and CLI variables without leaking credentials into test files or artifacts. agent-qa separates non-secret variables from secrets. Keep stable web test data in `.env`, sensitive values in `.env.secrets.local`, and temporary runtime values in hook output. ## Web variable example ```dotenv TASK_API_URL=http://localhost:3000/api TASK_TITLE=Release checklist review TASK_DESCRIPTION=Validate that a release checklist task can be created and completed. STATUS_IN_PROGRESS=In Progress STATUS_DONE=Done TASK_PRIORITY=Urgent ``` ```dotenv TASK_API_KEY=restricted-ci-token LOGIN_EMAIL=qa-user@company.test LOGIN_PASSWORD=temporary-test-password ``` ## workspace.envFile ```yaml workspace: envFile: .env ``` Description: Points agent-qa at a dotenv-style file for non-secret variables. Possible values: any non-empty workspace-relative path. Required: yes in `agent-qa.config.yaml`. Default: none. The referenced file must exist, even if it is empty. Use non-secret variables in test steps with `{{env:NAME}}`: ```yaml steps: - In the task title field, enter exactly "{{env:TASK_TITLE}}". - Set the task status to "{{env:STATUS_IN_PROGRESS}}". ``` If a variable is missing, the step fails before execution with an unresolved template error. ## workspace.secretsFile ```yaml workspace: secretsFile: .env.secrets.local ``` Description: Points agent-qa at a dotenv-style file for sensitive values. Possible values: any non-empty workspace-relative path. Required: yes in `agent-qa.config.yaml`. Default: none. Keep this file out of git. Use secrets with `{{secret:NAME}}` when a value must be inserted into an action at execution time: ```yaml steps: - Fill the email field with "{{secret:LOGIN_EMAIL}}". - Fill the password field with "{{secret:LOGIN_PASSWORD}}". ``` Secrets are also provided to hook containers as environment variables, so hook scripts can read `process.env.TASK_API_KEY`. ## Important security note Secret file contents are not stored as part of the run artifact. agent-qa stores metadata such as the secrets file path, load status, and secret count instead of the raw secret values. However, secrets can still appear indirectly if the application or hook sends them through observable channels. For example, a login API call could include a password or bearer token in captured network logs, request payloads, response bodies, browser console output, screenshots, or hook stdout before redaction can help. Use disposable credentials or credentials with strict restrictions for QA runs. Prefer short-lived accounts, isolated test workspaces, narrow API scopes, and secrets that can be rotated without affecting production users. ## .env variables ```dotenv TASK_TITLE=Release checklist review TASK_PRIORITY=Urgent ``` Description: Non-secret runtime variables loaded before the run. Possible values: dotenv key/value pairs. Default: none. ## CLI --var ```bash agent-qa run --var TASK_PRIORITY=High tests/task-create-and-complete.yaml ``` Description: One-off variable override for a single command. Possible values: `KEY=VALUE` pairs. Default: no CLI variables. ## Suite hook variables ```dotenv WORKSPACE_ID="ws_123" ``` Description: Variables exported by suite setup hooks. They are passed to later suite hooks and tests. Possible values: any key/value written to `/tmp/agent-qa.env` by a successful hook. Default: no variables unless hooks export them. ## Test hook variables ```dotenv TASK_FOUND="true" TASK_ID="task_123" ``` Description: Variables exported by test setup hooks, inline hooks, or teardown hooks. Possible values: any key/value written to `/tmp/agent-qa.env` by a successful hook. Default: no variables unless hooks export them. ## Captured variables ```yaml steps: - step: Copy the task key from the detail header. capture: variable: TASK_KEY method: regex pattern: "TASK-[0-9]+" ``` Description: Runtime values captured from a web page step and made available through `{{env:NAME}}`. Possible values: capture variables produced by `regex`, `selector`, or `ai` capture. Default: no captured variables. ## setVariable values ```yaml steps: - Remember the current release lane as SESSION_LABEL for later steps. ``` Description: Runtime variable set by the agent only when a step explicitly asks it to store a value. Possible values: any non-secret runtime value. Default: none. ## Hook output variables Hooks export variables by writing dotenv content to `/tmp/agent-qa.env`: ```js import { writeFile } from "node:fs/promises" await writeFile( "/tmp/agent-qa.env", [ 'TASK_FOUND="true"', 'TASK_ID="task_123"', "", ].join("\n"), "utf-8", ) ``` After a successful hook, later hooks and steps can read those values with `{{env:TASK_FOUND}}` and `{{env:TASK_ID}}`. ## Secret redaction boundaries agent-qa redacts known secret values from hook stdout, stderr, errors, and run data where the exact value is available to the redactor. Redaction is not a license to use production credentials. If a third-party service echoes a transformed token, a session cookie, or a derived credential into logs, that value may not match the original secret exactly. Keep test credentials scoped, disposable, and easy to rotate. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Hook](https://vostride.com/docs/agent-qa/configuration/hook.md) - Next: [Local config](https://vostride.com/docs/agent-qa/configuration/local-config.md) --- ## Global Config URL: https://vostride.com/docs/agent-qa/configuration/global-config Markdown: https://vostride.com/docs/agent-qa/configuration/global-config.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Configure workspace discovery, services, registries, plugins, analytics, and default run behavior in agent-qa.config.yaml. `agent-qa.config.yaml` is the project-wide source of truth. It tells agent-qa where to find tests, suites, hooks, variables, secrets, rules, and output; which browser targets and LLM configs exist; and which defaults apply before a suite, test, dashboard action, or CLI flag overrides them. This page is organized by config key so each option has a stable heading URL you can share in review comments. ## Complete web example This example uses a web app running locally. Native mobile fields are only included where the current schema expects a mobile app-state default. ```yaml workspace: testMatch: - tests/**/*.yaml suiteMatch: - suites/**/*.suite.yaml testPathIgnore: - tests/archive/**/*.yaml hooksFile: hooks.yaml agentRules: ./agent-rules.md envFile: .env secretsFile: .env.secrets.local services: dashboard: port: 3110 dbPath: .agent-qa/dashboard.sqlite artifactsDir: .agent-qa/artifacts mcp: enabled: true transport: http host: 127.0.0.1 port: 3481 path: /mcp cache: dir: .agent-qa/cache ttl: 7d authState: dir: .agent-qa/auth-states logging: level: warn recording: enabled: true accessibility: enabled: true standard: wcag2aa runAfter: navigation failOnViolation: false memory: enabled: true provider: local dir: agent-qa-memory registry: llms: - name: codex provider: openai-subscription model: gpt-5.5 screenshotSize: 50kb effectiveResolution: 500 targets: issue-tracker-web: platform: web product: issue-tracker url: http://localhost:3000 plugins: auth: - package: "@vostride/agent-qa-subscription-auth" use: browser: name: chromium headless: true viewport: width: 1280 height: 720 mobile: appState: preserve timeout: step: 5m test: 45m navigation: 90s healing: maxAttempts: 3 planner: maxSubActions: 12 previousStepCount: 6 logCapture: console: true network: true cache: true parallel: false llm: codex analytics: privacy: true passRateScope: attributes: branch: regex: "^(main|release/.+)$" ``` ## workspace The `workspace` block is required. All paths are resolved from the directory that contains `agent-qa.config.yaml`. ### workspace.testMatch ```yaml workspace: testMatch: - tests/**/*.yaml ``` Description: Glob patterns used by test discovery in the CLI and dashboard. Possible values: one or more non-empty glob strings. Required: yes. Default: none. The config must provide at least one pattern. ### workspace.suiteMatch ```yaml workspace: suiteMatch: - suites/**/*.suite.yaml ``` Description: Glob patterns used by suite discovery. Possible values: one or more non-empty glob strings. Required: yes. Default: none. The config must provide at least one pattern. ### workspace.testPathIgnore ```yaml workspace: testPathIgnore: - tests/archive/**/*.yaml ``` Description: Optional glob patterns excluded from test and suite discovery. Possible values: zero or more glob strings. Required: no. Default: not set. ### workspace.hooksFile ```yaml workspace: hooksFile: hooks.yaml ``` Description: Path to the hook registry file. Possible values: any non-empty workspace-relative path. Required: yes. Default: none. ### workspace.agentRules ```yaml workspace: agentRules: ./agent-rules.md ``` Description: Path to project-specific QA agent instructions. Possible values: any non-empty workspace-relative path. Required: yes. Default: none. ### workspace.envFile ```yaml workspace: envFile: .env ``` Description: Path to dotenv-style non-secret variables used with `{{env:NAME}}`. Possible values: any non-empty workspace-relative path. Required: yes. Default: none. The referenced file must exist, even if it is empty. ### workspace.secretsFile ```yaml workspace: secretsFile: .env.secrets.local ``` Description: Path to dotenv-style secrets used with `{{secret:NAME}}`. Possible values: any non-empty workspace-relative path. Required: yes. Default: none. Keep this file out of git. ## services.dashboard Dashboard service settings are optional. They affect local dashboard state and artifact locations. ### services.dashboard.port ```yaml services: dashboard: port: 3110 ``` Description: Port used by the local dashboard server. Possible values: a number. Required: no. Default: not set by schema. ### services.dashboard.dbPath ```yaml services: dashboard: dbPath: .agent-qa/dashboard.sqlite ``` Description: SQLite path for dashboard state. Possible values: a workspace-relative or process-resolved path string. Required: no. Default: not set by schema. ### services.dashboard.artifactsDir ```yaml services: dashboard: artifactsDir: .agent-qa/artifacts ``` Description: Directory used by the dashboard to read and write run artifacts. Possible values: a path string. Required: no. Default: not set by schema. ## services.mcp MCP hosts are intentionally local-only. Valid HTTP hosts are `localhost`, `127.0.0.1`, and `::1`. ### services.mcp.enabled ```yaml services: mcp: enabled: true ``` Description: Enables the local MCP service. Possible values: `true` or `false`. Required: no. Default: not set by schema. ### services.mcp.transport ```yaml services: mcp: transport: http ``` Description: Chooses how the MCP service is exposed. Possible values: `http` or `stdio`. Required: no. Default: not set by schema. ### services.mcp.host ```yaml services: mcp: host: 127.0.0.1 ``` Description: Loopback host used for HTTP MCP. Possible values: `localhost`, `127.0.0.1`, or `::1`. Required: no. Default: not set by schema. ### services.mcp.port ```yaml services: mcp: port: 3481 ``` Description: Port used by HTTP MCP. Possible values: an integer from 1 to 65535. Required: no. Default: not set by schema. ### services.mcp.path ```yaml services: mcp: path: /mcp ``` Description: HTTP path for MCP requests. Possible values: a non-empty string starting with `/`. Required: no. Default: not set by schema. ## services.cache ### services.cache.dir ```yaml services: cache: dir: .agent-qa/cache ``` Description: Directory for action cache data. Possible values: a path string. Required: yes when `services.cache` is present. Default: not set by schema. ### services.cache.ttl ```yaml services: cache: ttl: 7d ``` Description: Time to keep cache entries. Possible values: duration strings such as `30s`, `5m`, `1h`, and `7d`. Required: yes when `services.cache` is present. Default: not set by schema. ## services.authState ### services.authState.dir ```yaml services: authState: dir: .agent-qa/auth-states ``` Description: Directory for named web auth-state payloads and metadata. Keep this directory out of git. Possible values: any non-empty path string. Required: yes when `services.authState` is present. Default: `.agent-qa/auth-states` when auth state is resolved without an explicit directory. ## services.logging ### services.logging.level ```yaml services: logging: level: warn ``` Description: Default log level. CLI `--log-level`, `--verbose`, and `--quiet` can override it. Possible values: `silent`, `error`, `warn`, `info`, or `debug`. Required: yes when `services.logging` is present. Default: not set by schema. ## services.recording ### services.recording.enabled ```yaml services: recording: enabled: true ``` Description: Enables video recording defaults. CLI `--record` can also enable recording for a run. Possible values: `true` or `false`. Required: no. Default: not set by schema. ## services.accessibility ### services.accessibility.enabled ```yaml services: accessibility: enabled: true ``` Description: Enables accessibility checks for web runs. Possible values: `true` or `false`. Required: yes when `services.accessibility` is present. Default: not set by schema. ### services.accessibility.standard ```yaml services: accessibility: standard: wcag2aa ``` Description: Accessibility standard used by the checker. Possible values: `wcag2a`, `wcag2aa`, or `wcag2aaa`. Required: no. Default: not set by schema. ### services.accessibility.runAfter ```yaml services: accessibility: runAfter: navigation ``` Description: Controls when accessibility scans run during a web test. Possible values: `every-step`, `navigation`, or `test-end`. Required: no. Default: not set by schema. ### services.accessibility.failOnViolation ```yaml services: accessibility: failOnViolation: false ``` Description: Fails the run when accessibility violations are found. Possible values: `true` or `false`. Required: no. Default: not set by schema. ### services.accessibility.disableRules ```yaml services: accessibility: disableRules: - color-contrast ``` Description: Rule IDs to disable. Possible values: zero or more rule ID strings. Required: no. Default: not set. ### services.accessibility.exclude ```yaml services: accessibility: exclude: - "[data-test-preview]" ``` Description: Selectors excluded from accessibility scanning. Possible values: zero or more selector strings. Required: no. Default: not set. ## services.memory ### services.memory.enabled ```yaml services: memory: enabled: true ``` Description: Enables runtime memory injection. Possible values: `true` or `false`. Required: no. Default: `true`. ### services.memory.provider ```yaml services: memory: provider: local ``` Description: Memory storage provider. Possible values: `local`. Required: no. Default: `local`. ### services.memory.dir ```yaml services: memory: dir: agent-qa-memory ``` Description: Directory used by local memory. Possible values: a non-empty path string. Required: no. Default: the built-in local memory directory. ### services.memory.minTrust ```yaml services: memory: minTrust: 0.3 ``` Description: Minimum trust score required before memory is injected into a step. Possible values: a number from 0 to 1. Required: no. Default: `0.3`. ### services.memory.maxInjections ```yaml services: memory: maxInjections: 3 ``` Description: Maximum number of memory entries injected into a step. Possible values: an integer greater than or equal to 0. Required: no. Default: `3`. ### services.memory.curatorEnabled ```yaml services: memory: curatorEnabled: true ``` Description: Enables the memory curator after runs. Possible values: `true` or `false`. Required: no. Default: `true`. ### services.memory.curatorLockTimeout ```yaml services: memory: curatorLockTimeout: 120000 ``` Description: Curator lock timeout in milliseconds. Possible values: an integer of at least 1000. Required: no. Default: `120000`. ### services.memory.trustConfirmDelta ```yaml services: memory: trustConfirmDelta: 0.05 ``` Description: Trust increase applied when memory is confirmed. Possible values: a number from 0 to 1. Required: no. Default: `0.05`. ### services.memory.trustContradictDelta ```yaml services: memory: trustContradictDelta: 0.10 ``` Description: Trust decrease applied when memory is contradicted. Possible values: a number from 0 to 1. Required: no. Default: `0.10`. ### services.memory.ablationEnabled ```yaml services: memory: ablationEnabled: true ``` Description: Enables memory ablation behavior. Possible values: `true` or `false`. Required: no. Default: `true`. ### services.memory.circuitBreakerEnabled ```yaml services: memory: circuitBreakerEnabled: true ``` Description: Enables the memory circuit breaker. Possible values: `true` or `false`. Required: no. Default: `true`. ### services.memory.circuitBreakerWindowSize ```yaml services: memory: circuitBreakerWindowSize: 20 ``` Description: Window size for memory circuit checks. Possible values: an integer of at least 5. Required: no. Default: `20`. ### services.memory.circuitBreakerBaselineSize ```yaml services: memory: circuitBreakerBaselineSize: 3 ``` Description: Baseline sample size for memory circuit checks. Possible values: an integer of at least 2. Required: no. Default: `3`. ### services.memory.circuitBreakerThreshold ```yaml services: memory: circuitBreakerThreshold: 0.15 ``` Description: Failure threshold for the memory circuit breaker. Possible values: a number from 0 to 1. Required: no. Default: `0.15`. ## registry.llms Each LLM entry names a model configuration that `use.llm`, tests, suites, and auth storage can reference. ### registry.llms.name ```yaml registry: llms: - name: codex ``` Description: Stable config name for this LLM. Possible values: lowercase alphanumeric names with hyphens, starting with an alphanumeric character. Required: yes for each LLM entry. Default: none. ### registry.llms.provider ```yaml registry: llms: - name: codex provider: openai-subscription ``` Description: Provider ID used to resolve model access. Possible values: built-in provider modes such as `openai-compatible`, `anthropic-compatible`, `openai-subscription`, `anthropic-subscription`, and `gemini`, plus provider IDs added by auth plugins. Required: yes for each LLM entry. Default: none. ### registry.llms.model ```yaml registry: llms: - name: codex provider: openai-subscription model: gpt-5.5 ``` Description: Provider model name. Possible values: any provider-supported model string. Required: yes for each LLM entry. Default: none. ### registry.llms.baseURL ```yaml registry: llms: - name: local-openai provider: openai-compatible model: qwen2.5-vl baseURL: http://localhost:11434/v1 ``` Description: Base URL for compatible providers. Possible values: a URL string. Required: yes for `openai-compatible` and `anthropic-compatible` providers. Default: not set. ### registry.llms.providerHeaders ```yaml registry: llms: - name: claude-compatible provider: anthropic-compatible model: claude-sonnet-4-5 baseURL: https://api.anthropic.com providerHeaders: anthropic-beta: token-efficient-tools-2025-02-19 ``` Description: Extra provider headers for Anthropic-compatible requests. Header names that look like auth headers are rejected. Possible values: string-to-string header map. Required: no. Default: not set. ### registry.llms.screenshotSize ```yaml registry: llms: - name: codex provider: openai-subscription model: gpt-5.5 screenshotSize: 50kb ``` Description: Optional screenshot compression target for this model config. Possible values: size strings such as `50kb`, `256k`, and `1mb`. Required: no. Default: not set. ### registry.llms.effectiveResolution ```yaml registry: llms: - name: codex provider: openai-subscription model: gpt-5.5 effectiveResolution: 500 ``` Description: Optional positive integer describing effective image resolution. Possible values: positive integers. Required: no. Default: not set. ### registry.llms.contextWindow ```yaml registry: llms: - name: codex provider: openai-subscription model: gpt-5.5 contextWindow: 1mb ``` Description: Optional context window hint for a named LLM config. Possible values: size strings such as `256kb`, `1mb`, or `2mb`. Required: no. Default: not set. ## registry.targets Targets are the named apps under test. Web targets are the common path for most teams. ### registry.targets.product ```yaml registry: targets: issue-tracker-web: product: issue-tracker ``` Description: Optional product label for the target. Possible values: strings that do not contain `..`, path separators, or null bytes. Required: no. Default: not set. ### registry.targets.platform ```yaml registry: targets: issue-tracker-web: platform: web ``` Description: Target platform. Possible values: `web`, `android`, or `ios`. Required: yes for each target. Default: none. ### registry.targets.url ```yaml registry: targets: issue-tracker-web: platform: web url: http://localhost:3000 ``` Description: Start URL for a web target. Possible values: a URL string. Required: yes when `platform` is `web`. Default: none. ### registry.targets.bundleId ```yaml registry: targets: ios-release: platform: ios bundleId: com.company.issuetracker ``` Description: iOS bundle identifier for native iOS targets. Possible values: string. Required: no. Default: not set. ### registry.targets.appPackage ```yaml registry: targets: android-release: platform: android appPackage: com.company.issuetracker ``` Description: Android package name for native Android targets. Possible values: string. Required: no. Default: not set. ### registry.targets.appActivity ```yaml registry: targets: android-release: platform: android appActivity: .MainActivity ``` Description: Android launch activity for native Android targets. Possible values: string. Required: no. Default: not set. ### registry.targets.app.path ```yaml registry: targets: android-release: platform: android app: path: apps/android/release.apk ``` Description: Portable app artifact path for native targets. Absolute paths are rejected; local machine overrides belong in `agent-qa.local.yaml`. Possible values: relative path string. Required: no. Default: not set. ### registry.targets.app.browserstack ```yaml registry: targets: android-release: platform: android app: browserstack: bs://uploaded-app-id ``` Description: BrowserStack app reference for native targets. Possible values: BrowserStack app reference or relative value. Required: no. Default: not set. ## registry.devices Device profiles are reusable mobile device references. Web tests normally do not need this block. ### registry.devices.platform ```yaml registry: devices: android-local: platform: android ``` Description: Device platform. Possible values: `android` or `ios`. Required: yes for each device profile. Default: none. ### registry.devices.transport ```yaml registry: devices: android-local: platform: android transport: local ``` Description: Device transport. Possible values: `local` or `browserstack`. Required: yes for each device profile. Default: none. ### registry.devices.match ```yaml registry: devices: android-local: platform: android transport: local match: appPackage: com.company.issuetracker ``` Description: Provider-specific match fields. Local Android permits `avd`, `serial`, `appPackage`, `appActivity`, `automationName`, `browserName`, and `platformVersion`. Local iOS permits `udid`, `bundleId`, `automationName`, and `platformVersion`. Possible values: object with fields accepted by the chosen platform and transport. Required: no. Default: empty object. ## registry.providers ### registry.providers ```yaml registry: providers: browserstack: project: issue-tracker-web ``` Description: Provider-specific objects for integrations that need named settings. Keep secrets in `agent-qa.local.yaml`, environment variables, or auth storage. Possible values: provider-name keys with provider-specific objects. Required: no. Default: not set. ## plugins ### plugins.auth.package ```yaml plugins: auth: - package: "@vostride/agent-qa-subscription-auth" ``` Description: Auth plugin package loaded relative to the config file. Possible values: package import string. Required: either `package` or `path` is required for each auth plugin entry. Default: no auth plugins. ### plugins.auth.path ```yaml plugins: auth: - path: ./tools/agent-qa-auth-plugin.mjs ``` Description: Local auth plugin module. Possible values: path string. Required: either `package` or `path` is required for each auth plugin entry. Default: no auth plugins. ## use `use` contains defaults that cascade into suite and test runs. CLI flags and test-level `use` values can override them. ### use.browser.name ```yaml use: browser: name: chromium ``` Description: Browser engine used for web tests. Possible values: `chromium`, `firefox`, or `webkit`. Required: yes when `use.browser` is present. Default: not set by schema. ### use.browser.headless ```yaml use: browser: headless: true ``` Description: Runs web tests without a visible browser window. Possible values: `true` or `false`. Required: yes when `use.browser` is present. Default: not set by schema. ### use.browser.viewport.width ```yaml use: browser: viewport: width: 1280 ``` Description: Browser viewport width. Possible values: number. Required: no. Default: not set by schema. ### use.browser.viewport.height ```yaml use: browser: viewport: height: 720 ``` Description: Browser viewport height. Possible values: number. Required: no. Default: not set by schema. ### use.mobile.appState ```yaml use: mobile: appState: preserve ``` Description: Native mobile app-state behavior. Web tests do not use this value, but the current global config schema validates it when `use` is present. Possible values: `preserve` or `reset`. Required: yes when `use.mobile` is present. Default: none. ### use.timeout.step ```yaml use: timeout: step: 5m ``` Description: Default timeout for one test step. Possible values: duration strings such as `30s`, `5m`, and `1h`. Required: yes when `use.timeout` is present. Default: not set by schema. ### use.timeout.test ```yaml use: timeout: test: 45m ``` Description: Default timeout for the whole test. Possible values: duration strings. Required: yes when `use.timeout` is present. Default: not set by schema. ### use.timeout.navigation ```yaml use: timeout: navigation: 90s ``` Description: Default timeout for navigation actions. Possible values: duration strings. Required: yes when `use.timeout` is present. Default: not set by schema. ### use.healing.maxAttempts ```yaml use: healing: maxAttempts: 3 ``` Description: Default self-healing retry budget. Possible values: number. Required: yes when `use.healing` is present. Default: not set by schema. ### use.planner.maxSubActions ```yaml use: planner: maxSubActions: 12 ``` Description: Planner sub-action budget per step. Possible values: number. Required: yes when `use.planner` is present. Default: not set by schema. ### use.planner.previousStepCount ```yaml use: planner: previousStepCount: 6 ``` Description: Number of previous steps included as context. Possible values: number. Required: yes when `use.planner` is present. Default: not set by schema. ### use.logCapture.console ```yaml use: logCapture: console: true ``` Description: Captures browser console logs. Possible values: `true` or `false`. Required: yes when `use.logCapture` is present. Default: not set by schema. ### use.logCapture.network ```yaml use: logCapture: network: true ``` Description: Captures browser network logs. Possible values: `true` or `false`. Required: yes when `use.logCapture` is present. Default: not set by schema. ### use.cache ```yaml use: cache: true ``` Description: Enables action cache by default. CLI `--no-cache` disables it for one run. Possible values: `true` or `false`. Required: no. Default: not set by schema. ### use.llm ```yaml use: llm: codex ``` Description: Default LLM config name. Possible values: the `name` of an entry in `registry.llms`. Required: no. Default: not set by schema. ### use.parallel ```yaml use: parallel: false ``` Description: Allows parallel execution where supported. Possible values: `true` or `false`. Required: no. Default: not set by schema. ## analytics ### analytics.privacy ```yaml analytics: privacy: true ``` Description: Privacy flag for analytics behavior. Possible values: `true`. Required: no. Default: not set. ### analytics.passRateScope.attributes ```yaml analytics: passRateScope: attributes: branch: regex: "^(main|release/.+)$" ``` Description: Attribute filters used for pass-rate scoping. Possible values: string values or objects with a non-empty `regex` string. Required: no. Default: not set. ## Environment overrides Environment variables can override selected global config keys at load time. ### AGENT_QA_DASHBOARD_PORT ```bash AGENT_QA_DASHBOARD_PORT=3110 ``` Overrides: `services.dashboard.port`. ### AGENT_QA_MCP_PORT ```bash AGENT_QA_MCP_PORT=3481 ``` Overrides: `services.mcp.port`. ### AGENT_QA_CACHE_DIR ```bash AGENT_QA_CACHE_DIR=.agent-qa/cache ``` Overrides: `services.cache.dir`. ### AGENT_QA_CACHE_TTL ```bash AGENT_QA_CACHE_TTL=7d ``` Overrides: `services.cache.ttl`. ### AGENT_QA_LOG_LEVEL ```bash AGENT_QA_LOG_LEVEL=debug ``` Overrides: `services.logging.level`. ### AGENT_QA_HEADLESS ```bash AGENT_QA_HEADLESS=false ``` Overrides: `use.browser.headless`. Use `agent-qa run --config-debug` when you want to inspect the resolved config with source attribution. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Memory Reliability and Maintenance](https://vostride.com/docs/agent-qa/memory/reliability-maintenance.md) - Next: [Test](https://vostride.com/docs/agent-qa/configuration/test.md) --- ## Hook URL: https://vostride.com/docs/agent-qa/configuration/hook Markdown: https://vostride.com/docs/agent-qa/configuration/hook.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Register sandboxed hooks in hooks.yaml and implement scripts that prepare web test data, verify API side effects, and export runtime variables. Hooks are project scripts that agent-qa runs in Docker. They are useful for preparing data, calling product APIs, verifying web UI side effects, and cleaning up records that a browser test created. There are two parts: 1. The hook config file, usually `hooks.yaml`. 2. The hook implementation file, such as `scripts/verify-task.mjs`. ## Hook config file `workspace.hooksFile` in `agent-qa.config.yaml` points to the hook registry. ```yaml workspace: hooksFile: hooks.yaml ``` The registry schema has one root key: `hooks`. ```yaml hooks: - id: h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira name: Verify web task through API runtime: node file: scripts/verify-task.mjs deps: - scripts/api-client.mjs timeout: 45s network: true ``` ## hooks ```yaml hooks: - id: h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira name: Verify web task through API runtime: node file: scripts/verify-task.mjs timeout: 45s ``` Description: The list of hook definitions available to tests and suites. Possible values: zero or more hook objects. Required: yes in the hook registry file. Default: none. ## hooks.id ```yaml hooks: - id: h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira ``` Description: Generated stable hook ID. Possible values: canonical hook IDs that start with `h_` and contain 10 id-agent words. Required: yes. Default: none. Generate it with `agent-qa ids generate hook` or the dashboard. ## hooks.name ```yaml hooks: - name: Verify web task through API ``` Description: Unique human-readable hook name. Possible values: any non-empty string. Required: yes. Default: none. ## hooks.runtime ```yaml hooks: - runtime: node ``` Description: Runtime used inside the Docker hook runner. Possible values: `node`, `bun`, `python`, or `bash`. Required: yes. Default: none. ## hooks.file ```yaml hooks: - file: scripts/verify-task.mjs ``` Description: Hook entry file. agent-qa copies this file into the sandbox workspace before execution. Possible values: non-empty path string. Required: yes. Default: none. ## hooks.deps ```yaml hooks: - file: scripts/verify-task.mjs deps: - scripts/api-client.mjs ``` Description: Additional files copied beside the hook entry file. Possible values: zero or more path strings. Required: no. Default: empty list. ## hooks.packageFile ```yaml hooks: - file: scripts/verify-task.mjs packageFile: package.json ``` Description: Optional package file copied beside the hook entry file. Possible values: path string. Required: no. Default: not set. ## hooks.timeout ```yaml hooks: - timeout: 45s ``` Description: Hook execution timeout. Possible values: duration strings such as `30s`, `2m`, and `1h`. Required: yes. Default: none. ## hooks.network ```yaml hooks: - network: true ``` Description: Allows network access in the hook container. Set this to `true` for API verification hooks that call your local or staging web API. Possible values: `true` or `false`. Required: no. Default: `true`. ## Referencing hooks Use `setup` and `teardown` arrays when a hook should run before or after a test or suite: ```yaml setup: - h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira ``` Use inline `runHook` syntax when a hook should run at a specific point in the step list: ```yaml steps: - Run the API verification hook {{runHook:"h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira"}}. - Verify TASK_FOUND is exactly "true"; the current value is "{{env:TASK_FOUND}}". ``` Inline hooks run before variable interpolation for that step. Variables they export are available to later steps. ## Hook implementation Hook scripts receive environment variables from `.env`, CLI `--var`, previously successful hooks, and secrets from the configured secrets file. Secrets are available as environment variables inside the hook container. To return variables to agent-qa, write a dotenv file to `/tmp/agent-qa.env`. agent-qa reads that file after the container exits and merges the variables into the run. ```js // scripts/verify-task.mjs import { writeFile } from "node:fs/promises" function requiredEnv(name) { const value = process.env[name] if (!value || !value.trim()) { throw new Error(`Missing required environment variable: ${name}`) } return value.trim() } function escapeEnv(value) { return String(value ?? "") .replace(/\r?\n/g, " ") .replace(/\\/g, "\\\\") .replace(/"/g, '\\"') } async function writeHookEnv(values) { const body = Object.entries(values) .map(([key, value]) => `${key}="${escapeEnv(value)}"`) .concat("") .join("\n") await writeFile("/tmp/agent-qa.env", body, "utf-8") } const apiUrl = requiredEnv("TASK_API_URL") const apiKey = requiredEnv("TASK_API_KEY") const title = requiredEnv("TASK_TITLE") const response = await fetch(`${apiUrl}/tasks/search`, { method: "POST", headers: { Authorization: `Bearer ${apiKey}`, "Content-Type": "application/json", }, body: JSON.stringify({ title }), }) if (!response.ok) { throw new Error(`Task lookup failed with HTTP ${response.status}`) } const payload = await response.json() const task = payload.tasks?.find((item) => item.title === title) if (!task) { throw new Error(`Task "${title}" was not found`) } await writeHookEnv({ TASK_FOUND: "true", TASK_ID: task.id, TASK_STATUS: task.status, }) console.log(`Verified task ${task.id}: ${task.title}`) ``` ## /tmp/agent-qa.env ```dotenv TASK_FOUND="true" TASK_ID="task_123" TASK_STATUS="In Progress" ``` Description: Dotenv file that a hook writes when it needs to export variables back into the run. Required: only when the hook needs to return variables. Default: no variables are exported if the file is not written. ## Sandbox behavior agent-qa copies the hook entry file, dependency files, and optional package file into a temporary Docker workspace. The container runs read-only, mounts a writable `/tmp`, applies CPU and memory limits, and removes the temporary workspace after execution. ### node runtime Docker image: `vostride/agent-qa-hook-runner-node`. Use it for JavaScript and TypeScript-adjacent web API checks. ### bun runtime Docker image: `vostride/agent-qa-hook-runner-bun`. Use it when your hook relies on Bun-compatible scripts. ### python runtime Docker image: `vostride/agent-qa-hook-runner-python`. Use it for Python data setup or API verification scripts. ### bash runtime Docker image: `vostride/agent-qa-hook-runner-bash`. Use it for shell scripts that do not need a larger runtime. Hooks run sequentially. If one setup hook fails, later hooks in that hook group are skipped. A later successful hook can override a variable exported by an earlier hook. ## Secrets and output Known secret values are redacted from hook stdout, stderr, and error text. Variables written to `/tmp/agent-qa.env` are also filtered when their value exactly equals a known secret. Do not intentionally export secrets from hooks; export derived IDs, status flags, and other non-secret runtime data instead. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Suite](https://vostride.com/docs/agent-qa/configuration/suite.md) - Next: [Env & Secrets](https://vostride.com/docs/agent-qa/configuration/env-secrets.md) --- ## Configuration URL: https://vostride.com/docs/agent-qa/configuration Markdown: https://vostride.com/docs/agent-qa/configuration.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Understand the file-backed configuration system that powers agent-qa projects. agent-qa is designed to keep your QA system under complete team control. Configuration, tests, suites, hooks, custom rules, local bindings, auth metadata, and memory are all represented as files, so teams can review, version, and collaborate on QA changes the same way they review application code. The dashboard can create and edit those files for you, but the files remain the source of truth. You can change config toggles in the UI, write a test by hand, update a suite in a pull request, or let an AI coding agent propose a diff. The next run reads the same files. - [Global Configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md): Define workspace files, dashboard and MCP services, LLM configs, targets, devices, auth plugins, analytics, and default run behavior. - [Test](https://vostride.com/docs/agent-qa/configuration/test.md): Author one natural-language journey with generated IDs, target selection, hook references, variables, captures, and per-test overrides. - [Suite](https://vostride.com/docs/agent-qa/configuration/suite.md): Group tests into an ordered workflow with shared context, shared hooks, target selection, and suite-level execution overrides. - [Hook](https://vostride.com/docs/agent-qa/configuration/hook.md): Register sandboxed scripts in hooks.yaml, then implement Node.js, Bun, Python, or Bash hooks that export runtime variables. - [Variables & Secrets](https://vostride.com/docs/agent-qa/configuration/env-secrets.md): Load non-secret variables from .env, keep sensitive values in the secrets file, and pass temporary data between hooks and steps. - [Local Device Config](https://vostride.com/docs/agent-qa/configuration/local-config.md): Keep machine-specific device serials, local app paths, and provider credentials out of the portable global config. - [Agent Rules](https://vostride.com/docs/agent-qa/configuration/agent-rules.md): Tune the QA agent with project-specific execution rules without changing every test file. - [auth.json](https://vostride.com/docs/agent-qa/configuration/auth-json.md): Understand where saved LLM credentials live, what the credential schema looks like, and how to create it at CI runtime. ## File map Most teams start with this set of files: ```txt agent-qa.config.yaml # global workspace, services, registry, plugins, and use defaults tests/**/*.yaml # single natural-language test definitions suites/**/*.suite.yaml # ordered groups of tests hooks.yaml # hook registry scripts/*.mjs # hook implementation files .env # non-secret variables used by steps and hooks .env.secrets.local # secret values used through {{secret:NAME}} agent-qa.local.yaml # local device, app, and provider bindings agent-rules.md # additional QA agent instructions ~/.agent-qa/auth.json # saved LLM credentials, outside the project by default ``` Generated runtime data also remains file-backed. Dashboard state, artifacts, cache, and memory live under the configured paths, usually inside `.agent-qa/` or `agent-qa-memory/`. ## Configuration layers agent-qa resolves configuration from a small set of explicit layers: 1. `agent-qa.config.yaml` defines project-wide defaults. 2. Suite YAML can override supported `use` settings for every test in the suite. 3. Test YAML can override supported `use` settings for that test. 4. CLI flags such as `--device`, `--no-cache`, `--no-memory`, `--headless`, `--no-headless`, and `--var KEY=VALUE` apply to the current command. Use the dashboard when you want a guided editor. Use file edits when you want reviewable diffs. Both paths update the same source files. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Caching](https://vostride.com/docs/agent-qa/caching.md) - Next: [CLI](https://vostride.com/docs/agent-qa/cli.md) --- ## Local config URL: https://vostride.com/docs/agent-qa/configuration/local-config Markdown: https://vostride.com/docs/agent-qa/configuration/local-config.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Keep machine-specific provider credentials, device bindings, and app artifact paths in agent-qa.local.yaml instead of shared project config. `agent-qa.local.yaml` stores values that vary by developer machine or CI runner. The init command creates it and adds it to `.gitignore`. Web projects often need no local device binding at all. A web target can live entirely in `agent-qa.config.yaml`: ```yaml registry: targets: issue-tracker-web: platform: web product: issue-tracker url: http://localhost:3000 ``` Use `agent-qa.local.yaml` when credentials or device/app paths should stay off the shared config: ```yaml providers: browserstack: username: ${BROWSERSTACK_USERNAME} accessKey: ${BROWSERSTACK_ACCESS_KEY} ``` ## devices ```yaml devices: android-local: serial: RZCT90BCMWW appPackage: com.company.issuetracker ``` Description: Per-device local match data keyed by `registry.devices` name. Possible values: local Android and iOS match fields. Required: only when a selected mobile device profile needs local bindings. Default: no local device bindings. Web tests usually do not set `devices`. ### devices.android.avd ```yaml devices: android-local: avd: Pixel_8_API_35 ``` Description: Android emulator name. Possible values: Android Virtual Device name. Default: not set. ### devices.android.serial ```yaml devices: android-local: serial: RZCT90BCMWW ``` Description: Android device serial. Possible values: serial reported by Android tooling. Default: not set. ### devices.android.appPackage ```yaml devices: android-local: appPackage: com.company.issuetracker ``` Description: Android app package override for this machine. Possible values: package name string. Default: inherited from `registry.devices` or target config. ### devices.android.appActivity ```yaml devices: android-local: appActivity: .MainActivity ``` Description: Android app activity override for this machine. Possible values: activity string. Default: inherited from `registry.devices` or target config. ### devices.android.automationName ```yaml devices: android-local: automationName: UiAutomator2 ``` Description: Android automation engine. Possible values: provider-supported automation name. Default: not set. ### devices.android.browserName ```yaml devices: android-local: browserName: Chrome ``` Description: Browser name for Android browser sessions. Possible values: provider-supported browser name. Default: not set. ### devices.android.platformVersion ```yaml devices: android-local: platformVersion: "15" ``` Description: Android platform version. Possible values: version string. Default: not set. ### devices.ios.udid ```yaml devices: ios-local: udid: 00008110-001234567890801E ``` Description: iOS device UDID. Possible values: UDID string. Default: not set. ### devices.ios.bundleId ```yaml devices: ios-local: bundleId: com.company.issuetracker ``` Description: iOS bundle ID override for this machine. Possible values: bundle ID string. Default: inherited from `registry.devices` or target config. ### devices.ios.automationName ```yaml devices: ios-local: automationName: XCUITest ``` Description: iOS automation engine. Possible values: provider-supported automation name. Default: not set. ### devices.ios.platformVersion ```yaml devices: ios-local: platformVersion: "18" ``` Description: iOS platform version. Possible values: version string. Default: not set. ## apps ```yaml apps: android-release: path: apps/issue-tracker-debug.apk ``` Description: Per-target app install overrides keyed by `registry.targets` name. Possible values: `path` and `browserstack`. Required: only when a native target needs a machine-specific app artifact. Default: inherited from global target config. Web tests usually do not set `apps`. ### apps.path ```yaml apps: android-release: path: apps/issue-tracker-debug.apk ``` Description: Local app artifact path for this machine. Possible values: relative path string. Default: inherited from `registry.targets..app.path`. ### apps.browserstack ```yaml apps: android-release: browserstack: bs://uploaded-app-id ``` Description: BrowserStack app reference override. Possible values: BrowserStack app reference. Default: inherited from `registry.targets..app.browserstack`. ## providers ```yaml providers: browserstack: username: ${BROWSERSTACK_USERNAME} accessKey: ${BROWSERSTACK_ACCESS_KEY} ``` Description: Local provider credentials. For BrowserStack, agent-qa checks `agent-qa.local.yaml` first, then `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESS_KEY`. Possible values: provider-specific credential objects. Required: only when the provider needs local credentials. Default: no local provider credentials. ### providers.browserstack.username ```yaml providers: browserstack: username: ${BROWSERSTACK_USERNAME} ``` Description: BrowserStack username. Possible values: string or environment-variable interpolation handled by your shell/tooling before use. Default: falls back to `BROWSERSTACK_USERNAME` when local config does not provide a value. ### providers.browserstack.accessKey ```yaml providers: browserstack: accessKey: ${BROWSERSTACK_ACCESS_KEY} ``` Description: BrowserStack access key. Possible values: string or environment-variable interpolation handled by your shell/tooling before use. Default: falls back to `BROWSERSTACK_ACCESS_KEY` when local config does not provide a value. ## Generate local bindings Use the devices command to scan connected devices and generate local bindings: ```bash npx agent-qa devices init ``` Review the generated file before running mobile tests. Local config should not be committed. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Env & Secrets](https://vostride.com/docs/agent-qa/configuration/env-secrets.md) - Next: [Agent rules](https://vostride.com/docs/agent-qa/configuration/agent-rules.md) --- ## Suite URL: https://vostride.com/docs/agent-qa/configuration/suite Markdown: https://vostride.com/docs/agent-qa/configuration/suite.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Group multiple web tests into an ordered agent-qa suite with shared target, context, hooks, and run overrides. A suite YAML file runs multiple tests as one workflow. Use suites for release smoke checks, regression bundles, and end-to-end web journeys where several tests should share a target, context, or setup hook. ## Example web suite ```yaml suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis name: "Issue tracker: web task lifecycle" target: issue-tracker-web context: | Run this suite against the browser target configured as issue-tracker-web. The QA user is already signed in. setup: - h_seed-web-workspace-calm-cedar-lamp-river-field teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed use: browser: name: chromium headless: true viewport: width: 1280 height: 720 cache: false tests: - test: tests/task-create-and-complete.yaml id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila ``` ## suite-id ```yaml suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis ``` Description: Optional generated suite ID. Possible values: canonical suite IDs that start with `s_` and contain 10 id-agent words. Required: no. Default: not set. Generate it with `agent-qa ids generate suite` or the dashboard. ## name ```yaml name: "Issue tracker: web task lifecycle" ``` Description: Human-readable suite name. Possible values: any string. Required: yes. Default: none. ## target ```yaml target: issue-tracker-web ``` Description: Target name from `registry.targets`. Every listed test runs against this target unless runtime preparation resolves otherwise. Possible values: any configured target name. For web suites, use a target whose platform is `web`. Required: yes. Default: none. ## context ```yaml context: | Run this suite against the browser target configured as issue-tracker-web. The QA user is already signed in. ``` Description: Shared background passed to every test in the suite. Possible values: any string, usually a multi-line block. Required: no. Default: not set. ## setup ```yaml setup: - h_seed-web-workspace-calm-cedar-lamp-river-field ``` Description: Hook IDs that run before suite tests. Possible values: hook IDs registered in `hooks.yaml`. Required: no. Default: no setup hooks. ## teardown ```yaml teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed ``` Description: Hook IDs that run after suite execution. Possible values: hook IDs registered in `hooks.yaml`. Required: no. Default: no teardown hooks. ## use ```yaml use: browser: name: chromium headless: true cache: false ``` Description: Suite-level run overrides. Use `authState` here when every child web test should share one selected authenticated browser context. Possible values: the same override shape supported by test `use`: `browser`, `timeout`, `healing`, `planner`, `logCapture`, `cache`, `authState`, `mobile`, `llm`, `parallel`, and `device`. Required: no. Default: inherited from global config and CLI flags. ### use.browser ```yaml use: browser: name: chromium headless: true viewport: width: 1280 height: 720 ``` Description: Browser defaults for every web test in the suite. Possible values: `name`, `headless`, and optional `viewport.width` / `viewport.height`. Default: inherited. ### use.timeout ```yaml use: timeout: step: 3m test: 40m navigation: 90s ``` Description: Timeout defaults for every test in the suite. Possible values: duration strings. Default: inherited. ### use.healing ```yaml use: healing: maxAttempts: 2 ``` Description: Self-healing retry budget for suite tests. Possible values: number. Default: inherited. ### use.planner ```yaml use: planner: maxSubActions: 10 previousStepCount: 5 ``` Description: Planner defaults for suite tests. Possible values: numbers for `maxSubActions` and `previousStepCount`. Default: inherited. ### use.logCapture ```yaml use: logCapture: console: true network: true ``` Description: Browser console and network log capture defaults. Possible values: booleans for `console` and `network`. Default: inherited. ### use.cache ```yaml use: cache: false ``` Description: Enables or disables action cache for suite tests. Possible values: `true` or `false`. Default: inherited. ### use.authState ```yaml use: authState: qa-admin ``` Description: Named web auth state to load for the suite's shared browser context. Child tests inherit the suite auth state when they omit `use.authState`. Repeating the same child auth-state name is allowed; selecting a different child auth state is rejected. Possible values: a configured lowercase auth-state name. Default: not set. Only one primary auth state is supported per suite run. See [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md). ### use.llm ```yaml use: llm: codex ``` Description: LLM config name for suite tests. Possible values: a name from `registry.llms`. Default: inherited. ### use.parallel ```yaml use: parallel: false ``` Description: Allows parallel execution where supported. Possible values: `true` or `false`. Default: inherited. ### use.device ```yaml use: device: android-local ``` Description: Mobile device profile override. Web suites usually do not set this. Possible values: a name from `registry.devices`. Default: inherited or CLI-selected. ### use.mobile ```yaml use: mobile: appState: preserve ``` Description: Native mobile app-state override. Web suites do not use this value. Possible values: `preserve` or `reset`. Default: inherited. ## tests ```yaml tests: - test: tests/task-create-and-complete.yaml id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila ``` Description: Ordered list of test entries. Each entry links a file path to the expected test ID inside that file. Possible values: one or more objects with `test` and `id`. Required: yes. Default: none. ### tests.test ```yaml tests: - test: tests/task-create-and-complete.yaml ``` Description: Workspace-relative path to a test YAML file. Possible values: path string. Required: yes for each test entry. ### tests.id ```yaml tests: - id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila ``` Description: Generated `test-id` inside the referenced test file. This keeps identity stable even when filenames change. Possible values: canonical test IDs that start with `t_` and contain 10 id-agent words. Required: yes for each test entry. ## Hook variable flow Suite hooks run sequentially. Variables exported by a successful hook are passed to later hooks and to suite tests. Per-test setup hooks can add or override variables for that test. Suite teardown hooks receive accumulated suite variables. This makes suites a good place to create shared web records, authenticate a browser session, or clean up server state after all tests complete. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Test](https://vostride.com/docs/agent-qa/configuration/test.md) - Next: [Hook](https://vostride.com/docs/agent-qa/configuration/hook.md) --- ## Test URL: https://vostride.com/docs/agent-qa/configuration/test Markdown: https://vostride.com/docs/agent-qa/configuration/test.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Structure a single natural-language web test definition with target, hooks, variables, steps, and per-test run overrides. A test YAML file describes one user journey. It selects a target from `agent-qa.config.yaml`, gives the agent context, optionally runs hooks, and lists the steps the agent must execute and verify. This page uses a web app example because most agent-qa projects start with browser testing. ## Example web test ```yaml test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila name: Create urgent task and finish it target: issue-tracker-web use: browser: name: chromium headless: true viewport: width: 1280 height: 720 cache: false timeout: step: 3m test: 30m context: | The web app is available at the URL configured by the issue-tracker-web target. The user is already signed in to the QA workspace. Task data comes from the workspace .env file. steps: - Open the Tasks page. - Click the create task button. - In the task title field, enter exactly "{{env:TASK_TITLE}}". - In the task description field, enter exactly "{{env:TASK_DESCRIPTION}}". - Set the status to "{{env:STATUS_IN_PROGRESS}}". - Set the priority to "{{env:TASK_PRIORITY}}". - Submit the task. - Verify task "{{env:TASK_TITLE}}" appears in the Created view. - Run the API verification hook {{runHook:"h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira"}}. - Verify TASK_FOUND is exactly "true"; the current value is "{{env:TASK_FOUND}}". - Open task "{{env:TASK_TITLE}}" and change the status to "{{env:STATUS_DONE}}". - Verify the task no longer appears in the Assigned view. ``` ## test-id ```yaml test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila ``` Description: Generated stable ID for the test. Possible values: canonical test IDs that start with `t_` and contain 10 id-agent words. Required: yes. Default: none. Generate it with `agent-qa ids generate test` or the dashboard. ## name ```yaml name: Create urgent task and finish it ``` Description: Human-readable test name shown in the dashboard, CLI output, and run artifacts. Possible values: any string. Required: yes. Default: none. ## target ```yaml target: issue-tracker-web ``` Description: Target name from `registry.targets` in `agent-qa.config.yaml`. Possible values: any configured target name. For web tests, point this at a target whose platform is `web`. Required: yes. Default: none. ## context ```yaml context: | The web app is available at the URL configured by the issue-tracker-web target. The user is already signed in to the QA workspace. ``` Description: Background passed to the agent before the first step. Possible values: any string, usually a multi-line block. Required: no. Default: not set. Do not put secrets in context. Use `{{secret:NAME}}` or hook environment variables for credentials. ## setup ```yaml setup: - h_seed-task-workspace-bird-lake-slate-palm-cloud-frost ``` Description: Hook IDs that run before the test steps. Possible values: hook IDs registered in `hooks.yaml`. Required: no. Default: no setup hooks. ## teardown ```yaml teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed ``` Description: Hook IDs that run after the test finishes. Possible values: hook IDs registered in `hooks.yaml`. Required: no. Default: no teardown hooks. ## use ```yaml use: browser: name: chromium headless: true cache: false ``` Description: Per-test overrides for supported global and suite defaults. Use `authState` here for web tests that should start already signed in. Possible values: the `UseOverrideSchema` shape: `browser`, `timeout`, `healing`, `planner`, `logCapture`, `cache`, `authState`, `mobile`, `llm`, `parallel`, and `device`. Required: no. Default: inherited from suite, global config, and CLI flags. ### use.browser.name ```yaml use: browser: name: chromium ``` Description: Browser engine for this web test. Possible values: `chromium`, `firefox`, or `webkit`. Default: inherited. ### use.browser.headless ```yaml use: browser: headless: true ``` Description: Whether this test runs without a visible browser window. Possible values: `true` or `false`. Default: inherited. ### use.browser.viewport ```yaml use: browser: viewport: width: 1280 height: 720 ``` Description: Browser viewport override for this test. Possible values: numeric `width` and `height`. Default: inherited. ### use.timeout.step ```yaml use: timeout: step: 3m ``` Description: Timeout for one step. Possible values: duration strings such as `30s`, `3m`, and `1h`. Default: inherited. ### use.timeout.test ```yaml use: timeout: test: 30m ``` Description: Timeout for the whole test. Possible values: duration strings. Default: inherited. ### use.timeout.navigation ```yaml use: timeout: navigation: 90s ``` Description: Timeout for navigation actions. Possible values: duration strings. Default: inherited. ### use.healing.maxAttempts ```yaml use: healing: maxAttempts: 1 ``` Description: Self-healing retry budget for this test. Possible values: number. Default: inherited. ### use.planner.maxSubActions ```yaml use: planner: maxSubActions: 8 ``` Description: Planner sub-action budget for this test. Possible values: number. Default: inherited. ### use.planner.previousStepCount ```yaml use: planner: previousStepCount: 4 ``` Description: Previous-step context count for this test. Possible values: number. Default: inherited. ### use.logCapture.console ```yaml use: logCapture: console: true ``` Description: Captures browser console logs during this test. Possible values: `true` or `false`. Default: inherited. ### use.logCapture.network ```yaml use: logCapture: network: true ``` Description: Captures browser network logs during this test. Possible values: `true` or `false`. Default: inherited. ### use.cache ```yaml use: cache: false ``` Description: Enables or disables action cache for this test. Possible values: `true` or `false`. Default: inherited. ### use.authState ```yaml use: authState: qa-admin ``` Description: Named web auth state to load before creating the browser context. The name is resolved for the selected target; tests cannot reference arbitrary auth-state file paths. Possible values: a configured lowercase auth-state name. Default: not set. For capture, hook access, security, and mobile boundaries, see [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md). ### use.llm ```yaml use: llm: codex ``` Description: LLM config name to use for this test. Possible values: a name from `registry.llms`. Default: inherited. ### use.parallel ```yaml use: parallel: false ``` Description: Allows parallel execution where supported. Possible values: `true` or `false`. Default: inherited. ### use.device ```yaml use: device: android-local ``` Description: Mobile device profile override. Web tests usually do not set this. Possible values: a name from `registry.devices`. Default: inherited or CLI-selected. ### use.mobile.appState ```yaml use: mobile: appState: preserve ``` Description: Native mobile app-state override. Web tests do not use this value. Possible values: `preserve` or `reset`. Default: inherited. ## meta ```yaml meta: timeout: 20m retries: 1 record: true ``` Description: Test-level execution metadata. Possible values: `timeout`, `retries`, and `record`. Required: no. Default: not set. ### meta.timeout ```yaml meta: timeout: 20m ``` Description: Test timeout metadata. Possible values: duration strings. Default: not set. ### meta.retries ```yaml meta: retries: 1 ``` Description: Retry count metadata. Possible values: number. Default: not set. ### meta.record ```yaml meta: record: true ``` Description: Recording metadata for the test. Possible values: `true` or `false`. Default: not set. ## steps ```yaml steps: - Open the Tasks page. - Click the create task button. ``` Description: Ordered instructions for the agent. A test must contain at least one step. Possible values: string steps or structured step objects. Required: yes. Default: none. ### string step ```yaml steps: - Open the dashboard and verify the project switcher is visible. ``` Description: A plain natural-language instruction. ### structured step ```yaml steps: - step: Search for task "{{env:TASK_TITLE}}" and open the first matching result. timeout: 90s retries: 1 screenshot: true maxAttempts: 2 ``` Description: A step object with per-step controls. ### step.step ```yaml steps: - step: Search for task "{{env:TASK_TITLE}}". ``` Description: The natural-language instruction for a structured step. Required: yes for structured steps. ### step.timeout ```yaml steps: - step: Wait for the import status to finish. timeout: 90s ``` Description: Timeout for this step. Possible values: duration strings. Default: inherited step timeout. ### step.retries ```yaml steps: - step: Click retryable sync. retries: 1 ``` Description: Retry count for this step. Possible values: number. Default: not set. ### step.screenshot ```yaml steps: - step: Verify the task detail page is open. screenshot: true ``` Description: Requests a screenshot around this step. Possible values: `true` or `false`. Default: not set. ### step.maxAttempts ```yaml steps: - step: Save the task. maxAttempts: 2 ``` Description: Step-level max attempts. Possible values: number. Default: not set. ## capture ```yaml steps: - step: Copy the task key from the detail header. capture: variable: TASK_KEY method: regex pattern: "TASK-[0-9]+" - Verify the activity feed mentions "{{env:TASK_KEY}}". ``` Description: Captures runtime data from a structured step for later `{{env:NAME}}` interpolation. ### capture.variable ```yaml capture: variable: TASK_KEY ``` Description: Variable name written into the run environment. Required: yes. ### capture.method ```yaml capture: method: regex ``` Description: Capture strategy. Possible values: `regex`, `selector`, or `ai`. Required: yes. ### capture.pattern ```yaml capture: method: regex pattern: "TASK-[0-9]+" ``` Description: Regex pattern used by `regex` capture. Required: when the regex method needs a pattern. ### capture.selector ```yaml capture: method: selector selector: "[data-testid='task-key']" ``` Description: Selector or stable element reference used by `selector` capture. Required: when the selector method needs a selector. ### capture.description ```yaml capture: method: ai description: Extract the task key from the task detail header. ``` Description: Semantic extraction instruction used by `ai` capture. Required: when the AI method needs semantic guidance. ## Inline hooks ```yaml steps: - Run the API verification hook {{runHook:"h_decode-ouf-laid-remain-icing-iao-vang-bur-hem-vira"}}. - Verify TASK_FOUND is exactly "true"; the current value is "{{env:TASK_FOUND}}". ``` Description: Inline hooks run at a specific point in the step list. They run before variable interpolation for that step, and exported variables are available to later steps. Possible values: `{{runHook:"h_..."}}` with a hook ID registered in `hooks.yaml`. Use captures, hooks, or `setVariable` actions for runtime data. The older `variables` block is not part of the current test schema. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Configuration](https://vostride.com/docs/agent-qa/configuration.md) - Previous: [Global Config](https://vostride.com/docs/agent-qa/configuration/global-config.md) - Next: [Suite](https://vostride.com/docs/agent-qa/configuration/suite.md) --- ## Dashboard URL: https://vostride.com/docs/agent-qa/dashboard Markdown: https://vostride.com/docs/agent-qa/dashboard.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Source-backed tour of the agent-qa dashboard routes for runs, tests, hooks, suites, memory, insights, config, and queue-aware workflows. The dashboard is the local web UI for inspecting agent-qa runs, editing test assets, reviewing memory, and managing the workspace surfaces that back CLI execution. Launch it with: ```bash agent-qa dashboard --port 3470 --open ``` The dashboard command also supports `--db ` for the dashboard database and `--open` to open the browser after startup. The `serve` command starts the same dashboard-backed local services from configuration. ## Navigation The current dashboard sidebar exposes these labels: Runs, Tests, Hooks, Suites, Memory, Insights, and Config. ![agent-qa dashboard sidebar showing Runs, Tests, Hooks, Suites, Memory, Insights, and Config](/docs/dashboard/dashboard-runs-sidebar.png) ## Runs Runs open at `/runs`. A run detail page uses `/runs/:id`, and live in-progress execution uses `/runs/:id/live`. The route helper sends active runs to live view when the status should be watched in real time. Use Runs when you need to inspect recent execution, jump into a specific run, or follow a live run while an agent is acting. ![Runs table with search, filters, queue slots, statuses, targets, attributes, duration, and start time](/docs/dashboard/dashboard-runs-table.png) Run details keep the step timeline next to the captured browser state, tabs for Overview, Variables, Network, Console, ARIA Tree, and A11y, and per-action timing and model usage metadata. ![Run detail view showing setup hooks, passed steps, browser screenshot, and action breakdown panels](/docs/dashboard/dashboard-run-detail.png) Live execution uses the same split-view model while the run is active. The progress bar, cancel control, elapsed timer, current step selection, and verifier/action feedback update as the agent works. ![Live execution view showing active run progress, step timeline, and action feedback panels](/docs/dashboard/dashboard-live-execution.png) ## Tests Tests open at `/tests`. New tests use `/tests/new`. Existing tests can be viewed at `/test/:testId`, edited at `/test/:testId/edit`, and opened in live editing mode with `/test/:testId/edit?live=1`. Use Tests when you are authoring YAML journeys, reviewing generated test ids, or switching between view and edit modes for a test. The test builder includes Builder, YAML, and Memory tabs. From the builder you can edit the test name, target, context, setup or teardown hooks, and ordered steps, then validate, save, run, or connect a live session. ![Test builder showing target selection, setup hooks, ordered steps, validate, save, run, and live session controls](/docs/dashboard/dashboard-test-builder.png) ### Live Mode Live Mode connects the editor to a real browser session. The left side stays focused on test structure and step status, while the right side shows the browser surface plus inspection tabs such as Reasoning, Env, Network, Console, and ARIA Tree. ![Live test editor showing a connected browser session, step status, and the network inspection panel](/docs/dashboard/dashboard-live-test-editor.png) ## Hooks Hooks open at `/hooks`. New hooks use `/hooks/new`. Existing hooks can be viewed at `/hook/:hookId` and edited at `/hook/:hookId/edit`. Use Hooks to review setup and teardown automation that agent-qa can call before or after a run. Hook references also appear inside test and suite builders. Setup and teardown hook cards show the stable hook id that is saved back to YAML, so a dashboard edit remains reviewable in source control. ## Suites Suites open at `/suites`. New suites use `/suites/new`. Existing suites can be viewed at `/suite/:suiteId` and edited at `/suite/:suiteId/edit`. Use Suites to collect related tests into repeatable workflows and review suite-level setup, teardown, targets, and test references. ![Suite builder showing target selection, setup and teardown hooks, included tests, run suite, and live session controls](/docs/dashboard/dashboard-suite-builder.png) ## Memory Memory opens at `/memory`. Product-specific memory opens at `/memory/:product`. Test-level memory observation panels show observations, invalid memory files, refresh, delete, and empty states such as "No observations yet." Use Memory to inspect what the agent has learned about products, suites, and tests before trusting future runs that use runtime memory injection. ![Memory detail page showing product observations, suite and test sections, filters, and copy page control](/docs/dashboard/dashboard-memory-detail.png) ## Insights Insights opens at `/insights`. The dashboard also redirects older `/analytics` and `/trends` routes into Insights. Use Insights to review chart-oriented views that summarize run and quality signals. ![Insights dashboard showing pass rate, duration, token usage, memory curator, and observation charts](/docs/dashboard/dashboard-insights.png) ## Config Config opens at `/config`. Item-focused config links use `/config?bucket=...&item=...`. The dashboard configuration section includes Dashboard settings for Port, Database Path, Artifacts Directory, and Save Changes. Use Config to inspect and edit workspace-backed settings for dashboard services and other agent-qa configuration buckets. ![Configuration page showing execution defaults, default LLM selection, parallel execution, and save runtime defaults](/docs/dashboard/dashboard-config-execution-defaults.png) ## Queue and run management Queue management is source-backed through the CLI rather than a separate sidebar view. Use `agent-qa queue list --server http://localhost:3470` to inspect pending, running, and optionally completed jobs, and `agent-qa queue cancel --server http://localhost:3470` to cancel a pending or running job. Use this with Runs and live run routes when you need to decide whether to watch, inspect, or cancel work that is already in progress. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Hooks](https://vostride.com/docs/agent-qa/guides/hooks.md) - Next: [Curator](https://vostride.com/docs/agent-qa/memory/curator.md) --- ## Auth state URL: https://vostride.com/docs/agent-qa/guides/auth-state Markdown: https://vostride.com/docs/agent-qa/guides/auth-state.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Capture a named web login once, reuse it across tests and suites, and pass the active Playwright storage-state JSON to hooks without exposing credential material elsewhere. Auth state lets a web test start from an already signed-in browser session. Capture the session once for a target, give it a logical name, then select that name from a test or suite with `use.authState`. Use auth state when the product login flow is slow, protected by MFA, or not the thing the test is trying to verify. The first step of the product test should still prove the session is valid, such as verifying the dashboard, account menu, or tenant switcher is visible. Auth state is credential material. Keep it local, ignore it from git, avoid printing it, and refresh it manually when the product session expires. ## Configure storage `agent-qa init` creates the auth-state directory setting for new workspaces. If you are wiring the config by hand, keep auth state under `.agent-qa` beside cache and artifacts. ```yaml services: authState: dir: .agent-qa/auth-states ``` Also make sure the directory is ignored by git. ```text .agent-qa/auth-states/ ``` The state is scoped by target and logical name. The same logical name can exist for multiple environments because the target name is part of the internal resolution. ```yaml registry: targets: app-uat: platform: web url: https://uat.example.com app-staging: platform: web url: https://staging.example.com app-prod: platform: web url: https://app.example.com ``` If each target has an auth state called `qa-admin`, `use.authState: qa-admin` resolves to the state for the selected target. ## Capture from Live Mode Live Mode is the primary capture flow. ### Start a Live Mode web session Open the dashboard, start a web Live Mode session for the target, and let the browser open the product URL. ### Sign in manually Complete the product login flow as the QA account. This can include MFA, SSO, tenant selection, or any other interactive step that should not run in every test. ### Save auth state Use the Live Mode **Save auth state** action, enter the logical state name, and confirm replacement when updating an existing state. Live Mode writes the same Playwright-compatible storage-state JSON and metadata used by CLI capture. Saving auth state is an explicit action; normal product test runs do not update auth state. ## Capture from the CLI Use CLI capture when you want the same flow outside the dashboard. ```bash agent-qa auth-state capture --target app-staging --name qa-admin ``` The command opens a headed browser for the target URL. Sign in manually, then return to the terminal and press Enter to save. If the browser closes before confirmation, agent-qa does not save the state. The saved payload contains Playwright storage state with cookies, localStorage, and IndexedDB. SessionStorage is not part of the V1 contract. ## Run a test with auth state Select the auth state by logical name in a test. ```yaml test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila name: Verify billing dashboard target: app-staging use: authState: qa-admin steps: - Open the Billing page. - Verify the account menu is visible. - Verify the latest invoice table is visible. ``` `use.authState` is a per-test or per-suite capability grant. It is not a global default. A run can use exactly one primary auth state in V1. If the state is missing or unreadable, the run fails before creating the browser context and points you back to CLI capture. ## Run a suite with auth state For suites, put `use.authState` on the suite when every child test should share the same authenticated browser context. ```yaml suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis name: Billing smoke target: app-staging use: authState: qa-admin browser: name: chromium headless: true tests: - test: tests/billing-dashboard.yaml id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila - test: tests/billing-invoice-download.yaml id: t_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper ``` Child tests inherit the suite auth state when they omit `use.authState`. A child can repeat the same auth-state name, but a different child auth state is rejected because the suite has one shared authenticated runtime. ## Share the same test across environments Keep the test reusable by giving each environment the same logical auth-state name for its own target. ```yaml # tests/billing-dashboard.yaml name: Verify billing dashboard target: app-staging use: authState: qa-admin steps: - Open the Billing page. - Verify the account menu is visible. ``` Then run the same test against another configured target through the suite or CLI target selection used by your workflow. agent-qa resolves `qa-admin` inside the selected target's auth-state directory. Tests never reference auth-state file paths directly. ## Hook contract When setup, inline, or teardown hooks run inside an authenticated web test or suite, agent-qa exposes only the active auth state to that hook process. Hooks receive two environment variables: - `AGENT_QA_AUTH_STATE_JSON`: a structured JSON object with auth-state metadata and the raw storage-state path. - `AGENT_QA_AUTH_STATE_STORAGE_STATE_PATH`: direct path to the raw Playwright storage-state JSON. No auth-state environment variables are present when the run did not select an auth state. Hooks never receive all configured auth states. The JSON shape is stable and runtime-neutral. ```json { "version": 1, "kind": "web", "target": "app-staging", "name": "qa-admin", "capturedAt": "2026-05-17T15:06:00.000Z", "storageStatePath": "/workspace/.agent-qa-auth-state/storage-state.json" } ``` Hook authors parse the raw JSON themselves. Do not install Playwright inside hooks just to read auth state. ### Node ```js // scripts/read-auth-state.mjs import { readFile } from "node:fs/promises" const authStatePath = process.env.AGENT_QA_AUTH_STATE_STORAGE_STATE_PATH if (!authStatePath) { throw new Error("This hook requires use.authState") } const authState = JSON.parse(await readFile(authStatePath, "utf-8")) const sessionCookie = authState.cookies?.find((cookie) => cookie.name === "__session") const appOrigin = authState.origins?.find((origin) => origin.origin === "https://staging.example.com") const tenantId = appOrigin?.localStorage?.find((item) => item.name === "tenant_id")?.value if (!sessionCookie?.value) { throw new Error("Saved auth state did not contain the expected session cookie") } console.log(`Found tenant ${tenantId ?? "unknown"}`) ``` ### Bun ```js // scripts/read-auth-state.js const authStatePath = process.env.AGENT_QA_AUTH_STATE_STORAGE_STATE_PATH if (!authStatePath) { throw new Error("This hook requires use.authState") } const authState = JSON.parse(await Bun.file(authStatePath).text()) const sessionCookie = authState.cookies?.find((cookie) => cookie.name === "__session") const appOrigin = authState.origins?.find((origin) => origin.origin === "https://staging.example.com") const tenantId = appOrigin?.localStorage?.find((item) => item.name === "tenant_id")?.value if (!sessionCookie?.value) { throw new Error("Saved auth state did not contain the expected session cookie") } console.log(`Found tenant ${tenantId ?? "unknown"}`) ``` ### Python ```python # scripts/read_auth_state.py import json import os auth_state_path = os.environ.get("AGENT_QA_AUTH_STATE_STORAGE_STATE_PATH") if not auth_state_path: raise RuntimeError("This hook requires use.authState") with open(auth_state_path, "r", encoding="utf-8") as auth_state_file: auth_state = json.load(auth_state_file) session_cookie = next((cookie for cookie in auth_state.get("cookies", []) if cookie.get("name") == "__session"), None) app_origin = next((origin for origin in auth_state.get("origins", []) if origin.get("origin") == "https://staging.example.com"), {}) tenant_id = next((item.get("value") for item in app_origin.get("localStorage", []) if item.get("name") == "tenant_id"), None) if not session_cookie or not session_cookie.get("value"): raise RuntimeError("Saved auth state did not contain the expected session cookie") print(f"Found tenant {tenant_id or 'unknown'}") ``` ### Bash ```bash #!/usr/bin/env bash set -euo pipefail auth_state_path="${AGENT_QA_AUTH_STATE_STORAGE_STATE_PATH:-}" if [[ -z "$auth_state_path" ]]; then echo "This hook requires use.authState" >&2 exit 1 fi session_cookie="$(jq -r '.cookies[]? | select(.name == "__session") | .value' "$auth_state_path" | head -n 1)" tenant_id="$(jq -r '.origins[]? | select(.origin == "https://staging.example.com") | .localStorage[]? | select(.name == "tenant_id") | .value' "$auth_state_path" | head -n 1)" if [[ -z "$session_cookie" || "$session_cookie" == "null" ]]; then echo "Saved auth state did not contain the expected session cookie" >&2 exit 1 fi printf 'Found tenant %s\n' "${tenant_id:-unknown}" ``` ## Security boundary Auth state follows a narrower boundary than ordinary test variables. - Tests, suites, and hooks select auth state by logical name, not by filesystem path. - A hook receives only the selected active state for that authenticated run. - Auth-state payload contents and storage paths are excluded from run artifacts, dashboard run APIs, MCP run tools, logs, errors, and analytics by default. - Normal product runs are read-only with respect to auth state. - Capture and replacement require an explicit user action in Live Mode or the CLI. The auth-state management UI and metadata APIs can show target and name values because users need those values to manage states. They still do not return cookies, localStorage, IndexedDB, or raw storage-state paths. Do not echo the raw storage-state JSON from a hook. Redaction is best effort, and application cookies or tokens may not match values from `workspace.secretsFile`. ## Refresh stale state agent-qa does not try to prove that a saved session is still valid. Let the first product step validate the session. ```yaml steps: - Open the Dashboard page. - Verify the account menu for the QA user is visible. ``` If that step fails because the product redirected to login, refresh the state manually by running CLI capture again or saving from Live Mode with replacement. ## Native mobile boundary `use.authState` is web-only. For native Android and iOS apps, use `use.mobile.appState: preserve` when you want to keep installed app data between sessions. ```yaml target: issue-tracker-android use: device: android-local mobile: appState: preserve ``` App-state preservation is broader than auth. It can keep app data, caches, preferences, and other local state depending on the device and app lifecycle. agent-qa does not promise generic export of secure storage, keychain entries, shared preferences, app-private files, or native mobile auth tokens. Use mobile app-state preservation as the native-app fast path. If a product needs direct native token export, that is app-specific work and usually requires app cooperation, debug-only hooks, or a purpose-built test endpoint. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Web testing](https://vostride.com/docs/agent-qa/guides/web-testing.md) - Next: [Mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) --- ## Coding-agent verification loop URL: https://vostride.com/docs/agent-qa/guides/coding-agent-workflow Markdown: https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Turn a code change into a tested user journey, an evidence-backed diagnosis, and a focused rerun. Your coding agent already has the change, the repository and the product context. agent-qa gives it a repeatable way to check the resulting application: a reviewable test contract, live execution, step evidence and application memory. Complete the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) first if the runtime is not configured. ## Verify a change Start with the behavior the change is meant to deliver. Select an existing test when it covers that behavior; otherwise author the smallest new test that demonstrates it. Include important side effects and permissions when they are part of the requirement. A sequence of successful clicks can still end in the wrong outcome. Give your agent this task: ```text Verify my current change with agent-qa. Read the repository instructions and the diff, identify the affected user behavior, and reuse or extend the smallest relevant test. Work against the local or approved test environment. Validate the test definition, execute it, and inspect the completed run and step evidence. Preserve the expected behavior from the requirements. If the run fails, identify whether the problem is in the app, test, environment or model. Fix only issues within the scope I authorized and rerun the affected test. Report the actual outcome, evidence paths and exact rerun command. Do not call a dry run, queued job, interrupted run or missing credential a pass. ``` ## CLI path Run the project-local CLI through the repository's package manager. Target files explicitly, so an agent verifying one change does not launch every suite: ```bash npx agent-qa validate tests/app-smoke.yaml npx agent-qa run tests/app-smoke.yaml --headless --junit-output .agent-qa/app-smoke.junit.xml ``` Use `--dry-run` only to inspect which tests would execute. It does not drive the app or verify assertions. `--junit-output` supplies a standard result artifact for CI. Inspect the recorded outcome as well as the process exit status, especially when diagnosing setup or authentication failures. ## MCP path The public `https://vostride.com/mcp` service searches documentation and supplies the setup prompt. To run tests, connect to the **local** agent-qa MCP described in the [MCP reference](https://vostride.com/docs/agent-qa/mcp.md). Start the local dashboard-backed services when the tool needs execution state; opening their browser UI is optional. | Stage | Tools and evidence | | --- | --- | | Inspect the workspace | `agent_qa_discover`, `agent_qa_get_config`; reuse configured targets, models and paths. | | Author and validate | Generate canonical IDs, use the documented create/update tools, and validate the test or suite before running it. | | Enqueue execution | Use the installed server's execution tool and retain the returned job/run identifier. | | Wait for completion | Poll that same run while it is pending or running. A polling timeout does not prove the run stopped. | | Read the result | `agent_qa_get_run`, `agent_qa_get_run_steps`, `agent_qa_get_run_artifact` and logs. | | Diagnose | `agent_qa_classify_failure`; verify its hypothesis against artifacts and the app. | Discover the installed tool schemas rather than inventing parameters or assuming every release has identical tools. A classifier's confidence is supporting information, not proof of root cause. ## Diagnose and fix a failure ```text Investigate the failed agent-qa run I identify. Retrieve its completed run detail, failed step, screenshots and logs. Find the expected outcome and the earliest evidence explaining the failure. Classify it as an application, test-authoring, environment, authentication or model issue, and explain your evidence. If a fix is within my authorized scope, make the smallest relevant change, validate any changed test, and rerun it. Preserve the original failure and the new result. Do not delete assertions, substitute easier expectations or mark a skipped test as passing. If you cannot reproduce or execute the test, report that limitation. ``` The packaged `agent-qa-debug-fix` and `agent-qa-result-triage` [skills](https://vostride.com/docs/agent-qa/skills.md) contain the detailed workflow for these tasks. ## Keep verification meaningful agent-qa separates the execution runtime from the coding agent, but this alone does not enforce independent test authorship. A client granted test-mutation tools can edit tests. Review the test contract against product requirements, protect release acceptance tests where appropriate, and choose the client permissions your workflow requires. Self-healing finds another route to the same requested outcome. It does not authorize changing that outcome. Memory and caching reuse prior evidence and actions; the run still has to observe the app and check the current result. ## Reuse the next run Keep stable tests, configuration and useful application memory under the repository's normal review process. Preserve original and rerun identifiers in the agent's report. Record the app revision, environment and model so another teammate or agent can reproduce the result. Use [evaluation experiments](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) to measure whether memory and caching help your own workload. Faster execution is useful only when the assertions continue to catch regressions. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Skills](https://vostride.com/docs/agent-qa/skills.md) - Next: [Evaluate agent-qa on your app](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) --- ## Evaluate agent-qa on your app URL: https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa Markdown: https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Measure useful bug detection, repeated-run cost, and agent setup success with a reproducible experiment. agent-qa is a strong fit when you want natural-language end-to-end tests, web and native mobile support, repository-owned knowledge, and a CLI/MCP workflow that a coding agent can operate. The useful test is whether those capabilities improve verification on your application. ## Pick a representative journey Choose a flow with an observable business outcome: a saved setting, a completed checkout in a test environment, or an invitation with the expected permissions. Write the expected result before running the agent. Include the normal case and a controlled regression. Use a local fixture or isolated test environment. Keep the app revision, dataset, browser/device, test instructions, model version and execution settings in the report. Publish the fixture and commands when you can share them. Remove credentials and user data from artifacts before sharing. ## Measure correctness before speed | Measurement | What to record | | --- | --- | | Working application | How often the correct flow passes over repeated trials. | | Controlled regression | How often the broken behavior is detected. | | False pass | A broken flow reported as passing; inspect the assertion and evidence. | | False failure | A correct flow reported as failing; separate runtime, test and environment causes. | | Incomplete run | Authentication failures, cancellations and timeouts; keep these visible. | | Agent setup | Whether the agent completes setup from the public prompt, and what human input is needed. | Report counts and the number of trials, including unsuccessful attempts. Do not discard inconvenient runs or treat unexecuted tests as passes. Successful demonstrations show that a flow ran; they do not establish a general bug-detection rate. ## Compare repeated runs Use the same test and environment for each condition. Record total runtime, model requests, available token usage, cache hits and the outcome. Keep warm-up work separate from measured trials. ```bash # Normal configuration, with its configured memory and cache behavior npx agent-qa run tests/app-smoke.yaml --headless # Compare execution with each mechanism disabled npx agent-qa run tests/app-smoke.yaml --headless --no-cache npx agent-qa run tests/app-smoke.yaml --headless --no-memory npx agent-qa run tests/app-smoke.yaml --headless --no-cache --no-memory ``` These flags disable use on that run; they do not prove you started with an empty knowledge store. For a cold-start experiment, use a fresh isolated workspace with its own runtime and memory paths. Keep the app and test identical. Restore the same starting state between comparisons and vary the run order to reduce warm-up bias. The homepage's `42s → 8s` cache illustration explains the intended effect of skipping repeated planning. It is not a universal benchmark. Report your measured distribution, not a promised multiplier. If exact token accounting is unavailable from a provider, say so instead of estimating it as a measured value. ## Compare the right alternatives Compare agent-qa with the workflow you would actually use: maintained Playwright or Appium tests, a coding agent driving a browser directly, or a managed QA service. Hold the behavior and acceptance criteria constant. Include test maintenance effort and agent setup effort alongside execution cost. Existing deterministic tests remain useful. agent-qa does not replace unit tests, static checks, load testing or a dedicated security assessment. A browser-control tool may be sufficient for a one-off inspection. agent-qa adds durable test definitions, execution evidence, memory and reuse when the flow needs to be checked repeatedly. ## Publish a result others can check Include the fixture/revision, test YAML, exact commands, model and settings, trial counts, expected and observed outcomes, timing/token measurements, relevant artifacts and known limitations. The [recorded demonstrations](https://vostride.com/demo/agent-qa.md) are a place to inspect agent-qa's output before running your own experiment. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - Next: [Recorded test evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) --- ## Write your first test URL: https://vostride.com/docs/agent-qa/guides/first-test Markdown: https://vostride.com/docs/agent-qa/guides/first-test.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Create a source-grounded agent-qa test file with a target, context, optional hooks, per-test overrides, and natural-language steps. A test file describes one journey through one configured target. The required shape is small: `test-id`, `name`, `target`, and `steps`. Everything else is optional and should be added only when the run needs it. Use the [test configuration reference](https://vostride.com/docs/agent-qa/configuration/test.md) when you need every field. This page focuses on the first useful file. ## Create a target Tests point at a named target from `agent-qa.config.yaml`. A web target uses `platform: web` and must include `url`. ```yaml registry: targets: issue-tracker-web: platform: web url: http://localhost:3000 ``` ## Add a test file Create a YAML file under your test directory. The default workspace created by `agent-qa init` uses `tests/**/*.yaml`. ```yaml test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila name: Find a task by title target: issue-tracker-web context: | The QA user is already signed in. The Tasks page contains project work items for the current workspace. use: browser: name: chromium headless: true viewport: width: 1280 height: 720 timeout: step: 90s test: 10m logCapture: console: true network: true meta: timeout: 10m retries: 1 record: true steps: - Open the Tasks page. - Search for "billing review". - Verify the first matching task title is "Billing review". ``` `test-id` is a stable generated ID. Generate one with the dashboard or the CLI: ```bash npx agent-qa ids generate test ``` ## Add only the context the agent needs `context` is passed to the agent before the first step. Keep it about product state, user role, environment assumptions, and names the agent should recognize. ```yaml context: | The QA user is an admin in the Acme workspace. The product calls tasks "work items" in the sidebar. ``` Do not put credentials in `context`. Use env files, secrets, or hooks for runtime values that should not be written into prompts or artifacts. ## Use hooks when the run needs setup or cleanup `setup` and `teardown` contain hook IDs from your hook registry. Setup hooks run before steps. Teardown hooks run after the test finishes. ```yaml setup: - h_seed-task-workspace-bird-lake-slate-palm-cloud-frost teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed ``` Add hooks for data seeding, API assertions, or cleanup. Leave them out for a first browser-only smoke test. ## Write steps in plain language Steps can be plain strings: ```yaml steps: - Open the Tasks page. - Verify the Create task button is visible. ``` Use structured step objects when one instruction needs its own timeout, retry behavior, screenshot request, max-attempts budget, or capture rule. ```yaml steps: - step: Search for "billing review" and open the first matching task. timeout: 90s retries: 1 screenshot: true maxAttempts: 2 - step: Copy the task key from the task detail header. capture: variable: TASK_KEY method: regex pattern: "TASK-[0-9]+" - Verify the activity feed mentions "{{env:TASK_KEY}}". ``` `capture` writes a runtime variable that later steps can reference with `{{env:NAME}}`. Supported capture methods are `regex`, `selector`, and `ai`. ```yaml steps: - step: Read the task status badge. capture: variable: TASK_STATUS method: selector selector: "[data-testid='task-status']" - step: Identify the account name shown in the page header. capture: variable: ACCOUNT_NAME method: ai description: The visible account or workspace name in the header. ``` ## Run it Run a single test from the CLI: ```bash npx agent-qa run tests/find-task.yaml ``` The dashboard and CLI read the same file-backed definitions. Use the dashboard when you want the run timeline, screenshots, and agent reasoning while you tune the first test. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Recorded test evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) - Next: [Web testing](https://vostride.com/docs/agent-qa/guides/web-testing.md) --- ## Hooks URL: https://vostride.com/docs/agent-qa/guides/hooks Markdown: https://vostride.com/docs/agent-qa/guides/hooks.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Write sandboxed agent-qa hooks that prepare data, verify side effects, and export runtime variables back to tests and suites. Hooks are project scripts that agent-qa runs in a Docker sandbox. Use them when a test needs setup data, API verification, cleanup, or a runtime value that should be captured before a browser or mobile step runs. The full field reference lives in [Hook](https://vostride.com/docs/agent-qa/configuration/hook.md). This guide focuses on writing one hook and using it from tests or suites. ## Register a hook `workspace.hooksFile` points agent-qa at a hook registry, usually `hooks.yaml`. ```yaml workspace: hooksFile: hooks.yaml ``` The hook registry contains a `hooks` list. A hook needs an `id`, `name`, `runtime`, `file`, and `timeout`. `network` defaults to `true`; set it explicitly when the hook calls an external or local API. ```yaml hooks: - id: h_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper name: Fetch first Hacker News story runtime: node file: scripts/fetch-hn-top-story.mjs timeout: 30s network: true ``` Supported runtimes are `node`, `bun`, `python`, and `bash`. ## Fetch the Hacker News top story `agent-qa init` currently generates the Node version of this Hacker News hook for web projects. The Bun, Python, and Bash examples below implement the same hook contract: fetch top stories, validate the first story and title, then write `HN_FIRST_STORY_TITLE` and `HN_FIRST_STORY_ID` to `/tmp/agent-qa.env`. ### Node ```js // scripts/fetch-hn-top-story.mjs import { writeFile } from "node:fs/promises" const topStoriesUrl = "https://hacker-news.firebaseio.com/v0/topstories.json" async function getJson(url) { const response = await fetch(url) if (!response.ok) { throw new Error(`HN API request failed: ${response.status} ${response.statusText}`) } return response.json() } function escapeEnvValue(value) { return String(value) .replace(/\r?\n/g, " ") .replace(/\\/g, "\\\\") .replace(/"/g, '\\"') } const storyIds = await getJson(topStoriesUrl) const firstStoryId = Array.isArray(storyIds) ? storyIds[0] : undefined if (!Number.isInteger(firstStoryId)) { throw new Error("HN API returned no first story id") } const story = await getJson(`https://hacker-news.firebaseio.com/v0/item/${firstStoryId}.json`) const title = typeof story?.title === "string" ? story.title.trim() : "" if (!title) { throw new Error(`HN item ${firstStoryId} returned no title`) } await writeFile("/tmp/agent-qa.env", [ `HN_FIRST_STORY_TITLE="${escapeEnvValue(title)}"`, `HN_FIRST_STORY_ID=${firstStoryId}`, "", ].join("\n"), "utf-8") ``` ### Bun ```js // scripts/fetch-hn-top-story.js const topStoriesUrl = "https://hacker-news.firebaseio.com/v0/topstories.json" async function getJson(url) { const response = await fetch(url) if (!response.ok) { throw new Error(`HN API request failed: ${response.status} ${response.statusText}`) } return response.json() } function escapeEnvValue(value) { return String(value) .replace(/\r?\n/g, " ") .replace(/\\/g, "\\\\") .replace(/"/g, '\\"') } const storyIds = await getJson(topStoriesUrl) const firstStoryId = Array.isArray(storyIds) ? storyIds[0] : undefined if (!Number.isInteger(firstStoryId)) { throw new Error("HN API returned no first story id") } const story = await getJson(`https://hacker-news.firebaseio.com/v0/item/${firstStoryId}.json`) const title = typeof story?.title === "string" ? story.title.trim() : "" if (!title) { throw new Error(`HN item ${firstStoryId} returned no title`) } await Bun.write("/tmp/agent-qa.env", [ `HN_FIRST_STORY_TITLE="${escapeEnvValue(title)}"`, `HN_FIRST_STORY_ID=${firstStoryId}`, "", ].join("\n")) ``` ### Python ```python # scripts/fetch_hn_top_story.py import json import urllib.request TOP_STORIES_URL = "https://hacker-news.firebaseio.com/v0/topstories.json" def get_json(url): request = urllib.request.Request(url, headers={"User-Agent": "agent-qa-hook"}) with urllib.request.urlopen(request, timeout=20) as response: if response.status < 200 or response.status >= 300: raise RuntimeError(f"HN API request failed: {response.status}") return json.load(response) def escape_env_value(value): return str(value).replace("\r", " ").replace("\n", " ").replace("\\", "\\\\").replace('"', '\\"') story_ids = get_json(TOP_STORIES_URL) first_story_id = story_ids[0] if isinstance(story_ids, list) and story_ids else None if not isinstance(first_story_id, int): raise RuntimeError("HN API returned no first story id") story = get_json(f"https://hacker-news.firebaseio.com/v0/item/{first_story_id}.json") title = story.get("title", "").strip() if isinstance(story, dict) else "" if not title: raise RuntimeError(f"HN item {first_story_id} returned no title") with open("/tmp/agent-qa.env", "w", encoding="utf-8") as env_file: env_file.write(f'HN_FIRST_STORY_TITLE="{escape_env_value(title)}"\n') env_file.write(f"HN_FIRST_STORY_ID={first_story_id}\n") ``` ### Bash ```bash #!/usr/bin/env bash set -euo pipefail top_stories_url="https://hacker-news.firebaseio.com/v0/topstories.json" first_story_id="$(curl -fsSL "$top_stories_url" | jq '.[0]')" if ! [[ "$first_story_id" =~ ^[0-9]+$ ]]; then echo "HN API returned no first story id" >&2 exit 1 fi story_json="$(curl -fsSL "https://hacker-news.firebaseio.com/v0/item/${first_story_id}.json")" title="$(printf '%s' "$story_json" | jq -r '.title // ""')" if [[ -z "$title" || "$title" == "null" ]]; then echo "HN item ${first_story_id} returned no title" >&2 exit 1 fi escape_env_value() { printf '%s' "$1" | tr '\r\n' ' ' | sed 's/\\/\\\\/g; s/"/\\"/g' } { printf 'HN_FIRST_STORY_TITLE="%s"\n' "$(escape_env_value "$title")" printf 'HN_FIRST_STORY_ID=%s\n' "$first_story_id" } > /tmp/agent-qa.env ``` ## Use hooks from tests and suites Use `setup` when the hook should run before the test or suite starts. ```yaml setup: - h_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper steps: - Navigate to "https://news.ycombinator.com/" - Verify the page shows "{{env:HN_FIRST_STORY_TITLE}}" ``` Use `teardown` for cleanup hooks that should run after the test or suite. ```yaml teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed ``` Use inline `runHook` syntax when the hook belongs at one specific step. ```yaml steps: - Run the HN lookup hook {{runHook:"h_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper"}}. - Verify the first story id is "{{env:HN_FIRST_STORY_ID}}". ``` Inline hooks run before variable interpolation for that step. Variables exported by the hook are available to later steps. ## Export variables To return values to agent-qa, write a dotenv file at `/tmp/agent-qa.env`. ```dotenv HN_FIRST_STORY_TITLE="Launch HN: Example" HN_FIRST_STORY_ID=123456 ``` After the container exits, agent-qa reads that file and merges the variables into the run. A later successful hook can override a variable exported by an earlier hook. ## Add dependencies when needed `deps` copies extra files into the sandbox beside the hook entry file. `packageFile` copies a package file when the runtime needs package metadata. ```yaml hooks: - id: h_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper name: Fetch first Hacker News story runtime: node file: scripts/fetch-hn-top-story.mjs deps: - scripts/hn-client.mjs packageFile: package.json timeout: 30s network: true ``` Keep dependencies small. Files are copied into the temporary workspace for the hook run, not mounted from the repository. ## Read active web auth state Authenticated web runs can expose the selected auth state to setup, inline, and teardown hooks. See [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md) for the `AGENT_QA_AUTH_STATE_JSON` and `AGENT_QA_AUTH_STATE_STORAGE_STATE_PATH` contract, plus Bash, Python, Node, and Bun examples that parse the raw storage-state JSON without Playwright. ## Sandbox environment agent-qa runs hooks with Docker using the runtime image for the hook. - The hook entry file, `deps`, and optional `packageFile` are copied into a temporary host directory. - That directory is mounted at `/workspace`, and the hook command runs with `/workspace` as the working directory. - The container filesystem is read-only. - `/tmp` is writable and is where `/tmp/agent-qa.env` is written. - Resource limits are applied with `--memory 512m`, `--cpus 1`, and `--pids-limit 256`. - Environment variables from env files, CLI variables, previous successful hooks, and configured secrets are injected into the container. - Known secret values are redacted from hook stdout, stderr, and errors. Variables written to `/tmp/agent-qa.env` are filtered when their value exactly matches a known secret. Network access is enabled by default. Set `network: false` when the hook must not call the network. ```yaml hooks: - id: h_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper name: Validate exported HN variables offline runtime: bash file: scripts/validate-hn-env.sh timeout: 10s network: false ``` When `network: false` is set, agent-qa passes `--network none` to Docker for that hook container. ## Runtime images The current hook runner images are published under the `vostride` Docker Hub namespace: - [`vostride/agent-qa-hook-runner-node`](https://hub.docker.com/r/vostride/agent-qa-hook-runner-node) - [`vostride/agent-qa-hook-runner-bun`](https://hub.docker.com/r/vostride/agent-qa-hook-runner-bun) - [`vostride/agent-qa-hook-runner-python`](https://hub.docker.com/r/vostride/agent-qa-hook-runner-python) - [`vostride/agent-qa-hook-runner-bash`](https://hub.docker.com/r/vostride/agent-qa-hook-runner-bash) ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Suites](https://vostride.com/docs/agent-qa/guides/suites.md) - Next: [Dashboard](https://vostride.com/docs/agent-qa/dashboard.md) --- ## Mobile testing URL: https://vostride.com/docs/agent-qa/guides/mobile-testing Markdown: https://vostride.com/docs/agent-qa/guides/mobile-testing.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Configure native Android and iOS targets, local or BrowserStack devices, app installs, and required mobile app-state behavior. Mobile tests run against `android` or `ios` targets and a device profile. Native app tests must set `use.mobile.appState` to either `preserve` or `reset` so the runner knows whether to keep or reset installed app state. ## Configure a native app target Android targets use `appPackage` and usually `appActivity`. iOS targets use `bundleId`. ```yaml registry: targets: issue-tracker-android: platform: android appPackage: com.acme.issuetracker appActivity: .MainActivity app: path: apps/issue-tracker-debug.apk issue-tracker-ios: platform: ios bundleId: com.acme.issuetracker app: path: apps/IssueTracker.app ``` `app.path` must be relative to the config file. Keep large app binaries outside commits unless your repository intentionally stores test builds. ## Select a local device Local mobile runs use Appium and a device profile with `transport: local`. ```yaml registry: devices: android-local: platform: android transport: local match: avd: Pixel_8_API_35 automationName: UiAutomator2 ios-local: platform: ios transport: local match: udid: 00000000-0000-0000-0000-000000000000 automationName: XCUITest ``` Reference the device from the test with `use.device`: ```yaml test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila name: Open the mobile inbox target: issue-tracker-android use: device: android-local mobile: appState: reset steps: - Open the app. - Verify the inbox tab is visible. ``` Use `reset` when a test needs a clean app state. Use `preserve` when the test depends on an existing install, session, or prepared local state. ## Run on BrowserStack BrowserStack mobile runs use a device profile with `transport: browserstack`. Native app targets for BrowserStack must provide `app.browserstack`; local `app.path` is not used as a BrowserStack substitute. ```yaml registry: targets: issue-tracker-android: platform: android appPackage: com.acme.issuetracker appActivity: .MainActivity app: browserstack: bs://uploaded-app-id devices: android-browserstack: platform: android transport: browserstack match: deviceName: Google Pixel 8 osVersion: "14.0" ``` Then select that device in the test or with the CLI. ```yaml use: device: android-browserstack mobile: appState: preserve ``` ## Keep app state explicit The runner rejects native mobile tests that omit `use.mobile.appState`. Put it in the global config for one project-wide policy, in the suite for a group of tests, or in each test when flows need different behavior. ```yaml use: mobile: appState: reset ``` For suites, set one `use.device` when every child test should use the same mobile device. If the suite does not set a device, every child test must resolve to one device, and child tests cannot mix multiple devices in the same suite run. ## Run it Run a mobile test the same way you run a web test: ```bash npx agent-qa run tests/mobile-inbox.yaml ``` Use `--device` when you want to override the file-backed device selection for one run: ```bash npx agent-qa run tests/mobile-inbox.yaml --device android-browserstack ``` ## Auth state and mobile app state Web auth state is documented separately in [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md). Native Android and iOS runs do not use `use.authState`. For native apps, `use.mobile.appState: preserve` is the fast path when a test should reuse installed app data. This is broader than auth: it can preserve caches, preferences, and other app data. agent-qa does not export generic secure-storage, keychain, shared-preferences, app-private, or native mobile token data. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md) - Next: [Suites](https://vostride.com/docs/agent-qa/guides/suites.md) --- ## Recorded test evidence URL: https://vostride.com/docs/agent-qa/guides/recorded-evidence Markdown: https://vostride.com/docs/agent-qa/guides/recorded-evidence.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Inspect the recorded web and Android test results behind the agent-qa demo. These are recorded, sanitized agent-qa demonstrations against real interfaces. They show the journeys exercised and the evidence the runtime produces. Open a test to inspect its steps, screenshots, and available artifacts in the [run inspector](https://vostride.com/demo/agent-qa.md). This is a curated set of passing runs, not a benchmark, reliability estimate, or claim of endorsement by the products tested. It does not measure false passes or compare agent-qa with another tool. Follow the [evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) to test correctness and cost on your own application. The tables count step records in each saved test. Duration is the test record's execution duration, rounded to milliseconds; it is not total setup time, suite wall-clock time, or a prediction for another run. Dates are UTC. Values reflect the saved recordings and are not live checks of these products. ## Supabase Platform: **web**. Recorded: **2026-05-10**. | Recorded test | Verdict | Passed steps / total | Recorded duration | | --- | --- | --- | --- | | [Supabase login](/demo/run/supabase-login) | passed | 5 / 5 | 98.745 s | | [Create Supabase Hacker News table](/demo/run/supabase-create-table) | passed | 9 / 9 | 190.314 s | | [Insert and verify Hacker News story row](/demo/run/supabase-insert-row) | passed | 6 / 6 | 83.283 s | ## Linear Android Platform: **android**. Recorded: **2026-05-10**. | Recorded test | Verdict | Passed steps / total | Recorded duration | | --- | --- | --- | --- | | [Linear Android create urgent issue and complete it](/demo/run/linear-android-issue-lifecycle) | passed | 15 / 15 | 184.758 s | ## Polymarket Platform: **web**. Recorded: **2026-05-11**. | Recorded test | Verdict | Passed steps / total | Recorded duration | | --- | --- | --- | --- | | [Polymarket quote prices and win amounts match live API values](/demo/run/polymarket-quote-values) | passed | 13 / 13 | 257.861 s | ## PostHog Platform: **web**. Recorded: **2026-05-10**. | Recorded test | Verdict | Passed steps / total | Recorded duration | | --- | --- | --- | --- | | [PostHog login](/demo/run/posthog-login) | passed | 7 / 7 | 90.280 s | | [Create and verify PostHog feature flag](/demo/run/posthog-feature-flag) | passed | 7 / 7 | 140.792 s | | [Capture and verify PostHog demo event](/demo/run/posthog-event) | passed | 2 / 2 | 33.312 s | ## Clerk Platform: **web**. Recorded: **2026-05-10**. | Recorded test | Verdict | Passed steps / total | Recorded duration | | --- | --- | --- | --- | | [Clerk login](/demo/run/clerk-login) | passed | 5 / 5 | 92.933 s | | [Create Clerk user and verify through API](/demo/run/clerk-create-user) | passed | 12 / 12 | 115.698 s | | [Add Clerk organization access and verify in dashboard](/demo/run/clerk-org-access) | passed | 7 / 7 | 68.810 s | ## Hacker News Platform: **web**. Recorded: **2026-05-11**. | Recorded test | Verdict | Passed steps / total | Recorded duration | | --- | --- | --- | --- | | [Hacker News public top story check](/demo/run/hacker-news-top-story) | passed | 9 / 9 | 104.014 s | ## Try it on your own app Use the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) and its complete setup prompt to have a coding agent install agent-qa, write a test tied to a real requirement, and report the observed outcome. Keep a known failing case in your evaluation: a convincing pass needs evidence that the test can detect a regression too. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Evaluate agent-qa on your app](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) - Next: [Write your first test](https://vostride.com/docs/agent-qa/guides/first-test.md) --- ## Suites URL: https://vostride.com/docs/agent-qa/guides/suites Markdown: https://vostride.com/docs/agent-qa/guides/suites.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Combine multiple agent-qa tests into one ordered suite with shared target, context, hooks, and run overrides. A suite runs multiple test files as one workflow. Use suites for smoke checks, release gates, and multi-step regression flows where the tests should share setup, cleanup, target selection, or run defaults. For the complete schema reference, see [Suite](https://vostride.com/docs/agent-qa/configuration/suite.md). ## Create a suite file ```yaml suite-id: s_far-gnu-mean-junk-aga-visual-knife-lend-few-vis name: Issue tracker release smoke target: issue-tracker-web context: | Run against the browser target configured as issue-tracker-web. The QA user is already signed in to the Acme workspace. setup: - h_seed-web-workspace-calm-cedar-lamp-river-field teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed use: browser: name: chromium headless: true viewport: width: 1280 height: 720 timeout: navigation: 30s step: 90s test: 20m cache: false tests: - test: tests/task-create.yaml id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila - test: tests/task-complete.yaml id: t_rag-ember-rye-lamp-calm-track-sage-fair-yam-nix ``` `suite-id` is optional but useful for stable run identity. Generate one with: ```bash npx agent-qa ids generate suite ``` ## What the suite owns The suite schema supports: - `suite-id`: optional generated suite ID. - `name`: human-readable suite name. - `target`: target name from `registry.targets`. - `context`: shared background passed to each child test. - `setup`: hook IDs that run before suite tests. - `teardown`: hook IDs that run after suite execution. - `use`: suite-level run overrides. - `tests`: ordered child test entries. The `tests` list is required and must contain at least one entry. ## Tests list and identity Each child entry has a path and the expected test ID inside that file. ```yaml tests: - test: tests/task-create.yaml id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila ``` The runner loads each file, parses the test definition, and checks that the file's `test-id` matches the suite entry `id`. A mismatch fails the suite before that child test runs. Tests run in the order listed. If a child test fails, later children are marked skipped for the suite run. ## Target and overrides When `target` is set on the suite, that target overrides each child test target for the suite run. This lets the same test files run against a shared web, Android, or iOS target in different suites. ```yaml target: issue-tracker-web ``` Suite `use` values act as shared defaults. Child test `use` values can still override them. ```yaml use: browser: name: chromium headless: true ``` There is no separate field named `overrides` in suite YAML. Overrides are expressed through `target` and `use`. ## Precedence Config files are loaded before runtime overrides. ```txt config file -> env overrides -> CLI flags ``` Within test execution, `use` values are layered from broadest to most specific: ```txt global -> suite -> test -> CLI flags ``` Nested objects are deep-merged. For example, a global viewport can remain in effect while a suite changes only `browser.name`, and a child test changes only `browser.headless`. ```yaml # agent-qa.config.yaml use: browser: name: chromium headless: true viewport: width: 1280 height: 720 # suite use: browser: name: firefox # child test use: browser: headless: false ``` The child test keeps the inherited viewport, uses `firefox`, and runs with a visible browser. For keys that have CLI flags, flags win for that run. For example, `--device` selects the mobile device for the suite run. ## Setup and teardown Suite `setup` hooks run before the platform adapter is set up and before any child test starts. Variables exported by successful suite setup hooks are available to the suite target URL interpolation and to every child test. Per-test setup hooks run before that child test's steps. Per-test teardown hooks run after that child test finishes. Suite `teardown` hooks run after suite execution and receive the accumulated suite variables. ```yaml setup: - h_seed-web-workspace-calm-cedar-lamp-river-field teardown: - h_update-borg-artha-any-packet-derive-torch-front-plied-bed ``` Use suite hooks for shared records, login state, or cleanup that belongs to the whole group. Use test hooks when each child needs its own data. ## Mobile suites Mobile suites use the same suite shape. The suite target can be `android` or `ios`, and `use.mobile.appState` must resolve to `preserve` or `reset`. ```yaml target: issue-tracker-android use: device: android-local mobile: appState: reset tests: - test: tests/mobile-inbox.yaml id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila ``` If the suite does not set `use.device`, every child test must resolve to one device. Child tests in the same mobile suite cannot mix multiple device names unless the suite itself selects the device. ## Run a suite Run one suite file directly: ```bash npx agent-qa run suites/release-smoke.suite.yaml ``` Or run discovered suites from your workspace config: ```bash npx agent-qa run --suite ``` ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - Next: [Hooks](https://vostride.com/docs/agent-qa/guides/hooks.md) --- ## Web testing URL: https://vostride.com/docs/agent-qa/guides/web-testing Markdown: https://vostride.com/docs/agent-qa/guides/web-testing.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Configure web targets and browser overrides for agent-qa tests that run through Playwright-managed Chromium, Firefox, or WebKit. Web tests run against a target with `platform: web`. The target supplies the base `url`; the test supplies the journey. For the complete file-backed config shape, see [Global Config](https://vostride.com/docs/agent-qa/configuration/global-config.md) and [Test](https://vostride.com/docs/agent-qa/configuration/test.md). ## Configure a web target `url` is required for every web target. ```yaml registry: targets: issue-tracker-web: platform: web url: http://localhost:3000 ``` Point `url` at the environment you want the agent to open: a local dev server, preview URL, staging app, or production-safe test tenant. ## Set browser defaults Global `use` values apply to tests unless a suite, test, or CLI flag overrides them. ```yaml use: browser: name: chromium headless: true viewport: width: 1280 height: 720 timeout: navigation: 30s step: 90s test: 10m logCapture: console: true network: true ``` Supported browser names are `chromium`, `firefox`, and `webkit`. ## Override browser behavior per test Use per-test `use` when one flow needs a different engine, visible browser, viewport, timeout, or log capture behavior. ```yaml test-id: t_vog-earing-wap-git-tim-assert-teras-mill-aus-ila name: Verify responsive task filters target: issue-tracker-web use: browser: name: webkit headless: false viewport: width: 390 height: 844 timeout: navigation: 60s step: 2m logCapture: console: true network: false steps: - Open the Tasks page. - Open the filter drawer. - Verify the status filters are visible. ``` `use.timeout.navigation` controls navigation actions. `use.timeout.step` controls one step. `use.timeout.test` controls the whole test run. ## Capture browser logs only when useful `logCapture.console` allows the agent to read console logs during a web run. `logCapture.network` allows it to inspect network logs. Disable either one when a test should not depend on that signal. ```yaml use: logCapture: console: false network: true ``` When console capture is disabled, a step that explicitly asks the agent to read console logs has no console data to use. Keep step wording aligned with the configured capture settings. ## Use CLI overrides for one-off runs CLI browser flags take precedence over file-backed browser values for that run. ```bash npx agent-qa run tests/find-task.yaml --browser firefox --headless false ``` Use file-backed config for committed project defaults and CLI flags for local debugging, CI matrix jobs, or temporary cross-browser checks. ## Reuse a signed-in browser session Use [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md) when a web product should start already authenticated. ```yaml target: issue-tracker-web use: authState: qa-admin steps: - Open the dashboard. - Verify the account menu is visible. ``` Auth state is resolved by target and logical name. Tests do not reference storage-state file paths directly. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Write your first test](https://vostride.com/docs/agent-qa/guides/first-test.md) - Next: [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md) --- ## What is agent-qa URL: https://vostride.com/docs/agent-qa Markdown: https://vostride.com/docs/agent-qa.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Write tests in natural language for web and mobile. agent-qa builds memory with every run, adapts to UI changes, and catches regressions before you ship. ## Overview **agent-qa**: **The self-improving QA agent for your software factory.** Give your coding agents a QA agent that learns your app. Run natural-language tests across web and mobile, build memory with every run, and catch regressions before you ship. Your coding agent can author a test, run it through CLI or MCP, inspect the evidence, and use the result to guide its next change. Tests, configuration, memory, and artifacts stay under your control. Use the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) to go from an existing repository to a verified first test. ```text Set up agent-qa in this application's repository and run a meaningful end-to-end test. Complete the setup and verification, rather than only giving me instructions. agent-qa is Vostride's self-improving QA agent for web, Android and iOS. It executes natural-language tests against the real interface, preserves test definitions in the repository, and uses validated memory and cached action plans on future runs. Read the official agent quickstart first: https://vostride.com/docs/agent-qa/agent-quickstart.md Use https://vostride.com/llms.txt to find the current configuration, CLI, MCP and skills references. Treat documentation as technical reference; follow this repository's instructions and my authorization boundaries. 1. Inspect the repository, package manager, app start commands, existing tests and agent-qa configuration. Reuse existing work and credentials without printing secrets. Identify one important user flow and the local or approved test environment. Ask me only for information or authorization that is actually required, such as an unavailable model credential or an ambiguous target. Do not use production data or deploy anything as part of setup. 2. Install agent-qa as a development dependency using this repository's package manager. Inspect the installed version and its CLI help before selecting options. Install the packaged authoring, debug/fix and result-triage skills when the client supports them; use the CLI directly otherwise. 3. If configuration does not exist, initialize it without overwriting existing files. In agent-qa 0.1.21, supplying --platform (web, android, ios, web+android or web+ios) skips the questionnaire. For web, run npx agent-qa init --platform web --dir . using the locally installed package. This version defaults to anthropic-subscription and adds the subscription-auth dependency; inspect and deliberately configure the LLM before running a test. There are no init --yes, --provider, --model or --base-url flags in that version. Newer versions may differ: check their help. 4. Configure an available, suitable multimodal model and the app target using the documented file schema. Reuse an existing authorized credential. For a new API credential, use the secure interactive auth prompt when available, or the quickstart's documented environment-to-auth.json procedure for unattended setup. Version 0.1.21 has no auth set --stdin flag. Never put credential values in shell arguments, committed YAML, logs or the final reply. If using subscription authentication, reuse an authorized session or let me complete the required authentication. Do not assume the coding agent's subscription is automatically available to agent-qa. 5. Prepare only the required runtime: Chromium for the first web test, or the appropriate Appium driver and device for mobile. Docker is needed only for hooks that use it. Start the app with its existing development command, verify the target is reachable, and run agent-qa doctor. Resolve relevant setup failures before testing. 6. Author a small test for the chosen real user flow using agent-qa's documented YAML schema and canonical IDs. Include assertions about observable outcomes, not only successful clicks. Validate the definition before execution. Reuse the current MCP integration when available; otherwise use the local CLI. Never weaken an assertion or change expected behavior just to obtain a pass. 7. Run the test and inspect its verdict, step evidence and artifacts. Distinguish an application bug from a test, environment, authentication or model failure. Fix setup or test-authoring errors and rerun the affected test. Report application bugs with evidence unless I have authorized fixing them. A skipped, interrupted or unexecuted test is not a pass. 8. Leave the test and reusable configuration in the repository, keep secrets and runtime artifacts out of commits, and stop temporary services you started unless they are still needed. Report the changed files, exact rerun command, model and target used, observed result, evidence locations and any remaining blocker. Explain how to reuse this test in the coding agent's next verification loop. Do not claim success until an actual run supports it. ``` - [Agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md): Give your coding agent the complete setup task: initialize without menus, configure a model, and run a meaningful test. - [Explore demos](https://vostride.com/demo/agent-qa.md): Open curated agent-qa runs in the interactive demo viewer and inspect how real test steps execute. - [Configuration](https://vostride.com/docs/agent-qa/configuration.md): Review the file-backed configuration system for global settings, tests, suites, hooks, variables, secrets, devices, rules, and auth. - [Memory](https://vostride.com/docs/agent-qa/memory.md): Review how file-backed product, suite, and test observations make future runs more product-aware. - [Caching](https://vostride.com/docs/agent-qa/caching.md): Reuse validated action plans while the screen still matches, and measure the runtime and token savings on your own tests. - [Tests](https://vostride.com/docs/agent-qa/guides/first-test.md): Write one plain-English YAML journey with stable intent, reviewable steps, and source-controlled expectations. - [Web Testing](https://vostride.com/docs/agent-qa/guides/web-testing.md): Run browser flows against local or staged web apps without hardcoded selectors or fragile DOM coupling. - [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md): Capture a named web login once, reuse it by logical name, pass the active state to hooks, and keep credential material out of artifacts. - [Suites](https://vostride.com/docs/agent-qa/guides/suites.md): Bundle related tests, targets, configuration, and hooks into repeatable workflows for a feature area. - [Mobile Testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md): Use the same natural-language workflow model for mobile app journeys while preserving artifacts locally. - [Hooks](https://vostride.com/docs/agent-qa/guides/hooks.md): Prepare data, call project scripts, verify side effects, and clean up state before or after a run. ## How agent-qa works ### Write tests in natural language Describe the user journey in plain English: what to open, what to do, and what to assert. agent-qa turns that intent into repeatable execution. ### Bring your own LLM Use any hosted or local model. It also works with Codex and Claude Code subscriptions. ### Run tests Run tests from either the dashboard or the CLI. Group multiple tests into a suite to execute user journeys end-to-end. ### Generate memory and context Each run passes through a memory curator that records product behavior, test intent, and suite context, then creates contextual memory for future test runs. ### Improve the next run Future executions use the evolved agent context. That memory helps the agent adapt to product changes at runtime, recover from brittle UI drift, and become more reliable over time. ### Grow a product-aware QA agent The end state is a QA agent with enough context about your product to author tests, run them, inspect failures, and keep improving the testing loop. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Next: [Agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) --- ## MCP URL: https://vostride.com/docs/agent-qa/mcp Markdown: https://vostride.com/docs/agent-qa/mcp.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Use agent-qa MCP tools to discover project context, author tests and suites, enqueue runs, inspect artifacts, and classify failures from an AI agent. The local agent-qa MCP server exposes tools for AI coding agents to author and run tests. Start with the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) if the workspace or model is not configured yet. ## Public documentation MCP This website also serves a separate, public documentation MCP at **`https://vostride.com/mcp`**, using Streamable HTTP with no authentication. It provides: | Capability | Name | Result | | --- | --- | --- | | Tool | `search_agent_qa_docs` | Search documentation and return matching pages. | | Tool | `read_agent_qa_page` | Retrieve a documentation page as Markdown. | | Prompt | `setup_agent_qa` | Retrieve the complete onboarding task, including a real test and evidence review. | Connect that URL using your client's remote MCP configuration. Clients supporting MCP prompts can discover `setup_agent_qa` with `prompts/list` and retrieve it with `prompts/get`. The identical prompt is available at [/agent-setup.txt](/agent-setup.txt) for clients that only fetch URLs. Transport and capabilities are advertised in the [server card](/mcp/server-card). The public service reads documentation. It cannot run tests, read your repository, or authenticate your model. The local execution server described below uses your installed agent-qa workspace and configured services. ## Local execution MCP Start the stdio server from the [CLI](https://vostride.com/docs/agent-qa/cli.md): ```bash agent-qa mcp ``` The CLI attempts to load agent-qa config so analytics can use workspace settings. If config loading fails, the MCP startup path can still continue and start the server without loaded analytics config. ## Choose a transport Use the dashboard-backed HTTP endpoint for day-to-day agent work. It starts with the dashboard, binds to loopback by default, and pre-wires dashboard context for run, artifact, queue, and authoring tools. ```bash agent-qa dashboard --port 3470 --open ``` With the default service config, the MCP endpoint is: ```text http://127.0.0.1:3471/mcp ``` Use stdio when your client only supports local subprocess MCP servers, when you want a lightweight schema/discovery surface, or when you are integrating agent-qa into a client config that launches tools on demand. Stdio clients must pass `dashboardUrl` to dashboard-backed tools. When the stdio server connects, agent-qa writes a startup diagnostic to stderr: `agent-qa MCP server running over stdio. Waiting for MCP client messages on stdin. Stdout is reserved for MCP protocol traffic.` Keep stdout reserved for MCP protocol messages; route human-readable startup logs and client diagnostics through stderr or the client UI. ```yaml services: mcp: enabled: true transport: http host: 127.0.0.1 port: 3471 path: /mcp ``` ## Dashboard URL requirement Some MCP tools are local schema or discovery tools. Test, suite, hook, run, artifact, and triage tools are dashboard-backed and need a dashboard API base URL. Provide `dashboardUrl` to the tool call, or start MCP through a dashboard-backed path that supplies the dashboard URL. The server error is explicit: `dashboardUrl is required for authoring tools. Start MCP via agent-qa dashboard or pass dashboardUrl.` Use the [Dashboard](https://vostride.com/docs/agent-qa/dashboard.md) when an agent needs live run state, file-manager APIs, artifacts, logs, or queue cancellation. ## Client setup Install the `agent-qa` package in the project first, or make sure the `agent-qa` binary is available on your `PATH`. ### Codex CLI Use the dashboard HTTP endpoint when the dashboard is running: ```bash codex mcp add agent-qa --url http://127.0.0.1:3471/mcp codex mcp list ``` Use stdio when Codex should launch the MCP server itself: ```bash codex mcp add agent-qa -- agent-qa mcp codex mcp list ``` ### Claude Code Use the dashboard HTTP endpoint for the full dashboard-backed tool surface: ```bash claude mcp add --transport http agent-qa http://127.0.0.1:3471/mcp claude /mcp ``` Use stdio when Claude Code should launch `agent-qa mcp` as a local subprocess: ```bash claude mcp add agent-qa -- agent-qa mcp claude /mcp ``` ### OpenCode OpenCode can add MCP servers interactively: ```bash opencode mcp add opencode mcp list ``` For a project-local dashboard HTTP endpoint, add this to `opencode.json`: ```json { "$schema": "https://opencode.ai/config.json", "mcp": { "agent-qa": { "type": "remote", "url": "http://127.0.0.1:3471/mcp", "enabled": true } } } ``` For stdio, use a local MCP server entry: ```json { "$schema": "https://opencode.ai/config.json", "mcp": { "agent-qa": { "type": "local", "command": ["agent-qa", "mcp"], "enabled": true } } } ``` ### Other MCP clients For clients that accept a Streamable HTTP URL, point them at `http://127.0.0.1:3471/mcp` after starting the dashboard. For clients that only accept stdio servers, configure command `agent-qa` with args `["mcp"]`. If your client uses an `mcpServers` object, the stdio shape is usually: ```json { "mcpServers": { "agent-qa": { "command": "agent-qa", "args": ["mcp"] } } } ``` Always check the client docs for the exact HTTP field name. Some clients use `url`, some use `serverUrl`, and some require a helper process for remote endpoints. ## Verify the connection After registering the server, ask your agent to call `agent_qa_discover`. A healthy response should include tool names, schema resources, prompt metadata, endpoint metadata, and config path metadata. Then run a narrow dashboard-backed check: 1. Start `agent-qa dashboard --port 3470`. 2. Call `agent_qa_get_config`. 3. If using stdio, pass `dashboardUrl: "http://localhost:3470"` when calling authoring, run, artifact, or triage tools. 4. If using the dashboard HTTP MCP endpoint, the MCP server supplies dashboard context automatically. ## Discovery, config, schema, and IDs These tools do not require dashboard file APIs: - `agent_qa_discover`: returns MCP tools, resources, prompts, endpoint, dashboard, and config path metadata. - `agent_qa_get_config`: reads active config and returns masked raw config or a targets/devices/providers summary. - `agent_qa_schema_reference`: returns structured references for config, test YAML, suite YAML, hooks, or canonical IDs. - `agent_qa_validate_definition`: validates parsed config, test, suite, or hooks objects with agent-qa schemas. - `agent_qa_generate_id`: generates canonical IDs for test, suite, hook, run, or observation entities. - `agent_qa_validate_id`: validates canonical IDs and returns the expected id contract. ## Test, suite, and hook tools Use MCP authoring tools before editing YAML directly. They route through the dashboard workspace-safe APIs and still require `dashboardUrl`. Tests: - `agent_qa_list_tests` - `agent_qa_read_test` - `agent_qa_validate_test` - `agent_qa_create_test` - `agent_qa_update_test` - `agent_qa_delete_test` Suites: - `agent_qa_list_suites` - `agent_qa_read_suite` - `agent_qa_validate_suite` - `agent_qa_create_suite` - `agent_qa_update_suite` - `agent_qa_delete_suite` Hooks: - `agent_qa_list_hooks` - `agent_qa_read_hook` - `agent_qa_create_hook` - `agent_qa_update_hook` - `agent_qa_delete_hook` - `agent_qa_run_hook` ## Run and triage tools Use these dashboard-backed tools to enqueue work, inspect run evidence, and classify failures: - `agent_qa_enqueue_test_run`: queues a test run through `/api/runs/trigger`. - `agent_qa_enqueue_suite_run`: queues a suite run through `/api/runs/trigger`. - `agent_qa_get_run`: returns run detail, steps, attempts, and suite child context. - `agent_qa_get_run_steps`: returns run steps. - `agent_qa_get_run_logs`: returns run log rows with optional step, level, source, limit, and offset filters. - `agent_qa_get_run_execution_logs`: returns structured execution logs. - `agent_qa_get_run_artifact`: returns sanitized run artifacts, child artifacts, and missing artifact sections. - `agent_qa_cancel_run`: cancels a pending or running run through the dashboard queue. - `agent_qa_classify_failure`: classifies a failed run using run detail, artifacts, logs, execution logs, and recent related runs. ## Resources and prompt Schema resources are available at: ```text agent-qa://schema/{schema} ``` Current schema names are `config`, `test`, `suite`, `hooks`, and `ids`. The MCP prompt `agent_qa_authoring_context` gives agents the expected authoring sequence: inspect config, generate canonical IDs, validate definitions, and prefer MCP tools before file edits. ## Practical workflow 1. Call `agent_qa_discover`. 2. Call `agent_qa_get_config` to inspect targets, devices, providers, and dashboard settings. 3. Use `agent_qa_generate_id` for new IDs. 4. Validate definitions with `agent_qa_validate_definition`, `agent_qa_validate_test`, or `agent_qa_validate_suite`. 5. Create or update through MCP authoring tools when `dashboardUrl` is available. 6. Enqueue a run with `agent_qa_enqueue_test_run` or `agent_qa_enqueue_suite_run`. 7. Inspect evidence with run, logs, execution logs, and artifact tools. 8. Classify failures with `agent_qa_classify_failure` before deciding whether to edit tests, hooks, app code, or infrastructure. ## Security notes Keep local MCP endpoints on loopback hosts: `127.0.0.1`, `localhost`, or `::1`. Treat MCP access as workspace access: an enabled agent can read config, inspect artifacts, enqueue runs, mutate tests and suites through dashboard APIs, and classify failures using local evidence. Review any client-level auto-approval settings before enabling mutation tools. For shared projects, prefer project-local client config so each repository declares the exact agent-qa endpoint it expects. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [CLI](https://vostride.com/docs/agent-qa/cli.md) - Next: [Skills](https://vostride.com/docs/agent-qa/skills.md) --- ## Behavioral Memory Schema URL: https://vostride.com/docs/agent-qa/memory/behavioral-memory-schema Markdown: https://vostride.com/docs/agent-qa/memory/behavioral-memory-schema.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Read and review the markdown observation files that back agent-qa behavioral memory. Behavioral Memory is the reader-facing name for the observation files agent-qa stores on disk. Each observation is one markdown file named after its observation ID. The file has YAML frontmatter for metadata, followed by markdown body text for the observation itself. ## File path Observation files are stored under one of three tiers: ```txt agent-qa-memory/products//obs_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper.md agent-qa-memory/suites//obs_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper.md agent-qa-memory/tests//obs_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper.md ``` The file name and frontmatter `id` must match. ## Base fields Every observation includes these fields: - `id`: observation ID that starts with `obs_`. - `title`: short summary used in search and injected context. - `trust`: number from `0` to `1`. - `created`: ISO datetime for when the observation was created. - `last_confirmed`: ISO datetime for the latest confirmation or update. - `confirmed_count`: number of times evidence confirmed the observation. - `contradicted_count`: number of times evidence contradicted the observation. - `source_test`: test ID or source name that produced the observation. The markdown body is the observation content. Keep the body concise, factual, and safe to inject as context. ## Suite fields Suite observations add: - `position`: zero-based index of the child test in the suite. - `suite_snapshot`: ordered list of suite entries, each with `test` and `id`. The snapshot prevents a suite memory from being reused after the suite membership or order changes. ## Sample memory file ```md --- id: obs_aster-bloom-cloud-drift-ember-field-glade-hollow-ivory-jasper title: Hacker News first story title appears in the first row trust: 0.55 created: "2026-05-14T12:00:00.000Z" last_confirmed: "2026-05-14T12:10:00.000Z" confirmed_count: 1 contradicted_count: 0 source_test: t_quad-adar-micro-magic-cross-cue-open-agog-rang-cours position: 0 suite_snapshot: - test: tests/hacker-news-top-story.yaml id: t_quad-adar-micro-magic-cross-cue-open-agog-rang-cours --- On the Hacker News homepage, the title returned as the first item from the Firebase topstories API appears in the first story row. ``` The sample includes suite fields because it represents a suite-position observation. Product and test observations use the same base fields without `position` or `suite_snapshot`. ## Validation rules The parser requires frontmatter delimiters and a non-empty markdown body. If the frontmatter ID does not match the file name, the observation is ignored. The body text is intentionally outside the frontmatter so long observations stay readable in markdown review. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Memory overview](https://vostride.com/docs/agent-qa/memory.md) - Previous: [Curator](https://vostride.com/docs/agent-qa/memory/curator.md) - Next: [Runtime Memory Injection](https://vostride.com/docs/agent-qa/memory/runtime-injection.md) --- ## Curator URL: https://vostride.com/docs/agent-qa/memory/curator Markdown: https://vostride.com/docs/agent-qa/memory/curator.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Learn how the memory curator turns run evidence into added, confirmed, deprecated, deleted, or unchanged observations. The curator runs after test execution and decides whether the run produced useful behavioral memory. It is selective: memory should capture product behavior that helps future runs, not generic testing tactics or obvious page trivia. ## A.U.D.N. decisions The curator asks for A.U.D.N. decisions: - `add`: write a new behavioral observation. - `update`: confirm an existing observation that was relevant and correct. - `deprecate`: penalize an existing observation that the run contradicted. - `noop`: leave memory unchanged. The implementation records `update` decisions as confirmation deltas in the memory log. `noop` decisions do not write files. ## What gets added New observations start with trust `0.5`. ```txt new observation trust = 0.5 confirmed_count = 0 contradicted_count = 0 ``` The curator chooses a scope: - product scope for structural behavior that helps future tests across the product - suite scope for behavior tied to a suite sequence or position - test scope for behavior specific to one test Suite observations include the suite position and suite snapshot so they can be matched safely later. ## Confirmation and deprecation When the curator confirms an observation, trust increases by `trustConfirmDelta`, `last_confirmed` is updated, and `confirmed_count` increases by one. When the curator deprecates an observation, trust decreases by `trustContradictDelta` and `contradicted_count` increases by one. If trust reaches zero, agent-qa deletes the observation file instead of keeping a zero-trust memory entry. ## Failed runs For failed runs, agent-qa does not ask the curator to add new observations. It looks at observations injected into the failed step and deprecates those observations because they may have contributed to the bad run. If ablation later proves that memory caused a failure, the same deprecation path is used for all injected observations from that run. ## Suite cleanup Suite observations are tied to a `suite_snapshot`. After a suite run, the curator scans suite observations for that suite. If an observation's snapshot no longer matches the current suite entries, agent-qa deletes that stale suite observation. This prevents a memory from one suite order from being reused after tests are inserted, removed, renamed, or reordered. ## Curator lock The local provider uses a `.curator.lock` file under the memory root. The lock serializes writes so concurrent runs do not update or delete the same observation at the same time. `curatorLockTimeout` controls how long a run waits for the lock. A stale lock can be removed when the owning process is gone or the timestamp is old enough. ## Security checks Before a new or updated observation is written, agent-qa scans the title and body with the memory security scanner. Unsafe observation text is blocked and recorded as a curator error instead of being written to disk. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Memory overview](https://vostride.com/docs/agent-qa/memory.md) - Previous: [Dashboard](https://vostride.com/docs/agent-qa/dashboard.md) - Next: [Behavioral Memory Schema](https://vostride.com/docs/agent-qa/memory/behavioral-memory-schema.md) --- ## Memory overview URL: https://vostride.com/docs/agent-qa/memory Markdown: https://vostride.com/docs/agent-qa/memory.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Understand agent-qa memory as file-backed behavioral observations that make future runs more product-aware without replacing live evidence. Memory is the flagship differentiator in agent-qa. It lets the agent learn product behavior from previous runs, then bring that context back into future steps as reviewable evidence instead of hidden model state. agent-qa memory is file-backed. Observations are markdown files under a memory root, defaulting to `agent-qa-memory`, and are organized by `products`, `suites`, and `tests`. ## The lifecycle Memory works step by step: 1. A run starts with a product, test ID, and sometimes a suite ID plus suite position. 2. agent-qa builds an in-memory index from matching product, suite, and test observation files. 3. Before a step runs, the current step text is used to query the index. 4. Matching observations are injected as `` for that step. 5. The agent observes the live app and executes the step. 6. After the run, the curator reviews the result and decides whether to add, update, deprecate, or do nothing. The important boundary is that memory is contextual evidence, not a command channel. It helps the agent remember product behavior, but the current page, app state, logs, and test instructions still decide the run. ## File-backed tiers Memory lives in three tiers: ```txt agent-qa-memory/ products/ issue-tracker/ suites/ s_hill-gant-verb-nast-hunter-rita-home-store-amy-crest/ tests/ t_quad-adar-micro-magic-cross-cue-open-agog-rang-cours/ ``` Product memories apply broadly to a product target. Suite memories apply to a suite and, when position data is present, to a specific child test position. Test memories apply to one test ID. This file layout keeps memory inspectable. You can review diffs, remove stale observations, and understand why the agent saw a memory entry. ## What the curator does The curator is the post-run process that turns run evidence into memory changes. It can: - add a new observation when the run reveals useful product behavior - update an existing observation when new evidence confirms or refines it - deprecate an observation when the run contradicts it - do nothing when the run did not produce a useful memory change The curator writes markdown files through the memory provider and uses trust scores to keep unreliable observations from dominating future steps. ## Memory, cache, and config Memory is not the action cache. Cache reuses execution-level action results when that is safe. Memory stores behavioral observations about the product, suite, or test. Memory is also not static configuration. Configuration says how to run. Memory says what previous runs observed. Use configuration for known facts such as targets, browsers, mobile devices, LLMs, hooks, and timeouts. Use memory for facts that evolve from test execution, such as a product label, a common workflow outcome, or a suite-specific dependency between child tests. ## Where to go next - [Curator](https://vostride.com/docs/agent-qa/memory/curator.md) explains how observations are added, updated, deprecated, or left alone. - [Behavioral Memory Schema](https://vostride.com/docs/agent-qa/memory/behavioral-memory-schema.md) documents the markdown file format. - [Runtime Memory Injection](https://vostride.com/docs/agent-qa/memory/runtime-injection.md) explains how matching observations reach an agent step. - [Memory Reliability and Maintenance](https://vostride.com/docs/agent-qa/memory/reliability-maintenance.md) covers trust thresholds, ablation, circuit breaker behavior, stale suite snapshots, security scanning, and locks. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Quickstart](https://vostride.com/docs/agent-qa/quickstart.md) - Next: [Caching](https://vostride.com/docs/agent-qa/caching.md) --- ## Memory Reliability and Maintenance URL: https://vostride.com/docs/agent-qa/memory/reliability-maintenance Markdown: https://vostride.com/docs/agent-qa/memory/reliability-maintenance.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Configure memory trust, injection limits, curator locking, ablation, circuit breaker behavior, security scanning, and stale suite cleanup. Memory is useful only when it remains trustworthy. agent-qa combines trust scores, injection limits, curator review, security scanning, ablation, circuit breaker behavior, suite snapshot matching, and file locks to keep memory maintainable. ## services.memory Memory settings live under `services.memory`. ```yaml services: memory: enabled: true provider: local dir: agent-qa-memory minTrust: 0.3 maxInjections: 3 curatorEnabled: true curatorLockTimeout: 120000 trustConfirmDelta: 0.05 trustContradictDelta: 0.10 ablationEnabled: true circuitBreakerEnabled: true circuitBreakerWindowSize: 20 circuitBreakerBaselineSize: 3 circuitBreakerThreshold: 0.15 ``` `enabled` controls memory for the run. `provider` is currently `local`. `dir` sets the memory root; relative paths resolve from the config directory. ## Retrieval controls `minTrust` is the minimum trust an observation needs before it can be injected. `maxInjections` limits how many observations can be injected into one step. The local provider defaults to `0.3` minimum trust and `3` injected observations. Keeping those values conservative helps memory stay useful without overwhelming the step. ## Curator controls `curatorEnabled` controls post-run memory curation. When it is disabled, memory can still be queried, but the run does not create or update observations through the curator. `curatorLockTimeout` controls the `.curator.lock` wait time for file-backed writes. `trustConfirmDelta` increases trust for confirmed observations. `trustContradictDelta` reduces trust for contradicted observations. When deprecation pushes trust to zero, the observation is deleted. ## Ablation `ablationEnabled` controls the retry path for failures that used injected memory. When a run fails and at least one observation was injected, agent-qa can retry without memory. If the retry passes, memory is treated as the likely cause and injected observations are deprecated. If the retry also fails, the failure is not attributed to memory. ## Circuit breaker `circuitBreakerEnabled` controls run-level protection against harmful memory. The circuit breaker tracks outcomes with and without memory. It waits until both baseline and memory-backed samples reach `circuitBreakerBaselineSize`. Then it compares failure rates inside the `circuitBreakerWindowSize` window. If the memory-backed failure rate exceeds baseline by more than `circuitBreakerThreshold`, the breaker trips and memory injection is skipped for remaining steps. ## Suite snapshot maintenance Suite observations include `position` and `suite_snapshot`. Runtime injection uses both fields to avoid applying a suite memory to the wrong child test. After suite curation, agent-qa also scans suite observations and deletes stale entries whose stored `suite_snapshot` no longer matches the current suite. ## Security scanning Before local memory writes an observation, security scanning checks the title and body. Suspicious instruction-changing text, secret-reading commands, invisible control characters, and similar risky content are blocked before the observation reaches disk. Security scanning also runs while indexing memory. Unsafe observations are skipped instead of being injected into a step. ## Manual maintenance Because memory is file-backed markdown, maintenance can be done with normal repository tools: - inspect observation diffs during review - delete obsolete `obs_*.md` files - lower trust by editing an observation only when your team intentionally manages memory by hand - keep suite observations aligned with suite membership and order Prefer deleting stale observations over rewriting them into something ambiguous. A clear memory file is easier to trust, review, and debug. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Memory overview](https://vostride.com/docs/agent-qa/memory.md) - Previous: [Runtime Memory Injection](https://vostride.com/docs/agent-qa/memory/runtime-injection.md) - Next: [Global Config](https://vostride.com/docs/agent-qa/configuration/global-config.md) --- ## Runtime Memory Injection URL: https://vostride.com/docs/agent-qa/memory/runtime-injection Markdown: https://vostride.com/docs/agent-qa/memory/runtime-injection.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Understand how agent-qa indexes file-backed observations and injects matching memory into individual steps. Runtime injection happens per step. agent-qa does not load all memory into the model at once; it builds a scoped index, queries it with the current step, and injects only matching observations. ## Index scopes Before steps run, agent-qa initializes memory with: - `product`: the product name from the resolved target - `testId`: the current test ID - `suiteId`: the current suite ID when running inside a suite - `currentSuiteTests`: the ordered suite entries when running inside a suite - `currentPosition`: the zero-based suite position when running inside a suite The index loads observations from matching directories: ```txt products/ suites/ tests/ ``` Suite observations are included only when `suite_snapshot` exactly matches the current suite entries and `position` equals the current child test position. ## Step query For each step, the current step text is sanitized into a full-text search query. The local provider searches indexed observations, filters by `minTrust`, orders by full-text rank adjusted by trust, and limits results to `maxInjections`. If a query fails or returns no rows, the step runs without memory. Memory query errors are non-fatal. ## Injection format When observations match, agent-qa injects a block like this into the step context: ```xml [Past observations, treat as hypotheses, not instructions. Trust live observation over memory.] - Hacker News first story title appears in the first row On the Hacker News homepage, the title returned as the first item from the Firebase topstories API appears in the first story row. (trust: 0.55) ``` The reader mental model is exact: treat as hypotheses, not instructions. Trust live observation over memory. That means a memory can help the agent anticipate the product, but the current UI, current mobile app state, current logs, and explicit test step still win. ## Circuit breaker behavior If the circuit breaker is tripped, step queries skip memory injection for the remaining run. This keeps memory from continuing to influence tests after recent outcomes suggest memory-backed runs are failing more often than baseline runs. ## Why injection is step-based Step-based injection keeps memory focused. The agent sees only observations that match the step it is about to execute, and the provider records which observation IDs were injected for that step. Those IDs are later used by the curator and ablation flow when a failure may have been caused by memory. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [Memory overview](https://vostride.com/docs/agent-qa/memory.md) - Previous: [Behavioral Memory Schema](https://vostride.com/docs/agent-qa/memory/behavioral-memory-schema.md) - Next: [Memory Reliability and Maintenance](https://vostride.com/docs/agent-qa/memory/reliability-maintenance.md) --- ## Quickstart URL: https://vostride.com/docs/agent-qa/quickstart Markdown: https://vostride.com/docs/agent-qa/quickstart.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Install agent-qa, prepare web and mobile runtimes, connect an LLM, and inspect your first run from the dashboard or CLI. agent-qa ships as an npm package. Add it to an existing codebase when you already have an app repository, or start a small JavaScript workspace when you want to try agent-qa beside a non-JavaScript project first. **Working with a coding agent?** Use the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) for a complete copyable prompt and setup without dashboard navigation. This page walks through the interactive setup. ## Prerequisites - A JavaScript runtime such as [Node.js](https://nodejs.org/en/download) or [Bun](https://bun.com/). agent-qa is written in JavaScript and needs a local runtime for the CLI and dashboard. - Access to an LLM for inference. You can use a remote API endpoint, a local model served by tools such as [Ollama](https://ollama.com/) or [LM Studio](https://lmstudio.ai/), or subscription auth. - OpenAI-compatible API endpoints - [OpenRouter](https://openrouter.ai/docs) through its OpenAI-compatible API endpoint - Anthropic-compatible API endpoints - Gemini models - Codex or Claude Code subscriptions through the optional subscription auth plugin Use a multimodal model for the normal quickstart. Web and mobile runs inspect screenshots, so text-only models are not a good fit for visual QA workflows. - Docker is optional, but recommended. agent-qa can run JavaScript, Python, Bash, and Bun hooks inside an isolated Docker runtime. ## Install agent-qa and prepare the environment agent-qa runs independently from the application under test, so you can install it in JavaScript, Rails, Django, Laravel, Go, Java, Swift, Kotlin, web, or mobile repositories. You only need enough Node.js tooling to install and run the CLI. ```bash node --version npm --version ``` Skip this step when your repository already has a `package.json`. Otherwise, create one before installing agent-qa: ```bash npm init -y ``` Install `agent-qa` as a dev dependency so every teammate and CI job can run the same version. **Install agent-qa** ```bash # npm npm install -D agent-qa # pnpm pnpm add -D agent-qa # yarn yarn add -D agent-qa # bun bun add -d agent-qa ``` If you want to use Codex or Claude Code subscription auth instead of provider API keys, install the optional [subscription auth package](https://github.com/vostride/agent-qa-subscription-auth). **Install subscription auth support** ```bash # npm npm install -D @vostride/agent-qa-subscription-auth # pnpm pnpm add -D @vostride/agent-qa-subscription-auth # yarn yarn add -D @vostride/agent-qa-subscription-auth # bun bun add -d @vostride/agent-qa-subscription-auth ``` ## Set up the testing environment Let's set up the test environment by installing browser runtimes for web and the relevant platform tools for mobile. ### Web browsers Install browser runtimes for web tests. These agent-qa-managed browsers do not replace or interfere with browsers you already have installed. **Install browser support** ```bash # npm npx agent-qa install-browsers --all # pnpm pnpm exec agent-qa install-browsers --all # yarn yarn agent-qa install-browsers --all # bun bunx agent-qa install-browsers --all ``` ### Mobile drivers For Android or iOS tests, install the Appium runtime first: ```bash npm install -g appium appium --version ``` Then install the relevant mobile drivers: **Install mobile driver support** ```bash # npm npx agent-qa install-mobile-drivers --all # pnpm pnpm exec agent-qa install-mobile-drivers --all # yarn yarn agent-qa install-mobile-drivers --all # bun bunx agent-qa install-mobile-drivers --all ``` You also need the developer platform tools: - Android: install [Android Studio or Android SDK platform tools](https://developer.android.com/studio), then set up an emulator or connect a real device. - iOS: install [Xcode and command line tools](https://developer.apple.com/xcode/), then set up an iOS simulator or connect a real device. ### Hook runtime Hooks run in an isolated Docker environment. Install Docker from Docker's [Get Started page](https://www.docker.com/get-started/), start Docker Desktop or the Docker daemon, then confirm the CLI can reach it: ```bash docker --version docker info ``` You only need Docker for tests or suites that use hooks. If your first run does not use hooks, you can set it up later. ## Initialize agent-qa Run the init command to scaffold the config files, local settings, sample tests, and hook examples. **Initialize agent-qa** ```bash # npm npx agent-qa init # pnpm pnpm dlx agent-qa init # yarn yarn dlx agent-qa init # bun bunx agent-qa init ``` ### Verify the environment Run the doctor command after initialization to validate the local runtime pieces before your first test run. **Check agent-qa setup** ```bash # npm npx agent-qa doctor # pnpm pnpm exec agent-qa doctor # yarn yarn agent-qa doctor # bun bunx agent-qa doctor ``` The generated workspace usually looks like this. Exact sample file names can vary by version, but the shape is stable: project config at the root, optional hook scripts, suites, and tests. Select a file to inspect the generated content. > The generated workspace includes project and local configuration, agent rules, tests, suites, hook definitions, and optional hook scripts. Exact sample names can vary by package version. Keep generated run artifacts out of commits unless your team intentionally stores them. ## Open the dashboard Start the local dashboard from the project root. **Open dashboard** ```bash # npm npx agent-qa dashboard --open # pnpm pnpm exec agent-qa dashboard --open # yarn yarn agent-qa dashboard --open # bun bunx agent-qa dashboard --open ``` The `--open` command opens the agent-qa dashboard in your default browser. Before running your first test, connect the LLM model agent-qa should use. On the dashboard: 1. Go to `Config` > `LLM`. 2. Add an LLM configuration. 3. Choose the provider or subscription auth mode. 4. Test the connection. 5. Go to `Config` > `Execution Defaults` and select the new LLM configuration. 6. Save the config. The dashboard writes back to your local config files, so review the diff the same way you would review any other project configuration change. Model secrets, such as API keys or auth tokens, are stored in `~/.agent-qa/auth.json`. ## Run your first test from the dashboard Use the generated sample before writing a custom test. 1. Go to `Tests`. 2. Select `Example passing test`. 3. Click `Run` or press `R`. By default, runs execute in headless mode. Disable headless mode in the execution settings when you want to watch the browser or mobile session directly. While the test is running, the live view shows the active execution. After the run completes, open the run view and inspect the full timeline: - what the agent observed before each step - how it planned the next action - what it actually executed, such as clicking a button or filling an input - how it verified the result - how each assertion was evaluated, including the reasoning behind the pass or failure This view is the fastest way to learn whether a failure came from product behavior, test wording, environment setup, or model interpretation. ## Run your first test from the CLI agent-qa is designed to work with teams at scale. Run the same tests from CI, release jobs, or post-deploy checks to catch regressions before they reach users. For CI, start with a narrow command that targets the tests you trust, then expand to suites as coverage grows: ```bash npx agent-qa run tests/example-pass.yaml ``` The dashboard and CLI share the same file-backed definitions and run artifact storage. Use the dashboard for rich local debugging and the CLI for repeatable automation. Feel free to reach out if you face any issues. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [Agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md) - Next: [Memory overview](https://vostride.com/docs/agent-qa/memory.md) --- ## Skills URL: https://vostride.com/docs/agent-qa/skills Markdown: https://vostride.com/docs/agent-qa/skills.md > [Documentation index](https://vostride.com/docs.md) · [Site index](https://vostride.com/llms.txt) > Follow the Markdown links to read individual pages; the indexes list the available documentation. Practical guide to packaged agent-qa skills and the CLI command that lists them for AI agents. agent-qa packages reusable agent instructions for authoring, debugging, and result triage. List them from the [CLI](https://vostride.com/docs/agent-qa/cli.md): ```bash agent-qa skills ``` For machine-readable output: ```bash agent-qa skills --json ``` The JSON output returns the resolved skills directory. When you inspect or manually audit installed files, review whole skill directories, not only `SKILL.md`, because packaged skills can include references and client-specific agent metadata. ## Skill directory resolution ### Website onboarding skill This website also distributes **agent-qa-setup**, a focused skill for taking an existing app repository through installation, model authentication, a meaningful first test, and verification. It is maintained in the website repository and is separate from the three skills in the agent-qa npm package. Download [agent-qa-setup.zip](/.well-known/agent-skills/agent-qa-setup.zip), inspect its `SKILL.md`, and extract it into an `agent-qa-setup` folder in the project skill directory supported by your client. The [skill index](/.well-known/agent-skills/index.json) includes its SHA-256 digest alongside the packaged skills. Do not overwrite an existing local skill without reviewing the changes. For a one-off setup without installing a skill, copy the complete prompt from the [agent quickstart](https://vostride.com/docs/agent-qa/agent-quickstart.md). ### Packaged skills The `agent-qa skills` command resolves skills in this order: 1. `package`: the package root has a `skills` directory with skill subdirectories containing `SKILL.md`. 2. `source`: source checkout fallback at `../../skills` from the package root. 3. `missing`: no usable packaged or source skills directory was found. When skills are missing, non-JSON output writes an error like `agent-qa skills not found at ...` and sets exit code 1. JSON output prints a payload with `path`, `source`, and empty `skills`, then also sets exit code 1. ## Install skills into agent clients Use the cross-agent Skills CLI to install the packaged agent-qa skills from the public repository: ```bash npx skills add vostride/agent-qa --skill agent-qa-authoring --skill agent-qa-debug-fix --skill agent-qa-result-triage ``` To install every packaged agent-qa skill: ```bash npx skills add vostride/agent-qa --skill '*' ``` The package currently includes `agent-qa-authoring`, `agent-qa-debug-fix`, and `agent-qa-result-triage`. Run `agent-qa skills --json` when you want to inspect the packaged skills that shipped with your installed `agent-qa` version; use `npx skills add` when you want the client install step handled for you. ### Agent-specific installs Install for one or more detected agents: ```bash npx skills add vostride/agent-qa --skill '*' --agent codex npx skills add vostride/agent-qa --skill '*' --agent claude-code npx skills add vostride/agent-qa --skill '*' --agent opencode npx skills add vostride/agent-qa --skill '*' --agent codex --agent claude-code --agent opencode ``` Use `--global` for a user-wide install, or omit it for a project install: ```bash npx skills add vostride/agent-qa --skill '*' --agent codex --global npx skills add vostride/agent-qa --skill '*' --agent claude-code --global npx skills add vostride/agent-qa --skill '*' --agent opencode --global ``` Use `--copy` if your environment should copy skill files instead of symlinking them, and `--yes` for non-interactive setup: ```bash npx skills add vostride/agent-qa --skill '*' --agent codex --global --copy --yes ``` Codex global skills still load from `$CODEX_HOME/skills`, which is usually `~/.codex/skills`; project installs use `.agents/skills/`. Claude Code uses `.claude/skills` for project installs and `~/.claude/skills` globally. OpenCode can use `.agents/skills`, `.opencode/skills`, or `~/.config/opencode/skills` depending on scope and client configuration. Claude Code can invoke skills automatically from the description, or directly with `/agent-qa-authoring`, `/agent-qa-debug-fix`, and `/agent-qa-result-triage`. Codex and OpenCode can invoke them by name, such as `$agent-qa-authoring` or `$agent-qa-result-triage`, once their client has loaded installed skills. To keep skill use explicit in OpenCode, allow the skill tool in `opencode.json`: ```json { "$schema": "https://opencode.ai/config.json", "permission": { "skill": { "agent-qa-*": "allow" } } } ``` ### Cross-client project install Use `.agents/skills` when a repository is shared by Codex, OpenCode, and other clients that support the neutral agent-compatible location. Use `.claude/skills` when a repository is primarily shared by Claude Code users. The `npx skills add` installer can target those agents directly, so prefer it over hand-copying directories. ## `agent-qa-authoring` Use `agent-qa-authoring` when creating, editing, validating, or running tests, suites, or hooks. The skill prefers [MCP tools](https://vostride.com/docs/agent-qa/mcp.md) before CLI or YAML fallback: - Discover the local surface with `agent_qa_discover`. - Inspect config with `agent_qa_get_config`. - Generate IDs with `agent_qa_generate_id` or fallback to `agent-qa ids generate `. - Validate definitions with `agent_qa_validate_definition`, `agent_qa_validate_test`, or `agent_qa_validate_suite`. - Prefer `agent_qa_create_test`, `agent_qa_update_test`, `agent_qa_create_suite`, `agent_qa_update_suite`, and hook mutation tools before editing files directly. Use CLI/YAML fallback only when MCP tools are unavailable, and keep files inside configured workspace patterns. ## `agent-qa-debug-fix` Use `agent-qa-debug-fix` after a failed run when a code, YAML, hook, infrastructure, or product issue needs a patch. The workflow starts from evidence: - `agent_qa_get_run` - `agent_qa_get_run_steps` - `agent_qa_get_run_artifact` - `agent_qa_get_run_logs` - `agent_qa_get_run_execution_logs` - `agent_qa_classify_failure` Treat the classifier result as a hypothesis, inspect local files directly, apply the smallest change that explains the evidence, validate edited YAML, and rerun the narrowest affected test, suite, hook, or unit test. ## `agent-qa-result-triage` Use `agent-qa-result-triage` when the goal is evidence-backed classification rather than an immediate patch. The skill gathers run detail, steps, artifacts, logs, and execution logs, then calls `agent_qa_classify_failure`. It returns category, confidence, evidence, likely fix area, and next action. If code changes are needed, hand off to `agent-qa-debug-fix` after triage is complete. Fixed categories include `timeout`, `appium_startup`, `browser_disconnect`, `element_not_found`, `assertion_failure`, `hook_failure`, `infrastructure`, and `unknown_failure`. ## Use with MCP and CLI Prefer MCP tools for live project state and dashboard-backed authoring. Use CLI fallback for simple local checks such as `agent-qa validate`, `agent-qa ids validate --json`, or `agent-qa skills --json` when MCP is unavailable. ## Updating installed skills After upgrading `agent-qa`, run `npx skills update` or rerun the matching `npx skills add vostride/agent-qa ...` command. Replace or update the old directories as a unit so `SKILL.md`, `references/`, and `agents/` stay in sync. Review the files before enabling auto-approval in any agent client. Skills are instructions plus optional supporting files, and they should be treated as part of your local automation surface. ## Documentation navigation - [All documentation](https://vostride.com/docs.md) - Parent: [What is agent-qa](https://vostride.com/docs/agent-qa.md) - Previous: [MCP](https://vostride.com/docs/agent-qa/mcp.md) - Next: [Coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) --- # Use Cases --- ## AI browser testing that repairs the path, not the outcome URL: https://vostride.com/ai-browser-testing A web suite is the first thing a redesign breaks, because it is the only suite pinned to markup somebody just rewrote. agent-qa runs browser tests as plain-English steps against the live page and recovers from a moved control inside the same run. When no recovery reaches the step you asked for, the run fails and tells you what it found. ### The suite a redesign always breaks Every suite pays for change. The browser suite pays first. A component library upgrade renames classes, a modal becomes a page, a checkout gains a step, and none of that changes what a customer does. All of it changes the DOM your selectors were written against. The tests go red, somebody spends a morning on locators, and the web suite becomes the thing nobody wants to own. agent-qa holds no selectors, because it never generates a test script. Playwright is the execution kernel: agent-qa decides on each action from what is on the screen at that moment and hands the kernel a click, a fill, or a check. Nothing is recorded, so nothing goes stale when the markup does. The file below is a complete test. It names a target, gives the runtime some context, and lists seven sentences. There is no class, no id, no data-testid, and no wait anywhere in it, which is why it still runs on a layout its author never opened. ```yaml test-id: t_checkout-smoke name: Guest checkout completes target: storefront-web context: | The storefront runs at the URL configured by the storefront-web target. Product data is seeded by the workspace setup hook. steps: - Open the home page and search for "espresso grinder". - Open the first search result. - Add the product to the cart. - Start checkout as a guest. - Fill the shipping form with the workspace test address. - Place the order. - Verify the confirmation page shows an order number. ``` ### A recovery has to reach the step you asked for Self-healing is the claim every tool in this category makes, and the fair objection to all of them is the same: a runtime that keeps trying until something goes green is a runtime that hides bugs. agent-qa draws the line at the step. When a sub-action fails, because the click was intercepted, the control moved, or the field will not take the value, it re-observes the page and plans another route to the same step, inside the same run. What it will not do is find a different step. Take a step that asks for a $128.00 payment to be refunded in full. The refund dialog opens and reports that $35.60 of it already went back, so the button on offer refunds the remaining $92.40. Pressing that button would end the step successfully and leave the customer short. agent-qa declines it, declines the same amount typed into the field by hand, and fails the run with the reason it found. The attempt that failed stays in the trace either way. A healed step is marked healed and carries the sub-action that did not work, so a step that needed two tries never renders as a step that passed first time. That is the part most self-healing gives up, and it is the part that decides whether a green suite is evidence or decoration. ### Chromium, Firefox, and WebKit are a setting, not a second suite Cross-browser coverage usually means a parallel suite or a matrix nobody maintains. Here the engine is one value in the test file. All three engines run through the same kernel, so pointing an existing flow at another one costs a setting rather than a rewrite, and the steps never mention which browser they are in. The same block carries headless, viewport, and timeouts, plus console and network capture for the flows that need those logs. A workspace default covers the suite, one test overrides it, and the --browser and --headless CLI flags beat both for a single run, which is what a CI matrix job or a local debugging pass actually wants. A web run can also start signed in. Auth state is resolved by target and logical name, so a test asks for the qa-admin session rather than pointing at a storage-state file on disk. ```yaml test-id: t_checkout-smoke target: storefront-web use: browser: name: webkit headless: true viewport: width: 1440 height: 900 logCapture: console: true network: false authState: qa-admin ``` ### FAQ **What is AI browser testing?** A runtime drives a real browser from steps written as sentences instead of from scripted selectors. An agent-qa web test names a target and lists what the user does. The runtime observes the page, decides on each action, hands it to Playwright, and checks the result. Nothing is recorded, so nothing goes stale when the markup changes. **How is self-healing different from retrying until a test passes?** The recovery is bounded by the step. agent-qa will re-observe the page and take a different path to the outcome the step named, and it will decline a path that reaches a different outcome even when that path ends in a green button. A run with no path to what was asked fails and reports what it found, and the failed attempt stays in the trace so a healed step is never reported as a clean pass. **Can one test run on Chromium, Firefox, and WebKit?** Yes. The engine is a value in the test's use.browser block. A workspace default applies until a suite, a test, or the --browser CLI flag overrides it, and the flag wins for that run. Because the steps never name the engine, a cross-engine check is a setting on a test you already have. **How is agent-qa different from Playwright or Cypress?** agent-qa runs on Playwright, so this is not one browser runtime against another. A Cypress or hand-written Playwright suite is code you author and maintain: every locator, wait, and assertion is yours. agent-qa treats the runtime as an execution kernel and decides the actions itself while the run is happening. Teams usually keep scripted tests for the paths they want pinned exactly and move the flows that keep breaking over to agent-qa. **What does a failed web run give me to debug with?** Step-level artifacts: screenshots, the sub-actions that ran and their timings, any healing attempt and the reason for it, and a failure classification. Browser console and network capture are per-test settings, so a flow that needs those logs can have them and a flow that should not depend on them can turn them off. The local dashboard shows every step and sub-action of a run. ### Related resources - [Web testing guide](https://vostride.com/docs/agent-qa/guides/web-testing.md) - [Auth state](https://vostride.com/docs/agent-qa/guides/auth-state.md) - [Run dashboard](https://vostride.com/docs/agent-qa/dashboard.md) - [agent-qa vs Playwright](https://vostride.com/playwright-alternative.md) - [agent-qa vs Selenium](https://vostride.com/selenium-alternative.md) --- ## Natural language testing that is a file format, not a prompt URL: https://vostride.com/natural-language-testing An agent-qa test says what should happen and what proves it happened. The runtime reads the live interface and works out how to get there, so no step is pinned to a selector. The sentences sit inside a schema: a test ID, a target, context, hooks, and interpolated values, in YAML your team reviews like any other file. ### A step names the behavior, not the element Each step is one sentence. It names a control the way a person would say it out loud, and it names the outcome that should follow: 'Click New issue.' 'Verify the Issues table contains a row titled Fix mobile checkout.' The file never says which node that is, because it does not have to. agent-qa resolves the sentence against the live interface using visible roles, labels, and screen state, at the moment the step runs. Nothing is compiled in between. There is no recording and no generated script sitting between your app and the test, so there is no second artifact to keep in sync with the first. The sentence is the part you maintain, and it is also the part a reviewer reads. ### The structure around the sentences Plain English sounds imprecise until you see the rest of the file. A test declares a test-id, a name, and a target, plus optional context that tells the run what it is looking at before step one. Hooks run before the steps, after them, or at one exact point in the middle, in a sandboxed container. Values are interpolated rather than typed in: a URL arrives as {{env:LINEAR_URL}}, a credential as {{secret:API_TOKEN}}, and a fixture a hook just created arrives the same way. Because it is a schema, it can be checked. agent-qa validate reads your config, your test files, and the suite references between them before a browser ever opens. What lands in the diff is a contract: the behavior in the steps, and everything that has to be exact in the fields around them. ```yaml test-id: t_checkout-smoke name: Guest checkout completes target: storefront-web context: | The storefront runs at the URL configured by the storefront-web target. Product data is seeded by the workspace setup hook. steps: - Open the home page and search for "espresso grinder". - Open the first search result. - Add the product to the cart. - Start checkout as a guest. - Fill the shipping form with the workspace test address. - Place the order. - Verify the confirmation page shows an order number. ``` ### The same file, whoever wrote it A product manager can write the steps for a flow they specified. An engineer can add the target and the hooks. A QA engineer can tighten the assertion. A coding agent can write the whole file from the context it already holds about your product. The output is identical in each case: reviewable YAML in the repository, opened in a pull request beside the feature it covers. That works because there is no authoring tool to learn. No recorder, no vendor editor, no machine-only format. You create a file, run it with one command, and read the run back in a local dashboard. Anyone who can describe a user journey can add coverage, and everyone else can review what they added. ```bash # initialize a workspace npx agent-qa init # run a test npx agent-qa run tests/checkout-smoke.yaml # inspect runs in the local dashboard agent-qa dashboard --port 3470 --open ``` ### FAQ **What is natural language testing?** Natural language testing writes each step as a sentence about the product rather than as code against the page. In agent-qa a step names a control a user can see and the outcome that should follow, and the runtime resolves it against the live interface using visible roles, labels, and screen state. The tests are YAML files in your repository. **Plain English sounds flaky. What keeps a run deterministic?** The structure around the sentences. Each test declares a test-id, a target, and the context a run starts with. Data arrives by interpolation, so {{env:LINEAR_URL}} and {{secret:API_TOKEN}} resolve from your environment instead of being typed into a step. Hooks seed and verify state through your own code. agent-qa validate checks the file against the schema before anything runs. **What is actually in an agent-qa test file?** test-id, name, and target are required. context is an optional block that tells the run what it is looking at. setup and teardown list hook IDs, and a hook can also be called inline at one exact step. use carries per-test overrides such as browser, cache, and timeouts. steps is the ordered list of natural-language sentences. The test file reference documents every field. **How do URLs, credentials, and test data get into a test?** As variables, never as literals in a step. Stable values live in .env, secrets live in .env.secrets.local, and temporary values come from hook output merged into the active run. Steps reference them as {{env:NAME}} and {{secret:NAME}}, so one file runs against a local environment, a preview, or CI without being edited. **Who on the team writes these tests?** Anyone who can describe the journey. A product manager, an engineer, and a QA engineer each write in the language they would use to explain the flow out loud, and a coding agent produces the same file from the context it already has. Whoever typed it, the artifact is the same YAML, reviewed in the same pull request. ### Related resources - [Your first test](https://vostride.com/docs/agent-qa/guides/first-test.md) - [Test file reference](https://vostride.com/docs/agent-qa/configuration/test.md) - [Env and secrets](https://vostride.com/docs/agent-qa/configuration/env-secrets.md) - [Hooks](https://vostride.com/docs/agent-qa/guides/hooks.md) - [agent-qa vs testRigor](https://vostride.com/testrigor-alternative.md) --- ## Write the mobile flow once, point it at Android and iOS URL: https://vostride.com/ai-mobile-app-testing A mobile step names a control: the profile tab, the Sign in button. Turning that into the gesture a platform actually has is the execution kernel's job, not the author's. agent-qa decides each action from what is on the device at that moment and hands it to Appium to perform, so one flow covers an Android build and an iOS build. ### A step names a control, never an event Read a mobile step for what it does not say. It does not say tap, long press, or swipe. It names the control the user is reaching for, and working out the gesture that reaches it on this platform happens at run time. A click on a web target is a tap on a phone, and the author writes neither word. Appium is the execution kernel underneath, and a kernel is not a code generator. No test script is produced. Nothing is replayed from a recording. The agent decides on an action from what it can currently see, hands it over, and Appium performs it. That is why one flow survives being pointed at a platform it was not written on. ### The target changes, the test does not An Android target names an app package, and usually the activity to launch. An iOS target names a bundle identifier. That is the difference between the two, and nothing else about how the test is written changes. Only one line of the file below is platform-specific: the target. Point it at the Android target and Appium drives UiAutomator2. Point it at the iOS target and it drives XCUITest. The steps, the assertions, and everything the run remembers are untouched by that choice, so a flow that reads well on Android reads the same on iOS. ```yaml test-id: t_login-mobile name: Sign in on the mobile app target: shop-app-android steps: - Launch the app. - Dismiss the onboarding carousel if it appears. - Open the profile tab and choose "Sign in". - Enter the email "{{env:TEST_EMAIL}}" and password "{{env:TEST_PASSWORD}}". - Submit the form. - Verify the profile tab shows the account's display name. ``` ### App state and device are settings, not steps A native run has to say what happens to the installed app. Reset gives the run a clean install. Preserve keeps whatever the last run left behind, caches and preferences included. agent-qa refuses to run a native mobile test that leaves the choice out, and the choice can be made once for the workspace, once for a suite, or per test. Where the run lands is a device profile: a local Android emulator, an iOS simulator on your own machine, or a remote device you never have to hold. The profile is named in the file, and the command line can override it for a single run, so moving a flow onto different hardware never edits the flow. Steps, timings, artifacts, and anything learned are stored against the test, never against the platform that happened to run it. ```bash # initialize a workspace npx agent-qa init # run a test npx agent-qa run tests/checkout-smoke.yaml # inspect runs in the local dashboard agent-qa dashboard --port 3470 --open ``` ### FAQ **Can one agent-qa test run on both Android and iOS?** Yes. The steps name controls and the target names the build, so moving between platforms is a target change. An Android target carries an app package, an iOS target carries a bundle identifier, and Appium drives UiAutomator2 on the first and XCUITest on the second. The steps stay exactly as written. **Does agent-qa generate Appium code?** No. Appium is an execution kernel here, not a code generator. agent-qa decides on an action from what is on the device and asks Appium to perform it, one action at a time. No test script is produced, so there is no generated mobile code in your repository to review or maintain. **How is this different from recording a flow on a device?** A recording captures one pass through the app and plays it back. agent-qa plays nothing back. Every action is chosen during the run from the screen in front of it, which is why the same flow can be pointed at a platform whose screens were never recorded. **Which devices can a mobile run land on?** A device profile decides: a local Android emulator, an iOS simulator on your own machine, or a remote device. The profile is selected in the test file, and the command line can override it for a single run, so the same file can be pointed at different hardware. **Does a mobile test have to declare app state?** Yes, for native Android and iOS runs. Reset gives the run a clean install, preserve keeps what the last run left behind, and the runner rejects a native mobile test that says neither. Set it once for the workspace, once for a suite, or per test when flows need different behavior. Web auth state is a separate mechanism, and native runs do not use it. ### Related resources - [Mobile testing guide](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [Targets and device profiles](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [Run dashboard](https://vostride.com/docs/agent-qa/dashboard.md) - [agent-qa vs Appium](https://vostride.com/appium-alternative.md) - [agent-qa vs Maestro](https://vostride.com/maestro-alternative.md) --- ## AI test automation that leaves you no script to maintain URL: https://vostride.com/ai-test-automation Most AI testing means a model wrote a script once and you maintain that script forever. agent-qa generates no script at any point. It works the flow out from what is on screen at each run, and what it proves along the way is committed beside your tests as records you can read. ### Generating a script is not the same as testing The common shape of AI testing is generation. A model reads your app, emits Playwright or Selenium code, and hands it over. The intelligence was present for one afternoon and is gone by the first run. What you own afterwards is an ordinary script with ordinary selectors, and the maintenance bill arrives on the ordinary schedule. agent-qa generates nothing. There is no compile step and nothing to check in but the file below. At each run the agent reads the current screen, decides the next action, and hands that one action to an execution kernel to perform. The kernels underneath, Playwright and Appium, are never given a program to replay. ```yaml test-id: t_checkout-smoke name: Guest checkout completes target: storefront-web context: | The storefront runs at the URL configured by the storefront-web target. Product data is seeded by the workspace setup hook. steps: - Open the home page and search for "espresso grinder". - Open the first search result. - Add the product to the cart. - Start checkout as a guest. - Fill the shipping form with the workspace test address. - Place the order. - Verify the confirmation page shows an order number. ``` ### A passing run is evidence, not a green tick Most harnesses record a pass and discard everything that produced it. agent-qa keeps it: the elements that resolved, the flow that worked end to end, the timings that are real. That is the material a suite can actually compound on. The curator distills that evidence into durable records. Facts about your product, procedures that replay a proved flow, cautions about paths that misbehave, measurements taken from real runs. Each one lands as a reviewable bundle in a committed directory beside your tests, carrying the evidence that justified it. A record arrives as a diff in a pull request, not as a row in a database you cannot open. The model proposes and deterministic code applies. agent-qa never edits its own prompts and never edits its own code. The only thing that improves is knowledge, and knowledge is a file you can read, argue with, and revert. ### Nothing new is trusted on day one A record is born a candidate, and a candidate steers nothing. It is first replayed offline against the screens it came from, where it has to fix the decision it targets and flip nothing else. Then it has to earn attributed evidence from live runs: a later run that actually exercised it and did better for it. Only then does it go active and reach the next run. Failures are classified before anything is learned from them. A flake, a broken environment, a config error: no record is minted and no trust moves, and the test is flagged for a human instead. Trust moves only on failures attributed to wrong knowledge. That is what stops a bad afternoon from becoming a bad habit. Nor does it grade its own homework: the judge on replay is a separate model where one exists, and a deterministic diff where none does. The payoff is measurable. On a matched rerun a validated action is reused while the screen still matches, so the run spends its time acting rather than deciding: roughly 5x faster, 42s down to 8s, on about 60% fewer tokens. The moment the screen stops matching, the step is worked out again. This is reuse, never blind replay. ```bash # initialize a workspace npx agent-qa init # run a test npx agent-qa run tests/checkout-smoke.yaml # inspect runs in the local dashboard agent-qa dashboard --port 3470 --open ``` ### FAQ **What is AI test automation?** AI test automation puts a model in charge of how a test is carried out instead of encoding that in advance. What separates the tools is when the model runs. Most run it once, at authoring time, and hand you a script to own. agent-qa runs it at execution time, so each step is worked out against the screen in front of it and no generated code sits between the test and the app. **Does agent-qa generate test code I have to maintain?** No. Nothing is compiled and no script is emitted. The only artifact you keep is the plain-English test file. The execution kernel underneath is handed one decided action at a time, which is why there are no selectors in your repository waiting to go stale. **How does agent-qa avoid learning the wrong thing?** Everything it learns starts as a candidate, and a candidate steers no run. It has to survive an offline replay against the screens it came from, fixing the decision it targets and flipping nothing else, then earn attributed evidence from live runs before it is promoted. Flakes, broken environments and config errors are diagnosed and set aside rather than learned from, and the model that made a decision is never the one that judges it. **Can agent-qa rewrite its own prompts or code?** No, and that boundary is deliberate. The curator emits typed proposals, deterministic code applies them, and the only thing that changes is knowledge. Those records are committed files beside your tests, so an improvement you disagree with is a diff you can review and revert like any other. **If nothing is scripted, why do repeat runs get faster?** Because a validated action is kept and reused the next time the same screen appears. On a matched rerun that is roughly 5x faster, 42 seconds down to 8, on about 60% fewer tokens, since a reused action never reaches the model. As soon as the screen stops matching, the step is worked out from scratch again. ### Related resources - [Docs overview](https://vostride.com/docs/agent-qa.md) - [The curator](https://vostride.com/docs/agent-qa/memory/curator.md) - [Runtime memory injection](https://vostride.com/docs/agent-qa/memory/runtime-injection.md) - [Caching](https://vostride.com/docs/agent-qa/caching.md) - [Live demo](https://vostride.com/demo/agent-qa.md) --- ## End-to-end tests your coding agent can write, run, and read back URL: https://vostride.com/e2e-testing-for-coding-agents A coding agent changes more code in an hour than a person reads in an afternoon, so the slow step is no longer writing the feature. It is checking it. agent-qa hands that check to the agent: packaged Skills that teach it to author, debug, and triage a test, and an MCP server that gives it the calls to validate the file, queue the run, and read the artifacts back. ### The bottleneck moved to verification Writing code got cheap. Reading it did not. A branch arrives with twelve files touched, the diff looks reasonable, and nobody has exercised the feature. Approving it is a guess, and reviewers guess faster than they verify, which is how a plausible diff becomes a broken checkout. Unit tests do not settle the question. They pass on code that compiles and returns the right shape while the third screen of the flow is dead. What settles it is a run against the real product, and it has to happen while the agent still has the change in hand, not in a nightly job three merges later. ### Give your agent a direct interface to QA A coding agent needs a reliable way to set up testing, run a flow, and read the result. Some agents can operate a dashboard, but a CLI or MCP interface makes those operations explicit and repeatable. agent-qa exposes the QA workflow directly: initialize a workspace without menus, author a test from the product requirements, validate it, execute it, and inspect the evidence. agent-qa's surface is a package and a server instead. npx skills add installs the Skills into Claude Code, Codex, or OpenCode: one for authoring tests, suites, and hooks, one for debugging a failed run, one for evidence-backed triage. agent-qa mcp starts the MCP server, and that is where the calls live. The agent generates a canonical ID, validates the definition before it saves anything, queues a test or suite run, pulls back steps, logs, and artifacts, and asks for a failure classification. ```bash # expose agent-qa to coding agents over MCP agent-qa mcp # install packaged Skills for Claude Code and peers npx skills add vostride/agent-qa --skill '*' ``` ### It already has the context, and it hands back a diff A recorder watching a browser learns whatever the person driving it happened to click. Your agent holds more than that: the PRD that says what the banner should read, the route in the code, the support ticket about the promo code customers keep hitting. It writes the test out of context it already has, so the case is one a real customer ran into and the expected result is a product decision rather than the agent's opinion of one. The output is a file. Same schema a product manager writes by hand, same tests directory, same pull request. What a run learns about your app is committed too, in a visible directory beside the tests rather than a database you cannot read, so a new fact arrives as a diff carrying the evidence that justified it. Whoever typed it, a human reviews it the same way. ```yaml test-id: t_checkout-smoke name: Guest checkout completes target: storefront-web context: | The storefront runs at the URL configured by the storefront-web target. Product data is seeded by the workspace setup hook. steps: - Open the home page and search for "espresso grinder". - Open the first search result. - Add the product to the cart. - Start checkout as a guest. - Fill the shipping form with the workspace test address. - Place the order. - Verify the confirmation page shows an order number. ``` ### FAQ **How does a coding agent run an end-to-end test with agent-qa?** Through MCP, locally. Run agent-qa mcp for a stdio server the client launches itself, or start the dashboard and point the client at its MCP endpoint on loopback. Claude Code, Codex CLI, and OpenCode each register it in one line. The agent then calls the tools directly: read the config, generate an ID, validate the test, enqueue the run, and fetch the artifacts when it finishes. **What are agent-qa Skills, and which ones ship?** Skills are packaged instructions an agent loads the way it loads any other skill. Three ship today: agent-qa-authoring for creating and validating tests, suites, and hooks, agent-qa-debug-fix for patching after a failed run, and agent-qa-result-triage for evidence-backed classification. Install them per project or globally with npx skills add vostride/agent-qa. **Can the agent tell whether its own change caused the failure?** That is what the classifier is for. It reads run detail, steps, artifacts, logs, and execution logs, then returns a category with its confidence, the evidence behind it, the likely fix area, and a next action. The categories are fixed, from element_not_found and assertion_failure to hook_failure and infrastructure. The triage Skill treats the result as a hypothesis and checks the local files before anything is patched. **What stops an agent from writing a test that passes without checking anything?** Review, which is why the output is a file and not a row in a vendor's database. Validation catches a malformed definition before it is saved. A weak assertion is caught by a person reading the YAML diff in the pull request, and reading it takes no framework knowledge, because the step says 'Verify the confirmation page shows an order number'. **What does giving an agent MCP access to agent-qa actually grant?** Workspace access, so treat it as such. An enabled agent can read config, inspect artifacts, enqueue runs, mutate tests and suites through the dashboard APIs, and classify failures from local evidence. Keep the endpoint on a loopback host, prefer project-local client config so each repository declares the endpoint it expects, and review your client's auto-approval settings before turning the mutation tools on. ### Related resources - [Agent quickstart and setup prompt](https://vostride.com/docs/agent-qa/agent-quickstart.md) - [Verify a change with your coding agent](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [Evaluate agent-qa](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) - [MCP reference](https://vostride.com/docs/agent-qa/mcp.md) - [Skills](https://vostride.com/docs/agent-qa/skills.md) - [CLI](https://vostride.com/docs/agent-qa/cli.md) - [Dashboard](https://vostride.com/docs/agent-qa/dashboard.md) - [Manifesto](https://vostride.com/manifesto.md) --- # Comparisons agent-qa compared with browser agents, AI testing platforms, managed QA services, enterprise suites, and open-source frameworks. Index: https://vostride.com/alternatives Markdown index: https://vostride.com/alternatives.md This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. --- # agent-qa vs Midscene.js URL: https://vostride.com/midscene-alternative Markdown: https://vostride.com/midscene-alternative.md The Midscene.js alternative that remembers your app. Make every regression run part of a lasting QA workflow: reviewable YAML, curated product memory, and evidence your coding agent can inspect. Try agent-qa, a source-available QA runtime for repeatable web and mobile tests with application knowledge that stays in the repository. Tests declare the expected behavior, the runtime checks the live application, and the memory curator turns useful observations into reviewable files for later runs. ## The choice for recurring QA Choose agent-qa over Midscene.js for recurring regression testing when scoped application memory and an active MCP author-run-triage workflow matter. Both offer YAML, assertions, caching and mobile automation; agent-qa’s advantage is how it carries product knowledge through the QA cycle. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Repo-owned YAML expectations.** agent-qa: Yes; Midscene.js: Yes. Both keep tests in files. Midscene supports YAML actions and aiAssert, including the newer Midscene Test beta. agent-qa combines natural-language verification steps, stable test IDs, named targets and run overrides in its test contract. Sources: [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [Midscene YAML automation](https://midscenejs.com/automate-with-scripts-in-yaml), [Midscene Test overview (beta)](https://midscenejs.com/midscene-test/overview). - **Curated application memory.** agent-qa: Yes; Midscene.js: Partial. agent-qa curates product, suite and test observations into Markdown and retrieves relevant evidence for later steps. Midscene documents caller-supplied AI context and cached plans; those offer reuse, but the cited APIs do not describe an equivalent scoped behavioral-memory curator. Sources: [agent-qa behavioral memory](https://vostride.com/docs/agent-qa/memory.md), [Midscene agent API and AI context](https://midscenejs.com/reference/), [Midscene caching](https://midscenejs.com/caching). - **Coding-agent and MCP workflow.** agent-qa: Yes; Midscene.js: Partial. Both provide coding-agent Skills and CLI access. agent-qa also exposes local MCP tools for authoring, execution and failure triage. Midscene’s current docs retire its MCP servers and direct users to Skills; version 1.9.8 retains the older MCP packages. Sources: [agent-qa MCP reference](https://vostride.com/docs/agent-qa/mcp.md), [agent-qa coding-agent workflow](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), [Midscene Skills](https://midscenejs.com/skills), [Midscene MCP retirement](https://midscenejs.com/mcp). - **Reusable action plans.** agent-qa: Yes; Midscene.js: Yes. Midscene caches AI plans and eligible DOM locators with configurable read/write strategies. agent-qa caches plans by test, configuration, suite and platform context, re-observes the application, and invalidates the remaining path after a mismatch. Measure both on your own reruns. Sources: [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md), [Midscene caching](https://midscenejs.com/caching). - **Web and native mobile testing.** agent-qa: Yes; Midscene.js: Yes. Both support web, Android and iOS. agent-qa carries the same QA format, memory model and evidence workflow across those targets, with explicit mobile app-state settings. Midscene additionally supports desktop and HarmonyOS automation, which may matter beyond this QA scope. Sources: [agent-qa mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md), [agent-qa behavioral memory](https://vostride.com/docs/agent-qa/memory.md), [Midscene.js platforms and license](https://midscenejs.com/). - **Chromium, Firefox and WebKit targets.** agent-qa: Yes; Midscene.js: Partial. agent-qa exposes Chromium, Firefox and WebKit as browser choices in test configuration. Midscene recommends Chromium; its docs say basic Playwright operations may work on Firefox and WebKit, while CDP-dependent features can error. Evaluate each required browser explicitly. Sources: [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [Midscene Playwright integration and browser support](https://midscenejs.com/integrate-with-playwright). - **Evidence for failure diagnosis.** agent-qa: Yes; Midscene.js: Yes. Midscene provides visual reports and step logs. agent-qa records test and step outcomes, screenshots and artifacts, then exposes those records through its run inspector and MCP tools so the coding agent can diagnose a failure and rerun the affected test. Sources: [agent-qa recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md), [agent-qa coding-agent workflow](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), [Midscene Test overview (beta)](https://midscenejs.com/midscene-test/overview), [Midscene Playwright integration and browser support](https://midscenejs.com/integrate-with-playwright). - **Setup and API verification hooks.** agent-qa: Yes; Midscene.js: Yes. Midscene Test supports TypeScript Nodes and lifecycle hooks. agent-qa supplies a registered hook contract for Node, Bun, Python and Bash scripts in a Docker sandbox, including runtime variables that later verification steps can check. Sources: [agent-qa sandboxed hooks](https://vostride.com/docs/agent-qa/guides/hooks.md), [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [Midscene Test overview (beta)](https://midscenejs.com/midscene-test/overview). ## Why choose agent-qa for the next hundred regression runs ### Review what the agent learned Keep changing product behavior in product, suite and test memory files. agent-qa’s curator can add, refine or deprecate observations, while live evidence and your expected outcomes continue to decide whether the run passes. ### Keep MCP in the QA loop If your coding-agent workflow uses MCP, agent-qa connects test authoring, validation, runs, artifacts and failure classification through local tools. This gives your agent a supported path from a code change to an inspected regression result. ### Give the whole team one QA contract Review natural-language tests beside the feature they protect. Reuse named targets and hooks, select the browser or mobile device, and keep the resulting evidence and application memory available to humans, agents and CI. ## Verdict For a team building lasting regression coverage with coding agents, agent-qa is our pick: reviewable expectations, curated application memory, active MCP tooling and inspectable run evidence form one QA workflow. Midscene is a capable cross-platform testing system; the reason to choose agent-qa is the product knowledge and QA process you keep building after the first successful run. > Note: This comparison includes Midscene’s current documentation, including Midscene Test beta and the documented MCP retirement. The recommendation concerns recurring web and native mobile QA; it is not a measured superiority claim for every platform or workload. ## Evaluate agent-qa against Midscene.js 1. Choose one important journey and define the same visible result and API side effects in both tools, keeping the app revision, credentials and test data fixed. 2. Run a passing build and a deliberately broken build. Inspect the reports and confirm that neither implementation treats a completed click sequence as proof of the expected outcome. 3. Repeat the journey with and without caching. Inspect agent-qa’s memory files separately, then change one learned product detail and check how the next run handles the contradiction. 4. Have your coding agent retrieve the failed step, propose a scoped fix and rerun the same expectations. Repeat on every browser and native platform your release requires. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### Why choose agent-qa if Midscene.js already supports YAML testing? Choose agent-qa when maintaining application knowledge across regression runs is central to the job. Midscene already supports YAML, natural-language assertions and a test framework. agent-qa adds a defined memory lifecycle with product, suite and test scopes, plus MCP tools that let your coding agent author, execute and triage those tests. ### Does Midscene.js still support MCP? The current Midscene documentation says MCP servers have been retired and recommends Skills with platform CLIs. It identifies 1.9.8 as the last version containing MCP packages. agent-qa supports local MCP tools for test and suite authoring, run execution, artifact inspection and failure classification; execution tools need its dashboard-backed services. ### Can both tools use my models and cache repeated runs? Yes. Midscene supports compatible vision models, optional specialist models, and plan caching. agent-qa also lets you configure model providers and cache action plans. Its additional behavioral memory records what the application did, separately from execution caching. Compare uncached and repeated runs with equivalent assertions before drawing a cost or speed conclusion. ### Can I migrate Midscene Playwright or YAML tests incrementally? Yes. Start with one critical journey and translate its actions and aiAssert expectations into an agent-qa YAML test. Register the target, move necessary setup or API checks into hooks, and run both implementations against the same app revision. Keep your existing Midscene coverage while you verify the replacement detects the same intentional regression. The formats are different; this is not an automatic import. ### When might I keep Midscene.js alongside agent-qa? Keep Midscene where you need its desktop or HarmonyOS support, its existing Playwright integration, or its MIT license. agent-qa uses FSL-1.1-ALv2 and is our recommendation for a repository-owned web and native mobile QA workflow with curated application memory. You can move regression ownership gradually without replacing every automation tool. ## Sources - [Midscene.js platforms and license](https://midscenejs.com/) - [Midscene Test overview (beta)](https://midscenejs.com/midscene-test/overview) - [Midscene YAML automation](https://midscenejs.com/automate-with-scripts-in-yaml) - [Midscene agent API and AI context](https://midscenejs.com/reference/) - [Midscene caching](https://midscenejs.com/caching) - [Midscene Skills](https://midscenejs.com/skills) - [Midscene MCP retirement](https://midscenejs.com/mcp) - [Midscene model strategy](https://midscenejs.com/model-strategy) - [Midscene Playwright integration and browser support](https://midscenejs.com/integrate-with-playwright) - [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md) - [agent-qa behavioral memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md) - [agent-qa MCP reference](https://vostride.com/docs/agent-qa/mcp.md) - [agent-qa coding-agent workflow](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [agent-qa sandboxed hooks](https://vostride.com/docs/agent-qa/guides/hooks.md) - [agent-qa recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) - [agent-qa source and license](https://github.com/vostride/agent-qa) ## Compare other approaches - [agent-qa vs Browser Use](https://vostride.com/browser-use-alternative.md): Choose a complete QA workflow for recurring regressions: repo-owned expectations, scoped application memory, web and native mobile testing. - [agent-qa vs Stagehand](https://vostride.com/stagehand-alternative.md): Compare Stagehand's browser-agent SDK with agent-qa's YAML expectations, behavioral memory, local cache, and native mobile testing. - [agent-qa vs Playwright](https://vostride.com/playwright-alternative.md): Compare agent-qa with Playwright: AI-native natural-language E2E tests with memory versus hand-written browser automation scripts. --- # agent-qa vs Browser Use URL: https://vostride.com/browser-use-alternative Markdown: https://vostride.com/browser-use-alternative.md The Browser Use alternative built for recurring QA. Turn a successful browser task into a regression test your team can review, rerun and diagnose, with application memory that stays beside the code. Try agent-qa, a source-available QA runtime for maintained test coverage. It packages natural-language test definitions, application memory, web and native mobile execution, and step evidence into one repository-owned workflow that your coding agent and CI can run again. ## The choice for recurring QA Choose agent-qa over Browser Use when you need recurring regression tests with explicit expectations, scoped application memory and diagnosable results across web and native mobile. Browser Use supplies flexible browser automation; agent-qa supplies the QA contract and lifecycle around the task. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Versioned regression expectations.** agent-qa: Yes; Browser Use: Partial. agent-qa provides YAML tests with stable IDs, named targets and verification steps. Browser Use’s Python library lets you keep task prompts and custom checks in your repo, but you build the regression-test contract around its general-purpose Agent API. Sources: [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [Browser Use open-source library and Cloud distinction](https://github.com/browser-use/browser-use), [Browser Use agent configuration and custom tools](https://docs.browser-use.com/open-source/customize/agent/all-parameters). - **Scoped behavioral memory.** agent-qa: Yes; Browser Use: Partial. agent-qa curates reviewable Markdown observations by product, suite and test. Browser Use has execution history, and its Cloud offering documents persistent memory and workspace files. agent-qa’s distinction is its built-in QA scopes, trust controls and observation lifecycle in your repository. Sources: [agent-qa behavioral memory](https://vostride.com/docs/agent-qa/memory.md), [Browser Use action history and structured output](https://docs.browser-use.com/open-source/customize/agent/output-format), [Browser Use open-source library and Cloud distinction](https://github.com/browser-use/browser-use), [Browser Use persistent Cloud workspaces](https://docs.browser-use.com/cloud/agent/workspaces). - **Coding-agent access through MCP.** agent-qa: Yes; Browser Use: Yes. Both offer MCP integration. Browser Use documents local and Cloud servers for browser automation. agent-qa’s local MCP tools operate on tests, suites, execution jobs, artifacts and failure classification, giving a coding agent a QA workflow around the browser actions. Sources: [agent-qa MCP reference](https://vostride.com/docs/agent-qa/mcp.md), [Browser Use local MCP server](https://docs.browser-use.com/open-source/customize/integrations/mcp-server), [Browser Use Cloud MCP server](https://docs.browser-use.com/cloud/guides/mcp-server). - **Web, Android and iOS QA.** agent-qa: Yes; Browser Use: Partial. agent-qa runs web tests and native Android and iOS targets through the same test format and CLI. Browser Use’s documented products automate web browsers. Testing a responsive website there is useful, but does not provide agent-qa’s native mobile target and device workflow. Sources: [agent-qa native mobile targets](https://vostride.com/docs/agent-qa/guides/mobile-testing.md), [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [Browser Use hosted agent and browser products](https://docs.browser-use.com/cloud/quickstart), [Browser Use open-source library and Cloud distinction](https://github.com/browser-use/browser-use). - **Test evidence and CI results.** agent-qa: Yes; Browser Use: Partial. Browser Use returns action history, screenshots, errors and structured output. agent-qa organizes evidence into test and step records, offers JUnit output for CI, and lets an agent inspect and classify a failed run without assembling that QA reporting layer itself. Sources: [agent-qa recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md), [agent-qa coding-agent workflow and CI output](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), [Browser Use action history and structured output](https://docs.browser-use.com/open-source/customize/agent/output-format). - **Reuse across repeated runs.** agent-qa: Yes; Browser Use: Yes. Browser Use Cloud can save scripts in persistent workspaces, reuse them against live sites and repair them when needed. agent-qa provides action-plan caching with contextual keys and invalidation, alongside separate behavioral memory. Both should be evaluated against unchanged regression expectations. Sources: [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md), [agent-qa behavioral memory](https://vostride.com/docs/agent-qa/memory.md), [Browser Use rerunnable scripts](https://docs.browser-use.com/cloud/agent/scripts). - **Local execution and model choice.** agent-qa: Yes; Browser Use: Yes. Browser Use’s open-source library runs locally and supports multiple model providers; its hosted products are optional. agent-qa also runs locally or in CI with configured providers and endpoints. Its QA format lets the team review model selection together with test and run configuration. Sources: [agent-qa model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md), [agent-qa coding-agent workflow and CI output](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), [Browser Use open-source library and Cloud distinction](https://github.com/browser-use/browser-use), [Browser Use supported model providers](https://docs.browser-use.com/open-source/supported-models). - **Test setup and side-effect verification.** agent-qa: Yes; Browser Use: Partial. Browser Use supports custom tools and deterministic initial actions. agent-qa packages setup, teardown and inline hooks as registered scripts in Docker sandboxes, with exported variables that YAML verification steps can check for API or backend outcomes. Sources: [agent-qa sandboxed hooks](https://vostride.com/docs/agent-qa/guides/hooks.md), [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [Browser Use agent configuration and custom tools](https://docs.browser-use.com/open-source/customize/agent/all-parameters). ## Why choose agent-qa when the task becomes a test suite ### Make the expected result reviewable A useful regression test specifies what must remain true. agent-qa keeps those expectations in YAML with the target, identity and run controls, so a reviewer can assess coverage in the same pull request as the implementation. ### Keep product knowledge with the tests Let the memory curator preserve useful observations and deprecate contradicted ones. Product, suite and test scopes give future steps relevant application context without treating a past run as proof that today’s application still works. ### Complete the agent’s verification loop Give your coding agent one workflow to validate a definition, run the web or native mobile journey, inspect failed-step evidence and rerun after a fix. Test artifacts and JUnit output make the same result useful to teammates and CI. ## Verdict agent-qa is our choice for engineering teams that need maintained regression coverage across web and native mobile: the expected behavior, learned application context and failure evidence all have a defined place. Browser Use remains useful for general browser tasks and custom automation. Choose agent-qa when the result you need is a QA process the team can keep running and reviewing. > Note: Browser Use’s open-source library, hosted agent and managed browsers are distinct offerings. This page identifies Cloud-specific persistence and reuse explicitly; the recommendation is for recurring QA and does not assert benchmark superiority over Browser Use. ## Evaluate agent-qa against Browser Use 1. Select a release-critical flow and write down the same expected visible state and backend side effects for both implementations before running either agent. 2. Use the same app revision and test data to run a passing build and an intentional regression. Verify the failed expectation from recorded evidence, not only the agent’s final text. 3. Repeat the flow using each tool’s available reuse mechanism. Record the model, elapsed time, token usage and retained files, and check that the regression still fails. 4. Ask your coding agent to inspect a failure and rerun the unchanged expectations after a fix. Include a native Android or iOS journey if mobile coverage is part of the release. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### Is agent-qa a Browser Use alternative for AI testing? Yes. For recurring regression testing, agent-qa is our recommendation because YAML expectations, application memory, native mobile targets and QA evidence are part of its workflow. Browser Use is a flexible browser automation library and hosted service that you can use for QA; your application code supplies more of the testing structure around it. ### Is Browser Use only a paid cloud product? No. Browser Use has an MIT-licensed open-source Python library that can run on your machine with your chosen model provider, as well as optional hosted products. agent-qa is source-available under FSL-1.1-ALv2 with no paid tier. Model usage and infrastructure can cost money in either local setup, so compare total operating costs for the configuration you intend to use. ### Does Browser Use already have memory and reusable workflows? Yes. Its Cloud offering includes persistent memory and workspace files, and it documents rerunnable scripts that read live sites and can repair changed code. agent-qa’s reason to switch is its explicit QA memory lifecycle: reviewable product, suite and test observations, relevance-based retrieval and curation that can update or deprecate observations after runs. ### Can I keep Browser Use for scraping and use agent-qa for testing? Yes. Keep Browser Use tasks that collect data or complete general web workflows. Put release-critical journeys into agent-qa tests with explicit outcomes, then run them from CI or your coding agent. These tools can serve different parts of the same repository without requiring an all-at-once migration. ### How do I turn a Browser Use task into an agent-qa regression test? Start with the task’s intended outcome, identify the relevant named target and write its actions and verifications in an agent-qa YAML test. Move data preparation and API checks into registered hooks. Run both implementations against the same passing and intentionally broken builds, inspect the evidence, and retain the test only after it detects the expected regression. Python tasks do not import automatically. ## Sources - [Browser Use open-source library and Cloud distinction](https://github.com/browser-use/browser-use) - [Browser Use agent configuration and custom tools](https://docs.browser-use.com/open-source/customize/agent/all-parameters) - [Browser Use action history and structured output](https://docs.browser-use.com/open-source/customize/agent/output-format) - [Browser Use supported model providers](https://docs.browser-use.com/open-source/supported-models) - [Browser Use local MCP server](https://docs.browser-use.com/open-source/customize/integrations/mcp-server) - [Browser Use Cloud MCP server](https://docs.browser-use.com/cloud/guides/mcp-server) - [Browser Use persistent Cloud workspaces](https://docs.browser-use.com/cloud/agent/workspaces) - [Browser Use rerunnable scripts](https://docs.browser-use.com/cloud/agent/scripts) - [Browser Use hosted agent and browser products](https://docs.browser-use.com/cloud/quickstart) - [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md) - [agent-qa behavioral memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md) - [agent-qa MCP reference](https://vostride.com/docs/agent-qa/mcp.md) - [agent-qa coding-agent workflow and CI output](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa native mobile targets](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [agent-qa sandboxed hooks](https://vostride.com/docs/agent-qa/guides/hooks.md) - [agent-qa model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [agent-qa recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) - [agent-qa source and license](https://github.com/vostride/agent-qa) ## Compare other approaches - [agent-qa vs Midscene.js](https://vostride.com/midscene-alternative.md): Both support natural-language UI tests. Choose agent-qa for recurring QA with reviewable application memory and a complete MCP workflow. - [agent-qa vs Stagehand](https://vostride.com/stagehand-alternative.md): Compare Stagehand's browser-agent SDK with agent-qa's YAML expectations, behavioral memory, local cache, and native mobile testing. - [agent-qa vs QA Use](https://vostride.com/qa-use-alternative.md): Compare Desplega's QA Use CLI with agent-qa for local execution, behavioral memory, model control, and native mobile QA. - [agent-qa vs Playwright](https://vostride.com/playwright-alternative.md): Compare agent-qa with Playwright: AI-native natural-language E2E tests with memory versus hand-written browser automation scripts. --- # agent-qa vs QA Use URL: https://vostride.com/qa-use-alternative Markdown: https://vostride.com/qa-use-alternative.md A QA Use alternative that keeps the runner yours. QA Use by Desplega already brings YAML, MCP, and coding-agent skills to browser testing. agent-qa adds a locally controlled QA runtime, reviewable behavioral memory, and native Android and iOS coverage. Try agent-qa, a source-available QA runtime for a recurring verification loop your team owns: test expectations in YAML, execution on your laptop or CI runner, configurable models, and evidence your coding agent can inspect before a focused rerun. ## The choice for recurring QA agent-qa is the stronger fit for engineering teams that want repo-owned recurring QA across web and native mobile, with direct model configuration and behavioral memory they can review. QA Use already covers local test files and agent integration; the deciding advantage is control of the runtime and its retained product knowledge. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Reviewable YAML expectations.** agent-qa: Yes; QA Use: Yes. Both keep test definitions in YAML. QA Use includes named actions and assertions; agent-qa expresses journeys and expected outcomes as natural-language steps, with optional hooks and per-step controls. Existing YAML ownership is worth preserving during a move. Sources: [QA Use: test format, execution architecture and CI](https://github.com/desplega-ai/qa-use/blob/main/SETUP.md), [agent-qa: first test and expectations](https://vostride.com/docs/agent-qa/guides/first-test.md). - **Control of the execution runtime.** agent-qa: Yes; QA Use: Partial. QA Use documents local browser operation alongside a Desplega API-backed execution architecture. agent-qa runs the QA engine in your workspace or CI environment, with configured model and device services. A local browser and a locally controlled runner are separate choices. Sources: [QA Use: test format, execution architecture and CI](https://github.com/desplega-ai/qa-use/blob/main/SETUP.md), [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **MCP and coding-agent workflows.** agent-qa: Yes; QA Use: Yes. QA Use ships MCP and Claude Code verification skills. agent-qa also supplies MCP and packaged skills, including test authoring, run inspection and failure triage through local dashboard-backed services. Both can participate in an agent's development loop. Sources: [QA Use: MCP integration](https://github.com/desplega-ai/qa-use/blob/main/MCP.md), [QA Use by Desplega: repository and CLI](https://github.com/desplega-ai/qa-use), [agent-qa: local execution MCP](https://vostride.com/docs/agent-qa/mcp.md), [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Behavioral memory under review.** agent-qa: Yes; QA Use: Partial. Desplega's related Argus product says it remembers product context. agent-qa makes its memory lifecycle explicit: product, suite and test observations are editable Markdown, retrieved for later steps and curated after runs. The cited QA Use and Argus docs do not establish that same file-review workflow. Sources: [Desplega Argus: retained product context](https://www.desplega.ai/argus), [QA Use: test format, execution architecture and CI](https://github.com/desplega-ai/qa-use/blob/main/SETUP.md), [agent-qa: behavioral memory](https://vostride.com/docs/agent-qa/memory.md). - **Direct model configuration.** agent-qa: Yes; QA Use: Partial. agent-qa exposes named model configurations and compatible provider endpoints in project config. QA Use's documented CLI setup centers on a Desplega API key and app configuration. Its setup guide does not establish the same direct runner-level choice of model provider. Sources: [QA Use by Desplega: repository and CLI](https://github.com/desplega-ai/qa-use), [agent-qa: model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md). - **Web and native mobile targets.** agent-qa: Yes; QA Use: Partial. QA Use documents browser E2E testing. agent-qa also runs native Android and iOS targets through configured Appium devices, using the same CLI and test format. Native tests still need a device, app build and explicit app-state policy. Sources: [QA Use by Desplega: repository and CLI](https://github.com/desplega-ai/qa-use), [agent-qa: native mobile targets](https://vostride.com/docs/agent-qa/guides/mobile-testing.md). - **Evidence for a failed assertion.** agent-qa: Yes; QA Use: Yes. QA Use supports screenshots and downloads of recordings and HAR files. agent-qa records step outcomes and artifacts and exposes them to its run inspector and MCP tools. Evaluate which evidence lets your team explain a real failure, beyond seeing that the browser moved. Sources: [QA Use: test format, execution architecture and CI](https://github.com/desplega-ai/qa-use/blob/main/SETUP.md), [agent-qa: recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md), [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Portable definitions and CI.** agent-qa: Yes; QA Use: Yes. QA Use supports cloud-to-local export, synchronization and GitHub Actions. agent-qa runs repository tests through its CLI and can emit JUnit results. The reason to switch is the combined local QA runtime and memory, rather than an assumption that QA Use cannot work with Git or CI. Sources: [QA Use by Desplega: repository and CLI](https://github.com/desplega-ai/qa-use), [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). ## Where agent-qa takes your QA further ### Keep the execution decisions beside the tests Define targets, models, timeouts and hooks in your workspace, then run the same contract locally or in CI. agent-qa suits teams that need to review how verification runs as carefully as what it checks. ### Retain product knowledge between runs A suite can record observed product behavior in reviewable memory files. agent-qa retrieves that context for later steps while still inspecting the live application, so useful history is available without becoming an unchecked pass condition. ### Bring native mobile into the same workflow When the release spans a website and native apps, agent-qa gives the team one test format and CLI across web, Android and iOS. Keep target-specific journeys and device setup explicit while sharing the authoring and evidence workflow. ## Verdict For recurring QA owned by engineering, agent-qa offers the more complete combination of runtime control, behavioral memory and native mobile coverage. You retain the benefits of file-based tests and agent tooling while bringing the verification system into your own workspace. > Note: This page concerns Desplega's QA Use CLI. A partial rating can mean a different architecture or a capability not established in the cited documentation; it does not claim custom extensions are impossible. No comparative performance benchmark was run. ## Evaluate agent-qa against QA Use 1. Select one existing QA Use flow and preserve its setup, assertions and expected side effects in an agent-qa YAML test. 2. Run both against the same application revision and dataset; record the services, credentials, model settings and artifacts each requires. 3. Introduce a known regression and confirm both tests fail for the intended reason, then inspect the evidence available to a coding agent. 4. Rerun after a harmless UI change and review agent-qa's memory changes. If native mobile matters, evaluate a matching Android or iOS journey separately. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### Which QA Use project does this page compare? This page compares Desplega's QA Use at qa-use.dev, published as @desplega.ai/qa-use from desplega-ai/qa-use. Browser Use also publishes a separate repository named qa-use. That dashboard has its own comparison here; capabilities from one project should not be attributed to the other. ### Does QA Use already support YAML and MCP? Yes. Desplega's QA Use has local YAML tests, assertion actions, MCP and coding-agent skills. agent-qa's stronger case is the combination of a locally controlled QA runner, configurable models, curated behavioral memory and native mobile targets. YAML or MCP alone does not distinguish these products. ### Can I reuse QA Use tests when moving to agent-qa? Use your existing files as the migration specification. Translate action and assertion steps into agent-qa's test schema, configure the target and reproduce setup data. The schemas differ, so this is an explicit conversion, not a drop-in import. Keep the original expected outcomes and compare a passing run with a deliberately broken case. ### Does a local QA Use browser mean execution is independent of Desplega? Its documented local browser workflow still participates in an API-backed architecture and requires Desplega configuration. Inspect the services needed by the command you intend to use. With agent-qa, the engine runs in your environment and connects to the model and device providers you configure; those providers can still be remote. ### When is agent-qa the better QA Use alternative? Choose agent-qa when repeated release checks should retain reviewable product knowledge and run across web and native apps under your team's runtime configuration. QA Use remains a reasonable choice for teams invested in Desplega's browser testing workflow. Evaluate failure detection, evidence and operating dependencies on the same application before migrating. ## Sources - [QA Use by Desplega: repository and CLI](https://github.com/desplega-ai/qa-use) - [QA Use: test format, execution architecture and CI](https://github.com/desplega-ai/qa-use/blob/main/SETUP.md) - [QA Use: MCP integration](https://github.com/desplega-ai/qa-use/blob/main/MCP.md) - [Desplega Argus: retained product context](https://www.desplega.ai/argus) - [Browser Use's separate QA Use project](https://github.com/browser-use/qa-use) - [agent-qa: first test and expectations](https://vostride.com/docs/agent-qa/guides/first-test.md) - [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa: behavioral memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa: model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [agent-qa: native mobile targets](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [agent-qa: local execution MCP](https://vostride.com/docs/agent-qa/mcp.md) - [agent-qa: recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Compare other approaches - [agent-qa vs Browser Use QA Use](https://vostride.com/browser-use-qa-use-alternative.md): Compare Browser Use's QA Use dashboard with agent-qa for repo-owned expectations, local execution, memory, and web and mobile QA. - [agent-qa vs Browser Use](https://vostride.com/browser-use-alternative.md): Choose a complete QA workflow for recurring regressions: repo-owned expectations, scoped application memory, web and native mobile testing. - [agent-qa vs Stagehand](https://vostride.com/stagehand-alternative.md): Compare Stagehand's browser-agent SDK with agent-qa's YAML expectations, behavioral memory, local cache, and native mobile testing. - [agent-qa vs Playwright](https://vostride.com/playwright-alternative.md): Compare agent-qa with Playwright: AI-native natural-language E2E tests with memory versus hand-written browser automation scripts. --- # agent-qa vs Browser Use QA Use URL: https://vostride.com/browser-use-qa-use-alternative Markdown: https://vostride.com/browser-use-qa-use-alternative.md A Browser Use QA Use alternative built around your repo. Browser Use's QA Use puts natural-language checks in a self-hosted dashboard. agent-qa puts repeatable expectations, configurable execution, and retained product knowledge beside the code being tested. Try agent-qa, a source-available QA runtime for changes that need a reviewable verification contract. Keep tests and configuration as files, run them locally or in CI, and let a coding agent inspect the completed result before deciding what to fix. ## The choice for recurring QA Choose agent-qa for release checks that belong in the repository and must work across web and native mobile. It combines YAML expectations, local execution, behavioral memory and test-aware agent tools. Browser Use's QA Use is a different fit: a dashboard for cloud-executed browser checks. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Natural-language verification.** agent-qa: Yes; Browser Use QA Use: Yes. Both express browser checks in natural language. QA Use accepts steps and success criteria. agent-qa stores expected outcomes within YAML journeys, so the intended behavior can be reviewed with the application change. Sources: [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [agent-qa: YAML expectations](https://vostride.com/docs/agent-qa/guides/first-test.md). - **Tests as repository files.** agent-qa: Yes; Browser Use QA Use: Partial. QA Use is a dashboard application backed by PostgreSQL. agent-qa's contract is a YAML file read by the CLI and local dashboard. For pull-request review, that makes the exact acceptance behavior directly available as a source diff. Sources: [Browser Use QA Use: deployment services](https://github.com/browser-use/qa-use/blob/main/docker-compose.yaml), [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [agent-qa: YAML expectations](https://vostride.com/docs/agent-qa/guides/first-test.md). - **Local QA execution.** agent-qa: Yes; Browser Use QA Use: Partial. QA Use's documented setup self-hosts the dashboard and requires Browser Use Cloud. agent-qa runs its engine in your environment and uses the model and target services you configure, allowing the QA command to run alongside a local application or in CI. Sources: [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [Browser Use QA Use: application dependencies and API](https://github.com/browser-use/qa-use/blob/main/package.json), [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Configurable model endpoints.** agent-qa: Yes; Browser Use QA Use: Partial. agent-qa provides named model configurations with compatible endpoints. QA Use integrates the Browser Use API; the cited dashboard setup does not establish equivalent direct provider configuration. Confirm the options of that application instead of assuming it exposes every underlying Browser Use capability. Sources: [Browser Use QA Use: application dependencies and API](https://github.com/browser-use/qa-use/blob/main/package.json), [agent-qa: runtime and model configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md). - **Reviewable behavioral memory.** agent-qa: Yes; Browser Use QA Use: Partial. agent-qa curates Markdown observations by product, suite and test for later runs. QA Use's published overview does not document that file-backed memory lifecycle. The broader Browser Use ecosystem has its own memory features; this comparison is scoped to the QA Use application. Sources: [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [Browser Use: broader framework and cloud capabilities](https://github.com/browser-use/browser-use), [agent-qa: behavioral memory](https://vostride.com/docs/agent-qa/memory.md). - **Agent tools for test lifecycle.** agent-qa: Yes; Browser Use QA Use: Partial. agent-qa's local MCP covers definitions, run state, artifacts and triage. Browser Use offers separate browser and QA plugins for coding agents; those do not establish the same test-management tool surface in this QA Use dashboard. Sources: [Browser Use: separate coding-agent plugins](https://github.com/browser-use/plugins), [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [agent-qa: MCP tools](https://vostride.com/docs/agent-qa/mcp.md). - **Inspectable run evidence.** agent-qa: Yes; Browser Use QA Use: Yes. QA Use documents pass/fail reports, screenshots and recordings. agent-qa makes step evidence available in its inspector and local agent tools. Evaluate a seeded failure and check whether its cause is clear enough for another person or agent to reproduce. Sources: [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [agent-qa: recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md), [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Web and native mobile QA.** agent-qa: Yes; Browser Use QA Use: Partial. The QA Use project describes web testing. agent-qa's target configuration also includes native Android and iOS applications, device selection and explicit reset or preserve behavior. That gives cross-platform teams one authoring and execution workflow while keeping platform setup visible. Sources: [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use), [agent-qa: native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md). ## Make verification part of the change ### Review the expected outcome in the pull request Keep the acceptance steps in YAML beside the implementation. A reviewer can see which behavior changed, a CI runner can execute the same file, and an agent can rerun the specific journey that failed. ### Control the runner and its retained knowledge agent-qa keeps runtime configuration and curated observations accessible as files. Review model settings, inspect the context a later step receives, and remove stale observations through the same process used for other repository changes. ### Close the loop from failure to rerun Give the coding agent tools to retrieve the failed step, read its artifacts and check a focused fix. Preserve the original expectation and both run results so a successful rerun can be traced to an actual correction. ## Verdict agent-qa is the stronger choice for repo-owned release verification: expectations, execution configuration and behavioral memory are available to the engineers and agents changing the product. The benefit grows when the same team also needs native mobile QA. > Note: This comparison covers browser-use/qa-use, not every Browser Use product or Desplega's similarly named CLI. Partial ratings include capabilities not established by this application's cited documentation. No comparative performance benchmark was run. ## Evaluate agent-qa against Browser Use QA Use 1. Choose a dashboard check with explicit success criteria and represent the same requirement in agent-qa YAML. 2. Run both against an identical application revision and prepared account; document local services, cloud dependencies and evidence locations. 3. Break the expected behavior deliberately and verify that each result identifies the failure instead of passing because navigation succeeded. 4. Have a coding agent inspect the evidence, make an authorized fix and rerun the unchanged expectation. Compare the effort needed to preserve and review the whole loop. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### Is Browser Use QA Use the same as Desplega's QA Use? No. This page compares the dashboard in browser-use/qa-use. Desplega's @desplega.ai/qa-use is a separate CLI with YAML tests, MCP and agent skills, covered on the QA Use alternative page. The project URL is the clearest way to distinguish them. ### Can Browser Use QA Use schedule checks and send alerts? Yes. Its README lists suite schedules, parallel execution and failure emails. These are useful dashboard features. agent-qa is preferred here for repository-owned verification; use your CI or scheduler to trigger its CLI when recurring execution is needed. ### Does self-hosting the QA Use dashboard also self-host its agent? The published quickstart requires a Browser Use Cloud API key while Docker starts the dashboard services. Treat those as separate dependencies. agent-qa executes its QA engine in the environment you choose, with separately configured model and device services. ### How do I move a dashboard check into agent-qa? Carry over the journey, starting state and success criteria into an agent-qa YAML test. Configure its target, authentication and setup requirements, then run it against the same application revision. Keep the old result for comparison and add a known failing case to prove that the new test checks the requirement. ### Why choose agent-qa for recurring release QA? agent-qa connects a versioned test contract to a local runner, retained behavioral observations and inspectable step evidence. The same workflow extends to native mobile. That is a stronger match when engineering needs to review, reproduce and diagnose the verification performed for each release. ## Sources - [Browser Use QA Use: project scope and features](https://github.com/browser-use/qa-use) - [Browser Use QA Use: deployment services](https://github.com/browser-use/qa-use/blob/main/docker-compose.yaml) - [Browser Use QA Use: application dependencies and API](https://github.com/browser-use/qa-use/blob/main/package.json) - [Browser Use: separate coding-agent plugins](https://github.com/browser-use/plugins) - [Browser Use: broader framework and cloud capabilities](https://github.com/browser-use/browser-use) - [Desplega's separate QA Use CLI](https://github.com/desplega-ai/qa-use) - [agent-qa: YAML expectations](https://vostride.com/docs/agent-qa/guides/first-test.md) - [agent-qa: coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa: behavioral memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa: runtime and model configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [agent-qa: MCP tools](https://vostride.com/docs/agent-qa/mcp.md) - [agent-qa: recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) - [agent-qa: native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) ## Compare other approaches - [agent-qa vs QA Use](https://vostride.com/qa-use-alternative.md): Compare Desplega's QA Use CLI with agent-qa for local execution, behavioral memory, model control, and native mobile QA. - [agent-qa vs Browser Use](https://vostride.com/browser-use-alternative.md): Choose a complete QA workflow for recurring regressions: repo-owned expectations, scoped application memory, web and native mobile testing. - [agent-qa vs Stagehand](https://vostride.com/stagehand-alternative.md): Compare Stagehand's browser-agent SDK with agent-qa's YAML expectations, behavioral memory, local cache, and native mobile testing. --- # agent-qa vs Stagehand URL: https://vostride.com/stagehand-alternative Markdown: https://vostride.com/stagehand-alternative.md A Stagehand alternative with the QA workflow built in. Stagehand gives developers composable browser tools. agent-qa combines execution with reviewable expectations, behavioral memory, and evidence that a coding agent can use to verify the next change. Try agent-qa, a source-available QA runtime for a maintained regression suite. Author a YAML journey, name its expected outcomes, execute it through CLI or MCP, and retain useful product context for the next run. The same contract model covers web and native apps. ## The choice for recurring QA agent-qa is the better fit for recurring product QA because it supplies the test contract, execution workflow, curated behavioral memory and evidence inspection together. Stagehand is a capable SDK for building browser agents; teams adopting it for QA still choose and assemble the surrounding test workflow. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **A maintained test contract.** agent-qa: Yes; Stagehand: Partial. Stagehand v4 offers act, extract and observe alongside deterministic browser APIs. agent-qa adds a YAML test schema with expected outcomes, targets and run controls. Stagehand scripts can be repository-owned too; agent-qa supplies the QA contract and runner as one workflow. Sources: [Stagehand v4: browser-agent SDK and primitives](https://docs.stagehand.dev/v4/first-steps/introduction), [agent-qa: YAML test contracts](https://vostride.com/docs/agent-qa/guides/first-test.md). - **Product memory across runs.** agent-qa: Yes; Stagehand: Partial. Stagehand can persist cookies and site data through local profiles or Browserbase contexts. agent-qa additionally curates behavioral observations by product, suite and test, storing reviewable Markdown for retrieval in later steps. Browser session persistence serves a different purpose from this QA memory. Sources: [Stagehand v4: persistent browser data](https://docs.stagehand.dev/v4/best-practices/user-data), [agent-qa: behavioral memory](https://vostride.com/docs/agent-qa/memory.md). - **Caching for repeated work.** agent-qa: Yes; Stagehand: Yes. Stagehand v4 documents managed Browserbase caching and fallback to inference when recorded actions no longer resolve. agent-qa stores action plans locally and still observes the current app and verifies outcomes. Stagehand's v4 cache option requires a Browserbase browser; local sessions can reuse observed actions through code. Sources: [Stagehand v4: caching behavior](https://docs.stagehand.dev/v4/best-practices/caching), [Stagehand v4: actions, self-healing and deterministic replay](https://docs.stagehand.dev/v4/basics/act), [agent-qa: validated action caching](https://vostride.com/docs/agent-qa/caching.md). - **Coding-agent and MCP access.** agent-qa: Yes; Stagehand: Yes. Stagehand documents experimental persistent-browser integrations for Codex, Claude Code and other agents, including MCP. agent-qa's local MCP exposes QA definitions, runs, artifacts and triage. The distinction is the scope of the tools an agent receives, not whether agents can use either product. Sources: [Stagehand v4: coding-agent and MCP integrations](https://docs.stagehand.dev/v4/integrations/overview), [agent-qa: local execution MCP](https://vostride.com/docs/agent-qa/mcp.md). - **Bring your own model.** agent-qa: Yes; Stagehand: Yes. Both support model choice. Stagehand accepts provider configuration or a client-side LLM callback and offers Browserbase Model Gateway. agent-qa defines named models and compatible endpoints in project configuration. Choose agent-qa for how model selection fits the QA workflow, rather than treating provider choice as exclusive. Sources: [Stagehand v4: model configuration](https://docs.stagehand.dev/v4/configuration/models), [agent-qa: model configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md). - **Local browser execution.** agent-qa: Yes; Stagehand: Yes. Stagehand supports local Chromium browsers, CDP attachment and Browserbase sessions. agent-qa also runs in your environment, with a CLI that validates and executes test files and can write JUnit results. Local browser support is shared; the test lifecycle built around it is the decision. Sources: [Stagehand v4: local and hosted browsers](https://docs.stagehand.dev/v4/configuration/browser), [agent-qa: verification loop and CI](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Failure evidence in a QA loop.** agent-qa: Yes; Stagehand: Yes. Stagehand offers logs, metrics and tracing; Browserbase adds session replay. agent-qa ties captured artifacts and step outcomes to named tests and exposes them through its inspector and MCP tools. Both provide evidence; agent-qa packages it around diagnosing and rerunning a failed requirement. Sources: [Stagehand v4: observability](https://docs.stagehand.dev/v4/configuration/observability), [agent-qa: recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md), [agent-qa: verification loop and CI](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Web and native mobile QA.** agent-qa: Yes; Stagehand: Partial. Stagehand's documented runtime targets Chromium browsers. agent-qa adds native Android and iOS targets through configured Appium devices. Mobile browser emulation is useful web coverage, but a native app journey needs the app and device execution path that agent-qa provides. Sources: [Stagehand v4: browser-agent SDK and primitives](https://docs.stagehand.dev/v4/first-steps/introduction), [Stagehand v4: local and hosted browsers](https://docs.stagehand.dev/v4/configuration/browser), [agent-qa: native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md). ## Spend the effort on coverage ### Start with the acceptance contract agent-qa supplies a place for each journey's target, context, verification steps and run controls. Your team can review the requirement and execute it without first designing a test format around a browser SDK. ### Keep useful context after the browser closes File-backed behavioral memory makes earlier product observations available to later runs. Review the stored evidence, let the curator update stale context, and keep the current application state as the authority for each verdict. ### Use one QA workflow across the release Run web and native mobile journeys through agent-qa's common test model, CLI and evidence tools. Coding agents can use the same author, run, inspect and rerun process across platforms, with the necessary device configuration kept explicit. ## Verdict For teams building recurring QA, agent-qa supplies the more complete product: reviewable expectations, a local execution workflow, curated product memory and native mobile support. Choose it when the work should go into meaningful coverage and evidence instead of assembling that QA layer around browser primitives. > Note: Stagehand is extensible and can be embedded in a custom test harness. These ratings compare its documented v4 browser SDK with agent-qa's supplied QA workflow; they do not imply missing capabilities are impossible to build. No comparative speed, cost or reliability benchmark was run. ## Evaluate agent-qa against Stagehand 1. Take one Stagehand automation and write down its required final state and side effects before translating it into agent-qa YAML. 2. Run both on the same application revision, browser and dataset. Record models, hosting mode and cache settings so the comparison is reproducible. 3. Seed a regression that still permits successful clicks. Confirm the expected outcome fails and compare the artifacts available for diagnosis. 4. Repeat after a harmless UI change and measure warm and cold runs separately. Review memory updates and test a native mobile journey if it is part of your release. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### Is agent-qa a better Stagehand alternative for E2E testing? For a maintained E2E QA suite, agent-qa is the stronger fit: it combines YAML expectations, a runner, behavioral memory and test-aware evidence tools. Stagehand is well suited to developers building browser agents or custom automation systems. Its browser primitives can support tests, but your chosen harness supplies the surrounding QA workflow. ### Does Stagehand already have caching and self-healing? Yes. The reviewed v4 docs describe self-healing actions, Browserbase caching and fallback to inference when a cached action cannot resolve. agent-qa's advantage is a local action cache integrated with a QA runner that continues to observe the app and check outcomes. Neither cache proves that a test assertion is correct. ### Can Stagehand work with coding agents and different models? Yes. Stagehand documents experimental integrations with persistent browser tools, including MCP paths, and supports provider selection or a custom LLM callback. agent-qa also offers agent tools and configurable models, with its tools organized around test definitions, execution, artifact inspection and failure triage. ### How do I turn a Stagehand script into an agent-qa test? Identify the user journey and the assertions behind the script. Write those as agent-qa YAML steps, move preparation and cleanup into configured hooks where needed, and set the target and model. This is a schema change, not direct script compatibility. Validate both a passing case and a known regression before retiring the original check. ### Does this comparison apply to Stagehand v3 or v4? The cited Stagehand documentation is v4. Its managed cache requires Browserbase, while older v3 documentation also describes a cacheDir option for local action caching. Match the comparison to the version you actually use. The main reason to choose agent-qa remains its integrated QA contract, behavioral memory and native mobile workflow. ## Sources - [Stagehand v4: browser-agent SDK and primitives](https://docs.stagehand.dev/v4/first-steps/introduction) - [Stagehand v4: actions, self-healing and deterministic replay](https://docs.stagehand.dev/v4/basics/act) - [Stagehand v4: local and hosted browsers](https://docs.stagehand.dev/v4/configuration/browser) - [Stagehand v4: model configuration](https://docs.stagehand.dev/v4/configuration/models) - [Stagehand v4: caching behavior](https://docs.stagehand.dev/v4/best-practices/caching) - [Stagehand v4: persistent browser data](https://docs.stagehand.dev/v4/best-practices/user-data) - [Stagehand v4: coding-agent and MCP integrations](https://docs.stagehand.dev/v4/integrations/overview) - [Stagehand v4: observability](https://docs.stagehand.dev/v4/configuration/observability) - [Stagehand v3: earlier local cache behavior](https://docs.stagehand.dev/v3/best-practices/caching) - [agent-qa: YAML test contracts](https://vostride.com/docs/agent-qa/guides/first-test.md) - [agent-qa: verification loop and CI](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa: behavioral memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa: validated action caching](https://vostride.com/docs/agent-qa/caching.md) - [agent-qa: model configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [agent-qa: local execution MCP](https://vostride.com/docs/agent-qa/mcp.md) - [agent-qa: recorded evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) - [agent-qa: native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [agent-qa: fair evaluation experiments](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) ## Compare other approaches - [agent-qa vs Browser Use](https://vostride.com/browser-use-alternative.md): Choose a complete QA workflow for recurring regressions: repo-owned expectations, scoped application memory, web and native mobile testing. - [agent-qa vs Midscene.js](https://vostride.com/midscene-alternative.md): Both support natural-language UI tests. Choose agent-qa for recurring QA with reviewable application memory and a complete MCP workflow. - [agent-qa vs QA Use](https://vostride.com/qa-use-alternative.md): Compare Desplega's QA Use CLI with agent-qa for local execution, behavioral memory, model control, and native mobile QA. - [agent-qa vs Playwright](https://vostride.com/playwright-alternative.md): Compare agent-qa with Playwright: AI-native natural-language E2E tests with memory versus hand-written browser automation scripts. --- # agent-qa vs Skyvern URL: https://vostride.com/skyvern-alternative Markdown: https://vostride.com/skyvern-alternative.md A Skyvern alternative built around your QA suite. Skyvern automates browser work with AI, code and reusable workflows. Choose agent-qa when each code change needs a repeatable test, a clear expected result and application memory your team can review. Try agent-qa, a QA runtime that keeps test expectations in repository-owned YAML and connects execution, evidence and application memory through CLI, MCP and Skills. ## The choice for recurring QA agent-qa is the stronger fit for recurring product QA across web and native mobile: it packages test contracts, live verification, evidence and reviewable application memory into one workflow. Skyvern is a strong browser automation system with its own validation, caching and agent integrations. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **QA expectations as the contract.** agent-qa: Yes; Skyvern: Partial. Skyvern supports natural-language validation and exportable YAML workflows. agent-qa makes a test with explicit expected behavior the unit you validate, run and review, so a saved automation becomes part of a recurring QA suite. Sources: [Skyvern MCP and validation tools](https://www.skyvern.com/docs/developers/getting-started/mcp), [Skyvern workflow import and export](https://www.skyvern.com/docs/cloud/building-workflows/manage-workflows), [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Repository ownership and CI.** agent-qa: Yes; Skyvern: Yes. Skyvern's Python and TypeScript automation code can live in your repo and run in CI; self-hosting is supported. agent-qa brings that ownership to YAML tests, configuration and application memory, with JUnit output for CI results. Sources: [Skyvern browser automation](https://www.skyvern.com/docs/developers/browser-automations/overview), [Skyvern self-hosting](https://www.skyvern.com/docs/developers/self-hosted/overview), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), [agent-qa model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md). - **Coding-agent tools and Skills.** agent-qa: Yes; Skyvern: Yes. Both provide MCP, a CLI and Skills. Skyvern exposes browser actions, validation and workflow management. agent-qa's workflow centers on discovering tests, executing a selected test, inspecting step evidence and triaging its failure. Sources: [Skyvern MCP and validation tools](https://www.skyvern.com/docs/developers/getting-started/mcp), [Skyvern CLI and Skills](https://www.skyvern.com/docs/integrations/cli), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Reviewable application memory.** agent-qa: Yes; Skyvern: Partial. Skyvern documents reusable code and persistent browser profiles. agent-qa adds a different durable asset: Markdown observations scoped to products, suites and tests, retrieved for later steps and curated from run evidence. Profiles and execution caches serve different purposes from behavioral memory. Sources: [Skyvern browser profiles](https://www.skyvern.com/docs/developers/optimization/browser-profiles), [Skyvern code caching](https://www.skyvern.com/docs/developers/features/code-caching), [agent-qa application memory](https://vostride.com/docs/agent-qa/memory.md). - **Reuse on repeated runs.** agent-qa: Yes; Skyvern: Yes. Skyvern can execute generated code and fall back to AI when that code fails. agent-qa reuses matched action plans while continuing to observe the current application and verify outcomes. Compare correctness and measured resource use with each cache enabled and disabled. Sources: [Skyvern code caching](https://www.skyvern.com/docs/developers/features/code-caching), [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md), [Evaluate agent-qa on your application](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md). - **Evidence for failure diagnosis.** agent-qa: Yes; Skyvern: Yes. Skyvern exposes recordings, screenshots, reasoning logs, HAR files and traces. agent-qa connects step screenshots, logs and run artifacts to test results, with a documented coding-agent loop for diagnosis and a focused rerun. Sources: [Skyvern run artifacts](https://www.skyvern.com/docs/developers/debugging/using-artifacts), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Web and native mobile QA.** agent-qa: Yes; Skyvern: Partial. Skyvern's documented execution surface is the browser. agent-qa also configures native Android and iOS targets through Appium and local or BrowserStack devices, keeping mobile testing within the same test format and CLI workflow. Sources: [Skyvern browser automation](https://www.skyvern.com/docs/developers/browser-automations/overview), [Skyvern self-hosting](https://www.skyvern.com/docs/developers/self-hosted/overview), [agent-qa native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md). - **Choice of model and infrastructure.** agent-qa: Yes; Skyvern: Yes. Self-hosted Skyvern supports multiple providers, compatible endpoints and local models. agent-qa also configures providers and model endpoints in the workspace. Its QA advantage comes from the surrounding test lifecycle, not exclusive access to model choice. Sources: [Skyvern model configuration](https://www.skyvern.com/docs/developers/self-hosted/llm-configuration), [agent-qa model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md). ## Why choose agent-qa for QA after every change ### Make acceptance criteria a reviewed file Translate the user journey and its expected result into a YAML test beside the application. agent-qa gives the team a specific contract to review and rerun, while Skyvern's automation workflows remain useful for operational browser work. ### Carry product knowledge between tests A working action sequence is useful; knowing why the product behaves that way is useful too. agent-qa stores behavioral observations as reviewable Markdown, so future runs can consult application context while still checking the live result. ### Include the native app in the QA workflow When a release touches both a website and an Android or iOS app, agent-qa keeps target configuration, test authoring and execution in one runtime. Native targets use configured devices and Appium instead of browser emulation. ## Verdict For a repository-owned QA suite that must preserve expectations, explain failures and cover web plus native apps, choose agent-qa. Its advantage over Skyvern is a complete recurring verification workflow with reviewable product memory. > Note: This compares the documented Skyvern browser automation workflow with agent-qa's QA workflow. The recommendation is a product-fit judgment, not a measured claim about relative speed, cost or bug-detection rates. ## Evaluate agent-qa against Skyvern 1. Pick one product journey with a written expected outcome. Implement it in Skyvern and agent-qa against the same app revision, browser and fixture data. 2. Run both against the working journey and a controlled regression. Inspect the validation result and artifacts; count false passes, false failures and incomplete runs. 3. Repeat the journey using each product's documented caching options. Record runtime and model usage, and separately inspect what application context agent-qa memory contributes. 4. If native mobile is required, run the same business acceptance criteria on a configured Android or iOS target and include the extra tooling needed by each approach. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### When is agent-qa a better Skyvern alternative? Choose agent-qa when the deliverable is a recurring regression suite owned by the application team. Its test contracts, evidence retrieval, application memory and native mobile targets fit that job directly. Skyvern is well suited to browser workflows such as extracting information and completing operational tasks, and it can also validate page conditions. ### Does Skyvern already support MCP and coding-agent Skills? Yes. Skyvern documents an MCP server, CLI and bundled Skills, including setup for coding agents. agent-qa's advantage for QA is the test lifecycle those integrations operate: select or author a contract, validate it, run it, inspect the result and diagnose failures using the recorded evidence. ### How does agent-qa memory differ from Skyvern code caching? Skyvern's code cache reuses generated automation code, with AI fallback when execution fails. agent-qa has an action cache too, plus a separate store of behavioral observations about products, suites and tests. Those observations can be reviewed and updated as the application changes. Neither a cache hit nor a remembered observation proves a current test passes. ### Can Skyvern workflows be migrated into agent-qa YAML? Skyvern can export workflow definitions as YAML or JSON, and its automation code can already live in your repository. The schemas differ, so this is not a direct file import. Select a QA journey, retain its setup and acceptance criteria, and express those in an agent-qa test. Verify the rewritten test against a working case and a deliberate regression. ### Can I self-host Skyvern or choose its LLM instead? Yes. Skyvern supports self-hosting and multiple LLM providers, including compatible endpoints and local models. Compare it with agent-qa on your actual recurring QA task: authoring and review effort, failure diagnosis, application memory and any native mobile coverage you need. Include model usage and infrastructure in both evaluations. ## Sources - [Skyvern browser automation](https://www.skyvern.com/docs/developers/browser-automations/overview) - [Skyvern workflow import and export](https://www.skyvern.com/docs/cloud/building-workflows/manage-workflows) - [Skyvern MCP and validation tools](https://www.skyvern.com/docs/developers/getting-started/mcp) - [Skyvern CLI and Skills](https://www.skyvern.com/docs/integrations/cli) - [Skyvern code caching](https://www.skyvern.com/docs/developers/features/code-caching) - [Skyvern browser profiles](https://www.skyvern.com/docs/developers/optimization/browser-profiles) - [Skyvern run artifacts](https://www.skyvern.com/docs/developers/debugging/using-artifacts) - [Skyvern self-hosting](https://www.skyvern.com/docs/developers/self-hosted/overview) - [Skyvern model configuration](https://www.skyvern.com/docs/developers/self-hosted/llm-configuration) - [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md) - [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa application memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md) - [agent-qa native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [agent-qa model and runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [Evaluate agent-qa on your application](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) ## Compare other approaches - [agent-qa vs Browser Use](https://vostride.com/browser-use-alternative.md): Choose a complete QA workflow for recurring regressions: repo-owned expectations, scoped application memory, web and native mobile testing. - [agent-qa vs Stagehand](https://vostride.com/stagehand-alternative.md): Compare Stagehand's browser-agent SDK with agent-qa's YAML expectations, behavioral memory, local cache, and native mobile testing. - [agent-qa vs Puppeteer](https://vostride.com/puppeteer-alternative.md): Compare Puppeteer scripts with agent-qa for recurring QA: natural-language contracts, application memory, failure evidence and native mobile coverage. --- # agent-qa vs Puppeteer URL: https://vostride.com/puppeteer-alternative Markdown: https://vostride.com/puppeteer-alternative.md A Puppeteer alternative with the QA workflow included. Puppeteer gives developers direct browser control. agent-qa gives your team repeatable tests written as intent, with application memory, recorded evidence and a workflow coding agents can run. Try agent-qa, a QA runtime for plain-English YAML tests that combines planning, execution, application memory and evidence across web, Android and iOS. ## The choice for recurring QA Choose agent-qa over assembling a Puppeteer QA harness when you want recurring acceptance tests, application memory and evidence-driven agent triage in one runtime. Puppeteer remains a strong choice for direct browser programming and existing deterministic scripts. Sources reviewed 2026-09-07 by Vostride. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Reviewable test authoring.** agent-qa: Yes; Puppeteer: Yes. Both keep definitions in your repository. Puppeteer expresses automation through JavaScript APIs; agent-qa expresses the user journey and its expected behavior in plain-English YAML, with a runtime that plans the browser actions. Sources: [Puppeteer API and MCP introduction](https://pptr.dev/), [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md). - **Adapting actions to the UI.** agent-qa: Yes; Puppeteer: Partial. Puppeteer locators include automatic waiting, retries and accessible-name selectors, which help avoid brittle scripts. agent-qa can replan from test intent when the observed UI needs a different action path; the required outcome should stay unchanged. Sources: [Puppeteer locators and selectors](https://pptr.dev/guides/page-interactions), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Application memory between runs.** agent-qa: Yes; Puppeteer: No. agent-qa retrieves and curates behavioral observations as Markdown files scoped to products, suites and tests. Puppeteer's browser-control API does not supply that application-memory lifecycle; a team building it on Puppeteer would own the additional implementation. Sources: [Puppeteer API and MCP introduction](https://pptr.dev/), [agent-qa application memory](https://vostride.com/docs/agent-qa/memory.md). - **Coding-agent verification loop.** agent-qa: Yes; Puppeteer: Partial. Puppeteer points agents to Chrome DevTools MCP, which supports browser interaction, debugging and performance analysis. agent-qa supplies MCP and Skills around persistent tests, execution results, step artifacts and failure triage, ready for the next code change. Sources: [Puppeteer API and MCP introduction](https://pptr.dev/), [Chrome DevTools MCP repository](https://github.com/ChromeDevTools/chrome-devtools-mcp), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Evidence tied to test outcomes.** agent-qa: Yes; Puppeteer: Partial. Puppeteer provides screenshot and debugging APIs for a custom harness. agent-qa packages step evidence, run results and JUnit output into its testing workflow, so a coding agent can inspect a failed test and report a focused rerun. Sources: [Puppeteer screenshot API](https://pptr.dev/guides/screenshots), [Puppeteer debugging](https://pptr.dev/guides/debugging), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md). - **Web and native mobile targets.** agent-qa: Yes; Puppeteer: Partial. Puppeteer controls Chrome and Firefox and can emulate mobile browser settings. Native Android and iOS apps need a different driver. agent-qa includes configured Appium-based native targets alongside its web test workflow. Sources: [Puppeteer supported browsers](https://pptr.dev/supported-browsers), [Puppeteer device emulation](https://pptr.dev/api/puppeteer.page.emulate), [agent-qa native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md). - **Local and CI execution.** agent-qa: Yes; Puppeteer: Yes. Puppeteer runs in your environment and publishes a Docker setup. agent-qa likewise runs from a local or CI command, adding test validation, configured targets and result reporting. Hosting ownership is shared ground, not a reason to discard working Puppeteer scripts. Sources: [Puppeteer Docker setup](https://pptr.dev/guides/docker), [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md), [agent-qa runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md). - **Repeat-run execution model.** agent-qa: Yes; Puppeteer: Yes. A normal Puppeteer script repeats its code without requiring LLM inference. agent-qa uses models to interpret test intent and can reuse matched action plans while observing the live application. Evaluate that extra flexibility against its measured model usage and maintenance benefit. Sources: [Puppeteer API and MCP introduction](https://pptr.dev/), [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md), [Evaluate agent-qa on your application](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md). ## Why build recurring QA around agent-qa ### Review behavior without reviewing browser plumbing A teammate can review a YAML journey and its expected result alongside the feature change. agent-qa handles action planning and offers bounded recovery, reducing the amount of browser-control code your team needs to author for that journey. ### Give the next run useful product context agent-qa adds a file-backed memory lifecycle to the test suite. Observations can be inspected, corrected and reused across relevant tests, instead of requiring your team to design a separate application-knowledge layer around Puppeteer. ### Hand agents a complete verification task The coding agent can select a test, execute it, inspect failed-step evidence and rerun the affected journey through the documented QA loop. The same runtime can also target a native mobile app when the release crosses platforms. ## Verdict Choose agent-qa when the goal is recurring product verification with readable expectations, application memory and evidence an agent can use. It brings the QA workflow together above browser-control code, with native mobile coverage in the same runtime. > Note: This compares the documented Puppeteer library and its linked Chrome DevTools MCP integration with agent-qa's packaged QA runtime. Custom Puppeteer harnesses may already implement reporting or other workflow features. No head-to-head performance benchmark is claimed. ## Evaluate agent-qa against Puppeteer 1. Choose a maintained Puppeteer test with a meaningful assertion, retain its fixtures, and describe the same acceptance criteria in agent-qa YAML. 2. Run the two checks on identical working and deliberately broken app revisions. Record incorrect passes, incorrect failures, incomplete runs and the evidence explaining each verdict. 3. Make a harmless UI change while preserving behavior. Measure what maintenance each test needs, and verify that any agent-qa adaptation preserves the original expectation. 4. Repeat the tests and compare total runtime, agent-qa model usage and cache behavior, and the human work needed to diagnose failures. Include the model-free execution of ordinary Puppeteer scripts. [Set up your coding agent](https://vostride.com/docs/agent-qa/agent-quickstart.md) · [Evaluation guide](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) · [Recorded QA evidence](https://vostride.com/docs/agent-qa/guides/recorded-evidence.md) ## Frequently asked questions ### Is agent-qa a replacement for the Puppeteer library? agent-qa replaces the work of assembling a recurring QA workflow for suitable user journeys. It is not a drop-in implementation of Puppeteer's JavaScript API. Choose agent-qa for natural-language acceptance tests, application memory and agent-operated diagnosis; retain Puppeteer where precise browser scripting or an existing deterministic check already meets the requirement. ### Does Puppeteer support AI agents through MCP? Yes. Puppeteer's documentation recommends Chrome DevTools MCP, a Puppeteer-based server for controlling and inspecting Chrome. That integration includes debugging and performance tools. agent-qa's distinction is the persistent QA workflow around test contracts, run records and failure evidence, rather than exclusive access to agents or MCP. ### Are Puppeteer locators always brittle? No. Puppeteer locators offer automatic waiting and retries, and ARIA selectors can target accessible names and roles without depending on a particular DOM structure. agent-qa adds planning from natural-language intent when the action path needs to change. Test either approach against both a harmless UI change and a broken business outcome. ### Can agent-qa replace Puppeteer mobile emulation? For responsive website checks, configure the web target and viewport appropriate to your test. For native app QA, agent-qa supports Android and iOS targets using Appium and configured local or BrowserStack devices. Puppeteer's device emulation changes browser settings; it does not execute a native application. Native testing still needs the corresponding app build and device setup. ### How should I migrate an existing Puppeteer regression suite? Start with a user journey where ongoing script maintenance is costly. Reuse the fixture and expected outcome, then write an agent-qa YAML test and run both on the same working and deliberately broken app revisions. Keep useful deterministic checks. Compare failure evidence, maintenance effort and runtime cost before moving further coverage; ordinary Puppeteer scripts do not need LLM tokens. ## Sources - [Puppeteer API and MCP introduction](https://pptr.dev/) - [Puppeteer locators and selectors](https://pptr.dev/guides/page-interactions) - [Puppeteer screenshot API](https://pptr.dev/guides/screenshots) - [Puppeteer debugging](https://pptr.dev/guides/debugging) - [Puppeteer supported browsers](https://pptr.dev/supported-browsers) - [Puppeteer device emulation](https://pptr.dev/api/puppeteer.page.emulate) - [Puppeteer Docker setup](https://pptr.dev/guides/docker) - [Chrome DevTools MCP repository](https://github.com/ChromeDevTools/chrome-devtools-mcp) - [agent-qa test contract](https://vostride.com/docs/agent-qa/configuration/test.md) - [agent-qa coding-agent verification loop](https://vostride.com/docs/agent-qa/guides/coding-agent-workflow.md) - [agent-qa application memory](https://vostride.com/docs/agent-qa/memory.md) - [agent-qa action caching](https://vostride.com/docs/agent-qa/caching.md) - [agent-qa native mobile testing](https://vostride.com/docs/agent-qa/guides/mobile-testing.md) - [agent-qa runtime configuration](https://vostride.com/docs/agent-qa/configuration/global-config.md) - [Evaluate agent-qa on your application](https://vostride.com/docs/agent-qa/guides/evaluating-agent-qa.md) ## Compare other approaches - [agent-qa vs Playwright](https://vostride.com/playwright-alternative.md): Compare agent-qa with Playwright: AI-native natural-language E2E tests with memory versus hand-written browser automation scripts. - [agent-qa vs Stagehand](https://vostride.com/stagehand-alternative.md): Compare Stagehand's browser-agent SDK with agent-qa's YAML expectations, behavioral memory, local cache, and native mobile testing. - [agent-qa vs Midscene.js](https://vostride.com/midscene-alternative.md): Both support natural-language UI tests. Choose agent-qa for recurring QA with reviewable application memory and a complete MCP workflow. --- # agent-qa vs Momentic URL: https://vostride.com/momentic-alternative Markdown: https://vostride.com/momentic-alternative.md Momentic rents you AI testing. agent-qa gives you a QA agent that remembers, and it's yours. Momentic is a hosted product surface with vendor-controlled models and pricing. agent-qa puts plain-English tests, LLM choice, hooks, memory, and run evidence in the repo you already own. Try agent-qa, the source-available QA agent that rebuilds testing around repo memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Momentic: No. Momentic is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Momentic: No. Momentic keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Momentic: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Momentic: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Momentic: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Momentic: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Momentic: No. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Momentic: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. - **Browser control.** agent-qa: Yes; Momentic: Partial. agent-qa is designed for browser and mobile automation workflows where teams want the execution path visible and owned. ## Why teams switch ### Your tests stop being hostages Every test you write in Momentic makes leaving Momentic more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### You pick the model, not the vendor Momentic decides which AI runs your tests, when it changes, and what it costs you. agent-qa is bring-your-own-model: swap providers in one line, put a cheap model on smoke tests and a strong one on the flow that matters, or run against the internal endpoint your security team already signed off. ### The editor is not the product Momentic's polish lives in its hosted editor, which is exactly the problem. The durable asset in QA is the test intent, and Momentic keeps it in their app. agent-qa keeps it in files your team reviews, diffs, and ships like any other code. ## Verdict If you want AI testing as a subscription product, Momentic is a fine one. If you want AI testing as an engineering capability your team owns, with memory that compounds instead of a bill that does. That's agent-qa. ## Frequently asked questions ### Is agent-qa a good Momentic alternative? Yes, and the reason is structural rather than a feature count. Momentic is a hosted AI testing platform where tests, execution, and AI model choice live inside the vendor's product, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Momentic? Momentic is priced on vendor-set subscription plans on its hosted platform, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Momentic to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Your Momentic tests already describe user flows in high-level terms, so translating them into agent-qa's plain-English YAML is close to copy-editing. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Momentic? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Momentic is positioned primarily around web application testing; agent-qa treats mobile as a first-class target, not an add-on. ### Can I use my own LLM with Momentic or agent-qa? agent-qa is bring-your-own-LLM by design, configure any supported provider or compatible endpoint, route cheap models to smoke tests and stronger ones to complex flows. Momentic's AI execution runs on vendor-controlled models inside its platform. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Momentic public website](https://momentic.ai/) --- # agent-qa vs mabl URL: https://vostride.com/mabl-alternative Markdown: https://vostride.com/mabl-alternative.md mabl locks QA learning inside its platform. agent-qa keeps the memory in your repo. mabl sells a broad enterprise quality platform on quote-based contracts. agent-qa is the developer-owned harness: YAML in the repo, CLI and CI runs, and QA memory that compounds beside your code. Try agent-qa, the source-available alternative that stores QA memory beside code. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; mabl: No. mabl is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; mabl: No. mabl keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; mabl: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; mabl: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; mabl: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; mabl: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; mabl: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; mabl: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### No license fee, no seat math mabl runs on quote-based enterprise contracts sized for procurement, not for a team that just wants tests to run, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around mabl's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### An enterprise sales cycle for a developer problem Getting value from mabl means demos, quotes, onboarding, and training on their platform. Getting value from agent-qa means npx agent-qa init and a YAML file. E2E coverage is a developer-workflow problem, and it deserves a developer-workflow answer. ## Verdict mabl is what QA looks like when it's sold to a VP. agent-qa is what QA looks like when it's built for the engineers, and the coding agents, who actually ship the product. ## Frequently asked questions ### Is agent-qa a good mabl alternative? Yes, and the reason is structural rather than a feature count. mabl is a broad low-code enterprise quality platform covering web, mobile, API, and accessibility testing behind quote-based pricing, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to mabl? mabl is priced on quote-based annual enterprise contracts, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from mabl to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. mabl's low-code tests map to user journeys, and user journeys are exactly what agent-qa's natural-language tests describe. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like mabl? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. mabl also covers mobile in its platform tier; the difference is that agent-qa's mobile tests are files in your repo, not records in a vendor workspace. ### Why choose a source-available harness over mabl's enterprise platform? Because the test suite outlives the contract. mabl's tests, results, and learned behavior live in its platform and stop being useful when the subscription ends. agent-qa's tests, memory, and artifacts are repo files that survive vendor changes, budget cuts, and re-orgs. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [mabl public website](https://www.mabl.com/) - [mabl Series C announcement](https://www.mabl.com/awards-and-press/mabl-raises-40m-series-c) --- # agent-qa vs testRigor URL: https://vostride.com/testrigor-alternative Markdown: https://vostride.com/testrigor-alternative.md testRigor made English tests easy. agent-qa makes them learn. testRigor pushes no-code authoring through a managed platform. agent-qa makes plain-English tests pull-request artifacts, reviewed like code, run by coding agents, remembered across runs. Try agent-qa, the source-available way to turn plain-English runs into reusable memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; testRigor: No. testRigor is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; testRigor: No. testRigor keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; testRigor: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; testRigor: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; testRigor: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; testRigor: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; testRigor: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; testRigor: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting testRigor runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### Your tests stop being hostages Every test you write in testRigor makes leaving testRigor more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### No-code has a ceiling; code review doesn't Plain English is the right authoring format. testRigor got that part right. But serious teams need English tests that go through pull requests, diff cleanly, and sit next to hooks and config. That's a repo workflow, not a platform feature. ## Verdict testRigor proved plain-English testing works. agent-qa is what it looks like when plain English meets source control, coding agents, and memory, without the platform in between. ## Frequently asked questions ### Is agent-qa a good testRigor alternative? Yes, and the reason is structural rather than a feature count. testRigor is a no-code platform where plain-English tests are authored and executed inside the vendor's product, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to testRigor? testRigor is priced on tiered platform subscriptions, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from testRigor to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. testRigor tests are already written in plain English, which makes them the easiest kind to port. The intent transfers almost verbatim into agent-qa YAML. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like testRigor? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. testRigor supports mobile through its platform as well; agent-qa's difference is that the same YAML contract, memory, and artifacts serve web and mobile from your repo. ### Both use plain English, what's actually different? Where the English lives and what happens after a run. testRigor stores tests in its platform and each run stands alone. agent-qa stores tests in your repo, routes them through code review, and feeds every run's findings into memory and cache so execution gets faster and more reliable over time. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [testRigor public website](https://testrigor.com/) --- # agent-qa vs Autonoma URL: https://vostride.com/autonoma-alternative Markdown: https://vostride.com/autonoma-alternative.md Autonoma connects to your repo. agent-qa moves in, and remembers every run. Autonoma focuses on connected agent testing from its platform. agent-qa makes the repo the control plane: YAML, MCP, Skills, cache, hooks, and memory that compounds where your code lives. Try agent-qa, the source-available control plane where every run adds repo memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Autonoma: Yes. Both publish source. What differs is how much of the product that covers: agent-qa's engine, authoring format, memory store and agent workflow are all in the repo you clone, under FSL-1.1-ALv2. - **Repo-owned YAML.** agent-qa: Yes; Autonoma: Partial. agent-qa makes plain-English YAML the contract of record, so changing a test is a diff a reviewer can argue with rather than something somebody did in an app. - **Coding-agent native.** agent-qa: Yes; Autonoma: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Autonoma: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Autonoma: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Autonoma: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Autonoma: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Autonoma: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around Autonoma's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### Runs compound instead of resetting Autonoma runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### Connected-to is not the same as owned-by Autonoma reaches into your repo from its platform. agent-qa inverts that: the repo is the platform, and everything, tests, hooks, memory, cache, artifacts, is a file your team can read, review, and keep. ## Verdict Autonoma and agent-qa agree that agents should run QA. They disagree on who should own the result. agent-qa's answer: you, in your repo, forever. ## Frequently asked questions ### Is agent-qa a good Autonoma alternative? Yes, and the reason is structural rather than a feature count. Autonoma is an agent-testing product that connects to your repositories from its own platform, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Autonoma? Autonoma is priced on vendor-set product pricing, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Autonoma to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Since Autonoma already works at the level of user intent, rewriting its flows as agent-qa YAML tests preserves the intent while gaining code review. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Autonoma? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Autonoma covers app testing through its product; agent-qa runs web and mobile from the same repo-owned YAML and CLI path. ### Both are agentic, why does repo ownership matter? Because agents are only as good as the context they keep. agent-qa's memory, cache, and artifacts are files beside your code, versioned with it and visible to every coding agent you run. A platform-side agent's learning stays on the platform's side of the fence. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Autonoma public website](https://getautonoma.com/) --- # agent-qa vs Octomind URL: https://vostride.com/octomind-alternative Markdown: https://vostride.com/octomind-alternative.md Octomind is winding down. Your QA memory shouldn't die with it. Octomind publicly announced its app turns off at the end of May 2026. agent-qa is the source-available replacement path where your tests, and everything they learned, stay yours no matter what happens to any vendor. Try agent-qa, the source-available replacement path with memory your team can keep. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Octomind: No. Octomind is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Octomind: No. Octomind keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Octomind: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Octomind: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Octomind: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Octomind: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Octomind: No. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **Replacement path.** agent-qa: Yes; Octomind: No. The intent you have already written survives the move, in code you own, rather than being re-recorded into somebody else's format. ## Why teams switch ### Continuity risk stopped being hypothetical Octomind's wind-down is the case study every hosted-QA skeptic warned about: the tests, the run history, the learned behavior. All of it lives on infrastructure that is going away. Keeping tests and evidence in your own repo is the structural fix. ### Your tests stop being hostages Every test you write in Octomind makes leaving Octomind more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### Runs compound instead of resetting Octomind runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ## Verdict The lesson of Octomind isn't that AI testing failed. It's that renting your QA can fail you. agent-qa makes the replacement durable: source available, in your repo, with memory you keep. > Note: Octomind publicly announced that its app would turn off at the end of May 2026, so this page frames agent-qa as a replacement path. ## Frequently asked questions ### Is agent-qa a good Octomind alternative? Yes, and the reason is structural rather than a feature count. Octomind is an AI E2E testing product whose team publicly announced the app will shut down at the end of May 2026, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Octomind? Octomind is priced on a product that is being discontinued, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Octomind to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Treat migration as an intent transfer before the shutdown deadline: list your Octomind test cases, then re-express each as an agent-qa YAML flow. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Octomind? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Octomind focused on web E2E; agent-qa covers web and mobile from one contract, so the replacement also expands your coverage. ### What happens to my Octomind tests after the shutdown? Per Octomind's public wind-down notice, the hosted app turns off, which is exactly why the replacement should be source-owned. agent-qa tests are YAML files in your repository; no vendor decision can turn them off, and their accumulated memory stays with your code. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Octomind public website](https://octomind.dev/) - [Octomind public wind-down notice](https://octomind.dev/blog/a-letter-to-our-users-customers-and-readers/) --- # agent-qa vs Autosana URL: https://vostride.com/autosana-alternative Markdown: https://vostride.com/autosana-alternative.md Autosana runs your tests. agent-qa remembers what they proved. Autosana emphasizes natural-language tests through its product flow. agent-qa keeps the contract, local and CI execution, hooks, and artifacts under engineering control, and every run leaves memory behind. Try agent-qa, the source-available route for web and mobile QA that remembers prior runs. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Autosana: No. Autosana is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Autosana: No. Autosana keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Autosana: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Autosana: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Autosana: Yes. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Autosana: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Autosana: No. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Autosana: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting Autosana runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### You pick the model, not the vendor Autosana decides which AI runs your tests, when it changes, and what it costs you. agent-qa is bring-your-own-model: swap providers in one line, put a cheap model on smoke tests and a strong one on the flow that matters, or run against the internal endpoint your security team already signed off. ### Repo ownership outlasts any startup roadmap Betting your QA suite on an early-stage vendor's product decisions is a real risk, pricing changes, pivots, acquisitions, shutdowns. agent-qa is source available, and its tests, memory, and evidence stay inspectable in your repo whatever happens to anyone's roadmap. ## Verdict Autosana makes natural-language testing feel fast. agent-qa makes it compound: every run adds memory, every test is a file you own, and no product flow stands between your team and its QA. ## Frequently asked questions ### Is agent-qa a good Autosana alternative? Yes, and the reason is structural rather than a feature count. Autosana is a natural-language testing product where authoring and execution flow through the vendor's app, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Autosana? Autosana is priced on vendor-set product subscriptions, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Autosana to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Autosana flows are described in natural language already, so each maps one-to-one onto an agent-qa YAML test your team then owns in git. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Autosana? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Autosana covers mobile testing too; agent-qa's edge is that web and mobile share one repo-owned YAML contract, one memory store, and one CLI. ### Can agent-qa run tests locally and in CI like Autosana? Yes, and identically in both. The same npx agent-qa run command executes on a laptop, in a CI job, or from a coding agent via MCP, with artifacts and memory written to the same repo-visible locations every time. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Autosana public website](https://autosana.ai/) - [Autosana documentation](https://docs.autosana.ai/) --- # agent-qa vs SpurTest URL: https://vostride.com/spur-alternative Markdown: https://vostride.com/spur-alternative.md SpurTest sells QA confidence. agent-qa gives you QA memory you own. SpurTest is shaped around managed and e-commerce QA workflows. agent-qa is a general-purpose, repo-owned harness for teams that want engineers, and their agents, verifying product changes directly. Try agent-qa, the source-available path away from vendor workflows toward repo memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; SpurTest: No. SpurTest is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; SpurTest: No. SpurTest keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; SpurTest: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; SpurTest: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; SpurTest: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; SpurTest: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; SpurTest: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; SpurTest: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Your tests stop being hostages Every test you write in SpurTest makes leaving SpurTest more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around SpurTest's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### General-purpose beats vertical-shaped SpurTest's workflows lean toward e-commerce confidence checks. Your product is not a template. agent-qa tests whatever your users actually do, any web flow, any mobile flow, described in plain English and versioned with the code that implements it. ## Verdict Managed QA workflows buy comfort and cost you compounding. agent-qa puts the verification loop, and everything it learns, back inside your engineering team. ## Frequently asked questions ### Is agent-qa a good SpurTest alternative? Yes, and the reason is structural rather than a feature count. SpurTest is a managed QA product oriented around e-commerce and workflow confidence, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to SpurTest? SpurTest is priced on managed product plans, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from SpurTest to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. List the user journeys SpurTest watches for you, then write each as an agent-qa YAML test. The intent is the migration. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like SpurTest? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. agent-qa covers mobile E2E as a first-class target alongside web, with the same YAML authoring and evidence trail. ### Do I lose anything by leaving a managed QA workflow? You trade a vendor-run comfort layer for direct control, and gain speed. Failures land as artifacts and classified evidence in your workflow rather than tickets in someone else's queue, and coding agents can act on them immediately because they live where the code lives. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [SpurTest public website](https://www.spurtest.com/) --- # agent-qa vs Bug0 URL: https://vostride.com/bug0-alternative Markdown: https://vostride.com/bug0-alternative.md Bug0 manages your QA. agent-qa turns every run into memory your agents keep. Bug0 leans into managed QA help. agent-qa hands your own team the open framework, test files, model routing, runtime evidence, with no outsourced operating model in the loop. Try agent-qa, the source-available way to let agents learn from every proof. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Bug0: No. Bug0 is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Bug0: No. Bug0 keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Bug0: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Bug0: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Bug0: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Bug0: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Bug0: No. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Bug0: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting Bug0 runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### No license fee, no seat math Bug0 runs on managed-service pricing where the vendor's margin is built into every test cycle, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### Outsourced QA never compounds in-house When Bug0's service finds a bug, the knowledge of how it was found stays with the service. When agent-qa finds one, the test, the memory entry, and the run artifacts all land in your repo, your team and your coding agents get permanently smarter. ## Verdict Bug0 rents you a QA operating model. agent-qa gives your team one, source available, agent-native, and accumulating memory with every release. ## Frequently asked questions ### Is agent-qa a good Bug0 alternative? Yes, and the reason is structural rather than a feature count. Bug0 is a managed QA offering where the vendor operates the testing loop for you, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Bug0? Bug0 is priced on managed-service plans, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Bug0 to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Ask what flows Bug0 currently covers, then encode each as an agent-qa YAML test. You convert a service dependency into a repo asset. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Bug0? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. agent-qa treats web and mobile as equal targets under one repo-owned contract, wherever your product runs. ### Is a managed service easier than running agent-qa myself? Day one, maybe. Day ninety, no: agent-qa's authoring is plain English, runs are one CLI command or one MCP call from your coding agent, and memory plus cache mean the suite maintains itself more with every run. Easy that compounds beats easy that invoices. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Bug0 public website](https://bug0.com/) --- # agent-qa vs TestSprite URL: https://vostride.com/testsprite-alternative Markdown: https://vostride.com/testsprite-alternative.md TestSprite generates tests in its cloud. agent-qa learns in your repo. TestSprite focuses on autonomous generation and cloud verification. agent-qa turns agentic QA into explicit YAML, hooks, memory, and evidence that stay with the code your agents change. Try agent-qa, the source-available way to turn generated QA into remembered evidence. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; TestSprite: No. TestSprite is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; TestSprite: No. TestSprite keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; TestSprite: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; TestSprite: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; TestSprite: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; TestSprite: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; TestSprite: No. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; TestSprite: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around TestSprite's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### Your tests stop being hostages Every test you write in TestSprite makes leaving TestSprite more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### Generated tests you can't review are liabilities Autonomous generation without code review produces coverage you can't reason about. agent-qa keeps generation agentic but lands every test as reviewable YAML in a pull request, so a human (or another agent) can see exactly what's being promised. ## Verdict TestSprite automates testing into its cloud. agent-qa automates it into your codebase, where review, memory, and your coding agents already live. ## Frequently asked questions ### Is agent-qa a good TestSprite alternative? Yes, and the reason is structural rather than a feature count. TestSprite is an autonomous testing platform that generates and verifies tests in the vendor's cloud, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to TestSprite? TestSprite is priced on cloud platform subscriptions, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from TestSprite to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Export the intent of your TestSprite coverage, what each generated test verifies, and re-state it as agent-qa YAML flows your team reviews once and owns forever. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like TestSprite? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. agent-qa runs mobile E2E natively alongside web; generated or hand-written, the tests share one repo-owned format. ### Can coding agents generate agent-qa tests the way TestSprite generates tests? Yes. That's the native workflow. agent-qa ships MCP tools and packaged Skills so Claude Code, Cursor, and similar agents can author tests from product context, validate them, run them, and file the results, with everything landing in your repo instead of a vendor cloud. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [TestSprite public website](https://www.testsprite.com/) --- # agent-qa vs BaseRock URL: https://vostride.com/baserock-alternative Markdown: https://vostride.com/baserock-alternative.md BaseRock talks business signals. agent-qa remembers how your product actually behaves. BaseRock frames testing around business-signal validation. agent-qa focuses on source-owned web and mobile E2E execution that engineers and coding agents run directly, with proof that accumulates. Try agent-qa, the source-available way to keep E2E proof and memory in engineering hands. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; BaseRock: No. BaseRock is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; BaseRock: No. BaseRock keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; BaseRock: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; BaseRock: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; BaseRock: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; BaseRock: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; BaseRock: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; BaseRock: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting BaseRock runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### You pick the model, not the vendor BaseRock decides which AI runs your tests, when it changes, and what it costs you. agent-qa is bring-your-own-model: swap providers in one line, put a cheap model on smoke tests and a strong one on the flow that matters, or run against the internal endpoint your security team already signed off. ### Business signals aren't E2E proof Signal-level validation tells you something moved; it doesn't tell you the checkout flow works on the build you're about to ship. agent-qa produces behavioral proof, step-by-step runs with artifacts, tied to the exact source state that produced them. ## Verdict BaseRock reports on your product from the outside. agent-qa proves your product from the inside, run by run, release by release, in files you keep. ## Frequently asked questions ### Is agent-qa a good BaseRock alternative? Yes, and the reason is structural rather than a feature count. BaseRock is an AI testing product framed around validating business signals, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to BaseRock? BaseRock is priced on vendor platform pricing, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from BaseRock to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Identify the user-visible behaviors behind each BaseRock signal, then write those behaviors as agent-qa YAML tests. You go from monitoring signals to proving flows. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like BaseRock? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. agent-qa executes true device-level flows on Android and iOS as well as web, not just signal checks. ### Can agent-qa results feed business reporting like BaseRock? agent-qa emits structured run results, artifacts, and failure classifications you can pipe anywhere, dashboards, CI gates, Slack, or a coding agent's triage loop. The difference is the underlying evidence is a real E2E run, stored with your code. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [BaseRock public website](https://www.baserock.ai/) --- # agent-qa vs QA Wolf URL: https://vostride.com/qawolf-alternative Markdown: https://vostride.com/qawolf-alternative.md QA Wolf bills like an agency. agent-qa has no paid tier, is source available, and remembers every release. QA Wolf sells managed QA execution on service contracts, humans and infrastructure on their side of the fence. agent-qa is the self-service path where tests, models, hooks, and artifacts stay in-house. Try agent-qa, the source-available alternative where each release builds QA memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; QA Wolf: No. QA Wolf is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; QA Wolf: No. QA Wolf keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; QA Wolf: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; QA Wolf: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; QA Wolf: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; QA Wolf: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; QA Wolf: No. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; QA Wolf: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### No license fee, no seat math QA Wolf runs on quote-based managed-service contracts, you're paying for people and process, renewed annually, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### Runs compound instead of resetting QA Wolf runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### Humans in the loop don't scale with agents QA Wolf's model puts vendor humans between your release and your green check. As your own coding agents write more of the code, they need a QA loop they can call directly, MCP tools, CLI runs, artifacts, not a service queue on someone else's SLA. ## Verdict QA Wolf charges service-contract money to keep tests green. agent-qa makes green tests a property of your repo, self-improving, agent-runnable, and available without a paid tier. ## Frequently asked questions ### Is agent-qa a good QA Wolf alternative? Yes, and the reason is structural rather than a feature count. QA Wolf is a managed QA service where the vendor's team builds and maintains your test coverage under a service contract, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to QA Wolf? QA Wolf is priced on quote-based annual service contracts, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from QA Wolf to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. You already know your critical flows, they're what QA Wolf's team automated for you. Re-describe them as agent-qa YAML tests and the coverage comes in-house along with everything future runs learn. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like QA Wolf? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. QA Wolf has expanded toward mobile through its service; agent-qa gives you web plus Android and iOS from one source-available harness, without a service engagement. ### QA Wolf promises 80%+ coverage, can agent-qa match that? Coverage is a function of writing tests, and agent-qa makes tests radically cheap to write: plain-English YAML that engineers or coding agents produce in minutes, with memory and caching keeping runs stable as the product changes. The difference is you own the coverage instead of renting the team that maintains it. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [QA Wolf documentation](https://docs.qawolf.com/qawolf/Welcome-to-QA-Wolf) - [QA Wolf Series B announcement](https://www.prnewswire.com/news-releases/qa-wolf-raises-36-million-series-b-and-opens-waitlist-for-android-and-ios-test-automation-302204038.html) --- # agent-qa vs Katalon URL: https://vostride.com/katalon-alternative Markdown: https://vostride.com/katalon-alternative.md Katalon ships a suite. agent-qa ships a QA agent that learns your product. Katalon is a broad test automation platform with its own IDE and per-seat licensing. agent-qa is the lightweight repo-native path when the goal is agent-run E2E living beside the code. Try agent-qa, the source-available way to keep E2E tests and memory in one repo loop. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Katalon: No. Katalon is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Katalon: No. Katalon keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Katalon: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Katalon: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Katalon: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Katalon: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Katalon: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Katalon: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around Katalon's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### No license fee, no seat math Katalon runs on per-seat licensing across a studio IDE and platform tiers, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### The IDE is the lock-in Katalon coverage means Katalon Studio projects, Katalon formats, Katalon runtime engines. agent-qa coverage means YAML files any editor opens, any engineer reviews, and any coding agent extends, the suite sprawl never starts. ## Verdict Katalon optimizes for covering every testing category. agent-qa optimizes for the loop that matters: code changes, agent verifies, memory compounds. Pick the tool shaped like your workflow. ## Frequently asked questions ### Is agent-qa a good Katalon alternative? Yes, and the reason is structural rather than a feature count. Katalon is a broad test automation platform and IDE covering web, mobile, API, and desktop with per-seat licensing, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Katalon? Katalon is priced on per-seat and platform-tier licensing, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Katalon to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Katalon test cases encode user flows under studio-specific structure; strip them back to intent and each becomes a short agent-qa YAML file. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Katalon? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Katalon covers mobile through its studio tooling; agent-qa covers it through the same plain-English YAML and CLI you use for web, no separate tooling to learn. ### Is agent-qa lighter to adopt than Katalon? Substantially. There is no IDE to install, no project format to learn, and no license to assign: npx agent-qa init scaffolds the workspace, your first plain-English test runs minutes later, and everything lives in the repo your team already works in. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Katalon Studio public website](https://katalon.com/katalon-studio/) --- # agent-qa vs Functionize URL: https://vostride.com/functionize-alternative Markdown: https://vostride.com/functionize-alternative.md Functionize sells a platform. agent-qa is the QA agent that learns inside your workflow. Functionize packages AI testing as an enterprise platform sale. agent-qa keeps test ownership, model choice, and verification evidence in the developer workflow, no platform between you and your proof. Try agent-qa, the source-available workflow your coding agents can run and remember. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Functionize: No. Functionize is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Functionize: No. Functionize keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Functionize: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Functionize: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Functionize: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Functionize: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Functionize: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Functionize: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### You pick the model, not the vendor Functionize decides which AI runs your tests, when it changes, and what it costs you. agent-qa is bring-your-own-model: swap providers in one line, put a cheap model on smoke tests and a strong one on the flow that matters, or run against the internal endpoint your security team already signed off. ### Your tests stop being hostages Every test you write in Functionize makes leaving Functionize more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### Platform AI is a black box; repo AI is a diff When Functionize's cloud AI adapts a test, the adaptation happens inside the platform. When agent-qa adapts, the evidence lands in run artifacts and memory files you can read, and the test itself stays a reviewable YAML document with a git history. ## Verdict Functionize wraps AI testing in an enterprise platform. agent-qa strips the platform away and leaves what teams actually need: intent in YAML, adaptation with receipts, memory in the repo. ## Frequently asked questions ### Is agent-qa a good Functionize alternative? Yes, and the reason is structural rather than a feature count. Functionize is an enterprise AI testing platform where authoring, execution, and AI adaptation happen in the vendor's cloud, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Functionize? Functionize is priced on enterprise platform contracts, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Functionize to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Inventory the journeys your Functionize suites cover and restate each as an agent-qa plain-English test; the AI adaptation you relied on comes along, but inspectable this time. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Functionize? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. agent-qa's mobile support uses the same YAML contract and CLI as web, so one skill set covers both surfaces. ### How does agent-qa's self-healing compare to Functionize's? Both adapt when the UI changes. The difference is transparency and ownership: agent-qa re-plans from natural-language intent, records what changed in file-backed memory, and caches the corrected plan, all in artifacts your team can audit rather than a platform's internal state. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Functionize public website](https://www.functionize.com/) --- # agent-qa vs Tricentis URL: https://vostride.com/tricentis-alternative Markdown: https://vostride.com/tricentis-alternative.md Tricentis was built for procurement. agent-qa learns your product from inside the repo. Tricentis serves enterprise continuous-testing programs with enterprise licensing to match. agent-qa is for engineering teams that want fast, inspectable E2E checks in the same workflow as code changes. Try agent-qa, the source-available option for agent-run QA that learns without suite weight. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Tricentis: No. Tricentis is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Tricentis: No. Tricentis keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Tricentis: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Tricentis: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Tricentis: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Tricentis: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Tricentis: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Tricentis: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### No license fee, no seat math Tricentis runs on enterprise license agreements sized for testing programs, not product teams, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around Tricentis's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### Procurement-grade weight, repo-sized problem Tricentis exists for organizations that manage testing as a program, with the rollout time, training, and licensing that implies. If your actual problem is 'verify the app before every release,' agent-qa addresses it from your repo without a paid tier or platform fee. ## Verdict Tricentis is a testing program. agent-qa is a testing loop. Teams shipping continuously need the loop, and they need to own it. ## Frequently asked questions ### Is agent-qa a good Tricentis alternative? Yes, and the reason is structural rather than a feature count. Tricentis is an enterprise continuous-testing suite (Tosca and related products) sold through enterprise license agreements, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Tricentis? Tricentis is priced on enterprise license agreements, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Tricentis to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. You don't migrate a Tricentis program overnight. You carve out the product-team E2E layer first. Start with release-blocking user flows as agent-qa tests and let the suite earn its way outward. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Tricentis? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Tricentis covers mobile across its suite; agent-qa covers it with the same lightweight YAML-and-CLI loop it uses for web. ### Can agent-qa really replace an enterprise testing suite? For model-based enterprise programs spanning SAP and legacy estates, Tricentis is built for that scope. For web and mobile E2E verification wired into engineering and coding-agent workflows, agent-qa offers a repo-owned YAML, local/CI, and coding-agent workflow instead of an enterprise suite. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Tricentis public website](https://www.tricentis.com/) - [Tricentis GTCR investment announcement](https://www.tricentis.com/news/gtcr-makes-1-33-billion-investment-in-tricentis) --- # agent-qa vs LambdaTest URL: https://vostride.com/lambdatest-alternative Markdown: https://vostride.com/lambdatest-alternative.md LambdaTest sells you a grid. agent-qa builds QA memory you keep. LambdaTest supplies a large testing cloud priced around parallel sessions and platform tiers. agent-qa keeps the natural-language contract, LLM choice, and run evidence inside the repository workflow. Try agent-qa, the source-available path where test intent and memory stay out of dashboards. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; LambdaTest: No. LambdaTest is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; LambdaTest: No. LambdaTest keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; LambdaTest: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; LambdaTest: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; LambdaTest: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; LambdaTest: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; LambdaTest: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; LambdaTest: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Your tests stop being hostages Every test you write in LambdaTest makes leaving LambdaTest more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### Runs compound instead of resetting LambdaTest runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### Grid minutes meet agent economics Cloud-grid pricing made sense when humans wrote every script and ran suites nightly. Coding agents verify continuously, and per-session platform pricing punishes exactly that. agent-qa runs where your compute already is, gated only by the LLM budget you set. ## Verdict LambdaTest wants to be where your testing happens. agent-qa makes your repo that place, and lets you rent, own, or skip infrastructure as you see fit. ## Frequently asked questions ### Is agent-qa a good LambdaTest alternative? Yes, and the reason is structural rather than a feature count. LambdaTest is a cloud testing platform selling browser/device infrastructure and an AI test control plane, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to LambdaTest? LambdaTest is priced on per-parallel-session and platform-tier subscriptions, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from LambdaTest to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Keep any infrastructure you like. The migration is moving the test contract out of the platform: re-express your covered flows as agent-qa YAML and run them wherever suits you. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like LambdaTest? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. LambdaTest offers device clouds; agent-qa executes mobile E2E through your own local or CI device setup, keeping evidence in the repo. ### Does agent-qa provide testing infrastructure like LambdaTest? No, deliberately. agent-qa is the harness, not the hardware: it runs against local browsers, CI runners, emulators, or devices you already have. That separation is why there's no per-session meter and why your tests aren't coupled to anyone's cloud. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [LambdaTest public website](https://www.lambdatest.com/) - [LambdaTest Series D coverage](https://techcrunch.com/2024/12/10/software-testing-platform-lambdatest-secures-38-million-in-ai-push/) --- # agent-qa vs Applitools URL: https://vostride.com/applitools-alternative Markdown: https://vostride.com/applitools-alternative.md Applitools compares pixels. agent-qa remembers behavior. Applitools is strongest at visual AI validation, sold as an enterprise platform. agent-qa proves behavior: repo-owned natural-language flows, agent execution, hooks, and artifact review. Try agent-qa, the source-available way to remember behavior, not just compare pixels. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Applitools: No. Applitools is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Applitools: No. Applitools keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Applitools: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Applitools: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Applitools: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Applitools: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Applitools: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Applitools: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting Applitools runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### No license fee, no seat math Applitools runs on enterprise platform pricing structured around visual checkpoints and seats, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### A pixel diff can't tell you the flow works Visual validation catches what changed on screen; it can't tell you whether checkout completed, the API succeeded, or the right side effects fired. agent-qa verifies the behavior end to end and keeps screenshots as evidence, not as the definition of correctness. ## Verdict Applitools answers 'does it look right?' agent-qa answers 'does it work?', and remembers the answer for the next run. Most teams need the second question answered first. ## Frequently asked questions ### Is agent-qa a good Applitools alternative? Yes, and the reason is structural rather than a feature count. Applitools is a visual AI validation platform centered on screenshot comparison and visual regression, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Applitools? Applitools is priced on enterprise plans around visual checkpoints and seats, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Applitools to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Keep visual snapshots where they earn their keep; move the flow-correctness layer to agent-qa by writing your critical journeys as plain-English YAML tests. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Applitools? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Applitools validates mobile visuals; agent-qa runs full mobile flows, taps, inputs, assertions, side effects, with visual artifacts attached to each step. ### Can agent-qa do visual checks like Applitools? agent-qa captures screenshots and artifacts at every step and can assert on visible state as part of a flow. It doesn't try to be a pixel-perfect visual regression engine. It makes behavioral correctness the contract and visuals the evidence, which is the priority order most teams actually need. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Applitools public website](https://applitools.com/) - [Applitools funding announcement](https://applitools.com/blog/press-release-applitools-raises-31m/) --- # agent-qa vs Rainforest QA URL: https://vostride.com/rainforestqa-alternative Markdown: https://vostride.com/rainforestqa-alternative.md Rainforest QA rents you testers. agent-qa gives your team a QA agent that learns. Rainforest QA leans into managed and no-code QA workflows. agent-qa keeps the durable test contract, runtime choices, and evidence in engineering hands, where they compound. Try agent-qa, the source-available path where runs build team memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Rainforest QA: No. Rainforest QA is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Rainforest QA: No. Rainforest QA keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Rainforest QA: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Rainforest QA: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Rainforest QA: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Rainforest QA: Partial. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Rainforest QA: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Rainforest QA: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting Rainforest QA runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around Rainforest QA's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### Crowd testers don't learn your product Every Rainforest run starts from zero product knowledge. That's the nature of externalized QA. agent-qa's memory files accumulate your product's actual behavior, so runs get more reliable and more informed with every release instead of perpetually re-discovering the basics. ## Verdict Rainforest scales QA by adding people. agent-qa scales it by adding memory. One of those gets cheaper and smarter every release. ## Frequently asked questions ### Is agent-qa a good Rainforest QA alternative? Yes, and the reason is structural rather than a feature count. Rainforest QA is a managed QA platform combining no-code automation with human testing services, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Rainforest QA? Rainforest QA is priced on managed platform and service plans, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Rainforest QA to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Your Rainforest test cases are written as human-followable steps, which is precisely the format agent-qa consumes: turn each into a plain-English YAML test and automation replaces the crowd. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Rainforest QA? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. agent-qa covers Android and iOS flows with the same repo-owned YAML used for web, no service loop required. ### When do humans still matter if agent-qa automates the flows? Exploratory judgment, design taste, and edge-case hunting stay human. What shouldn't stay human is regression proof on every release. That's mechanical verification, and agent-qa makes it automatic and remembered without per-run service fees. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Rainforest QA public website](https://www.rainforestqa.com/) --- # agent-qa vs Autify URL: https://vostride.com/autify-alternative Markdown: https://vostride.com/autify-alternative.md Autify is another login. agent-qa is QA memory living inside your repo. Autify focuses on no-code AI test automation in its own platform. agent-qa gives engineering teams source-controlled YAML, local and CI runs, hooks, memory, and model choice, no extra surface to maintain. Try agent-qa, the source-available workflow that keeps QA memory out of another login. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; Autify: No. Autify is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; Autify: No. Autify keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; Autify: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; Autify: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; Autify: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; Autify: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; Autify: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; Autify: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Your tests stop being hostages Every test you write in Autify makes leaving Autify more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### You pick the model, not the vendor Autify decides which AI runs your tests, when it changes, and what it costs you. agent-qa is bring-your-own-model: swap providers in one line, put a cheap model on smoke tests and a strong one on the flow that matters, or run against the internal endpoint your security team already signed off. ### No-code platforms cap out at scenarios Platform QA is priced and shaped around scenario counts and plan tiers. Repo QA has no such ceiling: agent-qa tests are files, add as many as your product needs, organize them into suites, and let hooks and memory keep them honest. ## Verdict Autify moves QA into another product. agent-qa moves it into the product you already maintain, your codebase, and lets it learn there. ## Frequently asked questions ### Is agent-qa a good Autify alternative? Yes, and the reason is structural rather than a feature count. Autify is a no-code AI test automation platform where scenarios are created and run inside the vendor's product, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to Autify? Autify is priced on plan tiers shaped around scenarios and platform usage, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from Autify to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Autify scenarios are recorded user journeys; write each journey's intent as an agent-qa YAML test and you gain code review, memory, and LLM choice in the same move. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like Autify? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. Autify offers a mobile product line; agent-qa handles web and mobile in one harness with one authoring format and one memory store. ### Do I need to code to use agent-qa, unlike Autify's no-code approach? You write plain English in a YAML file, closer to a checklist than to code. If your team can write an Autify scenario description, it can write an agent-qa test; the difference is the file lands in git where reviews, agents, and history already work. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Autify public website](https://autify.com/) - [Autify Series A announcement](https://autify.com/news/series-a-ef4c7) --- # agent-qa vs ACCELQ URL: https://vostride.com/accelq-alternative Markdown: https://vostride.com/accelq-alternative.md ACCELQ spans every channel. agent-qa learns the one product you actually ship. ACCELQ covers enterprise codeless automation across web, mobile, API, and packaged apps, per-user licensed. agent-qa narrows the job to agent-run E2E checks that live with product code. Try agent-qa, the source-available alternative for fast, inspectable QA with memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; ACCELQ: No. ACCELQ is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; ACCELQ: No. ACCELQ keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; ACCELQ: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; ACCELQ: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; ACCELQ: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; ACCELQ: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; ACCELQ: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; ACCELQ: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### No license fee, no seat math ACCELQ runs on per-user enterprise licensing across its codeless platform modules, and that number grows with the coverage you add. agent-qa has no paid tier or licence fee for FSL-permitted use and its source is available under FSL-1.1-ALv2. You pay for tokens and infrastructure you control, on whichever provider is cheapest this quarter, and the cache cuts that too. ### Built for coding agents, not dashboards Your team already ships code with coding agents, and a coding agent cannot click around ACCELQ's interface. agent-qa ships MCP tools, packaged Skills and a CLI, so Claude Code, Cursor and their peers author the test, run it and triage the failure inside the same loop that wrote the change. ### Breadth is a tax when you need depth Every extra channel a platform covers is complexity you fund but may never use. agent-qa does one job, natural-language E2E for web and mobile, and does it with memory, cache, and hooks that make the hundredth run better than the first. ## Verdict ACCELQ is built to check every box in an RFP. agent-qa is built to verify your product on every commit. Those are different tools, and only one belongs in your repo. ## Frequently asked questions ### Is agent-qa a good ACCELQ alternative? Yes, and the reason is structural rather than a feature count. ACCELQ is an enterprise codeless test automation platform spanning web, mobile, API, and packaged applications, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to ACCELQ? ACCELQ is priced on per-user enterprise licensing, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from ACCELQ to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. Extract the business flows your ACCELQ scenarios encode and restate them as agent-qa plain-English tests, intent survives, platform structure doesn't need to. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like ACCELQ? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. ACCELQ includes mobile modules in its platform; agent-qa delivers mobile E2E from the same repo-owned YAML contract as web. ### We test more than web and mobile, does agent-qa still fit? agent-qa focuses on web and mobile E2E and pairs cleanly with whatever covers your API or packaged-app layers. Its hooks can call project scripts and API checks as part of a run. Focused tools composed in the repo beat one platform stretched across everything. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [ACCELQ public website](https://www.accelq.com/) --- # agent-qa vs MagicPod URL: https://vostride.com/magicpod-alternative Markdown: https://vostride.com/magicpod-alternative.md MagicPod automates clicks. agent-qa remembers why they matter. MagicPod runs web and mobile UI automation through its subscription product. agent-qa keeps natural-language test files, hooks, cache, memory, and artifacts in the developer workflow. Try agent-qa, the source-available way to keep UI test memory and LLM choice with your team. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Source access.** agent-qa: Yes; MagicPod: No. MagicPod is not positioned as a repo-owned framework with published source, so behaviour you disagree with is a support ticket. With agent-qa it is a pull request. - **Repo-owned YAML.** agent-qa: Yes; MagicPod: No. MagicPod keeps the test intent inside its own product. agent-qa keeps intent, config, hooks, memory and suites beside the code they cover, where your engineering process already works. - **Coding-agent native.** agent-qa: Yes; MagicPod: No. A coding agent cannot click through a hosted editor. agent-qa ships MCP tools, packaged Skills and a CLI, so the agent that changed the code writes the test, runs it and reads the failure without leaving the loop. - **Bring your own LLM.** agent-qa: Yes; MagicPod: No. Whoever picks the model sets your quality ceiling and your bill. agent-qa lets you point at any provider, any compatible endpoint, or a model on your own hardware, and change it in one line. - **Local and CI execution.** agent-qa: Yes; MagicPod: Partial. One command on a laptop, in CI, and from an agent. No run depends on somebody else's control plane being up, and nothing queues behind another tenant. - **Web and mobile QA.** agent-qa: Yes; MagicPod: Yes. Web, Android and iOS from the same natural-language flow and the same evidence model. The surface is a target named in a file, not a different product tier. - **Memory, cache, hooks.** agent-qa: Yes; MagicPod: Partial. Execution memory, a validated action cache and sandboxed hooks compound. A suite that has been running a month is faster, cheaper and better informed about your app than the day it was written. - **No platform lock-in.** agent-qa: Yes; MagicPod: No. Every durable asset stays in your repository. Cancel agent-qa tomorrow and the tests, the memory and the evidence are still there and still readable. ## Why teams switch ### Runs compound instead of resetting MagicPod runs a test and forgets. Every run starts from nothing, which is why the hundredth run costs exactly what the first one did. agent-qa writes what it learned into memory committed beside your tests, so the next run starts where the last one finished and the suite gets better at your app on its own. ### Your tests stop being hostages Every test you write in MagicPod makes leaving MagicPod more expensive. That is not an accident, it is the business model. Every test you write with agent-qa is a YAML file in your repository: reviewed in a pull request, portable to any runner, and still yours the day you cancel. ### Click automation without intent is brittle by design UI-level automation encodes how the screen looked when you recorded it. agent-qa encodes what the user is trying to do, so when the UI shifts, the runtime re-plans from intent instead of failing on a stale step, and writes the adaptation into memory. ## Verdict MagicPod keeps UI tests running inside its product. agent-qa keeps them learning inside yours. Over a year of releases, that difference is the whole game. ## Frequently asked questions ### Is agent-qa a good MagicPod alternative? Yes, and the reason is structural rather than a feature count. MagicPod is an AI-assisted test automation subscription product for web and mobile UI testing, which means the asset you are building lives on their side of the line. agent-qa is a source-available QA agent with no paid tier, governed by FSL-1.1-ALv2: tests are plain-English YAML in your repository, runs execute on your laptop, in your CI, or from your coding agent, and every run writes back into memory committed beside the tests. The suite gets better at your app whether or not you renew anything. ### How much does agent-qa cost compared to MagicPod? MagicPod is priced on monthly subscription plans, so the bill tracks how much you test. agent-qa has no paid tier, no seats and no platform fee; FSL-1.1-ALv2 governs permitted use. You pay for the model tokens and infrastructure you already control, on the provider you choose, and the validated action cache takes roughly 60% of the tokens off a matched rerun. Adding coverage does not add a line item. ### How do I migrate from MagicPod to agent-qa? You are re-describing intent, not porting code, which is why this is far smaller than a normal test migration. MagicPod test cases are step sequences over your UI; describe each sequence's goal in an agent-qa YAML test and the runtime rebuilds the steps, and keeps rebuilding them as your UI evolves. Run npx agent-qa init, write each critical flow as a plain-English YAML test, and let the runtime work out the selectors and the recovery. Most teams move a smoke suite in an afternoon, and there is nothing to un-pick later because the output is files in your own repository. ### Does agent-qa cover web and mobile like MagicPod? Yes, and from the same file. agent-qa runs end-to-end tests on web, Android and iOS with one natural-language format, one memory store and one evidence model, so a flow written once survives being pointed at another surface. MagicPod is notably mobile-friendly; agent-qa matches web and mobile coverage while keeping tests, memory, and evidence as repo files rather than platform records. ### How does agent-qa handle UI changes compared to MagicPod's auto-healing? MagicPod repairs recorded steps within its platform. agent-qa goes a layer deeper: tests are written as intent, so the runtime re-derives the right actions when screens change, caches the corrected plan, and records the behavioral change in memory your whole team, and your coding agents, can read. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [MagicPod public website](https://magicpod.com/) - [MagicPod public milestone announcement](https://magicpod.com/en/news/press-release/entry-535/) --- # agent-qa vs Playwright URL: https://vostride.com/playwright-alternative Markdown: https://vostride.com/playwright-alternative.md Love Playwright? Keep it. agent-qa is for the tests you're tired of maintaining, and it remembers. Playwright is a superb browser automation engine you program in TypeScript, Python, or Java. agent-qa operates a layer above: you state intent in plain English and the runtime plans, executes, adapts, and remembers. Try agent-qa, the source-available AI layer that turns test intent into runs that remember. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Plain-English authoring.** agent-qa: Yes; Playwright: No. Playwright tests are code, powerful, but every flow is programming work. agent-qa tests state user intent in natural-language YAML that anyone on the team can read and review. - **Survives UI changes.** agent-qa: Yes; Playwright: No. Playwright selectors and assertions break when the UI shifts, and a human fixes them. agent-qa re-plans from intent, caches the corrected plan, and records the change in memory. - **Execution memory.** agent-qa: Yes; Playwright: No. Playwright starts every run stateless. agent-qa accumulates file-backed behavioral memory, making later runs faster and more reliable. - **Source access.** agent-qa: Yes; Playwright: Yes. Playwright is open source; agent-qa is source available under FSL-1.1-ALv2. Both keep tests in the repo with no required platform or seats. - **Local and CI execution.** agent-qa: Yes; Playwright: Yes. Both run from a laptop, CI job, or automation pipeline with source-controlled commands. - **Mobile app testing.** agent-qa: Yes; Playwright: No. Playwright targets browsers (with experimental Android support). agent-qa covers native Android and iOS flows with the same YAML contract as web. - **Coding-agent native.** agent-qa: Yes; Playwright: Partial. Playwright has MCP-based browser control for agents; agent-qa ships a full agent QA loop, MCP tools, Skills, run artifacts, failure classification, and memory. - **Selector maintenance.** agent-qa: Yes; Playwright: No. agent-qa eliminates the selector-upkeep tax that consumes most Playwright suite maintenance time. 'Yes' here means no selectors to maintain. ## Why teams add agent-qa alongside Playwright ### Stop paying the selector tax Mature Playwright suites spend more engineering time on upkeep than on new coverage, every redesign breaks locators that encoded yesterday's DOM. agent-qa tests encode intent, so the runtime re-derives the steps when the UI moves on. ### Coverage at the speed of English A new Playwright spec is a programming task. A new agent-qa test is a paragraph. Teams cover the long tail of flows, the ones nobody had time to script, because writing them costs minutes, not hours. ### Your coding agents get a QA loop, not just a browser Playwright MCP lets an agent drive a browser; it doesn't give it a testing discipline. agent-qa gives agents the whole loop: author from product context, validate, run, read artifacts, classify failures, and remember, via MCP tools and packaged Skills. ## Verdict Playwright is a brilliant engine for programmed browser automation. agent-qa is the layer above it, intent in, verified behavior out, memory retained. Most teams need the layer more than another script. ## Frequently asked questions ### Does agent-qa replace Playwright? It replaces the hand-written E2E layer for many teams, and complements Playwright for others. If your pain is authoring and maintaining flow tests, agent-qa's natural-language tests with memory remove most of that work. Teams with deep custom automation keep Playwright for it and let agent-qa own the user-flow regression layer. ### Is agent-qa licensed like Playwright? No. Playwright uses the Apache 2.0 open-source license; agent-qa source is available under FSL-1.1-ALv2 and transitions to Apache 2.0 under its future-license terms. Its tests, config, hooks, memory, and artifacts remain files in your repository. ### What does agent-qa cost compared to Playwright? Playwright is free under Apache 2.0. agent-qa has no paid tier or license fee for FSL-permitted use, but it consumes model calls, reduced by its plan cache, from whichever LLM provider you configure. The operating-cost comparison is therefore engineer-hours, LLM tokens, and infrastructure. ### Can I migrate my Playwright tests to agent-qa? Yes, and it's usually simplification: each spec's intent, 'sign in, add item to cart, verify total', becomes a short plain-English YAML test. You delete selector logic rather than porting it. ### Is natural language reliable enough for CI? agent-qa is built for exactly that: deterministic YAML contracts, cached action plans reused across identical runs, file-backed memory reducing exploration, and artifacts plus failure classification for every step. It behaves like a test harness, not a chatbot. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Playwright public website](https://playwright.dev/) --- # agent-qa vs Cypress URL: https://vostride.com/cypress-alternative Markdown: https://vostride.com/cypress-alternative.md Cypress tests break when the UI changes. agent-qa re-plans, and remembers. Cypress made JavaScript E2E pleasant to write, with a paid cloud for the rest. agent-qa removes the script layer entirely: plain-English tests, self-adapting runs, memory in your repo, no cloud required. Try agent-qa, the source-available way to write tests in English and keep the memory of every run. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Plain-English authoring.** agent-qa: Yes; Cypress: No. Cypress tests are JavaScript/TypeScript. agent-qa tests are natural-language YAML, reviewable by anyone who understands the product, not just the framework. - **Survives UI changes.** agent-qa: Yes; Cypress: No. Cypress selectors fail on markup changes and wait for a human. agent-qa re-plans from intent and records what changed. - **Execution memory.** agent-qa: Yes; Cypress: No. agent-qa builds file-backed product memory across runs; Cypress runs are stateless. - **Source access.** agent-qa: Yes; Cypress: Yes. The Cypress runner is open source (its Cloud is a paid product). agent-qa is source available under FSL-1.1-ALv2, and its runs, artifacts, memory, and dashboard stay repo-local. - **Local and CI execution.** agent-qa: Yes; Cypress: Yes. Both execute locally and in CI. agent-qa adds coding-agent execution via MCP as a first-class path. - **Mobile app testing.** agent-qa: Yes; Cypress: No. Cypress is web-only by design. agent-qa runs native Android and iOS flows with the same YAML contract. - **Coding-agent native.** agent-qa: Yes; Cypress: No. agent-qa ships MCP tools, Skills, and structured artifacts so coding agents can author, run, and triage tests autonomously. - **Parallelization without paid cloud.** agent-qa: Yes; Cypress: Partial. Cypress's turnkey parallelization and analytics live in its paid Cloud. agent-qa's runs, artifacts, and local dashboard need no vendor cloud. ## Why teams move from Cypress to agent-qa ### Flake stops being a lifestyle Cypress flake usually traces to timing waits and selectors encoding a UI that moved. agent-qa executes from intent with adaptive planning, cached known-good plans, and memory of how your product actually behaves, attacking the root causes of flake rather than retrying around them. ### No cloud upsell in your critical path The open-source Cypress runner is real, but parallelization, analytics, and flake detection push you toward the paid Cloud. agent-qa keeps the complete workflow, execution, artifacts, memory, local dashboard, repo-local without an agent-qa platform subscription. ### One harness for web and mobile Cypress stops at the browser. If your product has an app, and it does, agent-qa covers those flows with the same plain-English format, instead of forcing a second framework and a second skill set. ## Verdict Cypress improved the developer experience of writing test scripts. agent-qa removes the scripts. English in, verified flows out, memory retained, on web and the mobile app Cypress can't touch. ## Frequently asked questions ### Is agent-qa a good Cypress alternative? Yes, particularly if your Cypress pain is flake, selector maintenance, or missing mobile coverage. agent-qa replaces scripted specs with plain-English YAML tests that self-adapt to UI changes, remember prior runs, and extend to Android and iOS. ### What does agent-qa cost compared to Cypress? The Cypress runner is free with a paid Cloud for parallelization and analytics. agent-qa has no paid tier and is source available under FSL-1.1-ALv2. You pay only your chosen LLM provider's token costs, moderated by plan caching. ### How do I migrate from Cypress to agent-qa? Rewrite intent, not code: each Cypress spec describes a user journey, and that journey, stated in plain English, is a complete agent-qa test. Most teams port their smoke suite first and retire specs as the equivalent agent-qa tests prove out. ### Does agent-qa have a dashboard like Cypress Cloud? agent-qa ships a local dashboard for runs, tests, suites, hooks, memory, insights, and live execution, running on your machine or CI artifacts, not a hosted subscription. ### Can agent-qa handle component testing like Cypress? No, component testing stays with your unit-level tooling. agent-qa focuses on user-level E2E flows across web and mobile, which is where scripted approaches are most expensive to maintain. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Cypress public website](https://www.cypress.io/) --- # agent-qa vs Selenium URL: https://vostride.com/selenium-alternative Markdown: https://vostride.com/selenium-alternative.md Selenium gave us browser automation. agent-qa adds intent, self-healing, and memory. Selenium is the two-decade veteran of programmatic browser control. agent-qa is what the same job looks like now: plain-English tests, adaptive execution, and a harness built for coding agents. Try agent-qa, the source-available AI QA agent that remembers what a decade of scripts never could. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Plain-English authoring.** agent-qa: Yes; Selenium: No. Selenium tests are code plus locators plus explicit waits. agent-qa tests are natural-language YAML anyone can review. - **Survives UI changes.** agent-qa: Yes; Selenium: No. XPath and CSS locators are Selenium's most famous failure mode. agent-qa re-plans from intent when the UI shifts and remembers the change. - **Execution memory.** agent-qa: Yes; Selenium: No. agent-qa accumulates behavioral memory across runs; Selenium sessions know nothing about the last run. - **Source access.** agent-qa: Yes; Selenium: Yes. Selenium is open source; agent-qa is source available under FSL-1.1-ALv2 and adds an AI harness on top of open foundations. - **Local and CI execution.** agent-qa: Yes; Selenium: Yes. Both run anywhere. agent-qa needs no Grid topology to plan, execute, and report a flow. - **Mobile app testing.** agent-qa: Yes; Selenium: Partial. Selenium's ecosystem reaches mobile through Appium as a separate stack. agent-qa covers web and native mobile in one contract. - **Coding-agent native.** agent-qa: Yes; Selenium: No. agent-qa exposes MCP tools, Skills, artifacts, and failure classification designed for autonomous agents; Selenium predates the idea. - **Wait/timing management.** agent-qa: Yes; Selenium: Partial. Explicit and implicit wait tuning is a Selenium discipline of its own. agent-qa's runtime observes actual app state while executing intent. ## Why teams retire Selenium suites for agent-qa ### Decades of locator debt, gone Legacy Selenium suites are archaeology: brittle XPath, page objects three refactors behind, waits tuned for servers that no longer exist. agent-qa lets you re-express what those tests were for, in English, and delete the debt instead of servicing it. ### The team that can write tests gets bigger Selenium coverage depends on engineers fluent in the framework. agent-qa tests are plain-English YAML, product engineers, QA specialists, and coding agents all author at the same speed, so coverage stops bottlenecking on framework expertise. ### A harness from the agent era Selenium was designed for humans writing scripts against browsers. agent-qa is designed for a world where coding agents change code continuously and need to verify it themselves, MCP tools, Skills, memory, and evidence built in from the start. ## Verdict Selenium earned its place in history. agent-qa is what you'd build today: source available, but with intent instead of locators, memory instead of amnesia, and coding agents as first-class operators. ## Frequently asked questions ### Is agent-qa a good Selenium alternative? Yes. It's the generational upgrade path. agent-qa replaces locator-based scripts with natural-language tests that self-adapt, remember prior runs, and produce reviewable evidence, while staying source available and repo-owned like the Selenium workflow you're leaving. ### What does agent-qa cost compared to Selenium? Selenium is free and open source. agent-qa is source available under FSL-1.1-ALv2 with no paid tier or license fee for FSL-permitted use. Selenium's hidden cost is the engineering time spent maintaining locators, waits, and Grid infrastructure; agent-qa trades that for LLM token spend under your control, with caching to keep repeat runs cheap. ### How do I migrate a large Selenium suite to agent-qa? Don't port code, harvest intent. Group your Selenium tests by user journey, write each journey as one agent-qa YAML test, and run both in parallel until trust is established. Teams typically find hundreds of scripts collapse into dozens of intent-level tests. ### Does agent-qa need Selenium Grid or similar infrastructure? No. agent-qa runs against local browsers, CI runners, and mobile emulators or devices you already have, there's no hub-and-node topology to operate, and no vendor cloud in the loop. ### Can agent-qa test legacy web apps that Selenium currently covers? Yes. agent-qa drives real browsers, so server-rendered and legacy UIs work the same as SPAs, often better than old Selenium scripts, because intent-based execution doesn't depend on the exact markup those scripts froze in time. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Selenium public website](https://www.selenium.dev/) --- # agent-qa vs Appium URL: https://vostride.com/appium-alternative Markdown: https://vostride.com/appium-alternative.md Appium drives your app. agent-qa understands it, and remembers every run. Appium is the standard for programmatic mobile automation, with the setup complexity to match. agent-qa makes mobile E2E a plain-English YAML file that runs on Android and iOS and gets smarter each run. Try agent-qa, the source-available path to mobile E2E that learns your app run over run. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **Plain-English authoring.** agent-qa: Yes; Appium: No. Appium tests are WebDriver code with platform-specific locator strategies. agent-qa tests describe the user journey in natural language. - **Survives UI changes.** agent-qa: Yes; Appium: No. Accessibility-ID and XPath locators break with app updates. agent-qa re-plans from intent and records the app's new behavior in memory. - **Execution memory.** agent-qa: Yes; Appium: No. agent-qa's file-backed memory accumulates app-specific behavior; Appium sessions start cold every time. - **Source access.** agent-qa: Yes; Appium: Yes. Appium is open source; agent-qa is source available under FSL-1.1-ALv2 with no license fee for FSL-permitted use. - **Setup complexity.** agent-qa: Yes; Appium: Partial. Appium setup spans drivers, capabilities, and platform toolchains. agent-qa initializes a workspace with npx agent-qa init and prepares runtimes from there. 'Yes' means minimal-setup. - **Web testing in the same harness.** agent-qa: Yes; Appium: Partial. Appium is mobile-first with some hybrid/web reach. agent-qa covers web and mobile under one YAML contract, one memory store, one CLI. - **Coding-agent native.** agent-qa: Yes; Appium: No. agent-qa ships MCP tools and Skills so coding agents can author and run mobile tests and triage failures autonomously. - **Local and CI execution.** agent-qa: Yes; Appium: Yes. Both run against local devices, emulators, and CI hardware. ## Why mobile teams move from Appium to agent-qa ### Mobile E2E without the ceremony Appium's power comes wrapped in drivers, capabilities, session config, and locator strategies per platform. agent-qa reduces the same outcome, a verified user flow on a real app, to a YAML file and a CLI command. ### One suite for the app and the web app Most products ship both. Appium covers one side; agent-qa covers both with a single authoring format, so your login flow is one test concept, not two codebases. ### Runs that learn your app Mobile UIs change constantly and Appium scripts pay for it release after release. agent-qa's memory records how your app actually behaves, navigation quirks, timing, changed screens, so stability compounds instead of eroding. ## Verdict Appium proved mobile automation could be open. agent-qa makes it effortless, one English sentence per step, both platforms, and a memory of every run. Keep Appium for device plumbing; give the flows to agent-qa. ## Frequently asked questions ### Is agent-qa a good Appium alternative? Yes for user-flow E2E on Android and iOS: agent-qa replaces WebDriver code and locator maintenance with plain-English tests that self-adapt and remember. Appium remains the lower-level choice when you need raw programmatic device control. ### What does agent-qa cost compared to Appium? Appium is free and open source. agent-qa is source available under FSL-1.1-ALv2 with no paid tier or license fee for FSL-permitted use. Appium's cost is the specialist time for setup and script upkeep; agent-qa's is LLM tokens on the provider you choose, cut by plan caching on repeat runs. ### How do I migrate Appium tests to agent-qa? Collapse each Appium test class into the journey it verifies and write that journey as an agent-qa YAML test. Platform-specific locator code has no equivalent to port, the runtime derives actions per platform from the same intent. ### Does agent-qa work with real devices and emulators like Appium? Yes, agent-qa runs mobile tests against local and remote devices and emulators, and the quickstart covers preparing mobile runtimes alongside web. ### Can agent-qa test both my iOS and Android apps with one test? That's the model: one plain-English test expresses the flow, and the runtime executes it per platform, with memory and artifacts tracked per target, no duplicated per-platform script trees. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Appium public website](https://appium.io/) --- # agent-qa vs Maestro URL: https://vostride.com/maestro-alternative Markdown: https://vostride.com/maestro-alternative.md Maestro made mobile flows simple. agent-qa makes them learn. Maestro popularized YAML-declared mobile UI flows with static commands. agent-qa keeps the YAML ergonomics but replaces static commands with AI-executed intent, plus web support and memory. Try agent-qa, the source-available QA agent where mobile flows compound into memory. > This comparison is based on publicly available information. Product capabilities and pricing can change; verify details with each vendor before making a purchase decision. ## Capability comparison - **YAML-based authoring.** agent-qa: Yes; Maestro: Yes. Both use YAML. Maestro's YAML lists UI commands (tapOn, assertVisible); agent-qa's YAML states user intent in plain English that the runtime plans against the live app. - **Survives UI changes.** agent-qa: Yes; Maestro: Partial. Maestro's text/ID matching tolerates small shifts but static commands still break on real redesigns. agent-qa re-plans the whole step from intent. - **Execution memory.** agent-qa: Yes; Maestro: No. agent-qa records behavioral observations across runs; Maestro flows execute the same commands cold each time. - **Source access.** agent-qa: Yes; Maestro: Yes. Maestro is open source and pairs with a paid cloud for scaled execution. agent-qa is source available under FSL-1.1-ALv2, and its full workflow stays repo-local. - **Web testing in the same harness.** agent-qa: Yes; Maestro: Partial. Maestro is mobile-first with some web support. agent-qa treats web and mobile as equal targets under one contract. - **Coding-agent native.** agent-qa: Yes; Maestro: No. agent-qa ships MCP tools, Skills, and structured evidence for autonomous agents; Maestro is operated by humans and CI scripts. - **Bring your own LLM.** agent-qa: Yes; Maestro: No. agent-qa's planning runs on the model provider you configure. Maestro's core doesn't use LLM planning. - **Local and CI execution.** agent-qa: Yes; Maestro: Yes. Both run locally and in CI without mandatory cloud services. ## Why Maestro users graduate to agent-qa ### Intent beats commands Maestro's tapOn and assertVisible commands are simpler than code, but they still encode a fixed UI. agent-qa's steps say what the user is doing, 'add the item to the cart and check out', and the runtime works out the taps on whatever the screen looks like today. ### The same YAML instinct, more leverage If your team already thinks in Maestro YAML, agent-qa feels native on day one, the file just says more with less, covers web too, and every run feeds memory and cache instead of evaporating. ### No cloud dependency for scale Maestro's scaled execution story leads to its paid cloud. agent-qa's runs, artifacts, memory, and dashboard are local and CI-native, scale is your infrastructure decision, not a subscription tier. ## Verdict Maestro proved YAML flows are the right ergonomics for mobile testing. agent-qa keeps the ergonomics and upgrades the engine: intent instead of commands, memory instead of amnesia, and web included. ## Frequently asked questions ### Is agent-qa a good Maestro alternative? Yes. It's the natural next step. You keep YAML-declared flows and gain AI execution that survives redesigns, memory that compounds across runs, first-class web support, and MCP/Skills integration for coding agents. ### What does agent-qa cost compared to Maestro? Maestro is free and open source, and monetizes scaled cloud execution. agent-qa is source available under FSL-1.1-ALv2 with no paid tier or license fee for FSL-permitted use. agent-qa's operating cost is LLM tokens on your chosen provider, moderated by caching. ### How do I migrate Maestro flows to agent-qa? It's the friendliest migration on this site: your Maestro YAML already names each flow's steps, so rewriting command sequences as plain-English intent is nearly mechanical, and the result is shorter than what it replaces. ### Does agent-qa support Android and iOS like Maestro? Yes, native flows on both platforms, plus web, all under the same YAML contract, memory store, and CLI. ### Why does memory matter for mobile testing? Mobile apps change UI constantly and devices add timing variance. agent-qa's memory records how your app actually behaves, screens, quirks, flows, so subsequent runs plan faster and flake less, which static command flows can never do. ## Sources - [agent-qa GitHub repository](https://github.com/vostride/agent-qa) - [Maestro public website](https://www.maestro.dev/) --- # Live demo URL: https://vostride.com/demo/agent-qa Markdown: https://vostride.com/demo/agent-qa.md The interactive run inspector contains recorded, sanitized demonstrations of agent-qa testing real web and mobile products. It exposes plain-English test steps, per-step timing, screenshots, artifacts, and verdicts. ## Available demonstrations - Supabase: Passing suite. https://vostride.com/demo/run/supabase - Linear Android: Passing Android suite. https://vostride.com/demo/run/linear-android - Polymarket: Passing web suite. https://vostride.com/demo/run/polymarket - PostHog: Passing suite. https://vostride.com/demo/run/posthog - Clerk: Passing suite. https://vostride.com/demo/run/clerk - Hacker News: Passing web suite. https://vostride.com/demo/run/hacker-news The demo is evidence of the product interface and output format, not a hosted test-execution service. --- # Pricing URL: https://vostride.com/pricing.md agent-qa has no paid tier, per-seat charge, or per-test platform fee. Product price: $0. Users control the cost of their chosen LLM provider, compute, browsers, mobile devices, CI, and storage. The validated action cache reduces repeat planning work and token use when test context is unchanged. agent-qa source is available under FSL-1.1-ALv2, with an Apache 2.0 future license. Full terms: https://vostride.com/license.md. --- # Company ## Vostride manifesto URL: https://vostride.com/manifesto Markdown: https://vostride.com/manifesto.md Opus 4.5 was an inflection point. Production-grade code could be generated at lightning speed, making software cheaper to produce, while the confidence that it would work in production remained questionable. Humans still need to verify the changes. At Vostride, our goal is to empower teams to ~~ship code~~ **ship quality confidently**. We envision an *agentic future* where humans are somewhere in the loop, owning the responsibilities of the agents they work with. Software teams are starting to look like factories, and we are focused on the verification layer. We are building ~~tools~~ **harnesses** to help **agents** and **humans** ship confidently. Vostride, founding team ## Contact Vostride URL: https://vostride.com/contact Markdown: https://vostride.com/contact.md Have a general question, need help with an integration, or want to talk through an agent-qa workflow? Email hello@vostride.com. ## Changelog URL: https://vostride.com/changelog Markdown: https://vostride.com/changelog.md Package versions and publication dates: https://www.npmjs.com/package/agent-qa?activeTab=versions Source changes and releases: https://github.com/vostride/agent-qa Agents should verify the current npm version before suggesting an install command. --- # License URL: https://vostride.com/license Markdown: https://vostride.com/license.md ## Abbreviation FSL-1.1-ALv2 ## Notice Copyright 2026 Pranshu Chittora ## Terms and Conditions ### Licensor ("We") The party offering the Software under these Terms and Conditions. ### The Software The "Software" is each version of the software that we make available under these Terms and Conditions, as indicated by our inclusion of these Terms and Conditions with the Software. ### License Grant Subject to your compliance with this License Grant and the Patents, Redistribution and Trademark clauses below, we hereby grant you the right to use, copy, modify, create derivative works, publicly perform, publicly display and redistribute the Software for any Permitted Purpose identified below. ### Permitted Purpose A Permitted Purpose is any purpose other than a Competing Use. A Competing Use means making the Software available to others in a commercial product or service that: 1. substitutes for the Software; 2. substitutes for any other product or service we offer using the Software that exists as of the date we make the Software available; or 3. offers the same or substantially similar functionality as the Software. Permitted Purposes specifically include using the Software: 1. for your internal use and access; 2. for non-commercial education; 3. for non-commercial research; and 4. in connection with professional services that you provide to a licensee using the Software in accordance with these Terms and Conditions. ### Patents To the extent your use for a Permitted Purpose would necessarily infringe our patents, the license grant above includes a license under our patents. If you make a claim against any party that the Software infringes or contributes to the infringement of any patent, then your patent license to the Software ends immediately. ### Redistribution The Terms and Conditions apply to all copies, modifications and derivatives of the Software. If you redistribute any copies, modifications or derivatives of the Software, you must include a copy of or a link to these Terms and Conditions and not remove any copyright notices provided in or with the Software. ### Disclaimer THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT. IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES, EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE. ### Trademarks Except for displaying the License Details and identifying us as the origin of the Software, you have no right under these Terms and Conditions to use our trademarks, trade names, service marks or product names. ## Grant of Future License We hereby irrevocably grant you an additional license to use the Software under the Apache License, Version 2.0 that is effective on the second anniversary of the date we make the Software available. On or after that date, you may use the Software under the Apache License, Version 2.0, in which case the following will apply: Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.