Run in CI

agent-qa run works the same in CI as in your terminal. With no terminal attached it prints no colors, spinners or prompts, and it exits with the run's verdict.

Add an agent-qa job to this repository's CI. Read https://vostride.com/docs/agent-qa/guides/ci.md first and follow it.

1. Find the CI system this repository uses, and the agent-qa suite or test files that should run on every change. Every test in one run must belong to the same product, so name a suite or explicit paths.
2. Install dependencies, then Chromium with npx agent-qa install-browsers --chromium --with-deps.
3. Create .env and the secrets file inside the job from CI secrets. They are not committed.
4. Save the model API key with npx agent-qa auth set --config <entry> --type api-key "$MODEL_API_KEY", where MODEL_API_KEY comes from a CI secret. Subscription sign-in does not work in CI.
5. Run the tests with --junit-output junit.xml, and always upload .agent-qa/runs/ and junit.xml, even when the job fails. Never upload the rest of .agent-qa/.

List the CI secrets I need to create, and never write secret values into the repository.

A GitHub Actions jobCopy Anchor Link

name: agent-qa
on: [push, pull_request]
jobs:
  agent-qa:
    timeout-minutes: 30
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
      - run: npm ci
      - run: npx agent-qa install-browsers --chromium --with-deps
      - name: Create the env files
        run: |
          touch .env
          printf 'TEST_PASSWORD=%s\n' "$TEST_PASSWORD" > .env.secrets.local
        env:
          TEST_PASSWORD: ${{ secrets.TEST_PASSWORD }}
      - name: Save the model key
        run: npx agent-qa auth set --config default --type api-key "$MODEL_API_KEY"
        env:
          MODEL_API_KEY: ${{ secrets.MODEL_API_KEY }}
      - run: npx agent-qa run suites/smoke.suite.yaml --junit-output junit.xml
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: agent-qa-evidence
          path: |
            .agent-qa/runs/
            junit.xml

The steps that matter:

• The env files must exist. init keeps .env and .env.secrets.local out of git, but every run needs both, even empty. Create them in the job, with secrets from your CI's secret store.
• The model key comes from the credential store. agent-qa does not read API keys from environment variables or the config. auth set with the key as its last argument saves it without a prompt, for the model entry your config uses.
• Name what to run. Run a suite or a set of test files. A bare agent-qa run runs every test in the project, and every test in one run must belong to the same product.
• Use an API key in CI. Subscription sign-in needs a person in a browser.

Exit codesCopy Anchor Link

--junit-output <path> writes a JUnit report that most CI systems display natively.

What to uploadCopy Anchor Link

Upload two things, even when the job fails:

• each run's folder under .agent-qa/runs/, with its events, evidence and screenshots;
• the JUnit report.

Never upload the rest of .agent-qa/. Its runs.db holds config snapshots that include your .env values, and it also holds saved logins. Never upload .env, the secrets file, or the lock files and queue in agent-qa-knowledge/.

Learning in CICopy Anchor Link

agent-qa detects CI from the CI variable and the usual vendor variables, and changes two things:

• It only proposes. What a CI run learns is recorded as candidates and never promoted, so a CI job cannot change what your team's runs trust. The curation line ends with (ci runs never promote learned knowledge).
• It writes the action cache on any branch, including your default branch.

After the verdict, agent-qa spends up to 60 seconds on analysis and curation. That never changes the exit code. Work still pending when the job ends is lost with the workspace, because the queue lives in local files. --detach skips that minute.

agent-qa never commits. Knowledge a CI run proposes stays in that workspace unless your job commits agent-qa-knowledge/ itself.

Faster repeat runsCopy Anchor Link

Persist .agent-qa/cache/ between jobs with your CI's cache step to replay actions across runs. Cached entries hold no secrets. See Caching.

Docker imagesCopy Anchor Link

vostride/agent-qa-web has agent-qa and every browser installed, and runs agent-qa as its entrypoint from /tests. vostride/agent-qa-android adds the Android SDK and Appium for emulator runs.