Models and credentials

agent-qa uses one model entry for a run. It plans and checks every step, then writes up analysis and knowledge after the verdict. Model entries live in registry.llms, and use.llm names the default.

registry:
  llms:
    - name: claude-code
      provider: anthropic-subscription
      model: claude-sonnet-5-5
use:
  llm: claude-code

Providers

Provider	Connects to	Signs in with	`init --llm`
`anthropic-subscription`	A Claude subscription	Browser sign-in	`claude-code`
`openai-subscription`	A Codex subscription	Browser sign-in	`codex`
`openai-compatible`	Any OpenAI Chat Completions endpoint: OpenAI, OpenRouter, a local server	API key, or none for a local server	`openai-compatible`
`anthropic-compatible`	Any Anthropic Messages endpoint	API key or bearer token, or none for a local server	`anthropic-compatible`
`gemini`	Google Gemini	API key	`gemini`

Provider plugins can add more. See Plugins.

Entry fields

Field	Required	Meaning
`name`	Yes	The entry's name. Lowercase letters, digits, dots and hyphens. Credentials are stored under it.
`provider`	Yes	One of the providers above, or a plugin's provider ID.
`model`	Yes	The model ID your provider expects.
`baseURL`	For the two compatible providers	The endpoint, such as `https://api.openai.com/v1`.
`providerHeaders`	No	Extra headers for `anthropic-compatible` only. Headers that carry credentials are refused.
`vision`	No	`true` by default. `false` runs text-only: no screenshots are sent.
`contextWindow`	No	The model's context window, such as `200k`. agent-qa sizes the knowledge brief from it.

Examples

registry:
  llms:
    # A Codex subscription
    - name: codex
      provider: openai-subscription
      model: gpt-6.1-sol

    # OpenAI's API
    - name: openai
      provider: openai-compatible
      model: gpt-5.5
      baseURL: https://api.openai.com/v1

    # OpenRouter
    - name: openrouter
      provider: openai-compatible
      model: anthropic/claude-sonnet-4.6
      baseURL: https://openrouter.ai/api/v1

    # A local server, such as LM Studio or Ollama, with no key
    - name: local
      provider: openai-compatible
      model: qwen3-vl-8b
      baseURL: http://127.0.0.1:1234/v1

    # Gemini
    - name: gemini
      provider: gemini
      model: gemini-3-flash-preview

A test or suite can use another entry with its own use.llm. In chat, /model <name> switches for the conversation.

Choosing a model

Use a model that accepts images. agent-qa sends a screenshot with each decision, which lets it check what is actually on the screen. services.vision limits how large those screenshots are.

A text-only model works with vision: false. agent-qa then reads the page structure and accessibility tree only, so it cannot check purely visual details such as colors or layout, and runs record no screenshots.

Credentials

Credentials are stored outside your project, in ~/.agent-qa/auth.json, or in $XDG_DATA_HOME/agent-qa/auth.json when that is set. The file is readable by you only, and each credential is stored under its entry's name. agent-qa does not read API keys from environment variables or from the config.

Command	Use it to
`agent-qa auth set --config <name> --type api-key`	Save an API key. It asks at a hidden prompt.
`agent-qa auth set --config <name> --type bearer-token`	Save a bearer token for an Anthropic-compatible endpoint.
`agent-qa auth login --config <name>`	Sign in a subscription entry.
`agent-qa auth status`	See which entries have credentials.
`agent-qa auth test --config <name>`	Send one short request and confirm the connection.
`agent-qa auth logout --config <name>`	Remove a credential.

For CI, pass the key as the last argument from a secret: agent-qa auth set --config default --type api-key "$MODEL_API_KEY". See Run in CI.

Subscriptions

Claude and Codex subscriptions need the sign-in plugin, installed in the project and declared in the config:

npm install -D @vostride/agent-qa-subscription-auth

plugins:
  auth:
    - package: "@vostride/agent-qa-subscription-auth"

init --llm claude-code or --llm codex writes the declaration and adds the plugin to package.json. Then sign in:

npx agent-qa auth login --config claude-code

agent-qa prints a sign-in URL and opens your browser. For Claude, paste the code the page shows back into the terminal. For Codex, the browser hands the sign-in back to agent-qa on your machine. A person has to complete either one.

Local models

The two compatible providers work with local servers. When baseURL points anywhere other than api.openai.com or api.anthropic.com, agent-qa runs without a saved key, and auth test reports Testing without a saved credential. Save a key with auth set if your server needs one.

When something is missing

agent-qa prints	Fix
`Sign in to "<provider>" for the "<name>" config`	`npx agent-qa auth login --config <name>`
`Save an API key for the "<name>" config`	`npx agent-qa auth set --config <name> --type api-key`
`needs the subscription-auth plugin`	Install and declare `@vostride/agent-qa-subscription-auth`.
`Provider "<id>" has been removed`	Use `openai-compatible` or `anthropic-compatible` with a `baseURL`. The old `anthropic`, `openai`, `google`, `ollama`, `lmstudio` and `custom` providers are gone.
`use.llm "<name>" does not match any name in registry.llms`	Fix the name in `use.llm`.

Models and credentials

ProvidersCopy Anchor Link

Entry fieldsCopy Anchor Link

ExamplesCopy Anchor Link

Choosing a modelCopy Anchor Link

CredentialsCopy Anchor Link

SubscriptionsCopy Anchor Link

Local modelsCopy Anchor Link

When something is missingCopy Anchor Link

On this page