auth.json

auth.json stores saved LLM credentials for named configs in registry.llms.

By default, agent-qa writes it outside the project:

~/.agent-qa/auth.json

If XDG_DATA_HOME is set, the path changes to:

$XDG_DATA_HOME/agent-qa/auth.json

The file is written with mode 0600 when agent-qa saves credentials.

Credential keys

Entries are keyed by the LLM config name, not by the provider name.

registry:
  llms:
    - name: codex
      provider: openai-subscription
      model: gpt-5.5
    - name: remote-openai
      provider: openai-compatible
      model: gpt-5.1
      baseURL: https://api.openai.com/v1
use:
  llm: codex

The matching auth.json keys are codex and remote-openai.

File shape

{
  "remote-openai": {
    "type": "api",
    "provider": "openai-compatible",
    "key": "sk-project-restricted-token"
  },
  "remote-anthropic": {
    "type": "bearer",
    "provider": "anthropic-compatible",
    "token": "restricted-bearer-token"
  },
  "codex": {
    "type": "oauth",
    "provider": "openai-subscription",
    "tokens": {
      "access": "access-token",
      "refresh": "refresh-token",
      "expires": 1799999999999,
      "accountId": "acct_123"
    }
  }
}

Description: JSON object keyed by LLM config name.

Possible values: api, bearer, and oauth credential objects.

Required: only for LLM configs that need saved credentials.

Default: no saved credentials.

type api

{
  "remote-openai": {
    "type": "api",
    "provider": "openai-compatible",
    "key": "sk-project-restricted-token"
  }
}

Description: API key credential.

Provider constraints: supported for openai-compatible, anthropic-compatible, and gemini.

Required fields: type, provider, and key.

Default: none.

type bearer

{
  "remote-anthropic": {
    "type": "bearer",
    "provider": "anthropic-compatible",
    "token": "restricted-bearer-token"
  }
}

Description: Bearer token credential.

Provider constraints: supported for anthropic-compatible.

Required fields: type, provider, and token.

Default: none.

type oauth

{
  "codex": {
    "type": "oauth",
    "provider": "openai-subscription",
    "tokens": {
      "access": "access-token",
      "refresh": "refresh-token",
      "expires": 1799999999999,
      "accountId": "acct_123"
    }
  }
}

Description: OAuth tokens from an auth plugin.

Provider constraints: used by subscription auth plugins such as openai-subscription and anthropic-subscription.

Required fields: type, provider, and tokens.

Default: none.

tokens.access

{
  "tokens": {
    "access": "access-token"
  }
}

Description: Access token used by the plugin-backed fetch wrapper.

Required: yes for OAuth credentials.

Default: none.

tokens.refresh

{
  "tokens": {
    "refresh": "refresh-token"
  }
}

Description: Refresh token used when the plugin refreshes credentials.

Required: yes for OAuth credentials.

Default: none.

tokens.expires

{
  "tokens": {
    "expires": 1799999999999
  }
}

Description: Expiration timestamp in milliseconds.

Required: yes for OAuth credentials.

Default: none.

tokens.accountId

{
  "tokens": {
    "accountId": "acct_123"
  }
}

Description: Optional account identifier.

Required: no.

Default: not set.

Managing auth locally

Use the CLI or dashboard rather than editing auth.json by hand when you are working locally.

>npx agent-qa auth set --config remote-openai --type api-key

For Anthropic-compatible bearer tokens:

>npx agent-qa auth set --config remote-anthropic --type bearer-token

For subscription providers, declare the auth plugin in agent-qa.config.yaml, install it, then authenticate from the dashboard or use the plugin-backed login flow when supported.

CI runtime auth

For CI, create auth.json at runtime from CI secrets. Do not commit it.

mkdir -p "$HOME/.agent-qa"
chmod 700 "$HOME/.agent-qa"

node -e '
const { writeFileSync } = require("node:fs");
const path = `${process.env.HOME}/.agent-qa/auth.json`;
const auth = {
  "remote-openai": {
    type: "api",
    provider: "openai-compatible",
    key: process.env.AGENT_QA_OPENAI_API_KEY
  }
};
writeFileSync(path, JSON.stringify(auth, null, 2), { mode: 0o600 });
'

npx agent-qa run --suite suites/web-release.suite.yaml

When CI uses an alternate data directory, set XDG_DATA_HOME and write to $XDG_DATA_HOME/agent-qa/auth.json.

Use CI secret values with the same discipline as test secrets: restricted scopes, disposable credentials when possible, and rotation-ready setup.