Skip to main content
agent-qa on GitHub,904stars
agent-qa

Safety and privacy

Copy page
View as MarkdownOpen the raw Markdown source.Open in ChatGPTAsk questions about this page.Open in ClaudeAsk questions about this page.Open in MistralAsk questions about this page.Open in VS CodeAsk Copilot Chat about this page.Open in CursorAsk Cursor about this page.

What agent-qa masks, what it refuses to do, what stays on your machine, and what usage data it sends.

agent-qa drives your real app with a model in the loop, so it is worth knowing exactly what it protects and what it leaves to you.

Secrets

Values in your secrets file, workspace.secretsFile (.env.secrets.local by default), are masked as [secret:<name>] everywhere text leaves agent-qa: model requests, knowledge records, logs, the terminal, the dashboard, MCP results, JUnit reports and analytics. A masked value is never restored.

  • Type a secret in a step with {{secret:NAME}}. The value is filled in at the moment of the action.
  • An action that contains the literal text [secret:NAME] is refused, because it would type the placeholder.
  • Steps that type a secret are never cached.

Nothing else is masked. Values in .env, other environment variables, cookies and saved logins, email addresses and other personal data, screenshots and session videos are kept as they are. Use temporary test accounts with limited access, and put anything sensitive in the secrets file.

Production targets

Mark a target that points at production:

registry:
  targets:
    checkout-prod:
      platform: web
      url: https://shop.example.com
      environment: production

A run that touches it is refused before anything happens, with exit code 2, unless the config also sets allowProduction: true at the top level. No flag or MCP argument can override it. environment can also be local or shared; a target without one counts as non-production.

No confirmations

v0.2 has no approval prompts. Every command, MCP tool and chat request applies in the call: runs start, tests are written, knowledge changes. Give an agent the agent-qa MCP server the way you would give it a shell in your project.

Local only

The dashboard, its run service and agent-qa mcp --http listen on 127.0.0.1 only. The dashboard refuses to start on any other address, and requests from other sites or hosts are rejected. None of them uses tokens, so reach them from another machine only through a tunnel or proxy that authenticates for you.

Hooks run in a sandbox

Hooks run in Docker containers with 512 MB of memory, one CPU and a read-only file system. They have no network unless the hook sets network: true. A hook receives the run's variables and every declared secret as environment variables, so review hook scripts like any code that handles credentials.

App content is data

Text from your app reaches the model as observations, wrapped as untrusted data. Learned knowledge is served as hints to weigh against the screen, never as instructions. agent-qa does not try to detect prompt injection, so test hostile content in an isolated environment.

Plugins

Provider plugins run inside agent-qa with full access to your machine, including credentials and screenshots. Use plugins you trust; agent-qa prints a warning when it loads one.

Usage analytics

Released versions of agent-qa send anonymous usage events to PostHog, at us.i.posthog.com, from run, mcp, chat, the daemon and init. The dashboard, ask, fn, runs, digest and entities send nothing.

Events hold only fixed values, numbers and IDs: the command, the platform, the verdict, durations, step and token counts, the agent-qa version, and which coding agent ran it. They never contain step text, page content or file paths, and they pass through the same secret masking. Your anonymous ID lives in ~/.agent-qa/analytics.json; CI runs use a fresh ID each time. PostHog may estimate a coarse location from your IP address.

To opt out, add this to agent-qa.config.yaml:

analytics:
  privacy: true

Update check

At most once a day, agent-qa run and the dashboard ask the npm registry whether a newer agent-qa exists. --quiet silences it.

What to keep out of git and CI uploads

  • .agent-qa/, except run folders you upload on purpose. Its runs.db holds config snapshots that include your .env values.
  • Saved logins in .agent-qa/auth-states/.
  • .env, the secrets file and agent-qa.local.yaml.
  • The knowledge folder's lock files and queue, which its own .gitignore already excludes.

Caching

How agent-qa replays the action that completed a step, when it trusts a replay, and how to control the cache.

memidex

agent-qa's memory. It keeps what runs prove about your app as small records, checks them against later runs, and stores them as files you can review.

On this page

SecretsProduction targetsNo confirmationsLocal onlyHooks run in a sandboxApp content is dataPluginsUsage analyticsUpdate checkWhat to keep out of git and CI uploads