How knowledge reaches a run
The brief each test gets, how it is ranked and budgeted, and the other ways memidex helps a step.
memidex reaches a running test in four ways: a brief built once per test, memory tools the model can call during a step, cautions shown after a failure, and procedures that can replay a known flow.
The brief
Before the first step of each test, memidex builds a brief and freezes it for that test. Every step's evidence records exactly what it was given, so you can always find out why the agent knew something.
The brief has three tiers:
| Tier | Holds | Trimmed to fit |
|---|---|---|
| First | Your policies, word for word, and facts that came from you. | Never |
| Second | Cautions: known ways this kind of test goes wrong. | Yes |
| Third | Facts learned from earlier runs. | Yes |
Each learned line reads like Observation: button "place order · $36.00" resolved as a verified element, followed by notes in brackets when they apply. The model is told the brief holds hypotheses to weigh against the live screen, not instructions.
| Note | Meaning |
|---|---|
UNVERIFIED, CANARY | A candidate, not yet promoted. See below. |
unconfirmed since build N | No run has confirmed it lately, so its trust is reduced. |
CONFLICT | Another record disagrees with it. |
What gets in, and in what order
The brief takes active facts and cautions that match the test's scope and are valid at this run's count. Ranking is plain code, so the same knowledge always produces the same brief:
- Cautions first, then facts.
- Within each, higher trust first, boosted when the record's anchors share words with the product or test.
- At most two records per anchor, so one busy page cannot fill the brief.
Candidates can appear early. Up to two candidates per brief, each confirmed at least once, are served with the UNVERIFIED note and half their trust. They take at most a fifth of the brief. This is how a new lesson collects evidence from real runs before it is promoted.
Budget
The brief's size follows the model. agent-qa takes the model's context window, sets aside room for its answer, the rules and the test itself, and gives the brief about 5% of the rest, between 256 and 2,048 tokens. Cautions get about 1%, and memory reads about 10%. Policies are never counted against the budget, so keep them short.
Memory tools
Models that can call tools, accept images and have a context window of at least 16,384 tokens also get two tools during a step: memory.search to look for records, and memory.read to open one. The model reaches for them when the brief does not cover what it sees.
Cautions after a failure
When an action fails, memidex looks for cautions tied to the same element, page or error, and shows them on the next planning turn. A known flaky dialog or a slow page then reaches the model at the moment it matters.
Procedures
A procedure is a flow that passed end to end, stored as actions with typed parameters. During a step, memidex matches procedures against the current page, the test and the step's intent.
A procedure replays without asking the model only when all of these hold:
- A capable model wrote it and later runs confirmed it at least twice. Procedures written by other models need four confirmations.
- Every parameter it needs can be filled from your variables, secrets or memory.
- The match is confident.
Each replayed action is checked against the screen first. Any mismatch hands control back to normal planning. A procedure that does not qualify is still offered to the model as a hint.
The index comes first
Briefs, memory tools and procedures all read the search index. When it is missing, a run gets your policies and nothing learned. A local run rebuilds the index after it finishes, and npx agent-qa knowledge rebuild-index rebuilds it on demand.
See what a run was given
| Question | Command |
|---|---|
| What did this run use and learn? | npx agent-qa runs knowledge <runId> |
| Why did a step's brief say what it said? | npx agent-qa knowledge brief-why <runId> --step <n> |
Both read what the run recorded at the time and never search again. Step numbers start at 0. In the dashboard, the Knowledge tab of a run shows the same thing per step: what was used, its effect on the run and what changed in memory afterwards.
When agent-qa writes a test
When agent-qa writes a test for you, through agent_qa_write_test, agent-qa ask or chat, it gets a brief of its own: cautions, procedures, up to four of your existing tests as examples, and facts about the product. Examples are chosen by how well those tests have run.