Building AI Research Agents
I built a suite of agents that did research for users. I have written separately about how we evaluated them. This is the other half, in the order it got built.
The loop
We initially had a simple loop. A model reasoning over its context and its tools, one turn at a time. The model reads what is in front of it, decides to call a tool, the result comes back into the context, and it goes again.
Everything we added after that was a change to what sits in that context and for how long.
Once the budget gets tight, the middle of the history is replaced by a structured summary. This is the lossy one, which is why it is last.
What goes into the context
We got good at injecting context, and most of that was the user's workspace.
What goes in is markers, not content. The last few files they viewed, the file they currently have open, the folder they are in, where their cursor is, and any selection state.
A marker is a few tokens and a file is thousands, and the agent has read_file - if it wants the content it can ask. Injecting the content up front is deciding on the agent's behalf which of those files matters, which is the decision we were trying to let it make.
Tools
We spent a significant amount of time on the tools. The set we ended up with:
| Tools | For |
|---|---|
| Web search, web fetch | Reading what is outside the workspace |
| Read, create, delete, rename, move file, and the plural forms where they made sense | Acting on files |
| Find file by term, find file by name, filter files by condition | Locating a file by something you already know about it |
| Semantic search over files | Locating a file by what it is about |
The last two rows are where the trouble was. Four genuinely different things, and a model that is not certain which one it wants will try two of them and waste a turn. We went through the names and the descriptions and made sure no two tools had overlapping scope, and that each description said what the tool was for and what it was not for.
That was the realisation that improved tool quality most. We had been treating them as an API surface and changing them casually, and once we started treating them as prompt they went through the same review as the rest of it.
Error messages got the same treatment. The string a failing tool returns is read by a model that has to decide what to do next, so it has to say what went wrong and what to do instead.
Task agents
After that we added sub agent parallelisation to speed up research. A lot of research work required independent explorations in multiple directions. And doing that sequentially would have been too slow. The base agent splits the question, each task agent works one direction, and each returns a summary. Around three task agents in a typical run.
Three explorations at once, so the request is bounded by the slowest branch at 5.5 min rather than their sum.
Durations are illustrative, obfuscated from the real ones.
A task agent's own history never reaches the base agent. Only the summary comes back, and that summary is a tool result like any other, which means it is subject to elision along with everything else.
Elision
Eval tests showed context being used up quickly, so we added elision and wrote tool results to external storage.
| Rule | Why |
|---|---|
| Elide a result older than 3 steps and larger than 2k tokens | Both conditions have to hold - a large result just received is probably the thing being worked on, and a small one sits in the history for nothing |
| Never elide a result cited in the last few turns | It is actively in use |
| Always keep the most recent result from each distinct tool | The model picks up a tool's output format from having seen it, and once every example is gone the next call is made from the description alone |
What the agent sees in place of the result:
[r_7f2a | search_web "Alcaraz 2026 clay season" | 12 results, 14k tokens
top sources: atptour.com, espn.com, tennis.com
covers: match results, ranking movement, one injury mention
fetch_result('r_7f2a')]
An id on its own would be useless, because the agent cannot tell what it is choosing to skip. The stub carries the source domains, what the result covers and how big it is, which is enough to decide whether fetching it back is worth a call.
Compaction
Elision buys a lot and a long enough run still fills up. Compaction is the last thing we reach for, because it is the only one of these that loses anything.
We trigger at 75% of budget and aim to come back down to about 30%. Not on every turn. Compaction is a separate model call, and the user sees "Compacting your messages before continuing" with a loading state, so every one of them is a pause in front of someone who is waiting.
Waiting until 75% costs 4 compaction calls over the same run. The 30% it cuts back to is not a dial - it is the pinned region plus the 20% verbatim tail plus the summary.
Figures illustrative, obfuscated from the real ones.
The 30% is a loose target rather than a dial. It is what the pinned region plus the verbatim tail plus the summary comes to. If the pinned material and the tail already add up to more than that, then they do.
Compaction is structured. We replace the middle of the history with structured output and keep the last N turns verbatim, where those N turns come to 20% of budget or less.
PINNED (never compacted, verbatim)
original task and user constraints
current plan / todo state
manifest of exact identifiers encountered
COMPACTED (structured, replaces the middle)
everything older than the recency window
VERBATIM TAIL
the last N turns, where N turns are 20% of budget or less
Pin by role, not by position. The task gets pinned because it is the task, not because it was first.
The middle compacts down to this:
{
"established_facts": [{ "claim": "...", "handle": "r_7f2a" }],
"exact_identifiers": ["EU 2024/1689 Art. 6", "FY25 Q3 revenue $1.42B"],
"dead_ends": [{ "approach": "...", "why_failed": "..." }],
"open_questions": ["..."],
"current_state": "...",
"next_action": "..."
}
One more rule we adhered to.
The crossing lands between the turn that works the problem out and the turn that acts on it. Turn 22 runs with the finding available as a line in the summary rather than as the reasoning that produced it.
What I would keep
The ordering. Elision is cheap and it loses nothing, so it runs continuously. Compaction loses something, so it waits behind a threshold and defers when the timing is bad.
And most of the work was not in the loop. It was in the tools, and in deciding what to put in front of the model.