Mistral Agentic Search: 3x on FinanceBench vs RAG

Agentic retrieval on the Mistral Search Toolkit: the five tools

Agentic search means that the model retrieves in several steps instead of once. Mistral's version is a retrieval layer, not a model.

It sits on an index built with the Mistral Search Toolkit, in the cloud or on-premises, and it gives the model five tools that work like file-system operations. The benchmarks below use long, dense material: financial filings and government Treasury Bulletins.

The five Agentic Search tools

Tool	What it does
search	Finds relevant documents across the corpus using the existing index
open	Opens a specific document
navigate	Moves to a page, section, or region within it
read	Retrieves the content at that location
grep	Finds a pattern within an open document

The work is split in two. The index picks the documents that are likely to hold the answer, and the model picks what to inspect inside them and across them. In one-shot RAG, the retrieval step fixes a set of chunks before the model answers.

No fine-tuning: why chunking stops capping RAG accuracy

The five tools need no fine-tuning and no model-specific training. In a one-shot setup, the chunking strategy limits what the model can see. Here the model can open the document and read around a chunk, so retrieval quality follows model capability instead.

The same performance patterns showed up on Mistral's own Medium 3.5 and on GLM-5.2 from Z.ai, which Mistral reads as a sign that Agentic Search is model-agnostic. Because Agentic Search builds on the index, a team that has already tuned its RAG chunking adds the loop on top of that work.

Agentic search vs RAG: where one-shot RAG breaks, agentic RAG recovers

One-shot RAG retrieves a fixed set of chunks and asks the model to answer in a single pass. That works when the answer is in a top result.

It falters on long reports, on references that point elsewhere, on comparisons across documents and on checking the evidence. Mistral gives three reasons.

• Retrieval without reasoning. The model has to answer from the first chunks, even when they are incomplete. It cannot decide that it needs another document or section.
• Chunk-level limit. Take a question about a company's tax rate for one quarter. The index can return the right filing, yet it has no way to open it, jump to the table and read the lines around the number.
• No iteration. Many questions need refining the search, inspecting a document, following a reference, comparing sources and trying a new route. One-shot RAG has no step for any of these.

In the RAG vs agentic RAG comparison, the difference is a design choice. One design gives a single retrieval pass. The other gives a loop of tool calls.

Agentic retrieval worked example: OfficeQA Pro and 1953 spending

The loop finds spending for all of 1953 in three tool calls, while one-shot retrieval stops at January to June. The question asks for the total U.S. national defense expenditures across all twelve calendar months of 1953. The Treasury Bulletins in the OfficeQA Pro corpus are organised by fiscal year, not by calendar year.

One-shot retrieval runs one search. It gets ten monthly bulletins framed by fiscal year, finds January to June, and stops without the other six months.

The agentic loop makes three tool calls. The first search returns the same partial year.

A second, reformulated search brings up treasury_bulletin_1954_02.pdf, page 15. Then read pulls Table 3, which lists all twelve months of 1953, and the sum is 44,463. The second search is the step that one-shot RAG cannot take, because it depends on the model seeing that the first results were incomplete.

Agentic search benchmark results: FinanceBench and OfficeQA Pro

In Mistral's own runs, correctness rises from 26.7% to 86% on FinanceBench and from 6.3% to 51.9% on OfficeQA Pro. That is about 3x on FinanceBench and a gain of 45.6 points on OfficeQA Pro. Mistral used the out-of-the-box stack with no tuning, and it calls the results floors, not ceilings.

Correctness, one-shot RAG vs the full loop (Mistral's figures)

The full loop reaches 86% on FinanceBench against 26.7% for one-shot RAG, and 51.9% on OfficeQA Pro against 6.3%. OfficeQA Pro stays near half correct even with the loop.

26.7%FinanceBench, one-shot RAG

86%FinanceBench, full loop

6.3%OfficeQA Pro, one-shot RAG

51.9%OfficeQA Pro, full loop

0255075%

The two benchmarks

	FinanceBench	OfficeQA Pro
Corpus	368 SEC filings (10-K / 10-Q / 8-K)	696 U.S. Treasury Bulletins
Scale	~147 pages each, ~53,900 total	~89,000 pages, scanned
Questions	150	133 (the "pro" subset, first pass)
Scoring	LLM judge calibrated against human labels	Verifiable numeric answers
One-shot RAG to full loop	26.7% to 86%	6.3% to 51.9%

FinanceBench: agentic RAG loop adds ~50 points, navigation ~7

Letting the model search repeatedly adds 47.3 and 52.6 points, and the navigation tools add 8.7 and 6.7 more. The ablation separates the loop from the navigation tools.

Moving from one-shot RAG to a loop that only calls search adds 47.3 points for Mistral Medium 3.5 and 52.6 points for GLM-5.2. Adding open, navigate, read and grep adds 8.7 and 6.7 points more.

So the loop, meaning the freedom to search more than once, carries most of the gain. A team that wants to test the idea can start there, with repeated search on its current index.

FinanceBench ablation (Mistral's figures)

Step	Mistral Medium 3.5	GLM-5.2
One-shot RAG to search-only loop	+47.3 points	+52.6 points
Search-only loop to full loop with navigation	+8.7 points	+6.7 points
Token usage, full loop vs search-only	-23.9%	-33.7%

Agentic search optimization: 39.6% lower p90 latency on FinanceBench

Navigation cuts p90 latency on FinanceBench by 39.6%, from 255s to 154s, and it uses fewer tokens. The full loop answers more questions correctly than the search-only loop and uses fewer tokens: 23.9% fewer for Mistral Medium 3.5 and 33.7% fewer for GLM-5.2.

Mean latency across FinanceBench drops from 108s to 71s. Mistral's explanation is that navigation replaces search retries that went nowhere with a direct move to the evidence.

OfficeQA Pro: why the harder benchmark tests agentic retrieval more

The full loop reaches 51.9% on OfficeQA Pro against 6.3% for one-shot RAG, a lower score than the 86% on FinanceBench. It asks for numeric answers that can be checked, over scanned, table-heavy Treasury Bulletins: 696 documents and about 89,000 pages. The first pass covers 133 questions.

A score of 51.9% means that about half of the questions are still missed, so this benchmark does not show that document question answering is solved. It does show how far a single retrieval pass falls behind on scanned documents that need deep table lookups.

When agentic RAG beats plain RAG accuracy, and when an index is enough

Use agentic retrieval when the answer sits inside a long document or across several, and keep the plain index for lookups. Take a support team whose questions are one-line lookups in short, clean help pages.

The answer sits in one of the first chunks, so extra retrieval steps are unlikely to help, and each extra turn adds tokens and time. That team keeps the index.

Take an analyst who must confirm a figure that sits in one table or footnote of a long filing. There the first chunks may miss it, and the loop can navigate to the page.

Mistral states the split in two lists, and the second names the cases where an index is the right starting point. The "Mistral's own guidance on which to use" table below sets the two lists side by side. In both cases the index stays in place, and the loop is added where a first-pass chunk is unlikely to hold the answer.

Mistral's own guidance on which to use

Use Agentic Search for	Indexed retrieval is the right starting point for
Long documents where the answer sits in one table, clause, figure, or footnote	Direct lookups in short, clean documents
Questions needing evidence compared across several sources	High-volume keyword or semantic lookup
Answers that must be verifiable at a stable document location	Simple, predictable questions with a known source
Tables and structured or scanned documents	Cases where extra retrieval steps are unlikely to help

Mistral Search Toolkit and agentic retrieval: triage failures

Sort the questions your current setup gets wrong into two groups before changing anything. In the first group the retrieved document is wrong, and the fix is the index. In the second group the document is right but the answer is somewhere else in it, and that is the group Agentic Search is built for.

Agentic Search is available in two ways. The Mistral Search Toolkit lets you add it to your own agents, workflows and customer deployments, in the cloud or on-premises.

Libraries in Studio and Vibe let you use it without building the retrieval system yourself. To try it on your own corpus, Mistral points to the Search Starter App, which creates a local index with a default configuration.