Agentic search

Context: Text search options covers regex, BM25, SPLADE and dense retrieval. Only result quality is discussed here, not speed or cost.

The latest step is not a new way of matching but a new driver. An LLM agent writes the queries, reads the results, and searches again. It can use any of the classic methods as tools: regex, BM25, dense retrieval, a reranker.

This brings regex back. An agent reads through many results and adjusts the query again and again, which people rarely do. And its adjustments are informed: a strong model brings a lot of knowledge about the language and the domain, such as synonyms, related terms and how things are usually phrased. That knowledge does the job that the analysis chain or the embedding model did in the earlier families, so exact matching becomes usable again. The “understanding” moves back to the user’s side, except the user is now a model.

The evidence so far

It is young and mixed.

Iteration narrows the gap but doesn’t close it

In BrowseComp-Plus ↗ (2025, ~100K documents, 830 questions), one-shot BM25 finds only 1.2% of the needed evidence in its top 5. An agent (gpt-5) iterating over BM25 eventually collects 62%.
Still, with the same agent and the same tool interface, a dense retriever wins for every model tested, by 10–22 points (e.g. gpt-5: 55.9% → 70.1% accuracy).
In Search-R1 ↗ (2025), an agent with BM25 makes more search calls to compensate, but only partly.

Rewriting rescues lexical search on hard queries

On BRIGHT ↗ (ICLR 2025), where relevance needs reasoning, GPT-4-rewritten queries raised BM25 from 14.5 to 27.0 nDCG@10. That is above a dense model given the same rewritten queries (24.8).

Raw grep can win, on literal tasks

In a 2026 preprint (DCI ↗), Claude Code with only grep, bash and file reading reached 80% on BrowseComp-Plus, vs 69% for the same model with a dense retriever. It found fewer relevant documents but checked exact text better once it found one.
Its accuracy fell to 37.5% when the corpus grew 4×, with over 120 tool calls per question.
The questions are puzzles built from exact clues, which favours grep.

Grep output is unranked, so agents filter rather than rank

The agent’s tool output is truncated, so with many matches it sees only the first N.
With grep those come in file order. With ripgrep they come in path order, or in no fixed order since it searches in parallel. So the agent sees an arbitrary N, not the best N.
The agent then narrows the pattern and tries again. That is filtering by repeated refinement, not ranking.
This is consistent with the call counts: in DCI, grep agents made about twice as many calls as agents with a ranked retriever (35 vs 18), and calls grew sharply with corpus size.

The weak point is the agent’s queries

Most retrieval failures are on the query side. In Liu et al. (2026) ↗, the agent never reaches the right topic in 55–67% of failures, and 77–94% of searches add no new evidence.
Agents give up after one empty search, or use a wrong query format and conclude nothing exists.
Writing correct patterns is hard. GPT-3.5 got only 33% of regex tasks right on the first try, and 52.5% within 10 tries (Siddiq et al., 2024 ↗).

My scepticism: markup

Raw documents are full of markup. A phrase split by **bold**, a link or a line break doesn’t match a plain pattern.
Agents often miss this. When they notice, they tend to make it worse by writing regex-based Markdown or HTML parsers.
I expect grep-style agentic search to work better over text with the markup removed, giving the same treatment that BM25’s analysis chain gives with markup stripping.
This is an untested hypothesis: no study yet measures frontier models writing search patterns, or patterns over Markdown or HTML.

Idea: a search tool for agents over a folder of documents

This is a high-level design, not a tested system. Details get decided when it is built.

The setting: an agent searches a folder of documents with markup (Markdown, HTML, PDF, LaTeX).

Preparing the text

Parse each document with a proper parser for its format. Markdown and HTML are straightforward. PDF and LaTeX are trickier.
Split each document into units of about 200–400 tokens along the parsed structure (see Chunk size for dense retrieval).
Remove the markup, but keep a mapping back to positions in the original text. A search result can then point to its position, and the agent can read the text around it if it wants to.

Direction: our own analysis pipeline

The BM25 formula is saturated (see Text search options). The analysis chain is the part left to tune, so we want it in our own hands rather than inside a search engine:

Experiment with tokenizers. Compare variants on the same corpus.
Correct tokenization with the extracted entities. Keep names like seL4 whole, and keep proper names away from the stemmer, using the gazetteer below.
Analyze by node type. For example, code blocks get no stemming, and prose gets the full chain.
Keep offsets in our own tables, relative to the unit, the same as for markup removal and entity mentions.

The engine then only stores and scores the terms we give it. That makes two requirements:

It accepts pre-tokenized input.
It stores positions, if phrase and proximity queries are needed.

Some engines that fit, as examples rather than a final choice:

SQLite FTS5 with a tokenizer written in Python (apsw ↗ 3.47+).
Tantivy (pre-tokenized text; in the Python bindings only on master as of October 2026).
Lucene.
PISA ↗, which accepts term → weight input for both BM25 and learned sparse. It has no positions, and its Python wheels are Linux-only.

Tools

Tool	Control for the agent	Chance to miss something
Grep over the clean text	High: the agent tunes the pattern exactly	Higher
BM25 with the usual additions (stemming, folding, etc.), or BM25 + dense vectors fused with RRF, or BM25 + reranking with a local model	Lower	Lower

Grep results are not fused with the others. Grep would mostly echo BM25, and mixing an exact tool with ranked ones blurs what each is for.

How the agent uses them

The first query runs all tools, and their results are shown separately. That way the agent learns what options it has. If grep misses something that BM25 finds, it won’t conclude that nothing exists and give up.
After that, the agent picks the tool per query, e.g. --grep or --bm25.

Building it up

The simplest version does everything on the fly: parse, split and search at query time.
The next step is a persistent index, rebuilt incrementally: a file is re-processed when it changes, and a unit is re-indexed only when its hash changes.
Then add dense vectors, fused with BM25 via RRF.
Optionally, extract entities, dates and numbers (see below).

Optional: extracting entities, dates and numbers

Text matching handles quantities, dates and exact entities poorly. The idea is to extract them into separate fields that the agent can use as filters or boosts, e.g. --after 2021 or --entity Q312. They are not appended to the text, so extraction errors don’t add noise to ordinary queries.

Sketch:

Mentions are stored as standoff annotations. Each is a span with offsets into the unit’s clean text, relative to the unit start, keyed by the unit’s hash rather than its position. A unit that moves, or appears in several documents, reuses its mentions. The unit’s offset map leads back to the original source.
Schema: document → units → mentions → entities.
- A mention is a span in a unit, with its type, a normalized value (ISO date, quantity in SI units), the recognizer and its version, and a confidence.
- An entity is the thing itself, with a canonical ID (DOI, ORCID, Wikidata ID) and aliases, shared across documents. “Apple Inc.”, “Apple” and “the company” are three mentions of one entity.
Document context is a separate layer. Some extractions need more than the unit: resolving relative dates (“last Tuesday”) against the document date, or carrying “Apple” = Apple Inc. through a document. This layer is keyed by document and recomputed when the document changes.
Recognizers are pluggable. No single system covers all entity types, so recognizers are chosen per folder, file type or language.
- Each declares the types it produces, its version, its languages, and whether it needs document context.
- The cache key is unit hash + recognizer + version, so upgrading one recognizer reprocesses only its own mentions.
- Overlapping results from different recognizers are all kept, labelled by source. The precision of each recognizer decides whether its mentions are safe as filters or only as boosts.

Potential entity types, with typical precision:

Kind	Examples	How to recognize	Typical quality	Use as
Identifiers	ISBN, ISSN, ORCID, DOI, arXiv ID, legal citations	Patterns + check digits, then resolve against Crossref / OpenAlex	~0.95–0.99 (DOI regex covers 99.3% of DOIs ↗)	Filters
Reference-list entries	”Vaswani et al. 2017. Attention is all you need…”	Match the whole string against Crossref	P 0.98 (Crossref ↗)	Filters
People, places, organizations	PERSON, GPE, ORG	Standard NER (spaCy, OntoNotes types), linking to Wikidata	F1 0.90–0.95 (spaCy trf ↗)	Filters
Dates and times	”4 March 2021”, “last Tuesday”	HeidelTime, SUTime, Duckling, plus the document date	Normalization F1 0.78 (HeidelTime, TempEval-3)	Filters
Quantities with units	”5 mg”, “$1,200”, “3.5 GB”	CQE, Duckling, Recognizers-Text	Value + unit F1 0.86 (CQE); other tools 0.34–0.59	Filters
Events, laws, products, works of art	EVENT, LAW, PRODUCT, WORK_OF_ART	Standard NER	F1 ~0.6 (spaCy trf), 0.35–0.4 (CPU models)	Boosts at most
Books, datasets, software	Book titles, dataset names, tools	Dictionary / gazetteer matching against a known list; ML is weaker	Dataset names: dictionary P 0.91 vs ML 0.64. Software: ML F1 0.71. Book titles: F1 0.4–0.6	Boosts, or filters with a dictionary
Scientific concepts	Methods, tasks, metrics; genes, diseases, chemicals	SciERC-style models, scispaCy	F1 0.64 (SciERC); 0.69–0.84 (biomedical)	Boosts
Any named type	”book title”, “dataset”, …	Zero-shot models (GLiNER) or LLMs	Average F1 ~0.48–0.61	Boosts, to explore

A custom gazetteer for less popular entities. General models and knowledge bases miss obscure names: internal projects, niche tools, local people. A gazetteer of our own covers them, and dictionary matching is the most precise of the ML-free options.

The agent can grow it. When it meets a name that recognizers miss, in documents or in queries, it adds an entry with type and aliases.
Entries keep their provenance (added by whom, from which document), since an agent can add wrong ones.
Changing the gazetteer only reprocesses that recognizer’s mentions.

The gazetteer also protects names in the BM25 analysis chain. Names are where tokenization and stemming go wrong:

Splitting: a word delimiter filter splits seL4 into se, l, 4, and camelCase names into parts.
Stemming: Porter stems Hastings to hast and Wells to well, so a name starts matching ordinary words.

Engines can protect specific words: Lucene/Elasticsearch have keyword_marker (skip stemming), stemmer_override, and protected_words in the word delimiter filter. The gazetteer can feed those lists.

Spotting the problems can be automated:

Run each gazetteer entry through the analyzer.
If it is split, or stems into a term that is frequent in the corpus, add it to the protected list.

The agent can do the same for names it notices in search results.

Two patterns stand out. The most reliable extractors are exact-match techniques: patterns with check digits, and dictionaries of known names. And scientific papers should be linked to OpenAlex or Crossref, not Wikidata: Wikidata holds about 46M scholarly works against OpenAlex’s ~328M.

Running it locally: candidate components

Not a final list. These are candidates for estimating what the system would require, as of October 2026. The agent’s LLM is assumed to be remote; this is only the search side. RAM figures are estimates (parameters × 2 bytes at bf16, about half at 8-bit), not published numbers.

Component	Candidates	Size / RAM	Quality	Runs on
Grep	ripgrep	—	—	CPU
BM25	Tantivy ↗, SQLite FTS5, bm25s ↗, Pyserini (Lucene)	small	depends on analyzers, not the engine	CPU
Learned sparse, documents only	OpenSearch doc-v3-gte ↗	133M; model needed only at indexing	BEIR 0.546; English only	Query side: tokenizer + lookup table, no model. Documents: sentence-transformers `SparseEncoder` (CPU, CUDA, MPS)
Embedder	Qwen3-Embedding ↗ 0.6B / 4B / 8B	~1 / ~4–5 / ~8–9 GB at 8-bit	MTEB English retrieval 61.8 / 68.5 / 69.4	llama.cpp, Ollama, TEI, sentence-transformers, mlx-embeddings
Embedder, small	EmbeddingGemma-300M ↗, gte-modernbert-base ↗	<200 MB quantized; 149M	strong for size; BEIR 55.3	llama.cpp, Ollama, TEI, ONNX
Reranker	Qwen3-Reranker ↗ 0.6B / 4B	~1 / ~4–5 GB	MTEB reranking 65.8 / 69.8	llama.cpp (`--rerank`), sentence-transformers
Late interaction (optional)	GTE-ModernColBERT ↗, answerai-colbert-small ↗	150M; 33M	BEIR 54.7; 53.8	PyLate ↗, sentence-transformers
Vector index	usearch ↗, LanceDB ↗, sqlite-vec ↗	—	exact search is fine for a folder-sized corpus	CPU

Notes on runtimes:

llama.cpp ↗ is the most portable accelerated option.
- It serves both the embedder and the reranker (llama-server --embedding, --rerank).
- It runs on Metal (Mac), CUDA (NVIDIA), HIP and Vulkan (AMD), and SYCL (Intel).
- Some community Qwen3-Reranker GGUF files return near-zero scores. Use properly converted ones.
MLX is partial. mlx-embeddings ↗ covers embedders, but there is no maintained path for text rerankers.
Ollama serves embeddings but has no rerank API ↗.
Avoid on Mac: FAISS (no Metal support) and ONNX Runtime’s CoreML backend (preview, partial operator support).

Rough totals for the search side:

Minimal: BM25 + sparse + grep. No model at query time.
Small: + 0.6B embedder + 0.6B reranker. About 2–3 GB.
Quality: + 4B embedder + 4B reranker. About 9–10 GB.

To verify

Whether extracted entities add information or mostly noise, beyond identifiers and dates.
Whether reranking grep hits helps. Grep returns matches unranked, so the agent sees the first N rather than the best N. A reranker could order them, but it isn’t clear this beats letting the agent refine the pattern.