Agentic Search vs RAG: Skip the Vector DB

Files, folders, a repo, a Drive? Agentic search. Give it list and grep and get out of the way. This is the case Anthropic already settled by pulling vector search out of Claude Code.

A database, a CRM, a ticket system?Agentic search, and honestly it isn't close. You have IDs, statuses, dates and foreign keys. Embedding a customer record so you can find it by vibes is a strange thing to do to a primary key.

Four years of support transcripts nobody tagged?RAG. There's no filter that narrows that, users describe problems in words that never appear in the ticket, and meaning is genuinely the only handle you have.

A 200-page policy PDF? Neither, probably. That fits in context now. Two years of retrieval architecture debate got quietly deleted by context windows getting bigger — I wrote about that in the 1M context window post.

Answering live, inside a phone call? Pre-computed retrieval, because agentic search costs you turns and turns cost you seconds, and a caller hears every one of them.

Classic RAG decides what the model sees before the model wakes up. You chunk every document, embed the chunks, and at question time a cosine-similarity function picks the top five and pastes them into the prompt. The model gets one shot at whatever the math handed it.

Agentic search inverts that. The model gets tools instead of a payload — list the directory, search for this string, open that file, query the orders table where email = x — reads what comes back, and decides whether it needs another look. Nobody pre-decides relevance. The agent narrows the way you would.

Anthropic's context engineering guidance calls this “just in time” retrieval: agents “maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime using tools.” The agent holds a pointer, not the payload — and dereferences it when it actually needs to.

That's not a research idea. It's the reason Claude Code reads your repo with grep rather than an index, and it's the same instinct behind handing an agent a CLI instead of a wrapper API: the tools people already use are usually better tools than the ones we build for models.

Here's the failure I hit over and over before I stopped defaulting to a vector store. Embeddings retrieve what reads like the question. Very often the thing that answers the question reads nothing like it.

Ask “why did the Tuesday invoice sync fail?” and cosine similarity happily returns three paragraphs of documentation about invoice syncing — topically perfect, useless. The record that answers it is a log line reading ECONNRESET at 03:14, which shares almost no semantic surface with the question. A grep for the date range finds it in one call.

Structure gets flattened too. A chunk boundary doesn't know that a function imports another file, that a ticket has a parent, that a row has a foreign key. Embeddings turn a graph into a bag of paragraphs, then you spend a month writing re-rankers to reconstruct the relationships you deleted.

And then there's staleness, which is the one that actually bit me in production. A client updated their pricing doc; the embedding job ran on a nightly cron; the agent confidently quoted old pricing to prospects for most of a business day. Nothing errored. Nothing alerted. The vector store was doing exactly what it was told. Any retrieval layer that can silently disagree with the source of truth will eventually do so at the worst possible moment.Agentic search can't drift, because it reads the real thing every time.

I don't want to sell you the fun half. Agentic search genuinely costs more per answer, and Anthropic says so directly: “runtime exploration is slower than retrieving pre-computed data.” Here's the whole trade.

Read that token row carefully, because it's where people get the cost wrong. Agentic search reads more tokens, but the expensive part — the system prompt and tool definitions — sits in a stable prefix that prompt caching bills at a fraction of the fresh rate. Meanwhile the RAG column's “lower, fixed” number quietly excludes the embedding bill, the vector host, and the engineer maintaining the pipeline. Compare total system cost, not the retrieval call.

I'm not anti-vector. I'm anti-default. There are three cases where I reach for embeddings without hesitating.

Large unstructured prose with no filters.Years of chat transcripts, research corpora, regulatory text. Nothing narrows the field, so you need meaning as the index. This is the original job and it's still the right tool.

Vocabulary mismatch between user and source.A customer types “the thing won't charge overnight” and the manual says “battery fails to reach full state of charge during standby.” No keyword search on earth connects those. Embeddings do, effortlessly.

A hard single-hop latency budget. Voice is the obvious one. In real-time voice agents you have a couple hundred milliseconds before the caller notices dead air. You cannot spend that on four exploratory tool calls. Pre-compute and take the staleness risk knowingly.

If you do land here, the store you pick matters less than people think — I broke down pgvector vs Qdrant vs Pinecone and for most workloads Postgres you already run beats a new managed service you now have to operate.

Anthropic's own framing allows for it: “the most effective agents might employ a hybrid strategy, retrieving some data up front for speed, and pursuing further autonomous exploration at its discretion.”

In practice that means semantic search becomes one tool among several, not the front door.Structured lookup and full-text search go first because they're exact, cheap and auditable. Vector search is what the agent calls when the cheap paths come back empty.

The nice side effect is that your index shrinks to the slice that genuinely needs meaning-based recall. That's a smaller embedding bill, a shorter reindex, and far less surface area for the staleness bug I described earlier. Ordering the tools this way isn't a compromise — it's the same “cheapest thing that answers it” principle behind how I cut agent costs generally.

One thing to get right if you build this: the tool descriptions. An agent choosing badly between four retrieval tools is almost always a description problem, not a model problem — it's near the top of n8n's own list of common agent failures. “Looks up a customer order by order ID or email address” works. “Handles data” does not.

My default is now boring: no vector database until a real question fails without one.

The rebuild that convinced me was an internal knowledge agent sitting on a client's Drive, Notion and ticket queue. Version one was textbook RAG — chunk everything, embed nightly, top-k into the prompt. It was fine at “explain our refund policy” and hopeless at “what did we tell Acme in June,” which is what people actually asked. Version two dropped the index entirely and gave it four tools: search Drive, read a doc, query tickets by account and date, read a ticket. Fewer moving parts, no nightly job, and the answers stopped being confidently wrong about specifics. It costs more per question. Nobody has ever mentioned that; they mention that it's right now.

Where I do keep embeddings, it's a narrow slice — usually one folder of long-form prose — exposed as one tool the agent may call. And I instrument every retrieval agent the same way I'd instrument any other, because “the answer was wrong” is unfixable without traces showing which tool it called and what came back.

The mistake I see most is treating “we need RAG” as a requirement rather than a hypothesis. It arrives in briefs pre-decided, usually because a tutorial made it look like the architecture rather than one option in it. Then six months later somebody owns a reindex cron nobody understands, serving answers a grep would have gotten right.