Search RAG Retrieval
Search RAG - Retrieval
Retrieval is where most RAG systems actually fail.
Why this matters in production
Generation gets blamed because users see the answer. But the answer is usually downstream of a retrieval mistake: missing evidence, wrong evidence, over-broad evidence, stale evidence, or unauthorized evidence.
Vector search is not a retrieval strategy. It is one signal. Production retrieval is query understanding, filtering, lexical search, dense retrieval, fusion, reranking, traceability, and evaluation.
What breaks
Embedding-only systems miss exact identifiers. Keyword-only systems miss paraphrases. Filter-after-search systems leak. Top-k systems overrepresent duplicated docs. None of these failures look dramatic; they produce plausible answers.
The most dangerous retrieval bug is not no answer. It is a confident answer from the wrong evidence.
What works
Build retrieval as a staged system. Resolve hard constraints first. Generate candidates through multiple independent paths. Fuse and deduplicate. Rerank with evidence quality, freshness, diversity, and source authority. Preserve the trace.
Do not optimize for answer fluency before measuring whether the right chunks reached the model.
Practical guidance
For every answer, log query transforms, filters, candidate counts, retrieval paths, per-signal scores, final chunks, citations, and excluded evidence counts.
What happened
A RAG assistant answered a configuration question with an obsolete parameter because vector search preferred a narrative migration guide over the exact API reference.
Why retrieval failed
Dense retrieval captured the concept but missed the exact config key and versioned reference page.
Why it was hard to detect
The generated answer was fluent and the cited guide looked relevant. Only a trace comparison showed the API reference never entered top-k.
What fixed it
The fix was hybrid retrieval, exact-match boosting for identifiers, version filters, and critical-chunk recall metrics.
Practical Guidance
Treat retrieval as the primary product.
Measure candidate quality before answer quality.
Use hybrid retrieval and trace-level debugging.