Search RAG Retrieval
Search RAG - Reranking and Output
Reranking is where recall becomes evidence.
Why this matters in production
Candidate generation should be broad. Generation context should be selective. Reranking is the narrow bridge between those two truths.
This works in demos but fails in production: sending top-k directly to the model because the retriever returned something plausible.
What breaks
Top-k often contains duplicates, stale chunks, low-quality chunks, near matches, and one document repeated five times. The model then appears to hallucinate, but it was starved of diverse evidence.
A retrieval output without scores, provenance, applied filters, and rank reasons is not a contract. It is a bag of text.
What works
Rerank with semantic relevance, lexical strength, source authority, freshness, quality score, permission state, and diversity. Keep enough evidence for synthesis, but not so much that context becomes noise.
Do not optimize reranking only for relevance if the answer needs multiple perspectives. Diversity is a correctness feature.
Practical guidance
The retrieval output should be structured: chunk ID, text, source, heading, scores, retrieval paths, applied filters, rank, and citation anchor.
What happened
A policy bot retrieved five chunks from the same FAQ and missed the exception table in a separate policy document.
Why retrieval failed
Reranking rewarded topical similarity and ignored document diversity and source authority.
Why it was hard to detect
The answer was consistent and well cited, but it was incomplete because the exception table never reached generation.
What fixed it
The fix was source diversification, authority boosts, and evaluation that required critical exception evidence in the final context.
Practical Guidance
Rerank for diversity and authority.
Return a structured retrieval contract.
Evaluate context completeness, not only top score.