Search RAG Ingestion
Search RAG - Ingestion - Data Enrichment
Enrichment is useful only when it changes retrieval behavior.
Why this matters in production
Summaries, keywords, generated queries, quality scores, and feedback signals can make chunks easier to retrieve. They can also create a second layer of synthetic text that outranks the source and lies more confidently than the document.
This works in demos but fails in production: generating beautiful summaries and then treating them as equivalent to evidence. Summaries are retrieval aids. They are not the source of truth.
What breaks
Keyword extraction can amplify noisy terms. Generated queries can overfit to imagined user intent. Quality scores can bury rare but critical evidence. Feedback signals can encode popularity instead of correctness.
The retrieval trace must show whether a chunk was selected because of source text, summary text, generated query, feedback, or metadata. Without that trace, enrichment becomes invisible ranking magic.
What works
Keep enrichment typed and accountable. Store which model created it, when it was created, which source span it came from, and how much it influenced ranking.
Do not let enrichment replace source text in final grounding. A slightly incomplete answer from source evidence is better than a perfect-sounding answer grounded in generated metadata.
Practical guidance
Use summaries for previews and recall, keywords for lexical hooks, generated queries for intent bridging, and quality scores for deprioritizing low-value content. Test each signal independently before combining them.
Make enrichment removable. If turning off a signal makes the system safer, the signal was too powerful or too poorly measured.
What happened
After adding generated queries, a sales-support RAG system started retrieving roadmap chunks for customer-support questions because the generated queries used broad future-looking language.
Why retrieval failed
The retriever matched the synthetic query field instead of the source text and over-weighted intent hints that were never validated against real user behavior.
Why it was hard to detect
Every retrieved chunk looked semantically close, and answer fluency improved. Only trace-level analysis showed synthetic queries dominating the candidate pool.
What fixed it
The fix was per-field attribution, capped enrichment boosts, generated-query evaluation, and a rule that final claims must cite source text, not generated hints.
Practical Guidance
Track which enrichment field caused retrieval.
Cap boosts from generated metadata.
Evaluate enrichment with retrieval traces, not aesthetic quality.