Search RAG Generation
Search RAG - Context Design
Context design is prompt engineering for evidence.
Why this matters in production
A prompt full of chunks is not a context strategy. The model needs to distinguish instructions, evidence, metadata, confidence, and output rules.
This is usually a mistake: maximizing context window usage. More context often makes answers less stable because the model must infer what matters.
What breaks
Instructions get mixed with evidence. Chunk IDs disappear. Metadata becomes prose. High-confidence and low-confidence chunks look identical. The model cites the wrong thing because the prompt did not make source identity durable.
A context package that humans cannot inspect will not be debuggable when the model misbehaves.
What works
Use structured evidence blocks. Include chunk IDs, source IDs, headings, timestamps, scores, confidence, and content type. Separate instructions from evidence and output rules.
Do not pass generated summaries without original source anchors unless you are comfortable with summaries becoming uncitable evidence.
Practical guidance
Keep the prompt boring. Make the evidence format regular, citation anchors explicit, and refusal rules simple. The more clever the context format, the harder it is to validate.
What happened
A model cited the wrong page because the prompt listed page metadata above a group of chunks instead of inside each chunk.
Why retrieval failed
Context assembly made source identity implicit, and the model copied the nearest visible page marker.
Why it was hard to detect
Automated checks saw a valid citation pattern. Only manual source review caught misattribution.
What fixed it
The fix was per-chunk metadata, deterministic citation anchors, and validation that citation IDs matched claim-supporting chunks.
Practical Guidance
Structure evidence explicitly.
Avoid context stuffing.
Validate citation anchors outside the model.