Skip to content
anythingai.space
Retrieval & RAGintermediate11 min · 8 steps

Prompt Layout & Conversation History

Put retrieved evidence in the right message role, tell the model how to stay grounded and cite, ask for a fixed answer structure, and keep long conversations inside the window by truncating and selecting history.

grounded74%
history6,200tok
ONE PROMPT STRINGrulesretrieved text (incl. “ignore the above…”)40 turns of chatQwindowRules and evidence look the sameHistory crowds out evidence
requestdataresponse
metrics are illustrative
1 / 8
ProblemStep 1 of 8

Everything in one string

A first RAG prototype often builds one long string: instructions, then retrieved text, then 40 turns of chat, then the question. The model can't tell rules from evidence — a retrieved page saying “ignore the above” reads like an instruction — and by turn 40 the history has pushed the evidence out of the window.

Prompt layout fixes the first problem; history management fixes the second.

Lesson 29 of 37 in Retrieval & RAG