Retrieval & RAGbeginner11 min · 9 steps
Choosing an LLM for RAG
The generator is one swappable box in a RAG pipeline — pick it by hosting, context size, modality, language and cost, then prove the choice on your own questions.
requestdatatokenresponse
Key ideaStep 1 of 9
The generator is one swappable box
Retrieval finds the evidence; the LLM turns question + retrieved chunks into an answer. It is the last stage of the pipeline and usually the most expensive one.
Four knobs decide whether a model fits: the context window (how many chunks fit), token limits on output, temperature (keep it low for factual answers) and access — a cloud API or a model you run on your own hardware.
InsightKeep the generator behind one interface. You will swap it more often than any other component.
What changed
- +Generator slot in the pipeline
Lesson 16 of 37 in Retrieval & RAG