Retrieval & RAGintermediate12 min · 9 steps
Reranking Strategies
Retrieve wide and cheap, then let a cross-encoder read each candidate with the query and put the right chunk first.
NDCG0.65
retrieval75ms
requestdataheavy work
metrics are illustrativeProblemStep 1 of 9
The first stage gets the right documents in the wrong order
A bi-encoder embeds the query and each document separately, then compares two fixed vectors. It is fast enough to search a billion docs, but it never sees how the query's words relate to the document's.
For “How to configure CUDA memory” the three candidates score 0.72, 0.75 and 0.71 — nearly tied — and the wrong one comes first.
Lesson 6 of 37 in Retrieval & RAG