Skip to content
anythingai.space
How LLMs Workintermediate13 min · 9 steps

Attention Mechanisms

How tokens look at each other — queries, keys and values, the causal mask, and the head-sharing tricks (GQA, MQA, MLA) that decide your KV-cache bill.

ONE SENTENCE, ELEVEN TOKENSTheanimaldidn'tcrossthestreetbecauseitwastootired“it” must find “animal” — and ignore “street”
datarequestresponse
1 / 9
ProblemStep 1 of 9

Meaning lives in other tokens

In “The animal didn't cross the street because it* was too tired”, the vector for it* means nothing on its own. To understand it, the model has to look back and decide that animal — not street — is the relevant word.

Attention is the mechanism that lets every position gather information from every other position directly, instead of passing messages down a chain.

InsightDistributed-systems analogy: an RNN is message passing along a chain; attention is pub/sub where every node can query every other node.

Lesson 3 of 6 in How LLMs Work