Skip to content
anythingai.space
How LLMs Workbeginner12 min · 8 steps

LLM Internals

Why transformers replaced RNNs, how Mixture-of-Experts splits memory from compute, and why serving a model is a bandwidth problem.

To link “it” back to “animal”, information must passthrough every hidden state in between — sequentially.Theh1animalh2didn't … becauseh3 … h7ith8
datarequesttoken
1 / 8
ProblemStep 1 of 8

Before 2017: one token at a time

RNNs and LSTMs read a sentence left to right, carrying a hidden state from step to step. Step t can't start until step t−1 finishes, so training can't be parallelised.

Long-range links are hard too. In “The animal didn't cross the street because it* was too tired”, the meaning of it* has to survive six hops through hidden states to reach animal.

InsightThink of an RNN as a single-threaded pipeline where every stage waits on the previous one.

Lesson 1 of 6 in How LLMs Work