Skip to content
anythingai.space
AI Foundationsintermediate14 min · 9 steps

Sequence Models: RNN, LSTM, GRU and the Transformer

How networks read data where order matters — a hidden state passed step by step, gates that protect long-term memory, and the attention mechanism that replaced recurrence.

SENTENCE AdogbiterbitesmanbittenSENTENCE BmanbiterbitesdogbittenWHAT A BAG-OF-WORDS MODEL SEES (IDENTICAL FOR A AND B)dog1bites1man1
tokendataresponsesignal
1 / 9
ProblemStep 1 of 9

Order carries meaning

“Dog bites man” and “man bites dog” contain exactly the same words. A plain feed-forward network that sees a bag of word counts gets the same input for both.

Language, speech, sensor readings and prices are sequences: what an element means depends on what came before it. We need a network that reads in order and remembers.

Lesson 3 of 34 in AI Foundations