AI Foundationsintermediate14 min · 9 steps
Sequence Models: RNN, LSTM, GRU and the Transformer
How networks read data where order matters — a hidden state passed step by step, gates that protect long-term memory, and the attention mechanism that replaced recurrence.
tokendataresponsesignal
ProblemStep 1 of 9
Order carries meaning
“Dog bites man” and “man bites dog” contain exactly the same words. A plain feed-forward network that sees a bag of word counts gets the same input for both.
Language, speech, sensor readings and prices are sequences: what an element means depends on what came before it. We need a network that reads in order and remembers.
← Previous lessonNeural Networks: From Perceptron to Deep Q-NetworkNext lesson →Large Language Models: GPT, BERT and ChatGPT
Lesson 3 of 34 in AI Foundations