Skip to content
anythingai.space
AI Foundationsbeginner12 min · 8 steps

Large Language Models: GPT, BERT and ChatGPT

What an LLM is, why GPT predicts the next token while BERT fills in blanks, and how GPT-3, GPT-4 and ChatGPT grew out of that idea.

pre-trainingpromptONE PROMPT, ONE LEARNED SKILLTranslate to French:Hello→BonjourText corpusbooks · web · codeLLMbillions of parametersMany taskstranslate · answer · code
dataresponserequesttokenerror
1 / 8
Key ideaStep 1 of 8

A large language model learns language by reading

A large language model (LLM) is a neural network with billions or trillions of parameters, trained on vast amounts of text. It learns the statistical patterns of language well enough to translate, summarise, answer questions and write code.

Pre-training is self-supervised: the text itself provides the answers, so no human labels are needed. Afterwards, fine-tuning adapts the model to specific tasks. GPT-4, Claude and LLaMA are all LLMs.

What changed
  • +Text corpus
  • +Pre-trained LLM

Lesson 4 of 34 in AI Foundations