Topic 01 of 06

Tokenization

A language model cannot read words. It reads integers. Tokenization bridges human-readable text and machine-readable numbers — converting any string into a sequence of integer IDs from a fixed vocabulary.

Why characters or words aren't enough

A character-level vocabulary is tiny but produces very long sequences, making attention computationally expensive. A word-level vocabulary would require hundreds of thousands of entries and can never handle unseen words. Subword tokenization hits the sweet spot: a vocabulary of ~50,000–100,000 units, with common words as single tokens and rare words expressed as composable subwords.

Byte-Pair Encoding (BPE)

The dominant algorithm for LLM tokenization. BPE starts with individual characters and iteratively merges the most frequent adjacent pair of symbols into a new unit, repeating until a target vocabulary size is reached. The learned merge rules are then applied to encode any new text.

Topic 02 of 06

Embeddings

An integer token ID is just a label — it carries no information about meaning. Embeddings convert those integers into dense floating-point vectors where geometric proximity reflects semantic similarity.

The embedding lookup table

The model maintains a learned matrix of size V × d where V is the vocabulary size and d is the embedding dimension (512–4096 in modern models). Looking up token ID 15496 retrieves row 15496 — a vector of d numbers called its embedding. This lookup is the very first operation in every forward pass.

Geometric structure of meaning

Word2Vec (2013) demonstrated that embedding arithmetic captures analogy: vec("king") − vec("man") + vec("woman") ≈ vec("queen"). This relationship emerges from statistical co-occurrence patterns — not programming. Transformer models learn vastly richer embedding spaces, but the same geometric intuition applies.

Static vs contextual embeddings

Word2Vec and GloVe assign one fixed vector per word regardless of context. The word "bank" had the same embedding whether used in "river bank" or "bank account". Transformers produce contextual embeddings — the vector for "bank" is different in every sentence, shaped by every other token via self-attention.

Topic 03 of 06

Self-Attention

Self-attention lets every token directly query every other token — capturing long-range dependencies that recurrent networks could not. It is the core mechanism of the Transformer.

Query, Key, Value

For each token embedding x, three linear projections produce a Query Q ("what am I searching for?"), a Key K ("what do I advertise?"), and a Value V ("what do I carry?"). The attention weight between tokens i and j is the dot product Q_i·K_j scaled by √d_k, passed through softmax. The output for token i is a weighted sum of all Value vectors.

Live heatmap demo

Hover any word below. The highlight intensity on other words shows how strongly an attention head attends to that token in context.

Multi-head attention

The Transformer runs h attention heads in parallel, each with independent Q/K/V projections. Different heads specialise — one may track syntactic dependencies, another co-reference, another positional proximity. Their outputs are concatenated and projected back to the model dimension.

Causal masking (GPT)

During generation, token i must not see token j > i. A causal mask sets those attention scores to −∞ before softmax, making them zero after normalisation. This enforces strictly left-to-right generation.

Topic 04 of 06

The Transformer

"Attention Is All You Need" (Vaswani et al., Google, 2017) replaced recurrent networks as the dominant sequence-modelling architecture. The key insight: attention mechanisms alone — without recurrence or convolution — are sufficient.

Architecture at a glance

Topic 05 of 06

RLHF

Reinforcement Learning from Human Feedback transforms a raw next-token predictor into a helpful, safe assistant. Without it, a pre-trained LLM is brilliant at imitating the internet — including its worst parts.

Phase 1 — Supervised Fine-Tuning (SFT)

Human contractors write ideal (prompt → response) pairs. The base model is fine-tuned on these examples, shifting it toward "helpful assistant" behaviour in style and tone. This alone is insufficient — the behaviour is inconsistent and the model still produces harmful outputs.

Phase 2 — Reward Model Training

The SFT model generates multiple responses to each prompt. Human raters rank them from best to worst. A separate neural network — the Reward Model (RM) — is trained on these rankings. It learns to score any (prompt, response) pair without further human involvement.

Phase 3 — RL via PPO

The SFT model is treated as a policy. The Reward Model evaluates its outputs; Proximal Policy Optimization (PPO) updates the policy's weights to maximise reward. A KL-divergence penalty prevents the policy from drifting so far from the SFT baseline that it "reward-hacks" — generating responses that score well on the RM but humans dislike.

Topic 06 of 06

Hallucinations

A hallucination is any model output that is fluent, confident, and factually false. The model has no concept of truth — only of generating statistically likely continuations. "Sounding correct" and "being correct" are distinct.

Why they happen

LLMs are trained to maximise the likelihood of the next token. When asked about something rare or outside their training distribution, they do not know to say "I don't know." They generate whatever sequence would have been most likely in similar contexts — which can be entirely fabricated, stated with equal confidence to true facts.

Active mitigations

Interactive Lab

Hands-on demos

Two live experiments. Interact directly with the mechanics you've read about.

Video Library

Watch to understand

Curated lectures from the sharpest educators in ML. Click any card to watch in an overlay.

3Blue1Brown

But what is a GPT?

Visual first-principles breakdown of Transformer architecture and text generation.

Architecture

Andrej Karpathy

Let's build GPT from scratch

Code a Transformer in Python from zero — the definitive technical walkthrough.

Deep Dive

Yannic Kilcher

RLHF — InstructGPT Paper

The alignment technique behind ChatGPT, explained via the original paper.

Alignment

Andrej Karpathy

Neural Networks: Zero to Hero

From backpropagation to a character-level language model. The complete series.

Foundations

Andrej Karpathy

Let's build the GPT Tokenizer

BPE from scratch — tiktoken internals and Unicode edge cases explained.

Tokenization

3Blue1Brown

Word Embeddings & Word2Vec

The geometric intuition behind semantic vector spaces, beautifully visualised.

Embeddings