Machine Learning from Scratch

The Big Question

We now have a transformer architecture and a way to turn text into tokens. The next question is the one that makes modern language models possible.

What exactly are we training the model to do?

The answer is simple: predict the next token. GPT-style models are not initially trained to answer questions or hold conversations. They are trained to assign high probability to the next token in ordinary text.

The Prediction Problem

Given this prefix:

Jessie ordered a

The model should estimate a probability distribution over the vocabulary:

Token	Probability
burger	0.72
salad	0.14
pizza	0.08
coffee	0.03
umbrella	0.000001

The model does not output a word directly. It outputs scores that become a probability distribution over possible next tokens.

Core Intuition

Next-token prediction sounds almost too small to matter. But to predict text well, the model must learn grammar, syntax, facts, style, code patterns, mathematical notation, and long-range dependencies.

None of those are labeled explicitly. They become useful because they reduce next-token prediction error.

One Sentence, Many Examples

The sentence Jessie ordered a burger. becomes many supervised training examples:

Input prefix	Target
Jessie	ordered
Jessie ordered	a
Jessie ordered a	burger
Jessie ordered a burger	.

A large corpus creates billions of these input-target pairs without human annotation. Ordinary text supplies its own labels.

The Full Pipeline

Text	Jessie ordered a burger	Raw training text
Tokens	[Jessie, ordered, a, burger]	Text units the model can index
Token IDs	[0, 1, 2, 3]	Integer vocabulary entries
Embeddings	dense vectors	Learned input representations
Transformer	contextual vectors	Causal attention processes the prefix
Logits	one score per vocabulary token	Unnormalized next-token scores
Softmax	probabilities	A distribution over possible next tokens
Loss	cross-entropy	Penalty for the true next token probability

Interactive Demo

Next-token training lab

Prefix length3

One sentence becomes many training examples

Input prefix	Target next token
Jessie	ordered
Jessie ordered	a
Jessie ordered a	burger
Jessie ordered a burger	.

Causal mask

see

mask

see

mask

see

mask

see

mask

see

A decoder-only transformer cannot look at future tokens while learning next-token prediction.

Current prediction problem

Jessie ordered a ?

Target next token: burger

burger5.80.909

salad3.10.061

pizza2.20.025

coffee0.40.004

.-1.80.000

Cross-entropy loss

-log(0.909) = 0.095

The model is rewarded for assigning high probability to the actual next token.

Move through the prefix lengths. Each position is a separate next-token classification problem trained with causal masking and cross-entropy.

Mathematics

Logits

The transformer does not directly output probabilities. It outputs logits: arbitrary real-valued scores, one per vocabulary token.

z=W_{\text{out}}h_t+b_{\text{out}}

If the vocabulary has 50,000 tokens, this vector has 50,000 logits.

Softmax

Softmax converts logits into a probability distribution:

P_i=\frac{e^{z_i}}{\sum_j e^{z_j}}

For logits $[2,1,0]$ , exponentiation gives $[7.39,2.72,1]$ . The sum is $11.11$ , so the probabilities are approximately:

[0.665,\;0.245,\;0.090]

Cross-Entropy Loss

If the correct next token is burger and the model assigns it probability $0.66$ , the loss is:

L=-\log(0.66)=0.415

If the model assigns burger probability $0.02$ , the loss becomes:

L=-\log(0.02)=3.91

Language modelling is multiclass classification repeated at every token position, with the vocabulary as the class set.

Teacher Forcing

During training, the model receives the correct previous tokens. When predicting burger, the input is Jessie ordered a, not a prefix corrupted by the model's earlier mistakes.

During generation, the model feeds back its own sampled tokens, so errors can compound over long outputs.

Implementation

The key training shift is that inputs predict the next token at each position.

import torch
from torch import nn

vocab_size = 100
batch = torch.randint(0, vocab_size, (8, 16))

# Inputs predict the next token at each position.
inputs = batch[:, :-1]
targets = batch[:, 1:]

logits = model(inputs)  # (batch, sequence, vocab_size)

loss = nn.CrossEntropyLoss()(
    logits.reshape(-1, vocab_size),
    targets.reshape(-1),
)

loss.backward()

Tokens up to position $t$ predict token $t+1$ . That simple shift creates the language modelling objective.

Interview Discussion

What is a language model?

A language model estimates a probability distribution over the next token given previous tokens.

What are logits?

Logits are unnormalized scores, one per vocabulary token, produced before softmax.

Why use softmax?

Softmax turns arbitrary logits into a probability distribution over the vocabulary.

Why is cross-entropy natural here?

The model is doing multiclass classification at each token position, and cross-entropy penalizes low probability on the correct next token.

What is teacher forcing?

During training, the model is fed the correct previous tokens rather than its own generated mistakes.

How is language modelling self-supervised?

Ordinary text supplies labels automatically: every token is the target for the prefix before it.

Active Recall

1. Explain the complete training pipeline for one next-token prediction.

2. What is the difference between a token ID, an embedding, a logit, and a probability?

3. Why does one sentence generate many training examples?

4. Why is language modelling just large-scale multiclass classification?

5. Why can next-token prediction lead to useful language abilities?

Common Mistakes

Thinking the model directly predicts words instead of logits over tokens.
Confusing embeddings with logits.
Forgetting that training uses teacher forcing.
Believing language modelling requires manually labeled question-answer pairs.
Assuming the model only memorizes exact continuations rather than learning a distribution.

Connection To Scaling Laws

Language modelling gives us the objective. The next question is why larger models, larger datasets, and more compute often produce predictably better models.