Large Language Models
How do we train a transformer into ChatGPT?
Turn transformer architecture into large language models through tokenisation, language modelling, scaling, training, alignment, and decoding.
1. Transformer as a Language Model
Use a transformer for next-token prediction with causal masking and a context window
2. Tokenisation
Convert text into model-readable tokens and vocabulary IDs
3. Language Modelling
Frame text generation as predicting the next token from previous tokens
4. Scaling Laws
Understand how performance changes with parameters, data, and compute
5. Pretraining
Learn general language patterns from massive self-supervised datasets
6. Fine-tuning
Adapt pretrained models with supervised data, instruction tuning, LoRA, and PEFT
7. Preference Optimisation
Align model behaviour with SFT, RLHF, DPO, and Constitutional AI
8. Inference and Decoding
Generate outputs with greedy decoding, beam search, temperature, top-k, and top-p