Machine Learning
from Scratch
Start Learning The Journey
What are we trying to build, and why does machine learning exist?
The opening act establishes the motivation, vocabulary, data concepts, and mathematical language needed before algorithms appear.
What is machine learning?
What Is Machine Learning?
Contrast hand-written rules with learned behaviour, then define models, parameters, training, inference, loss, and generalisation.
Datasets & Data Types
Understand observations, feature and label roles, data splits, and numerical, categorical, ordinal, text, time-series, and image data.
Types of Learning
When to use supervised, unsupervised, and reinforcement learning.
Exploratory Data Analysis
Inspect data quality, distributions, missing values, outliers, and early patterns before modelling.
Mathematical Foundations for Machine Learning
Vectors and matrices for representation, probability and statistics for uncertainty, and calculus for optimisation.
How does a machine learn, represent information, and generate intelligent behaviour?
This act follows the models themselves, from simple predictors to representation learning, deep networks, and pretrained language models.
How can a computer make predictions?
Linear Regression
Predict continuous values, measure vertical error with MSE, and learn good parameters with gradient descent.
Logistic Regression
Move from binary classification to probabilities, log-odds, maximum likelihood, and binary cross-entropy.
Optimising Models
Use derivatives, gradients, learning rates, and batch choices to train predictive models in practice.
Multiclass Classification
Extend classification beyond yes or no with softmax probabilities and categorical cross-entropy.
How can machines represent increasingly complex patterns?
Feature Engineering
Improve performance by changing the representation before changing the model.
Decision Trees
Replace one global equation with a hierarchy of local decisions.
Random Forests
Reduce tree variance by averaging many trees
Gradient Boosting
Correct errors one weak model at a time
Support Vector Machines
Find boundaries with maximum margin
Kernel Methods
Make non-linear patterns look linear
Representation Learning
Learn useful features instead of handcrafting them
Embeddings
Represent objects as meaningful vectors
Principal Component Analysis
Compress data while preserving structure
Clustering
Group examples without labels
How can machines automatically learn representations?
Perceptron
The simplest possible artificial neuron
Neural Networks
Combine many perceptrons into hidden layers
Activation Functions
Add non-linearity so stacked layers can learn richer functions
Backpropagation
Compute gradients through a network with the chain rule
Training Deep Networks
Train neural networks effectively with modern optimisation techniques
Convolutional Neural Networks
Learn spatial patterns in images with filters and feature maps
Sequence Models
Process ordered data with recurrent neural networks
LSTMs and GRUs
Remember information across longer sequences
Attention
Let tokens choose which other tokens matter
Transformers
Use attention as the primary computation
How do we train a transformer into ChatGPT?
Transformer as a Language Model
Use a transformer for next-token prediction with causal masking and a context window
Tokenisation
Convert text into model-readable tokens and vocabulary IDs
Language Modelling
Frame text generation as predicting the next token from previous tokens
Scaling Laws
Understand how performance changes with parameters, data, and compute
Pretraining
Learn general language patterns from massive self-supervised datasets
Fine-tuning
Adapt pretrained models with supervised data, instruction tuning, LoRA, and PEFT
Preference Optimisation
Align model behaviour with SFT, RLHF, DPO, and Constitutional AI
Inference and Decoding
Generate outputs with greedy decoding, beam search, temperature, top-k, and top-p
Once we have an intelligent model, how do we improve it, build applications with it, and deploy it?
The final technical act moves outside model internals into evaluation, adaptation, LLM applications, and production ML systems.
How do we know whether a model is actually good?
Bias versus Variance
Diagnose underfitting, overfitting, and the tradeoff between simple and sensitive models.
Regularisation
Discourage unnecessarily complex models by adding weight penalties to the loss.
Cross Validation
Estimate generalisation and compare model choices using held out folds.
Hyperparameter Tuning
Choose settings such as learning rate, tree depth, C, gamma, and regularisation strength.
Classification Metrics
Evaluate classifiers with confusion matrices, precision, recall, and F1.
Threshold Selection
Turn model scores into product decisions using thresholds, ROC curves, and precision recall curves.
Calibration
Check whether predicted probabilities mean what they claim.
Error Analysis
Inspect failures directly instead of trusting one aggregate score.
How do we improve or adapt an already-trained model?
Fine-tuning
Adapt pretrained models with supervised data, instruction tuning, LoRA, and PEFT.
Parameter-Efficient Fine-Tuning
Use LoRA, QLoRA, adapters, and other small-update methods to adapt large models cheaply.
Instruction Tuning
Turn pretrained completion models into models that follow tasks and conversational instructions.
Preference Optimisation
Align model behaviour with SFT, RLHF, DPO, GRPO, and Constitutional AI.
Distillation
Train smaller models to imitate larger or stronger teacher models.
Quantisation
Reduce numerical precision to make models cheaper and faster to run.
Continual Learning
Update models over time while managing forgetting, drift, and evaluation risk.
Now that we have an LLM, how do people actually build products with it?
Prompt Engineering
Design prompts from first principles using token prediction, context, and task framing
Retrieval-Augmented Generation
Ground model outputs with retrieval, embeddings, chunking, and re-ranking
Tool Use and Function Calling
Let models call external tools, APIs, databases, and structured functions
Memory
Store and retrieve user, task, and application state across interactions
Agents
Build model loops that plan, act, observe, and revise
Multi-Agent Systems
Coordinate multiple model roles, workers, critics, and supervisors
LLM Application Evaluation
Measure LLM application quality with tests, rubrics, model judges, and human review
Building Reliable AI Systems
Turn LLM prototypes into production systems with reliability, observability, guardrails, latency, and cost controls
How do we deploy machine learning in production?
Problem Definition
Define the product goal, model goal, and failure modes for YouTube comment toxicity detection.
Data Collection
Collect the examples, logs, and context needed for a production model.
Labels
Design reliable labels while handling ambiguity, disagreement, and noise.
Feature Pipelines
Compute features consistently for training and serving without leakage.
Training Pipelines
Turn data, code, configuration, and evaluation into repeatable model training runs.
Model Serving
Deliver model predictions with latency, reliability, and fallback constraints.
Monitoring
Watch deployed models for drift, latency, reliability, and failure patterns.
Retraining
Refresh models safely as data, users, and product behavior change.
A/B Testing
Measure production impact before fully launching model changes.
Where has the field come from, and where is it going?
The course closes by returning to the big picture: the historical sequence of ideas, bottlenecks, and breakthroughs.
How did machine learning become what it is today?
Machine Learning Timeline
Coming soon — a segmented timeline of classical ML, neural networks, deep learning, transformers, generative AI, and agentic systems.