Rules Break Where The World Gets Messy
Imagine building a spam filter. You could write: if an email contains “FREE,” mark it as spam. Then legitimate newsletters fail, spammers write “FR33,” and invoices containing “payment” trigger the wrong rule. Real language contains too many exceptions to enumerate.
Machine learning offers a different contract: provide examples of inputs and desired outcomes, choose a model family and objective, then let a training algorithm find useful parameter values.
Machine learning is a way to build behaviour from data when writing every rule is impractical.
One System, Two Phases
Training
Historical emails + spam labels → training algorithm adjusts parameters → trained model.
Inference
New unseen email → trained model → spam probability and decision.
Training is learning from examples. Inference is using what was learned. Evaluation asks whether inference works on examples not used to fit the parameters.
Rules Versus Learned Behaviour
Rules are explicit and inspectable but brittle. A learned model can combine patterns from examples, but its behaviour depends on the training data and objective.
The Vocabulary Every Later Lesson Uses
The behaviour we want: classify spam, predict price, rank videos.
One observed case: an email and, during supervised training, its answer.
Information made available to the model.
A parameterised function that maps inputs to outputs.
A learned internal value controlling the mapping.
The model’s output for one input.
A number measuring how wrong a prediction is.
The process of adjusting parameters to reduce loss.
Running the trained model on new inputs.
Working well on new cases, not merely memorising training cases.
Read this as: model with learned parameters turns input into prediction . Training changes ; inference keeps it fixed.
Worked Example: A Tiny Spam Model
Represent an email with two features: number of suspicious phrases and number of links . A simple model scores:
For an email with two suspicious phrases and one link, . A positive score predicts spam. The values 1.4, 0.8, and −2.0 are parameters. In real training they are learned from examples rather than chosen by hand.
Near-twin: compute the score for zero suspicious phrases and one link. Then decide whether the model predicts spam.
Reveal solution
0 + 0.8 − 2 = −1.2, so it predicts not spam.
Training And Inference In Code
# Tiny conceptual example: parameters improve from labelled examples.
weights = [0.0, 0.0]
bias = 0.0
def predict(features): # inference
return sum(w*x for w, x in zip(weights, features)) + bias
for features, label in training_examples: # training
error = label - predict(features)
for j in range(len(weights)):
weights[j] += 0.01 * error * features[j]
bias += 0.01 * error
new_email_score = predict([2, 1])Later lessons derive real loss functions and optimisation. For now, identify which lines belong to training and which belong to inference.
Common Misconceptions
“ML finds truth.”
It finds patterns useful for an objective in observed data.
“Training data is the program.”
Behaviour also depends on representation, model family, loss, optimisation, and decision rule.
“High training accuracy means success.”
A model can memorise. Performance on new, representative data is what matters.
Mastery Check
1. Define model, parameter, training, inference, and generalisation without notes.
2. In ŷ = f_θ(x), what changes during training?
3. Why is “contains FREE” brittle?
4. Give a task where rules are preferable because behaviour must be exact.
5. Tomorrow: retrieve all ten vocabulary terms. Three days: explain the two-phase diagram from memory.
Next: What Are Examples Made Of?
We have named the learning process. Now we need to inspect the raw material: rows, columns, labels, splits, and the very different kinds of data a column can contain.