Machine Learning from Scratch

The Big Question

Train an image classifier on cats and dogs. Then continue training it on cars and trucks. Suddenly cat and dog performance collapses. The new task overwrote the old one.

How can a model learn new tasks without forgetting previous ones?

Core Intuition

Learning French should not erase English. Neural networks often do not have that stability by default. Gradient descent optimises the new loss, and nothing in the new data necessarily protects parameters that were important for old tasks.

Continual learning studies lifelong adaptation under distribution shift, concept drift, limited memory, and changing tasks.

Interactive Demo

Continual learning lab

Sequential tasks

Replay buffer35%EWC strength0.40

Accuracy after current stage

Task A77%

Task B84%

Task C86%

Forgetting on Task A

17.3 pts

Replay rehearses old examples. EWC penalises movement away from parameters that were important for old tasks.

Why Forgetting Happens

After Task A, parameters sit near an optimum $\theta_A^*$ . Training on Task B minimises $L_B(\theta)$ , whose gradients may move parameters away from the region that solved Task A.

Elastic Weight Consolidation adds a penalty:

L=L_B+\lambda\sum_i F_i(\theta_i-\theta_i^*)^2

$F_i$ estimates how important parameter $\theta_i$ was for the old task. Important parameters receive larger penalties when they move.

Strategies

Replay

Keep a memory buffer of old examples and mix them with new examples.

Learning without Forgetting

Use distillation from the old model so new training preserves old outputs.

Parameter isolation

Use adapters, task-specific modules, or progressive networks to reduce interference.

Dynamic architectures

Add capacity for new tasks rather than forcing all tasks through one fixed set of weights.

Shift And Evaluation

Covariate shift changes inputs, label shift changes class proportions, and concept drift changes the relationship between input and label. Production foundation models face all three as languages, code, regulations, facts, and user behaviour evolve.

Sequential benchmarks track performance after each task. A simple forgetting metric is:

F_j=\max_{t< T} A_{t,j}-A_{T,j}

where $A_{t,j}$ is accuracy on task $j$ after training stage $t$ .

Implementation

class ReplayBuffer:
    def __init__(self, max_size):
        self.max_size = max_size
        self.items = []

    def add_batch(self, examples):
        self.items.extend(examples)
        self.items = self.items[-self.max_size:]

    def sample(self, n):
        idx = torch.randint(0, len(self.items), (n,))
        return [self.items[i] for i in idx]

for task in tasks:
    for batch in task.loader:
        replay_batch = buffer.sample(min(len(buffer.items), len(batch))) if buffer.items else []
        mixed = combine(batch, replay_batch)
        loss = model_loss(model, mixed)
        loss.backward()
        optimizer.step()
    buffer.add_batch(task.memory_examples)

def ewc_penalty(model, old_params, fisher):
    penalty = 0.0
    for name, param in model.named_parameters():
        penalty += (fisher[name] * (param - old_params[name]).pow(2)).sum()
    return penalty

loss = new_task_loss + lambda_ewc * ewc_penalty(model, old_params, fisher)

Interview Discussion

What is catastrophic forgetting?

Performance on old tasks collapses after training on new tasks.

How does replay help?

It mixes old examples with new ones so gradients still preserve old behaviour.

How does EWC help?

It penalises movement in parameters estimated to be important for previous tasks.

What are failure cases?

Replay privacy risk, limited storage, conflicting tasks, long task sequences, and evaluation leakage.

Active Recall

1. Why does gradient descent on a new task overwrite old capabilities?

2. Write the EWC objective.

3. How does Learning without Forgetting connect to distillation?

4. Distinguish covariate shift, label shift, and concept drift.