Model Development
How do we improve or adapt an already-trained model?
Adapt existing models with fine-tuning, parameter-efficient updates, preference optimisation, compression, and continual improvement.
1. Fine-tuning
Adapt pretrained models with supervised data, instruction tuning, LoRA, and PEFT.
2. Parameter-Efficient Fine-Tuning
Use LoRA, QLoRA, adapters, and other small-update methods to adapt large models cheaply.
3. Instruction Tuning
Turn pretrained completion models into models that follow tasks and conversational instructions.
4. Preference Optimisation
Align model behaviour with SFT, RLHF, DPO, GRPO, and Constitutional AI.
5. Distillation
Train smaller models to imitate larger or stronger teacher models.
6. Quantisation
Reduce numerical precision to make models cheaper and faster to run.
7. Continual Learning
Update models over time while managing forgetting, drift, and evaluation risk.