Supervised Fine-Tuning and LoRA

Learn behavior from curated demonstrations

Supervised fine-tuning (SFT) trains on input-output demonstrations, using next-token loss so desired answers become more likely. Dataset quality dominates: conflicting formats, unsupported facts, or leaked private text are learned rather than repaired by training.

Full fine-tuning updates all model weights and requires substantial memory for weights, gradients, and optimizer state. Parameter-efficient fine-tuning (PEFT) updates a much smaller set. Low-rank adaptation (LoRA) inserts small trainable matrices beside selected existing weights while leaving base weights frozen. The adapter is smaller to train, store, and swap than a full model copy.

base weight W stays frozen
effective transform = W + small learned update (A × B)
save adapter A,B plus exact base-model identity

Tip: Version the base model, tokenizer, adapter configuration, training data, and evaluation together. An adapter without its exact compatible base is not a reproducible artifact.

Warning: Fine-tuning may increase memorization of rare strings. Deduplicate data, remove secrets and PII, document rights and consent, and run extraction/privacy tests.