One word, several meanings
These are the terms the paper singles out as genuinely ambiguous. Each collapses techniques with different profiles.
Section 4.4
"We fine-tuned the model"
Whether every weight changed, only later layers, or none at all — three different scopes, three different validation burdens.
Ask: How much of the model was actually updated?
- FT (full) — Continues training every weight in the model on new task data — maximum flexibility, maximum cost, and no part of the model left unchanged.
- FT (partial) — Retrains only the later layers and freezes the early ones, so the general features learned during pre-training stay intact and less needs re-testing.
- PEFT (LoRA, adapters) — Freezes the base model and trains a small add-on instead, so the original weights are never touched and the adapter can be swapped out or removed cleanly.
Appendix C, Knowledge Transfer and Task Specialization
"We used few-shot learning"
FSL produces a new versioned model artifact requiring change control. ICL produces no artifact at all — the effect lasts one request.
Ask: Was a weight update performed?
Section 4.4
"We use RAG"
The corpus, the retrieval method, and the update cadence. A maintained corpus is version-persistent even when each request's context is ephemeral.
Ask: Is the retrieval artifact maintained and versioned?
- RAG — Retrieves relevant documents at query time and puts them in the context, so the model can use knowledge it was never trained on.
- CE — Manages everything that goes into the model's context — retrieved documents, tools, history — as a maintained, versioned production asset.
- ICL — Shows the model a few worked examples in the prompt, adapting it for that request only, with no training and no artifact left behind.
Section 4.4, Appendix C
"We did domain adaptation"
At the LLM tier the classical mechanism is superseded by continued pre-training or domain-specific PEFT. The label hides which was actually used.
Ask: Which model tier?
- DA — Trains a model to close the gap when the data seen at deployment looks different from the training data, without needing many labels for the new domain.
- SSL / CPT — Continues pre-training on unlabelled data from a specialist domain, so the model absorbs its patterns and vocabulary before any task-specific tuning.
- PEFT (LoRA, adapters) — Freezes the base model and trains a small add-on instead, so the original weights are never touched and the adapter can be swapped out or removed cleanly.
Appendix C, Training Strategies
"The model learns to learn"
At the LLM tier, in-context learning achieves rapid task adaptation without the bi-level training loop meta-learning implies.
Ask: Which model tier?
- Meta-Learning — Trains a model to be good at learning new tasks quickly, rather than good at any one task — "learning to learn".
- ICL — Shows the model a few worked examples in the prompt, adapting it for that request only, with no training and no artifact left behind.
Section 2
"The model was retrained"
Retraining and full fine-tuning are mechanistically identical. Only the goal and the data separate them — restoring lost performance is not the same as targeting a new task.
Ask: Why was it changed?
- Retraining — Trains the existing model again on newer data to recover accuracy lost as the world drifted away from the original training distribution.
- FT (full) — Continues training every weight in the model on new task data — maximum flexibility, maximum cost, and no part of the model left unchanged.