Knowledge Transfer and Task Specialization
FT (partial)
In brief — site editorial, not from the paper
Retrains only the later layers and freezes the early ones, so the general features learned during pre-training stay intact and less needs re-testing.
Definition
Partial FT targets Task Specialization via the same Parametric Update mechanism as Full FT but constrains updates to later, task-specific layers while freezing the early layers that encode broadly transferable representations [222]. This architectural discipline collapses the update surface from Whole-Model to Partial—a coordinate divergence from Full FT that carries both a data consequence and a validation consequence: the data requirement contracts to Small Labeled (the frozen layers perform feature extraction, leaving only the task-specific layers requiring supervision), and the regression testing footprint is structurally isolated to the modified layer subset, enabling targeted component-level rollback. Compared to PEFT, which introduces a fully decoupled Modular adapter that can be removed without touching any base-model layer, Partial FT embeds updates directly into the base model's later layers—rollback requires layer-level checkpoint restoration, not a clean module swap. The practical distinction matters for change-control documentation: a PEFT event generates a standalone adapter artifact; a Partial FT event generates a modified base-model checkpoint whose changed layers must be explicitly versioned. Partial FT necessitates versioned checkpointing of the modified layer subset and localized parametric regression testing bounded strictly to those layers; because the frozen early layers remain mathematically unchanged, they require no re-validation.
Verbatim from the paper — Knowledge Transfer and Task Specialization
Related techniques
- sub-technique PEFT (LoRA, adapters) — PEFT strictly extends Partial FT: same mechanism, plus a modular scope
the base model's weights remain strictly frozen throughout training
Appendix C, Knowledge Transfer and Task Specialization
Nearest profiles
Prompt Learning (0.04), FSL (0.17), PEFT (LoRA, adapters) (0.17), Meta-Learning (0.25), ReFT (0.25)
Computed from the taxonomy data (Gower distance over all six dimensions)
References
- [222] Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson 2014. How transferable are features in deep neural networks?. Advances in Neural Information Processing Systems
Numbered as in the paper
Cite this row
FT (partial) — six-dimensional profile D1: parametric-update D2: task-specialization D3: small-labeled D4: ad-hoc-permanent, scheduled-permanent D5: partial D6: dl, fm, llm, mllm Source: arXiv:2608.06246Compare with another technique Find in the explorer