Skip to content

Knowledge Transfer and Task Specialization

FSL

In brief — site editorial, not from the paper

Learns a new task from a handful of labelled examples by actually updating weights — as opposed to just showing examples in the prompt, which is ICL.

Definition

FSL is defined within this taxonomy exclusively as a Parametric Update intervention under extreme data scarcity. Depending on the implementation, it relies on either 1–5 examples for metric-based support sets (Few Demonstrations) or tens of examples for gradient-based updates (Small Labeled), producing a persistent weight modification (Ad-hoc Permanent, Scheduled Permanent) [185, 201]. This strict boundary resolves the taxonomy's most consequential terminological ambiguity: colloquial ``few-shot'' usage conflates two mechanistically and operationally distinct operations. The parametric form produces a versioned model artifact that has undergone a parameter-modifying training event and therefore demands formal change-control tracking and baseline regression testing. The non-parametric, prompting-based form shares Few Demonstrations but operates via Context Injection, producing only a Session-Ephemeral behavioral shift that leaves the underlying model artifact unmodified; this is reassigned to ICL. The D4 (Persistence) divergence is the taxonomic anchor: FSL creates a new versioned artifact requiring software change-control; ICL creates no artifact whatsoever. FSL's DL, FM ceiling mirrors DA's: at the LLM tier, ICL achieves rapid few-shot task specialization through context conditioning alone, making formal FSL training pipelines architecturally redundant for pure few-shot tasks at that scale. FSL necessitates version-control tracking of the resulting weight checkpoint; the extremely small Few Demonstrations dataset constitutes the sole training provenance record and should be fully documented.

Verbatim from the paper — Knowledge Transfer and Task Specialization

Notes from the table

For in-context FSL at the LLM level, see ICL.

Classification tensions

FSL. In this framework, the FSL categorization is strictly reserved for parametric approaches—such as gradient-based meta-learning or metric-based networks—where the model's weights undergo a Ad-hoc Permanent, Scheduled Permanent update based on limited labeled data. While the term FSL is colloquially used to describe providing examples in an LLM prompt, this framework classifies that non-parametric, Session-Ephemeral mechanism distinctly as ICL. ICL functions primarily to induce a Knowledge Update, Task Specialization mapping during inference, bypassing parameter updates entirely.

Appendix C

Related techniques

  • sub-technique ICL — Colloquial "few-shot" splits here: FSL updates weights, ICL does not
    underlying model artifact unmodified; this is reassigned to ICL Appendix C, Knowledge Transfer and Task Specialization

Nearest profiles

Meta-Learning (0.17), FT (partial) (0.17), Prompt Learning (0.21), DA (0.28), PEFT (LoRA, adapters) (0.33)

Computed from the taxonomy data (Gower distance over all six dimensions)

References

  1. [185] Jake Snell, Kevin Swersky, and Richard Zemel 2017. Prototypical networks for few-shot learning. Proceedings of the 31st International Conference on Neural Information Processing Systems
  2. [201] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray Kavukcuoglu, and Daan Wierstra 2016. Matching networks for one shot learning. Proceedings of the 30th International Conference on Neural Information Processing Systems

Numbered as in the paper

Cite this row

FSL — six-dimensional profile
D1: parametric-update
D2: task-specialization
D3: few-demonstrations, small-labeled
D4: ad-hoc-permanent, scheduled-permanent
D5: partial
D6: dl, fm
Source: arXiv:2608.06246
Compare with another technique Find in the explorer