Dimension D3

Data Requirements

What data is needed?

21 categories. Other dimensions: D1 D2 D4 D5 D6

II. Synthetic / Programmatic Data

Category	Definition	Techniques
Constitution-Deriv. Constitution-Derived	Data generated iteratively based on an explicit set of textual guidelines or constitutional principles, in which a model critiques and revises its own outputs according to the specified normative rules. The constitutional text itself functions as the primary normative artifact encoding the behavioral constraints that propagate into the adapted model through the generated training data.	RLAIF/CAI 1 technique(s)
Environment-Deriv. Environment-Derived	Data generated through interaction with a physics engine, game simulator, formal environment, or through the model's own autoregressive self-generation acting as an opponent or world model. The simulation parameters, reward mechanics, and environmental transition dynamics define the statistical properties of the generated data; the learning signal is derived from outcome-based environmental feedback rather than human judgment.	Self-Play 1 technique(s)
Instr.-Resp. Corpus Instruction-Response Corpus	A heterogeneous collection of prompt-response pairs spanning many tasks and domains used to train general instruction-following capability. The defining characteristic is corpus breadth across task types rather than depth within any single task;	SFT 1 technique(s)
Teacher-Deriv. Teacher-Derived	Data generated by a more capable or aligned model acting as an annotator or oracle (teacher). The generating model's biases, capability limitations, and output distribution are inherited by the student through this data pathway, making provenance characterization of the teacher a prerequisite for characterizing the data.	KD 1 technique(s)

V. Minimal / Decentralized / Delegated Data

Category	Definition	Techniques
Decent. Decentralized	Private data distributed across multiple non-collocated clients or institutions, each processed locally without sharing raw records with a central coordinator. The statistical heterogeneity across shards—arising from non-independent and non-identically distributed local distributions—constitutes the primary technical challenge for any aggregation mechanism that attempts to synthesize a coherent global model from local signals.	FL 1 technique(s)
Param.-Only Parameter-Only	A data-free regime in which adaptation operates exclusively on the model's pre-trained parameter vectors through algebraic composition, interpolation, or negation—requiring no training examples, labeled or unlabeled, as input to the adaptation procedure. The entire adaptation is derived from arithmetic relationships among existing weight tensors.	Model Compression, Task Arith. & Model Merging 2 technique(s)
Pipe.-Dep. Pipeline-Dependent	A delegation regime in which the effective data requirements of the technique are fully determined by and inherited from the downstream training method that the technique augments. The pipeline technique itself imposes no independent data regime; all data characterization is subsumed by the consuming training procedure.	Curriculum Learning, Active Learning, Data Augmentation 3 technique(s)
Unlabeled Unlabeled	Unannotated data used for self-supervised or unsupervised training objectives, continuous pre-training, or distribution sampling for post-training calibration or activation steering. The absence of ground-truth labels defines this regime: the learning signal is derived from the statistical structure of the raw data—through masked prediction, contrastive objectives, or density estimation—rather than from explicit human annotation.	DA, Semi-SL, SSL / CPT, Model Compression, Long-Context Ext., MoE, Calibration, Activation Steering 8 technique(s)
Zero-Shot Zero-Shot	Natural-language task directives provided in the model's input context without any accompanying labeled examples or demonstrations, relying entirely on the model's pre-existing instruction-following and generalization capability. The data requirement is zero labeled instances: only the task specification itself is provided, and all behavioral conditioning is achieved through the model's previously learned representations.	PE, APO, Test-Time Compute Scaling 3 technique(s)

IV. External / Targeted Data

Category	Definition	Techniques
Ext. Corpus External Corpus	A curated external knowledge repository queried at inference time to retrieve contextually relevant passages that augment model outputs. The corpus is a pre-existing, structurally static reference artifact that is independent of the active user session and of the model's internal parameter state.	RAG, CE 2 technique(s)
Forget Set Spec. Forget Set Specification	A precisely delineated set of training samples, factual associations, behavioral capability traces, or user data records whose influence is to be erased from model behavior.	Machine Unlearning 1 technique(s)

I. Human-Annotated Data

Category	Definition	Techniques
Few Demo. Few Demonstrations	A minimal set of input-output demonstration examples provided directly within the model's inference-time context window to condition its behavior without any gradient-based optimization. These examples function as behavioral exemplars that the model's in-context learning mechanism generalizes from within the active session, leaving the model's parameters entirely unmodified.	FSL, APO, ICL 3 technique(s)
Large Labeled Large Labeled	A high-volume corpus of annotated input-output pairs providing the statistical coverage necessary to update all model parameters reliably without overfitting. This data regime is the defining requirement for methods where the unrestricted optimization landscape requires broad supervision to constrain the high-dimensional parameter search space.	Training, FT (full), Adversarial Training, DP-FT 4 technique(s)
Pref. Pairs Preference Pairs	A dataset of paired model outputs annotated by human raters to indicate relative behavioral preference according to quality, helpfulness, or safety criteria. Unlike task-labeled data that specifies a correct output, preference pairs encode comparative judgments between two candidate responses, making the comparative elicitation methodology—rather than absolute correctness—the defining characteristic of this data regime.	RLHF (LLM Alignment), DPO 2 technique(s)
Small Labeled Small Labeled	A limited collection of annotated task-specific instances sufficient for parameter-efficient adapter training or partial FT but insufficient for full-parameter optimization without catastrophic forgetting or overfitting. The small scale constrains the effective update surface and necessitates parameter-efficient or regularized training strategies that limit the degrees of freedom available to the optimizer.	FT (partial), PEFT (LoRA, adapters), ReFT, DA, FSL, LP, LEACE, Semi-SL, DP-FT, Long-Context Ext., MoE, Prompt Learning, Calibration, Knowledge Editing, Activation Steering 15 technique(s)
Task Dist. Task Distribution	A distribution over multiple heterogeneous tasks to expose the model to diverse adaptation episodes. The distributional coverage over task space—rather than depth of supervision on any single task—is the defining property; the model optimizes for initial parameters or learning procedures that generalize rapidly to novel tasks encountered at deployment.	Meta-Learning, MTL 2 technique(s)

VII. Multi-Modal Paired Provenance

Category	Definition	Techniques
Pair. Multimod. Paired Multimodal	Aligned cross-modal paired samples used to train cross-modal alignment or grounding objectives. Each training example carries independent provenance chains for each modality, and the cross-modal correspondence itself introduces additional alignment-specific data characteristics.	Cross-Modal Alignment, Modality-Spec. Adapters, Multimodal Instr. Tuning 3 technique(s)

III. Live / Production Data

Category	Definition	Techniques
Seq./Inc. Sequential / Incremental	A temporally ordered stream of training examples arriving in batches or continuously over the model's operational lifecycle, whose distributional properties evolve across time. Adaptation mechanisms operating on this data regime must integrate new information without discarding previously consolidated knowledge;	Retraining, CL, TIL, DIL 4 technique(s)
Test-Only Test-Only	The live, unlabeled test instances encountered during active inference, used as the sole input signal for transient self-supervised adaptation without access to any external training data or ground-truth labels. The model adapts to each test batch or individual instance in a session-local manner; adaptation does not persist beyond inference completion.	TTA 1 technique(s)
User Interact. User Interaction Data	Logged behavioral signals and implicit telemetry generated directly by end-users during active deployment used to drive personalization or implicit feedback-based adaptation. This regime is operationally distinct from curated labeled datasets and comparative preference pairs in that it is generated passively during production use rather than collected through an explicit annotation pipeline.	CE 1 technique(s)

VI. Verifier-Defined Ground Truth

Category	Definition	Techniques
Verif. Task Verifiable Task	Tasks for which correctness can be assessed by a deterministic programmatic verifier without requiring human preference annotation. Ground truth is established through binary or structured programmatic evaluation rather than subjective human judgment, shifting the epistemological basis of the training signal from comparative human preference to objective computational verification.	RLVR/GRPO 1 technique(s)

Category definitions verbatim from the paper's appendix