MoE

MoE Adaptation introduces sparse gating over a population of expert sub-networks or integrating novel expert sub-networks into an existing MoE, enabling a model to route each input to the subset of parameters most relevant to that input's characteristics—achieving scalable multi-task specialization without proportionally scaling inference cost ^{[56, 157, 176]}. This structural distinction necessitates a compound D1 (Mechanism) profile: the topological addition of a new expert constitutes Architectural Modification—physically restructuring the computational graph—while the subsequent tuning of expert weights or the central routing mechanism constitutes Parametric Update. This graph-level intervention is what mandates Modular: the model is a collection of individually addressable expert modules coordinated by a router—each expert constituting an independently auditable behavioral unit whose activation patterns can be tracked per input. Compared to PEFT, which adds lightweight parameter overlays (adapters, LoRA) to a frozen base with Modular, Partial, MoE Adaptation adds structural routing logic that fundamentally changes the model's computational topology—adapter modules are additive overlays; expert modules are gated substitutes for dense computation. The compound D2 (Goal) profile—Capability Extension, Computational Efficiency—reflects a unique operational property: MoE adaptation is the only technique in the taxonomy that simultaneously scales systemic capacity while explicitly preserving fixed inference latency bounds. The Architectural Modification classification has stringent system-engineering implications in the taxonomy: a new model topology is being registered, not a parametric override of an existing one—requiring a complete re-specification of the model's computational graph and expert routing logic audit trail as components of the system configuration record, since any change to router architecture or gating threshold alters which computation paths are activated for which inputs.

MoE adaptation. Straddles architecture and adaptation; the row captures the adaptation act (training routers, adding experts) rather than the architectural pattern. RLVR vs.\ RLHF/DPO ( vs.\ ). All share Parametric Update but differ on D2 (Reasoning vs. Alignment, Safety) and D3 (Verifiable Task vs. Preference Pairs). RLVR and DPO share Partial, Whole-Model, while RLHF adds Surrogate Model. The boundary blurs when RLVR induces alignment-like properties or when RLHF pipelines incorporate verifiable components; in production, both are often applied sequentially to the same base model.