Knowledge Modification and Activation-Based Adaptation
Machine Unlearning
In brief — site editorial, not from the paper
Removes specific data or knowledge from a trained model, often to meet privacy obligations, aiming to leave it behaving as though it had never seen the material.
Definition
Machine Unlearning selectively erases learned knowledge, behavioral patterns, or training-data influence (Remove Knowledge) from a deployed model without retraining from scratch, addressing privacy deletion requirements, copyright compliance, and safety-critical knowledge removal (Privacy Preservation) [36]. Compared to LEACE, which removes a concept's linear influence from activations by modifying the weight matrix's projection geometry (Parameter-Only, applied to a concept direction), Machine Unlearning targets a broader knowledge construct—facts, training data points, behavioral patterns. Its D1 (Mechanism) coordinate spans two distinct mechanistic profiles that must be rigorously distinguished within system configuration management and regression tracking. Gradient-based approaches apply gradient ascent on the target forget set (Forget Set Specification) to mathematically reverse the historical learning signal, operating as Parametric Update with a Partial scope. Conversely, task-vector negation approaches algebraically subtract a localized task vector from the base model's weights to structurally strip out a target capability, operating as Parameter Composition with Fused Composition—mechanistically identical to Task Arithmetic executed with a negation operator. Consequently, they must be governed by the exact same change-control validation logic and interference testing suites. Adversarial membership inference attacks, targeted behavioral probes, and jailbreak-style elicitation of the supposedly forgotten knowledge constitute some available verification tools, and none provides a formal completeness guarantee; any system deploying Machine Unlearning as a compliance mechanism must document this verification gap explicitly and specify the probe suite used to assess forgetting fidelity [93].
Verbatim from the paper — Knowledge Modification and Activation-Based Adaptation
Related techniques
- hybrid Task Arith. & Model Merging — Task-vector-negation unlearning is mechanistically identical to task arithmetic
mechanistically identical to Task Arithmetic executed with a negation operator
Appendix C, Knowledge Modification and Activation-Based Adaptation
Nearest profiles
Knowledge Editing (0.50), LEACE (0.50), DP-FT (0.53), FT (partial) (0.58), PEFT (LoRA, adapters) (0.61)
Computed from the taxonomy data (Gower distance over all six dimensions)
References
- [36] Yinzhi Cao, and Junfeng Yang 2015. Towards Making Systems Forget with Machine Unlearning. 2015 IEEE Symposium on Security and Privacy
- [93] Shengyuan Hu, Yiwei Fu, Steven Wu, and Virginia Smith 2024. Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning.
Numbered as in the paper
Cite this row
Machine Unlearning — six-dimensional profile D1: parameter-composition, parametric-update D2: privacy-preservation, remove-knowledge D3: forget-set-specification D4: ad-hoc-permanent D5: fused-composition, partial D6: dl, fm, llm, mllm Source: arXiv:2608.06246Compare with another technique Find in the explorer