Alignment, Reasoning, and Trustworthiness
LEACE
In brief — site editorial, not from the paper
Surgically removes a concept's linear influence from the model's representations — used for fairness work — with a closed-form edit rather than training.
Definition
LEACE targets Fairness through a mechanistically novel D1 (Mechanism) pathway: rather than employing iterative GD, it operates via Parameter Composition—computing a covariance-based orthogonal projection from a Small Labeled and algebraically projecting the model's weight matrices into the null space of a target concept's linear representation [24], a parameter composition operation analogous to Task Arithmetic but operating on concept subspaces rather than task capability directions. This closed-form linear algebra operation bypasses the computational footprint of traditional parameter-updating methods entirely, producing a Ad-hoc Permanent, Partial modification at negligible training cost. Compared to LP, which trains a probe head on top of a frozen base to expose linear concept encodings, LEACE inverts the operation: it takes the same linear concept subspace identified by probing and algebraically projects it out of the base model's weight matrices—making LP and LEACE complementary, with LP as the diagnostic step and LEACE as the corrective intervention targeting the same identified representation. The Ad-hoc Permanent Permanent coordinate reflects that LEACE produces an irreversible, event-driven modification at negligible compute cost—unlike gradient-based fairness interventions that require training runs, LEACE can be applied as a post-hoc weight surgery. The Partial scope is bounded by the targeted layers: only the weight matrices encoding the concept's linear subspace are projected; all other parameters remain unchanged. Its operational and validation significance lies in the deterministic auditability of the intervention: because the modification is derived analytically from a strict concept specification, it is fully reproducible and mathematically characterizable; the concept subspace, projection matrix, and labeled probe dataset constitute a complete audit trail for the fairness intervention—validation requires verifying that the modified representation is linearly indecipherable for the target concept while baseline task performance is preserved, both of which are statistically testable properties.
Verbatim from the paper — Alignment, Reasoning, and Trustworthiness
Nearest profiles
Knowledge Editing (0.33), FT (partial) (0.42), Prompt Learning (0.46), Machine Unlearning (0.50), PEFT (LoRA, adapters) (0.50)
Computed from the taxonomy data (Gower distance over all six dimensions)
References
- [24] Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman 2023. LEACE: Perfect linear concept erasure in closed form. Advances in Neural Information Processing Systems
Numbered as in the paper
Cite this row
LEACE — six-dimensional profile D1: parameter-composition D2: fairness D3: small-labeled D4: ad-hoc-permanent D5: partial D6: dl, fm, llm, mllm Source: arXiv:2608.06246Compare with another technique Find in the explorer