Skip to content

Data-Centric and Privacy-Preserving Methods

Data Augmentation

In brief — site editorial, not from the paper

Expands the training set with transformed copies of existing data, improving robustness without collecting anything new.

Definition

Data Augmentation increases training data diversity through synthetic transformations of existing examples, without acquiring new labeled data [62, 181]. Its Pipeline-Mediated classification reflects its structural position: augmentation operates strictly upstream of the gradient loop, modifying training inputs before they reach the optimizer—it executes no forward pass on the model and computes no gradients. This upstream positioning means the technique shares Pipeline-Dependent and Pipeline-Dependent coordinates with all other boundary-extension methods (Curriculum Learning, Active Learning, Semi-SL), reflecting that the D4 (Persistence) and D5 (Scope) of any resulting model change are entirely determined by the downstream training method that consumes the augmented data. The compound Robustness, Task Specialization profile reveals a structurally important internal distinction within augmentation: standard transformations (flips, crops, synonym substitution) diversify the input distribution to improve generalization (Task Specialization), while adversarially constructed perturbations specifically target distributional robustness (Robustness) [195]. The latter variant overlaps in goal with Adversarial Training, but differs critically on D1 (Mechanism): adversarial training generates perturbations dynamically within the gradient loop (dParametric Update with adversarially shaped gradients), while adversarial augmentation applies fixed perturbation transforms as a static pre-processing step (Pipeline-Mediated)—an operationally meaningful distinction because the dynamic coupling in adversarial training produces adaptive perturbations that scale with the model's current vulnerability, while static augmentation perturbations cannot. The augmentation policy—comprising the exact set of transformation functions, their mathematical parameterizations, hyperparameter bounds, and random seed configurations—constitutes a component of the training data specification and must be version-controlled and documented as an immutable dependency of the training event; a change to the augmentation policy redefines the statistical distribution the model optimizes against and requires a new data-provenance record.

Verbatim from the paper — Data-Centric and Privacy-Preserving Methods

Nearest profiles

Curriculum Learning (0.08), Active Learning (0.11), Semi-SL (0.25), FT (full) (0.78), MTL (0.78)

Computed from the taxonomy data (Gower distance over all six dimensions)

References

  1. [62] Steven Y. Feng et al. 2021. A Survey of Data Augmentation Approaches for NLP. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021
  2. [181] Connor Shorten, and Taghi M. Khoshgoftaar 2019. A survey on Image Data Augmentation for Deep Learning. J Big Data
  3. [195] Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry 2018. Robustness May Be at Odds with Accuracy.

Numbered as in the paper

Cite this row

Data Augmentation — six-dimensional profile
D1: pipeline-mediated
D2: robustness, task-specialization
D3: pipeline-dependent
D4: pipeline-dependent
D5: pipeline-dependent
D6: dl, fm, llm, ml, mllm
Source: arXiv:2608.06246
Compare with another technique Find in the explorer