Negative Data Augmentation in Deep Learning

This video explains Negative Data Augmentation, a strategy for using label-corrupting, rather than label-preserving transformations in Deep Learning. The authors test this framework for training GANs and for Contrastive Learning such as CPC and MoCo. I think this is a really exciting direction for Data Augmentation and overcoming the challenge of learning from limited labeled data, I hope you find this video useful!

Content Links:
Negative Data Aug (Paper): https://arxiv.org/pdf/2102.05113.pdf
Self-Supervised Learning: The Dark Matter of Intelligence: https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence
Learning the difference that makes a difference: https://arxiv.org/abs/1909.12434

Chapters
0:00 Beginning
0:55 Semantically-Preserving Transformations
1:44 OOD Augmentations
3:25 NDA Strategy
6:00 Over-Generalization
7:18 Integration in GANs
8:00 Integration in Contrastive Learning
8:40 GAN Results
11:00 Contrastive Learning Results
12:18 Dark Matter - Energy-Based Learning
13:30 The Diff that makes a Diff