Negative Data Augmentation in Deep Learning

This video explains Negative Data Augmentation, a strategy for using label-corrupting, rather than label-preserving transformations in Deep Learning. The authors test this framework for training GANs and for Contrastive Learning such as CPC and MoCo. I think this is a really exciting direction for Data Augmentation and overcoming the challenge of learning from limited labeled data, I hope you find this video useful! Content Links: Negative Data Aug (Paper): https://arxiv.org/pdf/2102.05113.pdf Self-Supervised Learning: The Dark Matter of Intelligence: https://ai.facebook.com/blog/self-supervised-learning-the-dark-matter-of-intelligence Learning the difference that makes a difference: https://arxiv.org/abs/1909.12434 Chapters 0:00 Beginning 0:55 Semantically-Preserving Transformations 1:44 OOD Augmentations 3:25 NDA Strategy 6:00 Over-Generalization 7:18 Integration in GANs 8:00 Integration in Contrastive Learning 8:40 GAN Results 11:00 Contrastive Learning Results 12:18 Dark Matter - Energy-Based Learning 13:30 The Diff that makes a Diff