ICLR 2025: New Periodic Table in ML 🌐

- Paper Title: "I-Con: A Unifying Framework for Representation Learning"- Project Website: aka.ms/i-con
- Website: https://aka.ms/i-con
- Paper: https://arxiv.org/abs/2504.16929
- Code: https://github.com/ShadeAlsha/ICon
- MIT News Article: https://news.mit.edu/2025/machine-learning-periodic-table-could-fuel-ai-discovery-0423

This video introduces Information Contrastive Learning (I-Con) — a theoretical and empirical framework that unifies over 20 modern machine learning methods under a single information-theoretic equation. I-Con reveals that a wide range of techniques — including contrastive learning, clustering, dimensionality reduction, spectral methods, and supervised learning — can be viewed as minimizing an integrated KL divergence between learned and supervisory neighborhood distributions.

By recasting representation learning in this unified view, I-Con exposes deep structural connections across seemingly disparate approaches. It enables both rigorous theoretical analysis and the principled design of new loss functions. We demonstrate how this framework leads to state-of-the-art results in unsupervised image classification, including a +8% improvement on ImageNet-1K over prior methods.

Authors: Shaden Alshammari, John Hershey, Axel Feldmann, William T. Freeman, Mark Hamilton
Afflictions: MIT, Google, and Microsoft

#RepresentationLearning #ICLR2025 #UnsupervisedLearning #MachineLearning #ContrastiveLearning #Clustering