The Evolution of Reinforcement Learning: From Matchbox Computers to Robots 🤖

A full history of Reinforcement Learning's development, from Mitchie's matchbox computer to modern robotic systems. Traces the evolution of key concepts through games and physical control problems, showing how simulation-trained skills transfer to reality through domain randomization. Explores the emergence of human-like behaviors in AI agents and raises profound questions about the relationship between actions and language. Examines cutting-edge developments in embodied AI, from Tesla's Optimus (Figure, Atlas) to OpenAI's dexterous manipulation, and considers the future of action prediction models inspired by large language models. A thought-provoking exploration of how robots develop physical intelligence and what this means for the future of AI. 2025 Turing award winner! Thanks to Jane Street for sponsoring this video. They are hiring people interested in ML! learn more about their work and open roles (and support me), visit their website: jane-st.co/ml Featuring insights from: Claude Shannon Arthur Samuel Gerald Tesauro Richard Sutton David Silver Deep Mind/Open AI etc. 00:00 - Introduction 00:32 - Learning Tic Tac Toe 02:00 - Learning Cart and pole 04:20 - Shannon & Chess 06:50 - Samuel's Checkers 09:25 - TD Gammon (Gerald Tesaruo) 11:00 - TD Learning 14:30 - Learning Atari (DQN) 17:28 - DIrect Policy Gradiant 19:40 - Domain Randomization