Understanding Entropy in Data Science: A Clear and Simple Explanation 📊

Entropy is a fundamental concept in Data Science because it shows up all over the place - from Decision Trees, to similarity metrics, to state of the art dimension reduction algorithms. It's also surprisingly simple, but often poorly explained. Traditionally the equation is presented with the expectation that you memorize it without thoroughly understanding what it means and where it came from. This video takes a very different approach by showing you, step-by-step, where this simple equation comes from, making it easy to remember (and derive), understand and explain to your friends at parties.

For a complete index of all the StatQuest videos, check out:
https://statquest.org/video-index/

If you'd like to support StatQuest, please consider...

Patreon: https://www.patreon.com/statquest
...or...
YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...buying one of my books, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
https://statquest.org/statquest-store/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
https://twitter.com/joshuastarmer

0:00 Awesome song and introduction
1:28 Introduction to surprise
4:34 Equation for surprise
6:09 Calculating surprise for a series of events
9:35 Entropy defined for a coin
10:45 Entropy is the expected value of surprise
11:41 The entropy equation
13:01 Entropy in action!!!

#StatQuest #Entropy