Mastering the Bias-Variance Trade-Off in Machine Learning 📊

The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-theta-email-list
Want to work together? See here: https://truetheta.io/about/#want-to-work-together

Article on the topic: https://truetheta.io/concepts/machine-learning-and-other-topics/bias-variance-trade-off/

The Bias Variance Trade-Off is an essential perspective for developing models that will perform well out-of-sample. In fact, it's so important for modeling that most hyperparameters are designed to move you between the high bias-low variance and low bias-high variance ends of the spectrum. In this video, I explain what it says exactly, how it works intuitively, and how it's used typically.

SOCIAL MEDIA

LinkedIn : https://www.linkedin.com/in/dj-rich-90b91753/
Twitter : https://twitter.com/DuaneJRich

Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon: https://www.patreon.com/MutualInformation

TIMESTAMPS
0:00 The Importance and my Approach
0:46 The Bias Variance Trade off at a High Level
2:06 A Supervised Learning Regression Task and Our Goal
3:41 Evaluating a Learning Algorithm
5:39 The Bias Variance Decomposition
7:19 An Example True Function
8:07 An Example Learning Algorithm
9:41 Seeing the Bias Variance Trade Off
12:59 Final Comments

SOURCES

The explanation I've reviewed the most is in section 2.9 of [1]. Also, I found
Kilian Weinberger's excellent lecture [2] useful. If you'd like to learn how this concept generalizes beyond a regression model's square error, see [3]

[1] Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York: Springer.
[2] Weinberger, K. (2018). Machine Learning Lecture 19 Bias Variance Decomposition -Cornell CS4780 SP17, YouTube, https://www.youtube.com/watch?v=zUJbRO0Wavo&t=357s
[3] Tibshirani, R (1996), Bias, variance and prediction error for classification rules. Department of Preventive Medicine and Biostatistics and Department of Statistics, University of Toronto, Toronto, Canada