Mastering the Bias-Variance Tradeoff in Machine Learning 📊

In this video, we learn about the second most important tradeoff in Machine Learning: the bias variance tradeoff. Although we are simply rephrasing the results that we got in the previous video, there are two main reasons for doing this:

1. This is the common way of looking at the approximation generalization tradeoff, even though in my opinion it should be the other way around
2. The more ways that you can independently verify your conclusion (a classic probablist/data science way of thinking) the stronger you should feel about it

Link to my notes on Introduction to Data Science: https://github.com/knathanieltucker/data-science-foundations

Try answering these comprehension questions to further grill in the concepts covered in this video:

1. Where does the randomness come in training an ML algorithm (say linear regression)?
2. Do we ever really know the bias of a model if you only have a sample from a population?
3. Does bias depend on your data sample?
4. Why does bias + variance = test error?
5. When you have a lot of data points are low bias models better than high bias ones?
6. Why does variance always seem to work against us?
7. What is the lowest bias model?