Understanding Fisher Information in ML 📊

The machine learning consultancy: https://truetheta.io
Join my email list to get educational and useful articles (and nothing else!): https://mailchi.mp/truetheta/true-theta-email-list
Want to work together? See here: https://truetheta.io/about/#want-to-work-together

Article on the topic: https://truetheta.io/concepts/machine-learning-and-other-topics/the-fisher-information/

The Fisher Information quantifies how well an observation of a random variable locates a parameter value. It's an essential tool for measure parameter uncertainty, a problem that repeats itself throughout machine learning and statistics. In this video, I explain the Fisher Information rigorously and visually, starting in the one dimensional case and ending in the general case.

SOCIAL MEDIA

LinkedIn : https://www.linkedin.com/in/dj-rich-90b91753/
Twitter : https://twitter.com/DuaneJRich

Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon: https://www.patreon.com/MutualInformation

Sources and Learning More

[1] provides a complete and deep explanation of the Fisher Information. It's captures the abstract/general perspective while making the idea concrete with examples. As is typically the case, the wikipedia article [2] was helpful. Also, section 8.2.2 of [3] explains the use of a theorem on the asymptotic normality of the MLE via the Fisher Information, which I didn't cover here, but certainly informed how I think it connects to parameter uncertainty.

[1] Ly A., Marsman M., Verhagen J., Grasman R., Wagermarkers E.J., (2017), A Tutorial on the Fisher Information, Department of Psychological Methods, University of Amsterdam, The Netherlands
[2] Fisher information, Wikipedia, https://en.wikipedia.org/wiki/Fisher_information
[3] Hastie, T., Tibshirani, R., & Friedman, J. H. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2nd ed. New York: Springer.