Posterior and MAP Derivation for Categorical Distribution in TensorFlow Probability

We put a Dirichlet prior on the Categorical's parameter vector. Now let's derive the Posterior and the Maximum A Posteriori Estimate (MAP). Here are the notes: https://raw.githubusercontent.com/Ceyron/machine-learning-and-simulation/main/english/essential_pmf_pdf/categorical_posterior_and_map.pdf

The Dirichlet Distribution is the conjugate prior to the Categorical. We use this fact to intuitively derive the posterior and its mode, the Maximum A Posterior (MAP) Estimate.

-------

📝 : Check out the GitHub Repository of the channel, where I upload all the handwritten notes and source-code files (contributions are very welcome): https://github.com/Ceyron/machine-learning-and-simulation

📢 : Follow me on LinkedIn or Twitter for updates on the channel and other cool Machine Learning & Simulation stuff: https://www.linkedin.com/in/felix-koehler and https://twitter.com/felix_m_koehler

💸 : If you want to support my work on the channel, you can become a Patreon here: https://www.patreon.com/MLsim

-------

Timestamps:
00:00 Introduction
00:50 Motivation
01:17 Repetition: The Categorical
01:56 Directed Graphical Model
03:32 The joint distribution
05:51 Bayes' Rules
06:35 Proportional Posterior
08:02 Plugging in Dirichlet & Categorical
09:09 Simplifying Proportional Posterior
13:09 Why Dirichlet is conjugate prior
13:49 "Posterior Likelihood"
14:06 Two Paths
14:52 Deriving the Posterior
17:39 MAP: Setup
18:12 MAP: Log-Posterior Likelihood
19:15 MAP: Lagrange Multiplier
20:51 MAP: Maximization
28:12 Discussing the MAP
19:19 MAP for the One-Hot Categorical
30:18 TFP: Create a dataset
32:00 TFP: n observations per state
32:33 TFP: Calculating the MLE
32:52 TFP: Calculating the MAP
34:39 TFP: MLE/MAP for corrupt dataset
37:10 TFP: Posterior Distributions
38:47 Outro