Maximum Likelihood Estimation from Scratch: From Coin Flips to Gaussians

You've collected data and you have a model in mind — maybe a Gaussian, maybe a coin flip. But the model has parameters, and you need to find the values that best explain what you observed. How? Maximum Likelihood Estimation (MLE) answers this with a deceptively simple idea: choose the parameters that make your observed data most probable . By the end of this post, you'll implement MLE from scratch for three distributions, understand why we always work with log-likelihoods, and see how MLE connects to more advanced algorithms like EM . Quick Win: Estimate a Coin's Bias Let's start with the simplest possible case. You flip a coin 100 times and get 73 heads. What's the coin's bias? import numpy as np import matplotlib.pyplot as plt # Observed data: 73 heads out of 100 flips n_heads = 73 n_tails = 27 n_total = n_heads + n_tails # Compute likelihood for every possible bias value theta_values = np . linspace ( 0.01 , 0.99 , 200 ) likelihoods = theta_values ** n_heads * ( 1 - theta_values ) *