求泊松-二项分布（Poisson-binomial distribution）的MLE推导方法

Hey there! Since you already have a solid grasp on deriving MLEs for the Poisson and binomial distributions, let's extend that knowledge to the Poisson-binomial case. The core idea aligns with what you know, but there's a key twist that makes this one lack a neat closed-form solution—let's break it down step by step.

First, Recap the Poisson-Binomial Model #

The Poisson-binomial distribution describes the number of successes in n independent Bernoulli trials, where each trial has its own distinct success probability (p_1, p_2, ..., p_n). Unlike the binomial distribution (where all (p_i = p)), every trial can have a unique chance of success.

Its probability mass function (PMF) is:

P(X = k) = Σ [ product_{i ∈ S} p_i * product_{i ∉ S} (1-p_i) ]

where the sum runs over all subsets (S) of ({1,2,...,n}) with exactly (k) elements.

Setting Up the Likelihood & Log-Likelihood #

Suppose we have (m) independent observations (x_1, x_2, ..., x_m) (each (x_j) is the count of successes in (n) trials). The likelihood function is the product of the PMFs for each observation:

L(p₁, p₂, ..., pₙ) = Π₍ⱼ=1 to m₎ P(Xⱼ = xⱼ)

Taking the natural logarithm (to simplify differentiation), the log-likelihood becomes:

ℓ(p₁, p₂, ..., pₙ) = Σ₍ⱼ=1 to m₎ log(P(Xⱼ = xⱼ))

Deriving the MLE Equations #

To find the MLE, we take the partial derivative of the log-likelihood with respect to each (p_i), set it to 0, and solve for (p_i). Here's the critical trick for computing the PMF derivative:

For a single observation (x), rewrite the PMF by splitting it into two cases: whether the (i)-th trial was a success or not:

P(X = x) = p_i * P(X_{-i} = x-1) + (1-p_i) * P(X_{-i} = x)

where (X_{-i}) is the Poisson-binomial variable excluding the (i)-th trial (so it relies on (p_1,...,p_{i-1},p_{i+1},...,p_n)).

Now take the partial derivative of (P(X=x)) with respect to (p_i):

∂P(X=x)/∂p_i = P(X_{-i} = x-1) - P(X_{-i} = x)

The derivative of the log-PMF follows:

∂log(P(X=x))/∂p_i = [P(X_{-i} = x-1) - P(X_{-i} = x)] / P(X=x)

Applying this to the full log-likelihood, the partial derivative for each (p_i) is:

∂ℓ/∂p_i = Σ₍ⱼ=1 to m₎ [ P(X_{-i} = x_j - 1) - P(X_{-i} = x_j) ] / P(X = x_j) = 0

Why No Closed-Form Solution? #

Unlike the binomial distribution (where all (p_i = p), leading to a simple equation (Σx_j = nmp) with an explicit solution), the Poisson-binomial case has no closed-form MLE for individual (p_i)s. Each equation depends on the other (p_k)s (through (X_{-i})'s PMF), so we can't solve for (p_i) directly.

Instead, we use numerical optimization methods to find the (p_1,...,p_n) values that maximize the log-likelihood. Common approaches include:

• Newton-Raphson method: Iteratively updates (p_i)s using the gradient and Hessian of the log-likelihood.
• EM algorithm: Useful if you only observe total successes (not which specific trials succeeded).
• Gradient ascent: A simpler iterative method that follows the log-likelihood gradient until convergence.

A Quick Example to Illustrate #

Let's take (n=2) trials with (p_1, p_2), and (m) observations where:

• (c_0) are 0 successes,
• (c_1) are 1 success,
• (c_2) are 2 successes.

The log-likelihood is:

ℓ = c₀*log((1-p₁)(1-p₂)) + c₁*log(p₁(1-p₂) + p₂(1-p₁)) + c₂*log(p₁p₂)

Taking the partial derivative with respect to (p_1):

∂ℓ/∂p₁ = -c₀/(1-p₁) + c₁*( (1-p₂) - (1-p₁) ) / (p₁(1-p₂)+p₂(1-p₁)) + c₂/p₁ = 0

You can see this equation can't be rearranged to solve for (p_1) explicitly—it depends on (p_2), and vice versa. We'd need to solve these two equations numerically.

内容的提问来源于stack exchange，提问作者GOURISH GOUDAR