Sum of independent but differently distributed variables

It’s well known that a binomial random variable can be approximated by a Poisson random variable, and under what circumstances the approximation is particularly good. See, for example, this post.

A binomial random variable is the sum of iid (independent, identically distributed) Bernoulli random variables. But what if the Bernoulli random variables don’t have the same distribution. That is, suppose you’re counting the number of heads seen in flipping n coins, where each coin has a potentially different probability of coming up heads. Will a Poisson approximation still work?

This post will cite three theorems on the error in approximating a sum of n independent Bernoulli random variables, each with a different probability of success p_i. I’ll state each theorem and very briefly discuss its advantages. The theorems can be found in [1].

Setup

For i = 1, 2, 3, …, n let X_i be Bernoulli random variables with

Prob(X_i = 1) = p_i

and let X with no subscript be their sum:

X = X₁ + X₂ + X₃ + … + X_n

We want to approximate the distribution of X with a Poisson distribution with parameter λ. We will measure the error in the Poisson approximation by the maximum difference between the mass density function for X and the mass density function for a Poisson(λ) random variable.

Sum of p‘s

We consider two ways to choose λ. The first is

λ = p₁ + p₂ + p₃ + … + p_n.

For this choice we have two different theorems that give upper bounds on the approximation error. One says that the error is bounded by the sum of the squares of the p‘s

p₁² + p₂² + p₃² + … + p_n²

and the other says it is bounded by 9 times the maximum of the p‘s

9 max(p₁, p₂, p₃, …,  p_n).

The sum of squares bound will be smaller when n is small and the maximum bound will be smaller when n is large.

Sum of transformed p‘s

The second way to choose λ is

λ = λ₁ + λ₂ + λ₃ + … + λ_n

where

λ_i = -log(1 – p_i).

In this case the bound on the error is one half the sum of the squared λ’s:

(λ₁² + λ₂² + λ₃² + … + λ_n²)/2.

When p_i is small, λ_i ≈ p_i. In this case the error bound for the transformed Poisson approximation will be about half that of the one above.

Related posts

• Normal approximation to binomial
• Camp-Paulson approximation to binomial
• Relative error in normal approximations

[1] R. J. Serfling. Some Elementary Results on Poisson Approximation in a Sequence of Bernoulli Trials. SIAM Review, Vol. 20, No. 3 (July, 1978), pp. 567-579.