The product of two normal PDFs is proportional to a normal PDF. This is well known in Bayesian statistics because a normal likelihood times a normal prior gives a normal posterior. But because Bayesian applications don’t usually need to know the proportionality constant, it’s a little hard to find. I needed to calculate this constant, so I’m recording the result here for my future reference and for anyone else who might find it useful.
Denote the normal PDF by
Then the product of two normal PDFs is given by the equation
where
and
Note that the product of two normal random variables is not normal, but the product of their PDFs is proportional to the PDF of another normal.
I think it’s particularly elegant how the proportionality constant is expressed as a “normal”.
As is almost always the case, this all becomes unambiguously nicer if you work with variances instead of standard deviations. Better still, with reciprocal variances. If your means are m,n and your reciprocal variances are t,u then the new mean is (tm+un)/(t+u) — the weighted average of the means, weighted by the reciprocal variances — and the new reciprocal variance is t+u.
(It’s even better formally, but a bit too mysterious statistically, to work with the reciprocal variance and the mean times the reciprocal variance. Then these just add. That’s because a normal PDF is exp(polynomial(x)) and these are basically just the coefficients of x^2 and x.)
For multivariate normals, if A and B are the inverses of the covariance matrices and m,n the means — so that the PDFs are exp(-1/2 (x-m)^T A (x-m)) and similarly for B,n — then this generalizes nicely: the mean is (A+B)^-1 (Am+Bn) and the inverse covariance is A+B.
g: You probably know this, but to add some jargon for others: the reciprocal variance (AKA the precision) and the mean/variance are the “natural parameters” of the Gaussian when written as a member of the exponential family.
Multivariate generalizations of the results in this post can be found, for example, in these cribs:
Gaussian identities only: http://cs.nyu.edu/~roweis/notes/gaussid.pdf
Matrix Cookbook (much larger, contains a section on Gaussians): http://www2.imm.dtu.dk/pubdb/p.php?3274
Working with the inverse variance often arises in statistical estimation theory. The inverse variance is the Fisher Information of the true value. This works just like a quantity of information should. Given two normally distributed estimates of a parameter, we can find the combined Information by simply adding the Information from each of the individual estimates. The new mean is the information-weighted average of the individual means.
It is very straightforward and intuitive to think about normals in those terms.
can you describe how to obtain the mean and variance (or concentration) of circular convolution of two normal pdfS.
Also how to obtain the mean and variance (or concentration) of product of two raleigh pdfs.
I’m not sure what you mean by “circular” convolution, but the convolution of two PDFs is the PDF of the sum of the independent random variables. If X and Y are independent normals, E(X+Y) = E(X) + E(Y) and Var(X+Y) = Var(X) + Var(Y). I haven’t looked at the product of Rayleigh random variables.
Hi Thomas;
Thank you very much for this useful share. I had a related question (I’m a beginner in statistics btw, so sorry if my question may sound dumb):
1) Do you think it is still meaningful to use a tolerance interval (from engineering point of view) of “the mean minus/plus three sigmas” for this kind of distribution, too?
2) What about the mean and variance of “sums of products” of this kind? Can we assume that they will simply be the sums of the means and variances?
3) What would be the interpretation of tolerance intervals in the case of “sums of products”?
Regards,
Yilmaz
Thank you John, and G for your comment. Between you, you saved me a lot of work.
Oh, You’re assuming the same x. My work’s not done. My x1 is my m2. I assume the product is still proportional to a normal, but I haven’t proven it. Or what the mean is. Or if it’s not normal-proportional, at least the x that gives the highest product.
Some of the subscripts are reversed in the equation for the new mean. It should be μ = (μ_1/σ_2^2 + μ_2/σ_1^2) / (σ_1^2 + σ_2^2).
Tom: if you divide the numerator and denominator of your expression for μ by σ_1^2 σ_2^2 then I believe you’ll obtain the expression given in the post.
Because 1/(1/a + 1/b) = ab / (a+b) we can also refactor the formulas expressed with varriances to look a bit simpler as:
m = m1 v2 + m2 v1 / (v1 + v2)
v = v1 v2 / (v1 + v2)
This is probably the version Tom Lieber was thinking of when he commented above about the subscripts being reversed (they weren’t).