This post relates **moment generating functions** to the **Laplace transform** and to** exponential generating functions**. It also brings in connections to the **z-transform** and the **Fourier transform**.

Thanks to Brian Borchers who suggested the subject of this post in a comment on a previous post on transforms and convolutions.

## Moment generating functions

The **moment generating function** (MGF) of a random variable *X* is defined as the expected value of exp(*tX*). By the so-called rule of the unconscious statistician we have

where *f*_{X} is the probability density function of the random variable *X*. The function *M*_{X} is called the moment generating function of *X* because it’s *n*th derivative, evaluated at 0, gives the *n*th moment of *X*, i.e. the expected value of *X*^{n}.

## Laplace transforms

If we flip the sign on *t* in the integral above, we have the **two-sided Laplace transform** of *f*_{X}. That is, the moment generating function of *X* at *t* is the two-sided Laplace transform of *f*_{X} at −*t*. If the density function is zero for negative values, then the two-sided Laplace transform reduces to the more common (one-sided) Laplace transform.

## Exponential generating functions

Since the derivatives of *M*_{X} at zero are the moments of *X*, the power series for *M*_{X} is the **exponential generating function** for the moments. We have

where *m*_{n} is the *n*th moment of *X*.

## Other generating functions

This terminology needs a little explanation since we’re using “generating function” two or three different ways. The “moment generating function” is the function defined above and only appears in probability. In combinatorics, the (ordinary) generating function of a sequence is the power series whose coefficient of *x*^{n} is the *n*th term of the sequence. The exponential generating function is similar, except that each term is divided by *n*!. This is called the exponential generating series because it looks like the power series for the exponential function. Indeed, the exponential function is the exponential generating function for the sequence of all 1’s.

The equation above shows that *M*_{X} is the exponential generating function for *m*_{n} and the ordinary generating function for *m*_{n}/*n*!.

If a random variable *Y* is defined on the integers, then the (ordinary) generating function for the sequence Prob(*Y* = *n*) is called, naturally enough, the **probability generating function** for *Y.*

The **z-transform** of a sequence, common in electrical engineering, is the (ordinary) generating function of the sequence, but with *x* replaced with 1/*z*.

## Characteristic functions

The **characteristic function** of a random variable is a variation on the moment generating function. Rather than use the expected value of *tX*, it uses the expected value of *itX*. This means the characteristic function of a random variable is the **Fourier transform** of its density function.

Characteristic functions are easier to work with than moment generating functions. We haven’t talked about when moment generating functions exist, but it’s clear from the integral above that the right tail of the density has to go to zero faster than *e*^{−x}, which isn’t the case for fat-tailed distributions. That’s not a problem for the characteristic function because the Fourier transform exists for any density function. This is another example of how complex variables simplify problems.