Time series analysis and digital signal processing are closely related. Unfortunately, the two fields use different terms to refer to the same things.

Suppose you have a sequence of inputs *x*[*n*] and a sequence of outputs *y*[*n*] for integers *n*.

## Moving average / FIR

If each output depends on a linear combination of a finite number of previous *inputs*

*y*[*n*] = *b*_{0} *x*[*n*] + *b*_{1} *x*[*n*-1] + … + *b*_{q} *x*[*n* – *q*]

then time series analysis would call this a **moving average** (**MA**) model of order *q*, provided *b*_{0} = 1. Note that this might not really be an average, i.e. the *b*‘s are not necessarily positive and don’t necessarily sum to 1.

Digital signal processing would call this a **finite impulse response** (**FIR**) filter of order *q*.

## Autoregressive / IIR

If each output depends on a linear combination of a finite number of previous *outputs*

*y*[*n*] = *a*_{1} *y*[*n*-1] + … + *a*_{p} *y*[*n* – *p*]

then time series analysis would call this an **autoregressive** (**AR**) model of order *p*.

Digital signal processing would call this an **infinite impulse response** (**IIR**) filter of order *p. *

Sometimes you’ll see the opposite sign convention on the *a*‘s.

## ARMA / IIR

If each output depends on a linear combination of a finite number of previous inputs *and* outputs

*y*[*n*] = *b*_{0} *x*[*n*] + *b*_{1} *x*[*n*-1] + … + *b*_{q} *x*[*n* – *q*] + *a*_{1} *y*[*n*-1] + … + *a*_{p} *y*[*n* – *p*]

then time series analysis would call this an **autoregressive moving average **(**ARMA**) model of order *(**p*, *q*), i.e. *p* AR terms and *q* MA terms.

Digital signal processing would call this an **infinite impulse response** (**IIR**) filter with *q* feedforward coefficients and *p* feedback coefficients. Also, as above, you may see the opposite sign convention on the *a*‘s.

## ARMA notation

Box and Jenkins use *a*‘s for input and *z*‘s for output. We’ll stick with *x*‘s and *y’*s to make the comparison to DSP easier.

Using the backward shift operator *B* that takes a sample at *n* to the sample at *n*-1, the ARMA system can be written

φ(*B*) *y*[*n*] = θ(*B*) *x*[*n*]

where φ and θ are polynomials

φ(*B*) = 1 – φ_{1}*B* – φ_{2}*B*² – … φ_{p}*B*^{p}

and

θ(*B*) = 1 – θ_{1}*B* – θ_{2}*B*² – … θ_{q}*B*^{q}.

## System function notation

In DSP, filters are described by their **system function**, the *z*-transform of the impulse response. In this notation (as in Oppenheim and Shafer, for example) we have

The φ_{k} in Box and Jenkins correspond to the *a*_{k} in Oppenheim and Schafer. The θ_{k} correspond to the (negative) *b*_{k}.

The system function *H*(*z*) corresponds to θ(1/*z*) / φ(1/*z*).

## Related

DSP and time series consulting

Hi John: Very nice. I’ve been teaching myself basic EE lately and there’s also a lot of overlap with econometrics. All the distributed lag models in econometreics have a linear system viewpoint. As you said, totally different terminology but it’s all quite similar and sometimes equivalent. All three communities ( time-series statistics, time-series econometrics and DSP ) contain so much overlap that I never knew about or understood until the last 3-4 years. Great article. Your have of order q for the IIR in the AR section when you meant p. All the best.

Cool article, I’m an EE learning econometrics. All this stuff is strangely similar.