I was watching one of Brian Douglas’ videos on control theory (Discrete Control #5) and ran into a simple derivation of an approximation I presented earlier.

Back in April I wrote several post on simple approximations for log, exp, etc. In this post I gave an approximation for the exponential function:

The control theory video arrives at the same approximation as follows.

As I believe I’ve suggested before here, in a derivation like the one above, where you have mostly equalities and one or two approximations, pay special attention to the approximation steps. The approximation step above uses a first order Taylor approximation in the numerator and denominator.

The plot below shows that the approximation above (the bilinear approximation) is more accurate than doing a single Taylor approximation, approximating exp(*x*) by 1 + *x* (linear approximation).

Here’s a plot focusing on the error in the bilinear and linear approximations.

The bilinear approximation is hard to tell from 0 in the plot above for *x* up to 0.5.

The derivation above is simple, but why is the result so good? An explanation in terms of Padé approximation is given here.

Not math, but poetically: perhaps alternating “x” signs on the otherwise similar bilinear terms drive error in counter-balancing directions, cancelling out first order error terms.

If so, maybe this could be generalized, and splitting a function into even/odd symmetric parts (x —> -x) before approximation has similar value.

A more fair comparison might be to a 2nd order Taylor polynomial, since both have 3 degrees of freedom.

Here is a “derivation” that is not black magic. Using some plain Tex notation:

The exponential is the solution of the initial value problem

u'(x) = u(x), u(0) = 1.

We can approximate the solution with the trapezoidal formula

u(x) = u(0) + \int_0^x u'(x) dx \approx u(0) + {x \over 2} (u'(0) + u'(x))

so that

u(x) \approx u(0) + {x \over 2} (u(0) + u(x)) = 1 + {x \over 2} (1 + u(x)) .

Solve for u(x) and we have

u(x) \approx {1 + x/2 \over 1 – x/2},

which is the desired approximation.

@Symmetry of Approximation Error.

By (my) intuition the above derivation seemed to enforce

a rather symmetric approximation error.

The absolute error, however, is quite asymmetric.

But both the relative error

(exp(x) – (x+2)/(x-2))/exp(x)

and the absolute log scale error

x – ln((x+2)/(x-2))

are very symmetric and nearly coincide over [-1,1].

Very interesting blog, John.

I’d just add that linear and bilinear approximation have the same order $O(x^2)$ despite the bilinear formula containing a quadratic term $xy$. Therefore, the error might be smaller if the latter is used, but the order of accuracy remains the same.