Consider the following Taylor series for sin(θ/7)
and the following two functions based on the series, one takes only the first non-zero term
def short_series(x):
return 0.14285714*x
and a second that three non-zero terms.
def long_series(x):
return 0.1425714*x - 4.85908649e-04*x**3 + 4.9582515e-07*x**5
Which is more accurate? Let’s make a couple plots plot to see.
First here are the results on the linear scale.
Note that the short series is far more accurate than the long series! The differences are more dramatic on the log scale. There you can see that you get more correct significant figures from the short series as the angle approaches zero.
What’s going on? Shouldn’t you get more accuracy from a longer Taylor series approximation?
Yes, but there’s an error in our code. The leading coefficient in long_series is wrong in the 4th decimal place. That small error in the most important term outweighs the benefit of adding more terms to the series. The simpler code, implemented correctly, is better than the more complicated code with a small error.
The moral of the story is to focus on the leading term. Until it’s right, the rest of the terms don’t matter.
Update: Based on some discussion after publishing this post, I think my comment about short_series being simpler misdirected attention from my main point. Here’s a variation on the same argument based on two equally complex functions.
def typo1(x):
return 0.1425714*x - 4.85908649e-04*x**3 + 4.9582515e-07*x**5
def typo2(x):
return 0.14285714*x - 5.85908649e-04*x**3 + 4.9582515e-07*x**5
Both functions have one typo, typo1 in the first non-zero coefficient and typo2 in the second non-zero coefficient. These two functions behave almost exactly like long_series and short_series respectively. A typo in the leading coefficient has a much bigger impact on accuracy than a typo in one of the other coefficients.
Related post: Life lessons from differential equations
“there’s an error in our code. The leading coefficient in long_series is wrong in the 4th decimal place”
Saying errors in coding results in output errors is a tautology and a sleight of hand in this post. You’re pretending the simple code is better because it’s simple, but it’s better because it’s coded correctly.
I’m not just saying that incorrect code produces incorrect results. That is a tautology.
Neither of these functions is exactly correct. They’re both approximations. And there are two sources of approximation: truncating the Taylor series, and representing real numbers as floating point numbers. In this particular example, the latter is more important.
On a separate note, simpler code is more likely to be correct, all other things being equal, because one’s attention is spread over a smaller error. If the function
short_serieshad a typo, there’s only one place to look, but there are three places to look inlong_series.I deliberately introduced a typo in the function
long_series, but ironically I also had an unintentional error in the second term while writing the post. I corrected it before publishing, but it hardly made any difference, which is consistent with the theme of the post. The error in the most important term drowned out the error in the second most important term.…. I feel like this is a really important idea, but maybe not the best way to present it? It feels very bait-and-switch this way.