Comments on: Floating point numbers are a leaky abstraction

By: Will Dwinnell

Will Dwinnell — Thu, 28 Jan 2010 11:21:02 +0000

Tom Ochs wrote a series of articles about exactly this sort of thing in “Computer Language” magazine some years ago.

By: Tomas

Tomas — Wed, 20 Jan 2010 16:01:26 +0000

And of course Kevin did talk about it first, my oversight.

By: John

John — Wed, 20 Jan 2010 02:35:06 +0000

Tomas: I agree that the "move to the logspace" trick is very important. I talk about it here in the context of computing binomial probabilities.

By: Tomas

Tomas — Wed, 20 Jan 2010 02:27:27 +0000

I’m surprised to not see the “move to the logspace” basic trick… which is to compute exp(\sum \log x_i) in place of \prod(x_i). Works great for probabilities – they tend to be from some distribution and then central limit works for you…
Also normalizing small (or large) numbers to sum up to, say, 1.0 is better accomplished by first subtracting the max of all the logarithms.

By: Don’t invert that matrix — The Endeavour

Don’t invert that matrix — The Endeavour — Tue, 19 Jan 2010 14:48:46 +0000

[...] Floating point numbers are a leaky abstraction ? X [...]

By: links for 2009-04-13 | Yostivanich.com

links for 2009-04-13 | Yostivanich.com — Mon, 20 Apr 2009 02:17:05 +0000

[...] Floating point numbers are a leaky abstraction — The Endeavour Good remainder of some of the very real problems relying upon an abstracted system for accuracy. (tags: computer computerscience technology programming mathematics precision accuracy) [...]

By: John

John — Sun, 19 Apr 2009 18:40:30 +0000

Jack: Every floating point number does represent a rational number, so I could see your point. But I’d still say that floating point numbers are trying to approximate real numbers. Arithmetic operations are designed to be approximately correct in the analytical sense of the real line, not exactly correct in the algebraic sense of a quotient field. I think fixed-point numbers are a better model for rational numbers.

By: Jack

Jack — Sun, 19 Apr 2009 18:16:04 +0000

I disagree with your idea that floating-point is supposed to represent “real numbers”. They represent rational numbers.

If you want to approximate reals with rationals, that’s your choice, but you should at least understand the difference, and why you can get into trouble thereby.

By: Math Teachers at Play #5 « Let’s Play Math!

Math Teachers at Play #5 « Let’s Play Math! — Fri, 17 Apr 2009 11:16:58 +0000

[...] Cook presents Floating point numbers are a leaky abstraction. This is the first of two blog posts about how computer arithmetic works and what problems to look [...]

By: John

John — Fri, 17 Apr 2009 02:48:26 +0000

David Eisner: I updated the links to point to an updated version of Goldberg’s paper. Thanks for pointing out the error.

By: EastwoodDC

EastwoodDC — Thu, 16 Apr 2009 04:40:52 +0000

This also ties in with your post a while back on accurately calculation the variance.

By: John Cowan

John Cowan — Tue, 14 Apr 2009 01:28:41 +0000

Not to mention the problems that arise because we think in decimal but compute in binary. Does 0.1 + 0.2 = 0.3? No, not in binary floats, it doesn’t.

By: David Eisner

David Eisner — Mon, 13 Apr 2009 20:29:10 +0000

Did anybody else notice the error in Figure D-1 of David Goldberg’s paper (http://docs.sun.com/source/806-3568/ncg_goldberg.html#1374) ? The “0″ should be at the left of the number line, not at the hash mark corresponding to 1/2.

By: Kevin Peterson

Kevin Peterson — Mon, 13 Apr 2009 06:43:06 +0000

Don’t forget a Bayesian classifier — it involves multiplying large numbers of small probabilities, then normalizing them. After getting strange results, and switching to BigDecimal, I learned that the usual solution is to do all calculations in log space.

By: John Moeller

John Moeller — Mon, 06 Apr 2009 17:51:11 +0000

Nice post. This is a huge problem in sums too, especially when the addends are different by many orders of magnitude.