Here’s a strange detail of floating point arithmetic: computers have two versions of 0: positive zero and negative zero. Most of the time the distinction between +0 and -0 doesn’t matter, but once in a while signed versions of zero come in handy.
If a positive quantity underflows to zero, it becomes +0. And if a negative quantity underflows to zero, it becomes -0. You could think of +0 (respectively, -0) as the bit pattern for a positive (negative) number too small to represent.
The IEEE floating point standard says 1/+0 should be +infinity and 1/-0 should be -infinity. This makes sense if you interpret +/- 0 as the ghost of a number that underflowed leaving behind only its sign. The reciprocal of a positive (negative) number too small to represent is a positive (negative) number too large to represent.
To demonstrate this, run the following C code.
int main()
{
double x = 1e-200;
double y = 1e-200 * x;
printf("Reciprocal of +0: %g\n", 1/y);
y = -1e-200*x;
printf("Reciprocal of -0: %g\n", 1/y);
}
On Linux with gcc the output is
Reciprocal of +0: inf Reciprocal of -0: -inf
Windows with Visual C++ returns the same output except Windows prints infinity as 1#INF rather than inf. (See these notes for more on how Windows and Linux handle floating point exceptions.)
There is something, however, about signed zeros and exceptions that doesn’t make sense to me. The aptly named report “What Every Computer Scientist Should Know About Floating Point Arithmetic” has the following to say about signed zeros.
In IEEE arithmetic, it is natural to define log 0 = -∞ and log x to be a
NaNwhen x < 0. Suppose that x represents a small negative number that has underflowed to zero. Thanks to signed zero, x will be negative, so log can return aNaN. However, if there were no signed zero, the log function could not distinguish an underflowed negative number from 0, and would therefore have to return -∞.
This implies log(-0) should be NaN and log(+0) should be -∞. That makes sense, but that’s not what happens in practice. The log function returns -∞ for both +0 and -0.
I ran the following code on Linux and Windows.
int main()
{
double x = 1e-200;
double y = 1e-200 * x;
printf("Log of +0: %g\n", log(y));
y = -1e-200*x;
printf("Log of -0: %g\n", log(y));
}
On Linux, the code prints
Log of +0: -inf Log of -0: -inf
The results were the same on Windows, except for the way Windows displays infinities.
Related link:
IEEE floating point exceptions in C++
Anatomy of a floating point number
Math library functions that seem unnecessary
{ 4 trackbacks }
{ 9 comments… read them below or add one }
Jonas Elfström 06.15.10 at 08:56
I dabbled with infinites in C# and Ruby a while back. Now it seems my unnecessary bold statement that Ruby is IEEE 754-1985 compliant could be wrong, because in Ruby (1.8.7) both log(y) above sadly causes a “Numerical result out of range” error.
C# outputs the same as C++
double x = 1e-200;
double y = 1e-200 * x;
System.Console.WriteLine("Log of +0: {0}\n", Math.Log(y));
y = -1e-200 * x;
System.Console.WriteLine("Log of -0: {0}\n", Math.Log(y));
Log of +0: -INF
Log of -0: -INF
alfC 06.15.10 at 16:36
Ironically both -0. and +0. are equal to 0. and equal among them selves.
ie. y==0. is true in both cases!
if particular -0. in no less than 0.
y<0. is false
regarding log, maybe it makes sense define log(-0) as -inf, from the point of view of the algorithm of log. which probably ignores the sign bit after checking that the input is no less than 0.
maybe interpreting the IEEE -0. as a limit is wrong, to me it is more like a partial nan.
Anonymous22.1 06.15.10 at 17:19
Thanks for your investigations, Captain. Is it possible that most languages are not IEEE compliant? Did you ever get to grips with William Kahan’s “Why JAVA’s Floating Point Hurts Everyone Everywhere?” enough to know whether the criticisms relate to other languages too?
Joe User 06.15.10 at 18:02
Um, you should look up one’s compliment and two’s compliment
http://en.wikipedia.org/wiki/Ones_compliment
http://en.wikipedia.org/wiki/Twos_Compliment
whether or not you have -0 and +0 is usually dependent on how your hardware represents numbers. I wouldn’t expect -0 or +0 to be a valid numbers on x86 architecture, since I believe it’s two’s compliment.
Zane 06.15.10 at 20:07
Joe User, he’s talking about floating point representations of numbers, which is what he means by “IEEE floating point standard” and using “double” as the types of his variables. Modern x86 processors have floating-point units (aka “math coprocessors”) that operate on floating-point numbers — processors without floating-point units must implement all the wacky IEEE754 operations in software rather than in hardware.
http://en.wikipedia.org/wiki/IEEE_754-1985
http://en.wikipedia.org/wiki/IEEE_754-2008
You are correct about x86’s using two’s complement for integers, though.
Vinoth 06.16.10 at 00:57
I tested this on my Nokia phone (arm processor). The results are Inf and -Inf
Ted 06.17.10 at 22:32
I think Goldberg’s essay goes a little astray on the logarithm thing. The same reasoning would lead to a conclusion that sqrt(-0) should be NaN, which I don’t think anyone would advocate. I’m not expert on IEEE arithmetic (which doesn’t specify log(0) in any case, unless that has changed since I last paid attention) but as I understand it, IEEE has a signed zero as a result of a lot of thought about branch cuts for elementary functions of a complex variable, so thinking about it in terms of functions of a real variable doesn’t work very well.
For nonzero z, write z = r*exp(i*t), where r>0 and i is the imaginary unit. The choice of t is indeterminate, but we will have log(z) = log(r) + i*t for one of the possible values of t. As z goes to zero, log(z) goes to minus infinity no matter what the direction of approach. This suggests that log(-0) and log(0) should both evaluate to minus infinity. They should also raise the divide by zero exception, for the reasons given by Kahan in his notes on IEEE 754 (cf. page 10).
The real story with signed zero and the logarithm has to do with discontinuity across the negative real axis with the usual principal value, where you get an extra tiny slice of continuity by a definition like log(-1+0i) = pi and log(-1-0i) = minus pi.
leek 06.24.10 at 19:56
Actually sqrt(-0) = -0.
IEEE 754: “Except that squareRoot(–0) shall be –0, every valid squareRoot shall have a positive sign.”
I remember when Fortran 95 added the distinction between +0 and -0 to the Fortran language as an option. I had to change a compiler and libraries. I added a command-line option which allowed +0 and -0 to either be distinguished or not . The only places in the whole compiler/library chain which were affected, were the I/O routines which needed to be able to write +/- 0 and read it back in unchanged, and the SIGN(A,B) intrinsic, which combines B’s sign with A’s mantissa/exponent.
Fortran 77 and 90 mandated that -0 == +0 in all behaviors, so if you wrote -0 out, it would be written the same as +0, and if you read it back in it would be read in as +0, and if you used SIGN(A,-0) it would return ABS(A), just like SIGN(A,+0) would.
Fortran 95 changed this, allowing signed 0 distinctions. However, -0 == +0 still (-0 and +0 compare equal).
If you want to know how special functions should work with non-normal IEEE values, see this draft of the C99 standard, Annex F. It was mostly authored by a former colleague of mine, Jim Thomas. It even covers complex numbers.
Lance Walton 02.17.11 at 14:39
-0 and +0 exist in mathematics too.
Consider the Laplace Transform when applied to, for example, the Dirac delta function. In this case the lower bound is -0, and it’s well defined in terms of a limit.