Here’s an interesting problem that came out of a logistic regression application. The input variable was between 0 and 1, and someone asked when and where the logistic transformation

*f*(*x*) = 1/(1 + exp(*a* + *bx*))

has a **fixed point**, i.e. *f*(*x*) = *x*.

So given logistic regression parameters *a* and *b*, when does the logistic curve given by *y* = *f*(*x*) cross the line *y* = *x*? Do they always cross? Can they cross twice?

There’s always at least one solution. Because *f*(*x*) is strictly between 0 and 1, the function

*g*(*x*) = *f*(*x*) – *x*

is positive at 0 and negative at 1, and by the intermediate value theorem g(*x*) must be zero for some *x* between 0 and 1.

Sometimes *f* has only one fixed point. It may have two or three fixed points, as demonstrated in the graph below. The case of two fixed points is unstable: the logistic curve is tangent to the line *y* = *x* at one point, and a tiny change would turn this tangent point into either no crossing or two crossings.

If |*b*| < 1, then you can show that the function *f* is a **contraction map** on [0, 1]. In that case there is a unique solution to *f*(*x*) = *x*, and you can find it by starting with an arbitrary value for *x* and repeatedly applying *f* to it. For example, if *a*= 1 and *b* = 0.8 and we start with *x*= 0, after applying *f* ten times we get *x* = *f*(*x*) = 0.233790157.

There are a couple questions left to finish this up. How can you tell from *a* and *b* how many fixed points there will be? The condition |*b*| < 1 is sufficient for *f* to be a contraction map on [0, 1]. Can you find necessary and sufficient conditions?

**Related post**: Sensitivity of logistic regression prediction

The second question, at least, is reasonably easy to answer. Since f is differentiable, we can use its derivative to determine whether or not it is a contraction map. We have f'(x) = -b exp(a + bx) / (1 + exp(a + bx))^2. If we let u = exp(a + bx), then this is -b u / (1 + u^2), which is maximized (in absolute value) at b/2 when u = 1. Hence, f is a contraction on the real numbers if |b| 2. If |b| = 2, then f is a weak contraction.

But wait! Since f takes values in [0, 1], we might as well consider it as a function on [0, 1] when looking for fixed points. There are three cases:

1) u = 1 for some x in [0, 1]. This happens when one of a and a + b is positive, and the other is negative. The maximum value of f’ occurs somewhere in [0, 1], so the picture is the same as when considering f on the real numbers.

2) u < 1 for all x in [0, 1]. This happens when both a and a + b are negative. The maximum value of |f'| on [0, 1] occurs when u is maximized, which is at x = 0 if b is negative, or x = 1 if b is positive. If we calculate f' as above at this value, then f is a contraction if |f'| 1, and a weak contraction if |f’| = 1. Because [0, 1] is compact, a weak contraction is as good as a contraction for the purpose of finding fixed points.

3) u > 1 for all x in [0, 1]. This happens when both a and a + b are positive. Same as case 2, except the maximum value of |f’| is when u is minimized.

Nathan, I think you have misplaced the exponent (2):

if u=exp(a + bx), then f'(x) = -b u / (1 + u)^2, so the maximal value of |f’| is |b|/4, at u=1. Or in an other way: f'(x)= -b exp(a + bx) / (1 + exp(a + bx))^2 = -b f(x) (1-f(x)). Since 0< f(x) < 1, then f(x)(1-f(x)) reaches its maximum at f(x)=1/2, which is 1/4. So if |b| < 4, then f is a contraction.