Fixed points of logistic function

Here’s an interesting problem that came out of a logistic regression application. The input variable was between 0 and 1, and someone asked when and where the logistic transformation

f(x) = 1/(1 + exp(abx))

has a fixed point, i.e. f(x) = x.

So given logistic regression parameters a and b, when does the logistic curve given by yf(x) cross the line yx? Do they always cross? Can they cross twice?

There’s always at least one solution. Because f(x) is strictly between 0 and 1, the function

g(x) = f(x) – x

is positive at 0 and negative at 1, and by the intermediate value theorem g(x) must be zero for some x between 0 and 1.

Sometimes f has only one fixed point. It may have two or three fixed points, as demonstrated in the graph below. The case of two fixed points is unstable: the logistic curve is tangent to the line yx at one point, and a tiny change would turn this tangent point into either no crossing or two crossings.

Logistic curves crossing identity line

If |b| < 1, then you can show that the function f is a contraction map on [0, 1]. In that case there is a unique solution to f(x) = x, and you can find it by starting with an arbitrary value for x and repeatedly applying f to it. For example, if a= 1 and b = 0.8 and we start with x= 0, after applying f ten times we get xf(x) = 0.233790157.

There are a couple questions left to finish this up. How can you tell from a and b how many fixed points there will be? The condition |b| < 1 is sufficient for f to be a contraction map on [0, 1]. Can you find necessary and sufficient conditions?

Related post: Sensitivity of logistic regression prediction

2 thoughts on “Fixed points of logistic function

  1. The second question, at least, is reasonably easy to answer. Since f is differentiable, we can use its derivative to determine whether or not it is a contraction map. We have f'(x) = -b exp(a + bx) / (1 + exp(a + bx))^2. If we let u = exp(a + bx), then this is -b u / (1 + u^2), which is maximized (in absolute value) at b/2 when u = 1. Hence, f is a contraction on the real numbers if |b| 2. If |b| = 2, then f is a weak contraction.

    But wait! Since f takes values in [0, 1], we might as well consider it as a function on [0, 1] when looking for fixed points. There are three cases:

    1) u = 1 for some x in [0, 1]. This happens when one of a and a + b is positive, and the other is negative. The maximum value of f’ occurs somewhere in [0, 1], so the picture is the same as when considering f on the real numbers.
    2) u < 1 for all x in [0, 1]. This happens when both a and a + b are negative. The maximum value of |f'| on [0, 1] occurs when u is maximized, which is at x = 0 if b is negative, or x = 1 if b is positive. If we calculate f' as above at this value, then f is a contraction if |f'| 1, and a weak contraction if |f’| = 1. Because [0, 1] is compact, a weak contraction is as good as a contraction for the purpose of finding fixed points.
    3) u > 1 for all x in [0, 1]. This happens when both a and a + b are positive. Same as case 2, except the maximum value of |f’| is when u is minimized.

  2. István Á. Harmati

    Nathan, I think you have misplaced the exponent (2):
    if u=exp(a + bx), then f'(x) = -b u / (1 + u)^2, so the maximal value of |f’| is |b|/4, at u=1. Or in an other way: f'(x)= -b exp(a + bx) / (1 + exp(a + bx))^2 = -b f(x) (1-f(x)). Since 0< f(x) < 1, then f(x)(1-f(x)) reaches its maximum at f(x)=1/2, which is 1/4. So if |b| < 4, then f is a contraction.

Leave a Reply

Your email address will not be published.