Log concave functions

Posted on by John

Log concave functions have some very interesting and useful properties. I’ll list some of these shortly after a three definitions.

A function is convex if the line segment joining two points on the graph lies above the graph. In symbols, f(x) is convex if for every t between 0 and 1, and for every x and y,

f(tx + (1-t)y) ≤ t f(x) + (1-t) f(y).

A function f(x) is concave if –f(x) is convex. Equivalently, the line connecting two points on the graph lies below the graph. In symbols, reverse the direction of the inequality for convexity.

A function f(x) is log concave if log( f(x) ) is concave.

The basic properties of convex functions are obvious. It’s easy to show that the sum of two convex functions is convex, the maximum of two convex functions is convex, etc. It would seem that log concave functions would be unremarkable because they are so simply related to convex functions. But they have some surprising properties.

  • The Laplace transform of a non-negative function is a log convex.
  • The running average of a log concave function is also log concave.
  • If a random variable has a log concave PDF (probability density function), then the CDF (cumulative distribution function) is also log concave.
  • The convolution of two log concave functions is log concave.
  • If two independent random variables have log-concave PDFs, then their sum also has a log concave PDF.

A few properties of log concave functions do follow easily from analogous properties of convex and concave functions.

  • The product of log concave functions is log concave.
  • A function f(x) is log concave if and only if for every t between 0 and 1 and for every x and y, f(tx + (1-t)y) ≥ f(x)t f(y)1-t
  • A smooth function f(x) is log concave if and only if the inequality   f ∇2 f ≤ ∇ f ⋅ ∇ f   holds everywhere.

Here are a few specific functions that are log concave.

  • The exponential function eax
  • The inverse logit function ex/(ex + 1)
  • The function ∏ xi / ∑ xi
  • The function xa for a ≥ 0
  • Determinant is a log concave function on positive definite matrices

For more, see Stephen Boyd’s book Convex Optimization.

10 thoughts on “Log concave functions”

  1. gb

    Great post.

    One minor issue – I haven’t seen the function exp(x)/(1+exp(x)) called the inverse logic function before (which might simply be ignorance on my part).

    (I have seen it called the inverse logit function or the logistic function. )

    However, given “logit” and “logic” only differ by one character, I did wonder if it might have been a simple typo.

  3. seh chun

    Hi,

    This post is great and useful. About a point you have put, saying “The running average of a log concave function is also log concave”. May I know is there any book / reference that I can find more information about that?

    Thanks.

  4. John

    seh chun: For a reference, see the book by Stephen Boyd mentioned at the bottom on the post.

  5. KW

    John,

    A concave function could be increasing or decreasing fast in one segment of the curve and slow in other segments, while the upper half of a circle seems increasing and decreasing constantly. My question is how to interpret this in a mathematics form.

    Thanks.

    KW

  6. Brooks

    Thanks!　I have a question about “log concave”.
    Assume Q(x) is a log concave and it denotes the CDF.
    I can’t be sure that “p+(1-2p)Q(x)” is also a log concave function, if given p (0<p<1).
    Can you give some analysis for me , please?

  7. John

    I doubt your conjecture holds. Let f(x) = p + (1 – 2p)Q(x) and test whether f(x) f”(x) ≤ (f'(x))^2 for all p. I believe the inequality does not hold for p = 3/8.

    I only did a hurried calculation and so I may have made errors, but I believe that approach to resolving the question will work.

  8. Brooks

    Hi,
    Q(x) =erf(x);
    I find a intersting result. When p=1-2p, f =p+(1-2p)Q(x) will be a log concave function. The inequality always holds.
    %%%%%
    clear
    clc
    close all
    x=[-10:0.01:10];
    y=1/3+1/3.*erf(x);
    f1= 1/3 *1/(sqrt(2*pi)) *exp(-x.^2/2);
    f2= -1/3 *1/(sqrt(2*pi)) *exp(-x.^2/2) .* x;
    L=f2.*y;
    R=f1.^2;
    boot=(R>=L); %% (if R>=L, the value will be set to 1, or set to 0)
    plot(-10:0.01:10,L,’+’,-10:0.01:10,R,’*’,-10:0.01:10,boot,’s’)

    %%%

    Thanks.

  9. wen

    Hi,

    Thanks! I have a question: is the sum of infinitely many log concave functions a log concave function?

    Thanks!

Comments are closed.