Logarithms yearning to be free

I got an evaluation copy of The Best Writing on Mathematics 2021 yesterday. One article jumped out as I was skimming the table of contents: A Zeroth Power Is Often a Logarithm Yearning to Be Free by Sanjoy Mahajan. Great title.

There are quite a few theorems involving powers that have an exceptional case that involves a logarithm. The author opens with the example of finding the antiderivative of xⁿ. When n ≠ -1 the antiderivative is another power function, but when n = -1 it’s a logarithm.

Another example that the author mentions is that the limit of power means as the power goes to 0 is the geometric mean. i.e. the exponential of the mean of the logarithms of the arguments.

I tried to think of other examples where this pattern pops up, and I thought of a couple related to entropy.

q-logarithm entropy

The definition of q-logarithm entropy takes Mahajan’s idea and runs it backward, turning a logarithm into a power. As I wrote about here,

The natural logarithm is given by

$\ln(x) = \int_1^x t^{-1}\,dt$

and we can generalize this to the q-logarithm by defining

$\ln_q(x) = \int_1^x t^{-q}\,dt$

And so ln₁ = ln.

Then q-logarithm entropy is just Shannon entropy with natural logarithm replaced by q-logarithm.

Rényi entropy

Quoting from this post,

If a discrete random variable X has n possible values, where the ith outcome has probability p_i, then the Rényi entropy of order α is defined to be

$H_\alpha(X) = \frac{1}{1 - \alpha} \log_2 \left(\sum_{i=1}^n p_i^\alpha \right)$

for 0 ≤ α ≤ ∞. In the case α = 1 or ∞ this expression means the limit as α approaches 1 or ∞ respectively.

…

When α = 1 we get the more familiar Shannon entropy:

$H_1(X) = \lim_{\alpha\to1} H_\alpha(X) = - \left(\sum_{i=1}^n p_i \log_2 p_i \right)$

In this case there’s already a logarithm in the definition, but it moves inside the parentheses in the limit.

And if you rewrite

p^α

p p^α-1

then as the exponent in p^α-1 goes to zero, we have a logarithm yearning to be free.

Logarithms yearning to be free

q-logarithm entropy

Rényi entropy

Related

One thought on “Logarithms yearning to be free”