Covariant and contravariant

The terms covariant and contravariant come up in many contexts. An earlier post discussed how the terms are used in programming and category theory. The meaning in programming is an instance of the general use in category theory.

Vector fields can be covariant or contravariant too. This is also an instance of the categorical usage, except the terminology is backward.

Michael Spivak explains:

Nowadays such situations are always distinguished by calling the things which go in the same direction “covariant” and the things which go in the opposite direction “contravariant.” Classical terminology used these same words, and it just happens to have reversed this: a vector field is called a contravariant vector field, while a section of T*M is called a covariant vector field. And no one had the gall or authority to reverse terminology sanctified by years of usage. So it’s very easy to remember which kind of vector field is covariant, and which is contravariant — it’s just the opposite of what it logically ought to be.

Emphasis added.

In defense of classical nomenclature, it was established decades before category theory. And as Spivak explains immediately following the quotation above, the original terminology made sense in its original context.

From Spivak’s Differential Geometry, volume 1. I own the 2nd edition and quoted from it. But it’s out of print so I linked to the 3rd edition. I doubt the quote changed between editions, but I don’t know.

Related: Applied category theory

Geeky company names

I started a discussion on Twitter this evening about consulting company names. Here are some of the names.

  • Turing Machine Computing: If we can’t do it, it can’t be done.
  • Heisenberg Consulting: You can have speed or quality, but not both at the same time.
  • Perelman Consulting: Please don’t pay us. We don’t want your money.
  • Gödel Systems: Your job is done but we can’t prove it.
  • Gödel Consulting: because no one is supplying ALL your needs.
  • Lebesgue Consulting: We’ve got your measure.
  • Noether Consulting: We find the conserved values of your system.
  • Fourier consulting: We transform your world periodically.
  • Zorn’s Consulting: Your choice is axiomatic.
  • Spherical Computing: Without parallel.
  • Markov Chain Consulting: It doesn’t matter how we got here.
  • Dirac Consulting: We get right to the point.
  • Shannon Consulting: We’ll find a way to deliver your message.
  • Neyman & Pearson Consulting: No one is more powerful than us.
  • Complex Conjugate Consulting: We make your product real.
  • Hadamard Consulting: Real solutions by complex methods.
  • Zeno Consulting: We’ll get you arbitrarily close to where you want to be.
  • Hilbert Consulting: You think you have a problem?
  • Riemann Hypothesis Consulting: When your job is on the line and everything is critical

Here are footnotes explaining the puns above.

  • Turing: In computer science, Turing machines define the limits of what is computable.
  • Heisenberg: The Heisenberg Uncertainty Principle says that there is a limit to how well you can know a particle’s momentum and position. The more accurately you know one, the less you know about the other.
  • Perelman: Turned down prize money from the Fields Institute and Clay Institute after solving the Poincaré conjecture.
  • Gödel: His incompleteness theorem says that number theory contains true theorems that cannot be proved.
  • Lebesgue: Founder of measure theory, a rigorous theory of length, area, volume, etc.
  • Noether: Established a deep connection between symmetry and conservation laws.
  • Fourier: Known for Fourier transforms and Fourier series, expressing functions as sums or integrals of periodic functions.
  • Zorn: Known for Zorn’s lemma, equivalent to the axiom of choice.
  • Spherical: There are no parallel lines in spherical geometry.
  • Markov Chain: The probability distribution for the next move in a Markov chain depends only on the current state and not on previous history.
  • Complex Conjugate: A complex number times its conjugate is a real number. See xkcd.
  • Dirac: The reference here is to the Dirac delta function. Informally, a point mass. Formally, a distribution.
  • Shannon: Founder of communication theory.
  • Neyman-Pearson: The Neyman-Pearson lemma concerns most powerful hypothesis tests.
  • Hadamard: Said “The shortest path between two truths in the real domain passes through the complex domain.” That is, techniques from complex analysis are often the easiest way to approach problems from real analysis.
  • Zeno: Zeno’s paradox says you cannot get anywhere because first you have to get halfway there, then halfway again, etc.
  • Hilbert: Created a famous list of 23 research problems in math in 1900.
  • Riemann: The Riemann hypothesis says that all the non-trivial zeros of the Riemann zeta function line on the critical line Re(z) = 1/2.

Pulp science fiction and vibrations

This morning I ran across Pulp-o-mizer and decided my series of posts on mechanical vibrations could use a little sensational promotion.

The posts are

Consulting in differential equations

Damped, driven oscillations

This is the final post in a four-part series on vibrating systems. The first three parts were

and now we consider driven, damped vibrations. We are looking at the equation

m u'' + γ u' + k u = F cos ωt

where all coefficients are positive.

As in the previous post, we need to find one solution to the equation with the forcing term F cos ωt and add it to the general solution to the homogeneous (free) equation.

The particular solution we’re looking for is simply R cos(ωt– φ), but the dependence of R and φ on the coefficients m, γ, and k is complicated. We’ll come back to this later, but first we’ll see what we can understand without knowing the exact form of R and φ.

The solutions to the unforced equation decay exponentially over time because of damping. The long-term behavior of the system, known as the steady state solution, is just R cos(ωt– φ). The frequency of the steady state solution is the same as the frequency of the forcing function, though the phase will be different in general.

To find the amplitude and phase of the steady state solution, we first define Δ > 0 by

Δ2 = m202 – ω2)2 + γ2 ω2

where ω0 = √(k/m) is the natural frequency of the system.

Then R = F /Δ . Note that Δ is small when the driving frequency ω is close to the natural frequency. That’s not where Δ is minimized, though when γ is small the minimum of Δ occurs close to the natural frequency. The minimum of Δ, and hence the maximum of the amplitude R, occurs at a lower frequency, namely

ω2 = ω02 – γ2 / 2m2.

The phase φ satisfies

cos φ = m02 – ω2)/ Δ

and

sin φ = γω/ Δ.

For low frequency ω, the phase φ is small, i.e. the response is nearly in phase with the excitation. For ω =ω0, i.e. forcing at the natural frequency, the response lags the excitation by π/2, a quarter cycle.

* * *

If your company needs help with an application of differential equations, please call or email to discuss your project.

LET’S TALK

Exact chaos

Pick a number x between 0 and 1. Then repeatedly replace x with 4x(1-x). For almost all starting values of x, the result exhibits chaos. Two people could play this game with starting values very close together, and eventually their sequences will diverge.

It’s somewhat surprising that the iterative process described above can be written down in closed form. Starting from a value x0, the value after n iterations is

sin( 2n arcsin( √ x0 ) )2.

Now suppose two people start with the same initial value. One repeatedly applies 4x(1-x) and the other uses the formula above. If both carried out their calculations exactly, both would produce the same output at every step. But what if both used a computer?

The two approaches correspond to the Python functions f and g below. Because both functions are executed in finite precision arithmetic, both have errors, but they have different errors. Suppose we want to look at the difference between the two functions as we increase n.

from scipy import arcsin, sin, sqrt, linspace
from matplotlib import pyplot as plt

def f(x0, n):
    x = x0
    for _ in range(n):
        x = 4*x*(1-x)
    return x

def g(x0, n):
    return sin(2.0**n * arcsin(sqrt(x0)))**2

n = 40
x = linspace(0, 1, 100)
plt.plot(x, f(x, n) - g(x, n))
plt.ylim(-1, 1)
plt.show()

When we run the code, nothing exciting happens. The difference is a flat line.

Next we increase n to 45 and we start to see the methods diverge.

The divergence is large when n is 50.

And the two functions are nearly completely uncorrelated when n is 55.

Update

So which function is more accurate, f or g? As noted in the comments, the two functions have different kinds of numerical errors. The former accumulates arithmetic precision error at each iteration. The latter shifts noisy bits into significance by multiplying by 2^n. Apparently both about the same overall error, though they have different distributions of error.

I recomputed g using 100-digit precision with mpmath and used the results as the standard to evaluate the output of f and g in ordinary precision. Here’s a plot of the errors when n = 45, first with f

and then with g.

The average absolute errors of f and g are 0.0024 and 0.0015 respectively.

Wonky but free

Rachel Kroll wrote a blog post last Friday entitled I mortgaged my future with a Mac. The part I found most interesting is near the end of the post.

Instead of staying with my wonky-but-free ways of doing things, I shifted all of my stuff over to the Mac. … Now when I want to get back out, I have to do all of the work I thought I had managed to avoid by using a Mac in the first place.

I’m not interested in the pros and cons of using a Mac, but I really liked the phrase “wonky but free.” I have a growing appreciation for things that are wonky-but-free, technologies that make a poor first impression but have long-term benefits.

Simpler for whom?

Here’s an amusing sentence I ran across this morning:

The code was simplified in 2003 and is harder to understand.

Maybe the author is being sarcastic, but I doubt it. I believe he means something like this:

In 2003, the code was made more concise. This made it simpler from the perspective of the maintainers of the code, sophisticated programmers well versed with the project. Unfortunately, this makes the code harder for novices, such as the intended audience for this book, to understand.

Quotation from An Introduction to Programming in Emacs Lisp.

See Simple versus easy for a discussion of what it means to be simple. We often conflate the two terms. Rich Hickey argues that ease is subjective but that simplicity can be objectively defined. He makes an interesting case, though his definition of simplicity may not be what people have in mind or want when they talk about simplicity.

Undamped forced vibrations

This is the third in a series of four blog posts on mechanical vibrations. The first two posts were

We’re looking at solutions to the differential equation

m u'' + γ u' + k u = F cos ωt

The first two posts considered “free” vibrations, systems with no external input. The coefficient F is zero in such systems. Now we consider the effect of an external driving force of the form F cos ωt. We’ll leave out damping effects in this post, that is, we will assume γ = 0. The final post in this series will cover damped, driven oscillations, i.e. the case of γ and F both positive.

When the right-hand side of a linear differential equation is 0, we say the equation is homogeneous. When the right-hand side is not zero, we say the equation is non-homogeneous.

The general solution to a non-homogeneous linear differential equation is the general solution to the corresponding homogeneous equation plus any particular solution to the non-homogeneous equation. That is, all we have to do is find one solution to the equation with the forcing term, and add it to the general solution to the equation without the forcing term.

Recall that the natural frequency of our equation without forcing is ω defined by

ω02 = k/m.

and that the general solution without forcing is

u(t) = R cos(ω0t – φ)

where the amplitude R and the phase φ are determined by the initial conditions. To find the general solution to the non-homogeneous we need to find one solution to the non-homogeneous equation, and here’s one:

F cos(ωt) /(m02 – ω2))

provided that the natural frequency ω0 is not equal to the driving frequency ω. This says that the general solution is the sum of two periodic functions, one with the natural frequency ω0 and one with the frequency of the driving force ω. In general, the sum of two periodic functions can be fairly complicated, and not necessarily periodic. More on that here.

Next we will look at two phenomena that the solutions can illustrate: beats and resonance. The latter happens when the natural and driving frequencies are equal.

Beats

If the starts at position 0 and with velocity 0, then the amplitudes of the homogeneous and non-homogeneous solutions are the same. The solution is

F /(m02 – ω2)) ( cos(ωt) – cos(ω0t) )

We can apply the trig identity

cos(A) – cos(B) = 2 sin( (AB)/2 ) sin(( A+B)/2 )

to rewrite the solution as the amplitude 2F /(m02 – ω2)) times

sin( (ω0 – ω) t/2 ) sin( (ω0 + ω) t/2 )

If the natural and driving frequencies are similar, i.e. |ω0 – ω| is small, then the first term is a low frequency factor multiplying the second high frequency term. This phenomena is known as beats. You can hear it, for example, when two instruments are nearly but not quite in tune. If one is playing an A 440 and the other an A 441, you’ll hear a pitch of 440.5 getting louder and softer at a frequency of one cycle per second.

Here’s an example of beats, plotting sin(t) sin(10t). The solid blue line is the product. The dotted green lines are sin(t) and -sin(t).

illustrating beats

Incidentally, this is related to how AM radio works: the broadcast content is multiplied by the carrier wave.

Resonance

When we wrote down the solution above, we assumed the natural and driving frequencies were not equal. If they are, if the system is being driven at its natural frequency, we have resonance. In that case, the solution to the non-homogeneous equation is

(F/2mω) t sin ωt.

Because the solution is proportional to t sin ωt, this says we have oscillations whose amplitude grows linearly with time. In theory, the oscillations can become arbitrarily large. In reality there is always some damping, which eventually puts some limit on the amplitude of the oscillations. Also, the system may break before damping would reign in the oscillations.

Here’s an example of resonance, plotting 0.25 t sin(t) in solid blue and ± 0.25 t in dotted green.

Next

The final post in this series will look at the effect of damping on forced vibrations.

For daily posts about differential equations, follow @diff_eq on Twitter.

diff_eq twitter icon

How mathematicians see physics

From the preface to Physics for Mathematicians:

In addition to presenting the advanced physics, which mathematicians find so easy, I also want to explore the workings of elementary physics, and mysterious maneuvers — which physicists seem to find so natural — by which one reduces a complicated physical problem to a simple mathematical question, which I have always found so hard to fathom.

That’s exactly how I feel about physics. I’m comfortable with differential equations and manifolds. It’s blocks and pulleys that kick my butt.