People are invariably surprised when they hear it’s hardly ever necessary to invert a matrix. It’s very often necessary solve linear systems of the form Ax = b, but in practice you almost never do this by inverting A. This…
Blog Archives
Moore's law squared
In a review of linear programming solvers from 1987 to 2002, Bob Bixby says that solvers benefited as much from algorithm improvements as from Moore’s law. Three orders of magnitude in machine speed and three orders of magnitude in algorithmic…
Designed from the inside out
The most recent episode of the Plus Maths podcast describes how the London Velodrome was designed. Being a math podcast, it focuses on the optimization problems involved in the design and the finite element modeling of the structure. The beautiful…
Markov chains don’t converge
I often hear people often say they’re using a burn-in period in MCMC to run a Markov chain until it converges. But Markov chains don’t converge, at least not the Markov chains that are useful in MCMC. These Markov chains…
Where to wait for an elevator
Imagine a bank of three elevators along a wall. The elevators are in a straight line but they are not evenly spaced. Where do you stand in order to minimize the average distance you’ll need to walk to catch the…
Mathematically correct but psychologically wrong
The snowball strategy says to pay off your smallest debt first, then the next smallest, and so on until you’re out of debt. When I first heard of this I thought it was silly. Clearly the optimal strategy is to…
Rosenbrock’s banana function
Rosenbrock’s banana function is a famous test case for optimization software. It’s called the banana function because of its curved contours. The definition of the function is The function has a global minimum at (1, 1). If an optimization method…
Soft maximum
In applications you often want to take the maximum of two numbers. But the simple function f(x, y) = max(x, y) can be difficult to work with because it has sharp corners. Sometimes you want an alternative that sands down…
Solver Foundation optimization library
Microsoft’s Solver Foundation is a numerical optimization library capable of solving problems involving millions of variables and millions of constraints. When I listened Scott Hanselman interview Nathan Brixius from Microsoft’s Solver Foundation team, I expected Brixius to say that Solver…
Four reasons we don’t apply the 80/20 rule
Why can’t we make more use of the 80/20 rule? I’ll review what the 80/20 rule is, explain how it can be powerful, then give four reasons why we don’t take advantage of it. What is the 80/20 rule? The…
Convex optimization
I’ve enjoyed following Stephen Boyd’s lectures on convex optimization. I stumbled across a draft version of his textbook a few years ago but didn’t realize at first that the author and the lecturer were the same person. I recommend the…
