From an interview with Philip Wolfe: [John von Nuemann] always contended with Dantzig that the simplex method would take an absurdly long amount of time to solve linear programming problems. It appears to be, oh, so far as I know,…

From an interview with Philip Wolfe: [John von Nuemann] always contended with Dantzig that the simplex method would take an absurdly long amount of time to solve linear programming problems. It appears to be, oh, so far as I know,…

People are invariably surprised when they hear it’s hardly ever necessary to invert a matrix. It’s very often necessary solve linear systems of the form Ax = b, but in practice you almost never do this by inverting A. This…

In a review of linear programming solvers from 1987 to 2002, Bob Bixby says that solvers benefited as much from algorithm improvements as from Moore’s law. Three orders of magnitude in machine speed and three orders of magnitude in algorithmic…

The most recent episode of the Plus Maths podcast describes how the London Velodrome was designed. Being a math podcast, it focuses on the optimization problems involved in the design and the finite element modeling of the structure. The beautiful…

I often hear people often say they’re using a burn-in period in MCMC to run a Markov chain until it converges. But Markov chains don’t converge, at least not the Markov chains that are useful in MCMC. These Markov chains…

Imagine a bank of three elevators along a wall. The elevators are in a straight line but they are not evenly spaced. Where do you stand in order to minimize the average distance you’ll need to walk to catch the…

The snowball strategy says to pay off your smallest debt first, then the next smallest, and so on until you’re out of debt. When I first heard of this I thought it was silly. Clearly the optimal strategy is to…

Rosenbrock’s banana function is a famous test case for optimization software. It’s called the banana function because of its curved contours. The definition of the function is The function has a global minimum at (1, 1). If an optimization method…

In applications you often want to take the maximum of two numbers. But the simple function f(x, y) = max(x, y) can be difficult to work with because it has sharp corners. Sometimes you want an alternative that sands down…

Microsoft’s Solver Foundation is a numerical optimization library capable of solving problems involving millions of variables and millions of constraints. When I listened Scott Hanselman interview Nathan Brixius from Microsoft’s Solver Foundation team, I expected Brixius to say that Solver…

Why can’t we make more use of the 80/20 rule? I’ll review what the 80/20 rule is, explain how it can be powerful, then give four reasons why we don’t take advantage of it. What is the 80/20 rule? The…

I’ve enjoyed following Stephen Boyd’s lectures on convex optimization. I stumbled across a draft version of his textbook a few years ago but didn’t realize at first that the author and the lecturer were the same person. I recommend the…