Blog Archives

Von Neumann’s mistake

From an interview with Philip Wolfe: [John von Nuemann] always contended with Dantzig that the simplex method would take an absurdly long amount of time to solve linear programming problems. It appears to be, oh, so far as I know,

Tagged with:
Posted in Math

Example of not inverting a matrix: optimization

People are invariably surprised when they hear it’s hardly ever necessary to invert a matrix. It’s very often necessary solve linear systems of the form Ax = b, but in practice you almost never do this by inverting A. This

Tagged with: , ,
Posted in Python

Moore's law squared

In a review of linear programming solvers from 1987 to 2002, Bob Bixby says that solvers benefited as much from algorithm improvements as from Moore’s law. Three orders of magnitude in machine speed and three orders of magnitude in algorithmic

Tagged with:
Posted in Computing

Designed from the inside out

The most recent episode of the Plus Maths podcast describes how the London Velodrome was designed. Being a math podcast, it focuses on the optimization problems involved in the design and the finite element modeling of the structure. The beautiful

Tagged with: ,
Posted in Math

Markov chains don’t converge

I often hear people often say they’re using a burn-in period in MCMC to run a Markov chain until it converges. But Markov chains don’t converge, at least not the Markov chains that are useful in MCMC. These Markov chains

Tagged with: , ,
Posted in Statistics

Where to wait for an elevator

Imagine a bank of three elevators along a wall. The elevators are in a straight line but they are not evenly spaced. Where do you stand in order to minimize the average distance you’ll need to walk to catch the

Tagged with: ,
Posted in Math, Statistics

Mathematically correct but psychologically wrong

The snowball strategy says to pay off your smallest debt first, then the next smallest, and so on until you’re out of debt. When I first heard of this I thought it was silly. Clearly the optimal strategy is to

Tagged with: ,
Posted in Uncategorized

Rosenbrock’s banana function

Rosenbrock’s banana function is a famous test case for optimization software. It’s called the banana function because of its curved contours. The definition of the function is The function has a global minimum at (1, 1). If an optimization method

Tagged with: , ,
Posted in Math

Soft maximum

In applications you often want to take the maximum of two numbers. But the simple function f(x, y) = max(x, y) can be difficult to work with because it has sharp corners. Sometimes you want an alternative that sands down

Tagged with:
Posted in Math

Solver Foundation optimization library

Microsoft’s Solver Foundation is a numerical optimization library capable of solving problems involving millions of variables and millions of constraints. When I listened Scott Hanselman interview Nathan Brixius from Microsoft’s Solver Foundation team, I expected Brixius to say that Solver

Tagged with: ,
Posted in Computing, Software development

Four reasons we don’t apply the 80/20 rule

Why can’t we make more use of the 80/20 rule? I’ll review what the 80/20 rule is, explain how it can be powerful, then give four reasons why we don’t take advantage of it. What is the 80/20 rule? The

Tagged with: ,
Posted in Creativity

Convex optimization

I’ve enjoyed following Stephen Boyd’s lectures on convex optimization. I stumbled across a draft version of his textbook a few years ago but didn’t realize at first that the author and the lecturer were the same person. I recommend the

Tagged with: ,
Posted in Math