Comments on: Ten surprises from numerical linear algebra

By: Anselmo

Anselmo — Sun, 20 Nov 2011 02:48:16 +0000

For me, the greatest achievement of linear algebra is the simplex method. Linear optimisation problems with million variables may be solved in home computers in a few minutes. It is remarkably efficient, though it has an exponential worst case performance. In addition, most exact non-linear optimisation algorithms use the simplex method for solving the linear approximation model.

By: AV

AV — Wed, 14 Sep 2011 10:20:13 +0000

this article is a ‘trivality’.
any indian schoolkid knows this.

– of course linear algebra is balzing fast for FEA and related computations.
but the writer should state that the ’speed’ in fast lin algebra comes not from inherent properties of lin-algebra, but from the fact that we MASSIVELY approximate the actual behavior of the beams/trusses/etc. we approximate reality. and thats becoz we dont have good computational algos to handle the ‘true’ mathematical representations.

if that was ‘overhead transmission’ – then here’s a simplification– try to find a ‘fast’ solution to the Navier Stokes equations using ‘linear algebra’ – in their pure form.

By: Evan

Evan — Tue, 13 Sep 2011 23:16:32 +0000

Thanks mg. I’ll check that out as well.

By: mg

mg — Tue, 13 Sep 2011 23:07:36 +0000

@Evan my favorite text is Janich’s Linear Algebra.

By: martin

martin — Tue, 13 Sep 2011 21:53:45 +0000

Could you elaborate on number 6? I would have thought it would have taken close to a millisecond to just read a million values.

By: J

Tue, 13 Sep 2011 18:18:26 +0000

Charles Van Loan has a sweet pair of custom Nike kicks. Not even joking.

By: Juan Solano

Juan Solano — Tue, 13 Sep 2011 17:33:34 +0000

Let me recommend the videos in Khan Academy, is a very nice place to start and refresh the key concepts.. and then digest the Linear Algebra beauty…

http://www.khanacademy.org/#browse

Juan

By: fish

fish — Tue, 13 Sep 2011 17:32:46 +0000

Ha, good to know re: matrix inversion — I often use inverts in calculating color transforms (see http://www.brucelindbloom.com/index.html?Eqn_RGB_XYZ_Matrix.html) but hey. Thanks!

By: Utkarsh Upadhyay

Utkarsh Upadhyay — Tue, 13 Sep 2011 17:02:44 +0000

Some surprises indeed! However, a small note for point 4:

The efficiency of solving very large systems of equations has benefited at least as much from advances in algorithms as from Moore’s law.

At least solutions for spare system of equations have benefited two orders of magnitude more from algorithmic developments than hardware advancements (slide 7) to a total of 10 orders of magnitude improvement from 1970 to 2001. For other kinds of linear equations, the improvements indeed have been approximately the same..

By: Luis

Luis — Tue, 19 Jul 2011 23:45:47 +0000

Another interesting surprise is that, often, the matrices involved are very sparse.

Books: I love “Matrix algebra useful for statistics” by Searle.

By: Ron Modesitt

Ron Modesitt — Tue, 27 Jul 2010 00:46:12 +0000

What a thoroughly interesting blog! Glad I found you, with thanks to Stumble.

Ron

By: Michael Holmes

Michael Holmes — Thu, 01 Apr 2010 14:33:54 +0000

John suggested I post this here –

We recently released a MATLAB package called QLA that dramatically accelerates core linear algebra routines on dense matrices. It includes SVD, linear systems, least squares, PCA, etc., and speedups range from 10x to 1,000x+ on benchmarks. The gains come by trading a minuscule bit of accuracy for a mammoth increase in speed.

You can find more details and the free beta version here:
http://massiveanalytics.com.

By: Matt

Matt — Sun, 24 Jan 2010 12:46:23 +0000

By the way, has anyone here played with GotoBLAS?
Is its performance really as good as the benchmarks on its website suggest or is it just advertisement?
The source code is freely available, though only for research applications (commercial is not free). Here are relevant links:
http://www.tacc.utexas.edu/resources/software/gotoblasfaq/
http://en.wikipedia.org/wiki/Kazushige_Goto
http://www.tacc.utexas.edu/tacc-projects/
http://web.tacc.utexas.edu/~kgoto/
http://www.pathscale.com/building_code/gotoblas.html

By: Evan

Evan — Thu, 21 Jan 2010 20:54:30 +0000

John, thanks for your quick reply. I’ll definitely check those out!

By: John

John — Thu, 21 Jan 2010 06:11:14 +0000

Evan, there are a lot of linear algebra books out there. I got a review copy of Cheney and Kincaid's linear algebra book a few weeks ago and I thought it was a nice text. It's easy to read and has a good mix of theory and application. If you know basic linear algebra and want to jump into numerical linear algebra, I recommend Demmel's Applied Numerical Linear Algebra. Also, there's the classic Matrix Calculations by Golub and Van Loan.

By: Evan

Evan — Thu, 21 Jan 2010 04:48:29 +0000

So.. you have convinced me to learn Linear Algebra. Do you have any suggestions for books to get started? Thanks for the inspiration!

By: joeyo

joeyo — Wed, 20 Jan 2010 18:27:04 +0000

@Sean: LAPACK is built on top of BLAS/ATLAS.

By: Giles Warrack

Giles Warrack — Wed, 20 Jan 2010 17:31:20 +0000

Re No 3, Herbert Wilf (Penn) has a nice handout on the Kendall-Wei algorithm and Google

http://www.math.upenn.edu/~wilf/

By: Sean Devlin

Sean Devlin — Wed, 20 Jan 2010 16:53:56 +0000

No love for ATLAS? I thought this was use in favor of lapack nowadays.

http://math-atlas.sourceforge.net/

By: Tweets that mention Ten surprises from numerical linear algebra — The Endeavour -- Topsy.com

Wed, 20 Jan 2010 16:06:41 +0000

[...] This post was mentioned on Twitter by John D. Cook, Tracy Harms. Tracy Harms said: RT @JohnDCook: Ten surprises from numerical linear algebra http://bit.ly/5fHYB6 [...]