Gram matrix

Posted on by John

An elegant algebraic identity says

(a^2 + b^2)*c^2 + d^2) = (ac + bd)^2 + (ad - bc)^2

If x is the vector [a b] and y is the vector [c d] then this identity can be written

\begin{vmatrix} a & b \\ c & d \end{vmatrix}^2 = \begin{vmatrix} x\cdot x & x \cdot y \\ x\cdot y & y \cdot y \end{vmatrix}

where the dot indicates the usual dot product. I posted this on Twitter the other day.

Gram matrix

Now suppose that x and y are vectors of any length n. The matrix on the right hand side above is called the Gram matrix of x and y, and its determinant is called the Gram determinant or Gramian.

Correlation

Let θ be the angle between x and y in ℝn. Then

\cos\theta = \frac{\langle x , y\rangle} {||x|| \, || y ||}

where ⟨x, y⟩ is the dot product of x and y.

If x and y are data vectors with mean 0, then cos θ is their correlation. Since correlation is unaffected by scaling, we can scale y so that it has the same norm as x, i.e. ||x|| = ||y||. Then the Gram matrix G can be written in terms of cos θ:

G = \begin{pmatrix} \langle x, x\rangle & \langle x ,y\rangle \\ \langle x, y\rangle & \langle y ,y\rangle \end{pmatrix} = ||x||^2 \begin{pmatrix} 1 & \cos \theta \\ \cos \theta & 1 \end{pmatrix}

Generalization

The idea of the Gram matrix generalizes to more than two vectors. If we have m vectors, the Gram matrix is m × m whose (i, j) entry is the dot product of the ith and jth vectors. Note that the dimension n of the vectors does not have to equal the dimension m of the Gram matrix.

Let B be the matrix whose columns are the vectors xi. If the number of vectors m does equal the dimension of the vectors n then BT B equals the Gram matrix.

B^\intercal B = G

and so

|B^\intercal B| = |B|^2 = |G|

If m does not equal n then BT B cannot equal G because the two matrices have different dimensions, though the determinants of the two matrices are equal. That is, the square of the m-dimensional volume of the span of the x‘s inside ℝn equals their Gram determinant.

The Gram determinant can be defined for more general inner products than the dot product in ℝn. It could be, for example, the integral of the product of two functions.

Related posts