An elegant algebraic identity says
If x is the vector [a b] and y is the vector [c d] then this identity can be written
where the dot indicates the usual dot product. I posted this on Twitter the other day.
Now suppose that x and y are vectors of any length n. The matrix on the right hand side above is called the Gram matrix of x and y, and its determinant is called the Gram determinant or Gramian.
Let θ be the angle between x and y in ℝn. Then
where ⟨x, y⟩ is the dot product of x and y.
If x and y are data vectors with mean 0, then cos θ is their correlation. Since correlation is unaffected by scaling, we can scale y so that it has the same norm as x, i.e. ||x|| = ||y||. Then the Gram matrix G can be written in terms of cos θ:
The idea of the Gram matrix generalizes to more than two vectors. If we have m vectors, the Gram matrix is m × m whose (i, j) entry is the dot product of the ith and jth vectors. Note that the dimension n of the vectors does not have to equal the dimension m of the Gram matrix.
Let B be the matrix whose columns are the vectors xi. If the number of vectors m does equal the dimension of the vectors n then BT B equals the Gram matrix.
If m does not equal n then BT B cannot equal G because the two matrices have different dimensions, though the determinants of the two matrices are equal. That is, the square of the m-dimensional volume of the span of the x‘s inside ℝn equals their Gram determinant.
The Gram determinant can be defined for more general inner products than the dot product in ℝn. It could be, for example, the integral of the product of two functions.