Determinants and the Inverse

An analyst adds a new column to the flats table: size in square feet, right next to size in square metres. Training suddenly fails with "singular matrix". Nothing is wrong with the code. The two columns say the same thing, and the determinant is how you see that.

The rule

Determinant zero means information lost, and no inverse brings it back.

A 2 × 2 matrix $\begin{pmatrix} a & b \\ c & d \end{pmatrix}$ has determinant $ad - bc$. Think of the matrix as a machine that moves every point in the plane. The determinant is how much it scales areas:

• $\begin{pmatrix} 2 & 1 \\ 1 & 1 \end{pmatrix}$: determinant $2 - 1 = 1$. Areas keep their size.
• $\begin{pmatrix} 2 & 4 \\ 3 & 6 \end{pmatrix}$: determinant $12 - 12 = 0$. The second column is twice the first, so the whole plane is squashed onto a line.

The inverse $M^{-1}$ undoes the machine: $M M^{-1} = I$, the identity. If the plane was squashed onto a line, many points landed on the same spot, and no machine can tell them apart again. So:

• determinant not zero → an inverse exists;
• determinant zero → no inverse (the matrix is singular).

Because the inverse undoes the scaling, its determinant is one over the original: if $\det M = 0.5$ then $\det M^{-1} = 2$.

In ML, the classic linear-regression formula needs the inverse of $X^T X$. If one feature column is a multiple of another, like square feet and square metres, that matrix is singular. The fix is to drop the duplicate or add regularisation. Libraries solve the equations directly rather than forming the inverse, but the same singularity still breaks them.

The shortcut

For a 2 × 2 inverse: swap the diagonal, negate the off-diagonal, divide by the determinant.

Always compute the determinant first. If it is zero, stop: there is nothing to divide by.

Drills

Answer fast — trust the rule.

Remember

• Determinant zero means information lost, and no inverse brings it back.
• Shortcut for 2 × 2: swap the diagonal, negate the off-diagonal, divide by the determinant.
• Trap: duplicate or proportional features make the matrix singular.