I had never really heard of this result, sometimes called the Matrix Determinant Lemma, but it came up in the process of answering a relatively simple question. Suppose I have an -dimensional jointly Gaussian vector with covariance matrix . The differential entropy of is . Suppose now I consider some rank-1 perturbation . What choice of maximizes the differential entropy?
On the face of it, this seems intuitively easy — diagonalize and then pick to be the eigenvector corresponding to the smallest singular value of . But is there an simple way to see this analytically?
Matrix Determinant Lemma. Let be an positive definite matrix and and be two matrices. Then
To see this, note that
and take determinants on both sides.
So now applying this to our problem,
But the right side is clearly maximized by choosing corresponding to the largest singular value of , which in this case is the smallest singular value of . Ta-da!