I recently read G.W. Stewart‘s little paper On the Early History of the Singular Value Decomposition (free tech report version is at UMD). It talks about how Beltrami, Jordan, Sylvester, Schmidt, and Weyl all had different approaches to finding/proving the SVD. It’s worth a quick skim, because goodness knows it appears everywhere under all sorts of names. Part of the problem is characterizing the SVD, and the other is calculating it. Since numerical analysis was never part of my training, I don’t have as much sophisticated appreciation for the algorithmic aspects, but I certainly benefit from having efficient solvers.

One point Stewart makes is that we really shouldn’t call the approximation theorem for the SVD the Eckart-Young Theorem, since Schmidt was really the one who showed it much earlier in the context of “integral equations, one of the hot topics of the first decades of our [the 20th] century.” I’ve been guilty of this in the past, so it’s time for me to make amends. I suppose I better start saying Cauchy-Bunyakovsky-Schwarz too.

What was weird to me is that as an (erstwhile?) signal processor, there was not much mention of the Karhunen–Loève transform, even in the little paragraphs on “principal components.”