Unraveling principal component analysis: A long-form guide through the mathematical fundamentals of data science and machine learning

Principal component analysis is probably the most extraordinary method of data analysis ever devised. With a simple linear operation we can tease out human expression, age and sex from raw photographs. We can tell chimpanzee from human by the shape of a shoulder bone, and we can guess where someone lives, just from their DNA.

The method is easy to explain and easy to apply. But that doesn’t tell you where the magic comes from. It doesn’t tell you why a linear operation should be enough to turn a frowning person into a smiling one.

For that, we need to dig deeper. We need to dig up the foundations. That’s what this book is for. It starts with a quick and to-the-point introduction to what principal component analysis is, and how it works.

Then, it peels back the layers of the onion one by one. We dive into eigenvectors and -values, the spectral theorem, the singular value decomposition and rank decompositions.

Along the way, we’ll touch on subjects like complex numbers, determinants, characteristic polynomials and the fundamental theorem of algebra. We’ll do our best to cover them in the most intuitive way possible, with plenty of examples and illustrations. But the most important rule is that we leave no stone unturned. We will explain everything down to the foundations, without skipping any steps.

Current version: v1.2.0 See changelog here: https://github.com/pbloem/pca-book/releases