PCA Deletes Your Quietest Signals First

Classic Machine Learning Through the Eyes of an SRE — Part 7 Picture a client health metric that has been flat at 2 out of 10 for six months. Ask PCA to compress your client-health data and that metric will contribute almost nothing to the directions PCA decides to keep. Not because PCA is broken. Because PCA treats variance as importance, and a signal that barely moves contributes almost no variance. Reduce the data far enough and the independent information it carried is simply not there anymore. But a CSAT frozen at 2/10 is not noise. It is a crisis nobody is escalating. And after compression, it may no longer be available to anything downstream. That is the bet, and in ops data it is frequently wrong. The critical signals are often the quiet ones. There is a cheaper version of the same failure that catches most people first. PCA measures variance in whatever units your features happen to be in, so a metric ranging from 0 to 10,000 can dominate one ranging from 1 to 5 purely because it is bigger. Standardize before you compress, or your first principal component may just be an elaborate way of saying "ticket count." Same class of bug as unscaled features in K-Means and SVM, and it fails just as quietly. What PCA actually is Third answer-finding strategy in the unsupervised set, using the same shorthand as the last two articles. K-Means SEARCHES: iterate and hope. DBSCAN DEFINES: declare a rule and traverse. PCA SOLVES: an eigendecomposition or SVD gives a direct solution rather than an iterative local search. No convergence to babysit, no restarts, no local optima to escape. Two caveats on the word "direct," both worth knowing. Many libraries will use randomized SVD on large matrices, which is approximate and stochastic. And even with an exact solver, eigenvectors are only defined up to sign, so a component can come back inverted between runs or across implementations. The variance explained is identical either way, which is precisely why nobody notices. Hold ont