Dimensionality Reduction

The idea behind dimensionality reduction is simple: take high dimensional feature spaces ($k$) and project them onto lower dimensional subspaces ($m$) (where $m$ < $k$). Dimensionality reduction has several kind of appealing properties like solving the curse of dimensionality and overfitting, but it also allows us to visualize high dimensional data and to compress it. Collapsing high dimensional data that would otherwise be too difficult for us to understand or interpret suddenly becomes much more salient when we collapse it down into two or three dimensions. In this post, we walk through the application of the principal component analysis, a central dimensionality reduction algorithm.