289
views
0
recommends
+1 Recommend
0 collections
    48
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Linear Dimensionality Reduction: Survey, Insights, and Generalizations

      Preprint
      ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Linear dimensionality reduction methods are a cornerstone of analyzing high dimensional data, due to their simple geometric interpretations and typically attractive computational properties. These methods capture many data features of interest, such as covariance, dynamical structure, correlation between data sets, input-output relationships, and margin between data classes. Methods have been developed with a variety of names and motivations in many fields, and perhaps as a result the connections between all these methods have not been highlighted. Here we survey methods from this disparate literature as optimization programs over matrix manifolds. We discuss principal component analysis, factor analysis, linear multidimensional scaling, Fisher's linear discriminant analysis, canonical correlations analysis, maximum autocorrelation factors, slow feature analysis, sufficient dimensionality reduction, undercomplete independent component analysis, linear regression, distance metric learning, and more. This optimization framework gives insight to some rarely discussed shortcomings of well-known methods, such as the suboptimality of certain eigenvector solutions. Modern techniques for optimization over matrix manifolds enable a generic linear dimensionality reduction solver, which accepts as input data and an objective to be optimized, and returns, as output, an optimal low-dimensional projection of the data. This simple optimization framework further allows straightforward generalizations and novel variants of classical methods, which we demonstrate here by creating an orthogonal-projection canonical correlations analysis. More broadly, this survey and generic solver suggest that linear dimensionality reduction can move toward becoming a blackbox, objective-agnostic numerical technology.

          Related collections

          Author and article information

          Journal
          2014-06-03
          2016-03-18
          Article
          1406.0873
          4caa31be-38ba-46f3-9e78-bb4ea4946891

          http://arxiv.org/licenses/nonexclusive-distrib/1.0/

          History
          Custom metadata
          Journal of Machine Learning Research. 16(Dec): 2859-2900, 2015
          42 pages, 5 figures, 1 table
          stat.ML

          Machine learning
          Machine learning

          Comments

          Comment on this article