8
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Nonnegative Matrix Factorization for Semi-supervised Dimensionality Reduction

      Preprint
      ,

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          We show how to incorporate information from labeled examples into nonnegative matrix factorization (NMF), a popular unsupervised learning algorithm for dimensionality reduction. In addition to mapping the data into a space of lower dimensionality, our approach aims to preserve the nonnegative components of the data that are important for classification. We identify these components from the support vectors of large-margin classifiers and derive iterative updates to preserve them in a semi-supervised version of NMF. These updates have a simple multiplicative form like their unsupervised counterparts; they are also guaranteed at each iteration to decrease their loss function---a weighted sum of I-divergences that captures the trade-off between unsupervised and supervised learning. We evaluate these updates for dimensionality reduction when they are used as a precursor to linear classification. In this role, we find that they yield much better performance than their unsupervised counterparts. We also find one unexpected benefit of the low dimensional representations discovered by our approach: often they yield more accurate classifiers than both ordinary and transductive SVMs trained in the original input space.

          Related collections

          Most cited references5

          • Record: found
          • Abstract: not found
          • Article: not found

          $I$-Divergence Geometry of Probability Distributions and Minimization Problems

          I. Csiszar (1975)
            Bookmark
            • Record: found
            • Abstract: not found
            • Conference Proceedings: not found

            Confidence-weighted linear classification

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Semi-Supervised Nonnegative Matrix Factorization

                Bookmark

                Author and article information

                Journal
                16 December 2011
                Article
                1112.3714
                6fd829df-c198-4f0a-900a-5d458aaa4a83

                http://arxiv.org/licenses/nonexclusive-distrib/1.0/

                History
                Custom metadata
                Preprint submitted to Machine Learning Journal
                cs.LG

                Comments

                Comment on this article