11
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Self-regularizing restricted Boltzmann machines

      Preprint

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Focusing on the grand-canonical extension of the ordinary restricted Boltzmann machine, we suggest an energy-based model for feature extraction that uses a layer of hidden units with varying size. By an appropriate choice of the chemical potential and given a sufficiently large number of hidden resources the generative model is able to efficiently deduce the optimal number of hidden units required to learn the target data with exceedingly small generalization error. The formal simplicity of the grand-canonical ensemble combined with a rapidly converging ansatz in mean-field theory enable us to recycle well-established numerical algothhtims during training, like contrastive divergence, with only minor changes. As a proof of principle and to demonstrate the novel features of grand-canonical Boltzmann machines, we train our generative models on data from the Ising theory and MNIST.

          Related collections

          Author and article information

          Journal
          09 December 2019
          Article
          1912.05634
          1639fe48-0878-4de2-9b21-7c1ca6581680

          http://arxiv.org/licenses/nonexclusive-distrib/1.0/

          History
          Custom metadata
          28 pages, 11 figures
          cond-mat.dis-nn cond-mat.stat-mech hep-th stat.ML

          Condensed matter,High energy & Particle physics,Theoretical physics,Machine learning

          Comments

          Comment on this article