Multi-Species Mean Field Spin Glasses. Rigorous Results

By using a simple interpolation argument, in previous work we have proven the existence of the thermodynamic limit, for mean field disordered models, including the Sherrington-Kirkpatrick model, and the Derrida p-spin model. Here we extend this argument in order to compare the limiting free energy with the expression given by the Parisi Ansatz, and including full spontaneous replica symmetry breaking. Our main result is that the quenched average of the free energy is bounded from below by the value given in the Parisi Ansatz uniformly in the size of the system. Moreover, the difference between the two expressions is given in the form of a sum rule, extending our previous work on the comparison between the true free energy and its replica symmetric Sherrington-Kirkpatrick approximation. We give also a variational bound for the infinite volume limit of the ground state energy per site.

A specific type of neural networks, the Restricted Boltzmann Machines (RBM), are implemented for classification and feature detection in machine learning. They are characterized by separate layers of visible and hidden units, which are able to learn efficiently a generative model of the observed data. We study a "hybrid" version of RBMs, in which hidden units are analog and visible units are binary, and we show that thermodynamics of visible units are equivalent to those of a Hopfield network, in which the N visible units are the neurons and the P hidden units are the learned patterns. We apply the method of stochastic stability to derive the thermodynamics of the model, by considering a formal extension of this technique to the case of multiple sets of stored patterns, which may act as a benchmark for the study of correlated sets. Our results imply that simulating the dynamics of a Hopfield network, requiring the update of N neurons and the storage of N(N-1)/2 synapses, can be accomplished by a hybrid Boltzmann Machine, requiring the update of N+P neurons but the storage of only NP synapses. In addition, the well known glass transition of the Hopfield network has a counterpart in the Boltzmann Machine: it corresponds to an optimum criterion for selecting the relative sizes of the hidden and visible layers, resolving the trade-off between flexibility and generality of the model. The low storage phase of the Hopfield model corresponds to few hidden units and hence a overly constrained RBM, while the spin-glass phase (too many hidden units) corresponds to unconstrained RBM prone to overfitting of the observed data. Copyright © 2012 Elsevier Ltd. All rights reserved.

Aim of this paper is to give an extensive treatment of bipartite mean field spin systems, ordered and disordered: at first, bipartite ferromagnets are investigated, achieving an explicit expression for the free energy trough a new minimax variational principle. Furthermore via the Hamilton-Jacobi technique the same free energy structure is obtained together with the existence of its thermodynamic limit and the minimax principle is connected to a standard max one. The same is investigated for bipartite spin-glasses: By the Borel-Cantelli lemma a control of the high temperature regime is obtained, while via the double stochastic stability technique we get also the explicit expression of the free energy at the replica symmetric level, uniquely defined by a minimax variational principle again. A general results that states that the free energies of these systems are convex linear combinations of their independent one party model counterparts is achieved too. For the sake of completeness we show further that at zero temperature the replica symmetric entropy becomes negative and, consequently, such a symmetry must be broken. The treatment of the fully broken replica symmetry case is deferred to a forthcoming paper. As a first step in this direction, we start deriving the linear and quadratic constraints to overlap fluctuations.