+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: not found

      Convergence of stochastic learning in perceptrons with binary synapses.

      Physical review. E, Statistical, nonlinear, and soft matter physics
      Algorithms, Artificial Intelligence, Computer Simulation, Learning, physiology, Models, Neurological, Nerve Net, Neural Networks (Computer), Pattern Recognition, Automated, methods, Signal Processing, Computer-Assisted, Stochastic Processes, Synapses

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          The efficacy of a biological synapse is naturally bounded, and at some resolution, and is discrete at the latest level of single vesicles. The finite number of synaptic states dramatically reduce the storage capacity of a network when online learning is considered (i.e., the synapses are immediately modified by each pattern): the trace of old memories decays exponentially with the number of new memories (palimpsest property). Moreover, finding the discrete synaptic strengths which enable the classification of linearly separable patterns is a combinatorially hard problem known to be NP complete. In this paper we show that learning with discrete (binary) synapses is nevertheless possible with high probability if a randomly selected fraction of synapses is modified following each stimulus presentation (slow stochastic learning). As an additional constraint, the synapses are only changed if the output neuron does not give the desired response, as in the case of classical perceptron learning. We prove that for linearly separable classes of patterns the stochastic learning algorithm converges with arbitrary high probability in a finite number of presentations, provided that the number of neurons encoding the patterns is large enough. The stochastic learning algorithm is successfully applied to a standard classification problem of nonlinearly separable patterns by using multiple, stochastically independent output units, with an achieved performance which is comparable to the maximal ones reached for the task.

          Related collections

          Author and article information


          Comment on this article