4
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: not found

      Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization

      research-article

      Read this article at

      ScienceOpenPublisherPMC
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Significance

          Artificial neural networks can suffer from catastrophic forgetting, in which learning a new task causes the network to forget how to perform previous tasks. While previous studies have proposed various methods that can alleviate forgetting over small numbers ( 10) of tasks, it is uncertain whether they can prevent forgetting across larger numbers of tasks. In this study, we propose a neuroscience-inspired scheme, called “context-dependent gating,” in which mostly nonoverlapping sets of units are active for any one task. Importantly, context-dependent gating has a straightforward implementation, requires little extra computational overhead, and when combined with previous methods to stabilize connection weights, can allow networks to maintain high performance across large numbers of sequentially presented tasks.

          Abstract

          Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically causes them to forget previously learned tasks. This phenomenon is the result of “catastrophic forgetting,” in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly nonoverlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks, particularly when combined with weight stabilization. We show that this method works for both feedforward and recurrent network architectures, trained using either supervised or reinforcement-based learning. This suggests that using multiple, complementary methods, akin to what is believed to occur in the brain, can be a highly effective strategy to support continual learning.

          Related collections

          Most cited references14

          • Record: found
          • Abstract: found
          • Article: not found

          Dynamic predictions: oscillations and synchrony in top-down processing.

          Classical theories of sensory processing view the brain as a passive, stimulus-driven device. By contrast, more recent approaches emphasize the constructive nature of perception, viewing it as an active and highly selective process. Indeed, there is ample evidence that the processing of stimuli is controlled by top-down influences that strongly shape the intrinsic dynamics of thalamocortical networks and constantly create predictions about forthcoming sensory events. We discuss recent experiments indicating that such predictions might be embodied in the temporal structure of both stimulus-evoked and ongoing activity, and that synchronous oscillations are particularly important in this process. Coherence among subthreshold membrane potential fluctuations could be exploited to express selective functional relationships during states of expectancy or attention, and these dynamic patterns could allow the grouping and selection of distributed neuronal responses for further processing.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration.

            Sleep is universal, tightly regulated, and its loss impairs cognition. But why does the brain need to disconnect from the environment for hours every day? The synaptic homeostasis hypothesis (SHY) proposes that sleep is the price the brain pays for plasticity. During a waking episode, learning statistical regularities about the current environment requires strengthening connections throughout the brain. This increases cellular needs for energy and supplies, decreases signal-to-noise ratios, and saturates learning. During sleep, spontaneous activity renormalizes net synaptic strength and restores cellular homeostasis. Activity-dependent down-selection of synapses can also explain the benefits of sleep on memory acquisition, consolidation, and integration. This happens through the offline, comprehensive sampling of statistical regularities incorporated in neuronal circuits over a lifetime. This Perspective considers the rationale and evidence for SHY and points to open issues related to sleep and plasticity. Copyright © 2014 Elsevier Inc. All rights reserved.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Structure-stability-function relationships of dendritic spines.

              Dendritic spines, which receive most of the excitatory synaptic input in the cerebral cortex, are heterogeneous with regard to their structure, stability and function. Spines with large heads are stable, express large numbers of AMPA-type glutamate receptors, and contribute to strong synaptic connections. By contrast, spines with small heads are motile and unstable and contribute to weak or silent synaptic connections. Their structure-stability-function relationships suggest that large and small spines are "memory spines" and "learning spines", respectively. Given that turnover of glutamate receptors is rapid, spine structure and the underlying organization of the actin cytoskeleton are likely to be major determinants of fast synaptic transmission and, therefore, are likely to provide a physical basis for memory in cortical neuronal networks. Characterization of supramolecular complexes responsible for synaptic memory and learning is key to the understanding of brain function and disease.
                Bookmark

                Author and article information

                Journal
                Proc Natl Acad Sci U S A
                Proc. Natl. Acad. Sci. U.S.A
                pnas
                pnas
                PNAS
                Proceedings of the National Academy of Sciences of the United States of America
                National Academy of Sciences
                0027-8424
                1091-6490
                30 October 2018
                12 October 2018
                : 115
                : 44
                : E10467-E10475
                Affiliations
                [1] aDepartment of Neurobiology, The University of Chicago , Chicago, IL 60637;
                [2] bThe Grossman Institute for Neuroscience, Quantitative Biology and Human Behavior, The University of Chicago , Chicago, IL 60637
                Author notes
                1To whom correspondence may be addressed. Email: masse@ 123456uchicago.edu or dfreedman@ 123456uchicago.edu .

                Edited by Terrence J. Sejnowski, Salk Institute for Biological Studies, La Jolla, CA, and approved September 11, 2018 (received for review March 4, 2018)

                Author contributions: N.Y.M. and D.J.F. designed research; N.Y.M. and G.D.G. performed research; N.Y.M. and G.D.G. analyzed data; and N.Y.M., G.D.G., and D.J.F. wrote the paper.

                Author information
                http://orcid.org/0000-0002-9094-1298
                Article
                PMC6217392 PMC6217392 6217392 201803839
                10.1073/pnas.1803839115
                6217392
                30315147
                1d3a83bc-3fc7-4248-8e63-200c386d21b8
                Copyright @ 2018

                Published under the PNAS license.

                History
                Page count
                Pages: 9
                Funding
                Funded by: HHS | National Institutes of Health (NIH) 100000002
                Award ID: R01EY019041
                Award Recipient : David Freedman
                Funded by: HHS | National Institutes of Health (NIH) 100000002
                Award ID: R01MH092927
                Award Recipient : David Freedman
                Funded by: National Science Foundation (NSF) 100000001
                Award ID: NCS 1631571
                Award Recipient : David Freedman
                Categories
                PNAS Plus
                Biological Sciences
                Neuroscience
                Physical Sciences
                Computer Sciences
                PNAS Plus

                artificial intelligence,synaptic stabilization,continual learning,catastrophic forgetting,context-dependent gating

                Comments

                Comment on this article