+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Recurrent computations for visual pattern completion


      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          The ability to complete patterns and interpret partial information is a central property of intelligence. Deep convolutional network architectures have proved successful in labeling whole objects in images and capturing the initial 150 ms of processing along the ventral visual cortex. This study shows that human object recognition abilities remain robust when only small amounts of information are available due to heavy occlusion, but the performance of bottom-up computational models is impaired under limited visibility. The results provide combined behavioral, neurophysiological, and modeling insights showing how recurrent computations may help the brain solve the fundamental challenge of pattern completion.


          Making inferences from partial information constitutes a critical aspect of cognition. During visual perception, pattern completion enables recognition of poorly visible or occluded objects. We combined psychophysics, physiology, and computational models to test the hypothesis that pattern completion is implemented by recurrent computations and present three pieces of evidence that are consistent with this hypothesis. First, subjects robustly recognized objects even when they were rendered <15% visible, but recognition was largely impaired when processing was interrupted by backward masking. Second, invasive physiological responses along the human ventral cortex exhibited visually selective responses to partially visible objects that were delayed compared with whole objects, suggesting the need for additional computations. These physiological delays were correlated with the effects of backward masking. Third, state-of-the-art feed-forward computational architectures were not robust to partial visibility. However, recognition performance was recovered when the model was augmented with attractor-based recurrent connectivity. The recurrent model was able to predict which images of heavily occluded objects were easier or harder for humans to recognize, could capture the effect of introducing a backward mask on recognition behavior, and was consistent with the physiological delays along the human ventral visual stream. These results provide a strong argument of plausibility for the role of recurrent computations in making visual inferences from partial information.

          Related collections

          Most cited references24

          • Record: found
          • Abstract: found
          • Article: not found

          Neural networks and physical systems with emergent collective computational abilities.

          J Hopfield (1982)
          Computational properties of use of biological organisms or to the construction of computers can emerge as collective properties of systems having a large number of simple equivalent components (or neurons). The physical meaning of content-addressable memory is described by an appropriate phase space flow of the state of a system. A model of such a system is given, based on aspects of neurobiology but readily adapted to integrated circuits. The collective properties of this model produce a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size. The algorithm for the time evolution of the state of the system is based on asynchronous parallel processing. Additional emergent collective properties include some capacity for generalization, familiarity recognition, categorization, error correction, and time sequence retention. The collective properties are only weakly sensitive to details of the modeling or the failure of individual devices.
            • Record: found
            • Abstract: found
            • Article: not found

            Top-down influences on visual processing.

            Re-entrant or feedback pathways between cortical areas carry rich and varied information about behavioural context, including attention, expectation, perceptual tasks, working memory and motor commands. Neurons receiving such inputs effectively function as adaptive processors that are able to assume different functional states according to the task being executed. Recent data suggest that the selection of particular inputs, representing different components of an association field, enable neurons to take on different functional roles. In this Review, we discuss the various top-down influences exerted on the visual cortical pathways and highlight the dynamic nature of the receptive field, which allows neurons to carry information that is relevant to the current perceptual demands.
              • Record: found
              • Abstract: found
              • Article: not found

              A feedforward architecture accounts for rapid categorization.

              Primates are remarkably good at recognizing objects. The level of performance of their visual system and its robustness to image degradations still surpasses the best computer vision systems despite decades of engineering effort. In particular, the high accuracy of primates in ultra rapid object categorization and rapid serial visual presentation tasks is remarkable. Given the number of processing stages involved and typical neural latencies, such rapid visual processing is likely to be mostly feedforward. Here we show that a specific implementation of a class of feedforward theories of object recognition (that extend the Hubel and Wiesel simple-to-complex cell hierarchy and account for many anatomical and physiological constraints) can predict the level and the pattern of performance achieved by humans on a rapid masked animal vs. non-animal categorization task.

                Author and article information

                Proc Natl Acad Sci U S A
                Proc. Natl. Acad. Sci. U.S.A
                Proceedings of the National Academy of Sciences of the United States of America
                National Academy of Sciences
                28 August 2018
                13 August 2018
                13 August 2018
                : 115
                : 35
                : 8835-8840
                [1] aProgram in Biophysics, Harvard University , Boston, MA 02115;
                [2] bChildren’s Hospital, Harvard Medical School , Boston, MA 02115;
                [3] cProgram in Software Engineering, Institut für Informatik, Universität Augsburg , 86159 Augsburg, Germany;
                [4] dProgram in Software Engineering, Institut für Informatik, Ludwig-Maximilians-Universität München , 80538 München, Germany;
                [5] eProgram in Software Engineering, Fakultät für Informatik, Technische Universität München , 85748 Garching, Germany;
                [6] fMolecular and Cellular Biology, Harvard University , Cambridge, MA 02138
                Author notes
                2To whom correspondence should be addressed. Email: gabriel.kreiman@ 123456tch.harvard.edu .

                Edited by Terrence J. Sejnowski, Salk Institute for Biological Studies, La Jolla, CA, and approved July 20, 2018 (received for review November 10, 2017)

                Author contributions: H.T., M.S., W.L., C.M., D.C., and G.K. designed research; H.T., M.S., W.L., C.M., A.P., J.O.C., W.H., and G.K. performed research; H.T., M.S., W.L., C.M., and G.K. analyzed data; and H.T., M.S., W.L., and G.K. wrote the paper.

                1H.T., M.S., and W.L. contributed equally to this work.

                Copyright © 2018 the Author(s). Published by PNAS.

                This open access article is distributed under Creative Commons Attribution-NonCommercial-NoDerivatives License 4.0 (CC BY-NC-ND).

                Page count
                Pages: 6
                Funded by: HHS | NIH | National Eye Institute (NEI) 100000053
                Award ID: R01EY026025
                Award Recipient : Gabriel Kreiman
                Funded by: National Science Foundation (NSF) 100000001
                Award ID: 1231216
                Award Recipient : Gabriel Kreiman
                Biological Sciences
                Social Sciences
                Psychological and Cognitive Sciences

                visual object recognition,computational neuroscience,pattern completion,artificial intelligence,machine learning


                Comment on this article