13
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Divisive normalization is an efficient code for multivariate Pareto-distributed environments

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Significance

          Divisive normalization is a ubiquitous computation commonly thought to be an implementation of the efficient coding principle. Despite empirical evidence that it reduces statistical redundancy present in naturalistic stimuli, making the relationship between this neural code and the statistics of a stimulus precise has remained elusive. This paper closes this gap by providing a necessary and sufficient condition for divisive normalization to generate an efficient code. The multivariate Pareto distribution found to be efficiently encoded exhibits many stylized features of naturalistic stimulus statistics and provides testable predictions. In an empirical analysis, we find that the Pareto distribution captures the statistics of natural images well, suggesting that divisive normalization may have evolved to efficiently represent stimuli from such distributions.

          Abstract

          Divisive normalization is a canonical computation in the brain, observed across neural systems, that is often considered to be an implementation of the efficient coding principle. We provide a theoretical result that makes the conditions under which divisive normalization is an efficient code analytically precise: We show that, in a low-noise regime, encoding an n-dimensional stimulus via divisive normalization is efficient if and only if its prevalence in the environment is described by a multivariate Pareto distribution. We generalize this multivariate analog of histogram equalization to allow for arbitrary metabolic costs of the representation, and show how different assumptions on costs are associated with different shapes of the distributions that divisive normalization efficiently encodes. Our result suggests that divisive normalization may have evolved to efficiently represent stimuli with Pareto distributions. We demonstrate that this efficiently encoded distribution is consistent with stylized features of naturalistic stimulus distributions such as their characteristic conditional variance dependence, and we provide empirical evidence suggesting that it may capture the statistics of filter responses to naturalistic images. Our theoretical finding also yields empirically testable predictions across sensory domains on how the divisive normalization parameters should be tuned to features of the input distribution.

          Related collections

          Most cited references106

          • Record: found
          • Abstract: not found
          • Article: not found

          A Mathematical Theory of Communication

          C. Shannon (1948)
            Bookmark
            • Record: found
            • Abstract: not found
            • Article: not found

            Power-Law Distributions in Empirical Data

              Bookmark
              • Record: found
              • Abstract: not found
              • Article: not found

              Information Theory and Statistical Mechanics

                Bookmark

                Author and article information

                Journal
                Proc Natl Acad Sci U S A
                Proc Natl Acad Sci U S A
                pnas
                PNAS
                Proceedings of the National Academy of Sciences of the United States of America
                National Academy of Sciences
                0027-8424
                1091-6490
                26 September 2022
                4 October 2022
                26 September 2022
                : 119
                : 40
                : e2120581119
                Affiliations
                [1] aDepartment of Economics, New York University , New York, NY 10012;
                [2] bDepartment of Computer Science, University of Tübingen , 72076 Tübingen, Germany;
                [3] cDepartment of Computational Neuroscience, Max Planck Institute for Biological Cybernetics , 72076 Tübingen, Germany;
                [4] dStern School of Business, New York University , New York, NY 10012;
                [5] eTandon School of Engineering, New York University , Brooklyn, NY 11201;
                [6] fNew York University Shanghai , Shanghai, China 200122
                Author notes
                1To whom correspondence may be addressed. Email: stefan.bucher@ 123456nyu.edu .

                Edited by Wilson Geisler, The University of Texas at Austin, Austin, Texas; received November 11, 2021; accepted July 11, 2022

                Author contributions: S.F.B. designed research; S.F.B. and A.M.B. developed theoretical results; S.F.B. performed numerical and data analyses; and S.F.B. and A.M.B. wrote the paper.

                Author information
                https://orcid.org/0000-0002-0843-4685
                https://orcid.org/0000-0002-5274-5296
                Article
                202120581
                10.1073/pnas.2120581119
                9546555
                36161961
                a137b116-d120-4546-9be0-75cd0e3b7946
                Copyright © 2022 the Author(s). Published by PNAS.

                This open access article is distributed under Creative Commons Attribution License 4.0 (CC BY).

                History
                : 11 July 2022
                Page count
                Pages: 10
                Funding
                Funded by: NYU | Leonard N. Stern School of Business, New York University (NYU Stern) 100006720
                Award ID: n/a
                Award Recipient : Adam M Brandenburger
                Funded by: New York University Shanghai (NYU Shanghai) 100008410
                Award ID: n/a
                Award Recipient : Adam M Brandenburger
                Funded by: NYU | NYU Grossman School of Medicine (NYUGSM) 100016027
                Award ID: n/a
                Award Recipient : Stefan F Bucher
                Funded by: Alexander von Humboldt-Stiftung (AvH) 100005156
                Award ID: n/a
                Award Recipient : Stefan F Bucher
                Funded by: New York University (NYU) 100006732
                Award ID: n/a
                Award Recipient : Stefan F Bucher
                Categories
                424
                431
                Biological Sciences
                Neuroscience
                Social Sciences
                Psychological and Cognitive Sciences

                divisive normalization,efficient coding,natural stimulus statistics,histogram equalization,pareto distribution

                Comments

                Comment on this article