17
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Discounting of reward sequences: a test of competing formal models of hyperbolic discounting

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Abstract

          Humans are known to discount future rewards hyperbolically in time. Nevertheless, a formal recursive model of hyperbolic discounting has been elusive until recently, with the introduction of the hyperbolically discounted temporal difference (HDTD) model. Prior to that, models of learning (especially reinforcement learning) have relied on exponential discounting, which generally provides poorer fits to behavioral data. Recently, it has been shown that hyperbolic discounting can also be approximated by a summed distribution of exponentially discounted values, instantiated in the μAgents model. The HDTD model and the μAgents model differ in one key respect, namely how they treat sequences of rewards. The μAgents model is a particular implementation of a Parallel discounting model, which values sequences based on the summed value of the individual rewards whereas the HDTD model contains a non-linear interaction. To discriminate among these models, we observed how subjects discounted a sequence of three rewards, and then we tested how well each candidate model fit the subject data. The results show that the Parallel model generally provides a better fit to the human data.

          Related collections

          Most cited references27

          • Record: found
          • Abstract: found
          • Article: not found

          Area under the curve as a measure of discounting.

          We describe a novel approach to the measurement of discounting based on calculating the area under the empirical discounting function. This approach avoids some of the problems associated with measures based on estimates of the parameters of theoretical discounting functions. The area measure may be easily calculated for both individual and group data collected using any of a variety of current delay and probability discounting procedures. The present approach is not intended as a substitute for theoretical discounting models. It is useful, however, to have a simple, univariate measure of discounting that is not tied to any specific theoretical framework.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops.

            Evaluation of both immediate and future outcomes of one's actions is a critical requirement for intelligent behavior. Using functional magnetic resonance imaging (fMRI), we investigated brain mechanisms for reward prediction at different time scales in a Markov decision task. When human subjects learned actions on the basis of immediate rewards, significant activity was seen in the lateral orbitofrontal cortex and the striatum. When subjects learned to act in order to obtain large future rewards while incurring small immediate losses, the dorsolateral prefrontal cortex, inferior parietal cortex, dorsal raphe nucleus and cerebellum were also activated. Computational model-based regression analysis using the predicted future rewards and prediction errors estimated from subjects' performance data revealed graded maps of time scale within the insula and the striatum: ventroanterior regions were involved in predicting immediate rewards and dorsoposterior regions were involved in predicting future rewards. These results suggest differential involvement of the cortico-basal ganglia loops in reward prediction at different time scales.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              Within-subject comparison of real and hypothetical money rewards in delay discounting.

              A within-subject design, using human participants, compared delay discounting functions for real and hypothetical money rewards. Both real and hypothetical rewards were studied across a range that included $10 to $250. For 5 of the 6 participants, no systematic difference in discount rate was observed in response to real and hypothetical choices, suggesting that hypothetical rewards may often serve as a valid proxy for real rewards in delay discounting research. By measuring discounting at an unprecedented range of real rewards, this study has also systematically replicated the robust finding in human delay discounting research that discount rates decrease with increasing magnitude of reward. A hyperbolic decay model described the data better than an exponential model.
                Bookmark

                Author and article information

                Journal
                Front Psychol
                Front Psychol
                Front. Psychol.
                Frontiers in Psychology
                Frontiers Media S.A.
                1664-1078
                06 March 2014
                2014
                : 5
                Affiliations
                [1] 1Deparment of Psychological and Brain Sciences, Indiana University Bloomington, IN, USA
                [2] 2Department of Experimental Psychology, Ghent University Ghent, Belgium
                Author notes

                Edited by: Philip Beaman, University of Reading, UK

                Reviewed by: Zheng Wang, Ohio State University, USA; Timothy Pleskac, Michigan State University, USA

                *Correspondence: William H. Alexander, Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, B-9000 Ghent, Belgium e-mail: william.alexander@ 123456ugent.be

                This article was submitted to Cognitive Science, a section of the journal Frontiers in Psychology.

                Article
                10.3389/fpsyg.2014.00178
                3944395
                0f420fad-9fff-4a0c-8a4d-96cdb28589e7
                Copyright © 2014 Zarr, Alexander and Brown.

                This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

                Page count
                Figures: 2, Tables: 4, Equations: 13, References: 35, Pages: 9, Words: 7080
                Categories
                Psychology
                Original Research Article

                Comments

                Comment on this article