52
views
0
recommends
+1 Recommend
0 collections
    0
    shares
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Model-based influences on humans’ choices and striatal prediction errors

      research-article

      Read this article at

      Bookmark
          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

          Summary

          The mesostriatal dopamine system is prominently implicated in model-free reinforcement learning, with fMRI BOLD signals in ventral striatum notably covarying with model-free prediction errors. However, latent learning and devaluation studies show that behavior also shows hallmarks of model-based planning, and the interaction between model-based and model-free values, prediction errors and preferences is underexplored. We designed a multistep decision task in which model-based and model-free influences on human choice behavior could be distinguished. By showing that choices reflected both influences we could then test the purity of the ventral striatal BOLD signal as a model-free report. Contrary to expectations, the signal reflected both model-free and model-based predictions in proportions matching those that best explained choice behavior. These results challenge the notion of a separate model-free learner and suggest a more integrated computational architecture for high-level human decision-making.

          Related collections

          Most cited references76

          • Record: found
          • Abstract: found
          • Article: not found

          Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action.

          Recent behavioral studies in both humans and rodents have found evidence that performance in decision-making tasks depends on two different learning processes; one encoding the relationship between actions and their consequences and a second involving the formation of stimulus-response associations. These learning processes are thought to govern goal-directed and habitual actions, respectively, and have been found to depend on homologous corticostriatal networks in these species. Thus, recent research using comparable behavioral tasks in both humans and rats has implicated homologous regions of cortex (medial prefrontal cortex/medial orbital cortex in humans and prelimbic cortex in rats) and of dorsal striatum (anterior caudate in humans and dorsomedial striatum in rats) in goal-directed action and in the control of habitual actions (posterior lateral putamen in humans and dorsolateral striatum in rats). These learning processes have been argued to be antagonistic or competing because their control over performance appears to be all or none. Nevertheless, evidence has started to accumulate suggesting that they may at times compete and at others cooperate in the selection and subsequent evaluation of actions necessary for normal choice performance. It appears likely that cooperation or competition between these sources of action control depends not only on local interactions in dorsal striatum but also on the cortico-basal ganglia network within which the striatum is embedded and that mediates the integration of learning with basic motivational and emotional processes. The neural basis of the integration of learning and motivation in choice and decision-making is still controversial and we review some recent hypotheses relating to this issue.
            Bookmark
            • Record: found
            • Abstract: found
            • Article: not found

            Remembering the past to imagine the future: the prospective brain.

            A rapidly growing number of recent studies show that imagining the future depends on much of the same neural machinery that is needed for remembering the past. These findings have led to the concept of the prospective brain; an idea that a crucial function of the brain is to use stored information to imagine, simulate and predict possible future events. We suggest that processes such as memory can be productively re-conceptualized in light of this idea.
              Bookmark
              • Record: found
              • Abstract: found
              • Article: not found

              The neural basis of loss aversion in decision-making under risk.

              People typically exhibit greater sensitivity to losses than to equivalent gains when making decisions. We investigated neural correlates of loss aversion while individuals decided whether to accept or reject gambles that offered a 50/50 chance of gaining or losing money. A broad set of areas (including midbrain dopaminergic regions and their targets) showed increasing activity as potential gains increased. Potential losses were represented by decreasing activity in several of these same gain-sensitive areas. Finally, individual differences in behavioral loss aversion were predicted by a measure of neural loss aversion in several regions, including the ventral striatum and prefrontal cortex.
                Bookmark

                Author and article information

                Journal
                8809320
                1600
                Neuron
                Neuron
                Neuron
                0896-6273
                1097-4199
                15 March 2011
                24 March 2011
                24 March 2012
                : 69
                : 6
                : 1204-1215
                Affiliations
                [1 ] Center for Neural Science and Department of Psychology, New York University
                [2 ] Department of Psychology and Neuroscience Institute, Princeton University
                [3 ] Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London
                [4 ] Gatsby Computational Neuroscience Unit, University College London
                Article
                NIHMS280176
                10.1016/j.neuron.2011.02.027
                3077926
                21435563
                16e1819b-4aa7-4bf6-9976-5619360c74fb

                Open Access under CC BY 3.0 license.

                History
                Categories
                Article

                Neurosciences
                Neurosciences

                Comments

                Comment on this article