The modulation of savouring by prediction error and its effects on choice

Abstract

When people anticipate uncertain future outcomes, they often prefer to know their fate in advance. Inspired by an idea in behavioral economics that the anticipation of rewards is itself attractive, we hypothesized that this preference of advance information arises because reward prediction errors carried by such information can boost the level of anticipation. We designed new empirical behavioral studies to test this proposal, and confirmed that subjects preferred advance reward information more strongly when they had to wait for rewards for a longer time. We formulated our proposal in a reinforcement-learning model, and we showed that our model could account for a wide range of existing neuronal and behavioral data, without appealing to ambiguous notions such as an explicit value for information. We suggest that such boosted anticipation significantly drives risk-seeking behaviors, most pertinently in gambling.

DOI: http://dx.doi.org/10.7554/eLife.13747.001

eLife digest

People, pigeons and monkeys often want to know in advance whether they will receive a reward in the future. This behaviour is irrational when individuals pay for costly information that makes no difference to an eventual outcome. One explanation is that individuals seek information because anticipating reward has hedonic value (it produces a feeling of pleasure). Consistent with this, pigeons are more likely to seek information when they have to wait longer for the potential reward. However, existing models cannot account for why this anticipation of rewards leads to irrational information-seeking.

In many situations, animals are uncertain about what is going to happen. Providing new information can produce a “prediction error” that indexes a discrepancy between what is expected and what actually happens. Iigaya et al. have now developed a mathematical model of information-seeking in which anticipation is boosted by this prediction error.

The model accounts for a wide range of previously unexplained data from monkeys and pigeons. It also successfully explains the behaviour of a group of human volunteers from whom Iigaya et al. elicited informational and actual decisions concerning uncertain and delayed rewards. The longer that the participants had to wait for possible rewards, the more avidly they wanted to find out about them. Further research is now needed to investigate the neural underpinnings of anticipation and its boosting by prediction errors.

DOI: http://dx.doi.org/10.7554/eLife.13747.002

Contributors

Naoshige Uchida: Role: Reviewing editor

Journal

Journal ID (nlm-ta): eLife

Journal ID (iso-abbrev): Elife

Journal ID (hwp): eLife

Journal ID (publisher-id): eLife

Title: eLife

Publisher: eLife Sciences Publications, Ltd

ISSN (Electronic): 2050-084X

Publication date (Electronic): 21 April 2016

Publication date Collection: 2016

Volume: 5

Affiliations

[1 ]deptGatsby Computational Neuroscience Unit , University College London , London, United Kingdom

[2 ]deptThe Wellcome Trust Centre for Neuroimaging , University College London , London, United Kingdom

[3 ]Max Planck UCL Centre for Computational Psychiatry and Ageing Research , London, United Kingdom

Harvard University , United States

Harvard University , United States

Author notes

kiigaya@ 123456gatsby.ucl.ac.uk

Article

Publisher ID: 13747

DOI: 10.7554/eLife.13747

PMC ID: 4866828

PubMed ID: 27101365

Copyright statement: © 2016, Iigaya et al

License:

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Product

Funding

Funded by: FundRef http://dx.doi.org/10.13039/501100000324, Gatsby Charitable Foundation;

Award Recipient : Kiyohito Iigaya Peter Dayan

Funded by: FundRef http://dx.doi.org/10.13039/100004440, Wellcome Trust;

Award ID: 091593/Z/10/Z

Award Recipient : Giles W Story Zeb Kurth-Nelson Raymond J Dolan

Funded by: FundRef http://dx.doi.org/10.13039/100004440, Wellcome Trust;

Award ID: 098362/Z/12/Z

Award Recipient : Giles W Story Zeb Kurth-Nelson Raymond J Dolan

Funded by: The Joint Initiative on Computational Psychiatry and Ageing Research between the Max Planck Society and University College London;

Award Recipient : Giles W Story Zeb Kurth-Nelson Raymond J Dolan

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Categories

Subject: Research Article

Subject: Neuroscience

Custom metadata

elife-xml-version 2.5

Author impact statement The anticipation of rewards turns out to have its own hedonic value, on top of that of the reward itself; a wide range of behavioral and neurophysiological data suggest that this anticipation is boosted by prediction errors.