Believing in dopamine

Abstract

Dopamine signals are widely thought to report reward prediction errors that drive learning in the basal ganglia. However, dopamine has also been implicated in a variety of probabilistic computations, such as encoding uncertainty and controlling exploration. These different facets of dopamine can be brought together under a common reinforcement learning framework. The key idea is that multiple sources of uncertainty impinge upon reinforcement learning computations: uncertainty about the state of the environment, the parameters of the value function, and the optimal action policy. Each of these sources plays a distinct role in the prefrontal/basal ganglia circuit for reinforcement learning, and are ultimately reflected in dopamine activity. The view that dopamine plays a central role in the encoding and updating of beliefs brings the classical prediction error theory into alignment with more recent theories of Bayesian reinforcement learning.