Activity patterns of serotonin neurons underlying cognitive flexibility

Abstract

Serotonin is implicated in mood and affective disorders. However, growing evidence suggests that a core endogenous role is to promote flexible adaptation to changes in the causal structure of the environment, through behavioral inhibition and enhanced plasticity. We used long-term photometric recordings in mice to study a population of dorsal raphe serotonin neurons, whose activity we could link to normal reversal learning using pharmacogenetics. We found that these neurons are activated by both positive and negative prediction errors, and thus report signals similar to those proposed to promote learning in conditions of uncertainty. Furthermore, by comparing the cue responses of serotonin and dopamine neurons, we found differences in learning rates that could explain the importance of serotonin in inhibiting perseverative responding. Our findings show how the activity patterns of serotonin neurons support a role in cognitive flexibility, and suggest a revised model of dopamine–serotonin opponency with potential clinical implications.

DOI: http://dx.doi.org/10.7554/eLife.20552.001

eLife digest

Serotonin is a molecule that plays various roles in the human body. In the brain, it is involved in regulating mood and emotions. Growing evidence suggests that serotonin also helps animals – including humans – adapt their behavior to changes in their environment. To allow for such behavioral flexibility, serotonin might promote changes in the underlying brain structures and activity.

In a type of learning known as ‘reversal learning’, for instance, it is necessary to adapt to a sudden change in a previously familiar environment. For example, if there were a road closure on a person’s way to work, they might want to learn to stop following their usual route and learn a new and better one. Previous research has shown that when serotonin signaling is reduced, people persevere. That is, they will keep following the old route even if it is no longer the best choice. How this process works is still largely unknown.

To start unraveling these mechanisms, Matias et al. trained mice in a reversal learning task while manipulating and recording the activity of the neurons that produce serotonin. The results showed that when the activity in serotonin neurons was experimentally blocked, the mice tended to keep looking for a reward that was no longer available. Then, by recording the activity of serotonin neurons, Matias et al. found that it was the surprise of discovering a change in a previously familiar environment that activates serotonin neurons. It did not matter whether the change was better or worse than expected. The findings suggest that together with dopamine, another molecule involved in learning from rewards, serotonin could play an important role during reversal learning.

One next step will be to determine if serotonin mainly stops perseverance in its tracks, or whether it works by helping to unlearn the old behavior, or a combination of both. In the future, this could further our understanding of depression, which can be viewed as a disorder characterized by patients being unable to adapt to adverse situations, leaving them trapped to repeat behaviors and thoughts that are not beneficial. Future studies could also build on these findings to guide the development of new treatments for depression that involve serotonin.

DOI: http://dx.doi.org/10.7554/eLife.20552.002