+1 Recommend
0 collections
      • Record: found
      • Abstract: found
      • Article: found
      Is Open Access

      Learning from Sensory and Reward Prediction Errors during Motor Adaptation

      1 , 2 , * , 1

      PLoS Computational Biology

      Public Library of Science

      Read this article at

          There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.


          Voluntary motor commands produce two kinds of consequences. Initially, a sensory consequence is observed in terms of activity in our primary sensory organs (e.g., vision, proprioception). Subsequently, the brain evaluates the sensory feedback and produces a subjective measure of utility or usefulness of the motor commands (e.g., reward). As a result, comparisons between predicted and observed consequences of motor commands produce two forms of prediction error. How do these errors contribute to changes in motor commands? Here, we considered a reach adaptation protocol and found that when high quality sensory feedback was available, adaptation of motor commands was driven almost exclusively by sensory prediction errors. This form of learning had a distinct signature: as motor commands adapted, the subjects altered their predictions regarding sensory consequences of motor commands, and generalized this learning broadly to neighboring motor commands. In contrast, as the quality of the sensory feedback degraded, adaptation of motor commands became more dependent on reward prediction errors. Reward prediction errors produced comparable changes in the motor commands, but produced no change in the predicted sensory consequences of motor commands, and generalized only locally. Because we found that there was a within subject correlation between generalization patterns and sensory remapping, it is plausible that during adaptation an individual's relative reliance on sensory vs. reward prediction errors could be inferred. We suggest that while motor commands change because of sensory and reward prediction errors, only sensory prediction errors produce a change in the neural system that predicts sensory consequences of motor commands.

          Author Summary

          It is thought that motor adaptation relies on sensory prediction errors to form an estimate of the perturbation. Here, we present evidence that motor adaptation can be driven by both sensory and reward prediction errors. We found that learning from sensory prediction error altered the predicted consequences of motor commands, leaving behind a sensory remapping, whereas learning from reward prediction error produced comparable change in motor commands, but did not produce a sensory remapping. It is possible that the neural basis of learning from sensory and reward prediction errors are distinct because they produce different generalization patterns.

          Related collections

          Most cited references 41

          • Record: found
          • Abstract: found
          • Article: not found

          Noise in the nervous system.

          Noise--random disturbances of signals--poses a fundamental problem for information processing and affects all aspects of nervous-system function. However, the nature, amount and impact of noise in the nervous system have only recently been addressed in a quantitative manner. Experimental and computational methods have shown that multiple noise sources contribute to cellular and behavioural trial-to-trial variability. We review the sources of noise in the nervous system, from the molecular to the behavioural level, and show how noise contributes to trial-to-trial variability. We highlight how noise affects neuronal networks and the principles the nervous system applies to counter detrimental effects of noise, and briefly discuss noise's potential benefits.
            • Record: found
            • Abstract: found
            • Article: not found

            A computational neuroanatomy for motor control.

            The study of patients to infer normal brain function has a long tradition in neurology and psychology. More recently, the motor system has been subject to quantitative and computational characterization. The purpose of this review is to argue that the lesion approach and theoretical motor control can mutually inform each other. Specifically, one may identify distinct motor control processes from computational models and map them onto specific deficits in patients. Here we review some of the impairments in motor control, motor learning and higher-order motor control in patients with lesions of the corticospinal tract, the cerebellum, parietal cortex, the basal ganglia, and the medial temporal lobe. We attempt to explain some of these impairments in terms of computational ideas such as state estimation, optimization, prediction, cost, and reward. We suggest that a function of the cerebellum is system identification: to build internal models that predict sensory outcome of motor commands and correct motor commands through internal feedback. A function of the parietal cortex is state estimation: to integrate the predicted proprioceptive and visual outcomes with sensory feedback to form a belief about how the commands affected the states of the body and the environment. A function of basal ganglia is related to optimal control: learning costs and rewards associated with sensory states and estimating the "cost-to-go" during execution of a motor task. Finally, functions of the primary and the premotor cortices are related to implementing the optimal control policy by transforming beliefs about proprioceptive and visual states, respectively, into motor commands.
              • Record: found
              • Abstract: found
              • Article: not found

              Learning of action through adaptive combination of motor primitives.

              Understanding how the brain constructs movements remains a fundamental challenge in neuroscience. The brain may control complex movements through flexible combination of motor primitives, where each primitive is an element of computation in the sensorimotor map that transforms desired limb trajectories into motor commands. Theoretical studies have shown that a system's ability to learn action depends on the shape of its primitives. Using a time-series analysis of error patterns, here we show that humans learn the dynamics of reaching movements through a flexible combination of primitives that have gaussian-like tuning functions encoding hand velocity. The wide tuning of the inferred primitives predicts limitations on the brain's ability to represent viscous dynamics. We find close agreement between the predicted limitations and the subjects' adaptation to new force fields. The mathematical properties of the derived primitives resemble the tuning curves of Purkinje cells in the cerebellum. The activity of these cells may encode primitives that underlie the learning of dynamics.

                Author and article information

                Role: Editor
                PLoS Comput Biol
                PLoS Computational Biology
                Public Library of Science (San Francisco, USA )
                March 2011
                March 2011
                10 March 2011
                : 7
                : 3
                [1 ]Department of Biomedical Engineering, Johns Hopkins School of Medicine, Baltimore, Maryland, United States of America
                [2 ]Department of Human Media Systems, The University of Electro-Communication, Chofu, Tokyo, Japan
                Northwestern University, United States of America
                Author notes

                Conceived and designed the experiments: JI RS. Performed the experiments: JI. Analyzed the data: JI. Contributed reagents/materials/analysis tools: RS. Wrote the paper: JI RS.

                Izawa, Shadmehr. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
                Page count
                Pages: 11
                Research Article

                Quantitative & Systems biology


                Comment on this article