SSCC TD: A Serial and Simultaneous Configural-Cue Compound Stimuli Representation for Temporal Difference Learning

There is no author summary for this article yet. Authors can add summaries to their articles on ScienceOpen to make them more accessible to a non-specialist audience.

Abstract

This paper presents a novel representational framework for the Temporal Difference (TD) model of learning, which allows the computation of configural stimuli – cumulative compounds of stimuli that generate perceptual emergents known as configural cues. This Simultaneous and Serial Configural-cue Compound Stimuli Temporal Difference model (SSCC TD) can model both simultaneous and serial stimulus compounds, as well as compounds including the experimental context. This modification significantly broadens the range of phenomena which the TD paradigm can explain, and allows it to predict phenomena which traditional TD solutions cannot, particularly effects that depend on compound stimuli functioning as a whole, such as pattern learning and serial structural discriminations, and context-related effects.

Related collections

Most cited references 35

Record: found
Abstract: found
Article: not found

A framework for mesencephalic dopamine systems based on predictive Hebbian learning.

PR Montague, P. Dayan, TJ Sejnowski (1996)

We develop a theoretical framework that shows how mesencephalic dopamine systems could distribute to their targets a signal that represents information about future expectations. In particular, we show how activity in the cerebral cortex can make predictions about future receipt of reward and how fluctuations in the activity levels of neurons in diffuse dopamine systems above and below baseline levels would represent errors in these predictions that are delivered to cortical and subcortical targets. We present a model for how such errors could be constructed in a real brain that is consistent with physiological results for a subset of dopaminergic neurons located in the ventral tegmental area and surrounding dopaminergic neurons. The theory also makes testable predictions about human choice behavior on a simple decision-making task. Furthermore, we show that, through a simple influence on synaptic plasticity, fluctuations in dopamine release can act to change the predictions in an appropriate manner.

0 comments Cited 342 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: found

Is Open Access

Dopamine signals for reward value and risk: basic and recent data

Wolfram Schultz (2010)

Background Previous lesion, electrical self-stimulation and drug addiction studies suggest that the midbrain dopamine systems are parts of the reward system of the brain. This review provides an updated overview about the basic signals of dopamine neurons to environmental stimuli. Methods The described experiments used standard behavioral and neurophysiological methods to record the activity of single dopamine neurons in awake monkeys during specific behavioral tasks. Results Dopamine neurons show phasic activations to external stimuli. The signal reflects reward, physical salience, risk and punishment, in descending order of fractions of responding neurons. Expected reward value is a key decision variable for economic choices. The reward response codes reward value, probability and their summed product, expected value. The neurons code reward value as it differs from prediction, thus fulfilling the basic requirement for a bidirectional prediction error teaching signal postulated by learning theory. This response is scaled in units of standard deviation. By contrast, relatively few dopamine neurons show the phasic activation following punishers and conditioned aversive stimuli, suggesting a lack of relationship of the reward response to general attention and arousal. Large proportions of dopamine neurons are also activated by intense, physically salient stimuli. This response is enhanced when the stimuli are novel; it appears to be distinct from the reward value signal. Dopamine neurons show also unspecific activations to non-rewarding stimuli that are possibly due to generalization by similar stimuli and pseudoconditioning by primary rewards. These activations are shorter than reward responses and are often followed by depression of activity. A separate, slower dopamine signal informs about risk, another important decision variable. The prediction error response occurs only with reward; it is scaled by the risk of predicted reward. Conclusions Neurophysiological studies reveal phasic dopamine signals that transmit information related predominantly but not exclusively to reward. Although not being entirely homogeneous, the dopamine signal is more restricted and stereotyped than neuronal activity in most other brain structures involved in goal directed behavior.

0 comments Cited 174 times – based on 0 reviews      Review now

Bookmark

Record: found
Abstract: found
Article: not found

Similarity and discrimination: a selective review and a connectionist model.

John M. Pearce (1994)

The 1st part of this article evaluates the extent to which 2 elemental theories of conditioning, stimulus sampling theory and the Rescorla-Wagner (1972) theory, are able to account for the influence of similarity on discrimination learning. A number of findings are reviewed that are inconsistent with predictions derived from these theories, either in their present form or in various modified forms. The 2nd part of the article is concerned with developing an alternative, configural account for discrimination learning. In contrast to previous configural theories, the present version is set within the framework of a connectionist network.

0 comments Cited 111 times – based on 0 reviews      Review now

Bookmark

All references

Author and article information

Contributors

Gregg Roman: Role: Editor

Journal

Journal ID (nlm-ta): PLoS One

Journal ID (iso-abbrev): PLoS ONE

Journal ID (publisher-id): plos

Journal ID (pmc): plosone

Title: PLoS ONE

Publisher: Public Library of Science (San Francisco, USA )

ISSN (Electronic): 1932-6203

Publication date Collection: 2014

Publication date (Electronic): 23 July 2014

Volume: 9

Issue: 7

Electronic Location Identifier: e102469

Affiliations

[1 ]Centre for Computational and Animal Learning Research, St Albans, United Kingdom

[2 ]Institute for Complex Systems Simulations, University of Southampton, Southampton, United Kingdom

[3 ]Department of Computer Science, City University London, London, United Kingdom

[4 ]School of Psychology, University of Nottingham, Nottingham, United Kingdom

[5 ]Institute of Neuroscience, Newcastle University, Newcastle upon Tyne, United Kingdom

University of Houston, United States of America

Author notes

* E-mail: e.mondragon@ 123456cal-r.org

Competing Interests: The authors have declared that no competing interests exist.

Conceived and designed the experiments: EM JG EA CB DJJ. Performed the experiments: EM JG EA CB DJJ. Analyzed the data: EM JG EA CB DJJ. Contributed reagents/materials/analysis tools: EM JG EA CB DJJ. Wrote the paper: EM JG EA CB DJJ.

Article

Publisher ID: PONE-D-14-02534

DOI: 10.1371/journal.pone.0102469

PMC ID: 4108321

PubMed ID: 25054799

SO-VID: 79aed593-e3ec-4d56-b1b6-5d634d0f593b

License:

This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

History

Date received : 6 February 2014

Date accepted : 18 June 2014

Page count

Pages: 24

Funding

Jonathan Gray's contribution has been supported by an EPSRC Doctoral Training Centre grant (EP/G03690X/1). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

SSCC TD: A Serial and Simultaneous Configural-Cue Compound Stimuli Representation for Temporal Difference Learning

Read this article at

Abstract

Related collections

PLOS Climate

Most cited references 35

A framework for mesencephalic dopamine systems based on predictive Hebbian learning.

Dopamine signals for reward value and risk: basic and recent data

Similarity and discrimination: a selective review and a connectionist model.

Author and article information

Contributors

Journal

Affiliations

Author notes

Article

History

Page count

Funding

Categories

Comments

Comment on this article

Similar content 89

Cited by 2

Most referenced authors 162