AI/ML Wiki

schultz-1993-responses-of-monkey-dopamine-neurons

3 min read

Source Summary: Schultz et al. 1993

Full title: Responses of Monkey Dopamine Neurons to Reward and Conditioned Stimuli during Successive Steps of Learning a Delayed Response Task
Authors: Wolfram Schultz, Paul Apicella, Tomas Ljungberg
Affiliation: Institut de Physiologie, Université de Fribourg, Switzerland
Venue: The Journal of Neuroscience, March 1993, 13(3): 900–913
PDF: papers/schultz-1993-responses-of-monkey-dopamine-neurons.pdf

Contributions

  1. Reward-to-CS transfer during learning — dopamine neuron responses to primary reward diminish as the animal learns, while responses to conditioned stimuli (CSs) predicting reward emerge.
  2. Learning-stage dependence — during acquisition of each new subtask, ~25% of DA neurons respond to reward delivery; this drops to ~9% once the task is established.
  3. Population response during learning — average population activity shows a clear reward response during learning but not after established performance.
  4. Phasic alerting function — DA neurons respond transiently to behaviorally significant stimuli (instruction cue, trigger stimulus) regardless of learning stage; no sustained delay-period activity.

Methodology

Animals and Tasks

  • Two male Macaca fascicularis monkeys (3.5 kg), one development monkey
  • Previously trained on simple reaction time task (Ljungberg et al., 1992)
  • Trained through three successive spatial tasks:
    1. Spatial choice task: instruction + trigger lights appear simultaneously; animal chooses lever
    2. Instructed spatial task: instruction 1 sec before trigger; animal must hold then move
    3. Spatial delayed response task: instruction 1 sec, then extinguished; trigger appears 2.5–3.5 sec later; animal must remember target location (working memory)

Electrophysiology

  • Single-unit recording with moveable microelectrodes
  • Recording in substantia nigra pars compacta (area A9) and ventral tegmental area (A10, A8)
  • 76 DA neurons recorded during learning; 163 neurons during established performance
  • DA neurons identified by: long-duration biphasic waveform, low spontaneous firing rate (2–5 Hz), broad action potentials

Stimuli

  • Instruction light (LED above target lever) — conditioned stimulus 1, predicts target location
  • Trigger light (central LED) — conditioned stimulus 2, conditioned incentive predicting reward and timing response
  • Liquid reward (0.15–0.20 ml apple juice), 500 ms after correct touch

Results

Reward responses

  • During learning: 25% of 76 DA neurons showed phasic reward responses
  • After learning established: only 9% of 163 neurons responded to reward
  • Area A10 (VTA) showed significantly more reward responses than A9 (SNc) during learning

Conditioned stimulus responses

  • Instruction cue: 56% of neurons during learning, 44% after learning
  • Trigger stimulus: 36% during learning, 40% after learning
  • CS responses were phasic (short-latency bursts), not sustained

No delay-period activity

  • Zero DA neurons showed sustained activity in the 2.5–3.5 sec delay between instruction and trigger
  • Unlike striatal and frontal cortex neurons, DA neurons do not encode working memory or spatial expectation

Key Findings

  1. DA neuron reward responses decrease with learning — transferred to CS once contingency learned
  2. DA neuron CS responses persist even after task is established — reflecting the behavioral significance of the CS, not novelty
  3. DA neurons respond to attentional/motivational salience of stimuli, not to the specific spatial or temporal properties of the task
  4. The function is phasic and transient — dopamine encodes events, not states

Interpretation

The paper does not yet use the TD (temporal difference) learning framework explicitly, but the data pattern is consistent with it:

  • Reward responses during learning = prediction error at reward time (when reward is unexpected)
  • Reduction after learning = reward becomes predicted; no prediction error at reward time
  • CS responses = prediction error transferred to earlier, predictive stimulus

This was the empirical foundation that Schultz et al. (1997) would later interpret formally as prediction error signaling.

Key References

  • Schultz 1986 — earlier observations of DA responses to reward
  • Romo & Schultz 1990 — DA responses to conditioned incentive stimuli
  • Ljungberg, Apicella & Schultz 1992 — DA responses during simple reaction time task learning (immediate precursor study)

Graph View

Backlinks