JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2017, 107, 301–320 NUMBER 3 (MAY)

BEHAVIOR ANALYSIS AND NEUROSCIENCE: COMPLEMENTARY DISCIPLINES

JOHN W. DONAHOE
UNIVERSITY OF MASSACHUSETTS/AMHERST

Behavior analysis and neuroscience are disciplines in their own right but are united in that both are
subfields of a common overarching field—biology. What most fundamentally unites these disciplines is
a shared commitment to selectionism, the Darwinian mode of explanation. In selectionism, the order
and complexity observed in nature are seen as the cumulative products of selection processes acting
over time on a population of variants—favoring some and disfavoring others—with the affected variants
contributing to the population on which future selections operate. In the case of behavior analysis, the
central selection process is selection by reinforcement; in neuroscience it is natural selection. The two
selection processes are inter-related in that selection by reinforcement is itself the product of natural
selection. The present paper illustrates the complementary nature of behavior analysis and neurosci-
ence through considering their joint contributions to three central problem areas: reinforcement—
including conditioned reinforcement, stimulus control—including equivalence classes, and memory—
including reminding and remembering.
Key words: reinforcement, selectionism, neuroscience, conditioned reinforcement, stimulus control,
equivalence classes, memory

This article illustrates the complementary
nature of behavior analysis and neuroscience,
each a scientific discipline in its own right, but
each informed and enriched by the other.
The underlying premise is that the advantages
of the integration of behavior analysis and
neuroscience is foreshadowed by the earlier
benefits of integrating the sciences of evolu-
tion and heredity, now known as the Modern
Synthesis (Dobzhansky, 1937, cf. Donahoe,
2003). Of this earlier synthesis, the biologist
and philosopher of science Paul Gayon wrote,
“If there is a key event in the history of Dar-
winism, it must be its confrontation with Men-
delism. It was through its contact with the new
science of heredity that the theory of selection
became truly intelligible” (Gayon, 1998,
p. 253). As it was with evolution through natu-
ral selection and genetics, so it may well be
with selection by reinforcement and its neural
mechanisms.
Efforts to integrate findings from behavior
analysis and neuroscience are nothing new,
of course. Edward Thorndike’s earliest writ-
ings emphasized the importance of not only
the Law of Effect but also of what he termed
the “Law of Acquired Brain Connections”
Address correspondence to: Prof. John W. Donahoe,
Department of Psychological and Brain Sciences, Univer-
sity of Massachusetts/Amherst, Amherst, MA 01002; Email:
jdonahoe@psych.umass.edu
doi: 10.1002/jeab.251
(Thorndike, 1903, p. 165; cf. Donahoe,
1999), although he lamented the absence of
direct experimental knowledge of the latter.
Donald Hebb was perhaps the first American
psychologist to attempt a wide-ranging inte-
gration of the two disciplines in his Organiza-
tion of Behavior (1949), but again direct
knowledge of the neural processes remained
largely absent. Hebb’s proposal that the con-
nections between two neurons increased in
strength when they were co-active was simply
a restatement in neural terms of the behav-
iorally established contiguity requirement.
Skinner regarded behavior analysis as “a rig-
orous, extensive, and rapidly advancing
branch of biology” (1974, p. 255) but noted
two prerequisites for such an integration. Of
behavior: “A rigorous description at the level
of behavior is necessary for the demonstra-
tion of a neurological correlate.” And, of neu-
roscience: “A science of the nervous system
will someday start from the direct observation
of neural processes … It is with such a sci-
ence that the neurological point of view must
be concerned if it is to offer a convincing
‘explanation’ (sic) of behavior” (Skinner,
1938, p. 422). These two prerequisites are
becoming increasingly approximated, if not
satisfied, by the present state of development
of each discipline.
Before describing specific examples of the
salutary effects of integrating behavior analysis
and neuroscience, two general orienting

© 2017 Society for the Experimental Analysis of Behavior

301
302 JOHN W. DONAHOE
principles are identified. First, both evolution-
ary biology and behavior analysis are
selectionist sciences (Catania, 1987; Donahoe,
2003; McDowell, 2013a; Palmer & Donahoe,
1992; Staddon, 1983). That is, the diverse and
potentially complex outcomes of evolution
and reinforcement are the cumulative pro-
ducts of relatively simple selection processes
acting over time. Any given instance of selec-
tion occurs under conditions that exist at that
moment, conditions which may or may not be
shared with future moments. Strictly speaking,
selection processes “prepare” the organism for
conditions that existed in the past, not in the
future except in so far as the future resembles
the past. Second, the cumulative products of
selection processes are the net effects of many
individual instances of selection, whether
reproductive fitness acting on a population of
different organisms in the case of evolution or
reinforcement acting on a population of dif-
ferent environment-behavior relations of a sin-
gle organism in the case of behavior
(Donahoe, 2003). The nature of the popula-
tion on which the selecting events act—
different organisms in the case of evolution or
different behaviors of the same organism in
the case of behavior analysis—is what most
fundamentally distinguishes much of psychol-
ogy from behavior analysis. Psychology often
seeks its principles through averaging steady-
state observations from different organisms, a
method appropriate for population genetics
but not necessarily for a science of behavior
(e.g., Estes, 1956; Sidman, 1960).
In subsequent sections, the relevant behav-
ioral findings are first briefly summarized fol-
lowed by the neural mechanisms that
mediate that behavior. Skinner’s admonition
that a “rigorous description at the level of
behavior” must precede “the demonstration
of a neurological correlate” is amply illus-
trated. The first findings considered are
those related to the reinforcement process
itself. Just as natural selection informs all of
evolutionary biology, so a properly under-
stood conception of selection by reinforce-
ment should illuminate the full range of
behavior from simple conditioning to com-
plex human behavior. The discussion of
selection by reinforcement, including condi-
tioned reinforcement, is followed by sections
on stimulus control (including equivalence
classes) and memory. The body of the article
is self-contained, but supplementary informa-
tion is occasionally introduced by means of
footnotes.
Selection by Reinforcement
The approach to reinforcement described
here is a logical extension of Skinner’s view
that selection occurs at the moment when a
reinforcing stimulus (that is, a reinforcer)
appears in close proximity to a stimulus in the
Pavlovian procedure or to a response in the
operant procedure. Figure 1 depicts these two
temporal relations. Note, however, that in the
Pavlovian procedure some response necessar-
ily occurs in proximity to the reinforcer
whereas in the operant procedure some stimu-
lus necessarily occurs in proximity to the rein-
forcer. Thus, what truly distinguishes between
the two procedures is not whether a stimulus
or a response precedes the reinforcer, but the
reliability with which that event precedes the
reinforcer. The classical and operant proce-
dures clearly differ, but whether the proce-
dural difference portends a fundamental
difference in the conditioning process is a sep-
arate matter.
Skinner implicitly appreciated this point in
his demonstrations of superstitious condition-
ing. Superstitions of the first kind occurred
when the frequency of a response changed if
it occurred by chance before a reinforcing
Fig. 1. Diagram illustrating the similarity and differ-
ence between the Pavlovian and operant procedures. In
both procedures, a reinforcing stimulus (SR) is introduced
that elicits a response (Relicited). In the Pavlovian proce-
dure SR most frequently follows a specific stimulus whereas
in the operant procedure SR most frequently follows a spe-
cific response (Rj) . Relicited typically goes unmeasured in
the operant procedure. Note, however, that stimuli and
responses necessarily precede the reinforcer in both
procedures.
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
stimulus (Skinner, 1948; cf. Staddon & Sim-
melhag, 1971). Superstitions of the second
kind occurred when a reinforced response
occurred by chance during a stimulus and the
response then came under the control of that
stimulus (Morse & Skinner, 1957). Viewed in
this way, the distinction between Pavlovian
and operant conditioning is procedural, and
not necessarily a difference in the condition-
ing process itself. Skinner was open to this
possibility. However, he chose not to pursue
the commonalities in the conditioning proc-
ess in the two procedures but, instead, to
explore the profound differences in the con-
sequences of the procedural difference. In
the Pavlovian procedures, an elicited response
(the respondent) comes under the control of
arbitrary stimuli1 whereas in the operant pro-
cedure the entire behavioral repertoire of the
organism potentially comes under the control
1
“Experimental analysis,” as Skinner used the term,
referred to circumstances in which all of the effects of the
variables relevant to a phenomenon are either manipu-
lated, measured, or controlled. This idealized set of cir-
cumstances is only approximated even in the laboratory.
Classical procedures in which the conditioned response
and the elicited response closely resemble one another—
such as the salivary response of the dog (Pavlov, 1927/
1960) and the nictitating-membrane response of the rab-
bit (Gormezano, 1966) or the pigeon (Stickney, Dona-
hoe, & Carlson, 1981)—very closely approximate these
requirements. However, other classical procedures reveal
the more complex outcomes that can be produced by the
conditioning process. As one example, the conditioned-
suppression procedure devised by one of Skinner’s stu-
dents (Estes & Skinner, 1941) demonstrated that an
unmonitored unconditioned response (the response to
shock) may interfere with the execution of a monitored
operant (bar pressing), thereby providing an indirect but
sensitive measure of conditioning. This procedure eventu-
ally led to the important discovery of the discrepancy
requirement, which is described subsequently (Kamin,
1968; Rescorla, 1968). As a second example of a complex
outcome produced by a classical procedure, the response
measured after conditioning may sometimes appear to be
the opposite to the response to the US: For example, a con-
ditioned stimulus paired with the injection of glucose pro-
duces a decrease in circulating glucose when the stimulus is
presented alone. However, the true unconditioned
response to the injection of glucose is not an increase in
circulating glucose, but a decrease in the glucose liberated
from glucose-storing organs when neurons detect an
increase in exogenous glucose (see Eickelboom & Stewart,
1982). In general, pre-existing homeostatic and other
mechanisms resulting from natural selection must be
taken into account when interpreting the results of any
conditioning procedure (cf. Dworkin, 1993; Solomon &
Corbit, 1974).
303
of arbitrary stimuli, thereby opening the path
to the emergence of complex behavior. Skin-
ner’s recognition of the possible unity of the
selection process was acknowledged without
dissent in his very early writings (Skinner,
1938, p. 111): “Hilgard (1937) … points out
that … reinforcement is essentially the same
process in both [procedures]” and “Mowrer
(1937) holds out that … the two processes
may eventually be reduced to a single
formula.”
Behavioral Findings
Contiguity. The first variable identified as
critical to selection by reinforcement was the
temporal relation between the reinforcer and
the environment (the conditioned stimulus, or
CS) in the Pavlovian procedure or the behav-
ioral event (the operant) in the operant pro-
cedure. In the Pavlovian procedure, an
effective interval is typically described as no
more than a few seconds (e.g., Smith, Cole-
man, & Gormezano, 1969). The effect of vary-
ing the interval between the operant and the
reinforcer is more difficult to analyze experi-
mentally because the onset of the operant can-
not be controlled and the events that
intervene between the operant and the rein-
forcer may include other instances of the
operant, as in intermittent schedules of rein-
forcement. In addition, stimuli that function
as conditioned reinforcers may bridge the gap
between the operant and the reinforcer
(e.g., Catania, 1971; Lattal & Gleeson, 1990).
In his initial operant procedure, Skinner
sought to minimize these intrusions by imme-
diately following the operant by the “click” of
a feeder, an auditory stimulus that had previ-
ously immediately preceded the delivery of
food according to a Pavlovian procedure
(Skinner, 1938, p. 53). A commitment to tem-
poral contiguity was also expressed in Ferster
and Skinner’s treatment of schedules of rein-
forcement: “The only contact between [the
schedule] and the organism occurs at the
moment (emphasis added) of reinforcement....
Under a given schedule of reinforcement, it
can be shown that at the moment of reinforce-
ment a given set of stimuli will usually prevail.
A schedule is simply a convenient way of
arranging this.” (Ferster & Skinner, 1957,
pp. 2-3).
304 JOHN W. DONAHOE
The general conclusion is that a reinfor-
cing stimulus produces changes in stimulus
control in the Pavlovian procedure and
changes in response frequency in the oper-
ant procedure only over relatively brief inter-
vals of time.2 As Skinner put it, “To say that
a reinforcement is contingent upon a
response may mean nothing more than that
it follows the response” (Skinner,
1948, p. 168).
Behavioral discrepancy. With the advent of
the blocking procedure, it became clear that,
although contiguity was necessary, it was not
sufficient to produce selection by reinforce-
ment. As demonstrated initially with a Pavlov-
ian procedure (Kamin, 1968; 1969) and
subsequently with an operant procedure (vom
Saal & Jenkins, 1970), something in addition
to temporal contiguity was required.3 Studies
showed that if a stimulus or a response was fol-
lowed by a putative reinforcer with a temporal
relation known to be effective for condition-
ing, but that the stimulus or response was
accompanied by another stimulus that had
previously preceded the same reinforcer, then
the putative reinforcer would not be effective.
Prior conditioning blocked the acquisition of a
new environment–behavior (E–B) relation if
the same reinforcer had previously occurred
in that context. Blocking has also been shown
2
The statement that conditioning in the classical proce-
dure occurs over only brief temporal intervals between the
CS and the reinforcer is based on procedures in which the
reinforcer-elicited response has a short latency and brief
duration. Findings reported later indicate that it is not the
temporal relation of the CS to the reinforcing stimulus
per se that is critical but the relation of the CS to the
behavior evoked by the reinforcer. For example, in taste
aversions a novel taste (CS) followed by the ingestion of a
nonlethal poison becomes aversive even though the behav-
ioral effects (Relicited) of ingestion occur some hours later.
In the evolutionary history of organisms, gustatory and
olfactory stimuli inevitably preceded the gastric conse-
quences of ingestion. This conjunction has permitted the
selection of neural structures relating such stimuli to gas-
tric responses. Neural traces of these stimuli endure and
are then contiguous with the gastric consequences
(Chambers, 1990). In respects other than the temporal
disjunction between the taste CS and the gastric Relicited to
the poison US, findings from the conditioning of taste
aversions are consistent with those from other condition-
ing preparations (Domjan & Galef, 1983).
3
Several other studies conducted around the time of
these critical experiments pointed toward the insufficiency
of temporal contiguity for conditioning (e.g., Johnson,
1970), but their implications were not fully appreciated
(cf. Williams, 1975).
to occur with conditioned reinforcers. Using a
concurrent schedule in which one response
produced a visual stimulus that had previously
been paired with a reinforcer and a second
response produced a different visual stimulus
that had been paired equally often with the
same reinforcer but only in the presence of a
previously conditioned stimulus, responding
was maintained for only the first response
(Palmer, 1986).
What is the nature of the additional varia-
ble? Associationist psychology proposed that a
given reinforcer would support only a fixed
maximum association value and that, because
prior conditioning had already attained that
maximum value, no further increases in associ-
ative strength were possible (Rescorla &
Wagner, 1972). The formalization of this pro-
posal in the Rescorla-Wagner model has
proven exceptionally fruitful (Siegel & Allan,
1996, cf. Papini & Bitterman, 1990), especially
when implemented in real-time computer
simulations (e.g., Donahoe, Burgos, & Palmer,
1993; Sutton & Barto, 1981). The model is
silent, however, with respect to the biobeha-
vioral mechanisms that implement it
(Donahoe, 1984). The physical events that are
available to an organism on its first exposure
to a reinforcing stimulus are the stimulus itself
and the responses that are elicited by that
stimulus. Subsequent research with the block-
ing procedure revealed that when the reinfor-
cing stimulus remained constant but, through
various means, the response elicited by
the reinforcer was changed, conditioning
occurred to a stimulus that would have other-
wise been blocked (Brandon, Betts, & Wagner,
1994; Stickney & Donahoe, 1983). In short,
blocking was prevented when the uncondi-
tioned response (UR) was changed. Based on
these findings, the second requirement for
selection by reinforcement may be described
as follows: The reinforcing stimulus must not
only be contiguous but it must also evoke a
change in the behavior that would otherwise
occur at that moment (Donahoe et al., 1993;
Donahoe, Crowley, Millard, & Stickney, 1982).
This is known as the behavioral-discrepancy
requirement.
The importance of the behavior evoked by
the reinforcing stimulus was further con-
firmed by other work demonstrating that the
critical temporal relation in the Pavlovian
procedure was not between the CS and the
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
US as conventionally understood, but
between the CS and the behavior evoked by
that stimulus, that is, the UR (Donahoe &
Vegas, 2004). Using the throat-movement
preparation of the pigeon, water or air
injected into the oral cavity of a restrained
pigeon evokes a UR of a sufficiently long
latency (approximately 200 ms) and duration
(approximately 2 s) to permit the temporal
relation of the CS to the US and to the UR
to be unconfounded. It was found that
whether the CS occurred before the US–UR
(the standard forward-conditioning proce-
dure), only between the CS and the UR, or
only after the onsets of both the CS and
UR (but during the UR), the level of condi-
tioned responding to the CS was the same
when measured on CS-alone test trials.
(Appropriate controls were instituted to rule
out alternative interpretations such as sensiti-
zation.) Thus the CS–UR relation, not the
CS–US relation, is the critical temporal rela-
tion for conditioning in the classical proce-
dure. The apparent importance of the CS–US
relation was based largely on procedures in
which the US elicited a short latency, brief
duration UR, thereby confounding the CS–
US and CS–UR temporal relations.
These findings provide an interpretation of
a number of otherwise problematic findings
(Donahoe & Vegas, 2004). Examples include:
(a) how the same CS–US temporal relation
conditions some responses but not others
when the URs differ in latency or duration
(e.g., Betts, Brandon, & Wagner, 1996),
(b) how a backward conditioning arrange-
ment (US–CS) conditions long-duration
URs, such as autonomic responses, but not
short-duration URs, such as eye blinks
(e.g., Albert & Ayres, 1997; McNish, Betts,
Brandon, & Wagner, 1997), and (c) how a
backward US–CS arrangement conditions
components of a temporally extended sequ-
ence of responses that are contiguous with
the CS (e.g., Silva, Timberlake, & Cevik,
1998). The behavioral-discrepancy require-
ment is also consistent with the a priori
conjecture that the ancestral selecting envi-
ronment (that is, natural selection) would
more likely produce a conditioning process
that was sensitive to the expressed behavior of
an organism than to its mere perception of a
stimulus. Thomas Huxley’s words resonate:
“The great end of life is not thought but
305
action.”4 The view that the same conditioning
process, requiring both contiguity and dis-
crepancy, is engaged by the Pavlovian and
operant procedures is summarized by the uni-
fied reinforcement principle (Donahoe
et al., 1993).
Neuroscientific Findings
The present discussion of neural mechan-
isms focuses on the neural systems that imple-
ment the reinforcement process (cf. Kirk,
Babtie, & Stumpf, 2015). As a result, the
emphasis is generally upon the net effects of
the neuroanatomical and cellular components
(neurons) of these systems and not the intra-
cellular processes themselves. A neural system
refers to those brain structures and their inter-
connections that are most relevant to the
behavior under consideration. The neural sys-
tem of concern here implements reinforce-
ment in the operant-conditioning procedure.
Figure 2 is a diagram of a lateral view of the
human cerebral cortex showing (with dashed
lines) the subcortical structures and intercon-
nections that are central to implementing
reinforcement. The general flow of neural
activity begins with stimulation of the senses,
such as vision and audition. This stimulation
4
The following section of the paper takes the position
that an analysis of moment-to-moment events reveals the
dynamical processes of which relations between environ-
mental and behavioral events observed over more
extended periods of time are the product. It is generally
agreed that stable molar relations, as useful as they may be
for some purposes, are silent with respect to the more
“molecular” processes of which they are the asymptotic
expression (e.g., Donahoe, 2012; Marr, 1992; Staddon,
2014). In addition, studies indicate that at least some
molar E-B relations, such as matching, may be the con-
certed product of multiple discriminated operants
(e.g., Crowley & Donahoe, 2004; Killeen, 2015; MacDonall,
2009; Pliskoff, 1971). Indeed, computer simulations that
implement moment-to-moment processes have yielded
asymptotic results that reproduce some molar relations
such as matching (e.g., Calvin & McDowell, 2016; McDo-
well, 2013a) and the effects of varying the C/T ratio on
acquisition (Burgos, 2005), where C is the length of the
CS-US interval and T is the total session length (Gibbon &
Balsam, 1981). A conceptually independent but often con-
flated difference between theoretical approaches to behav-
ior theory concerns whether it is sufficient for a theory to
serve as a guide for the behavior of the theorist
(e.g., Gallistel & King, 2009; Jensen, Ward, & Balsam,
2013; cf. Donahoe, 2010) or whether theory should also
embody the biobehavioral processes that underlie the
functional relations captured by the theory (McDowell,
2013b; cf., Donahoe, 2013).
306 JOHN W. DONAHOE
Fig. 2. Lateral view of the human cerebral cortex show-
ing the major cortical regions and, using dashed lines, the
subcortical structures—nucleus accumbens (NAC) and
ventral tegmental area (VTA)—and pathways central to
the neural mechanisms of selection by reinforcement.
(The divisions of the cortex designated in the diagram are
for heuristic purposes only. For example, the region desig-
nated primary sensory cortex is largely the primary visual
cortex and the region designated sensory-association cor-
tex also includes the primary auditory cortex of the tempo-
ral lobe.)
activates neurons in the primary sensory cor-
tices subsequent to interactions involving vari-
ous subcortical structures (not shown). Activity
in primary sensory neurons propagates to
sensory-association cortex whose polymodal
neurons respond to various combinations of
their inputs from neurons in primary sensory
and sensory-association cortices. Sensory and
polymodal neurons activate, in turn, neurons
in prefrontal and motor cortex (among
others), with the latter leading most directly to
behavior via the striatum and various other
subcortical and spinal structures (not shown).
Neurons in prefrontal cortex, which includes
the adjacent supplementary motor and premo-
tor areas, coordinate neural activity in the
motor cortex. This account of the neural sys-
tems mediating E–B relations is simplified and
does not reflect other, often reciprocal, path-
ways between structures—some of which are
considered in a later section. The account is
suitable for present purposes however.
Given the many neurons in the human
brain—perhaps 1012, the many potential con-
nections to a single neuron—perhaps 103 in
the case of motor neurons, and the “spontane-
ous” activity of many cortical neurons, natural
selection faced a formidable challenge: What
mechanism could be devised whereby the
strengths of connections between neurons
mediating a specific reinforced E–B relation
were strengthened? As with any selection proc-
ess, the challenge cannot be met on a single
occasion and is fallible (Donahoe, 2003;
Palmer & Donahoe, 1992). (This challenge is
known as the “assignment-of-credit” problem
in neural-network research.) The mechanism
that evolved has the effect of disproportion-
ately, but not exclusively, strengthening the
“right” connections. (Personification of selec-
tion processes—whether natural selection or
selection by reinforcement—is an expository
device only, of course.)
What is the mechanism that permits selec-
tion by reinforcement of a wide range of spe-
cific E–B relations? When a response is
followed by a reinforcer, it must be true that
pathways sufficient to enable the reinforced
E–B relation are among the many that are
active at that moment. However, the neural
mechanisms implementing the reinforcement
process must not only affect the pathways of
that specific reinforced E–B relation but also
potentially a wide range of other pathways that
might mediate other E–B relations under dif-
ferent contingencies of reinforcement. Such a
mechanism is required if almost any stimulus
is to potentially control almost any operant.
Dopamine and the selection of pathways
mediating reinforced E–B relations. The neu-
ral capability for widespread effects of rein-
forcement is accomplished through the
liberation and subsequent diffusion of the
neuromodulator dopamine (DA) along pro-
jections from cells in the ventral tegmental
area (VTA) to the prefrontal and motor cor-
tices (among others). The diffusion of DA
allows a reinforcer to affect a wide range of
synapses and, thereby a wide range of poten-
tial E–B relations mediated by the pathways
involving those synapses.5 See Figure 2. For
purposes of the current discussion, the neural
inputs to the prefrontal and motor cortices
from sensory areas are presumed to be suffi-
cient to control the reinforced behavior. This
matter is considered further in a later section.
5
Although dopamine plays a critical role in reinforce-
ment, not all instances of long-lasting changes in synaptic
plasticity are dopamine-dependent (e.g., Bauer, Schafe, &
LeDoux, 2002).
 BEHAVIOR ANALYSIS AND NEUROSCIENCE 307

DA is generally designated a neuromodula-

tor rather than a neurotransmitter because it
alters the effect of other neurotransmitters on
the functioning of neurons. One may specu-
late that the central role of DA in reinforce-
ment should not be completely surprising
given that DA also plays an important role in
the digestive process where it governs gut
motility (Li, Schmauss, Cuenca, Ratcliffe, &
Gershon, 2006). This leads to the conjecture
that the digestion of nutrients and reinforce-
ment were intertwined in evolutionary history.
The neural activity initiated by receptors sti-
mulated by taste and smell and by sexual activ-
ity activate DA neurons via pathways traversing
various intermediate subcortical structures.
Drugs of abuse also typically activate DA neu-
rons (e.g., Volkow, Fowler, & Wang, 2007;
Wise, 2002). The widely projecting outputs
(axons) of VTA neurons liberate DA that dif-
fuses from varicosities along their lengths. DA Fig. 3. The frequency of firing of dopaminergic
molecules remain present for several seconds (DA) neurons in the ventral tegmental area (VTA) during
before being degraded (Yagishita et al., 2014). a differential operant-conditioning procedure. Panel A
shows the DA response to the reinforcing stimulus (SR) at
The relatively short-lived presence of DA is the outset of conditioning. Panel B shows the DA response
consistent with the contiguity requirement. to the discriminative stimulus (SD) and SR on a reinforced
DA also plays a critical role in the discrepancy trial after conditioning. Panel C shows the DA response to
requirement as described shortly. SD on a trial after conditioning in which the reinforcer
was omitted. See the text for additional information. (Data
Figure 3 indicates the frequency of firing from Schultz, 1997; Schultz, Apicella, & Ljungberg, 1993.)
of VTA-DA neurons in our fellow primate,
the crab-eating monkey. The task was a differ-
ential conditioning procedure in which press-
ing one of two levers was reinforced with
apple juice depending on which of two function (Dinsmoor, 1950; cf. Williams, 1994a,
spatially separated lights was illuminated 1994b). Although conditioned reinforcers may
(Schultz, Apicella, & Ljungberg, 1993; Schultz, acquire additional functions, particularly in
Dayan, & Montague, 1997). The top panel free-operant procedures (e.g., Shahan, 2010),
(A) shows the frequency of firing of DA neu- neuroscience indicates that both uncondi-
rons when the reinforcing stimulus (SR) was tioned and conditioned reinforcers cause DA
presented at the outset of conditioning. Note to be liberated by VTA neurons. The route
that the baseline level of activity of DA neurons whereby VTA neurons are activated by condi-
increased abruptly shortly after the reinforcing tioned reinforcers differs, however. Whereas
stimulus occurred. Transition to the “bursting” unconditioned reinforcers activate VTA neu-
mode is required for DA to be liberated along rons by pathways originating from receptors
the axons of VTA neurons (Grace & Bunney, such as those for taste or sexual stimulation
1984; Grace, Floresco, Goto, & Lodge, 2007; (e.g., Balfour, Yu, & Coolen, 2004; Smith,
Johnson, Seutin, & North, 1992). The middle Liu, & Vogt, 1996), conditioned reinforcers
panel (B) shows the frequency of firing of DA activate VTA neurons by a less direct route
neurons after conditioning when the discrimi- involving the prefrontal cortex (e.g., Pears, Par-
native stimulus (SD) was presented and a cor- kinson, Hopewell, Everitt, & Roberts, 2003; Wil-
rect response was followed by SR. The burst of son & Bowman, 2004)
firing now occurred at the onset of SD, not SR. As shown in Figure 2, the neural activity
This finding is consistent with the long- initiated by SD stimulates neurons in prefron-
standing behavioral finding that discriminative tal cortex and pathways originating from pre-
stimuli also acquire a conditioned reinforcing frontal neurons activate cells in the nucleus
308 JOHN W. DONAHOE
accumbens (NAC) that, in turn, cause VTA-
DA neurons to be activated. Thus both condi-
tioned and unconditioned reinforcers ulti-
mately engage the same VTA reinforcement
system, but by different routes.6 For experi-
enced organisms in which many E–B relations
have been previously acquired, the environ-
ment offers many opportunities for engaging
the neural mechanisms of conditioned rein-
forcement. In this way, activities that require
temporally extended sequences of behavior,
such as problem solving, are maintained by a
relatively continuous stream of conditioned
reinforcers, which increases as the target
behavior is more closely approximated
(Donahoe & Palmer, 1993, p. 285 ff.).7
Panel C of Figure 3 reveals an additional
finding that relates to the behavioral-
discrepancy requirement. The lower panel
shows the activity of DA neurons on a trial in
which SD was presented, but SR was omitted.
Note that DA activity was actually inhibited
shortly after the time when the reinforcer was
otherwise scheduled to occur.8 If another
6
The conditioning of autonomic responses with operant
procedures has been notoriously difficult to achieve
(Dworkin & Miller, 1986). Autonomic responses are
mediated at the level of the midbrain and need not
involve cortical mechanisms, including the circuits
involved with conditioned reinforcement described here
(e.g., Oakley & Russell, 1972). Computer simulations indi-
cate that conditioned reinforcement is needed to bridge
the temporal gap between the operant and the reinforcer
(Donahoe & Burgos, 2005). These circuits are not availa-
ble at the level of the midbrain. It is of interest that the
few studies that have successfully demonstrated operant
conditioning of autonomic response have minimized delay
of reinforcement by using immediate electrical stimulation
of the brain as a reinforcer (e.g., Trowill, 1967).
7
Deficiencies in the neural mechanisms of conditioned
reinforcement have been proposed as an important factor
in the behavioral deficits found along the autism spectrum
(Donahoe & Vegas, in press). In one study, diminished
activity in the NAC to a visual conditioned reinforcer was
found with autistic subjects (Dichter, Felder, Green, Riten-
berg, Sasson, & Bodfish, 2012). If the numbers and
sources of connections from regions in prefrontal cortex
to NAC and, possibly, VTA are reduced for those with
autism, then the potential for stimuli that activate those
regions to serve as conditioned reinforcers would be
impaired. As an example, suppose that neurons in the
region of the prefrontal cortex that receive inputs from
the sensory areas involved in face perception do not have
connections to NAC. Under these circumstances, seeing
human faces would not serve as a source of conditioned
reinforcement. Instead, eye contact with another person
would be viewed as a threat gesture, the typical reaction in
other primates. (Emery, Lorincz, Perrett, Oram, &
Baker, 1997).
stimulus had accompanied this SD in a block-
ing design, then the reinforcer would not have
produced an increase in DA activity and would
not have become a discriminative stimulus.
The inhibition of VTA neurons following the
time at which the reinforcer previously
occurred is the neural basis of the discrepancy
requirement (Burgos & Donahoe, 2016; Dona-
hoe, Burgos, & Palmer, 1993; Waelti, Dickin-
son, & Schultz, 2001).9
Dopamine and the cellular mechanisms of
selection (LTP). The liberation of DA from
VTA neurons by reinforcers during condition-
ing comports well with what is known from
behavioral research about unconditioned
8
The present discussion of neural mechanisms describes
the major systems underlying reinforcement. Although
much is known of these systems and their underlying pro-
cesses, much remains to be known (e.g., Gerfen & Surme-
ier, 2011). As an example relating to the DA-ergic
mechanisms of the discrepancy requirement, DA neurons
in the VTA are maintained at their low baseline (tonic)
rates of firing by inhibitory neurons acting on VTA neu-
rons from the output of NAC. (Lalive et al., 2014). Uncon-
ditioned reinforcers drive the VTA neurons sufficiently to
overcome this tonic inhibition. The onset of conditioned
reinforcers activate Glu pathways from prefrontal cortex
to neurons in NAC and these increase DA activity by stim-
ulation of DA VTA neurons (Geizler, Derst, Veh, & Zahm,
2007) and/or by inhibiting inhibitory neurons from NAC
to VTA, thereby briefly liberating DA from VTA neurons
before tonic inhibition is reinstated (Aggarwai, Hyland, &
Wickens, 2012; Creed, Ntamati, & Tan, 2014; Moorman &
Aston-Jones, 2010; Tong, Overton, & Clark, 1996). As an
additional complication, inputs from the hypothalamus to
the region of the VTA co-release the neuropeptide
orexin/hypocretin which provides a mechanism whereby
the state of deprivation affects DA VTA activity
(cf. Burdakov, Liss, & Ashcroft, 2003; Gao & Horvath,
2014; Seidenbecher, Balschun, & Reymann, 1995). Finally,
NAC is itself not a homogeneous structure; the core and
shell of the nucleus are differently innervated (Dreyer,
Vander-Weele, & Lovic, 2016) and have somewhat differ-
ent functions (Corbit & Balleine, 2011). (For a different
view of the place of DA in learned behavior see Ber-
ridge, 2007).
9
The behavioral expression of the UR is not necessary
for conditioning as demonstrated by cases in which the
behavioral UR is prevented (as when transmission at the
neuromuscular junction is blocked; e.g., Solomon &
Turner, 1962) or by cases in which the environment does
not support expression of the UR (as when an auditory CS
is paired with food for the pecking response of the pigeon
in an autoshaping experiment; e.g., Leyland & Mackin-
tosh, 1978). The response elicited by the reinforcer is the
behavioral event that is most closely synchronized with the
occurrence of the DA-ergic neural mechanisms of rein-
forcement. (Donahoe & Vegas, 2004). However, once
these neural mechanisms had been naturally selected their
behavioral expression is not necessary for the environment
to engage them.
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
reinforcement, conditioned reinforcement,
and blocking. In addition, the diffusion of DA
in prefrontal and motor cortex (Yagishita
et al., 2014) allows DA to simultaneously affect
the functioning of many neurons along the
diverse pathways required to implement a
wide range of reinforced E–B relations. What
is the cellular process that enables this
function?
The process that affects the strengths of
connections along pathways mediating E–B
relations is long-term potentiation (LTP;
Bliss & Lømø, 1973). Before describing LTP,
several technical terms are briefly reviewed.
LTP refers to an increased ability (potentiation)
of receptors on a neuron to be activated by a
neurotransmitter released from the axons of
Fig. 4. An axon terminal of a presynaptic neuron and
a dendritic spine of a postsynaptic neuron. The excitatory
neurotransmitter glutamate (Glu) released by the presyn-
aptic neuron binds to two types of postsynaptic Glu
receptors—the fast-acting AMPA receptor and the voltage-
sensitive NMDA receptor. When the postsynaptic neuron
is sufficiently depolarized by the action of Glu, the NMDA
receptor allows calcium ions (Ca2+) to enter the cell. If
dopamine (DA) is simultaneously bound to DA receptors,
then a series of intracellular processes (second messen-
gers) migrate to genetic material in the cytoplasm and/or
nucleus of the postsynaptic cell stimulating protein synthe-
sis. The synthesized proteins diffuse down the dendrite
where they act upon those AMPA receptors to which Glu
was previously bound. This produces long-lasting changes
in those Glu receptors that make them more responsive to
Glu on subsequent occasions. See the text for additional
information.
309
other neurons. Neurotransmitters diffuse
across the synapse between presynaptic and post-
synaptic neurons. The neurotransmitter
released by presynaptic neurons may either
increase (excite) or decrease (inhibit) the activ-
ity of postsynaptic neurons through its effect
on the receptors to which the neurotransmitter
binds on the postsynaptic neuron. Receptors
are typically located within the membrane of
the dendrites of postsynaptic neurons. Neurons
normally maintain a negative resting potential
across the cell membrane. This potential
becomes more positive when neurons are acti-
vated and more negative when they are inhib-
ited. The membrane potential is governed by
the net influx of negative and positive ions
across the cell membrane.
Figure 4 is a schematic representation of an
axon terminal of a presynaptic neuron and a
juxtaposed spine on a dendrite of a postsynap-
tic neuron. Single neurons have many such
axon terminals and dendritic spines. Mole-
cules of the neurotransmitter glutamate (Glu)
are released through the cell membrane of
the presynaptic neuron and diffuse into the
synapse. (Glu is the primary excitatory neuro-
transmitter in the brain.) Molecules of Glu
then bind to two types of Glu receptors on the
postsynaptic neuron—AMPA receptors and
NMDA receptors. (The abbreviations stand for
the Glu analogs, α-amino-3-hydroxy-5-methyl-4-
isoxazolepropionic acid and N-methyl-D-aspar-
tate, respectively, that selectively bind to the
two types of Glu receptors.) If Glu binds to
AMPA receptors, entry of positive ions into
the cell increases and the membrane potential
of the neuron becomes less negative. When
the membrane potential becomes less nega-
tive, even if not enough to cause the postsy-
naptic neuron to “fire,” those specific AMPA
receptors acquire a molecular “tag” that
enhances their response to Glu for perhaps an
hour or so (Frey, 1997; Frey & Morris, 1997).
This brief enhancement is called early LTP.
When the concerted action of AMPA recep-
tors depolarizes the postsynaptic neuron suffi-
ciently, additional effects occur: If Glu has also
bound to NMDA receptors, which permit cal-
cium ions (Ca2+) to enter the cell, and if DA is
present, then a series of intracellular events
are initiated that result in the synthesis of new
proteins (see Fig. 4). These proteins, which
require time to synthesize, diffuse down the
dendrite of the postsynaptic cell and produce
310 JOHN W. DONAHOE
a long-lasting enhancement in the response to
Glu to only the previously tagged AMPA recep-
tors. This produces late LTP, which is a long-
lasting change in the ability of Glu to activate
specific receptors on the postsynaptic neu-
ron.10 In this way, DA “forges” pathways
through the frontal lobes from the inputs
arriving from sensory and sensory-association
cortex to motor neurons that mediate observa-
ble behavior (Reynolds, Hyland, & Wickens,
2001).11 The finding that DA plays an espe-
cially important role in the acquisition of
effortful responses or long sequences of
responses may reflect that such tasks requite
the potentiation of receptors on large num-
bers of neurons (Salamone, Correa, Nunes,
10
DA receptors of the type involved here are coupled to
a particular G protein that catalyzes the synthesis of cAMP
(cyclic adenosine monophosphate), which is the starting
point for a series of intracellular second messengers that
stimulate protein synthesis. Studies have shown that the
injection of cAMP into a cell with tagged Glu receptors
produces LTP of those receptors even though the cell was
not depolarized sufficiently to otherwise cause LTP. LTP
may result from prolonging the opening of ion channels
of Glu receptors or by producing additional Glu receptors
(so called “latent” receptors) in the postsynaptic mem-
brane (Ju et al., 2004). See Frey, 1997 and Martin, Grim-
wood, and Morris, 2000, for additional information. As
indicated later, neuromodulators other than DA can ena-
ble LTP.
11
The presentation has focused on the effect of DA aris-
ing from the VTA that acts on neurons in the prefrontal
cortex and NAC. These projections make up the mesocor-
tical and mesolimbic DA systems, respectively. Another sys-
tem of DA projections arises from the substantia nigra pars
compacta (SN), a midbrain nucleus adjacent to the VTA.
The SN system projects to a region called the striatum, the
region referenced in this footnote. DA-ergic neurons in
the SN are also activated by reinforcing stimuli (Lee &
Tepper, 2009). In addition to these DA-ergic inputs, neu-
rons in the striatum receive converging inputs from motor
regions of the prefrontal cortex and from the thalamus
(Plenz & Wickens, 2010). Striatal neurons are thus well
positioned to integrate activity from their sensory thalamic
inputs with their reinforced prefrontal motor inputs
(cf. Bradfield, Bertran-Gonzalez, Chieng, & Baleine,
2013). More “downstream” motor systems then lead to
observable behavior. A provocative speculation is that DA
from the VTA acts upon neural circuits in prefrontal cor-
tex that specify those striatal neurons that are most active
prior to reinforcement and that DA from SN then
strengthens connections from sensory inputs to those co-
active striatal neurons. As the result of such a process,
reinforced E-B relations could ultimately be mediated
before fully engaging prefrontal activity, for example the
activity mediating subvocal verbal behavior
(i.e., consciousness). Through this “short-circuiting” proc-
ess, such behavior would become “automatic” (Donahoe,
1997, p. 354).
Randall, & Pardo, 2013; cf. Georgopoulos,
Schwartz, & Kettner, 1986; Sanes & Dono-
ghue, 2000). DA also potentiates Glu-ergic
receptors on postsynaptic neurons in the
NAC, which subserve conditioned reinforce-
ment. If DA is not present when pre- and post-
synaptic neurons are coactive, then the
enhanced response of postsynaptic Glu recep-
tors declines, producing long-term depression
(LTD). Note that at each of the various levels
of analysis—the behavioral, the neural, and
the cellular—the mode of explanation follows
the Darwinian model of selectionism: That is,
reinforcers select among E–B relations;
reinforcer-evoked DA selects among neural
pathways that mediate the E–B relations, and
DA-instigated protein synthesis selects among
tagged postsynaptic receptors (Donahoe,
1997).12
Stimulus Control
Behavioral Findings
The vast range of phenomena encompassed
by the field of stimulus control was foresha-
dowed by Skinner’s early comment that “dis-
criminative stimuli are practically inevitable
after conditioning” (Skinner, 1937, p. 273;
cf. Donahoe, Palmer, & Burgos, 1997). The
effects of only two sets of phenomena are con-
sidered here: discrimination formation and
equivalence classes. Other areas are omitted,
notably concept formation initiated by the pio-
neering work of Richard Herrnstein
(Herrnstein, Loveland, & Cable, 1976) and
continuing as an active area of current
research (e.g., Zentall, Wasserman, &
Urcuioli, 2013).
Discrimination formation. If a reinforcer
occurs in an environment, whether in a Pav-
lovian or an operant procedure (e.g., Gynther,
1957, and Guttman & Kalish, 1956, respec-
tively), then behavior comes under the control
of not only that environment but also other
environments to the extent that they share sti-
muli in common. This is the so-called
common-elements account of stimulus gener-
alization formalized by Skinner’s former
12
A wide variety of conditioning phenomena have been
simulated in neural-network research that implements the
foregoing mechanisms of reinforcement using networks
whose architecture is consistent with the neural systems
described here (see Burgos & Donahoe, 2016).
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
student William Estes (Estes, Burke, & Atkin-
son, 1957). If a differential conditioning pro-
cedure is instituted, then the controlling
stimuli are confined to those that are specific
to the environment in which the reinforcer
occurred. Stimuli that are not shared with the
conditioning environment become the occa-
sion for whatever previously conditioned and
unconditioned responses are supported by
that environment. The behavioral effects of
differential conditioning are most apparent
with experimental procedures in which differ-
ent operants are conditioned to the different
environments. For example, lever presses of
two different durations were differentially rein-
forced in the presence of two light intensities
and tests of stimulus generalization were then
conducted with various intermediate light
intensities (Crowley, 1979). The result was that
responding during generalization tests with
the other light intensities was confined to the
two previously reinforced response durations,
their proportions varying with the proximity of
the test intensity to the two training intensities.
Findings of similar import have been obtained
with a variety of other procedures
(e.g., Migler, 1964; Migler & Millenson, 1969;
Scheuerman, Wildemann, & Holland, 1978;
Wildemann & Holland, 1972). The general
conclusion from this work is that new environ-
ments do not control new behavior but new
mixtures of previously conditioned behavior
(Bickel & Etzel, 1985; Donahoe & Wessells,
1980, pp. 194-196). Any “creativity” arises from
variation in the environment, not from within
the organism itself. As E–B relations accumu-
late, the variation increases upon which future
selections by reinforcement may act, with the
emergent possibility of ever more complex E–
B relations.
Equivalence classes. Equivalence classes
(Sidman & Tailby, 1982) are typically studied
using matching-to-sample procedures with
multiple, arbitrary (symbolic) conditional dis-
criminations. For example, conditional discri-
minations using A–B and A–D are trained
where the pairs of letters stand for different
sets of sample and comparison stimuli, respec-
tively. After these conditional discriminations
have been acquired, unreinforced probe trials
are occasionally introduced using previously
untrained combinations of sample and com-
parison stimuli, for example D–B. An equiva-
lence class is said to form if an organism
311
reliably responds to the comparison stimulus
associated with a sample stimulus that was pre-
viously trained with a different but corre-
sponding comparison stimulus. Equivalence
classes meet the three criteria of mathematical
equivalence: identity (A = A), symmetry
(if A = B, then B = A), and transitivity
(if A = B and B = C, then A = C). In the pre-
ceding example of a B–D probe trial, respond-
ing to the appropriate D comparison stimulus
requires both symmetry (only A–B was trained,
not B–A) and transitivity (B–A–D). Equiva-
lence classes have important implications for
the interpretation of complex behavior
because they demonstrate that the effects of
reinforcement on the environmental guidance
of behavior transcend the specific E–B rela-
tions that were explicitly reinforced, even
beyond those of stimulus generalization. The
controlling stimuli in the training environ-
ment (for example, A) do not have stimulus
elements in common with those of the test
environment (for example, D).
Neuroscientific Findings
When the discriminative stimuli in condi-
tioning and generalization-test situations are
well defined in physical terms, the findings are
relatively straightforward. For example, rabbits
trained using a classical-conditioning proce-
dure to respond to a particular frequency of
an auditory stimulus were then tested with
other frequencies. Generalization tests
included measurement of the responses of
neurons in primary auditory cortex as well as
behavioral responses. The greater the overlap
between the number of cells firing to the test
stimuli, and those firing to the training stimu-
lus, the greater the behavioral response
(Thompson, 1965). That is, responding
occurred to the extent that the conditioning
and test environments activated neural ele-
ments in common.
But what of equivalence-class procedures in
which the discriminated stimuli in the condi-
tioning environment share no readily identifia-
ble physical similarity to those in the testing
environment? To address this question it is
necessary to consider additional neural sys-
tems, specifically the relation between sensory-
association (S-A) cortex and the hippocampus.
Figure 5 depicts that relation. As previously
noted, neurons in S-A cortex receive their
312 JOHN W. DONAHOE
inputs from primary sensory and other S-A
neurons. Thus the activation of S-A neurons
reflects the co-occurrence of multiple environ-
mental inputs. The axons of S-A neurons give
rise to a multisynaptic pathway (among
others) that innervates the hippocampus, a
subcortical structure located beneath the tem-
poral lobes. After interactions involving other
neurons within the hippocampus, neurons in
the CA1 region of the hippocampus give rise
to a multisynaptic pathway that projects back
to S-A cortex. (CA is an abbreviation for the
Latin phase, cornu Ammons, or horn of
Ammons, that refers to the shape of the hip-
pocampus.) It has been proposed that the out-
put of the hippocampus from the CA1 region
projects back to the very regions and/or neu-
rons of the S-A cortex that gave rise to the hip-
pocampal inputs (Amaral, 1987). The
synchronized activation of S-A neurons by
their sensory inputs and, very shortly thereaf-
ter, by their hippocampal inputs depolarizes
those S-A neurons and, in this way, potentiates
specific receptors on the neuron (cf. Strycker,
1986). These receptors are then tagged and
potentially undergo LTP.13 The net effect of
this process produces S-A neurons that
respond to their multimodal inputs. As a sim-
plistic example, if the inputs to a given S-A
neuron arise from the co-occurrence of a tone
and a light, then that S-A neuron becomes
functionally a tone–light neuron. The net
result is that S-A neurons become sensitive to
the conjunctions of spatio-temporally contigu-
ous, environmental events, or environment–
environment (E–E) relations.
The acquisition of E–E relations between
sample and comparison stimuli has been pro-
posed to underlie the formation of equivalence
13
DA-dependent LTP was first discovered in CA1 neu-
rons (Bliss & Lomo, 1973). While DA enables LTP at CA1
synapses, other agents play a critical role at synapses within
the trisynaptic circuit of the hippocampus (e.g., Stein,
Xue, & Belluzzi, 1993). In S-A cortex, neurotransmitters
such as noradrenaline, acetylcholine, and/or serotonin
play a role that is functionally similar to that of DA in fos-
tering LTP in prefrontal cortex and CA1 synapses (Singer,
1997). Early studies of the hippocampus documented its
role in spatial discriminations (O’Keefe & Nadel, 1978).
Spatial discriminations require integrating multiple sen-
sory inputs to specify a particular location within the physi-
cal environment. However, the role of the hippocampus is
much more general in that it integrates multisensory
inputs in a wide variety of situations, with spatial discrimi-
nation being but one example.
Fig. 5. Lateral view of the human cerebral cortex show-
ing the major cortical regions and subcortical structures
and pathways (indicated by dashed lines) that play an
important role in the selection of environment–
environment relations.
relations (Donahoe & Palmer, 1993, p. 148).
Recent neuroscientific evidence supports this
proposal. Rhesus monkeys that had acquired
an arbitrary matching-to-sample task using
computer-generated, abstract visual stimuli
developed single neurons in the S-A cortex that
responded equally to either the sample or the
comparison stimulus of pairs to which correct
responding had been reinforced. Such neu-
rons were not found for unreinforced pairs of
stimuli, although they had occurred together
equally often (Fujimichi, Naya, & Miyashita,
2005; Sakai & Miyashita, 1991). Neurons that
responded equally to either member of a rein-
forced pair were designated “pair-coded neu-
rons.” For such neurons, the two stimuli were
literally equivalent. The effect of reinforce-
ment on the acquisition of E–E relations is
facilitated by DA-ergic projections from the
VTA to CA1 neurons (see Fig. 5), the output
neurons of the hippocampus to SA cortex
(Duncan, Ketz, Inati, & Davachi, 2012; Gas-
barri, Verney, Innocenzi, Campana, & Pacitti,
1994; Li, Cullen, Anwyl, & Rowan, 2003; Swan-
son & Kohler, 1986). As already noted, without
the contribution of DA, receptors on CA1 neu-
rons would be potentiated for only a few hours
(early LTP). With the contribution of DA,
potentiation of CA1 and S-A neurons may
endure indefinitely until “over-written” by
other E–E relations.
With nonhuman organisms, behavioral
research provides clear evidence of both
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
identity (generalized identity matching) and
transitivity, but little or no compelling evi-
dence of symmetry (but see Kastak, Schuster-
man, & Kastak, 2001; Urcuioli, 2008). In
humans, strong evidence exists for all three
components of equivalence classes (Lionello-
DeNolf, 2009). What is responsible for this
apparent interspecies difference? Research in
neuroscience offers some clues. In primates,
unlike other mammalian orders, the bidirec-
tional pathways that loop between the S-A cor-
tex and hippocampus arise exclusively from
and synapse upon S-A neurons (Amaral,
1987). Organisms that have a lesser capacity
for acquiring pair-coded neurons—or lack
homologous neural structures that achieve the
same end—may have a diminished or absent
capacity to form the pair-coded neurons that
support symmetric discriminations (cf. -
Sidman, Rauzin, Lazar, Cunningham, Tailby, &
Carrigan, 1982). For organisms whose neuroa-
natomy permits the formation of pair-coded
neurons, the development of neurons that
respond to multisensory inputs should be the
norm if the contingencies of reinforcement
foster their selection.14 As Sidman proposed,
equivalence-class formation may be a norma-
tive effect of reinforcement (Sidman, 2000).
Memory
Behavioral Findings
In a behaviorally informed approach to
memory, as the field is conventionally known,
phenomena may be divided into two cases:
reminding and remembering (Donahoe &
Palmer 1993; Palmer 1991). Reminding refers
to those cases in which some behavior is
scheduled for reinforcement and the behavior
takes place in an environment in which that
behavior has been previously reinforced. As an
example, when asked “Who was the President
during the Civil War,” the response “Lincoln”
is scheduled for reinforcement and has almost
certainly been previously reinforced in that
verbal context. This is simply a straightforward
14
A considerable array of procedures other than stand-
ard matching-to-sample procedures has provided evidence
of the formation of equivalence classes (e.g., Arntzen,
2006; Arntzen, Nartey, & Fields, 2015, Arntzen, Norbom &
Fields, 2015; Fields, Reeve, Varelas, Rosen, & Belanich,
1997; Grisante et al., 2013; Leader, Barnes, & Smeets,
1996). I thank Prof. Erik Arntzen for providing some of
these references.
313
example of stimulus control. Using language
more congenial to the field of memory, the
environment may be said to remind the organ-
ism of the appropriate behavior. By contrast,
remembering refers to cases in which some
behavior is scheduled for reinforcement but
the contemporaneous environment does not
include a sufficient number of the stimuli to
which the target response was previously con-
ditioned. Readily observable behavior can
sometimes produce the requisite controlling
stimuli as when one consults a smartphone to
find the telephone number of AAA after one’s
car has broken down. Consulting a smart-
phone to find a telephone number and then
keying in that number have both been previ-
ously reinforced. A greater challenge is pre-
sented by those cases of remembering in
which the behavior required to produce the
stimuli needed to control the appropriate
response is covert. For example, upon seeing
the approach of casual acquaintances, you
struggle to recall their names. You may then
subvocally sound out the letters of the alpha-
bet hoping to hit upon letters that, together
with the appearance of their faces, prompt the
correct names. The sight of the approaching
acquaintances controls covert behavior (mne-
monic behavior) that in the past has been effec-
tive in producing covert stimuli sufficient for
the emission of the overt response: in this case
uttering the person’s name (Donahoe &
Palmer, 1993, pp. 275-277). In brief, remem-
bering requires engaging in behavior, whether
overt or covert, that produces stimuli that are
sufficient to remind oneself, so to speak, of
the behavior needed to produce the rein-
forcer. However, note that when the remind-
ing behavior is covert, this is an interpretation of
memory, not an experimental analysis
(cf. Skinner, 1974, p. 194; Palmer, 2011). That
is, the account is an extrapolation from behav-
ioral observations that draws only upon pro-
cesses that have been analyzed experimentally
in previous contexts and that are highly likely
to have occurred in the current context.
Neuroscientific Findings
The behavioral interpretation of remember-
ing requires that the behavior scheduled for
reinforcement is already within the repertoire
of the organism, but that stimuli adequate to
control that behavior are not present. For
314 JOHN W. DONAHOE

example, the question “Who was the Union

General who won the Battle of Gettysburg?”
may not be sufficient to control the verbal
response “Meade.” However, a response of the
desired form can be emitted under other cir-
cumstances, for example, as a textual response
to the printed word “Meade.” A biobehavioral
interpretation proposes that remembering is
accomplished at the neural level by exploiting
the extensive pathways (tracts) that course
back and forth between the S-A and prefrontal
cortex (Donahoe & Palmer, 1993; cf. Fuster,
2015). Repeated cycles of these neural interac-
tions can potentially activate motor neurons
that produce the target behavior.
A behavioral interpretation of remembering
has received support from the experimental
analysis of neuroscience. To appreciate these
findings, additional neuroanatomical informa-
tion is required. Figure 6 is a schematic view
looking down on the upper surface of the
cerebral cortex. The right hemisphere has
been deflected back to reveal the large fiber Fig. 6. Schematic view of the upper surface of the cere-
bundle (corpus callosum, shown in black) that bral cortex showing the left and right hemispheres. The
interconnects neurons in the two hemi- right hemisphere is deflected back to reveal the corpus
spheres. A monkey was trained with multiple, callosum (black), the large fiber bundle that interconnects
neurons in the two hemispheres. The curved arrow indi-
abstract, conditional discriminations using a cates the tracts that connect the sensory-association (S-A)
matching-to-sample task in which visual stimuli area of the right hemisphere with the prefrontal cortex of
were presented in such a way that sample sti- that same hemisphere. (Comparable tracts exist within the
muli activated only neurons in the right visual left hemisphere but are not shown.) The block arrows
cortex. The corresponding two comparison sti- indicate other tracts between the two hemispheres that
course through the corpus callosum. The portion of the
muli activated only neurons in the left visual corpus callosum that was first severed is indicated by A;
cortex (Tomita, Ohbayashi, Nakahara, Hase- the portion that was later severed is indicated by B
gawa, & Miyashita, 1999). In an intact brain, (Tomita , Ohbayashi, Nakahara, Hasegawa, & Miyashita,
stimuli that directly activate neurons in only 1999). See the text for additional information.
one hemisphere can also activate neurons in
the contralateral hemisphere by means of
pathways (shown as an inverted V-shaped,
double-headed, block arrow) coursing block arrow in Fig. 6) connected neurons in
through the posterior portion of the corpus the prefrontal cortex of the right hemisphere
callosum (denoted by A in Fig. 6). After the to neurons in the S-A area of the left hemi-
subjects had acquired the matching-to-sample sphere via the anterior portion of the corpus
task to a high level of proficiency, the A por- callosum (denoted B in Fig. 6). In short, feed-
tion of the corpus callosum was severed. back from activity in prefrontal cortex was suf-
Despite the absence of connections between ficient to control discriminative performance.
the S-A cortices of the two hemispheres, per- This feedback is the neural counterpart of
formance remained at a high level. Perfor- mnemonic behavior proposed in a behavioral
mance was maintained by two sets of alternate interpretation of remembering (Donahoe &
pathways: One pathway (indicated by a curved Palmer, 1993): The test environment (the sam-
block arrow) connected neurons in the S-A ple stimulus) was not sufficient to control
cortex to neurons in the prefrontal cortex of responding to the reinforced comparison stim-
the right hemisphere. The second set of path- ulus, but it instigated activity in the prefrontal
ways (shown as a single-headed, right-angled, cortex that was sufficient to appropriately
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
control responding. When the anterior por-
tion of the corpus callosum was severed (indi-
cated by B in Fig. 6), performance returned to
chance levels and could not be recovered. The
experiment demonstrates that neural activity
that does not evoke responses at the behav-
ioral level of observation can nevertheless con-
trol behavior when the contemporary
environment fails to provide stimuli in whose
presence the response was previously rein-
forced. The behaviorally based distinction
between reminding and remembering is thus
consistent with independent convergent evi-
dence from neuroscience where the two
classes of memory are known as bottom-up
versus top-down processing, respectively
(Fujimichi et al, 2005).15 In humans, covert sti-
muli, such as those produced by subvocal ver-
bal behavior that occurred at the time of
acquisition, can facilitate these neural pro-
cesses if they occur later during remembering,
Verbal behavior, whether overt or covert, is
not necessary for remembering, however, as
shown by the aforementioned findings with
monkeys.
Conclusions
These findings and their implications are
but a few examples illustrating the comple-
mentary nature of behavior analysis and neu-
roscience. Behavioral research indicates that
contiguity and discrepancy are required for
selection by reinforcement, and neuroscience
is well on its way to identifying the physiologi-
cal mechanisms that implement these require-
ments. The understanding of reinforcement
arising from behavioral research constrains
the search for its neural mechanisms to those
that are capable of affecting the strength of a
wide range of E–B relations while simultane-
ously confining the cumulative effects of selec-
tion to the particular class of E–B relations
that precede the reinforcer. The widely pro-
jecting DA system from the VTA to the frontal
lobes accommodates the potentially wide-
spread effects of selection by reinforcement;
15
Various proposals in cognitive neuroscience are not
necessarily inconsistent with the reminding-remembering
distinction. These include distinctions between the roles
of hippocampal and cortical systems (e.g., Kumaran, Has-
sabis, & McClelland, 2016) as well as between declarative
and procedural memory (Squire, 2004).
315
the specificity of LTP to particular postsynap-
tic receptors satisfies the specificity require-
ment. In addition, the same neural systems
provide an integrated account of uncondi-
tioned and conditioned reinforcement, with
the latter forming a critical element in the
emergence of complex behavior. Our knowl-
edge of the neural mechanisms of reinforce-
ment remains incomplete, of course,
particularly with regard to a detailed account
of the interactions between neurons in pre-
frontal cortex, NAC, and VTA, but the discrep-
ancy requirement arising from behavioral
research continues to constrain the search for
these mechanisms.
For more complex behavior such as that
observed in the study of equivalence classes
and memory, behavioral research can offer
verbal interpretations (e.g., Donahoe &
Palmer, 1993, pp. 139ff, 223ff ) that draw upon
observations of phenomena that have been
subjected to experimental analysis in other
contexts: The understanding of equivalence
classes can appeal to what is known about joint
stimulus control when conditioning serial and
compound discriminative stimuli; the study of
memory can appeal to covert behavior that is
consistent with what is known about overt
behavior in otherwise comparable situations.
Be that as it may, neuroscience offers the pos-
sibility of supplementing these behavioral
interpretations with experimental analyses at
the neural level of observation: Pair-coded
neurons provide a mechanism whereby stimuli
that bear no physical similarity to one another
become equivalent at the neural level and
feedback from activity in prefrontal cortex to
sensory-association cortex provides a mechan-
ism whereby memories occur under circum-
stances in which the contemporaneous
environment bears no specific similarity to the
environment in which the behavior was origi-
nally acquired.
A strong case can be made that the integra-
tion of behavior analysis and neuroscience
portends a biobehavioral science whose impli-
cations for understanding complex behavior,
including human behavior, are as profound as
the earlier integration of the sciences of
heredity and genetics for understanding com-
plex structure. Skinner explicitly acknowl-
edged “the spatial gap between behavior and
the variables of which it is a function” and
argued that this gap “can be filled only by
316 JOHN W. DONAHOE
neuroscience, and the sooner… the better”
(Skinner, 1938, p. 470). Now seems a propi-
tious time to vigorously pursue that goal
(cf. Ortu & Vaidya, 2016).
References
Aggarwai, M., Hyland, B. I., & Wickens, J. R. (2012). Neu-
ral control of dopamine transmission: Implications
for reinforcement learning. European Journal of Neuro-
science, 35, 1115–1123.
Albert, M., & Ayres, J. J. (1997). One-trial simultaneous
and backward excitatory fear conditioning in rats:
Lick suppression, freezing, and rearing to CS com-
pounds and their elements. Learning & Behavior, 25,
210–220.
Amaral, D. G. (1987). Memory: Anatomical organization
of candidate brain regions. In V. B. Mountcastle
(Ed.), Handbook of physiology. Section I, The nervous sys-
tem (Vol. V. Higher functions of the brain (pp. 211–294).
Washington, D. C.: American Physiological Society.
Arntzen, E. (2006). Delayed matching to sample and stim-
ulus equivalence: Probability of responding in accord
with equivalence as a function of different delays. The
Psychological Record, 56, 135–167.
Arntzen, E., Nartey, R. K., & Fields, L. (2015). Enhancing
responding in accordance with stimulus equivalence
by the delayed and relational properties of meaning-
ful stimuli. Journal of the Experimental Analysis of
Behavior, 103, 524–541.
Arntzen, E., Norbom, A., & Fields, L. (2015). Sorting: An
alternative measure of class formation? The Psychologi-
cal Record, 65, 615–625.
Balfour, M. E., Yu, L., & Coolen, L. M. (2004). Sexual
behavior and sex-associated environmental cues acti-
vate the mesolimbic system in male rats.
Neuropsychopharmacology, 29, 718–730.
Bauer, E. P., Schafe, G. E., & LeDoux, J. E. (2002). NMDA
receptors and L-type voltage-gated calcium channels
contribute to long-term potentiation and different
components of fear memory formation in the lateral
amygdala. Journal of Neuroscience, 22, 5239–5249.
Berridge, K. C. (2007). The debate over dopamine’s role
in reward: The case for incentive salience. Psychophar-
macology, 191, 391–431.
Betts, S. L., Brandon, S. E., & Wagner, A. R. (1996). Disas-
sociation of the blocking of conditioned eyeblink and
conditioned fear following a shift in US locus. Animal
Learning & Behavior, 24, 459–470.
Bickel, W. K., & Etzel, B. C. (1985). The quantal nature of
controlling stimulus-response relations in tests of stim-
ulus generalization. Journal of the Experimental Analysis
of Behavior, 44, 245–270.
Bliss, T. V., & Lømø, T. (1973). Long-lasting potentiation
of synaptic transmission in the dentate area of the
anaesthetized rabbit following stimulation of the per-
forant path. Journal of Physiology, 232, 331–356.
Bradfield, L. A., Bartran-Gonzales, J., Chieng, B., &
Balleine, B. W. (2013). The thalamostriatal pathway
and cholinergic control of goal-directed action: Inter-
lacing new with existing learning in the striatum. Neu-
ron, 79, 153–156.
Brandon, S. E., Betts, S. L., & Wagner, A. R. (1994). Discri-
minated lateralized eyeblink conditioning in the
rabbit: An experimental context for separating spe-
cific and general associative influences. Journal of
Experimental Psychology: Animal Behavior Processes. 20,
292–307.
Burdakov, D., Liss, B., & Ashcroft, F. M. (2003). Orexin
excites GABAergic neurons of the arcuate nucleus by
activating the sodium–calcium exchanger. Journal of
Neuroscience, 23, 4957–4951.
Burgos, J. E. (2005). Theoretical note: The C/T ratio in
artificial neural networks. Behavioural Processes, 69,
249–256.
Burgos, J. E., & Donahoe, J. W. (2016). Unified principle
of reinforcement in a neural-network model: Reply to
N. T. Calvin and J. J McDowell. Behavioural Processes,
126, 46–54.
Calvin, O. L., & McDowell, J. J (2016). Extending unified-
theory-of-reinforcement neural networks to steady-
state operant behavior. Behavioural Processes, 127,
52–61.
Catania, A. C. (1987). Some Darwinian lessons for behav-
ior analysis: A review of Bowler’s The Eclipse of Darwin-
ism. Journal of the Experimental Analysis of Behavior, 47,
249–257.
Catania, A. C. (1971). Reinforcement schedules: The role
of responses preceding the one that produces the
reinforcer. Journal of the Experimental Analysis of Behav-
ior, 15, 271–287.
Chambers, K. C. (1990). A neural model for conditioned
taste aversions. Annual Review of Neuroscience, 13,
373–385.
Corbit, L. H., & Balleine, B. W. (2011). The general and
outcome-specific forms of Pavlovian-instrumental
transfer are differently mediated by the nucleus
accumbens core and shell. Journal of Neuroscience, 31,
11786–11794.
Creed, M. C., Ntamati, N. R., & Tan, K. R. (2014). VTA
GABA neurons modulate specific learning behaviors
through the control of dopamaine and cholinergic
systems. Frontiers in Behavioral Neuroscience, 8, 1–7.
Crowley, M. A. (1979). The allocation of time to tempo-
rally defined behaviors: Responding during stimulus
generalization. Journal of the Experimental Analysis of
Behavior, 32, 191–197.
Dichter, G. S., Felder, J. N., Green, S. R., Ritenberg, A. M.,
Sasson, N. J., & Bodfish, J. W. (2012). Reward circuitry
function in autism spectrum disorders. Social Cognitive
and Affective Neuroscience, 7, 160–172.
Dinsmoor, J. A. (1950). A quantitative comparison of the
discriminative and reinforcing functions of a stimulus.
Journal of Experimental Psychology, 40, 458–472.
Dobzhansky, T. G. (1937). Genetics and the origin of species.
New York: Columbia University Press.
Domjan, M., & Galef, B. G. (1983). Biological constraints
on instrumental and classical conditioning: Retro-
spect and prospect. Animal Learning & Behavior, 11,
151–161.
Donahoe, J. W. (1984). Skinner—The Darwin of ontog-
eny? Behavioral and Brain Sciences, 7, 487–488.
Donahoe, J. W. (1997). Neural plasticity. In
J. W. Donahoe & V. P. Dorsel (Eds.), Neural-network
models of cognition (pp. 80–81). New York: Elsevier Sci-
ence Press.
Donahoe, J. W. (1999). Edward L. Thorndike: The selectionist
connectionist. Journal of the Experimental Analysis of Behav-
ior, 72, 451–454.
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
Donahoe, J. W. (2003). Selectionism. In K.A. Lattal & P.
N. Chase (Eds.), Behavior Theory and Philosophy
(pp.103–128). Dordrecht, Netherlands: Kluwer Aca-
demic Publishers.
Donahoe, J. W. (2010). Man as machine: A review of
“Memory and the computational brain: Why cognitive
science will transform neuroscience” by C.R. Gallistel
and A.P. King. Behavior and Philosophy, 38, 83–101.
Donahoe, J. W. (2012). Origins of the molar-molecular
debate. European Journal of Behavior Analysis, 13,
195–200.
Donahoe, J. W. (2013). Theory and behavior analysis.
Behavior Analyst, 36, 361–371.
Donahoe, J. W, & Burgos, J. E. (2005). Selectionism: Com-
plex outcomes from simple processes. Behavioural and
Brain Sciences, 28, 429–430.
Donahoe, J. W., Burgos, J. E., & Palmer, D. C. (1993). A
selectionist approach to reinforcement. Journal of the
Experimental Analysis of Behavior, 60, 17–40.
Donahoe, J. W., Crowley, M. A., Millard, W. J., &
Stickney, K. A. (1982). A unified principle of rein-
forcement: Some implications for matching. In
M. L. Commons, R. J. Herrnstein, & H. Rachlin
(Eds.), Quantitative analyses of behavior: Vol. 2. Matching
and maximizing accounts (pp. 493–521). Cambridge,
MA: Ballinger.
Donahoe, J. W., & Palmer, D. C. (1993/2005). Learning
and complex behavior. Boston: MA: Allyn & Bacon.
(Reprinted Richmond, MA: Ledgetop Publishing).
Donahoe, J. W., Palmer, D. C., & Burgos, J. E. (1997). The
S-R issue: Its status in behavior analysis and Donahoe
and Palmer’s Learning and Complex Behavior. (1997).
Journal of the Experimental Analysis of Behavior, 67,
193–211.
Donahoe, J. W., & Vegas, R. (2004). Pavlovian condition-
ing: The CS-UR relation. Journal of Experimental Psychol-
ogy: Animal Behavior Processes, 30, 17–33.
Donahoe, J. W., & Vegas, R. (in press). Respondent
(Pavlovian) conditioning. In W. W. Fisher,
C. C. Piazza, & H. S. Roane (Eds.), Handbook of applied
behavior analysis (2nd edition). New York: Guilford
Press.
Donahoe, J. W., & Wessells, M. G. (1980). Learning, lan-
guage, and memory. New York: Harper & Row.
Dreyer, J. K., Vander-Weele, C. M., & Lovic, V. (2016).
Functionally distinct dopamine signals in nucleus
accumbens core and shell in the freely moving rat.
Journal of Neuroscience, 36, 96–112.
Duncan, K., Ketz, N., Inati, S. J., & Davachi, L. (2012). Evi-
dence for area CAi as a match/mismatch detector: A
high-resolution fMRI study of the human hippocam-
pus. Hippocampus, 22, 389–398.
Dworkin, B. R. (1993). Learning and physiological regulation.
Chicago: University of Chicago Press.
Dworkin, B. R., & Miller, N. E. (1986). Failure to replicate
visceral learning in the acute curarized rat prepara-
tion. Behavioral Neuroscience, 100, 299–314.
Eickelboom, R., & Stewart, J. (1982). Conditioning of
drug-induced physiological responses. Psychological
Review, 89, 507–528.
Emery, N. J., Lorincz, E. N., Perrett, D. I., Oram, M. W., &
Baker, C. I. (1997). Gaze following and joint attention
in rhesus monkeys (Macaca mulatto). Journal of Compar-
ative Psychology, 111, 286–293.
317
Estes, W. K. (1956). The problem of inferences from
curves based on group data. Psychological Bulletin, 53,
134–140.
Estes, W. K., Burke, C. J., & Atkinson, R. C. (1957). Proba-
bilistic discrimination learning. Journal of Experimental
Psychology. 54, 233–239.
Estes, W. K., & Skinner, B. F. (1941). Some quantitative
properties of anxiety. Journal of Experimental Psychology,
29, 390–400.
Ferster, C. B., & Skinner, B. F. (1957). Schedules of reinforce-
ment. New York: Appleton-Century-Crofts.
Fields, L., Reeve, K. F., Varelas, A., Rosen, D., &
Belanich, J. (1997). Equivalence class formation using
stimulus-pairing and yes-no. The Psychological Record,
47, 661–686.
Frey, U. (1997). Cellular mechanisms of long-term potenti-
ation: Late maintenance. In J. W. Donahoe and
V. P. Dorsel (Eds.), Neural-network models of cognition:
Biobehavioral foundations (pp. 105–128). New York:
Elsevier Science Press.
Frey, U, & Morris, R. G. M. (1997). Synaptic tagging and
long-term potentiation. Nature, 385, 533–536.
Fujimichi, R., Naya, Y., & Miyashita, Y. (2005). Neural basis
of semantic-like memory: Representation, activation,
and cognitive control. In M. S. Gazaniga (Ed.), The
cognitive neurosciences III (pp. 905–918). Cambridge,
MA: MIT Press. http://www.physiol.m.u-tokyo.ac.jp/
en/review/review_en_03.html
Fuster, J. (2015). The prefrontal cortex (9th ed.). New York:
Academic Press.
Gao, X.-B., & Horvath, T. (2014). Function and dysfunc-
tion of hypocretin/orexin: An enegertics point of
view. Annual Review of Neuroscience, 37, 101–116.
Gallistel, C. R., & King, A. P. (2009). Memory and the compu-
tational brain: Why cognitive science will transform neurosci-
ence. New York: Wiley.
Gasbarri, A., Verney, C., Innocenzi, R., Campana, E., &
Pacitti, C. (1994). Mesolimbic dopaminergic neurons
innervating the hippocampal formation in the rat: a
combined retrograde tracing and immunohistochem-
ical study. Brain Research, 668, 71–79.
Gayon, J. (1998). Darwin’s struggle for survival: Heredity
and the hypothesis of natural selection. (Trans.
M. Cobb from Darwin et l’aprés-Darwin: une histoire de
l’hypothèse de sélection naturalle. Paris: Editions Kimé,
Paris, 1992.) Cambridge, UK: Cambridge University
Press.
Geisler, S., Derst, C., Veh, R. W., & Zahm, D. S. (2007).
Glutamatergic afferents of the ventral tegmental area
in the rat. Journal of Neuroscience, 27, 5730–5743.
Georgopoulos, A. P., Schwartz, A. B., & Kettner, R. E.
(1986). Neural population coding of movement direc-
tion. Science, 233, 1416–1419.
Gerfen, C. R., & Surmeier, D. J. (2011). Modulation of stri-
atal projection neurons by dopamine. Annual Review
of Neuroscience, 34, 441–466.
Gibbon, J., & Balsam, P. D. (1981). The spread of associa-
tion in time. In C. Locurto & H. S. Terrace (Eds.),
Conditioning theory. New York: Academic Press.
Gormezano, I. (1966). Classical conditioning. In
J. B. Sidowski (Ed.), Experimental methods and instru-
mentation in psychology. New York: McGraw-Hill.
Grace, A. A., & Bunney, B. S. (1984). The control of firing
pattern in nigral dopamine neurons: Burst firing. Jour-
nal of Neuroscience, 4, 2877–2890.
318 JOHN W. DONAHOE
Grace, A. A., Floresco, S. B., Goto, Y., & Lodge, D. J.
(2007). Regulation of firing of dopaminergic neurons
and control of goal-directed behaviors. Trends in Neu-
roscience, 30, 220–227.
Grisante, P. C., Galesi, F. L., Sabino, N. M., Debert, P.,
Arntzen, E., & McIlvane, W. J. (2013). Go/no-go pro-
cedure with compound stimuli: Effects of training
structure on the emergence of equivalence classes.
The Psychological Record, 63, 63–72.
Guttman, N., & Kalish, H. J. (1956). Discriminabilty and
stimulus generalization. Journal of Experimental Psychol-
ogy, 51, 79–88.
Gynther, M. D. (1957). Differential eyelid conditioning as
a function of stimulus similarity and strength of
response to the CS. Journal of Experimental Psychology,
53, 408–416.
Hebb. D. O. (1949). The organization of behavior. New York:
Wiley.
Herrnstein, R. J., Loveland, D. H., & Cable, C. (1976). Nat-
ural concepts in pigeons. Journal of Experimental Psy-
chology: Animal Behavior Processes, 4, 285–302.
Jensen, G., Ward, R. D., & Balsam, P. D. (2013). Informa-
tion: Theory, brain, and behavior. Journal of the Experi-
mental Analysis of Behavior, 100, 408–431.
Johnson, D. F. (1970). Determiners of selective stimulus
control in the pigeon. Journal of Comparative and Physi-
ological Psychology, 76, 298–307.
Johnson, S. W., Seutin, V., & North, R. N. (1992). Burst fir-
ing in dopamine neurons induced by N-methyl-D-
Aspartate: Role of electrogenic sodium pump. Science,
258, 665–667.
Ju, W., Morishita, W., Tsui, J., Gaiette, G., Deerinck, T. J.,
Adams, S. R., …Malenka, R. C. (2004). Activity-
dependent regulation of dendritic synthesis and traf-
ficking of AMPA receptors. Nature Neuroscience, 7,
244–253.
Kamin, L. J. (1968). "Attention-like" processes in classical
conditioning. In M. R. Jones (Ed.), Miami symposium
on the prediction of behavior (pp. 9–31). Coral Gables,
FL: University of Miami Press.
Kamin, L. J. (1969). Predictability, surprise, attention and
conditioning. In B. A. Campbell & R. M. Church
(Eds.), Punishment and aversive behavior (pp. 279–296).
New York: Appleton-Century-Crofts.
Kastak, C. R., Schusterman, R. J., & Kastak, D. (2001).
Equivalence classification by California sea lions using
class-specific reinforcers. Journal of the Experimental
Analysis of Behavior, 76, 131–156.
Killeen, P. R. (2015). The logistics of choice. Journal of the
Experimental Analysis of Behavior, 104, 74–92.
Kirk, P. D. W., Babtie, A. C., & Stumpf, M. P. H. (2015).
Systems biology (un)certainties. Science, 350, 386–388.
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016).
What learning systems do intelligent agents need?
Complementary learning systems theory updated.
Trends in Cognitive Sciences, 20, 512–534.
Lalive, A. L., Munoz, M. B., Bellone, C., Slesinger, P. A.,
Luscher, C., & Tan, K. R. (2014). Firing modes of
dopamine neurons drive bidirectional GIRK channel
plasticity. Journal of Neuroscience, 34, 5107–5114.
Lattal, K. A., & Gleeson, S. (1990). Response acquisition
with delayed reinforcement. Journal of Experimental Psy-
chology: Animal Behavior Processes, 16, 27–39.
Leader, G., Barnes, D., & Smeets, P. M. (1996). Establish-
ing equivalence relations using a respondent-type
training procedure. The Psychological Record, 46,
685–706.
Lee, C. R., & Tepper, J. M. (2009). Basal ganglia control
of substantia nigra dopaminergic neurons. Journal of
Neural Transmission Supplement, 73, 71–90.
Leyland, C. M., & Mackintosh, N. J. (1978). Blocking of
first- and second-order autoshaping in pigeons. Ani-
mal Learning & Behavior, 6, 392–394.
Li, S., Cullen, W. K., Anwyl, R., & Rowan, M. J. (2003).
Dopamine-dependent facilitation of LTP induction in
hippocampal CA1 by exposure to spatial novelty.
Nature Neuroscience, 6, 326–331.
Li, Z. S., Schmauss, C., Cuenca, A., Ratcliffe, E., &
Gershon, M. D. (2006). Physiological modulation of
intestinal motility by enteric dopaminergic neurons
and the D2 receptor: Analysis of dopamine receptor
expression, location, development, and function in
wild-type and knock-out mice. Journal of Neuroscience,
26, 2798–2807.
Lionello-DeNolf, K. M. (2009). The search for symmetry.
Learning & Behavior, 37, 18–203.
MacDonall, J. S. (2009). The stay/switch model of concur-
rent choice. Journal of the Experimental Analysis of Behav-
ior, 91, 21–39.
Marr, M. J. (1992). Behavior dynamics: One perspective.
Journal of the Experimental Analysis of Behavior, 57,
249–266.
Martin, S. J., Grimwood, P. D., & Morris, R. G. M. (2000).
Synaptic plasticity and memory: An evaluation of the
hypothesis. Annual Review of Neuroscience, 23, 649–711.
McDowell, J. J (2013a). A quantitative evolutionary theory
of adaptive behavior dynamics. Psychological Review,
120, 731–750.
McDowell, J. J (2013b). Representations of complexity:
how nature appears in our theories. Behavior Analyst,
36, 345–359.
McNish, K. A., Betts, S. L., Brandon, S. E., & Wagner, A. R.
(1997). Divergence of conditioned eyeblink and con-
ditioned fear in backward Pavlovian training. Animal
Learning & Behavior, 25, 43–62.
Migler, B. (1964). Effects of averaging data during stimu-
lus generalization. Journal of the Experimental Analysis of
Behavior, 7, 303–307.
Migler, B., & Millenson, J. R. (1969). Analysis of response
rates during stimulus generalization. Journal of the
Experimental Analysis of Behavior, 12, 81–87.
Moorman, D. E., & Aston-Jones, G. (2010). Orexin/hypo-
cretin modulates response of ventral tegmental dopa-
mine neurons to prefrontal activation: Diurnal
influences. Journal of Neuroscience, 17, 15585–15599.
Morse, W. H., & Skinner, B. F. (1957). A second type of
superstition in the pigeon. American Journal of Psychol-
ogy, 70, 308–311.
Oakley, D. A., & Russell, I. S. (1972). Neocortical lesions
and Pavlovian conditioning. Physiology & Behavior, 8,
915–926.
O’Keefe, J., & Nadel, L. (1978). The hippocampus as a cogni-
tive map. Oxford, UK: Oxford University Press.
Ortu, D., & Vaidya, M. (2016). The challenges of integrat-
ing behavioral and neural data: Bridging and break-
ing boundaries across levels of analysis. The Behavior
Analyst, 39, 1–16.
Palmer, D. C. (1986). The blocking of conditioned reinforce-
ment. Unpublished doctoral dissertation, University of
Massachusetts, Amherst, MA.
 BEHAVIOR ANALYSIS AND NEUROSCIENCE
Palmer, D. C. (1991). A behavioral interpretation of mem-
ory. In L. J. Hayes & P. N. Chase (Eds.), Dialogues in
verbal behavior (pp. 261–279). Reno, NV: Context
Press.
Palmer, D. C. (2011). Consideration of private events is
required in a comprehensive science of behavior.
Behavior Analyst, 34, 201–207.
Palmer, D. C., & Donahoe, J. W. (1992). Essentialism and
selectionism in cognitive science and behavior analy-
sis. American Psychologist, 47, 1344–1358.
Papini, M. R., & Bitterman, M. E. (1990). The role of con-
tingency in classical conditioning. Psychological Review,
97, 396–403.
Pavlov, I. P. (1960). Conditioned reflexes. New York: Dover
Press.
Pears, A., Parkinson, J. A., Hopewell, L., Everitt, B. J., &
Roberts, C. (2003). Lesions of the orbitofrontal but
not medial prefrontal cortex disrupt conditioned
reinforcement in primates. Journal of Neuroscience, 23,
11189–11201.
Plenz, D., & Wickens, J. (2010). The striatal skeleton:
Medium spiny projection neurons and their lateral
connections. In H. Steiner & K. Y. Tseng (Eds.).
Handbook of behavioral neuroscience, vol. 20 (pp. 99–
112). New York: Elsevier Science Press.
Pliskoff, S. S. (1971). Effects of symmetrical and asymmet-
rical changeover delays on concurrent performances.
Journal of the Experimental Analysis of Behavior, 16,
249–256.
Rescorla, R. A. (1968). Probability of shock in the pres-
ence and absence of CS in fear conditioning. Journal
of Comparative and Physiological Psychology, 66, 1–5.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pav-
lovian conditioning: Variations in the effectiveness of
reinforcement and nonreinforcement. In
A. H. Black & W. F. Prokasy (Eds.), Classical condition-
ing: Current research and theory (pp. 64–99). New York:
Appleton-Century-Crofts.
Reynolds, J. N. J., Hyland, B. I., & Wickens, J. R. (2001). A
cellular mechanism for learning. Nature, 413, 67–70.
Salamone, J. D., Correa, M., Nunes, E. J.,
Randall, P. A., & Pardo, M. (2013). The behavioral
pharmacology of effort-related choice behavior: Dopa-
mine, adenosine, and beyond. Journal of the Experimen-
tal Analysis of Behavior, 97, 97–125.
Sakai, K., & Miyashita, Y. (1991). Neural organization for
the long-term memory of paired associates. Nature,
354, 152–155.
Sanes, J. N., & Donoghue, J. P. (2000). Plasticity and pri-
mary motor cortex. Annual Review of Neuroscience, 23,
393–415.
Scheuerman, K. V., Wildemann, D. G., & Holland, J. G.
(1978). A clarification of continuous repertoire devel-
opment. Journal of the Experimental Analysis of Behavior,
30, 197–203.
Schultz, W. (1997). Adaptive dopaminergic neurons report
the appetitive value of environmental stimuli. In
J. W. Donahoe & V. P. Dorsel (Eds.), Neural-network
models of conditioning (pp. 317–35). New York: Elsevier
Science Press.
Schultz, W., Apicella, P., & Ljungberg, T. (1993). Condi-
tioned stimuli during successive steps of learning a
delayed response task. Journal of Neuroscience, 13,
900–913.
319
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural
substrate of prediction and reward. Science, 275, 1593–
1599.
Seidenbecher, T., Balschun, D., & Reymann, K. G. (1995).
Drinking after water deprivation prolongs unsatu-
rated LTP in the dentate gyrus of rats. Physiology &
Behavior. 57, 1001–1004.
Shahan, T. A. (2010). Conditioned reinforcement and
response strength. Journal of the Experimental Analysis of
Behavior, 93, 269–289.
Sidman, M. (1960). Tactics of scientific research. New York:
Basic Books.
Sidman, M. (2000). Equivalence relations and the rein-
forcement contingency. Journal of the Experimental
Analysis of Behavior, 74, 127–146.
Sidman, M., Rauzin, R., Lazar, R., Cunningham, S.,
Tailby, W., & Carrigan, P. (1982). A search for sym-
metry in the conditional discriminations of rhesus
monkeys, baboons, and children. Journal of the Experi-
mental Analysis of Behavior, 37, 23–44.
Sidman, M., & Tailby, W. (1982). Conditional discrimina-
tion vs. matching to sample: An expansion of the test-
ing paradigm. Journal of the Experimental Analysis of
Behavior, 37, 5–22.
Siegel, S., & Allan, L. G. (1996). The widespread influence
of the Rescorla-Wagner model. Psychonomic Bulletin &
Review, 3, 314–321.
Silva, F. J., Timberlake, W., & Cevik, M. O. (1998). A
behavior systems approach to the expression of back-
ward associations. Learning & Motivation, 29, 1–22.
Singer, W. (1997). Development and plasticity of neocorti-
cal processing architectures. In J. W. Donahoe &
V. P. Dorsel (Eds.), Neural-network models of cognition
(pp. 142–159). New York: Elsevier Science Press.
Skinner, B. F. (1937). Two types of conditioned reflex: a
reply to Konorski and Miller. Journal of General Psychol-
ogy, 16, 272–279.
Skinner, B. F. (1938). The behavior of organisms. New York:
Appleton-Century-Crofts.
Skinner, B. F. (1948). "Superstition" in the pigeon. Journal
of Experimental Psychology, 38, 168–172.
Skinner, B. F. (1974). About behaviorism. New York: Ran-
dom House.
Smith, M. C., Coleman, S. P., & Gormezano, I. (1969).
Classical conditioning of the rabbit nictitating mem-
brane response at backward, simultaneous, and for-
ward CS-US intervals. Journal of Comparative and
Physiological Psychology, 69, 226–231.
Smith, D. V., Liu, H., & Vogt, M. B. (1996). Responses of
gustatory cells in the nucleus of the solitary tract of
the hamster after NaCl or miloride adaptation. Journal
of Neurophysiology, 76, 47–58.
Solomon, R. L., & Corbit, J. D. (1974). An opponent-
process theory of motivation: I. Temporal dynamics of
affect. Psychological Review, 81, 119–145.
Solomon, R. L., & Turner, L. H. (1962). Discriminative
classical conditioning in dogs paralyzed by curare can
later control conditioned avoidance responses in the
normal state. Psychological Review, 69, 202–219.
Squire, L. R. (2004). Memory systems of the brain: A brief
history and current perspective. Neurobiology of Learn-
ing and Memory, 82, 171–177.
Staddon, J. E. R. (1983). Adaptive behavior and learning.
New York: Cambridge University Press.
320 JOHN W. DONAHOE
Staddon, J. E R. (2014). On choice and the Law of Effect.
International Journal of Comparative Psychology, 27,
569–584.
Staddon, J. E. R., & Simmelhag, V. L. (1971). The “super-
stition” experiment: A reexamination of its implica-
tions for the principles of adaptive behavior.
Psychological Review, 78, 3–43.
Stein, L., Xue, B. G., & Belluzzi, J. D. (1993). A cellular
analogue of operant conditioning. Journal of the Experi-
mental Analysis of Behavior, 60, 41–55.
Stickney, K., & Donahoe, J. W. (1983). Attenuation of
blocking by a change in US locus. Animal Learning &
Behavior, 11, 60–66.
Stickney, K. J., Donahoe, J. W., & Carlson, N. R. (1981).
Conditioning preparation for the nictitating mem-
brane of the pigeon. Behavior Research Methods &
Instrumentation, 13, 653–656.
Stryker, M. P. (1986). Binocular impulse blockade pre-
vents the formation of ocular dominance columns in
cat visual cortex. Journal of Neuroscience, 6, 2117–2133.
Sutton, R. S., & Barto, A. G. (1981). Toward a modern the-
ory of adaptive networks: Expectation and prediction.
Psychological Review, 88, 135–170.
Swanson, L. W., & Kohler, C. (1986). Anatomical evidence
for direct projections from the entorhinal to the
entire cortical mantle in the rat. Journal of Neuroscience,
6, 3010–3023.
Thompson, R. F. (1965). The neural basis of stimulus
generalization. In D. I. Mostofsky (Ed.), Stimulus
generalization (pp. 154–178). Stanford, CA: Stan-
ford University Press.
Thorndike, E. L. (1903). Elements of psychology. New York:
A. G. Seiler.
Tomita, H., Ohbayashi, M., Nakahara, K., Hasegawa, I., &
Miyashita, Y. (1999). Top-down signal from prefrontal
cortex in executive control of memory retrieval.
Nature, 401, 699–703.
Tong, Z. Y., Overton, P. G., & Clark, D., (1996). Stimula-
tion of the prefrontal cortex in the rat induces pat-
terns of activity in midbrain dopaminergic neurons
which resemble natural burst events. Synapse. 22,
195–208.
Trowill, J. A. (1967). Instrumental conditioning of the
heart rate in the curarized rat. Journal of Comparative
and Physiological Psychology, 63, 7–11.
Urcuioli, P. J. (2008). Associative symmetry, antisymmetry,
and a theory of pigeons’ equivalence-class formation.
Journal of the Experimental Analysis of Behavior, 90,
257–282.
Volkow, N. D., Fowler, J. S., & Wang, G-J. (2007). Dopa-
mine in drug abuse and addiction. Archives of Neurol-
ogy, 64, 1575–1579.
vom Saal, W., & Jenkins, H. M. (1970). Blocking the devel-
opment of stimulus control. Learning & Motivation, 1,
52–64.
Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine
responses comply with basic assumptions of formal
learning theory. Nature, 412, 43–48.
Wildemann, D. G., & Holland, S. G. (1972). Control of a
continuous response dimension by a continuous stim-
ulus dimension. Journal of the Experimental Analysis of
Behavior, 18, 419–434.
Williams, B. A. (1975). The blocking of reinforcement
control. Journal of the Experimental Analysis of Behavior,
24, 215–226.
Williams, B. A. (1994a). Conditioned reinforcement:
Neglected or outmoded explanatory construct? Psycho-
nomic Bulletin & Review, 1, 457–475.
Williams, B. A. (1994b). Conditioned reinforcement:
Experimental and theoretical issues. The Behavior
Analyst, 17, 261–285.
Wilson, D. I., & Bowman, M. (2004). Nucleus accumbens
neurons in the rat exhibit differential activity to con-
ditioned reinforcers and primary reinforcers within a
second-order schedule of saccharin reinforcement.
European Journal of Neuroscience, 20, 2777–2788.
Wise, R. A. (2002). Brain reward circuitry: Insights from
unsensed incentives. Neuron, 36, 229–240.
Yagishita, S., Hayashi-Takagi, A., Ellis-Davies, G. C. R.,
Urakubo, H., Ishii, S., & Kasai, H. (2014). A critical
time window for dopamine actions on the structural
plasticity of dendritic spines. Science, 345, 1616–1620.
Zentall, T. R., Wasserman, E. A., & Urcuioli, P. J. (2013).
Associative concept learning in animals. Journal of the
Experimental Analysis of Behavior, 101, 130–151.
Received: July 13, 2016
Final Acceptance: February 7, 2017
Copyright of Journal of the Experimental Analysis of Behavior is the property of Wiley-
Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv
without the copyright holder's express written permission. However, users may print,
download, or email articles for individual use.