Chapter 5: Conditioning

Part One: Classical Conditioning

Classical Conditioning Why Classical Conditioning Evolved Reflexes Pavlov's Dog The Conditional Response Acquisition of the Conditional Response The Acquisition Curve Extinction and Spontaneous Recovery Generalization Discrimination Release from Habituation Pavlovian Conditioning: "It's not what you think it is" Summary: Classical Conditioning

Part Two: Applications of Classical Conditioning

Applications of Classical Conditioning Conditioning of Neurons Conditioning and Drug Tolerance Conditional Immune Response Conditional Emotional Responses Sexual Anticipation CERs from Traumatic Events Taste Aversion Summary: Applications of Classical Conditioning

Part Three: Operant Conditioning

Operant Conditioning Starting Out in a "Rat Lab " Positive Reinforcement The Premack Principle Extinction of an Operant Response Negative Reinforcement Punishment Response Cost ("Negative Punishment") A 2 x 2 Table of Consequences Stimulus Control Discriminative Stimuli Backward Chaining Intermittent Reinforcement and Resistance to Extinction Extinction-Induced Resurgence (the "extinction burst") Spontaneous Backward Chaining in a Blue Jay Avoidance and Escape Learning Observational Learning Summary: Operant Conditioning

Part Four: Applied Behavior Analysis

Applied Behavior Analysis Behavioral Analysis Baselining Self-Monitoring as a Method of Behavior Change Using Reinforcement Using the Premack Principle Natural Social Reinforcers Secondary Reinforcement Using Tokens More about Shaping Prompting and Fading Differential Reinforcement DRO and DRL Using Negative Reinforcement Using Punishment Treatment of Uncontrollable Sneezing Punishment from the Environment The Punishment Trap Response Cost with a Rabbit Applied Analysis of Antecedents Summary: Applied Behavior Analysis

CLASSICAL CONDITIONING: WHAT IS IT?

How might a sea slug make a simple anticipatory response? Classical conditioning is a very simple form of learning. (It must be, if a sea slug can do it.) Classical conditioning happens when a stimulus the animal can identify (such as an odor in the water) is followed by a biologically significant event (such as the appearance of food). After several such pairings, the animal learns the connection. The next time the stimulus occurs, the animal acts like it expects the same event. That is, it makes an anticipatory response. If it is an Aplysia, maybe it extends its feeding tube.
What is the "rule" underlying classical conditioning?

This is not rocket science. Creatures do not need much of a nervous system to notice that when stimulus A happens, event B happens next. With a few neurons the sea slug can be really clever and make a response that prepares it for event B. The rule is, "Whenever you notice A, prepare for B." That is classical conditioning: an anticipatory biological response.

WHY CLASSICAL CONDITIONING EVOLVED

Classical conditioning is simple, but it was a major advance for creatures on Earth, because in its own humble way it is aprediction of the future. The sea slug senses the odor of food, and extends its feeding tube even before food is encountered. This trick enables the clever sea slug to feed quickly and consume more food than its less learned cousins who wait until they bump into food before responding to it. So the clever sea slug lives well and reproduces, and its descendants take over the seas.

How might the sea slug gain an advantage by making a response early?

Classical conditioning occurred in simple animals, and continues to occur in humans, because any animal can benefit from making anticipatory reactions. All animals need to prepare for the future. Creatures like the sea slug follow the simplest rule: "If it happened before, it may happen again." After A is paired with B a few times, A is treated as though A predictsB.
How does classical conditioning relate to the "head vs. heart" distinction discussed in Chapter 3?

Because it is so simple and basic, classical conditioning does not require conscious thinking or language. It can influence people-or sea slugs-without a conscious thought process. We hinted at this in Chapter 3 (States of Consciousness) when discussing the distinction between two modes of processing in human mental life. Different labels are used to describe the two modes: serial vs. parallel processing, intellectual vs. emotional processing, head vs. heart, analytic vs. intuitive thought,conscious vs. unconscious thought, and so forth. Classical conditioning comes down squarely on the "parallel, emotional, heart, intuitive, unconscious" side. The effects of classical conditioning are automatic. They tend to involve emotional processes rather than intellectual processes. They are involuntary rather than a result of will power. This distinction proves to be important when discussing the applications of classical conditioning to psychotherapy. Almost by definition, people requesting therapy are facing an unpleasant emotional reaction they cannot control simply by wishing it away or exercising conscious effort. That is why they seek help. In almost every case, the unpleasant emotional reaction can be analyzed as a case of classical conditioning. That will be a prominent theme in later parts of this chapter, as well as Chapter 13 (Therapies).

REFLEXES
In the 1800s, scientists knew a little about the nervous system, although a lot less than they know today. They knew the body had nerves. They knew the nerves were involved inreflexes that were input/output circuits built into the nervous system. In a reflex, some stimulus (an input to the nervous system) produces a response (an output from the nervous system). Here is a classic diagram of a spinal motor reflex from the 1920s. It shows the essential elements.

In a reflex, there is a sensory component (stimulus) and a motor component (response).

What components are found in all reflexes?

All reflexes have a stimulus component and a response component, corresponding to sensory and motor neural circuits, respectively. The simplest examples are spinal motor reflexes such as the knee jerk reflex (patellar reflex) or the finger withdrawal reflex. In the knee jerk reflex, a sensory neuron sends signals to the spine when the knee is tapped with a rubber hammer. In the spine, the sensory neuron synapses on the dendrites of a motor neuron, which sends nerve impulses back out to the muscle fibers, making the leg jerk. In the finger withdrawal reflex, pain such as the heat from a candle causes the finger to be withdrawn. The following figure, from the frontispiece of William James's Psychology: The Briefer Course(1892), shows the neural circuitry underlying the finger withdrawal reflex.

The finger-withdrawal reflex as depicted by William James in 1892. Inputs go through the sensory nerves (1 and S) and responses come out the motor nerves (M and 2).

The following table lists some reflexes in humans. In each case a stimulus activates sensory neurons, which trigger a motor response. The response is always biologically significant. It always helps to preserve the individual or the species. Name of reflex Gag reflex Tear reflex Startle reflex Patellar reflex Salivary reflex Orgasm Stimulus (input) Throat obstruction Eye irritation Loud noise Tap tendon below knee Food in mouth Sexual stimulation Baby at breast Light Response (output) Gagging, vomiting Tears Head, arms. Body, eyes move Lower leg jerks Saliva in mouth Pleasure, glandular and muscular responses Milk released Pupil contracts

Letting down Pupillary reflex

What are some examples of reflexes in humans?

Reflexes are stereotyped in form. They can be recognized in all normal members of a species. They are built inthey do not need to be learned. Snakes, for example, are born with all the reflexes they need. They emerge from an egg and slither away to hunt without any learning.
What did the word "reflex" mean to Pavlov and other Russian scientists?

The word reflex had a broader meaning in the late 1800s when Pavlov used the term. In those days the word reflex was used to describe almost everything we would today callbiologically based behavior. Indeed, Russian psychologists called themselves reflexologists from the 1880s through about the 1920s. They believed reflexes were the basic building blocks of all behavior.
How did Pavlov's discovery seem to provide a missing link?

In the 1890s, an emphasis on reflexes seemed to make psychology more scientific, in the 1800s, by anchoring it in biology. However, this also presented a problem. Human beings were obviously influenced to a very great extent by learning. How could reflexes patterns that seemed to be fixed into place by biologybe the building blocks of human behavior? The missing link seemed to be provided by Pavlov when he showed there was a way to modify reflexes...to bring them under the control of learning.

PAVLOV'S DOG
Ivan Pavlov was one of the most prominent scientists in the world at the beginning of the 20th Century. His discovery of classical conditioning actually came late in his career. For decades, Pavlov did research on digestive reflexes: the biological processes of digestion triggered by inputs to the stomach. He was an exceptionally good researcher who received a Nobel Prize for his research. When Pavlov delivered his acceptance speech at the Nobel Prize banquet in 1904, he surprised the crowd. He lectured about something he accidentally discovered while doing his digestion researchclassical conditioningrather than the digestion research that won him the prize. Pavlov announced that he had discovered conditional reflexes, reflex responses occurring as the result of learning.

Pavlov giving his Nobel Prize speech in 1904.

The discovery occurred when Pavlov connected a clear tube to the dog's salivary gland in the cheek, so he could measure the amount of salivation that took place after food was placed in the mouth. A similar set-up (that of Pavlov's co-worker G.F. Nicolai) is shown below. The dog was restrained in a harness with its head held still so the tube would not be ripped out. The researcher puffed meat powder into the dog's mouth to start the digestive process. Dogs salivate ("slobber") when they eat, so the meat powder stimulated lots of saliva. The saliva dripped out of the tube into a beaker where it could be measured.

A dog with a tube connected to its salivary gland

For what work was Pavlov awarded the Nobel Prize? How did he surprise the audience? In what sense did Pavlov's dog respond to a psychological stimulation?

With a set-up like this, Pavlov probably could not help but notice that dogs anticipated their meals. When Pavlov or an assistant entered the laboratory carrying meat powder, the saliva began dripping out of the tube. Pavlov realized this was significant. A biological reflex (salivation) was being modified by something psychological, namely, anticipation. In Pavlov's terminology, the dog's prediction was a form of "psychic stimulation" that activated the reflex. How could this happen? Reflexes were biological, yet the reflex was influenced by psychological factors.

Ivan Pavlov

Pavlov next devised a systematic version of his accidental observation. He (1) sounded a tone, and then (2) fed the dog meat powder. After a few repetitions, the dog started salivating when it heard the tone, even before the meat powder entered its mouth.

Pavlov was in his late 40s, but he changed his research program quickly to focus on this phenomenon and continued studying it until shortly before his death at age 87. In 1906 he followed up on his Nobel Prize speech by publishing an article in the American journal Science, summarizing his findings. The Science article was titled, "The Scientific Investigation of the Psychical Faculties or Processes in the Higher Animals." In those days, psychical meant the same thing as psychological.
What was Pavlov's one-word label for classical conditioning? What research did Watson do, using Pavlov's method?

Pavlov had a one-word label for classical conditioning. He called it signalization. That is not a bad label for classical conditioning, which occurs when a signal triggers a reflex-like response. In America, John B. Watson (the "father of behaviorism" described in Chapter 1), heard about Pavlov's research. Watson used Pavlovian conditioning in his own research. For example, he carried out many studies of the fingertip withdrawal reflex. Watson would ring a bell then quickly shock a person's fingertip with a small amount of electricity, causing involuntary withdrawal of the fingertip. Soon the person would withdraw his or her fingertip whenever the bell rang. Pavlov's dog and Watson's fingertip illustrate the basic pattern found in all classical conditioning. An organism learns that a signal predicts theactivation of a reflex. After learning this, the organism reacts to the signal with an anticipatory response similar to the reflex response

CONDITIONAL RESPONSE
Now we come to the labels for elements of a classical conditioning experiment. These are the terms which students apparently find difficult to remember. The best way to learn them is to notice what they mean. For example, think of conditional as meaning conditional upon learning or dependent upon conditions. Pavlov labeled the signal that occurs before a biologically significant event a conditional stimulus (CS). Its ability to trigger salivation is conditional, that is, "depending on conditions." It does not automatically trigger salivation in an untrained dog. Another way to remember what CS stands for is to think of the word CUES. The conditional stimulus is what cues the conditional response. Pavlov sounded a tone (the CS) before giving the dog meat powder.
What do the abbreviations CS and CR stand for? Why is the word "unconditional" used?

How did Pavlov initiate the biological reflex of salivation? He squirted meat powder in the dog's mouth. Even before learning, salivation is triggered by meat powder. So this response is said to be unconditional (not dependent upon special learning conditions). The meat powder is called the unconditional stimulus (US) It is the stimulus that provokes a response automatically. Because the salivation response to meat powder is born-in and unconditional, it is called the unconditional response (UR) Every reflex such as those listed earlierconsists of an unconditional stimulus that triggers an unconditional response. To repeat: classical conditioning occurs when a signal (CS) is put before the reflex (or paired with it as many textbooks say, although conditioning is faster if the CS comes before the reflex). For example, the tone is sounded before the meat powder is squirted into the dog's mouth. Then the dog anticipates the meat powder when it hears the tone. Therefore it salivates to the tone. The salivation that occurs in response to the tone is the conditional response (CR). It depends on a prior act of learning, hence the name "conditional." After training, the CR occurs in response to the CS (conditional stimulus).

How is the CR different from the UCR?

The unconditional response and conditional response obviously resemble each other, but they are not identical. They have different causes (one is a reaction to the biologically natural stimulus; the other is a learned reaction to a signal). They also occur at a different strength. The conditional response is typically somewhat weaker than the natural response to an unconditional stimulus. A dog does not salivate as much to the signal as it does to real food. Watson's subjects probably did not withdraw their fingertips quite as vigorously when they heard a bell as when they actually received a shock. So the two have different names. The original, reflex response is the unconditional response. The learned response is the conditional response. Due to a mistranslation of Pavlov's early works, most psychologists refer to "conditioned responses" rather than "conditional responses." Conditional is actually the correct translation of the Russian word, which is uslovnye (Fitzpatrick, 1990).
What mistranslation of Pavlov's terms occurred? Why is "conditional" arguably more meaningful than "conditioned"?

The word conditional is arguably more meaningful than conditioned. In the classic Pavlovian setup, the dog salivates because of previous experience. Therefore the CR is literally conditional upon the association of the CS with the reflex. The word conditioned carries no such meaning, unless one make a comparison between the establishment of a conditional response and the conditioning of a piece of metal by pounding it into shape-a metaphor Pavlov did not intend. Cyril Franks, a noted behavior therapist, wrote the following in an essay titled "Behavior Therapy and Its Pavlovian Origins."
What did Cyril Franks say about Pavlov's view of CRs?

Pavlov distrusted absolutism in any form, and it was no accident that he employed the term "conditional" in his writings rather than "conditioned." His intent was to convey the essentially temporary nature of the connections thus formed, connections that lacked the certainty and regularity of innate or "unconditional" reflexes. For Pavlov...the conditional reflex was a creative, emergent activity of the organism, not a stereotyped or unchanging process. (Franks, 1973)
What happened to Pavlov's theory after the communist revolution in 1917?

Franks may have written "Pavlov distrusted absolutism in any form" to counteract a widespread belief that Pavlov sold out to the communist regime after the Bolshevik revolution in 1917. Lenin, leader of the Russian revolution, had the highest respect for Pavlov, one of his country's greatest scientists. Lenin's endorsement made Pavlovian psychology the state-approved psychology of the Soviet Union. To this day, some Russian psychologists believe classical conditioning can explain a large percentage of human behavior. To be fair, some American psychologists feel the same way about operant conditioning.

ACQUISITION OF THE CONDITIONAL RESPONSE

In conditioning experiments, "acquisition" is basically learning. In acquisition, a creature acquires a conditional response. One lecturer-Thomas Landauer, then of Bell Laboratories in New Jersey-used a simple technique to demonstrate acquisition of a conditional response in a large introductory psychology class. While lecturing about Pavlov, he periodically raised a starter pistol, shouted "NOW!" and fired a shot.

How did Landauer establish classical conditioning, quickly, in a large crowd? What is the best timing, for conditioning of a simple motor response?

Conditioning occurs most quickly when the signal comes about half a second before the reflex is activated. This is the optimum interval to use for rapid conditioning of a simple motor response (muscle movement). Therefore, Landauer's timing was close to optimum. He shouted "NOW!" half a second before creating a loud noise that caused a startle response (made everybody jump). It was natural that everybody jumped in reaction to the gun report. A loud noise is an unconditional stimulus (US) that leads to an unlearned startle response (UR). Five more times in the next 20 minutes of his lecture about classical conditioning Landauer raised his arm, shouted "NOW!" and fired his pistol. The sound "NOW!" was the conditional stimulus (CS). After repeating this a few times, Landauer raised his hand and shouted "NOW!" but did nothing. Everybody jumped. He had conditioned the entire crowd in less than half an hour.
Why did the students jump?

Why did this happen? The essence of classical conditioning is putting a signal before a reflex so that an organism can get a "jump" or head start on the reflex. Remember we described classical conditioning as a primitive form of prediction. Landauer's students were making an anticipatory response. After hearing "NOW!" five times, followed immediately by a gunshot, their nervous systems learned about the predictable relationship between these events. When they heard the signal again, they responded (they jumped). Of course, it was a prediction of a simple sort: they were not making a conscious plan to jump. They just did it. Classical conditioning does not require a conscious thought process.

THE ACQUISITION CURVE

Acquisition of a conditional response often occurs rapidly, in just a few trials or repetitions of the CS-reflex pair. The Landauer example from the previous page illustrates this. He only had to shout "NOW!" and fire the pistol a few times before everybody was jumping to the word "NOW!" Experiments with animals illustrate the same thing: a conditional response can be set up quickly.

A typical acquisition curve involving 15 trials

How quickly can a conditional response be learned? What "true-to-life variability" is shown in the curve?

The figure shows a learning curve. On the Y-axis is the amount of saliva secreted during each training session. Notice there is evidence of conditioning by the 7th trial (each trial being one unit of training, one pairing of the signal with activation of the reflex). Unlike many textbook diagrams, which are smoothed, this particular example shows some trueto-life variability. It goes up and down, not just up. Conditioning does not always proceed smoothly from no response to a large response, and there may be setbacks during training when the organism does not respond as much as before.

EXTINCTION AND SPONTANEOUS RECOVERY

A classically conditioned response can be eliminated or extinguished by eliminating the predictive relationship between the signal and the reflex. This is accomplished by presenting the signal or CS while preventing the reflex from occurring. How can the reflex be prevented from occurring? One technique is to activate a behavior incompatible with the reflex. For example, a cat will be fearful of a box in which it has been given an electric shock. The cat can be given "therapy" later by feeding it in the box. That prevents the anxiety response, so the cat loses its fear of the box. Research like this led to a therapy called desensitization designed to eliminate fears and phobias. Desensitization was very successful as a therapy and marked the beginning of behavior therapy as a discipline. If a CS occurs many times but the reflex is never activated, the organism learns that the signal no longer has the same meaning as before. The signal no longer predicts the activation of the reflex, and the conditional response disappears. This overall process is called extinction. Landauer demonstrated extinction in his lecture. After establishing the conditional response (students jumping when he said, "NOW!") he extinguished it. He stuck up his arm periodically and shouted "NOW!" without firing the starter pistol. At first people continued to jump. However, as he repeated the action, shouting "NOW!" without firing the starter pistol, the response weakened. Eventually nobody jumped. By repeatedly giving the signal without firing the gun, he eliminated the predictive relationship between the conditional stimulus ("NOW!") and the CR or conditional response (the jump). Soon there was no conditional response. Extinction had occurred.
How does one extinguish a classical conditioned response? What is spontaneous recovery?

As time passes after extinction, a classically conditioned response typically recovers a bit of strength. This recovery takes place "spontaneously" without any additional training, so it is called spontaneous recovery. After Landauer first extinguished the startle response, he lectured on a different topic for 20 minutes. Then, without warning, he pushed his arm in the air and shouted "NOW!" (still without firing the pistol). Most of the crowd jumped again. Landauer had demonstrated spontaneous recovery.

Spontaneous recovery after the first extinction period

How does one completely extinguish a CR?

In order to completely extinguish a classically conditioned response, one must go through the extinction procedure repeatedly, eliminating spontaneous recovery each time it occurs. Incidentally, this diagram is idealized, which means it is unrealistically smoothed. A genuine record of behavior would never be this clean and simple.

GENERALIZATION
Generalization occurs when an organism makes the same response to similar stimuli. The size of the response typically depends on the degree of similarity. If a dog receives meat powder after hearing a 500 Hz tone, it will probably salivate when hearing a 450 Hz tone also, but not as much as it would to another 500 Hz tone. It would salivate less to a 400 Hz or 600 Hz toneand even less to a 300 Hz or 700 Hz tone. Pavlov found that the greater the resemblance between stimuli used during training and stimuli used during testing the greater the generalization. In other words, more salivation would occur if a tone was close to the training tone, less salivation would occur if the tone was very different from the training tone.
When does generalization occur? When the stimuli are tones, what determines the amount of generalization? How can generalization be a more complex issue in real-world situations?

Generalization is often an important phenomenon in real-world settings. A therapist may work with a client to reduce anxiety states triggered by some kind of stimulus (let's say, dirt and contamination) by having the client imagine the feared situation (touching dirt) while staying deeply relaxed on the therapist's couch. But will the exercises performed in the therapist's office generalize to real world settings? One can never tell. The greater the resemblance of the training situation to the testing situation, the more generalization should occur. That is why in vivo (naturalistic or real life) therapies often work best. For example, a person who is anxious about contamination by dirt is encouraged to touch actual dirt, dug from the ground, while the therapist stands by for encouragement. A person who is anxious about flying is best taught to remain calm during actual airplane flights. That helps insure the results of therapy will carry over to future real world settings where the therapist is not present. However, no two situations are identical. If an organism notices differences between situations rather than similarities, generalization will not occur. For example, a horse that responds well to one rider may be stubborn for another. Would the students in Landauer's class jump if one of their classmates stood up and shouted "Now!"? Maybe some would jump. If they did, they would be showing generalization. But others might not jump. They would be showing discrimination, the opposite of generalization. Discrimination is described in the next section.

How do little children show overgeneralization?

Generalization often involves our knowledge of the world. If two stimuli are interpreted as the same, a person will generalize between them. Little children go through a phase in which they show overregularization (a form of overgeneralization) in language and thinking. A one-year-old child may call all 4-legged creatures "doggies." This shows the crude nature of the child's categorization processes.
In what way is generalization often made the focus of therapy?

Therapists often try to help their clients redefine traumatic experiences to avoid overgeneralization. If a person has a bad experience with a member of the opposite sex, will that permanently affect all relationships with all individuals of the opposite sex? If so, that might be overgeneralization. Re-thinking or re-categorizing experiences can prevent overgeneralization. For example, the therapist might encourage a person to see that "one person hurt you" but "not all men are alike" or "not all women will hurt you."

DISCRIMINATION
The opposite of generalization is discrimination. Discrimination occurs when an organism responds differently to two stimuli. An adult scratched by an angry cat typically would notdevelop a fear of dogs. A one-year old might! Our knowledge of the world includes sharp distinctions between animals such as cats and dogs. We make discriminations a one-year-old does not yet make.
What is discrimination learning?

In classical conditioning, discrimination occurs when one stimulus triggers a conditional response but another does not. To set up discrimination in the laboratory, a researcher creates a situation in which the two stimuli predict different things. For example, a green light is followed by meat powder, but a red light is not. Soon the dog discriminates between green and red lights. It salivates to the green light but not to the red light.
How can discrimination learning be used to investigate sensory capabilities of an animal?

Discrimination is evidence that an animal notices the difference between two or more stimuli. That makes the technique of discrimination learning very useful. We cannot ask a dog whether it sees in color, but we can easily arrange a discrimination experiment that reveals the answer. Two stimuli are matched in shape and brightness, differing only in color. One is made to predict delivery of food powder. The other never predicts food powder. The positions of the two stimuli are randomized, so the dog cannot tell the stimuli apart by their position, only by their color. If the dog learns to salivate to one light and not the other, then we know it can tell the difference between them. (And, yes, experiments like this show that dogs can see in color.)
How is discrimination learning used to study young babies?

Similarly, in developmental psychology research, very young babies can provide research data through discrimination learning techniques. In one procedure, a baby sucks on a nonnutritive nipple (which is basically a pacifier connected to a computer). If the baby sucks faster when it sees the mother's face rather than a stranger's, this shows the baby can tell them apart. In other

words, it can discriminate between them. Such experiments have shown that babies can detect their mother's faces, voices, and odors.

RELEASE FROM HABITUATION

One useful discrimination learning technique is called release from habituation. Habituation is the gradual cessation (stopping) of a response when the same stimulus is repeated many times. Habituation happens reliably in a variety of situations (it is "a robust phenomenon" as scientists say). If there is a loud ticking in the room, soon you will not hear it. If you live near traffic, soon you "tune it out." If you repeat a tone, soon a baby will ignore it. But if a different sound is played, the baby may show a startle response. If that happens, then obviously the baby can tell the difference between the new tone and the earlier tones. This phenomenon, called release from habituation is an indicator of discrimination. It shows that an organism reacts differently to two categories of stimuli.
What is "release from habituation"? What are some discrimination abilities in babies?

Experiments using release from habituation have shown remarkable discrimination abilities in babies. For example, babies have a primitive sense of number. A baby can be shown a series of slides in which there are always three objects in various different positions (different objects each time). Soon the baby gets bored and stops responding to the pictures. But if a picture with four objects appears, the baby perks up and pays attention to it. Similar studies have shown that babies have the ability to distinguish between language sounds used in all different human languages before the age of one, but after the age of one their discrimination of some of these sounds disappears. Soon they discriminate only between language sounds they hear in their environments.
How did Pavlov produce an "experimental neurosis"?

Pavlov believed that being pressured to make excessively fine discriminations could trigger neurosis or mental disturbance. In one experiment, Pavlov taught a dog to discriminate between a circle and an oval. The location of the circle and the oval was changed randomly, so the dog had to discriminate between them on the basis of shape, not location. When the dog pointed its nose at a circle, it received food. When it pointed at the oval shape, it received an electric shock. Gradually Pavlov made the oval rounder and rounder. Soon it was hard to tell the oval from the circle. The dog began showing signs of distress, whining and defecating. Pavlov said this showed an experimental neurosis.
How did old-time Soviet psychologists use this concept to explain crazy Americans?

As mentioned earlier, the Soviet Union endorsed Pavlov's psychology, making it in effect the officially approved version of psychology in the country during the Communist era. I heard a Soviet psychologist who was visiting the U.S. use Pavlov's ideas about neurosis to explain the (to him) strange behavior of hippies in the mid-1960s. Observing longhaired young men tossing Frisbees on the University of Michigan campus during the late 1960s, the psychologist said with a confident gleam in his eyethat he knew what caused this odd behavior. "It is your supermarkets. You have 20 different brands of soap, and who can tell which is best? The result is neurosis." In other words, he believed that the presence of many similar stimuli, requiring finer and finer discriminations, caused American youth to be mentally disturbed.

PAVLOVIAN CONDITIONING: "IT'S NOT WHAT YOU THINK IT IS"

Robert A. Rescorla of the University of Pennsylvania was probably the most influential classical conditioning researcher in the late 20th Century. In a 1988 American Psychologistarticle addressed to his fellow academic psychologists titled "Pavlovian Conditioning: It's not what you think it is," Rescorla complained that psychologists had ignored developments in research on classical conditioning during the preceding 20 years. Rescorla cited the out-of-date description of classical conditioning in a 1966 introductory psychology textbook he had as a student. [Quoting the 1966 textbook] "The essential operation in conditioning is pairing of two stimuli. One, initially neutral in that it elicits no response, is called theconditioned stimulus ( CS) ; the other, which is one that consistently elicits a response, is called the unconditioned stimulus (US). The response elicited by the unconditioned stimulus is the unconditioned response (UR). As a result of the pairing of the conditioned stimulus (CS) and the unconditioned stimulus (US), the previously neutral conditioned stimulus comes to elicit the response. Then it is called the conditioned response (CR)." Rescorla noted, "This description is typical of those found in both introductory and advanced textbooks 20 years ago." The problem was that the same sort of description was continuing to appear in 1988, when Rescorla wrote his article, despite "dramatic conceptual changes that had taken place." In fact, it is still typical of how classical conditioning is described in many textbooks, an additional 20 years later, as we approach 2008. What was wrong with the old-fashioned description of conditioning? Rescorla cited these problems:
What were problems with many textbook descriptions of classical conditioning, in Rescorla's view?

1. The textbook description emphasized contiguity [closeness in time] of the conditional stimulus and unconditional stimulus, while modern studies emphasize the informative or predictive nature of the conditional stimulus. 2. The textbook description gave the impression any two stimuli could be associated as conditional stimulus and unconditional stimulus. Modern research shows that some stimuli are much easier to associate with a particular biological response than others. 3. The old textbook description gave the impression conditioning was slow and gradual, requiring many repetitions or trials. Rescorla wrote, "Although conditioning can sometimes be slow, in fact most modern conditioning preparations routinely show rapid learning" requiring from 1 to 8 trials. Rescorla also wanted to change the image of classical conditioning. He said, his professors back in the 60s conveyed the impression classical conditioning was "all spit and twitches," because famous experiments relied on reflexes like salivation, eyeblinks, and finger withdrawal. Rescorla pointed out that classical conditioning in the modern era is relevant to much more than spit and twitches. Classical conditioning "is intimately involved in the control of central psychological processes, such as emotions and motivation." The following section of the chapter contains many examples that support Rescorla's arguments. Classical conditioning is now known to be involved with the immune system, sexual anticipation, tolerance to addictive drugs, and much more.

SUMMARY: CLASSICAL CONDITIONING

Ivan Pavlov, a Russian physiologist who was studying digestion in dogs, discovered classical conditioning accidentally. Pavlov noticed that a dog salivated at the sight of a food bowl. Pavlov recognized this as an important phenomenon. It represented the triggering of a biological reflex (salivation) by learning (in this case, by the sight of the bowl). Pavlov studied this phenomenon in the laboratory and called it signalization. Classical conditioning always starts with a reflex : an unlearned stimulus-response circuit in the nervous system. In many situations, an organism benefits by making a reflex response to appropriate situations slightly early. Therefore animals are sensitive to cues that predict the activation of a reflex. Such signals allow the animal to make an anticipatory biological response. This is classical conditioning. Learning or acquisition of a classically conditioned response occurs when a signal or cue is put before the activation of a reflex. Learning is fastest if the signal comes about a half second before the reflex, if the reflex involves skeletal muscle movement, such as a knee jerk or withdrawal of a fingertip. Learning typically occurs after only a few pairings of signal with reflex. Extinction or unlearning of a classically conditioned response occurs when the predictive relationship between the signal and the reflex is destroyed. The signal is presented but the reflex is prevented or a competing reflex is activated. Soon the signal no longer has predictive power and the animal ceases to respond to it. After a time, or if the context is changed, an extinguished response may reappear. To completely eliminate a conditional response, the response must be extinguished several times. Generalization is the name for responding the same way to different situations. The more similar the situations, the more generalization will occur. In naturalistic settings, a conditional response may occur or not depending upon how an organism categorizes stimuli. Discrimination is the name for responding differently to distinct stimuli. It is the opposite of generalization. An animal discriminates between stimuli when it responds differently to them. An example is release from habituation. It occurs even in small babies and can be used to determine whether they distinguish between different categories of stimuli.

APPLICATIONS OF CLASSICAL CONDITIONING

Psychologists have applied the concepts of classical conditioning to many new situations in the last several decades. Here is a set of links to earlier parts of this textbook where classical conditioning is applied or put into action. 1. Chapter 2 mentioned Kagan and Snidman's (1991) speculation that shyness might be due to an overactive amygdala. Emotional reactions are generally classically conditioned reactions, as we will see in the coming section on CERs (conditional emotional responses).
What are some examples involving classical conditioning, from earlier chapters?

2. Chapter 3 described the repeated re-discovery of two modes of consciousness. The two can be doubly dissociated by brain injury. Bechara, Tranel, Damasio, Adolphs, Rockland, and Damasio (1995) used a fear conditioning classical conditioning task to show this. 3. Chapter 4 mentioned perception without awareness as a category of experiences that resemble ESP without actually involving unknown psychic powers. In fact, the "powers" involved

are probably classical conditioning, in many cases, because classical conditioning does not necessarily involve conscious processes. To use the example from that chapter, a person could respond to a police car seen in peripheral vision or in the rear view mirror without realizing what the stimulus was or where it came from, leading to what seems like an uncanny coincidence. And we could go on. Research using classical conditioning is booming in the 21st Century. However, this was not always the case. For most of the middle 20th Century, American psychologists paid classical conditioning little attention, except for teaching their introductory psychology students about Pavlov's dog. That situation started to change in the late 1960s with the discovery that individual nerve cells could be conditioned using Pavlov's methods.

CONDITIONING OF NEURONS
In the 1960s and 1970s scientists found they could carry out classical conditioning with neurons surgically removed from simple animals such as sea slugs. The clear implication is that classical conditioning is a very basic form of learning not even requiring a complete nervous system or a brain. That helps to explain why classical conditioning can be involved in all sorts of unconscious biological processes.
How was classical conditioning demonstrated in a single neuron?

Toledo-Morrell and colleagues (1979) did one such experiment. In their case, the signal (conditional stimulus) was activation of an incoming synapse, not strong enough by itself to make the cell fire a nerve impulse. The researchers simply followed the signal with an input that was strong enough to trigger a nerve impulse. After repeated pairings, activation of the incoming synapse was enough to trigger a nerve impulse by itself.
What changes were observed at synapses involved in conditioning?

Similarly, Farley and colleagues (1983) showed classical conditioning in a single photoreceptor cell of a mollusk. Kelso and Brown (1986) showed classical conditioning of individual neurons called pyramidal cells from the rat hippocampus. Other researchers showed that classical conditioning produces noticeable changes at the junction between neurons, the synapse. A synapse involved in successful conditioning often showed a thickening on the presynaptic side (before the synapse). Eric Kandel of Columbia University won a Nobel Prize in 2000 (shared with Arvid Carlsson and Paul Greengard) for his decades-long research program on this and related topics. The main laboratory task Kandel used to study neural changes induced by learning was classical conditioning of the sea slug or Aplysia.

CONDITIONING AND DRUG TOLERANCE

One form of unconscious learning that appears to be due to classical conditioning is drug tolerance Drugtaking behavior (such as using a needle or even opening a bottle of beer) functions as a signal or CS that predicts the introduction of the drug into the body. Eventually the act of drug taking triggers an anticipatory response: the secretion of drug antagonists that help eliminate the drug from the body.

How does classical conditioning explain drug tolerance? What factors increase the risk of overdose from heroin, and how does this relate to the rat studies?

The ability of experienced drinkers to "hold their liquor" (consume a lot of alcohol without showing much effect) is a sign that the body is adapting to the drug. Classical conditioning has occurred. Alcohol-consumption now triggers a strong anti-drug action that reduces the effect of the drug. The same thing happens with cigarette smoking and coffee drinking. As a person becomes addicted, the drug has less and less effect. Siegel and colleagues (1981) demonstrated that drug tolerance was due to classical conditioning. They gave rats morphine, a relative of heroin. Sometimes the rats received a signal followed by a period of no morphine. Therefore the signal indicated a drug-free period. Gradually the rats built up a tolerance to the morphine. Its pain-killing effects disappeared. But when morphine was given after the signal (typically a drug-free period) the rats had no tolerance for the morphine and the full pain-killing effects returned. Intrigued by this finding, Siegel and co-workers (1982) gathered data on heroin overdose deaths in humans. They found that victims of heroin overdose were typically occasional or weekend users, not daily users. Often the fatal overdose occurred in unfamiliar environments. Drawing an analogy to their rat studies, the researchers proposed that addicts who took the drug in an unfamiliar setting, or took it after a period of time not using the drug were in special danger of overdosing. Their bodies failed to perform the usual anticipatory response of secreting opiate antagonists, so they had less tolerance.
What are possible reasons that Sid Vicious overdosed on heroin after being released from prison?

The punk rocker Sid Vicious, subject of the movie Sid and Nancy, died of a heroin overdose the day after being released from prison, pending trial for the murder of his girlfriend Nancy. He died upstairs in his mother's house. Probably there were two things that made his overdose more likely. He had been drug-free for several weeks while in prison, and his mother's house was not his usual drug-taking environment. Therefore a classically conditioned response did not take place. His body did not secrete opiate antagonists in the usual amount, and he did not tolerate his usual dosage of heroin.

CONDITIONAL IMMUNE RESPONSE

A potentially important application of Pavlovian conditioning involves the body's immune system. Like other body systems, it can be activated or suppressed through classical conditioning. This has exciting implications. If learning can stimulate immune system activity, people should be able to arrange conditions to improve health or healing. Perhaps humans have already been doing this for thousands of years. Classical conditioning may shed a light on healing rituals and trances practiced by pre-modern cultures. An early experiment reported by Schmeck (1985) involved a team of researchers at the University of Alabama medical school. They studied effects of classical conditioning on activity of natural killer cells (NK cells) that destroy germs and other invaders in the body. In the experiment, mice were exposed for three hours at a time to a powerful odor (camphor ). Exposure to this odor, by itself, had no effect on the mice. Next, the odor was made to predict a significant biological event. After exposure to the odor, mice in an experimental group were given injections of a synthetic chemical called poly I:C (for polyinosinic-polycytidilic acid) that stimulates activity of natural killer cells. Mice in the control group did not receive the poly I:C.

How did researchers demonstrate a conditional immune response in mice?

For the experimental group, the odor of camphor was paired with exposure to Poly I:C nine times. In the 10th session, the mice were exposed only to the odor of camphor. Every mouse in the experimental group showed large increases in natural killer cell activity. Their bodies were "predicting" the injection of poly I:C and responding with immune system activity. In the control group, which was exposed only to the odor of camphor, no such response occurred. This is typical of research on classical conditioning. It is capable of demonstrating remarkable, subtle biological effects. However, analyzing the exact mechanism can be difficult. How exactly does a mouse's "knowledge" that poly I:C is about to be delivered to its bloodstream stimulate the production of NK cells? If researchers knew that, perhaps they could help human patients boost production of NK cells when needed, as well. Researchers suspect that neuroimmunomodulation takes place at every level of the nervous system. The word neuroimmunomodulation contains the word roots for nerve (neuro) andimmune (immuno). So neuroimmunomodulation means modulating immune system activity with nervous system activity. The discipline of psychoneuroimmunology-arose in the 1970s and 1980s to study psychological influences on immune system functioning. However, research in this area has produced disappointing results. The positive effects of psychological intervention on health are easy to document, but evidence relating these benefits to immune system changes has been elusive (see the section on psychoneuroimmunology in Chapter 14).

CONDITIONAL EMOTIONAL RESPONSES

Conditional emotional responses (CERs ) are learned emotional reactions like anxiety or happiness that occur as a response to predictive cues. Most American psychologists use the -ed form of the word, calling CERs "conditioned emotional responses." However, to be consistent with earlier arguments about the translation of Pavlov's terms from Russian to English, we will stay with the term conditional except when quoting people who use the -ed form. The process of acquiring a CER is similar to the process of acquiring any other conditional response. A signal comes before a biologically significant event, and the organism learns the correlation (literally "making the connection" on a neural level). For example, we could sound a tone and give you an electric shock. Such fear conditioning establishes a CER quickly. After one or two pairings, the sound of the tone will send a wave of apprehension through amygdalar circuits of any organism capable of processing fear.
What is a CER? What is the ideal CS-UCS interval for an emotional response?

The ideal interval for creating a CER is somewhat longer than the ideal interval for creating a conditional response involving skeletal muscles. You might recall that the ideal CS-UCS interval for motor responses (responses involving skeletal muscle movement, such as finger withdrawal) is half a second. For emotional responses, the ideal interval is 2-10 seconds, sometimes longer, and the timing does not matter as much as it does with motor responses.
What are some CERs commonly seen by therapists

Emotional responses are typically regulated by the autonomic nervous system. As discussed in Chapter 2, the autonomic nervous system consists of two subdivisions, the sympathetic and parasympathetic systems. They have largely opposite functions. The sympathetic nervous

system is activated in the so-called "fight or flight" reaction, which produces a raised heartbeat, sweating, and other symptoms of arousal.
What did Eysenck assert?

Many psychologists believe CERs involving the sympathetic nervous system are responsible for panic attacks, stage fright, test anxiety, and similar unpleasant emotions. These responses tend to be unconsciously learned and therefore difficult to control, so they drive people to seek help. British psychologist Hans Eysenck once asserted, "...all neuroses are essentially conditioned emotional responses" (Cunningham, 1984).
How does a CER get set up in natural situations?

How does a CER get set up, in natural circumstances? All it takes is an experience that causes strong emotion. In the case of CERs that send people to therapy, the strong emotion is a negative emotion such as pain, fear, or anxiety. A car crash, for example, will normally be preceded by certain stimuli such as driving a car. After the car crash, the prospect of driving a car might fill a person with dread. The dread would be a CER.
How can a teacher trigger a CER with a few words?

I asked a colleague, Doug Marshall, if he had a handy, real-life example of a conditional emotional response. He said, "Sure. Walk into your class and say, 'Take out a blank sheet of notebook paper, and put your books on the floor.'" These words uttered by a teacher lead to a wave of anxiety in college students. The anxiety is an anticipatory response to a stressful event: a surprise test.
How does odor become a CER?

CERs can involved any stimulus, including odor. The emotions involved can be pleasant, or highly complex. A certain odor may evoke memories of a grandparent's home. Students who love horses often report that the smell of horse manure is pleasant for them, no doubt because of good associations with horses. My children, when young, told me they loved the smell of gasoline. Perhaps they associated it with pleasant trips in the car, or with rides around the yard on the lawn tractor. The smell of gasoline might provoke a very different reaction in someone recovering from a car wreck. After breaking up a close relationship, many people respond to the smell of perfume or cologne that their ex-lover wore; it creates a wave of nostalgia, regret, or perhaps relief in some cases.

SEXUAL ANTICIPATION
Sexual response is another complex reaction modulated by neurochemicals. Like the others, it is subject to classical conditioning. Graham and Desjardins (1980) provided male rats with sexual partners in the same environment each day. Soon the rats showed an eightfold rise in sex hormones when placed into the cage, even when there were no other rats present.
How did Graham and Desjardins demonstrate classical conditioning of sex hormone response? What was a similar finding with quail?

Domjan, Blesbois, and Williams (1998) did a similar experiment with quail. They did not find a rise in testosterone, but they found that male quail produced more sperm of better quality after being presented with a stimulus (the sight of a female's head and neck) that previously came before

copulation. The researchers pointed out the implication: quick learning of the Pavlovian variety can improve an animal's reproductive and evolutionary success. A student reported how classical conditioning was used to help gather sperm for artificial insemination of horses: I wanted to tell you about a form of classical conditioning that I've encountered. In high school. I planned on majoring in veterinary medicine, and I attended a number of workshops offered by the University of Georgia. In one workshop we were learning about sperm collection for artificial insemination of farm animals. Apparently in previous years they had a student hold a stallion in place while other students paraded mares in front of the stallion until he was aroused. Then the mares would be escorted out and the stallion would be allowed to mount a fake horse that would collect the sperm in a bottle attached to it.
How did agriculture experts use classical conditioning to help with artificial insemination?

Realizing that students could get hurt, the agriculture officials chose to use classical conditioning. They kept the stallion in his pen and several times they let the mares in. Just before the mares entered a red flag was raised. Eventually the stallions would become aroused when the flag was raised. That meant nobody had to hold the stallion or parade the mares; they could just raise the flag, open the pen, and the stallion would mount the fake horse. [Author's files]
How might paraphilias be due to classical conditioning?

Some psychologists speculate that odd sexual attractions, technically called paraphilias, are the result of classical conditioning. For example, shoes are sexually arousing to some people. Perhaps that person experienced the sight and smell of a shoe right before a sexually arousing experience. The result might be a "shoe fetish" (sexual arousal from sight or contact with shoes). There is no proof this explanation is correct, but it is one of the few reasonable ways to explain paraphilias, many of which are quite odd and otherwise hard to explain.

CERS FROM TRAUMATIC EVENTS

Conditional emotional responses (CERs) can be set up by a single traumatic incident. On July 17, 1981, a poorly designed skywalk in the Kansas City Hyatt Regency Hotel collapsed beneath a dancing crowd. Many people died. Psychiatrist Charles Wilkinson studied survivors. He found that nearly half of those surveyed startled easily. "Sudden noises, particularly those which reminded subjects of the skywalks tearing away from their supports and falling, produced striking overreactions, often to the subject's embarrassment." (Creekmore, 1984)
What are common ways that survivors of traumatic incidents show CERs?

Survivors of World Wars I and II reported conditional emotional reactions to loud, sudden sounds that reminded them of bombs or to sirens that resembled air raid sirens. Soldiers returning from Iraq may feel anxiety and fear when a car backfires nearby, making a sound like a gunshot or improvised exposive device. "A pothole gets them jittery because it reminds them of potential bombs," one social worker on an army base said (Urbina, 2007).The reaction is like a reflex; it occurs before they have time to think about it. In Chapter 13 (Therapies) we discuss how classical conditioning can be used in a therapy technique called desensitization to extinguish such responses. Wolpe and Rachman (1960) made a famous re-interpretation of one of Freud's classic case studies, the case of Little Hans, which they re-interpreted as a conditional emotional response

(CER). Little Hans was a five-year-old boy who developed a fear of horses after an accident involving a horse-drawn carriage. Freud interpreted the boy's fear of horses as symbolic of sexual conflicts. The problem (Freud said) was that Hans had unconscious sexual desires for his mother and wanted his father out of the way. Hans feared that his father knew this and might castrate him. The accident symbolized castration. The horse, a penis symbol, was an uncomfortable reminder of his father's power. So, as Freud explained it, fear of the father was converted into fear of the horse. (This is actually quite typical of Freud's thinking.)
How did behaviorists re-interpret Freud's Little Hans story?

Wolpe and Rachman mocked Freud's explanation and pointed to an obvious alternative. They suggested that the boy developed his fear of horses as a conditional response. The sight of the horse preceded the carriage accident. Therefore when Little Hans saw a horse (a conditional stimulus) he grew fearful and anxious (a conditional response). To most psychologists, this is a much better explanation than Freud's. It is not only less speculative; it is based on a well-known phenomenon that can be demonstrated in any laboratory or classroom: the conditional (or "conditioned") emotional response.

TASTE AVERSION
Taste aversionlearning to avoid a food that makes you sickis an intriguing form of classical conditioning. The signal or CS is the taste of a food. The reflex that follows it is sickness. Organisms quickly learn to associate taste with sickness. Taste aversion is interesting to researchers because it appears to violate several rules of classical conditioning.
What is taste aversion and how is it unusual among examples of classical conditioning?

1. It emerges in full strength after only one CS-UCS pairing. 2. The CS-UCS gap is very longup to six hours or moreyet the conditioning is strong. 3. The association is very selective. People associate the sickness with the odor or taste of a food, not with sights or sounds or other stimuli in the environment. 4. The learned response resists unlearning. Taste aversion can occur even though a person knows that an illness occurred because of a virus, not because of food. It does not matter; the body jumps to the conclusion that the food was bad, and the food becomes repulsive to us. In ancient times, that was a good thing. Food was often the cause of illness, and it was important to learn quickly to avoid ingesting the same poison twice. In modern times, we might know intellectually that a virus was responsible for our illness...but that does not matter. The heart (or gut) overrules the head, and the innocent food repels us. This illustrates again how classical conditioning involves automatic, involuntary, primtive processes in the human brain.
How did John Garcia achieve fame?

The tendency to blame food for illness, even if the food had nothing to do with the illness, is called the Garcia Effect. John Garcia gave rats a radiation treatment that made them sick after

they drank sweetened water in a red-lighted room. He found that after getting sick the rats would then avoid sweetened water, but they would not avoid red light. This made sense to Garcia (food or water can make animals sick, light normally cannot). However, the idea that animals would naturally associate sickness with one stimulus instead of another flew in the face of established wisdom in the 1960s. Most psychologists at that time believed that all stimuli should be equally easy to associate. Garcia's articles were initially rejected by prestigious publications like Science, but he did not give up. He published in lesser journals and continued to replicate his experiment in several variations. Eventually his evidence convinced the scientific establishment, and Garcia's name was attached to the new phenomenon.
What is tumor anorexia? How might it be explained?

Bernstein (1978) reported that children receiving chemotherapy, which causes extreme nausea, develop taste aversions for foods consumed before treatment. Tumor anorexia is a generalized loss of appetite experienced by cancer patients who are not necessarily receiving chemotherapy. Bernstein and Sigmundi (1980) suggest tumor anorexia may be due to a generalized conditional aversion to the entire diet. In effect, all foods are associated with feelings of illness, so the patient develops an aversion to all eating.
What is bait shyness? How did psychologists make it work in favor of ranchers?

Taste aversion can work against attempts to control predation with poison bait. Perhaps you have heard the term bait shyness Ranchers in the western United States put poison bait in their fields to kill coyotes that preyed on herds of sheep. After the coyotes sampled the bait and got sick from it, they became "bait shy" and would not touch it, so the bait no longer worked. This is a form of taste-aversion conditioning. Psychologists (including Garcia) suggested a different strategy that would make taste aversion work in the rancher's favor. The ranchers put out poisoned bait that was mutton (sheep meat). Again the coyotes sampled it and got sick. However, this time the "bait shyness" worked to the ranchers' advantage. The coyotes developed an aversion to mutton and left the sheep alone (Gustafson, Kelly, Sweeny, and Garcia, 1976).
How did Nicolaus and colleagues combat egg predation in crows?

Nicolaus and colleagues (1983) showed that taste aversion could be used to control crow predation on eggs-a problem for bird sanctuaries and farmers with outdoor chickens. The researchers put a sickness-causing agent in several eggs, and then left them where crows could get them. This eliminated the egg-eating habit in a population of crows. A therapy called sensitization attempts to do the same thing with humans, conditioning them against drinking or smoking cigarettes by deliberately making them sick when they indulge.

SUMMARY: APPLICATIONS OF CLASSICAL CONDITIONING

Classical conditioning is apparently a very basic form of learning. It can be demonstrated even with single nerve cells. Conditional release of neurotransmitter antagonists may underlie the phenomenon of drug tolerance. This helps to explain why overdoses are more common after a period of abstinence, or when the drug is taken in an unfamiliar setting. Under those conditions, conditional release of antagonists is less likely and tolerance is reduced. Classical conditioning has also been shown to affect responses of the immune system. This is important because it implies that conditioning techniques might affect health.

A CER or conditional emotional response is a learned response to cues that occur before an emotionally powerful event. CERs can result from pleasant or unpleasant emotions. Those involving very unpleasant emotions sometimes motivate people to seek therapy, because (like other forms of classical conditioning) the responses are largely involuntary and resistant to conscious control. CERs can be pleasant, for example, when odors awaken fond memories. Taste aversion is a special form of CER that occurs when a person gets sick after eating a particular food. Even if the illness is known to be caused by a virus, not by the food, the taste aversion can be strong and durable.

OPERANT CONDITIONING
Pavlov had many early admirers in America. John B. Watson, who gave behaviorism its name, studied Pavlovian conditioning early in the 20th Century. However, by the 1940s and 1950s American psychologists spent most of their time studying operant conditioning, a form of learning distinct from Pavlov's. Operant conditioning does not involve triggering reflexes. Operant conditioning involves exploratory or goal-seeking actions and their consequences. Many of today's students have heard about operant conditioning. It is typified by the rat in a Skinner Box (itself named after B.F. Skinner, a leading proponent of operant conditioning). The rat presses a lever that sticks out from the side of the cage. The goal is to obtain food reinforcement.

Distinguishing Between Operant and Classical Conditioning

Here are some contrasts between classical and operant conditioning.
What are differences between classical and operant conditioning?

Classical vs. Operant conditionings

How are the concepts of "elicited" and "respondent" behavior related?

How does operant conditioning contrast with Pavlovian or classical conditioning? Classical conditioning always involves anticipatory biological responses triggered by a signal. The response is drawn out of the organism or elicited. In operant conditioning, by contrast, the animal generates the behavior on its own, as a way of achieving a goal. The behavior is emitted.

Operant conditioning always involve behavior, which is basically the same thing as activity. A behavior reinforced or punished in operant conditioning is called an operant. Biologically, classical conditioning is more likely to involve the autonomic nervous system the one response for gut reactions and emotions. Operant conditioning is more likely to involve largescale motor movements of the sort humans consider voluntary. As discussed earlier in this chapter, memory for classically conditioned responses occurs throughout the nervous system at the neural level, while memory for patterns of operant responses (i.e. complex non-instinctive behavior) typically requires the hippocampus, the part of the brain responsible for event memory. The two forms of conditioning are intermingled within the living organism, and they are not always conceptually distinct even to human psychologists. For example, each successful reinforcement in operant conditioning (such as giving a food pellet to a rat) triggers a reflex such as swallowing food. Operant and classical conditioning effects are mingled in course of normal animal behavior. Some behaviors, such as key pecking in the pigeon, can be studied as reflexes or as operants with equal success. However, for the beginning student, the challenge is to tell these two forms of conditioning apart. Usually the easiest way to do that is to look for a reflex: a biological, born-in stimulus-response circuit. If the reflex is activated by a signal, then one is talking about classical conditioning. If the animal is engaging in something like exploratory or strategic activity followed by payoff or a punishment, then one is talking about operant conditioning.
What is a formal definition of operant?

Technically, an operant is defined as one of a class of behaviors thatoperate on the environment in an equivalent way. For example, a pigeon pecks a key. This is an example of the key peck operant, because the pigeon operates on the environment. Even if it kicks the key with its foot instead of pecking it, the operant is the same. An operant is defined by itseffect. Perhaps the most-studied operant in American psychology is the bar-press response. A rat presses down a little bar sticking out from the side of a metal box called an operant chamber. Any behavior that results in the bar being pressed-whether the rat does it with its paw or its nose-is called a bar-press operant because the effect (the bar-press) is the same.

STARTING OUT IN A "RAT LAB "

If you signed up for a psychology class titled Conditioning and Learning or Animal Learning Laboratory, you might find yourself in a so-called "rat lab." Although many students start out with negative feelings about rats, they often end up enjoying such a class. Rats turn out to be friendly, intelligent creatures that often enjoy human company when tamed. Many a retired rat lab rodent has lived out its life as a human pet. During its working life in the lab, the rat serves as an ideal subject for studying learning under controlled conditions.
What are typical features of an operant chamber?

The operant chamber or Skinner Box for a rat

Often the first thing students do in a rat lab is handle the rat. This helps the rat become accustomed to human contact and not be too anxious to perform during future experimental sessions. The next task might be to observe the rat in the operant chamber, a little cage where conditioning takes place. A typical operant chamber is shown here. Researchers can put a light or buzzer on the wall, or dispense food pellets one at a time from the automated dispenser. A lever (labeled bar) sticks out of the wall of the operant chamber. This is the switch that the rat learns to press to get food. The food might be Purina Rat Chow or some other commercial rat food formed into little pellets that can be dispensed automatically.
What are early activities of students in a rat lab?

If you are a student just starting out to learn about operant conditioning, you may be asked to watch a rat in the operant chamber and write down all its behaviors for fifteen minutes. If the rat presses the bar, you make a note of it. The rat may not press the bar at all, or it may press the bar a few times. The unreinforced rate of response is the baseline level of responding. Next comes a procedure called magazine training. ("Magazine" in this context means "storage area.") Each cage has a food cup, the magazine, where food pellets are delivered. The researcher can press a button on a remote control unit to deliver a food pellet at any time. To teach the rat where this food is delivered, a student can moisten a few pellets of food to increase their odor, rub them on the hopper, and leave a few pellets in the hopper. The rat, if deprived of food overnight, will go toward the odor of the moistened pellet, find the pellets in the hopper, and consume them.
What does a rat learn during magazine training?

Now the rat knows where to find food. After the food is consumed, the rat starts exploring the cage looking for more food. The student in the rat lab waits until the rat is near the food magazine, preferably facing it. Then the student presses a remote control button that causes a dry food pellet to drop into the metal dish with a clattering sound. The rat quickly learns that a certain stimulus (the sound of a dry food pellet clattering into a bowl) can be followed by a response (running to the food hopper) that will be reinforced(followed by a stimulus, in this case food, which makes the behavior more frequent in the future). So far, the only behavior that has been reinforced is running toward the food bowl.

What is shaping?

Now a student might start an operant conditioning procedure called shaping, also called handshaping and the method of successive approximations. Shaping is a way to get an animal to perform a new behavior. The behavior that the animal is supposed to learn is called target behavior. In an operant conditioning laboratory using rats, the first target behavior after magazine training is likely to be the bar-press, which is the act of pushing down a little rounded metal bar that protrudes from the side of the operant chamber.
How is a rat shaped to press the bar?

The method of successive approximations or shaping consists of reinforcing small steps toward the target behavior. First the student gives the rat a food pellet when it is anywhere near the little metal bar. The rat learns this and soon it stays near the bar. Then the rat may be required to touch the bar before it receives a food pellet. (It is likely to touch the bar at some point, if it is hungry and exploring.) Finally, it is required to press the bar to receive a food pellet. This process of reinforcing gradual changes in behavior is shaping or the method of successive approximations. We will discuss shaping more in the later section on applied behavior analysis.

POSITIVE REINFORCEMENT
Reinforcement is defined by its effect on behavior. Specifically, a reinforcer is any stimulus that increases the frequency or probability of a behavior it follows. This is a functionaldefinition. A reinforcer is defined by how it works or functions.
What is the functional definition of reinforcement? How do you know if something is a reinforcer?

One can usually make an educated guess about what reinforcers will work to increase the frequency of behavior. Food usually works with hungry animals. But one never knows for sure what stimulus will function as a reinforcer until it is actually tried. If it increases the frequency or probability of a behavior it follows, then it is a reinforcer, by definition. This is true even if the stimulus does not seem like it should be a reinforcer. The word positive in positive reinforcement does not refer to the pleasantness of the stimulus. It means a stimulus is added or applied to the situation. Any stimulus that works to increase the frequency of a behavior it follows is a positive reinforcer, even if it does not seem like it should be rewarding. That turns out to be an important insight, as we will see later in the chapter. For example, sometimes a behavior intended as punishment (such as yelling at a child) can function as a reinforcer, making undesirable behaviors more frequent instead of less frequent. A stimulus intended as a reinforcer may not function as a reinforcer. One must measure results before deciding. If a stimulus actually has the effect of reducing the frequency of the behavior it follows, then it is a punisher, not a reinforcer. Praising a student in class for raising a hand and offering an answer, for example, may be intended by a teacher as encouragement for the student. It is intended as reinforcement. But if the student never again raises his or her hand, then (in effect) the student was punished by this attention.
What are primary and secondary reinforcers? What are examples of each?

Psychologists commonly distinguish between primary and secondary reinforcers. Primary reinforcers are unlearned. They are based on some biological drive or need, such as the need for food, water, attention, or a comfortable temperature.

Secondary reinforcers, by contrast, are learned or symbolic reinforcers. They have value because they can be used to obtain other reinforcers or have been associated with other reinforcers. Money is an example of a secondary reinforcer. You cannot eat or drink money or satisfy any biological need with it, but you can exchange it for other things that do satisfy basic needs. Grades are secondary reinforcers, worthless in themselves, but a means to obtain other reinforcers like pride and fruits of employment after graduation.

THE PREMACK PRINCIPLE

How do you find a reinforcer that will "work" with a particular individual? One technique is to observe what the individual chooses to do voluntarily. As a rule, preferred behaviors can be used to reinforce unpreferred behaviors. This is called the Premack principle after psychologist David Premack who proposed the rule. A formal statement of the Premack principle is as follows: high-probability behaviors (those performed frequently under conditions of free choice) can be used to reinforce lowprobability behaviors.
What is the Premack principle?

If a rat likes to run on a wheel, you can reinforce a bar-press by allowing it to run on its wheel only after a bar-press. If you have a hamster that always wants to escape from its cage, then you can reinforce the hamster for climbing onto your hand by offering your hand as a way to escape from the cage. The preferred behavior (escaping) is a reinforcer for the non-preferred behavior (climbing onto the hand of a human). This actually works. Parents sometimes use a variation of the Premack principle. If they want children to eat vegetables (sometimes a non-preferred behavior) they can make a rule that vegetables must be consumed before the children get to eat a dessert (usually a preferred behavior). Similarly, children may be asked to do their homework (sometimes a non-preferred behavior) before watching TV or playing video games (usually a preferred behavior).

EXTINCTION OF AN OPERANT RESPONSE

What happens when reinforcement stops? The animal may still perform the behavior for a while, but eventually-if the reinforcement never resumes-the animal will stop performing the behavior. This is the operant conditioning version of extinction. The behavior that disappears is said to be extinguished. As with classical conditioning, the extinguished behavior typicallyreappears after a time, in what is called spontaneous recovery. An extinction procedure must be carried out several times to be totally effective.
How is extinction carried out, in operant conditioning? What is a simple way to explain spontaneous recovery?

To understand spontaneous recovery, it helps to put yourself in the place of the laboratory animal. If you were receiving food pellets at a particular place (a food bar, let us say) and one day the food did not arrive, how would you interpret the situation? You might conclude that the food was permanently gone, but you might also wonder if the problem was temporary. You would check again later. Spontaneous recovery is like testing the possibility that delivery of reinforcers will resume at a later time. If extinction continues through the period of spontaneous recovery (the food never comes back) eventually the animal gives up entirely, and extinction is complete.

NEGATIVE REINFORCEMENT
Negative reinforcement occurs when a behavior is reinforced by removal of a stimulus. The word "negative" does not mean "unpleasant." It means a stimulus is removed or "subtracted" from the situation as a form of reinforcement.
What is negative reinforcement? What does the word "negative" refer to? Why do students often seem to misunderstand the concept?

Negative reinforcement is one of the most misunderstood concepts in all of introductory psychology. Students commonly assume that the word negative refers to something unpleasant, so they jump to the conclusion that negative reinforcement is a form of punishment. But negative reinforcement is not a form of punishment. Negative reinforcement is a form of reinforcement. It increases the frequency or probability of a behavior by "taking away something bad." A colleague (Dr. Gary McClure) suggested that students should think of arithmetic. Positive means something is added, while negative means something is taken away. Negative reinforcement is the type of reinforcement in which something is taken away. Here is a simple example of negative reinforcement. Suppose your teacher said you could skip the final exam by studying an extra chapter and taking a quiz on it. You might study an extra chapter (your studying behavior would be made more frequent) because of the promise of an unpleasant stimulus being removed (no final exam). For additional examples see the later section on using negative reinforcement.

PUNISHMENT
Punishment occurs when a stimulus is applied and has the effect of making a behavior less frequent. Sometimes this is called positive punishment. "Positive" in this context means a stimulus is added. However, few psychologists use the word "positive" when discussing punishment. This would only make beginning students more confused! When you see the word punishment by itself, this means an aversive stimulus is applied .
What is punishment? What is a punisher, by definition?

Punishment, like reinforcement, is defined by its effect. A stimulus that decreases the probability of a behavior it follows is a punisher, by definition. Some surprising things can be punishers. We used the example earlier of praising a student who raises his or her hand in a classroom. If the student never again raises his or her hand, then praising the student was punishment , by definition. Similarly, if you praise a child for drawing pictures, and the child stops drawing pictures, then your praise was punishing rather than reinforcing. We will provide examples of all these concepts in the next section, on applications of operant conditioning. For now, get acquainted with the basic ideas, especially this concept of defining

reinforcement and punishment by their effects. It is one of the more subtle and valuable aspects of operant conditioning. It can give you an "edge" in analyzing certain situations. When you see a behavior get more frequent, you look for the stimulus that follows that behavior, and then assume the stimulus is a reinforcer. This can allow you to identify some weird reinforcers, like yelling at a child, or giving sympathetic attention to a pet. Such reinforcers can establish troublesome behavior patterns, because people do not recognize that a stimulus is functioning as a reinforcer. See, for example, the later section on the punishment trap that occurs when parents ignore children and only pay attention to bad behavior.
Why do some psychologists avoid using the word "reward" as a synonym for "reinforcement"?

Some psychologists caution against using the word reward as a synonym for reinforcement , because a reward is usually intended as a reinforcement but may not actually function as a reinforcer. Whether a reward functions as a reinforcer depends on whether it increases the frequency of behavior it follows. If not, it is not a reinforcer. A reward (something intended as a reinforcer) may actually function as a punisher, if it has the effect of decreasing the frequency or probability of a behavior it follows. The only way to tell is to observe the behavior.

RESPONSE COST ("NEGATIVE PUNISHMENT")

Response cost or negative punishment is another way to make behavior less frequent. It is therefore a form of punishment. It occurs when a stimulus is taken away as a consequence of behavior and the effect is to reduce the frequency of the behavior. The word "negative" in "negative punishment" comes from the fact that a stimulus is removed.
What is response cost or negative punishment? How are penalties negative punishment?

In general, any time you use the word penalty you are talking about response cost. A speeding ticket is a negative reinforcer. Your money is taken away to reduce the frequency of speeding behavior. This is a form of punishment because it is a stimulus that makes the behavior it follows less frequent in the future. Technically it is negative punishment because a stimulus is removed or subtracted as a form of punishment. The alternative label-response cost- is perhaps more intuitive. It labels the fact that a response (such as driving too fast) "costs" you. Extinction and response cost both make a behavior less frequent by taking away something good. The distinction between them is fairly subtle, but here it is. In extinction the reinforcer that maintains a behavior is withheld. This means the behavior has been analyzed and the reinforcer causing the behavior has been identified and taken away. That produces extinction. An example would be extinguishing the bar press operant by turning off the food dispenser in a Skinner Box.
How is extinction distinguished from response cost? Why is a speeding ticket response cost rather than extinction? How would you extinguish speeding?

Response cost, by contrast, involves any valued stimulus being removed, whether or not it caused the behavior. If you get a speeding ticket, your money (a valued stimulus) is taken away from you. However, the money was probably not the reason you were speeding. Therefore a speeding ticket is categorized as response cost (negative punishment) rather than extinction.

How would you extinguish speeding? If a person drove fast for thrills, then to extinguish speeding one would have to eliminate the thrills. This might occur naturally if, for example, a person matured and eventually grew bored with driving over the speed limit. The result would be extinction of speeding behavior in that individual. The usefulness of all these concepts is directly linked to their abstract quality. People have intuitions about what is reinforcing and punishing, and often these intuitions are wrong. By stepping back and analyzing the situation ("Is a stimulus being added or subtracted? Is the behavior getting more frequent or less frequent?") one can categorize the situation and identify a reinforcer or a punisher.
How can analysis of operant behavior produce insights?

For example, you might know somebody who teases you. You respond in a way that you assume will make the teasing stop, for example, by showing some irritation. But if the teasing gets more intense instead of stopping, then your response functioned as reinforcement. Whatever you did, it must be reinforcement because the frequency of teasing increased after the stimulus was applied. Therefore it is time to try something different. One web site that advertised a dramatic new technique for reducing bullying. The author found that protests by the bullied child were ineffective and actually encouraged more bullying, but he said he had discovered a technique that worked much better. It consisted of teaching a bullied child to react to a bully with friendliness. The author of the web site was not very familiar with behavioral techniques and reacted with astonishment when I mentioned that, technically, his procedure was punishing the bully. To him, punishment had to hurt! In the anti-bullying technique he described, however, the punishing stimulus was friendly behavior, because it reduced the incidence of bullying.

A 2 X 2 TABLE OF CONSEQUENCES
A handy 2x2 table of consequences shows the types of reinforcement and punishment we just discussed.

Four types of stimuli that change the frequency of the behavior they follow

STIMULUS CONTROL
Stimulus control is a term used to describe situations in which a behavior is triggered by the presence or absence of some stimulus. For example, if you always eat when you watch TV, your eating behavior is controlled by the stimulus of watching TV. (This can be an important insight to some people.) If you are talkative with your friends but you never speak out in a classroom, your speech behavior is controlled by your social environment.
What is stimulus control? What is the difference between consequences and antecedents?

Antecedents (ant-a-SEED-ince) are things that come before. In operant conditioning, antecedent stimuli are those occurring before a behavior. Teachers of operant conditioning sometimes say behavior is controlled by its consequences. That sums up much of operant conditioning, but the statement is incomplete. Antecedents can also control behavior. When they do, it is called stimulus control. Here is an example. When we have a powerful thunderstorm in our lightning-prone area of the country, my wife and I unplug our computers. Our behavior is "controlled" by the occurrence of the thunderstorms, which are potentially antecedent to a damaging electrical surge. Even the best surge protectors cannot protect against a nearby lightning strike. By reacting to the antecedent stimuli of thunderclaps, we attempt to avoid the punishing stimulus of ruined computers.

DISCRIMINATIVE STIMULI
Psychologists say an operant behavior is under stimulus control if it is triggered (or suppressed) by certain stimuli. Because an organism must discriminate between these stimuli or "tell them apart" in order to respond to them in different ways, these are called discriminative stimuli.
What is an S+? What is an S-? How can you create an S+ for an animal?

An S+ is a discriminative stimulus that tells an animal reinforcement is available. An S- is a discriminative stimulus that tells an animal reinforcement is not available. Animals quickly learn to approach an S+ and avoid an S-. A stimulus is made into an S+ by consistently following it with reinforcement. For example, one student told how her father tossed a hula-hoop into a fishpond then tossed food inside the ring. After a few weeks, fish would swim to the inside of the ring as soon as it hit the water. (She said her father originally intended to use this technique to catch the fish for eating, but he became fond of his trained fish and would not eat them.)
How do teachers observe the effects of time, or words, as an S+?

Time of day can become a discriminative stimulus. For example, students may fidget or begin closing books before the end of a class hour. Certain words can trigger this behavior. A teacher must be careful about uttering the words "In conclusion..." near the end of a classroom hour, or books slam shut. Those words become an S+ for the negative reinforcer of finishing class. A person can be a discriminative stimulus. A person who is kind to a dog becomes an S+; the dog will approach the person. A person who is mean or cruel to a dog becomes an S-; the dog will avoid the person. Discrimination learning can be quite complex. In a classic study, Herrnstein , Loveland, and Cable (1964) reinforced pigeons with grain for discriminating pictures containing human beings. The pigeons were reinforced for key-pecks only when they saw a picture containing a human.
What was a classic study by Herrnstein and Loveland with pigeons?

Approximately half the photographs contained at least one human being; the remainder contained no human beingsin the experimenter's best judgment. In no other systematic way did the two sets of slides appear to differ. Many slides contained human beings partly obscured by intervening objects: trees, automobiles, window frames, and so on. The people were distributed throughout the pictures: in the center or to one side or the other, near the top or the bottom, close up or distant. Some slides contained a single person; others contained groups of various sizes.

The people themselves varied in appearance: they were clothed, semi-nude, or nude; adults or children; men or women; sitting, standing, or lying; black, white, or yellow. (p.287) Despite the variety of people and poses, all five pigeons learned to identify slides which contained humans. Their performance continued to improve over a period of months. When new slides were used, although the pigeons had never before seen them, the pigeons' responses were accurate. They identified the new slides containing humans. The pigeons showed generalization by treating different pictures of humans the same way. They showed discrimination by reacting differently to pictures with and without humans. In follow-up studies, the pigeons were taught to recognize things like fish and trees that they had never seen, being raised in a laboratory. This showed that the pigeon's discrimination abilities did not require prior experiences with the objects they learned to recognize.

BACKWARD CHAINING
Backward chaining is the secret of complex animal performances in movies and animal shows. Stimulus control is used in this technique. The name backward chaining comes from the fact that training starts with the last behavior in the to-be-learned sequence of behaviors. The trainer works backward from there. Suppose a rat in a Skinner box learns to press a lever to get food pellets. Once the rat learns the bar-press, the experimenter can easily establish an S+. For example, the rat might be given a pellet for a bar press only when the green light is on. This makes the green light an S+.
How is backward chaining used to teach animals complex behavior sequences?

Here is the key to backward chaining. An S+ can function as a reinforcer for other behaviors. For example, if a green light (signifying that a bar-press will produce a food pellet) is illuminated only when the rat climbs the ladder, the rat will learn to climb the ladder in order to see the green light. After seeing the green light, the rat climbs off the ladder, presses the bar, and gets a food pellet. In backward chaining this pattern is repeated many times to build up a chain of consecutive behaviors. If ladder climbing is reinforced by the green light only after a tone has been sounded, then soon the rat will wait for the tone before climbing the ladder. If the tone is sounded, the rat climbs the ladder and waits for the green light, then runs over presses the bar to get a food pellet when the green light turns on. If the tone is not sounded, nothing works; the rat can climb the ladder, but the green light does not turn on and the rat never gets a food pellet. So the tone comes to control the behavior of ladder climbing. The rat climbs the ladder only after the tone.
What dual function does the S+ serve, in backward chaining?

Now the tone functions as an S+, and it can be used as a reinforcer for another behavior. Perhaps you want to rat to crawl through a tunnel before it climbs the ladder. Put a little tunnel in the rat's cage, and as soon as the rat climbs through it, sound the tone that means "time to climb the ladder." The tone not only serves as an S+ for climbing the ladder; it also reinforces the rat for crawling through the tunnel. One by one, you keep adding new requirements for the rat, building a chain of behaviors. You can see why the technique is called backward chaining. The chain of behaviors stretches backward from the final behavior (consuming the food pellet). Each S+ serves as a reinforcer for the previous behavior and a discriminative stimulus for the following behavior. Very complex chains can be built up this way.

Following is a sequence of S+s and behavioral responses learned by one prodigy, the star of a short movie titled, Barnabus: the Educated Rat. S+ (light) Response (climbing spiral staircase to platform) S+ (drawbridge is visible) Response (pushing down and crossing drawbridge) S+ (ladder is visible) Response (climbing ladder) S+ (chain is accessible) Response (pulling car by chain) S+ (car near platform) Response (entering car and pedaling through tunnel) S+ (stairs are visible) Response (climbing stairs) S+ (tube is visible) Response (going through tube) S+ (elevator is accessible) Response (entering elevator) S+ (string attached to flag is accessible) Response (pull the string, raising the flag) S+ (elevator arrives at ground floor) Response (exit the elevator) S+ (buzzer sounds) Response (bar press) S+ (sound of food pellet) Response: (consuming food pellet)
Why is backward chaining necessary to use, for moviemakers who want to use animals as stars?

Backward chaining is the only way to teach animals arbitrary sequences of behaviors. The word "arbitrary" allows for naturalistic exceptions such as route processing and nest building. Animals ranging from squirrels to jumping spiders can travel in complex, multi-part journeys to reach a goal, and common bird species go through a complex series of steps to build a nest. However, such behaviors are essential to the survival of the species and the ability to perform them has been shaped up by evolution. When animals in movies perform sequences of behavior that do not "come naturally," it is due to backward chaining.
What was discovered when backward chaining was used with Army technicians?

With humans, backward chaining is not necessarily helpful. In one study (Cox & Boren, 1965) United States Army researchers used backward chaining to teach technicians a 72-action procedure involving missile preparation. In other words, they started with the last movement (action #72) and worked backward. There was no advantage over the reverse procedure: starting with action #1 and working forward. Humans are unique in their ability to guide behavior with language, which allows long sequences of arbitrary behaviors to be learned by practicing under the guidance of a written or spoken plan.

INTERMITTENT REINFORCEMENT AND RESISTANCE TO EXTINCTION

One of the useful principles discovered by behavioral psychologists is that intermittent reinforcement increases resistance to extinction. The word intermittent means not every time. Intermittent reinforcement contrasts with continuous reinforcement. Under conditions of continuous reinforcement, the organism is reinforced every time it makes the required response.
What effects do continuous and intermittent reinforcement have upon speed of extinction? Why?

For example, under continuous reinforcement, every time the rat hits the bar, it receives a food pellet. Under intermittent reinforcement, the rat might be required to hit the bar 50 times to get the pellet, or the rat might be reinforced only once every five minutes, or the rat might be reinforced only when you are in the room, or in accordance with some other pattern, butnot every time. Any pattern of reinforcement other than continuous reinforcement is a form of intermittent reinforcement. Extinction, as you recall, is a process of eliminating a behavior by stopping the delivery of reinforcers responsible for maintaining the behavir. Intermittent reinforcement makes extinction slower or harder to accomplish. The reason is that intermittent reinforcement makes an extinction period harder for animals to discriminate. During an extinction period, a behavior is never reinforced. If the response has been continually reinforced in the past, the animal will quickly notice this; it will discriminate the extinction period. It will stop responding soon. By contrast, if a response is intermittently reinforced, then the animal grows accustomed to periods of no reinforcement. If an experimenter tries to extinguish the behavior by cutting off all reinforcement, the animal is less likely to notice that extinction is taking place, or more likely to persist with the behavior in the expectation that reinforcement may resume again as it has in the past. The result is that animals with a history of intermittent reinforcement do not stop a behavior as quickly as animals with a history of continuous reinforcement. Instead, they show resistance to extinction.
How might this knowledge be important for parents?

The fact that intermittent reinforcement produces persistence or resistance to extinction is an important insight for parents. Parents try to discourage a child from throwing tantrums, but some parents tire under the onslaught of the child's rage and "cave in" by reinforcing the child. In effect, they are putting the child on an intermittent reinforcement schedule. This makes the tantrums harder to stop in the future. The child will show resistance to extinction, having learned in the past that persistence pays off. Experts on parenting advise parents never to reinforce a tantrum. For example, if a child misbehaves in a store, yelling and screaming in an attempt to get a piece of candy or other desired item, the parent should simply remove the child from the store. To resist the tantrum for a while, then break down and reinforce the child, makes the behavior more persistent in the future.

EXTINCTION-INDUCED RESURGENCE (THE "EXTINCTION BURST")

During an extinction period, animals usually show a great variety of behaviors. The animal appears to test variations of behavior to see if anything will make the reinforcers start again. Epstein (1985) called this "extinction-induced resurgence." Others call it an extinction burst.
What is extinction-induced resurgence? How is it useful to animal trainers? How was it shown by some dolphins?

Extinction-induced resurgence is a handy phenomenon for animal trainers. If they want animals to learn new tricks, instead of repeating old ones, they put the animals temporarily on an extinction schedule (stop delivering reinforcers). This results in an increase in the activity level and variability of behavior. Then the trainer can pick out a new behavior to be reinforced. In 1992 two trained porpoises escaped from an enclosure and swam out to sea. A few weeks later they turned up in a waterway next to a golf course, performing tricks for amazed golfers. The porpoises had been trained to perform tricks for fish, but by escaping they put themselves on an extinction schedule (except for catching their own fish). Their behavior at the golf course was lively and variable...and it worked. Before long some golfers bought a load of fresh fish, and the porpoises were feasting. Porpoises are social animals that often make eye contact before and after a trick. Attention is a powerful reinforcer for most social animals-including humans. Suppose you were on a golf course by the ocean, and you saw a porpoise close to the shore, doing what appeared to be a show trick (such as swimming on its tail, with its head high out of the water) while looking you straight in the eye. Wouldn't you feel the urge to go buy some fish? In short, it was not just the porpoises that were being reinforced! This brings to mind the old cartoon in which a rat is sitting in a cage telling another rat, "Boy, have I got that human trained...every time I hit this bar, the human gives me a food pellet." Humans think they are training the animals, but the animals are just as surely training the humans.

SPONTANEOUS BACKWARD CHAINING IN A BLUE JAY

Something similar to backward chaining can occur spontaneously with the aid of extinction-induced resurgence. When I was in graduate school, my wife and I fed peanuts to the large squirrels in Ann Arbor, Michigan. Soon a number of squirrels were showing up every day to receive handouts of fresh roasted peanuts. A blue jay took note and started stealing peanuts when we tossed them onto the ground for the squirrels. To prevent this, I had to hand the peanuts directly to the squirrels, which (in effect) put the blue jay on an extinction schedule.
How did a blue jay show impressive learning and memory?

The jay responded with extinction-induced resurgencea flurry of novel behaviors such as preening and fluffing up. Such a behavior is normally used for staying warm or in courtship displays. When we saw the blue jay fluffed up, we thought it was cute, so we tossed it a peanut. Otherwise we ignored it. Soon the bird was fluffing up every time it saw us.

Next we noticed that, as we walked through our house, the bird was flying from branch to branch outside our windows, fluffing up, trying to elicit peanuts. Apparently it could see us through the windows. It received several peanuts after such displays. This game continued until the bird left for the winter. To our surprise, the same bird showed up again the following year. This means it carried out a complex chain of behaviors. It found our house, sat on a branch outside the window, spotted a human, fluffed up, andjust like the year beforeit got a peanut tossed to it from the front porch.

Left to right: the blue jay appears on a branch, preens, fluffs up, and receives reinforcement

How did the blue jay's learning experiences resemble backward chaining?

The last behavior in the chain (flying down to the ground to get a peanut) actually came first in the bird's learning experiences. The other behaviors (learning to fluff for a peanut, then learning to go to a window to look for a human, then finding its way back to the house) were added, one-by-one, as requirements for getting a peanut. They formed a natural "backward chain."

AVOIDANCE AND ESCAPE LEARNING

So far we have spoken mainly of positive reinforcement. But animals also respond in predictable ways to avoid punishment. The term aversive control is used to cover situations in which behavior is motivated by the threat of an unpleasant stimulus. There are two main categories of behavior under aversive control: avoidance behavior and escape behavior.
What is aversive control? What are the two main types?

Escape conditioning occurs when the animal learns to perform an operant to terminate an ongoing, aversive stimulus. It is a "get me out of here" or "shut this off" reaction, aimed at escape from pain or annoyance. The behavior that produces escape is negatively reinforced (reinforced by the elimination of the unpleasant stimulus). For example, you could get a rat to jump off a platform into some water which a rat is normally reluctant to doby electrifying the top of the platform, giving the rat a mild shock. When the rat jumps off, it escapes the shock. If the platform is not electrified, and is the only place to rest from swimming, the rat will stay on the platform until it gets shocked. The jump is an escape behavior.
How can escape conditioning be converted into avoidance conditioning?

Escape conditioning is converted into avoidance conditioning by giving a signal before the aversive stimulus starts. If the animal receives acue or signal that an aversive stimulus is coming, then after one or two occurrences of the punishing stimulus the cue will trigger an avoidance behavior. This kind of learning occurs quickly and is very durable. For example, if you sounded a tone before you electrified the platform, after one or two trials the rat would respond to the tone by jumping into the water. It would not wait for the shock. This is a form of stimulus control, because it puts behavior under control of a stimulus, in this case the warning tone.
Why are avoidance behaviors so persistent?

Avoidance behaviors are incredibly persistent. This is true even when there is no longer anything to avoid. The reason is that an animal that performs an avoidance reaction never experiences the aversive stimulus. But it receives negative reinforcement in the form of relief. Because of this, avoidance behavior is self-reinforcing. It keeps going forever, because relief functions as a reinforcer even if the original threat is removed. In the preceding example, the rat will respond to the tone by jumping into the water hundreds of times, even if you turn off the shock generator and never use it again. Each time it jumps, it probably figures (in a rat way) that it has avoided the shock.
What was Solomon's shuttle-box experiment, and what point does it illustrate?

Psychologist Richard Solomon demonstrated persistence of avoidance behavior in his classic shuttle box experiment. In this research, Solomon placed a dog in a large cage with two chambers. A low wall separated the two chambers. When Solomon electrified the floor on one side, the dog jumped to the other side. This was escape conditioning because the dog emitted the behavior of jumping after it experienced the aversive stimulus (electric shock). If Solomon electrified the floor on the other side of the cage, the dog jumped back to the original side (which was no longer electrified). To convert the situation into one of avoidance learning, Solomon arranged for a signal to occur before each shock. He used an audible tone. After he sounded the tone, he turned on the shock, and the dog jumped to the other side. Soon the dog jumped as soon as it heard the tone. The escape behavior turned into avoidance behavior. Now Solomon turned off the shock generator. But he continued to sound the tone, and the dog continued to jump back and forth over the barrier from one side to the other, as many times as Solomon sounded the tone. The behavior never extinguished. No matter how many times this was repeated, the dog continued to jump. It never allowed itself to discover that the shock generator had been turned off.
In what sense is avoidance conditioning "self-reinforcing"?

This is an important pattern because something similar happens to humans. Avoidance behavior is self-reinforcing. Relief is the reinforcer, and relief occurs whether or not the threat is still present. Avoidance conditioning can go on forever, even if there is no longer a reason for it . A student who has trouble with math in high school may feel relief by avoiding math in college. Given a choice, the student might never take another math class, even if (in reality) the student would do well in a college math class. The student feels relief each time math is avoided. The avoidance behavior could last forever unless the student summons up the courage to take math and find out "it is really not so bad."

OBSERVATIONAL LEARNING
In an influential 1968 book, Behavior Modification: Theory and Practice, Albert Bandura rallied evidence to support the importance of learning through observation. As the name suggests, observational learning is learning stimulated by observing the behavior of another organism. Modeling is one form of observational learning. It occurs when one person performs a behavior, while others look on and learn from it. Specialized nerve cells called mirror neurons may account for this ability. Mirror neurons are found in many different primate species. They fire the same patterns whether a creature is performing an action itself or watching another member of the same species performing the action. This provides a way for actions seen externally to be taken into the nervous system, through a sort of automatic motor empathy.
What is observational learning? How might it be aided by mirror neurons?

Non-primate species show observational learning in special situations. For example, lion cubs learn to hunt by watching from a concealed location as the adult lions hunt. Parrots are also good at imitating. Modeling is used in the rival mate technique for training parrots. If a trainer wants an adult male parrot to perform tricks, the bird is allowed to watch a rival male show off in front of a female. The untrained parrot will then imitate many of the tricks with no special training.

SUMMARY: OPERANT CONDITIONING

Operant conditioning is distinguished from classical conditioning in several ways, including the country of origin (the United States instead of Russia), the type of behavior involved (learned instead of biologically inborn), and the prototypical experimental set-up (an operant chamber instead of Pavlov's dog). Operant conditioning is also called instrumental conditioning, because the animal uses its own behavior as an "instrument" to pursue some goal. An operant is defined as a behavior producing a certain effect on the environment. Thus a bar-press operant is any behavior which results in a bar press, whether it is accomplished (for example) with the animal's paw or the animal's nose. In a "rat lab" students start by teaching a rat to find food pellets in a small enclosure, the food magazine. Next the rat is reinforced (given food pellets) for any behavior that brings it close to the bar that sticks out of the cage wall. Then the rat is required to actually press the bar, to receive a pellet. This process of gradually reinforcing steps toward a desired behavior is called shaping. A reinforcing stimulus is a stimulus that makes the behavior it follows more frequent. Extinction occurs when the reinforcer that maintains a behavior is stopped, and the behavior goes away. A punishing stimulus is one that makes the behavior it follows less frequent or probable. It is not the same thing as negative reinforcement. Negative reinforcement is a form of reinforcement (therefore it makes behavior more frequent). Negative reinforcement occurs when the reinforcing event is removal of a stimulus. "Negative punishment" or response costoccurs when the removal of a stimulus after a behavior makes the behavior less frequent. Antecedents-stimuli that come before a behavior-are also used in operant conditioning. An S+ is a stimulus that indicates reinforcementis available if a behavior is performed. An S- is a stimulus that indicates reinforcement is not available. An S+ or an S- can be called a discriminative stimulus, because it helps the animal discriminate between situations when reinforcement is or is not available.

When animals learn to perform a behavior to escape from pain or other aversive stimulation, this is called escape conditioning. When animals receive a stimulus indicating something aversive is about to happen, they will try to escape ahead of time to avoid the unpleasant event. This is called avoidance learning. Avoidance conditioning is marked by its persistence. The relief produced by avoidance is reinforcing, so avoidance behavior tends to continue forever even when the original threat is no longer relevant. Operant conditioning can occur through observational learning. Modeling is the process of demonstrating a behavior that others learn by observation. This is a talent uniquely well developed in primate species, perhaps because of specialized nerve cells called mirror neurons that respond the same way whether an action is (1) performed by the animal itself, or (2) observed being performed by a different animal.

APPLIED BEHAVIOR ANALYSIS

In preceding sections of this chapter you have been introduced to the main tools of the applied behavior analyst, a behavioral psychologist who specializes in operant conditioning. There are two main tools: (1) systematic arrangement of consequences (reinforcement and punishment) and (2) careful analysis and arrangement of antecedents (S+ and S-). Together, these skills can be called contingency management. A contingency is a dependency between events, such as the delivery of food when an animal performs a behavior. Contingency management is used whenever animals are motivated by incentives (such as getting paid for a job) or penalties (such as paying a fine for doing something wrong). Applied behavior analysis is the application of conditioning principles to any tasks or problems outside the laboratory. We already discussed applications of classical conditioning in an earlier section of this chapter. In this section we will concentrate on applications of operant conditioning.
How did a professor start a class on applied behavior analysis, and what was the point?

One professor started a graduate class on applied behavior analysis by generating a huge list of problems. The professor told the students, "Think of any problem a person can have, and we will design a behavioral therapy for it. Let's get a big list on the board." At first the class responded slowly. Someone suggested "marriage problems" and the professor said, "Put it in terms of behaviors." "OK," said the student, "let's say the problem is not enough time spent together." The professor wrote that on the board. Another student suggested the problem of eliminating "writer's block," defined behaviorally as the problem of increasing the frequency of certain types of writing. Another student suggested the problem of eliminating unwanted involuntary movements (tics). Other students mentioned problems like nailbiting and quitting cigarettes.
How are problems defined in behavioral terms?

As the list grew, the students realized this process could take quite a while. The list of possible human problems is neverending. Most can be defined in behavioral terms. In other words, most problems can be described in terms of some observable, measurable activity (behavior) that must be made either more frequent or less frequent. That is the essence of the behavioral approach: problems are defined as behaviors that can be made more frequent or less frequent, and the therapy consists of applying conditioning techniques to make the desired changes. After the list filled the board, the professor gave the class its assignment. Each student had to select a problem and, by the end of the term, design a behavioral treatment for it. The professor

was making a point, not just an assignment. Behavioral approaches can be applied to virtually any conceivable problem.

BEHAVIORAL ANALYSIS
The first step in applied behavior analysis is to analyze the problem. The analysis must be behavioral; that is, one must state the problem in terms of behaviors, controlling stimuli, reinforcers, punishers, or observational learning...the concepts we have covered in this chapter. Antecedent and consequent stimuli must be identified. After this analysis, one can make an educated guess about which intervention strategy or "treatment" might be best.

LINDSLEY'S SIMPLIFIED PRECISION MODEL

Green and Morrow (1974) offered a convenient, four-step guide to successful behavior change. Developed by Ogden Lindsley, it is called Lindsley's Simplified Precision Model.
What is Lindsley's Simplified Precision Model?

1. Pinpoint the target behavior to be modified. 2. Record the rate of that behavior. 3. Change consequences of the behavior. 4. If the first try does not succeed, try and try again with revised procedures. The first step in Lindsley's list was to pinpoint the behavior to be modified. This is often the most crucial step. If you cannot specify a behavior, how can you modify it?
What "heroic" efforts are exemplified by the list of speech problems?

Behavior modifiers and therapists sometimes go to heroic lengths to identify specific, modifiable behaviors. For example, a team of behavior therapists at a speech clinic came up with 49 different, specific verbal problems. The checklist included the following: 1. Overtalk. A person speaks considerably more than is appropriate. 2. Undertalk. A person doesn't speak enough. 3. Fast talk. A person talks too fast. 4. Slow talk. A person speaks too slowly. 5. Loud talk. A person speaks too loudly. 6. Quiet talk. A person speaks too softly. 7. Singsong speech. A person talks as if singing or chanting. 8. Monotone speech. A person speaks with unvarying tone. 9. Rapid latency. A person speaks too quickly after someone else. 10. Slow latency. A person responds only very slowly. 11. Affective talk. A person talks with great emotion, crying, whining, screaming, or speaking with a shaky voice. 12. Unaffective talk. A person speaks in a flat, unemotional voice even when emotion is appropriate. 13. Obtrusions. A person too often "butts in" to conversation. ...and the list goes on, up to #49, which is "illogical talk." (Adapted from Thomas, Walter & O'Flaherty, 1974, pp. 248-251.)

What happened when a client first entered the speech clinic? What happened once the problem was specified?

When a client first entered the speech clinic, the therapists checked off which behaviors defined the client's problem. Each category specifies a type of behavior: something that can be recognized, singled out for reinforcement, extinction, or punishment. If a person clearly alternates between logical and illogical talk, it is possible to reinforce one and extinguish the other. Once the problem is specified in terms of something that can be measured or detected, a behavior, then a therapist can attempt to change the antecedents or consequences of the behavior, to alter the frequency of the behavior.

BASELINING
The next step, after specifying a behavior to be changed, is to measure its frequency of occurrence before attempting to change it. Baselining is keeping a careful record of how often a behavior occurs, without trying to alter it. The purpose of baselining is to establish a point of reference, so one knows later if the attempt to change behavior had any effect. A genuine behavior change (as opposed to a random variation in the frequency of a behavior) should stand out sharply from the baseline rate of the behavior.
What is baselining? What is its purpose? How long should baselining continue, as a rule?

As a general rule, baselining should continue until there is a definite pattern. If the frequency of the behavior varies a lot, baseline observations should continue for a long time. If the behavior is produced at a steady rate, the baseline observation period can be short. While taking baseline measurements of an operant rate-the frequency of some behavior-an applied behavior analyst should pay careful attention to antecedents -stimuli that comebefore the behavior. As we saw earlier, discriminative stimuli (both S+s and S-s) act as if they control behavior, turning it on or off.
In what important respect was Lindsley's model incomplete?

During the baseline period, one should keep a record of potentially important antecedent stimuli, as well as the record of the frequency of the behavior itself. One weakness of Lindsley's Simplified Precision Model (previous page) was that it did not mention antecedents. It only mentioned changing consequences of a behavior. Often the relevance of antecedents will be obvious, once they are noticed. For example, if a child's misbehavior occurs every day when the child is dropped off at a nursery school, a good behavior analyst will target this period of the day and try to arrange for a reinforcing event to occur if the child remains calm following the departure of the parent at such a time.

SELF-MONITORING AS A METHOD OF BEHAVIOR CHANGE

Sometimes behavior will change during a baseline observation period, due to the measurement itself. Although old-time behaviorists never would have used this language, the likely explanation is that a person becomes more conscious of the behavior and starts to alter it when it is measured. Measurement of one's own behavior is called self-monitoring , and it can be an effective tool for behavior change. For example, many people wish to lose weight, but few people keep detailed records of their calorie intake. People who keep a record of every calorie consumed often find that they lose weight as a result. The act of recording data focuses attention on each bite of food and its consequences, and this (or the fear of having bad eating habits exposed) motivates a person to eat less.
What is self-monitoring? What sorts of problems respond well to self-monitoring?

Self-monitoring often works especially well with impulsive habits, like snacking, cigarette smoking, or TV-watching. These are all things a person may start to do "without thinking." Of course, to really work, self-monitoring must be done honestly without cheating. It forces a person to think about every occurrence of a behavior. It also draws attention to the consequences of behavior. "If I eat this, I am over my limit," or "If I start watching TV I won't get my homework done." Selfmonitoring, as a behavior change procedure, lacks any specially arranged reinforcement or punishment, but it forces attention to natural reinforcements and punishments.

USING REINFORCEMENT
In some cases, mere measurement and informative feedback is not enough. A direct form of intervention is required. Therefore the third part of Lindsley's method consists of arranging a contingency involving reinforcement or punishment. Green and Morrow (1974) offer the following example of Lindsley's Simplified Precision Model in action. Jay, a twenty-year-old spastic, retarded man urinated in his clothes. A physician had ruled out any organic basis for this behavior. Recently Jay was transferred from a state hospital to a nursing home.... The nursing home director said that Jay must leave if he continued to be enuretic. He agreed, with reservation, to let [a student] try a program to eliminate the wetting behavior. Conveniently, the nurses had been routinely recording the number of times Jay wet his clothes each day....Jay's daily rate of wets was...plotted on a standardized graph form... Questionable punishment procedures, insisted upon by the nursing home director, were used in the first two modification phases. First, Jay was left wet for thirty minutes following each wet. Second, Jay was left in his room for the remainder of the day after he wet once. Throughout both punishment phases the median rate remained unchanged. What to do? Lindsley's step #4 is "if at first you don't succeed, try and try again with revised procedures." That is a rather strange sounding rule, but it proves important. Behavior analysts are not automatically correct in their first assessment of a situation. The first attempt at behavior change may not work. Like other scientists, they must guess and test and try again. In the case of Jay, behavior analysts decided to stop the "questionable punishment procedures" and try positive reinforcement instead.
How does the story of "Jay" illustrate step #4 of Lindsley's procedure?

In a fourth phase, following consultation by the student with the junior author, and with the nursing home director's reluctant consent, Jay was given verbal praise and a piece of candy each time he urinated in the toilet. No punishment was used. Candy and praise were chosen as consequences after discussion with the nursing home personnel disclosed what Jay seemed to "go for." The procedure essentially eliminated wetting. In an "after" phase (after reinforcements were discontinued), the rate remained at zero except for one lapse. Presumably, approved toilet behavior and nonwetting were now maintained by natural consequences, such as social approval and the comfort of staying dry. (Green & Morrow, 1974, p.47)) Note the "after" phase. Proper intervention procedures also include a follow-up to make sure the change is permanent.

USING THE PREMACK PRINCIPLE

The search for effective reinforcers sometimes requires creative thinking. Here is a case history in which the Premack principle was employed with good results. The Premack principle, discussed earlier, is the idea that preferred or high frequency behaviors can be used to reinforce less preferred, low frequency behaviors.
How was the Premack principle used to help Burton, the strange 13-year-old?

Case #S-13. Burton had been forced out of school because of his bizarre mannerisms, gestures, and posturing. It was generally assumed that he was a severely schizophrenic child, albeit a highly intelligent 13-year-old. He acted belligerently toward his parents and was destructive of home property. He had been known to punish his parents by such behaviors as pouring buckets of water over his head in the middle of the living room, but his high probability behaviors were to publicly assume a semifetal position, and, alternately, to lock himself alone in his room for long hours. Reading the homework assigned by his visiting teacher was low probability behavior. Neither he nor his parents rated any objects or people as reinforcing. Initially, therefore, the reinforcement of "retiring to his room" was used, contingent upon completing his homework assignment. Later, he was returned to a special education classroom. Low probability behavior was classwork, and high probability was escaping alone to a corner of the schoolyard. A contingency was established in which Burton was allowed to leave the class after completion of his assignment. Later, school attendance became a high probability behavior. At that point, he was allowed to attend school only contingent upon more acceptable behavior at home. (Tharp and Wetzel, 1969, p.47)

NATURAL SOCIAL REINFORCERS

Probably the most commonly used reinforcer in human and animal affairs is natural social reinforcement. Natural social reinforcement includes all sorts of positive social contact, including (among humans) smiles, hugs, compliments, or simple attention and company.
What are natural social reinforcers?

Common social reinforcers among non-human animals are attention, touching, grooming, and cleaning. For example, small fish sometimes linger in the area of larger fish and clean them by eating parasites and debris from the larger fish. Walcott and Green (1974) showed that this cleaning symbiosis was reinforcing to the larger fish. In other words, the larger fish would perform a behavior more often when it was followed by contact with the smaller, cleaning fish. Among humans, social reinforcers can be ruined if they are perceived to be fake or manipulative. A primary rule of social reinforcement is "be sincere." If flattery is honest and true, it is a powerful reinforcer. Perhaps the word flattery is a bad choice. To some people, it implies deceit, as if flattering someone means buttering them up, not really meaning what you say. The Dale Carnegie course, which teaches "how to win friends and influence people," says flattery is not recommended as a technique for winning friends, but appreciation is very effective! Natural social reinforcement can be useful in professionally planned behavior modification programs. The following example is from Tharp and Wetzel's book Behavior Modification in the Natural Environment (1969).
How was natural social reinforcement used with Rena?

Case #50. Rena was referred by her parents who were very concerned about her inappropriate behavior at school. Rena, an elementary school student, was known throughout the school for her aggressiveness toward her peers, disruptive classroom behavior, and general defiance. After interviewing her parents, we discovered that Rena was exhibiting, on a somewhat lesser scale, the same behavior at home... An intervention plan was set up whereby Rena's teacher could inform the parents each day her behavior was satisfactory. Since reinforcers at home were so limited, we had to rely on the positive attention her father could give her when he got home. They would play simple card games or play in

the yard skipping rope, etc. Rena's father had occasionally done this with her, and by making it contingent, this interaction became very meaningful to her. When Rena's behavior was not satisfactory at school, this reinforcement did not occur. The plan took effect rather rapidly, and before long Rena was no problem at school. And, as hoped, her behavior at home also improved.

SECONDARY REINFORCEMENT USING TOKENS

Secondary reinforcers, you may recall, are learned or symbolic reinforcers. Money is a secondary reinforcer, because you cannot eat or drink money or get any other primary reinforcement directly from it. However, you can trade money for primary reinforcers such as food and drink. Secondary reinforcers get their reinforcing properties from their association with primary reinforcers. Grades are an example. They are worthless in themselves, but they can lead to primary reinforcers like pride, attention, and the fruits of employment after graduation.

How are secondary reinforcers used in token economies?

One well-known application of secondary reinforcement is in token economies. Token economies are like miniature economic systems, using plastic poker chips or similar tokens instead of money. Originally they were used in mental hospitals, but more recently they have been found useful in institutions serving learning-disabled individuals. One student writes: Everyone has a need and a want for food, sleep, and love. These are just a few examples of what is known as primary reinforcements. Sometimes reinforcements are earned to buy or attain primary reinforcements. This kind of reinforcement is known as secondary reinforcement. My sister is mentally retarded, and I can tell when she has been positively reinforced for something she has done, especially at school. She attends Gordon County Training Center. The teachers there have set up a system based on tokens. If the students at GCTC do their work well or get along with the other students, they receive a certain amount of tokens. At the end of each week, the students can go to the "store" in the school and spend their tokens on something that they want. My sister always comes home telling us how many tokens she earned and what she spent them on. She really enjoys getting tokens or any other kind of secondary reinforcement, such as trophies or ribbons, for her achievements. The secondary reinforcements show her that she is doing something good and acceptable in the eyes of others. [Author's files]
Why are tokens useful in institutional settings?

Tokens are useful in group settings like a training center for several reasons. (1) the same reinforcer can be given to everybody, and (2) reinforcement can be given immediately after a behavior. In a treatment facility, candy might be an effective reinforcer with some people...but not everybody, and different people like different types of candy. With tokens, you do not need to worry about having the right type of reinforcer on hand; you can give reinforcements immediately (in tokens) and the patient can "spend" the tokens later at a store in the hospital.

MORE ABOUT SHAPING

Before you can reinforce a behavior, the behavior must occur. What if the behavior is not occurring? Then you must use a technique called shaping , mentioned earlier in connection with teaching a rat to press a bar in a Skinner Box.
What is the technical name for "shaping"?

Shaping is well described by its technical name: the method of successive approximations. To approximate something is to get close to it. To do successive approximations is to get closer by small steps. Shaping works by starting with whatever the organism can already do and reinforces closer and closer approximations to a goal. Here are five simple rules for shaping.
What are five rules to observe, while using shaping?

1. Make sure the target behavior is realistic and biologically possible. 2. Specify the current (entering) behavior and desired (target) behavior. 3. Plan a small chain of behavioral changes leading from the entering behavior to the target behavior. 4. If a step proves too large, break it into smaller, more manageable steps. 5. Use reinforcers in small quantities, to avoid satiation (getting "full").
How are the five rules illustrated by teaching a dog to catch a Frisbee?

To illustrate the five rules, consider the task of teaching a dog to catch a Frisbee. If you have ever seen dogs catch a Frisbee, you know it is quite impressive. Suppose you want to teach your dog this trick. How do you do it? According to rule #1, you have to decide whether your dog is physically capable of such an act. The national champion Frisbee-catching dogs are usually dogs like whippets with a lean, muscular build which permit them to leap high into the air to snatch Frisbees out of the breezes. Other breedsbulldogs, Pekinese, and dachshundsmight be less able to learn this skill. Suppose you have a dog that is physically capable of catching a Frisbee. Rule #2 says "specify the current (entering) behavior." This must be a behavior the dog can already perform. It should be a behavior that can be transformed, in small steps, into the target behavior. Frisbee catching requires that the dog take a Frisbee into its mouth, so you might start by reinforcing the dog for the entering behavior of holding the Frisbee in its mouth. Most dogs are capable of doing this without any training. In fact, they will gladly puncture a Frisbee with their canine teeth, so use a cheap Frisbee you do not mind ruining. The dog enters the experimental situation with this behavior already in its repertoire. That is why it is called an entering behavior. Rule #3 says to devise a series of small steps leading from the entering behavior (holding the Frisbee in his mouth) to the target behavior (snatching the Frisbee from the air). Finding such a sequence of steps is the trickiest part of shaping. How can you get from "here" to "there"? One approach is to toss the Frisbee about a foot in the air toward the dog, hoping it will perform the skill so you can reinforce it. Unfortunately, this probably will not work. The dog does not know what to do when it sees the Frisbee coming, even if the dog has chewed on it in the past. It hits the dog on the nose and falls to the ground. This brings us to rule #4. If a step is too large (such as going directly from chewing the Frisbee to snatching it out of the air) you must break it into smaller steps. In the Frisbee-catching project, a good way to start is to hold the Frisbee in the air. The dog will probably rise up on his hind legs to bite it. You let the dog grab it in his mouth, then you release it. That is a first, simple step. Next, you release the Frisbee a split second before the dog grabs it. If you are holding the Frisbee above the dog, you might drop it about an inch through the air, right into the dog's mouth. Now the most critical part of the shaping procedure takes place. You gradually allow the Frisbee to fall a greater and greater distance before the dog bites it. You might start one inch above the dogs mouth, work up to two inches, then three, and so on, until finally the dog can grab the

Frisbee when it falls a whole foot from your hand to the dog's mouth. (For literate dogs outside the U.S. and Britain, use centimeters and meters.) Keep rule #4 in mind; if the dog cannot grab the Frisbee when it falls 8 inches, you go back to 6 inches for a while, then work back to 8, then 10, then a foot. Eventually, if the dog gets into the spirit of the game, you should be able to work up to longer distances. Once the dog is lunging for Frisbees that you flip toward it from a distance of a few feet, you are in business. From there to a full-fledged Frisbee retrieval is only a matter of degree. Rule #5 says to have reinforcers available in small quantities to avoid satiation. Satiation (pronounced SAY-see-AY-shun) is "getting full" of a reinforcergetting so much of it that the animal (or person) no longer wants it. If satiation occurs, you lose your reinforcer, and your behavior modification project grinds to a halt. Suppose you are using Dog Yummies to reinforce your Frisbee-catching dog. If you use 50 yummies getting it up to the point where it is catching a Frisbee that falls eight inches, you will probably not get much further that day. The dog is likely to decide it has enough Dog Yummies and crawl off to digest the food.
Why is satiation unlikely to be a problem in this situation?

Actually, dogs respond well to social reinforcement (praise and pats), and that never gets old to a loving dog, so dog trainers usually reserve their most powerful reinforcers for occasional use. Retrieval games are themselves reinforcing to many dogs. When I took a dog obedence course, the trainer used retrieval of a tennis ball to reinforce her dog at the end of a training session. That was a fine example of the Premack Principle in action because a preferred behavior (retrieving a tennis ball) was used to reinforce non-preferred behaviors (demonstrating obedience techniques).

PROMPTING AND FADING

Prompting is the act of helping a behavior to occur. This is a useful way to start teaching a behavior. A coach who helps a small child hold a baseball bat, to teach a proper swing, is using prompting. Fading is said to occur when the trainer gradually withdraws the prompt. For example, the baseball coach gradually allows the child to feel more and more of the bat's weight, until the coach is no longer holding it. Eventually the child swings the bat alone. The prompt has been "faded away."
What is prompting and fading?

Prompting and fading is commonly used in dog obedience training. For example, to teach a dog to sit, one gives the command (sit) then forces the dog to comply with it by gently sweeping the arm into the dog's back knees from behind, so the dog's back legs buckle gently and its rump goes down to the ground. Meanwhile one holds the dog's collar so the head stays up. This forces the dog to sit. When the dog sits, the trainer praises it or offers it a morsel of food.
How is prompting and fading used in dog obedience training?

The command is a stimulus that eventually functions as an S+. The upward tug on the collar and the arm behind the back knees are called a prompt because they help or prompt the behavior to occur. The procedure of gradually removing the prompt is called fading. The prompt becomes weaker and weaker; it is "faded out." After about 20 repetitions there is no need to touch the back of the dog's legs; one says "sit" and the dog sits.
How did a city use prompting and fading?

Prompting and fading was used by one city when it switched from signs with the English words "No Parking" to signs with only an international symbol (a circle with a parked car in it and a diagonal line crossing it out). For the first three months, the new signs contained both the international symbol and the English words. Then the words were removed. People hardly

noticed the transition to the new signs, because their behavior was transferred smoothly from one controlling stimulus to another.

DIFFERENTIAL REINFORCEMENT
Differential reinforcement is selective reinforcement of one behavior from among others. Unlike shaping, differential reinforcement is used when a behavior already occurs and has good form (does not need shaping) but tends to get lost among other behaviors. The solution is to single out the desired behavior and reinforce it.
What is differential reinforcement? How is it distinguished from shaping? What is a "response class"?

Differential reinforcement is commonly applied to a group of behaviors. For example, if one was working in a day care center for children, one might reinforce any sort of cooperative play, while discouraging any fighting. The "cooperative play" behaviors would form a group singled out for reinforcement. Such a group is labeled a response class. A response class is a set of behaviorsa category of operantssingled out for reinforcement while other behaviors are ignored or (if necessary) punished. The only limitation on the response class is that the organism being reinforced must be able to discriminate it. In the case of preschoolers at a day care center, the concept of cooperative play could be explained to them in simple terms. Children observed to engage in cooperative play would then be reinforced in some way that worked, for example, given praise or a star on a chart or more time to do what they wanted.
How did Pryor reinforce creative behavior?

Karen Pryor is a porpoise trainer who became famous when she discovered that porpoises could discriminate the response class of novel behavior. Pryor reinforced two porpoises at the Sea Life Park in Hawaii any time the animals did something new. The response class, in this case, was any behavior the animal had never before performed . Pryor set up a contingency whereby the porpoise got fish only for performing novel (new) behaviors. At first this amounted to an extinction period. The animals were getting no fish.
How did the porpoises' natural reaction to extinction help Pryor?

As usual when an extinction period begins, the porpoises showed extinction-induced resurgence. In other words, the variety of behavior increased, and the porpoises showed a higher level of activity than normal. They tried their old tricks but got no fish. Then they tried variations of old tricks. These were reinforced if the porpoise had never done them before. The porpoises caught on to the fact that they were being encouraged to do new and different things. One porpoise "jumped from the water, skidded across 6 ft of wet pavement, and tapped the trainer on the ankle with its rostrum or snout, a truly bizarre act for an entirely aquatic animal" (Pryor, Haag, & O'Reilly, 1969). The animals also emitted four new behaviorsthe corkscrew, back flip, tailwave, and inverted leapnever before performed spontaneously by porpoises.

DRO AND DRL

A special form of differential reinforcement is differential reinforcement of other behavior, abbreviated DRO. Other behavior basically means any behaviors except the one you want to get rid of. In the behavioral laboratory, DRO is technically defined as "delivery of a reinforcer if a specified time elapses without a designated response." In other words, the animal can do whatever it wants, and as long as it does not do a particular behavior for a certain period of time, it receives a reinforcer.
What is DRO? What are situations in which DRO might be useful?

DRO is used to eliminate a behavior without punishment. Suppose you have a roommate who complains constantly about poor health. You could say, "Stop talking about your health" but that would be rude. So how can you encourage your roommate to stop talking about health? One approach is to use DRO. If the roommate spends a minute without talking about health, you pay attention and act friendly. If the conversation turns to aches and pains, you stop talking. Eventually, if the procedure works, your roommate will stop talking about health problems. As this example shows, DRO involves extinction of the problem behavior. You cut off reinforcements to the behavior you want to get rid of (extinction) and you reinforce any otherbehavior (DRO).
How can DRO supplement or replace punishment?

Whenever punishment is used, DRO should be used as well. For example, if you feel you must discipline a child, you should not merely punish the wrong responses, you should positively reinforce correct responses. Most of the time, you can achieve what you want through positive reinforcement alone without any punishment. Another variation of differential reinforcement is DRL or differential reinforcement of a low rate of behavior. DRL occurs when you reinforce slow or infrequent responses. Psychologists were initially surprised that such a thing as DRL could exist. After all, reinforcement is defined as increasing the rate of behavior. However, many animals can learn a contingency in which responding slowly produces reinforcement.
What is DRL?

A student reports using DRL to deal with a roommate problem: My experience with my roommate is an example of DRL. My roommate is a wonderful person, but she talks too much. A simple yes or no question receives a "15 minute lecture" answer. She talks constantly. After the psychology lecture on differential reinforcement for a low rate of behavior, I decided to try this method. When I asked her a simple question and received a lengthy answer, I simply ignored her or left the room. When she gave a simple reply, I tried to seem interested and even discussed her answer. Now my roommate talks less and I don't get as aggravated with her. [Author's files]

USING NEGATIVE REINFORCEMENT

So far we have considered variations on the theme of positive reinforcement. Negative reinforcement also has many uses. One is to increase productivity in industry. Most people will work extra hard if they can get some time off from work as a reward. This is negative reinforcement because the level of work is increased (the behavior becomes more frequent) as the result of having an aversive stimulus removed (time off from work). Sometimes this works too well! Here is a story told by a guest lecturer in a Behavior Modification class.
How does the story about the automatic transmission assembly line illustrate the potential power of negative reinforcement?

A major automobile manufacturer asked some behavioral psychologists to help solve a morale and production problem at an automobile assembly plant in Ypsilanti, Michigan. Workers on one of the lines were going too slow, holding up the entire plant. They were supposed to manufacture 60 transmissions per hour, but they seldom achieved this objective. When managers urged them to speed up, they complained about being exploited, because (as they saw it) they were being asked to produce more work for the same salary. They felt abused. The supervisors were disgusted. Bad feelings existed all around.

The psychologists suggested a contingency. If the workers produced their quota of 60 transmissions before the end of the hour, they could take a break until the end of the hour. The psychologists conceived of this as a negative reinforcement contingency. They arrived at the idea when they realized there was no positive reinforcer available, because the management would not allow any extra salary incentives. OK, said the psychologists, then we will use a negative reinforcer. If the workers hate being "pushed" all the time, we will let them take time off (and avoid being pushed) when they meet their quota. The program worked like magic. Productivity leaped. 60 transmissions were produced in 50 minutes, leaving 10 minutes for a break every hour. The supervisors felt strange seeing the workers "goof off" every hour, but at least the line was finally meeting its quota. Then a funny thing happened. The line began to manufacture 60 transmissions in 45 minutes, then 60 transmissions in 40 minutes. Soon workers on the other lines were grumbling, "Hey, how come those guys are getting a 20 minute break every hour?" The plant managers had not expected this to happen, and they had no plans for dealing with it. They boosted the production quota to 80 transmissions per hour. The workers grumbled, but given the alternative of going back to the old plan with no hourly break, they accepted the new quota. Nobody expected the "problem" that occurred next. At first, producing 80 transmissions took almost the whole hour. But soon 80 transmissions took only 50 minutes...and then only 45 minutes, and the workers were back to taking a 15 minute break every hour. Workers on the other assembly lines started asking for a similar system. At this point, the plant managers decided the experiment had to end. It was unacceptable to have people taking a break every hour. The managers sent the behavior modifiers home and went back to the old system. "We knew it wouldn't work," the managers said. [Author's files] The assembly line story is an example of negative reinforcement. The reinforcer (time off from work) involved removal of an aversive stimulus so it was "negative" reinforcement. Like positive reinforcement, it produced a higher frequency of behavior...until the managers refused to let it continue. Eventually a simplified form of this incentive did find a home in the auto industry, and it even became part of union contracts. In an echo of the above story (which I heard as an undergraduate in the early 1970s) news articles in 1997 reported that workers in a stamping plant at Flint, Michigan were going home after only 4 hours instead of the usual 8. Apparently the union had negotiated a production quota based on the assumption that certain metal-stamping machines were capable of stamping out 5 parts per hour, but actually the machines were capable of 10 per hour. The workers speeded them up to 10 parts per hour and met the quota specified in their union contract within 4 hours. GM decided to eliminate the "go home early" policy, and this was one issue in a 1998 strike against General Motors. In the end, a modified version of the policy with a higher rate of production was re-affirmed in a new contract.
What is reinforced by an hourly wage? By piecework? What is the disadvantage of piecework?

If you think about it, an hourly wage reinforces slow behavior. The less energy a worker puts into the job, the more money the worker receives per unit of energy expended. By contrast, when people are paid according to how much they produce (so-called "piecework" systems) they work very quickly to maximize their gain. The obvious disadvantage is that workers who are rushing to complete their quota might produce a poor quality product, or endanger themselves, unless steps are taken to maintain quality control and safety on the job.

Babies as master behavior modifiers

Babies are masters of behavior modification who use negative reinforcement to increase the frequency of parenting behaviors in adults. Non-human babies of all different types practice this form of operant conditioning instictively. For example, juvenile birds fresh out of the nest will fly from branch to branch, following their parents, making unpleasant noises until fed.
How are babies "masters of behavior modification"?

Almost nothing is more aversive to parents than the cry of a baby, and adult humans will do almost anything to eliminate or prevent that stimulus. They will feed a baby, change its diapers, dance around with it, get up in the middle of a deep sleepall to prevent crying. Fortunately for parents, crying (in babies under one year old) is not reinforced by the application of love.
What finding surprised behavior modifiers?

This finding actally surprised behavior modifiers when it was first discovered, because it was so counter-intuitive. Studies showed that babies whose parents responded less to their crying (leaving the baby crying longer or more often) actually suffered more crying in the future. Perhaps this is because crying cannot be completely extinguished. Parents must respond eventually, so parents who respond slowly are essentially using intermittent reinforcement and are teaching their babies persistence, not quietude.

USING PUNISHMENT
Punishment is the application of a stimulus after a behavior, with the consequence that the behavior becomes less frequent or less likely. Most people assume the stimulus has to be unpleasant (aversive), but that is not always the case. Any stimulus that has the effect of lowering the frequency of a behavior it follows is a punisher, by definition, even if it does not seem like one. Electric shock is often the most effective punishing stimulus. Perhaps because electricity is an unnatural stimulus, or because it disrupts the activity of nerve cells, organisms never become accustomed to it, and they will do almost anything to avoid it. Whatever the reason, electric shock "penetrates" when other punishers fail to work.
What is punishment?

Treatment of head-banging Whaley and Mallott (1971) tell of a nine-year-old, mentally retarded boy who caused himself serious injury by head-banging. The boy had to be kept in a straitjacket or padded room to keep him from hurting himself. This prevented normal development; he acted more like a three-yearold than a nine-year-old. Left unrestrained in a padded room, the boy banged his head up to a thousand times in an hour. Something had to be done.
How did punishment help the child who banged his head?

The researchers decided to try a punishment procedure. They placed shock leads (electrodes) on the boy's leg, strapping them on so he could not remove them. Each time he banged his head, they delivered a mild shock to his leg. The first time he banged his head and was given a shock, Dickie stopped abruptly and looked about the room in a puzzled manner. He did not bang his head for a full three minutes, and then made three contacts with the floor in quick succession, receiving a mild shock after each one. He again stopped his head-banging activity for three minutes. At the end of that time he made one

more contact, received a shock, and did not bang his head for the remainder of the one-hour session. On subsequent sessions, after a shock was given the first time Dickie banged his head, he abandoned this behavior. Soon the head banging had stopped completely and the mat was removed from the room. Later, a successful attempt was made to prevent Dickie from banging his head in other areas of the ward. Currently Dickie no longer needs to be restrained or confined and has not been observed to bang his head since the treatment was terminated; therefore, in his case, punishment was a very effective technique for eliminating undesirable behavior. The psychologist working with Dickie stressed that the shock used was mild and, compared to the harm and possible danger involved in Dickie's head banging, was certainly justified (Whaley & Mallott, 1971). Twenty years after this technique was developed, it was still being debated. It worked and spared the child further self-injury, plus it stopped a destructive habit that might have persisted for years if left unchecked. However, people who regarded any use of electric shock with humans as unacceptable attacked this technique as cruel.

TREATMENT OF UNCONTROLLABLE SNEEZING

If you try to identify the common element in problems that respond well to aversive treatment, they often involve a "stuck circuit"a biologically based behavior pattern that, for some reason, is triggered again and again with self-injurious consequences. In these cases, one could argue, punishment therapy is justifiable, even merciful, because it is so quick and effective. Consider another case reported by Whaley and Mallott. It involves uncontrollable sneezing in a 17-year-old girl. She sneezed every few minutes for six months. Her parents spent thousands of dollars consulting with medical specialists, but nobody could help. The problem was solved (again) with mild electric shocks.
How does the case history of the sneezing girl illustrate therapeutic use of electric shock?

The shock began as soon as the sneeze was emitted and lasted for half a second after its cessation. Within a few hours the sneezing became less frequent, and six hours later it had stopped completely. For the first time in six months...the girl spent a full night and day without a single sneeze. Two days later she was discharged from the hospital and, with occasional subsequent applications of the shock apparatus, her sneezing remained under control. ...The total time that the teenager actually received shocks during the entire treatment was less than three minutes. (Whaley & Mallott, 1971)

PUNISHMENT FROM THE ENVIRONMENT

Punishment often has negative side effects, Animals lose their trust of humans who punish them, making medical treatment and other interactions difficult. Punishment can cause avoidance and emotional disturbance. When humans punish animals, the animals often fail to learn because they do not know which specific behavior is being punished. To be effective, a punishment must occur immediately after a behavior, and it need not be injurious. A mother wolf (or lion or tiger) shows effective punishment procedures with its babies. Misbehavior is followed by a quick and largely symbolic act of dominance, such as a swat or pretend bite. The punishment does not cause injury, but it conveys disapproval, and it comes immediately after the problem behavior.
What are some negative side effects of punishment? What typically happens when a human tries to punish a cat?

Dunbar (1988) noted that if a cat owner sees a cat performing a forbidden act such as scratching the furniture, punishment is not usually effective. If the human punishes the cat, the cat merely learns to avoid the human (so the human becomes an S-). Typically the cat will continue to perform the same forbidden act when the human is not present. If the human discovers evidence of a cat's forbidden behavior upon coming home, and punishes the cat, the cat learns to hide when the human comes home. This does not mean the cat feels "guilt." It means the cat has learned that the human does unpleasant things when first arriving home. The cat does not associate punishment with the forbidden behavior, which typically occurred much earlier.
What is "punishment from the environment" and how can it be used to keep cats off the kitchen counter?

If punishment from a human does not work very well, a good alternative is punishment from the environment. It works with all animals, even cats. Dunbar points out, "A cat will only poke its nose into a candle flame once." For similar reasons, "a well-designed booby trap usually results in onetrial learning." For example, a cat can be discouraged from jumping on a kitchen counter by arranging cardboard strips that stick out about 6 inches from the counter, weighted down on the counter with empty soda cans. When the cat jumps to the counter it lands on the cardboard. The cans go flying up in the air, and the whole kit and caboodle crashes to the floor. The cat quickly learns to stay off the counter. Meanwhile the cat does not blame this event on humans, so the cat does not avoid humans, just the kitchen counter.

What conditional response bedevils cat owners? How do automatic gadgets help?

Sometimes cats get into the nasty habit of defecating or urinating on a carpet. Once the problem starts, it is likely to continue, because even enzyme treatments designed to eliminate the odor do not eliminate all traces of it, and the odor "sets off" the cat in the manner of a conditional response. The behavior occurs when no human is present, and punishment by a human does not deter it, for reasons discussed above (punishment comes too late and the animal fails to connect the punishment with the behavior). What to do? The problem is urgent and motivates online buying, so entrepreneurs have responded. Gadgets designed to deter this behvaior typically combine a motion sensor with a can of pressurized air or a high-frequency audio alarm. The blast of air (or alarm) is triggered by the presence of the cat in the forbidden area. According to reviews by troubled cat owners at places like amazon.com, these devices sometimes work when all else has failed. They are also a good example of punishment from the environment.
What are several reasons dog trainers recommend against harsh punishment?

Dog trainers also recommend not using harsh punishment. Some dogs will "take it," but some will respond with an active defense reflex that could involve biting, even if the dog is normally friendly. (Terrier breeds are particularly prone to this problem, and a usually-friendly dog can surprise a child with a vicious response to being harassed.) Moreover, punishment is unnecessary with dogs. Dogs have been bred to desire the approval of humans. They respond very well to positive reinforcement as simple as a word of praise. When punishment is used with any pet or domesticated animal, it should be as mild as possible. For example, if cat owners have a kitty that likes to wake them up too early in the morning, the simplest and gentlest approach is negative punishment or response cost. Simply put the kitty out of the room. If that fails, a squirt gun works. Gentle methods are to be preferred with all animals. Trainers who handle wild horses no longer "break" them, the way they did a century ago. Modern

horse trainers win horses over with gentle and consistent positive reinforcement. It works just as well and results in a horse that enjoys human company.
How should cat owners respond to unwanted morning awakenings?

Is electric shock punishment ever justified? Some people argue against all use of electric shock, in principle, as if shock is always inhumane. But electric shocks come in all sizes. Small shocks do not cause physical injury, and they are very effective punishers that discourage a repetition of harmful behavior. Sometimes this is necessary and desirable.
What is an example of "effective and humane" use of electric shock?

In the case of electric fences used by ranchers, for example, shock is effective and humane. You can touch an electric fence yourself, and although you will get a jolt, you will not be harmed. But even large animals like horses will not test an electric fence more than a few times. Then they avoid it. Avoidance behaviors are self-reinforcing, so large animals will continue to avoid a thin electric fence wire, even if the electricity is turned off. They do not "test" it the way they test nonelectric fences (often bending or breaking them in the process). Electric fences also allow farmers and ranchers to avoid using barbed wire, which can injure animals severely.

THE PUNISHMENT TRAP

Ironically, stimuli intended as punishment may sometimes function as reinforcers. How can you tell when something intended as punishment is functioning as reinforcement? Observe the frequency of the behavior. If the behavior becomes more frequent, the intended punisher is actually a reinforcer. If a child responds to punishment by doing more of the same bad behavior, most parents will step up the level of punishment. Sometimes this only makes the behavior worse. If so, the parents are caught in the punishment trap.
How can you tell when attempts at punishment are actually reinforcing a behavior? What is the "punishment trap"? What typically happens when children are well behaved?

How could such a pattern occur? Consider these facts. The average parent is very busy, due partly to having children. The parent enjoys peace and quiet when children are being good or playing peacefully. Therefore, when children are well behaved, parents tend to ignore them. By contrast, when children misbehave, parents must give attention. Parents must break up fights, prevent damage to furniture or walls or pets, and respond to screams or crying. Most children are reinforced by attention. So there you have all the necessary ingredients for the punishment trap. Children learn to misbehave in order to get attention. One student noticed the misbehavior-for-attention pattern while visiting a friend: I was at my friend's trailer one weekend visiting with her and her small daughter. I played with Jessie, the little girl, for a while. Then Dee-Ann and I sat down to talk, leaving Jessie to play with her toys. She played quietly for a while, but all of a sudden she got up, stuck her hand in the potted plant, and threw dirt on the floor. Dee-Ann quickly scolded her and got her to play with her toys again. Dee-Ann then sat back down and continued with our conversation. In a few minutes, Jessie was throwing dirt again. Again, Dee-Ann got her to play with her toys, and then sat back down. But in a few minutes Jessie was throwing dirt.
Why did little Jesse throw dirt?

Dee-Ann could not understand why Jessie was acting like that. I then remembered the story about the parents hitting the kids for messing with things, but the kids wanting attention and doing it more often. So I thought maybe Jessie was being reinforced for throwing dirt because each time she threw dirt, Dee-Ann's attention reverted to her. I explained this to Dee-Ann, and the next time Jessie messed with the plant, Dee-Ann simply ignored her, picked up the plant and sat it out of Jessie's reach. That ended the dirt-throwing problems. [Author's files] Little Jessie probably got lots of loving attention when her mother was not engrossed in conversation with a friend. But some children receive almost no attention unless they are "bad." In such cases, serious behavior problems may be established. One student remembers this from her childhood: When I was a little girl, I always told lies, even if I did not do anything wrong... I think the only reason I lied was to get attention, which my parents never gave me. But one thing puzzles me. Why would I lie when I knew my dad was going to spank me with a belt? It really hurt. [Author's files]
How can a stimulus intended as punishment actually function as a reinforcer, in this type of situation?

The answer to this student's question, probably, is that she wanted attention more than she feared pain. Any attention-even getting hit with a belt-is better than being totally ignored. Similar dynamics can occur in a school classroom. One of my students told about a teacher in elementary school who wrote the names of "bad" children on the board. Some children deliberately misbehaved in order to see their names on the board.

How can a parent avoid the punishment trap?

The solution? It is contained in the title of a book (and video series) called Catch Them Being Good. Parents should go out of their way to give sincere social reinforcement-love, attention, and appreciation-when it is deserved. When children are playing quietly or working on some worthy project, a parent should admire what they are doing. When they are creative, a parent should praise their products. When they invent a game, a parent should let them demonstrate it. If you are a parent with a child in a grocery store, and you observe other children misbehaving, point this out to your own children and tell them how grateful you are that they know how to behave in public. Don't wait for them to misbehave. Point out how "mature" they are, compared to those kids in the next aisle who are yelling and screaming. Sincere social reinforcement of desirable behavior is a very positive form of differential reinforcement. It encourages a set of behaviors that might be called sweetness. With such a child, the occasional reprimand or angry word is genuinely punishing. This reduces the overall level of punishment considerably. A child who loves you and trusts you and looks forward to your support may be genuinely stricken by harsh words. A loving parent realizes this and adopts a gentler approach, which is usually adequate when a child cares about pleasing the parent.
What did Tanzer write about in the book titled Your Pet Isn't Sick ?

Pets are also capable of something like the punishment trap. They, too, can learn to misbehave or pretend to be ill, in order to get attention from humans. One veterinarian saw so many malingering animals trying to get attention by acting ill, coughing, or limping, that he wrote a book called Your Pet Isn't Sick [He Just Wants You to Think So] (Tanzer, 1977). It explained how owners who accidentally reinforced symptoms of illness caused pet problems. Owners will run over to a pet and comfort it, if it makes a funny noise like a cough. Soon the pet would be coughing all the time. Tanzer found that if the animals were not reinforced for the symptoms (after a thorough check to rule out genuine problems) the symptom would go away.

What did one vet call the "single most common problem" he encountered?

Another vet specialized in house calls so he could see a pet misbehave in context. He said unwitting reinforcement of undesired behavior was the single most common problem he encountered. The solution was the same as with many child behavior problems: "catch them being good." Praise the pet and give it lots of love when it acts healthy; ignore it when it starts coughing or limping. Usually the problem goes away. Of course, first you have to rule out genuine medical problems.

RESPONSE COST WITH A RABBIT

Recall that there are two forms of reinforcement (positive and negative) just as there are two forms of punishment (positive and negative). Negative punishment is more commonly called response cost and consists of removing a reinforcing stimulus. This has the effect of punishing a behavior, making the behavior less frequent. On the internet, several discussion groups cater to rabbit owners. Here is an example from one of them in which the solution to a problem involved response cost. [An American list member writes:] I have a 1 1/2 year old French lop and for his entire 1 1/2 years he has been obsessed with peeing on the bed. We discovered that if we kept the bed covered with a tarp, it would usually deter him from the bed, though not always. Up until about a month ago, we thought we had him pretty well trained, with only a few infrequent accidents. But then, my husband and I got married and Jordy (we call him Monster) and I moved in with my husband. He seems to have adjusted quite well, with the exception of his bed habit...
How was response cost used with a rabbit?

...Please help us, we want to keep Jordy as happy as possible, but we can't keep washing bedding every day. [A British list member responded:] We have two "outdoor" rabbits that come inside for about an hour a day. The older (male) rabbit used to pee on the bed. Whenever he did this he would immediately be put back in the hutch outside. After about 10 times of peeing on our bed, he learnt that if he peed, quite simply, he wouldn't be able to play with "Mum" and "Dad." We haven't had a problem since then. I imagine that if Jordy is put outside of the bedroom and denied affection for the rest of the evening he'll learn pretty quickly. Good luck! This is a fine example of response cost, because the rabbit's behavior was punished by removing a reinforcing stimulus. In this case the reinforcing stimulus was being allowed in the house. This stimulus was removed. Eventually, after about 10 repetitions, he learned the consequence of his behavior, and the problem behavior was eliminated.

APPLIED ANALYSIS OF ANTECEDENTS

So far most of our examples of applied behavior analysis have involved changing the consequences of behavior. But antecedents of behavior are important, too. Antecedents are things that happen before an event, and they may control behavior by signaling when a behavior will or will not be followed by a reinforcer. Recall the discussion of discriminative stimuli. An S+ is a stimulus indicating reinforcement is available. An Sis a signal that reinforcement is not available, or that punishment may be coming. Naturally, animals learn to perform a behavior in the presence of an S+ and to suppress it in the presence of an S-. Behavior reliably emitted or suppressed in the presence of a particular stimulus is said to be under stimulus control.

How can you manipulate antecedent stimuli to help study more?

B.F. Skinner

The powers of daily habit can jumpstart important life activities. Books about studying in college typically advise that students set aside a particular time and place for study. The familiar time and location triggers the studying behavior. That is important with studying because getting started is half the battle. Usually studying is not too painful once one gets started. Problems occur when a person never gets started or procrastinates until there is too much work for the remaining time.
How did B.F. Skinner apply this principle to increase his writing productivity?

B.F. Skinner, whose research on operant conditioning underlies virtually all of the second half of this chapter, used stimulus control to encourage his scholarly work. He followed a rigid daily schedule. At 4 a.m. he got up and ate breakfast, then he wrote for about five hours. Time and the environment of his home office served as discriminative stimuli to get him started on his writing. Around 10 a.m. Skinner took a walk down to campus (Harvard) to meet his morning classes. Walking probably became a stimulus for mulling over his lectures for the day. In the afternoon he attended meetings and scheduled appointments. With this routine he was always able to put in a few good hours of writing every day during his prime time, early morning, while also scheduling adequate time for his other activities.

SUMMARY: APPLIED BEHAVIOR ANALYSIS

Applied behavior analysis is the application of principles from operant conditioning to "real life" problems outside the conditioning laboratory. Lindsley's Simplified Precision Model recommends first pinpointing the behavior to be modified, recording the rate of that behavior, changing the consequences of the behavior, then (if one fails at the first attempt to change behavior) trying again with revised procedures. The first step in any behavioral intervention is to specify the behaviors targeted for change. Next, baseline measurements should continue until a stable pattern of behavior is observed. . During baselining, antecedent stimuli should also be observed; they are often important in behavior analysis. Baseline measurement may itself produce behavior change. When this is done deliberately (for example, to help people stop smoking) it is called self-monitoring. The Premack principle suggests that a preferred behavior can be used to reinforce less likely behaviors. Shaping is a technique that employs positive reinforcement to encourage small changes toward a target behavior. Prompting and fading is a technique in which a behavior is helped to occur, then help is gradually withdrawn or faded out until the organism is performing the desired behavior on its own. Differential

reinforcement is the technique of singling out some behaviors for reinforcement, while ignoring other behaviors. Negative reinforcement works wonders when employees are given "time off" as a reinforcer for good work. Babies are master behavior modifiers who use negative reinforcement to encourage nurturing behavior in parents. Punishment is effective in certain situations. Electric fences are arguably more humane than alternatives such as barbed wire for horses and other grazing animals. In human child-rearing, parents must beware of the "punishment trap," which occurs when children are ignored until they misbehave. The solution is to "catch them being good." Animals can also learn to misbehave or act ill, if it gets them attention. They, too, respond better to kindness than punishment. Analysis of antecedents can prove helpful in changing behavior. Dieters are often advised to avoid eating in front of the TV, so television does not become an S+ for eating. Time of day can be used as a discriminative stimulus for desirable behaviors such as studying. B.F. Skinner used this technique when he set aside a certain time every morning for writing.