Zaller Feldman - A Simple Theory of Survey Response

A Simple Theory of the Survey Response: Answering Questions versus Revealing Preferences Author(s): John Zaller and Stanley
Feldman Source: American Journal of Political Science, Vol. 36, No. 3 (Aug., 1992), pp. 579-616 Published by: Midwest Political Science Association Stable URL: http://www.jstor.org/stable/2111583 Accessed: 15/10/2010 09:07
Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at http://www.jstor.org/action/showPublisher?publisherCode=mpsa. Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.
Midwest Political Science Association is collaborating with JSTOR to digitize, preserve and extend access to American Journal of Political Science.
http://www.jstor.org
A SimpleTheory theSurvey of Response: Answering versus Questions Revealing Preferences*

Los John Zaller,University California, Angeles of StateUniversity NewYork Stony at Brook Stanley Feldman, of
variance: of of research besetbytwomajortypes "artifactual" is Opinion hugeamounts overfor trivial and timeresponse instability thecommon tendency seemingly changesin questionnaire a converts anomalous of We this form affect expression attitudes. propose simplemodelthat to the of intothenature publicopinion.The model variance" intosourcesof substantive ; error insight mostpeoplepossessopinions thelevelof notion that at abandons conventional implausible the but of items-and instead assumesthatmostpeopleare internally conflicted specificity typical survey on most to ideas overmost issues-and that political respond survey questions thebasisof whatever of Numerous areat thetopof their headsat themoment answering. are empirical regularities shown tobe consistent these with assumptions.
on that Virtually publicopinion all research proceeds theassumption cition zens possessreasonably well formed attitudes majorpolitical issuesandthat of viewis that surveys passivemeasures theseattitudes. standard are The when X favor they simply are a state survey respondents they say describingpreexisting offeeling favorably toward X. Accumulating evidence thevagaries masspolitical on of attitudes, however, has madethisviewincreasingly dubious.If,as is wellknown, peopleareasked the same questionin a seriesof interviews, theirattitude are reports highly evidence also shows, react to in as changeable. Many, much strongly thecontext in are and whichquestions asked,to theorder whichoptions presented, to are in Thesephenomena more wholly nonsubstantive changes question wording. are raise seriousdoubtsabout whatpublic thanmethodological curiosities; they measure. opinion surveys In view of this,we proposea new understanding themass survey of redo we at attitudes sponse.Most citizens, argue,simply notpossesspreformed in in around their thelevelof specificity demanded surveys. Rather, they carry Whenquesconsistent ideas and considerations. headsa mixof onlypartially
at annualmeeting thePolitical of for So*Originally prepared delivery thefifth Methodology The dataused in thepaperwerecollected theBoardof Overseers theNational of ciety. by Election Studiesunder grant a ScienceFoundation. thesedatamadeunusually from National the Collecting for at demands thestaff theInstitute Social Research theUniversity Michigan, on of of heavy where to thestudy was conducted. thisconnection are grateful ZoAnneBlackburn, In we StevePinney, and SantaTraugott. also thank Don Kinder, We Barbara MattLyons,and Geddes,ShantoIyengar, drafts thepaper.Finally, wouldliketo of on we Douglas Riversfortheir helpful comments various in thank assistance preparing dataforanalysis. the Neither these indiHiroaki Minatoforinvaluable for of or that vidualsnorinstitutions anyresponsibility anyerrors judgment fact bear mayappearin this paper. American Journal PoliticalScience,Vol. 36, No. 3, August1992,Pp. 579-616 of ? 1992bytheUniversity TexasPress,P.O. Box 7819, Austin, 78713 of TX
580
Feldman John Zallerand Stanley
of an tioned, theycall to minda sampleof theseideas, including oversample and to and recent events, use them by ideas madesalient thequestionnaire other cases,reflect choicesdo not,inmost offered. their But chooseamong options the as the that rather, reflect thoughts they anything can be described trueattitudes; at of that most are accessiblein memory themoment response. a better account claimscan, as we show, provide far A modelbasedon these suchrelated matters as evidence political on attitudes, including of theexisting of thancan currently consistency theeffects politicalawareness, and attitude of models thesurvey dominant response. of modelsofpublicopinion then and We beginwith review existing a outof modelanddrawupona range newandexisting evidence to linean alternative demonstrate value. its Theories LimitsofExisting instability Response research beenthediscovhas of One ofthemostunsettling findings opinion in of to quespeople'sanswers survey eryofa largecomponent randomness most in interviews, only If tions. thesamepeopleareaskedthesamequestion repeated The of about halfgivethesameanswers. datainTable1, basedon interviews the illustrate problem. can be seenfrom the As the samepersons months six apart, on entries the maindiagonals,only45% to 55% gave the same answerboth ' eventhough about30% couldhavedoneso bychancealone. The amount times, differs from issueto another Feldman one 1989),but (see instability ofresponse in typical. thecases shown Table 1 arefairly in of ConIn his famous paper"The Nature BeliefSystems Mass Publics," is to who instability due mainly individuals verse(1964) arguedthatresponse but attitudes nevertheless by indulgeinterviewers politely lack meaningful in of the between response options in front them-butchoosing an put choosing of he fashion. almost random "simply "Largeportions an electorate," suggested, the do not have meaningful beliefs,even on issues thathave formed basis forintense periodsof time" amongelitesforsubstantial politicalcontroversy (1964, 245). whocontend has Thisconclusion beenstrongly that, challenged scholars by fluctuate citizens haveunderlying greatly, responses" although people's"survey stable(Achen 1975, 1983; Dean and that "trueattitudes" are overwhelmingly that Moran 1977; Erikson1979; Feldman1989). The fluctuations appearin where are to responses attributed "measurement people's overtsurvey error,"
to 'Given data from onlytwo pointsin time,it is impossible distinguish systematic attitude of three- five-wave and change from random fluctuation. However, analysis datafrom panelsstrongly randomrather than systematic suggeststhatalmost all responseinstability represents change (Feldman1989).
A THEORY OF SURVEY RESPONSE
581
overRepeatedInterviews: Table 1. ResponseStability TWoExamples

American Relations withRussia (Corner Percentaging) Attitudesin January1980 Cooperate June 1980: Cooperate Middle Tougher Unsure N = 25% 7 5 4 (338) Middle 8 4 5 2 (153) Tougher 8 5 17 3 (266) Unsure 2 1 2 4 (74)
Level of GovernmentServices (Corner Percentaging) Attitudesin January1980 Cut June 1980: Cut Middle Keep Same Unsure N = 24% 8 5 8 (362) Middle 6 4 4 2 (122) Keep Same 5 2 15 3 (208) Unsure 6 2 3 6 (138)
were:"Some people feelit is important us to try for Note: The exactquestions to too hardto getalongwithRussia. Others feelit is a big mistake try hard very on wouldyouplaceyourself this togetalongwith Russia.Where scale,orhaven't on werethenaskedto place themselves a aboutthis?"Respondents you thought as scale. In thistable,points1, 2, and 3 havebeencounted "cooperseven-point as as 5, ate";4 is counted middle; 6, and7 havebeencounted "tougher." should the was: "Somepeoplethink government provide The secondquestion in fewerservices,even in areas such as healthand education, orderto reduce for to the Otherpeople feel it is important thegovernment continue spending. would in Where evenifit meansno reduction spending. services nowprovides it aboutthis?"Itemwas on you place yourself thisscale, or haven'tyou thought as recoded above. Election Source:National Studies,1980PanelSurvey.
of the one's attitudes is sucherror said to stemfrom inherent difficulty mapping of ontotheunavoidably questions. vaguelanguage survey to have critical deficiencies. Both approaches responseinstability Conof which takesanyinstability evidence a "nonattitude," an as was verse's thesis, to attitudes on highly abstract extreme claimintended characterize issues. only
582
Feldman Zallerand Stanley John
people'sattitudes and (1979) argue, issues,as Converse Markus typical On more or more lessstable. of and or maybe more less"crystallized" are,as a result this, apart can of how crystallization be measured But thisonlyraisesthequestion Sinceno one haseversaid,attitude stability. on effect response from supposed its out, (1988) havepointed more as and remains, Krosnick Schuman crystallization of stability. theory attitude than testable a a metaphor appears of error"theory responseinstability The newer"measurement agree,meacore. When, all estimates as at equallyunderspecifieditstheoretical of of one-half more thevariance typical or constitutes typically "error" surement of consists and this"error" whatexactly wonders one naturally items, attitude questions aboutthese knowso little Yet howithas beengenerated. we presently variance." "unexplained an namefor essentially alternative remains the that term Effects Response to variance hasbeenattributed meathat response to In addition therandom efartifactual "response variancefrom there existssystematic error, surement Soviet toward on Cold Warexperiment attitudes a Consider well-known fects." werewillingto allow In sample,37% of respondents journalists. a split-half half-sample, States.Yetwhen,in theother in reporters theUnited communist be should allowedin Russia U.S. reporters were askedwhether respondents first here to the agreeing allow Russianreporters (whichmostfavored), percentage to doubled 73%. of otherfindings thistype:people are less likelyto Thereare numerous in they just themselves interested politics after havebeenaskedabout as describe to1984); people'sattitudes and issues(Bishop,Oldendick, Tuchfarber obscure or e.g., religion by are wardabortion affected thekindsof items(concerning, et and that women'srights) precedeit (Tourangeau Rasinski1988; Tourangeau answersto open-ended questionsthan al. 1989); people give quite different options thatask themto choose amonga .eriesof prespecified to questions of features survey designdo irrelevant (Schumanand Scott 1987). Seemingly be people who might affect be onlyunsophisticated not,as might suspected, (Krosnick of all of they nonattitudes; ensnare types respondents suspected having andSchuman 1988;see also Bishop1990). thus survey questions effects makesit clearthat on The literature response it measure publicopinion.Theyalso shapeand channel by the do notsimply set and thealternatives, otherwise the issues,order frame in they manner which that to led researchers a conclusion seemsindisThishas of context thequestion. of the political at is but putable that fundamentallyodds with assumptions most attitudes: of peopledo notmerely namely, scientists aboutthenature political someconsiderableextent, areusing people on attitudes surveys;to revealpreexisting and are "attitudes" (Bishop,Oldendick, to thequestionnaire decide whattheir 1990). Tuchfarber 1984;Zaller1984;Feldman and mostdirectly. Tourangeau Rasinski makethisargument Psychologists
583
that to (1988), for example, argue responses attitude questions be understood can as theoutcome a question-answering of in process which people(1) decidewhat for theissueis; (2) canvastheir minds relevant thoughts; combine (3) ideas into attitude ontoavailableresponse and a coherent attitude; (4) map theresulting of features theinterview can affect options. Because, as they maintain, process can affect whatgetsreported each of thesesteps,thequestionnaire also readily as publicopinion. Morerecently, Wilsonand Hodges(1991) haveproposed modelin which a attitudes "temporary are constructs" aremadeup at themoment response that of on thebasis of ideas in a largebutinternally conflicted "database." In perhaps the mostambitious to attempt deal withthe question-answering process(and and an much else), Wyer Srull(1989) offer information-processing having model more dozenelements. than three What NeedstoBe Done from and Despitetheevidence psychologists survey methodologists, public of opinionresearchers largely ignoreboththe longstanding problem massive on over-time and response instability thenewerfindings questionnaire effects. of the makewhat amount essenMoreover, many thosewhorecognize problems of view.In thecase of response the tiallyto patch-ups thetraditional effects, to the from consists trying prevent problem of patch-up becoming conspicuous; thisis doneby,for order sureto keepquestion constant across example, making time seriessurveys in somecomputer-assisted or, surveys, randomizing question order acrossrespondents. thecase ofresponse In the consists instability, patch-up corrections create comthat of statistical corrections measurement for the error, the fortable existbeneath enormous illusionthatfixed"trueattitudes" surface noise. The challenge, is that both then, to devisea theory accommodates response and is to and effects that crafted thekindsof problems instability response and of to data facinganalysts publicopinion.This is whatwe attempt do in this The theory propose we admit, we than wouldbe necessary to paper. is, simpler all that have But explain ofthefindings psychologists nowdocumented. a theory to wouldhavelittle valuetomost sufficiently complex do this political scientists, it and our aim is, above all, to reachthisgroup-convincing that conventhe tionalunderstanding publicopinion unworkable that practical of is and a alternative available. is An Alternative Model oftheSurveyResponse choose whichever to attitude individuals According conventional theory, ownposition. if,as we contend, comesclosestto their But prespecified option do on maketheir peopletypically nothavefixed positions issues,howdo they choices? takesthe"true as Since mostsurvey research attitude" itsprimitive of unit
584
John Zallerand Stanley Feldman
analysis, little attention beendevoted thisquestion. however, turns has to If, one to studies employ that depth one useful interviews, finds much evidence. Among thebestoftheseis Hochschild's (1981) study attitudes of toward equality, What's Fair?From interviews 28 persons, her with Hochschild found peoplewould, that ifaskedto do so, readily answer fixed-choice questions, that but giventheopportunity talk,"peopledo notmakesimple to statements; shade,modulate, they or deny, retract, justgrind a haltin frustration. to Thesemanifestationsuncerof are tainty justas meaningful interesting thedefinitive and as of statements a beliefsystem" (238). Hochschild particularly emphasizes ambivalence many herresponthe of of dents. Thisambivalence leads to frequently them contradict themselves-which is to say,to givetemporally in unstable responses thecourseofa singleconversation.Consider account theattitudes one ofhersubjects this of of toward government incomeguarantees: "Vincent Sartori cannot decidewhether notthe or government shouldguarantee incomes,because he cannotdecide how much weight giveto thevalueof productivity. believes to He that richare mostly the at and. . . yethe is angry 'welfare cheats'whorefuse work.... to undeserving his between desireforequality his knowledge existing and of Caught injustice, will on theone hand,andhisfear that guaranteed a income benefit shirkers, even on theother, remains he aboutpoliciestoward poor"(252). ambivalent the models Current attitude seemquiteirrelevant these to observations. reaThe Sartori's is sonfor vacillation not,as students Converse of he might that has say, no opinion thisquestion, is itthat, usersofmeasurement models on nor as error has attitude" Hochschild unable measure that is might Sartori a "true say, to relior that has conably It is rather Sartori conflicting opinions, at leastconflicting that at siderations, lead himtogivedifferent responses different times, depending on howhe thinks abouttheissue. It is easy to objectto thelimitations rigor on in inherent depth interviews we that suchas Hochschild's. about Nonetheless, are persuaded thebasic point individuals and ambivalence-that possess multiple often opinions conflicting an toward Muchpsychological important issues-represents important insight. this for research reinforces view.Memory haveshown that researchers, example, of in but peoplestore hugeamounts information their long-term memories, can and of retrieve use onlya fraction itat one time.The particular material do they of recalldependson a combination chanceand recency activation. of Hence, and on peoplemakequitedifferent judgments beliefstatements, depending the information happento recall from and they long-term memory (Raaijmakers and Shiffren 1981;Wyer Hartwick 1984). Another research concerned socialcognition, with focuses tradition, mainly of on theorganization ideas in themind.A central in concept muchof thisreis a from search the"schema," term has beenadapted that cognitive psychology. A schema a cognitive is structure organizes that information experience and prior
585
ofnewinformaguidestheinterpretation around central a valueor idea andthat tionandexperience. of haveseveral them is peopletypically aboutschemas that A critical point For any availableforunderstanding givenphenomena. example,an individual ifthe quitedifferently newperson wouldreact to beingintroduced a "professor" associations That is, different of as were insteaddescribed "a mother four." different wouldbe noticed, of different qualities theperson wouldcometo mind, In and the would be drawnfrom person'smannerisms, so forth. conclusions Thus,Tesser wouldbe different. attitude toward person the the short, perceiver's psythrust muchcognitive of that the (1978), in statements represent dominant of writes: a feature themodel propose, we and nicely capture central chology that proof at pointin timeis the result a constructive "An attitude a particular any is toward objectbut,rather, an cess. . . . And, there nota singleattitude of on availableforthinking depending thenumber schemas of number attitudes or do abouttheobjects"(297-98). And"persons nothavea singlefeeling evaluschema cognitive depending upontheparticular vary ation an object.Feelings of we 'tunein'" (307). Hochschild's concerning insights These studiestendbothto corroborate the and to undermine conventional politicalscienceassumption ambivalence modeland Achen's black-and-white (whichis at theheartof bothConverse's error it for to measurement model)that is normal individuals have "true attitude" on attitude issues. In view of this,our modelwill follow coherent a single, that around their in headsa mixofmore or in Hochschild assuming peoplecarry where consideration defined a reason a is as for less consistent "considerations," thananother. who thinks one (E.g., a person favoring side of an issue rather a has aboutdefense spending waste"whiledeciding question about"Pentagon her on that issue.) a raised consideration maywellcontrol decision that as: axiomofourmodelmaynowbe stated The first considaxiom.Mostpeoplepossessopposing AXIoM1: The ambivalence lead them deto that on erations mostissues,that considerations might is, cidetheissueeither way. as the of that concept consideration, used in thisaxiom,is We emphasize it of wordforschema.First, is cast in thelanguage everyday notjust another a for rather intended political discourse Kelley1983),as befits term (see political to structures than analysis.Second,it makesno reference mental psychological to that of rawsensory suchas theinterpretation input, arecentral or operations, of theconcept schema. is the transform diverse considOurnext problem todecidehowindividuals in as erations their heads intoclosed-ended One possibility, Taylor responses. is makechoices"off top of the the and Fiske (1978) suggest, thatindividuals
586
oftenthe first one satisfactory thatcomes along. single, colorfulpiece of case history evidence.
head"on thebasis of thefirst that idea comesto mind.Thus,peoplemaymake socialjudgments seizingon "a single,sufficient salient by and . explanation . .
. . . .
rateor consensus information logically, influenced a peopleare moreoften by
. [I]nsteadof employing base
evidence bearsupona particular that use problem, peoplefrequently theinformation whichis mostsalientor availableto them, thatis, thatwhichis most to easily brought mind"(251). Tversky and Kahneman's (1982) well-known workon framing effects reinforces viewthat the individuals often overly are influenced a single, dominant consideration. by much Atthesametime, datainboth science(Campbell al. 1960) political et andcognitive that psychology (Anderson 1974) indicate on other occasions,individuals reachdecisions averaging acrossa range competing of ideas. Thus, by voters seemto decidewhich Kelley(1983) showsthat candidate to presidential "likes"and "dislikes" abouteach party support summing all of their by and up the the presidential candidate choosing one with highest total. and net The axiomswe propose to allowindividuals respond survey to on questions on one thebasis ofeither or many considerations, depending howmany happen in the is tobe readily accessible memory themoment question posed: at AxIoM2: The response axiom.Individuals answer survey questions avby acrosstheconsiderations happen be salient themoment eraging that to at of where is response, saliency determined theaccessibility by axiom. AxIoM3: The accessibility axiom.The accessibility anygiven of considerationdependson a stochastic sampling process, where considerations that havebeenrecently thought aboutaresomewhat more likely be sampled.2 to For thecase in whicha person devotes great and thought attention an issue, to Axiom3 implies that there maybe multiple considerations salient memory in at themoment answering of abouttheissue and hencemany questions considerationsto be averaged across.But a person who rarely thinks aboutan issue and who is confronted an interview by situation requires succession quick that a of answers (Feldman1990)mayhaveonlyoneconsideration immediately available inmemory, which in case theaveraging reduces answering thebasisof rule to on a single as "top-of-the-head" and consideration, suggested Taylor Fiske. by Thesethree axioms,although spareandinformal, be usedboth orgacan to nize muchexisting research to generate testable newhypotheses and aboutthe of nature themasssurvey as response, we shallnowbeginto show.
2Theempirical warrant thisaxiomis extremely for strong Bargh al. 1986;Higgins (see et and and King 1981;Bodenhausen Wyer1987).
. Instead of reviewingall the
587
Data on the of Sincewe base much ouranalysis datafrom 1987PilotStudy the of to pausingbriefly describethis NationalElectionStudies(NES), it is worth to inter that alia, the"considerations" unstudy. survey The attempted measure, to closed-ended items. The study was survey derliepeople'sresponses standard 457 wereinterviewed theMay in in apart; persons conducted twowavesa month in wave. All hadpreviously participated the1986Nawaveand 360 in theJune Other technical detailsof thestudy availablethrough are tionalElectionStudy. of theNES attheUniversity Michigan. was and to The basic method to ask peoplea closed-ended policyitem then ownwordsabouttheissuesitraised.The closed-ended ask them talkin their to aid of NES items job guarantees, on items weretelephone versions thestandard services spending. form respondents In A, were and to blacks,and government asked the open-ended after answering givenclosedthe probesimmediately probes was: of open-ended endedpolicy item. Theexactform the"retrospective" I'd Stillthinking aboutthequestion just answered, likeyouto tellme you Exactly what as that what ideascametomind youwereanswering question. went things through mind.(Anyothers?) your imto This probewas designed elicita "memory dump"of theconsiderations and Simon(1984) in workby Ericsson mediately salient people'sminds.Prior if after showsthat suchprobescan workeffectivelyaskedimmediately a given taskhas beencarried out. in In form interviewers theitems theusualway, read but, without waiting B, to an askedrespondents givetheir reactions theprincipal to idea for answer, they the in elements thequestion.For thejob guarantees question, probeswereas follows: of kinds couldyoutellme what me Before telling howyoufeelaboutthis, surethat aboutgovernment come to mindwhenyou think making things has of (Anyothers?) every person a goodstandard living? each personget aboutletting Now,whatcomes to mindwhenyou think own?(Anyothers?) ahead on their rereadthe original theseprobes,the interviewer Immediately following to is and closed-ended wording reply it. (Fullquestion question tooktheperson's intheappendix.) The "retrospective" The two typesof probesare clearlynot equivalent. in the whichwereposed after peoplehad answered question thenormal probes, was at weredesigned find what to out exactly on people'sminds themoment way,
588
of response. The "prospective" "stop-and-think"3 or probes, theother on hand, weredesigned inducepeople to searchtheir to memories morecarefully than wouldforpertinent considerations. that stop-and-think Note the they ordinarily in probesdo notraisenew ideas or pushtherespondent a particular direction; the to whatmeaning or she athe they simply require respondent say explicitly to of taches thedefining phrases thequestion. Respondents wererandomly assignedto question form and answered the in The three itemsand same typeof thequestion each wave of thestudy. test neartheendofeach waveofthesurvey. associated open-ended probes appeared Interviewers wrote downas faithfully possible responses theopen-ended as all to incidental comments side probes,including (e.g., "This is a tough one"). The weresubjected an elaborate to transcribed comments classification with scheme, each probe.4 on as many four as comments codedfor Respondents thestop-andthink averaged side about3.7 codablecomments policy with per item, almost all at The on respondents offering leastonecodablecomment. average theretrospectivesidewas 2.9. All comments, side comments, wereratedon severalvariables including codersat theInstitute Social Research theUniversity Michifor at of by staff was a gan. Becausethecoding project considered difficult onlyexperienced one, coderswereused. The mostimportant variable was "directional of thrust comwhichside of theissue,ifany, remark the Alwhichindicated favored. ment," noted this and 75% though variable ambivalence, confusion, nonissue concerns, had thrust. other codingclassification The of comments a cleardirectional key that was "frame reference,"variable included of a morethan140 categories and to of The tried capture substantive the content eachremark. frame codesreferred the and to general individualism principles (e.g., equality, roleof government), of the workethic,the fairness theeconomicsystem, particular groups(e.g., and blacks,theelderly), personal experience, particular government programs. in aboutthesecodes maybe found Table5 and associated information (Further in all described thestudy text; codesarefully codebook.) Thesedataarenotwithout limitations. most The obviousis coderreliability. About tenth all interviews double-coded, although a of were exactreliability and, the a "difference" coddataarenotavailable, coding between supervisor reported rate as for erson 10% to 15% of all cases. Thisdifference was regarded normal In 10% but than wouldhopefor. addition, one material thistype,5 itis higher of a that couldnotbe assigned directional thrust. of remarks wereso unclear they is in one A final limitation thedifficulty confidently distinguishing "considera3This designation theinvention Kathleen is of apt Knight. 4Fortheaid to blacksitem,there wereup to six probes: three questions, each followed a by As as remarks werecoded in connection each of thesesix queryfor"anyothers?" many four with On there weretwoinitial probes. theother items, probes, eachwith follow-up probes. 5Personal from who communication StevePinney, supervised project ISR. our at
589
opposing thrusts, is no this haveclearly tion"from another. Whentworemarks offer seriesof remarks thesamesideofan on a problem. peoplesometimes But or on considerations,justelaborations represent separate issue.Do suchremarks to as occurred, we as a singleidea? Even a person listening theinterviews they be codersworking from imperfect an transcript did,wouldsometimes uncertain; uncertainty. wouldexperience greater we could have been Even in retrospect, are notsurehow theseproblems in of We feelthePilot ameliorated thecontext a masssurvey. thus significantly attainable directly for are the examining ideadataarethebestthat readily Study of but that they are,nonetheless, tional underpinnings massattitudes, we admit far from perfect. TestsoftheModel Axiom Preliminary CheckoftheAmbivalence a checkof the of We beginassessment themodelby making plausibility ideas holdmultiple, mostindividuals conflicting on most claimsthat axiomthat in the of waysto measure extent ambivalence the issues.Ourdatagiveus three as public, follows: remarks eachpersonthat can be 1. A countofthenumber opposing by of a makestwocomments pairedagainstone anotherIf,forexample, respondent his a thrust, scoreon theconflict with liberal a thrust twowith conservative and scale is two.Ifhe makesthree more)on one sideoftheissueandonlytwoon (or of theother, conflict scoreis stilltwo,sincethenumber opposing comments the remains two. that be paired can 2. A countofthetimes ambivalence diffiexpress or people spontaneously to minds.A specialcode was created capture suchrein culty making their up conflict it "Mention indicates ambivalence, (e.g., 'I see marks; readsas follows: meritin both sides'; 'That's a toughquestion'; 'Depends'; 'Both are valid points')." Incomments." 3. A countofthenumber times of people make"two-sided a of codes are special"starcodes" thatindicate cludedin theframe reference to but with to directhrust thecomment also ambivalence respect that directional had but tion.Starcodes applyto cases in which respondents a preference were side to someattention theother oftheissue.Instructionscoders to clearly paying for ofstar codesreadas follows: use is or A star code is used onlyforcases in whichthere a singlethought that two elements I comment encompasses opposing (e.g., "Althoughthink Y" in R favor Starcodesareusedfor comments which sees X, I nevertheless twosidesto an issue.
590
own,but Examplesof starcodes are "people shouldtryto get ahead on their problem(s) "admits and respondent shouldhelp whennecessary" government A anyway." it but or withanyprogram typeof program, insists is worthwhile a of remarks maythusbe considered measure ambivacountof thestar-coded lence. on of a a we measures created fourth:summary theindices three From these are and Because conflict ambivalence whicha personscored + 1 or higher. or the whether occurwithin courseof one interview they equallyconsequential acrossbothwavesof the all interviews, indicesare calculated acrossseparate survey. are on distributions thesefourmeasures shownin Table 2. As Frequency a of substantial amounts ambivalence, result captures can be seen,each measure of evidence conservative with axiom.Evenon themore that consistent thefirst is the which involve onlyone queryin each wave,6 sumtheretrospective probes, amare indicates 36% to 48% ofrespondents to somedegree that mary measure What an issues. And thisis surely understatement. the on bivalent thesethree has is the as capture, explained, thereason person answered probes retrospective like of cannot anything thefullrange ideas capture theitemas he justhas; they to the probesweredesigned tap a in theperson'shead. However, prospective 75% them, roughly from on of wider range theideasinpeople'sminds; evidence on issues. conflicted thethree areat leastsomewhat ofrespondents axiom,the for clear initial support themodel'sfirst provide These results be we two cannot tested directly, turn axiom.Sincetheother axioms ambivalence of axiomstaken of implications thethree nowto an examination thedeductive all together. the Deductions First from Model the are straightforward from modelthat entirely We beginwithdeductions and ones. If,as to useful important and andperhaps uninteresting proceed more of consideration depends axiomclaims,theaccessibility a given theaccessibility that devoted an issue,we shouldfind to peoplewho of on theamount thought at involved havemoreconsiderations thetopof morepolitically are,in general, This is thefirst survey questions. headsand availableforuse in answering their we 1). from modelthat makeandtest the (Deduction (To keep of 18 deductions in as each willbe numbered parentheses, here.)Despite of track thedeductions, confirms this inthe of someindication nonmonotonicity data,Table3 essentially
expectation.7
in interest an issueto whohavegreater we persons Similarly, wouldexpect
we weredirective, makeno use ofthem. weremade,butsincethey probes 6More of version ourmodelleadsone sophisticated is probably real,as a more 7Thenonmonotonicity chap. 8). toexpect (see Zaller,inpress,
on of and Ambivalence PoliticalIssues Table 2. Expressions Conflict Probes Retrospective Considerations Conflicting Jobs Count
0
Probes Stop-and-Think Considerations Conflicting Jobs 36.9% 27.3 22.2 13.1 Services 30.7 29.0 21.6 18.7 Aid to Blacks 29.0 21.6 25.0 24.4
Services 57.8 33.6 5.2 3.4
Aid to Blacks 73.4 22.6 4.0 0.0
1 2 3+
73.9%a 22.6 3.5 0.0
of Expressions Ambivalence 0 1+ 76.9 23.1 83.5 15.5 78.8 27.8
of Expressions Ambivalence 63.2 36.8 72.2 27.8 71.1 28.9
(StarCodes) Two-SidedRemarks 0 1+ 75.0 25.0 91.7 8.3 81.4 18.6
(StarCodes) Remarks Two-Sided 64.9 35.1 85.2 14.8 72.3 27.7
TotalIndications 0 1 2 3 60.2 15.7 18.5 5.6 51.4 37.6 7.3 3.7 63.6 16.9 12.7 6.8 25.9 39.1 9.8 25.3
TotalIndications 24.3 50.9 14.8 10.1 24.7 44.0 12.0 19.3
in are Note:aMeasures described text. Source: 1987NES PilotStudy.
on of Table 3. Effect PoliticalAwareness Volumeof Open-EndedComments Awareness Level ofPolitical Low of Standard living services Government Aid toblacks N= 2.3 2.3 3.3 44 2.9 3.2 3.9 53 Medium 3.5 4.3 4.4 54 3.5 4.6 5.2 38 High 3.2 4.7 4.4 46
in remarks thegivencell. These of substantive are Note:Cell entries theaveragenumber discrete from probesshowthesame retrospective from probes,butresults stop-and-think data are derived in is measure described theappendix. The pattern. awareness Source:1987NES PilotStudy.
592
have,all else equal, more thoughts aboutthat issuereadily accessible memory in thanother persons (Deduction Since thePilotStudy notdirectly re2). did ask spondents how important interesting of thepolicyissues was to them, or each to ourability testthisexpectation limited. did, however, that is We find blacks raisedmoreconsiderations whites connection than in withtheaid to minorities item < .01). Government (p employees hadslightly also more saythan to other aboutthegovernment persons servicesitem(p = .07). Unemployed persons, were not more likelyto raise considerations however, to pertinent the job item. guarantees Axioms2 and 3 claimpeopleanswer survey questions averaging by across whatever considerations salient memory this so, we should are in If is find strong correlations between of measures people'sthoughts they as answer survey a item and thedirection decisionon theitemitself of (Deduction Thus if,forex3). makestworemarks favor liberalside of theissue and ample,a person that the one that favors conservative we wouldexpect theperson the that side, would,on this average,takethe liberalside of the issue. Although inference mayseem worth it it hardly testing, is by no meansobviousthat can be confirmed. Social in psychologists, haveturned cases working thedomainof social cognition, up in which direction people'sopen-ended the of is thoughts uncorrelated, even or with of negatively issue. correlated, evaluations thegiven As Hastieand Park(1986) havecontended an influential in essay,thesurnoncorrelations occurbecausepeopletypically notconstruct prising do attitude statements from ideas theycan retrieve from as memory theyare questioned. formed an earlier at Instead, peoplerecallattitudes time.Thus,they maintain, there no necessary is correlation between ideas top-of-the-head andattitude statements Lodge,McGraw, Stroh and (see 1989). substantial Notwithstanding ourdataindicate this, correlations between the ideas mostaccessibleto individuals themoment response theresponse at of and we additive indices people'sopen-ended of given.To showthis, created remarks, We coded fordirectional thrust. then correlated theseindices with to responses theclosed-ended as items, shownin Table4. On thestop-and-think correside, between indicesand their the lations associated item dichotomous in each wave ofthesurvey about.40. Whenan indexofall remarks both over averaged waves is of thesurvey correlated witha scale that consists responses theclosedof to endeditems from twowaves,thecorrelations the about.50. In theother average of the between half thestudy, correlations individuals' remarks and retrospective in their closed-ended .70. When responses thesamewaveofthesurvey averaged weresummed correlated remarks items and and acrosswavesofthe retrospective the in survey, correlations averaged.80. (Much of thedifference correlations between stop-and-think and theretrospective appearsto be due to the side side in error thecodingof theclosed-ended responses, which,as discussed below, seemsto have been higher thestop-and-think Giventhat closedon the side.)
593
Table 4. Relationship between AvailableThoughts and Closed-EndedItems Wave 1 Wave2 Combined
Correlations Remarks with Made Just before Answering Closed-Ended Question Jobguarantees Government services Aid to blacks .39 (212) .31 (187) .57 (220) .39 (161) .36 (153) .48 (165) .50 (173) .41 (165) .63 (166)
Correlations Remarks with Made Just after Closed-Ended Answering Question Jobguarantees services Government Aid to blacks .79 (126) .79 (137) .67 (144) .70 (123) .70 (105) .83 (114) .79 (105) .78 (106) .83 (112)
is correlation Note:Cell entry Pearson between remarks madejustbefore justafter) open-ended (or and First closeditem scoreson thecloseditem. column showscorrelations first from wave answering of survey; from secondwave; third secondcolumnshowscorrelations columnshowscorrelations between remarks from bothwavesandcombined item scoreson bothwaves. Source: 1987NES PilotStudy.
in endeditems thesetests essentially are and of dichotomies,8 thefragile nature ouropen-ended are data,these sizablecorrelations.9 We shouldadd thatthe findings correlations of between top-of-the-head and are thoughts attitude reports not,takenalone, clinching evidence thatthe former havecausedthelatter. the Rather, correlations-which, we haveindias in for of cated,couldnotbe taken granted light pastresearch-simply represent one of morethan dozencases in which havebeenable to develop a we evidence is that consistent thethree-axiom with model.
8A fewrespondents volunteered depends" "it middle which wereaccepted. responses, of 9The substantial do show that closed-ended magnitudes thesecorrelations not,however, valid of in survey responses after perfectly indicators theconsiderations people'sheads.As are, all, are we noteimmediately considerations salient different at below,different times, leading peopleto at makedifferent attitude times.WhatTable4 showsis onlythat closed-ended attireports different of tudereports reliableindicators theconsiderations are salient themoment making are that at a of response.
594
Instability Explaining Response one interviews as we haveindicated, is, over Responseinstability repeated withthe regularities associated and empirical of themostimportant disturbing we to In mass survey response. thissection, attempt use our modelto explain and proof We simpleillustrations ourapproach then thisinstability. beginwith ceedto moresystematic analysis. abouttheproper level of government Whenasked in the May interview favored higher emphatically one respondent, identified a teacher, as services, crisis, was an and spending. The country facing educational levelsof services for weredrastically needed. theteacher said, and moreexpenditures education reducethealready wouldinevitably servicesor spending Anycuts in federal a however, same the Just month later, funds availableforeducation. inadequate was spending. Government too big and cutsin government individual favored crisisthathad to had to be cut back. Therewas no reference theeducational 10 just preoccupied individual a fewweeksearlier. this people can answeridentical have long knownthatdifferent Researchers as different the of concerned topics.What vignette thevacillatquestions ifthey at the can the ingteacher showsis that sameperson answer samequestion differto enttimesas if it involved different topics.This can happen,according the answers vary that people'ssurvey model,becausetheconsiderations determine resurvey acrossinterviews. felt, Thus,peoplecan give strongly contradictory on their offeelings theissueor consciously mix without either changing sponses considerations that or experiencing ambivalence conflict-iftheparticular any their haveshifted. responses determine survey and view to detecting measuring Our data werecollectedwitha specific resummaries thesedataforfour of closed-ended suchshifts. Table5 presents the rethesedataare from stop-and-think in which side, spondents. Notethat " more than to aboutissuessomewhat wereencouraged think fully they spondents would. ordinarily of to standard living A. reaction a guaranteed Consider respondent His first with it by ideals;he was also bothered the was that was inconsistent American to aboutletting thosewhorefuse work.Yethe worried of unfairness supporting thatsome peopleneed specialhelp individuals ahead on their own, saying get In to and thatsocietyhas an obligation help theneedy. thesecondinterview, A there was no signof thisambivalence. Respondent gavesix reasons however, a of to own,including restatement his ought getaheadon their whyindividuals
of verbatim said. respondents transcriptions whatthisand other '?We would like to present of has the committee theUniversity Michigan determined suchuse at that However, HumanSubjects of oftherawprotocols wouldbe an invasion therespondents' toprivacy. right to that two who weresubject theconstraint we wanted respondents werestable " The selections items twowhowereunstable. and on theclosed-ended
Table 5. Contentof Open-EndedResponseson Job GuaranteesQuestion

about... when think you kinds things cometomind of Couldyoutellme what SecondWave First Wave mak... government every ingsurethat personhas a good standard living? of A: Respondent 110. Idea is unAmerican (Con.) eachper... letting son getahead on own? their ... government makeach ingsurethat person a good has of standard living? ... letting pereach son getahead on their own?
on (StableConservative FixedResponse) 180. Some people needhelp(Lib.) 161. Dutyto helpthe needy (Lib.) 158. Tax burden too great (Con.) 110. Idea is unAmerican (Con.) 346. Program/food stamps (Con.) 140. Individualism/ work ethic (Con.) 133. Equal opportufor nity exists all (Con.) 344. Program/education(Con.)
B: Respondent too 158. Tax burden (Con.) great 156. Gov't. redtape (Con.) 144. Valueofcompetition
if 136. Unfair some don'twork (Con.) make 145. All should italonebut someneedhelp (ETU) on (StableConservative FixedResponse)
344. Program/educa- 344. Program/educa- 140. Individualism/ work ethic tion(Lib.) tion(Lib.) (Con.) people 142. Shiftless 344. Program/educa- 219. Groupref.to tion(Lib.) deserve fate middle class (Con.) (Con.) to 152. Limited 147. Motivation 110. Idea is ungov't. work American (Con.) (Con.) (Con.) 140. Individualism/ work ethic (Con.) Conservative Liberal) to (Unstable, 140. Individualism/ work ethic (Con.) 151. Gov't. must insureequal opp. (Lib.) 140. Individualism work ethic (Con.)
C: Respondent 140. Individualism/ work ethic (Con.) 161. Dutyto helpthe (Lib.) needy D: Respondent
Liberalto Conservative) (Unstable, 150. Idea of welfare state (Lib.) 111. Fairness of Amer. system (Con.)
344. Program/educa- 140. Individualism/ work ethic tion(Lib.) (ETU)
596
John Zallerand Stanley Feldman Table5 (continued)

Couldyoutellmewhatkinds things of cometo mind when think about. you First Wave SecondWave
... government makingsurethat every personhas a good standard living? of 357. Program/housing(Lib.) 356. Program/welfare (Lib.)
. . . letting eachperson getahead on their own?
. . . government makeach ingsurethat personhas a good standard living? of 348. Program/health (Lib.) 143. Work welfare & (Lib.)
. . . letting eachpersongetahead on their own?
of Note:A fulldescription these codescan be found theICPSR codebook the1987PilotStudy. in for Each remark identified parentheses having liberal, is in as a or directional conservative, uncertain thrust that (ETU indicates theevaluational thrust theremark unclear). of was Evaluational thrust was codedindependently substantive of content remark. of
that feeling job guarantees un-American, are without raising opposing any considerations. Respondent who opposed government guarantees bothof his A, job in closed-ended attitude is reports, thus less stablein hisreaction theguaranteed to standard livingquestion of thanhis stableclosed-ended responses wouldsuggest. He wentfrom beingan ambivalent on conservative thisissue to beinga confident conservative. Giventheambivalence theinitial of interview, would one notbe surprised overmany if, encounters this with question slightly or rephrased versions thequestion, occasionally thecentral of he saw issueas aid totheneedy rather un-American than idealsand,on thisbasis,expressed support thesofor cial welfare option. he Certainly hasrealimpulses that in direction. Nowconsider C. respondent Although is ambivalent bothinterviews she at on thebasis of similar she considerations, changesherclosed-ended response from to conservative liberal. One can imagine shemayhave,ineffect, that tossed a mental coin in deciding how to answerthefixed question-not because,as Converse's nonattitudes thesissuggests, had no meaningful she reaction the to issue (clearly, did), and notbecause,as themeasurement modelssugshe error gest,she was notquitesurewhat question the asked(she saw itthesamewayat both but interviews), becauseshewas undecided between stable conlargely but flicting impulses. These cases showthateven if, as we contend, people base their attitude on reports theideasthat mostimmediately are salient them, is byno means to it easy to explainover-time In response instability. themoststraightforward case, arisesfrom in instability changes theconsiderations are immediately that salient
597
teacher. as an report, inthecase ofthevacillating at thetimeof making attitude as these are scenarios also possible, we haveseen.With quitedifferent Butother, of examination rein complexities mind,let us proceedto a moresystematic sponseinstability. on considerations possesscompeting If, as themodelclaims,individuals to ideashappen be atthe answer thebasisofwhatever on most issues,andifthey of of one a amount at minds themoment response, wouldexpect fair topoftheir is 4). (Deduction Thereason that in reports instability people'sattitude over-time not might that accessibleat one interview theconsideration(s) are stochastically is by supported a massof at be so prominent thenext.This inference strongly evidence (e.g., Table 1). existing instability, also expectsit to but response The modelnotonlyanticipates in 80% oftheconsiderations oneperson's Supposethat structure. havea definite her on issue,while20% induce a response a given her headinduce toward liberal proand that a secondperson, for these response; suppose toward conservative a on If responses a one-element are portions reversed. each based her survey in of the samplefrom distribution considerations herhead,each wouldexhibit wouldbe person over instability time,butoverthelongrun,thefirst response 80% 80% of thetimeand thesecondwouldbe conservative ofthetime. liberal tendencies are stableovertime,buttheir that wouldhavecentral Thus,citizens around thesecentral tendencies (Dewouldfluctuate greatly attitude statements in fact, thepattern has beenobtained that repeatedly duction Thisis, exactly 5). tradition error" in 1979; (Achen1975;Erikson byresearchers the"measurement and Krosnick1981; Feldman1989; Milburn1980; Judd,Milburn, Juddand Krosnick 1988). Zaller1990;see, however, havea larger in awarepersons number as shown Table3, more politically If, head and accessibleforuse in answering at of considerations thetop of their in stability their greater theyshould,all else beingequal, exhibit questions, formed from averan attitude The reports closed-ended responses. reasonis that indicator theunderlying of will age of manyconsiderations be a morereliable than basedonjustone ortwoconsideraof population considerations an average tions 6). (Deduction this Feldman failed confirm expectation, to initial research (1989) Although data setsthat shownin separate political and Zaller (1990) havemorerecently in assowitha reduction thechancevariation awareness in fact,associated is, between initial later the and ciatedwith (The difference reports. people'sattitude work usestests political of information is the of tests thisexpectation that recent of as themeasure political awareness.) to responses closelogic,peopleshouldbe morestablein their By parallel issues-thatis, issuesso closetoeveryendedpolicyitems doorstep concerning to 7). (Deduction that give peopleroutinely somethought them dayconcerns most
598
of in 7 Deduction maybe found thediscussion racial to tending support Evidence issues in Converse(1964), the discussionof moralissues in Converseand in (1989). (1979), andgenerally Feldman Markus to ought be ambivalence of Another implication our modelis thatgreater 8). (Deduction Since,as we with levelsofresponse instability higher associated within single a no ambivalence who justsaw,someindividuals exhibit apparent this in that, testing it be may interview nonetheless quiteconflicted, is essential that a of of implication themodel,we employ measure ambivalence spansboth an to of we a Accordingly, havebuilt measure theextent which indiinterviews. side of a givenissue one favor or theother consistently vidual'sconsiderations We this by acrossbothwavesof thesurvey. constructed measure meansof the formula: following
X
remarks) I (liberalremarks) X (conservative + (liberal) + E (conservative) X (ambivalent)
the remarks were that person's wouldindicate A scoreof one on thismeasure whilea scoreof zerowould or thrust all conservative, all liberalin their either of indicate thatthepersonhad madean equal number liberaland conservative for items cases in which on stability theclosed-ended We higher remarks. expect The remarks inthesamedirection. datainTable6 support run all oftheperson's was with statistia In of this thisexpectation. five six trials, measure associated in case, therelationship in stability; thesixth increase response callysignificant statistical (p significance = .07). achieves marginal might If peopleformed be. as as The results not,however, strong they are ratesshould stability acrossaccessibleconsiderations, by responses averaging dividedin their to considerations) 100% 50% (forpeopleevenly varybetween on consistent Particularly thestopconsiderations). (forpeople withperfectly How can this from expectation. this depart and-think thedatasignificantly side, be explained? is from expectations codingerror. reasonfortheshortfall The mostlikely on indicated codersdisagreed the that random check-coding earlier, As reported and obviously of remarks, suchmiscoding coding 10% to 15% ofall open-ended and be stable which should perfectly to respondents our impairs ability determine random. perfectly which an results Table6, we developed in for this To test explanation theimperfect of it the and of measure codingerror correlated with magnitude the item-level in shown Table6. The correlation as exwas, relationships stability-consistency on error consistently was 12 higher thestopcoding quitehigh. Moreover, pected,
that of we error item-level percentages remarks thecoders rates, used item-level '2Toestimate to a one varied from lowofabout5% for item remarks The rate uncodable of found be uncodable. to relationthat of We 20% for another. expected themagnitude thestability-consistency a high nearly of
599
of and Consistency Considerations Table 6. ResponseStability

Job Guarantees Government Services Considerations Retrospective Aid to Blacks
Consistency ofconsiderations: .00 .01 to .50

.51 to .99 1.00
N .50 (7) .80 (20)

.77 (15) .91 (63) (p < .01)
N .59 (11) .70 (25)

.78 (16) .87 (54) (p < .02) Considerations Stop-and-Think
N .57 (7) .71 (19)

.80 (15) .96 (71) (p < .01)
Consistency ofconsiderations: .00 .01 to .50 .51 to .99 1.00
N .63a (16) .68 (74) .73 (37) .88 (45)

(p < .02)
.54 .77 .80 .73
(14) (63) (50) (37)
.57 .83 .84 .88
(14) (66) (44) (41)
(p < .07)
(p < .01)
wave 1 towave from items to responses closed-ended stablein their are Note:Cell entries proportion measure. P-valuesarebasedon uncollapsed in is of 2. Measure consistency described text. Source: 1987NES PilotStudy.
in is from expectations higher and-think side, whichexplainswhytheshortfall 13 of form thesurvey. that about in the of the Within limits thedata,then, results Table6 areprobably at effects convensizablestability We as as strong could be expected. obtained and of in levelsofstatistical significance five sixtrials werecloseto signiftional a measurement error, smallsample,and significant despite icanceon thesixth, timebeshort low owingto therelatively abnormally base ratesof instability reinterviews. tween in pointneeds to be made. The measureof consistency One additional
for slopes,wouldbe greatest those regression by shipsin Table 6, as summarized unstandardized the This remarks. was strongly case (r2 = 0.93 on a logafithof rates uncodable with lowest the items in test chap.4). of details this arereported Zaller(in press, micfit; = 5; p < .05). Further df side. on ratesare higher thestop-and-think It that add, notsurprising error '3Itis, we might has if thrust a remark theremark been of to the wouldseemeasier,a priori, determine directional if as side, of just madeinjustification an opinion rendered, on theretrospective than ithasbeengiven side. as to a means therespondent, on thestop-and-think of phrase of as part a discussion what given
6oo
in Table6 was calculated overbothwavesofthesurvey order capture to consistencyof considerations within both interviews acrossthem. and The consistency measurecan, however, calculatedwithin singleinterview. be a Whenthisis the a clearfinding: one-wave measure consistof done,we comeup with strong, no encyof considerations almost capacity predict has to over-time response instability either (in form). This finding indicates theconflict that most responsible response for instais that than interviews that bility conflict occursacrossrather within and responas dentsare often unaware their of conflict theyanswerquestions. Thus, the no overgovernment vacillating teacher exhibited conflict services within either Mostlikely, but conflict acrossinterviews. oncetheteacher interview, substantial item the services of "bloated beganto viewthegovernment through prism either or that government" "education crisis,"he or she fell intoa mindset blocked fits aboutthe other thinking pointof view.'4This suggestion nicelywithour holdsthat and model,which peopleanswer survey questions hastily on thebasis searches. ofincomplete memory Explaining "Response Effects" effects masssurin The modelcan also explainseveral important response of or irrelevant features question order veys,thatis, cases in whichseemingly we of the a designaffects responses given.In thissection survey variety these response effects. "order such of Consider first on effects," as theeffect a prior question supof port therights communist for reporters, discussed as earlier. as themodel If, on ambivalent issues butansweron thebasis of claims,people are normally of whatever new ideas are mostaccessibleat the moment answering, raising to in should able to affect be the considerations immediate proximity a question different considerations salient answers given making by (Deduction 9). or of into The intrusion unexpected novel considerations the questionto be all answering processwould not, however, expected affect respondents that in equally.Some peoplemaypossessconsiderations are so consistent supissue that admission one competing of the consideration portof one side of an ambivalent theissueon should haveno effect. Others, however, be deeply may These evenbalanceofproandconconsiderations. that maypossessa roughly is, affected artificial are thepersonswho shouldbe moststrongly by changesin order question (Deduction 10). for et Tourangeau al. (1989) havereported support thisexpectation. People whoreported they mixedfeelings had aboutan issue(and whoalso said the that
'4Tourangeau Rasinski and (1988) distinguish between "interpretation" question "reof a and trieval" information of relevant answering In terms thisdistinction, results it. to of our suggest that is the of interpretation more source response important instability.
6oi
issue was veryimportant them)werequitesusceptible question to to order effects-or whattheauthors "carryover call effects" whileother persons were notsusceptible all. The carryover at effects thevulnerable in groups (where the groups varied from issueto issue)ranged from lowsof4 and8 percentage points on abortion welfare, highsof 34 and 36 percentage and to on points aid to the Contra with average an rebelsanddefense spending, of carryover 19 percentage points. is Another effect whatmaybe called the"enimportant of response type In dorsement effect." thiscase, reference a political to or figure groupsystemaltersthepublic'sresponses a givenpolicydilemma. to atically For example, that Mueller (1973) found questions indicated President that that Johnson favored a particular for policyoptionled to greater references to support it. Similarly, for communism increased in systematically support U.S. involvement Koreaand This anticommunism Vietnam. endorsement to effect, according Mueller, sug"somewhat observations. theonehand,support thewar On gested for conflicting was clearly to theanti-Communist in America thetime.To generate tied spirit at one a kindof warfever, merely to toss thewords,'Communist had invasion,' intothediscussion. theother On was hand,theCommunist element notentirely builtintotheresponse thewarbecauseAmericans to be reminded it to had of before their anti-Communism fully was activated" (1973, 48). The tendency of on peopleto base attitude are to reports theideas that mostimmediately salient in as sucheffects them, specified Axioms2 and3, wellexplains (Deduction 11). mostresponse are effects considered Although "methodological artifacts," or are names interpretations. Consider they sometimes givensubstantive these: 1. Race of interviewer after 1986 New York a Timespoll effects. Shortly that rate blackswas 37%, a Washingfound President Reagan'sapproval among tonPostpollestimated blackapproval Reaganwas only23%. The differof that whiletheTimes normal ence was traced thefactthat, to followed interview prowho their the cedures, Postused blackinterviewers informed blackrespondents in wouldbe participating a study theattitudes blackAmericans. of of that they effect thiswas to induceblackreof As Sussman(1986) maintains, likely the of to black"intheir evaluations Reagan'sperformance first spondents "think (the 15 item thesurvey). on
as '5This other and race-of-interviewer might interpreted"socialdesirability" effects be effects. If by social desirability effects meanscases in which one in attitudes peopleconsciously misreport is at a social desirability with argument. our order avoid embarrassment, to interpretation variance with socialdesirability taken meancases is to Butif,as seemsequallyconsistent available evidence, what inwhich their attitudes butareinfluenced theimmediate are context give to peopleareunsure by to effects then another greater weight a particular consideration, social desirability represent simply on in typeof situation whichpeoplemakeattitude reports thebasis of theideas mostimmediately salient them. to
602
experiment, In group effects. a classic social psychology 2. Reference to weremorelikely RomanCatholics that (1958) found and Charters Newcomb control) doctrine (e.g., on birth with wereconsistent church that stateattitudes Thiseffect to was their to if,justprior questioning, religion madesalient them. on attitudes. of of evidence theimportance "reference was taken as groups" and to news.According Iyengar Kinder of 3. "Priming effects" television certain to functions "prime" newsoften (1991), television (1987) and Iyengar presidential themmoreaccessibleforuse in evaluating making ideas, thereby and assessingthenain candidates elections, between deciding performance, attitude In problems. thisway,TV newsis said to affect tion'smostimportant attitudes. underlying individuals' changing permanently without reports orderIftheconsidand wording question of 4. "Framing effects" question can by can questions be primed TV news,they peopleuse in answering erations Lau and For or are by also be primed howquestions worded framed. example, shown that (1990) haveexperimentally Sears(1983) and Lau, Sears,andJessor with correlated evalare status morestrongly financial aboutpersonal questions are the when questions askedincloseproximity. of politicians uations incumbent one used is a The reason,presumably, that consideration in answering question a inducing correthereby questions, subsequent remains availableforanswering 16 lation. appearsto be a tenfor responsible each of theseeffects The mechanism have on at questions leastpartly thebasisofideasthat for dency peopletoanswer as maybe counted reto salient them.As such,they been made momentarily givesan explanation response for sponseeffects whichourmodelof thesurvey 12, (Deductions 13, 14, 15). order way in whichquestion yet Milbum(1987) has documented another that to that askingrespondents "tellme everything can effects occur.He found causedsubsequent of comesto mindwhenyou think a Liberal(Conservative)" thanwere with consistent one another to attitude reports be moreideologically thiseffect, however, of group.Milbumobtained theattitude reports a control to liberalor conservative leanings beginwith. who had either onlyforpersons conducted in thesefindings an experiment replicated Price(1991) has recently national data samples.In both and on collegestudents in nonexperimental from on to thatsimply askingpersons place themselves a liberalcases, Pricefound of the to increase ideological consistency scale was sufficient conservative rating who posagain,onlyforpersons to policyitems-but, responses subsequent or sessed a clear ideologicalleaning(i.e., were notcentrists unableto place scale). on theself-rating themselves
estimates "pocketof inflated effects artificially have these on is '6There disagreement whether 1990),butno disagree(see in bookvoting" NES surveys Lewis-Beck1985;Lau, Sears,andJessor do could,inprinciple, so. effects ment framing that
603
This type question of order effect also explainable is from model.Havthe orientations salient them prior answering inghadtheir ideological made to to just thoserespondents possesssuchorientations more policyitems, who are likely to in relyon them a consideration formulating as responses subsequent to policy thoseresponses morestrongly withtheir questions, thereby making correlated and with ideological positions hencealso moreideologically consistent one another (Deduction 16). Thissection examined variety empirical has a of regularities-question ordereffects, endorsement race-of-interviewer reference effects, effects, efgroup frame and fects, priming effects, question of effects, theeffect making ideology Within attitude salient. conventional someofthese theory, empirical regularities aretaken substantive as and findings someas methodological artifacts. have We as shown, however, all maybe explained manifestationsa common that of theoretical the of mechanism, namely, normal tendency peopleto respond survey to on to questions thebasis of theideas that happen be, forwhatever reason,immediately salient them. to the Explaining Effects ExtraThought of as Survey responses, conceivedhere,are not"attitudes" se; theyare per indicators themixof considerations theperson's of in unreliable mind-unreliable because,amongother answerwithout things, people normally retrieving from all considerations. however, memory relevant If, peoplecouldbe artificially a than normal of induced retrieve larger to number considerations,should it imtoclosed-ended prove reliability their the of responses items. in Our intent designing stop-and-think the was to create probes suchan into ducement. requiring individuals discusstheelements a question of By before to them call to mind takeaccount a wider and answering we wereinducing it, of would.We therefore that rangeof ideas thanthey normally expected responses the treatment wouldbe, all else equal, morereliable following stop-and-think of indicators the set of underlying considerations responses than made in the standard that intheretrospective condition way, is, (Deduction 17). our to is Unfortunately, ability testthisexpectation compromised an arby "no in tifact. Because of theuse of an explicit interest" option theretrospective condition notin thestop-and-think more failed the in but condition, respondents to to retrospective condition respond the issue item.Low-awareness persons weremostaffected thisquestion their rateaveraged by difference; no opinion condition only4% inthestop-and-think but 38% intheretrospective condition.'7 Thismeans that less-aware retrospective respondents, especially ones,area more and this selected to group would,for reason alone,be expected be more ideologthan This runs icallyconsistent their stop-and-think counterparts. artifact against
in the ratesacrossforms '7Bycontrast, difference no opinion averaged 13% (16% vs. 3%) in thehighest information quintile.
604
Zallerand Stanley Feldman John
it difficultdemto thegrain theanticipated of stop-and-think making more effect, onstrate effect, the particularly less-aware for respondents. response reliability We developed twotests theexpectation increased of of a of in thestop-and-think condition. thefirst we expected measure social In test, withthetarget welfare ideology (see appendix) be morestrongly to correlated and condiitems (jobs, government services, aid toblacks)inthestop-and-think condition. usedthefollowing We interactive regrestionthan theretrospective in where Form refers question to sionmodeltotest expectation, this form: Item= bo + b, x Form+
b2 x
Ideo. + b3 x Form x Ideo.
for who in 40% Whenwe estimated model respondents scored theupper this we that critical the of of ourmeasure political coefficient, b3, awareness, found for but direction all three statistical signifiranin theexpected items, achieved the of test cancein onlyone case. To increase statistical power theinteraction in of our smallsample-the number cases in each testaveraged about 140-we the be reestimated modelunder constraint all coefficients equal across the that The results, shown thetoppanelofTable7, confirmed in thethree items. expecthe of is a condition, tations: effect ideology twiceas largein thestop-and-think difference is statistically that significant. the for We also estimated model,under same constraints, respondents the 40% oftheawareness measure. Herewe found as in that, also scoring thebottom
on Table 7. The Effect Stop-and-Think of IdeologicalConsistency Low Awareness Intercept Ideology Form Ideologyx Form 0.01 0.89 (.21) -0.36 (.21) -0.24 (.27) 434 HighAwareness - .20 0.62 (.14) 0.00 (0.20) 0.62 (.26) 437
N=
modelwas estimated acrossthe Note:Model is shownin text; simultaneously aid wave 1 of the jobs, services,and minority items.Data are takenfrom "don'tknow" wave on answered survey, exceptin cases in which respondents if wave2 responses, any, wereused.Testwas conducted case their 1, inwhich in 40% and top40% of awareness scale. forrespondents scoring thebottom in Measures described theappendix. are Source: 1987NES PilotStudy.
605
in shown Table7, thestop-and-think notonlyfailed increase test to consistency, butmight havereduced We shallreturn thisapparent it. to in actually reversal a But we 17. moment. first, havea secondtestof Deduction If extra thought inattitude it enhance onlycorrelations not ducesmore reliable reports, should with of ideology also theover-time but stability theseresponses. can be seen in As the fail this In Table 8, however, data completely to support expectation. fact, less-aware peopleexhibit consistency thestop-and-think less in condition, while more-aware showno effect. ones to theseresults light theselection in of It is essential evaluate we artifact havedescribed. gaininreliability The informed in among highly persons Table7 of runs against grain theartifact so is especially the and likely be real.Thenull to
Table 8. The Effect Stop-and-Think of on Test-Retest Correlations Retrospective Jobguarantees Government services aid Minority .68 (40) .56 (41) .79 (53) Stop-and-Think .45 (62) .43 (58) .53 (57)
Low Awareness
MiddleAwareness Jobguarantees services Government aid Minority .64 .48 (31) .81 (33) .70 (39) .61
(37) (29)
.41 .38 (51) .51 (58) HighAwareness .55 (51) .58 .86
(60)
Jobguarantees services Government aid Minority
.70
(48) (45)
(32)
are Pearsoncorrelations. Numbers in parenare Note: Cell entries test-retest theses. Source: 1987NES PilotStudy.
6o6
Zallerand Stanley Feldman John
because theymight have been caused by the findings apparent and reversals, real not also artifact, moresuspect. are Butmight thereversals represent effects seriesof Indeed,they might. an impressive In of thestop-and-think treatment? Tim Wilson and colleagues (Wilson et al. 1989; Wilson and experiments, peopleto articuthat, contrary ourmodel,asking to Hodges 1991) haveshown reliability consistently reduces predictive the latethereasonsfortheir attitudes less knowledgeable aboutthegiven of attitude reports, especiallyforpersons attitude object. et The explanation thedisruptive for effects thought, Wilson al. mainof as attitudes causes aboutthereasonsfortheir tain,is thataskingpeople to think in are of them sampleideas that too heavily to weighted thedirection cognitive reports are that than ones.Attitude reactions theattitude to objectrather affective than based on thisunrepresentative sampleare, as they conclude,less reliable in are most accessible memory reports basedon theideasthat otherwise assertion ourmodel,which of is the Notethat argument this accepts central et behavthat show, people'sattitude reports (and also, as theWilson al. studies of are the iors)reflect ideas that at thetopof thehead at themoment decision In because thanany deepertypeof "trueattitude." fact,it is precisely rather in on salient ideasthat extra thought, bringattitude reports depend immediately of disruptive. inga biasedsample ideastothetopofthehead,proves of et at The argument Wilson al. aboutoversampling cognitions theexpense we in couldexplaintheunexpected results obtained Tables7 and 8. of feelings and evidence that (1991) haveamassedconsiderable Sniderman, Brody, Tetlock to on whereas less-aware are reports feelings, persons morelikely base attitude on It more-aware tendto respond thebasisof ideological ones principle. would with followfrom thatthestop-and-think this cognitive treatment, itsrelatively of to shouldprovemostdisruptive theattitude reports theless aware flavor, we which what havefound. is runs of Sincethis artifact, posthocexplanation inthedirection theselection makestheuseful that point we cannotbe surethatit is correct. nonetheless It that to failure confirm manipulation (Deduction17) maybe due to a thought If crafted failed mimic to carefully thought processes. so, a more people'snatural our the gain manipulation might produce reliability that modelanticipates.'8 yet Summary of The modelwe haveproposed like all models,a simplification what is, thanconventional precision actuallyoccurs. In addition,it has less formal
that that either showing thought manipulations stress '8There in fact, are, psychological studies of predictably different effects, disrupting attitude theaffective cognitive or aspects situations produce discussion these of reports somesituations notothers MillarandTesser1986). Forfurther in but (see chap. issuesas they relate themodel,see Zaller(forthcoming, 5). to
607
to error models(see Achen 1975) and paysless attention mental measurement But, despiteits limimodelsof psychologists. processesthando the attitude model is, we believe, uniquelysensitive tations,the simple three-axiom rewiththemass survey associated regularities to thewide rangeof empirical criticize modeland propose our is sponse.Our hope, therefore, thatas others broad on theorizing a sufficiently planethatit can they alternatives, keep their we regularities haveexamined. of the accommodate range empirical which an our for phenomena which modeloffers explanation, The empirical headings: three general under in we summarize Table9, maybe grouped search.Because memory on reports probabilistic of 1. Dependence attitude and thatare bothprobabilistic searches are reports based on memory attitude on (2) overtime; centered the tend reports to be (1) unstable attitude incomplete, of with and considerations; (3) correlated theoutcomes meanof theunderlying 3-5). Thisis also whypeoplewhoaremorecon(Deductions searches memory closed-ended in unstable their are considerations more underlying in flicted their 8). (Deduction responses survey survey that The notion individuals' madesalient. of 2. Effects ideas recently has salient been of in can responses be deflected thedirection ideasmaderecently race-of-interviewer effects, endorsement order effects, used to explainquestion and effects, TV newspriming framing effects, question reference group effects, 9-16). effects (Deductions aboutan that The on reports. notion thinking 3. Effects thought attitude of enablespeopletorecall levelsofpolitical awareness, issue,as gaugedbygeneral has responses reliable and of number considerations henceto makemore a larger response exhibit greater awarepersons politically whymore beenusedtoexplain on issues(Deand stability whythepublicas a wholeis morestable "doorstep" and awarepersons, persons whymorepolitically ductions 7). It also explains 6, relevant to thoughts aboutan issue,are able to recallmore concerned especially remakesattitude that thought greater the 1, it (Deductions 2). Finally, notion the with success,to explain onlylimited has morereliable beeninvoked, ports 17). to an issue(Deduction of at thought themoment responding effects extra of for of explanations many theparalternative it Although is easyto imagine be our with model,itwouldnot,we think, we ticular phenomena haveexplained of and that an alternative hastherange simplicity ourmodel.Yet easyto develop of that a representsimplificationa process ourmodeldoes, as we haveadmitted, have of Whatsorts simplifications we made? more complicated. be must much As processing. the involves issue of "on-line" One of themostimportant peopleoften arguethat others, HastieandPark(1986), among earlier, indicated "on-line" they as their to continuously attitudes operator" update use a "judgment in are said to storetheirupdatedattitudes information. People acquirenew
6o8

Table 9. List of EmpiricalPhenomenaThat theModel Claims to Explain
at considerations the awarehavemore more politically 1 Peoplewhoare,in general, questions. survey use headsandavailablefor in answering topoftheir have,all else equal, more in interest an issueshould 2. Peoplewhohavegreater persons. than accessiblein memory other issuereadily aboutthat thoughts the between ideasat thetopof people'sminds correlations 3. Thereshouldbe strong themselves. on decisions theitems and items their survey as they answer reports. in instability people'sattitude of amount over-time 4. Thereshouldexista fair tendencies havecentral should measurement to that subject repeated are 5. Opinions tendencies. central these around but that stableovertime, shouldfluctuate are over-time greater exhibit should awarepersons of reports politically 6. The attitude persons. than stability thoseof less-aware conpolicyitems to responses closed-ended 7. Peopleshouldbe morestablein their that concerns most issues-that is, issuesso close to everyday doorstep cerning to pay peopleroutinely someattention them. instawith levelsofresponse to higher ambivalence ought be associated 8. Greater bility. the affect should to in proximity a question 9. Raisingnewconsiderations immediate salient. different considerations answers givenby making by be affected manipulations on 10. Peoplewhoareambivalent an issueshould most abouttheissue. to in proximity a question that raisenewconsiderations immediate should affect intoa question or politician group the 11. Inserting nameof a prominent effect"). (the to thepublic'sresponses thequestion "endorsement the to affect responses quesshouldat leastsometimes 12. The raceof an interviewer he tionswhich or she asks. to responses can group affect that 13. Manipulations raisethesalienceof a reference has position. the on group a well-known questions which reference for accessible them more making certain ideas,thereby can 14. Newsreports "prime" effect"). on (the subjects "priming attitude statements related use in formulating with correlations proxicertain inducing ideas,thereby order "prime" can 15. Question items. materelated in orientation close proximity to abouttheir ideological individuals think 16. Inducing for content "prime" can ideology use in answering ideological to questions having thosequestions. an stating opinion aboutan issuebefore carefully more 17. Inducing peopleto think of (Notconfirmed.) report. the enhance reliability theopinion should memoryand retrievethemas required,ratherthan, as in our model, long-term each new surveyquestion. on statements the spot as theyconfront create attitude to processingand believe thatsome We are sympathetic on-lineinformation formof it eventuallymustbe included in models of political attitudes.Yet even advocates of on-line processing so farhave foundno evidence of the strongest of of (as such processingin the formation policy attitudes against the formation The issues about which evaluationsof persons). And this is scarcely surprising.
609
citizens mustanswer are survey questions too numerous, multidimensional, too for to and,in most cases, too obscure itto be feasible engageinon-line processingofall relevant information. Discussion In closing wouldliketoconsider we three broad questions about model. our Is it plausible? for Whatare its implications future opinion research? Whatare itsnormative for in implications theroleofmassopinion a democracy? Substantive Plausibility theModel of Of thevarious claimswe havemade,theambivalence axiom(alongwith its that do and implication individuals normally nothavea single,fixed, firm attitudeon issuesbutinstead havemany, is potentially opposing "considerations") It the and contradicts perhaps leastfamiliar hencetheleastintuitive. also flatly of theories political attitudes. Achen(1975) assumes thedominant academic that and that alall respondents "trueattitudes" maintains thesetrueattitudes, have to vacillate becauseofmeasurement areoverwhelmingly though appearing error, stable.Converse assumes have (1964) likewise that, although many respondents those have fixed nonattitudes, whodo haveattitudes perfectly andstable ones. of ideal and Despitethebreadth itsappeal,theconventional offixed stable "true is One attitudes" well pastdue forstrong questioning. needonlyconsider and in elitedebate,policyquestions that,in bothpublicopinionsurveys are in of framed terms summary abortion shouldbe typically judgments (whether or schoolchildren shouldbe busedto promote racialinpermitted not;whether of an or the tegration not)so that making these judgments requires aggregation diverse concerns. acrossfrequently Thereis absolutely reano ofone'sfeelings abouteach of theelements son to supposethata personmustfeelconsistently suchsummary that or she aggregates he acrosswhenmaking judgments. Thus, to her someone whosupports woman's a schedule need right control reproductive with fetuses of notalso feelcomfortable aborting (Scott1989); advocates govwith ernment to theneedyneednotfeelcomfortable big government aid (FeldmanandZaller1992). is for of of there is The heart ourargument that most people,most thetime, their no needto reconcile evento recognize or reactions events to contradictory a with andissues.Each representsgenuine feeling, capableofcoexisting opposon reingfeelings and,depending itssaliencein theperson's mind, controlling of sponsesto survey questions. Analysts publicopinion longhavebeen aware in consistent" their to isthat fewcitizens "ideologically are responses different is sues (Converse1964). Our argument that citizens equallyinconsisare many tent their in reactions different to an aspectsof thesameissue,where "issue"is asks respondents aggregate thata pollster to simplyany bundleof concerns a acrossinthecourseofanswering question. this it Although lineofargument seemunobjectionable,is easytoovermay
6to
look its implications. mostimportant thatindividuals The is typically not do develop"trueattitudes" thetypethat of opinion analysts routinely assume,but possessa seriesof autonomous often and inconsistent reactions thequestions to askedbypollsters. to putitanother Or, way,mostopinions mostissueshave on both central a and tendency a variance. Weemphasize, that in however, nothing ourmodeldenies that somepersons maydevelopwhollyconsistent of considerations respect some issets with to sues. Indeed,there no doubt is somepeople,including that many political activistsandothers whomight driven be toward cognitive consistency, exactly do this. Suchpeoplearewellcaptured conventional "true attitude" by models.Butthese peoplecan be accommodated equallywellbyourmodel,which, although holdingthat most individuals to someextent are ambivalent, allowssomeindividuals tobe unambivalent. Thus,ourmodelaccommodates themajority persons both of whoare,as Table2 indicates, inconsistent their in reactions diverse to aspects of issuesandtheminority arestable consistent. who and Empirical Potential theModel of The value of our modelforimproving on research publicopinionlies in three directions. One involves of opening themicrofoundationsattitude reup to ports empirical and For scrutiny analysis. example, researchers often refer to attitudes aremore less "crystallized," orless ideological, inother that or more or acrosspeopleand issues(Rivers1988; Sniderman, waysheterogeneous Brody, andTetlock is term 1991). But sincethe"attitude" alwaystheprimitive of analhavehad no wayof directly theseimputations. ysis,they Within our verifying a attitude be is model,however, crystallized to might one that found be basedon a larger morehomogeneous of underlying or set an considerations; similarly, be attitude or ideological might one basedon abstract principled considerations. The empirical of differences attitudinal in consequences thesemeasured microbe foundations couldthen investigated. Another for is direction future research in thearea of communication and Studiesof political assumethat persuasion. attitude persuasion typically change involves conversion in a one attitude structure reexperience which crystallized raisesdifferent Our mostnotably places another. model,however, possibilities, if The "persuasion framing." idea hereis that an eliteor someother by persuader one discusses issueso as tohighlight setofconsiderations an rather another, than thepublicwouldbe expected respond basingitsopinion to statements the on by considerations madesalient.Edelman's thus (1977) and Bennett's (1980) work on theeffects massopinion symbolic of on issuemanipulation elites,Kinder by and Sanders's(1990) workon the effects different of and questionframes, on use voter Popkin's (1991) observations howcandidates issuestomobilize supall the of mass the port affirm importance manipulating opinion manipulating by considerations aresalient thepublic. to that
6i i
of our possiblewithin conception attitudes, remains persuasion Traditional mainform. Zaller(1984, forthcoming) As different butitassumesa somewhat must as within framework notbe understood an all-orattitude our change tains, in but in nothing shift a "trueattitude," as an adjustment themixof considerato tions relating an issue. of ambivalent publiccan helpdevelop the Finally, notion a fundamentally process. between publicopinionand thepolicymaking linkages moreeffective in and DemocV. Forexample, 0. Key (1961) found PublicOpinion American prosocial welfare stronger of wanted a racythat sizablefraction theelectorate off tax putting someimportant whilealso favoring cuts"evenifitmeans tections much thespirit ourmodel,Key in of that things needto be done."In a comment were contradictory, real. As he appearing though thatbothattitudes, observed support taxreof makessimultaneous "a wrote, simplecalculusof self-interest . of entirely consistent. . . For activities duction and expansion social welfare combination irrational is this as thesystem a whole,however, typeof opinion in making.. . . The balance of forcesdrives and createsproblems program which and maybe regrestaxation, back concealed indirect policymakers toward siveinitsincidence" (168). (1967), who in their A similar pointhas been made by Free and Cantril a thepublicexhibited comfound that The BeliefsofAmericans study Political andoperational towelfare state ofphilosophical suppolicies bination opposition This"schizoid" belief patthefullrangeof welfare programs. for port virtually for to difficult government argued,made it extremely tern,Free and Cantril policy-making. engageinrational a that that on Thus,we feelthat conception emphasizes "opinion" an issue of than attitude" be more will a rather a single"true is generally range reactions links massopinion thepolitical and process. for fruitful investigating between Normative of Implications theModel thesis extremely are of nonattitudes The normative implications Converse's attitudes-aclaimthat Converse case bleak.In thelimiting of a publicwithout sense. As Achen(1975, makeslittle make-self-government did not actually In the loses itsstarting point." contrast, implitheory 1,227)putit,"Democratic mostmemwas optimistic: cationof Achen'sempirical investigation relatively stable. Thedisturbattitudes arealmost that perfectly bersofthepublichavetrue Achenargued, are discovered Converse, by change ingly highlevelsofresponse of researchers than rather to thevagueminds due to thevaguequestions survey ofcitizen respondents. and the between Converse Achenpositions. Our position fallssomewhere deal of uncertainty, tentativeness, that there a great is We agreewithConverse The we in masssurvey andincomprehensionthetypical response. problem, furwe Andyet,with than Achen, reject ther deeper vaguequestions. agree,is much
612
mostresponse black-and-white model,whichis that thepremise Converse's of by random guessing peoplewho haveno meanfluctuation due to essentially is ingful opinions. instabilevenwhenpeopleexhibit highlevelsofresponse Ourclaimis that Even the theyexpressmaystillbe based on real considerations. ity, opinions they the statements turn whentheseconsiderations outto be transitory, opinion in necessarily lacking authenticity. that reason, generate not,for are of survey results. This argument extendsto the interpretation aggregate of that approve thewayGeorgeBushis report they Thus,if55% of Americans it as that majority a of doinghisjob as president, shouldnotbe taken evidence it be of Rather, should taken thepublic unequivocally is supportive thepresident. Bush'sjob perare toward to meanthat 55% of Americans on balancepositive will 55% approval naturally though particular whoexpress the formance-even accidents of depending thecross-cutting on one to changefrom survey thenext, searches (Page andShapiro1992). people'smemory many different of between assertions individual-level our Thereis, then, inconsistency no in of on and ambivalence instability, theone hand,andbelief themeaningfulness one on poll aggregate-level results, theother-provided readspollsas revealing than of attitudes." rather as counts people's"true a balanceofconsiderations of of Thisconception an ambivalent publicmayfallshort ouridealof what in and to oratory ought be like,as thisidealis expressed political publicopinion in is that But gloridemocratic mythology. ifdemocracy possible a country both the to state;thatprofesses fieseconomicindividualism demands welfare and on racialdiscrimination; insists bothhigher that cherish and practices equality services hatesCongress rebut levelsof government and lowertaxes;and that incumbents extremely rates-thenitis also possible at electscongressional high of under understandingmasspublic our opinion. 2 1991 submittedAugust Manuscript 4 received December 1991 Final manuscript
APPENDIX Attitude Items in shouldsee to it that thatthegovernment Washington Some people think Jobguarantees. think government that should of has just let every person a job and a good standard living.Others own. eachperson aheadon their get kinds things of cometomind couldyoutellmewhat me Before telling howyoufeelaboutthis, has sure about of (Any when making that every person a goodstandard living? government youthink others) aboutletting eachperson ahead of cometo mind whenyouthink get Now,whatkinds things on their own?(Anyothers)
613
Government services.Some peoplethink government the should provide fewer services, even in areassuchas health education, order reduce and in to spending. Other peoplefeelit is important evenifitmeans increase spending. an in for government provide to the many more services Before telling howyoufeelaboutthis, me couldyoutellmewhat kinds things of cometomind when youthink aboutfewer government services? (Anyothers) Now,whatkindsof things come to mindwhenyouthink aboutincreases government in services?(Anyothers) in Aid to blacks. Some peoplefeelthegovernment Washington shouldmakeevery effort to of the feel should makeany not improve socialandeconomic position blacks.Others that government to should specialeffort helpblacksbecausethey helpthemselves. Before telling howyoufeelaboutthis, me couldyoutellmewhat kinds things of cometo mind of when youthink about"thesocialandeconomic position blacks?" (Anyothers) What comesto mind whenyouthink to aboutefforts improve social and economic the position ofblacks?(Anyothers) And (whatcomes to mind)whenyou hearthephrase blacksshouldhelp themselves? (Any others) Political Awareness Scale A 19-point an of v635to v642,v202(up to four scale, having alphareliability .85, as follows: candidate location points).Two recognition tests,v242, v244 (one itemeach). Fourcomparative v730 and v731; v749 and v750; v811 andv812; v831 andv832;plusv723 (a noncomparative items: location test). Scale Social Welfare Ideology of The social welfare scale consists 14 items individualism equality, and ideology concerning of as v620 to v622, v624, v626, v701 to plus two measures ideological self-designation, follows: v706,v2176,v2178,v2179,plusv722andvI 010. REFERENCES 1975. "Mass Political and Achen,Christopher. Attitudes theSurvey Response." American Political ScienceReview 69:1218-31. . 1983. "Toward of at Theories Data." Presented theannual convention theAmerican of Political ScienceAssociation, Chicago. Norman. 1974. "Information A Anderson, In in Integration Theory: Brief Survey." Advances Mathematical Psychology, 2, ed. David Krantz al. vol. et and Bargh, John, N. Bond,W. J.Lombardy, M. E. Tofa. 1986. "TheAdditive R. of Nature Chronic and Temporary Construct Journal Personality Social Psychology and Accessibility." of 50: 869-78. Bennett,Lance W. 1980. Public Opinion in AmericanPolitics. New York: Harcourt Brace Jovanovich. Bishop, George. 1990. "PoliticalInvolvement ResponseEffects." and Public OpinionQuarterly 53:209-18. W. Bishop,George,Robert Oldendick, A. J.Tuchfarber. and 1984."What MustMy Interest Poliin ticsBe IfI Just ToldYou 'I Don't Know'?"PublicOpinion 48:510-19. Quarterly GallenV., andRobert Wyer.1987. "Social Cognition Social Reality." Social S. and In Bodenhauser, Information and ed. Processing Survey Methodology, Hans J.Hippler, Norbert and Schwarz, Sudman.New York: Seymour Springer-Verlag.
614
E. DonaldStokes, Warren and Campbell, Angus, Miller.1960.TheAmerican Philip Converse, Voter. New York: Wiley. W. and Theodore Newcomb.1958. "Some Attitudinal Charters, W., Jr., Effects Experimentally of In Salienceof GroupMembership." Readingsin Social Psychology, ed., ed. E. Increased 3d and Theodore New York: Macoby, Newcomb, E. Hartley. Holt,Rinehart. of in In Converse, PhilipE. 1964. "The Nature BeliefSystems Mass Publics." Ideology Disconand FreePress. ed. New York: tent, David Apter. Markus.1979."Plus ,a Change. . : The NewCPS PanelStudy." Converse, Philip andGregory E., Political ScienceReview 73:32-49. American Mass Political Dean, Gillian,andThomasMoran. 1977. "Measuring Attitudes: Changeand Unreli4:383-401. Political ability." Methodology 1977.Political Academic Edelman, Murray. Press. Language.New York: and K. Simon. 1984.Protocol Ericsson, Anders, Herbert MIT Press. Analysis. Cambridge: Erikson, Robert.1979. "The SRC PanelData andMass Political Attitudes." British Journal Politiof cal Science9:16-49. 1989. "Measuring IssuePreferences: Problem Response The Feldman, Stanley. of Instability." Political Analysis 1:25-60. and . 1990. "Theory, Presented theannual at Measurement, theSurvey Response." meeting of Political theAmerican ScienceAssociation, Atlanta. and of Feldman, Stanley, John Zaller. 1992. "Political Culture Ambivalence: Ideological Responses Journal Political totheWelfare State." American of Science,36: 268-307. Free,Lloyd,and HadleyCantril.1967. The PoliticalBeliefs Americans. of New York:FreePress. Park.1986."TheRelationship between Hastie,Reid,andBernadette and Memory Judgment Depends on Whether Judgment the Task Is On-lineor MemoryBased." Psychological Review93: 258-68. 1989. "Noteson theDistinction Hastie,Reid, and NancyPennington. between and Memory-based In PersonPerception, John On-line ed. Bassili. Hillsdale,NJ: Judgments." On-line Cognitive Erlbaum. of Higgins,E. Tory,and Gary A. King. 1981. "Accessibility Social Constructs: Informationof and In Processing Consequences Individual Contextual Variability." Personality, Cognition, F. Cantor John Kihlstrom. and and Social Interaction, Nancy ed. NJ: Hillsdale, Erlbaum. Hans J.,and Norbert Schwarz.1987. "Response Effects Surveys." Social Information in In Hippler, and Survey ed. Processing Methodology, Hans J. Hippler, Norbert Schwarz,and Seymour Sudman.New York:Springer-Verlag. 1981. What's Fair? Americans'Attitudes towardDistributive Hochschild,Jennifer. Justice. Harvard Press. Cambridge: University Shanto.1991.Is Anyone of Iyengar, Responsible? Chicago:University ChicagoPress. of Iyengar, Shanto,and Donald Kinder.1987. News ThatMatters.Chicago: University Chicago Press. and and Lusk. 1984. "Knowledge Structures Evaluative Judd, Charles, Cynthia Effects of Judgments: Structural Variableson Attitude Journal Personality Social Psychology and Extremity." of 8:331-38. and 1980."TheStructure Attitude of in Judd, Charles, MichaelMilburn. Systems theGeneral Public: of A Comparison a Structural American Review 45:627-43. Equations System." Sociological MichaelMilburn, Jon and Krosnick. 1981."Political Involvement Attitude and Judd, Charles, StrucAmerican Review in Public." ture theGeneral 46:660-69. Sociological Jr. Elections. Princeton: Princeton Press. Kelley, Stanley, 1983.Interpreting University and New York:Knopf. Key,V. O., Jr.1961.PublicOpinion American Democracy. Political Debatewith Kinder, Donald,and LynnSanders.1990. "Mimicking The Survey Questions: on Action Blacks."Social Cognition for Case ofWhite 8:73-103. Opinion Affirmative
6i5
Krosnick, Jon. 1988. "Attitude Importance Attitude and Change."Journal Experimental of Social 24:240-55. Psychology Krosnick, Jon, Howard and Schuman.1988. "Attitude Intensity, Importance, Certainty Susand and Journal Personality Social Psychology and ceptibility Response to Effects." of 54:940-52. Lane, Robert.1962.Political Ideology. New York: FreePress. Lau, Richard, David0. Sears. 1983."Inducing and Apparently Self-interested Political Preferences." American Journal Political of Science27:233-52. David 0. Sears,andTomJessor. 1990. "FactorArtifact Revisited: Lau, Richard, Survey Instrument Politics." Political Behavior12:217-42. Effects Pocketbook and Lewis-Beck,Michael. 1985. "Pocketbook Votingin U.S. NationalElections:Fact or Artifact?" American Journal Political of Science29:348-56. and Lodge, Milton, Kathleen McGraw, Patrick Stroh.1989. "An Impression-Driven ModelofCandidate Evaluation." American Political ScienceReview 83:399-420 E. and M. 1985."Cognitive Loftus, Elizabeth Stephen Feinberg, Judith Tanur. F., Psychology Meets theNational American 40:175-80. Survey." Psychologist and Milbum,Michael. 1987. "IdeologicalSelf-schemata Schematically-Induced Attitude ConsisSocial Psychology 23:383-98. Journal Experimental tency." of of and and Tesser.1986. "Effects Affective Cognitive Focus on AttitudeMillar,Murray, Abraham Journal Personality Social Psychology and 51:270-76. Behavior Relationship." of . 1989. "The Effects Affective-Cognitive of Consistency and Thought the Attitudeon Social Psychology Behavior Relation." Journal Experimental of 25:189-202. John. 1973. War, and New York: Mueller, Presidents, PublicOpinion. Wiley. Page, Benjamin, and Robert Shapiro.1992. TheRationalPublic. Chicago:University Chicago of Press. Sam. 1991. TheReasoning Voter. Press. Popkin, Cambridge: Cambridge University Price,Vincent.1991. "Priming IdeologicalLeanings:The Impactof Salience and Commitment." BoardofOverseers, Arbor, Ann to Election Studies Report theNational Michigan. Shiffren. 1981. "SearchofAssociated Review Raaijmakers, andRichard J., Memory." Psychological 88:93-134. in Rivers, Douglas. 1988."HeterogeneityModelsofVoter H. Choice."American Journal Political of Science. and Presser. 1981.Questions Answers Attitude and in NewYork: Schuman, Howard, Stanley Surveys. Academic Press. in and Scott.1987."Problems theUse ofSurvey to Schuman, Howard, Jacqueline Questions Measure PublicOpinion." Science236: 957-59. 1989. "Conflicting BeliefsaboutAbortion: and Scott,Jacqueline. Legal Approval MoralDoubts." 52: Social Psychology Quarterly 319-26. Lau. 1983. "Inducing Self-interested Political Sears,David O., andRichard Apparently Responses." of American Journal Political Science27:223-52. and PhilipTetlock.1991. Reasoningand Choice. Cambridge: Sniderman, Paul, RichardBrody, Press. Cambridge University of 1986. "Do BlacksApprove Reagan?It Dependson Who'sAsking." Sussman, Barry. Washington Postweekly 10 February. ed. and Taylor, ShelleyE., and Susan Fiske. 1978. "Salience,Attention, Attribution: of theHead Top In in ed. New York: Phenomena." Advances Social Psychology, Leonard Berkowitz. Academic Press. in Abraham.1978. "Self-generated Attitude ed. Tesser, Change."In Advances Social Psychology, Press. Leonard Berkowitz. New York: Academic and Kenneth Rasinski.1988. "Cognitive Processes Context Effects Tourangeau, Roger, Underlying inAttitude Bulletin 103:299-314. Measurement." Psychological
6i6
and Norman Bradbum, Roy D'Andrade. 1989. "Carryover Rasinski, Roger,Kenneth Tourangeau, 53:495-524. Quarterly PublicOpinion in Surveys." Effects Attitude of of Amos, and Daniel Kahneman.1982. "The Framing Decisionsand thePsychology Tversky, San and ed. Consistency, RobinHogarth. Francisco: Framing Response Choice."In Question Jossey-Bass. of Effects and D. Lisle. 1989. "The Disruptive Dana S. Dunn, Dolores Kraft, Wilson,Timothy, vol. in WhyWe Feel theWayWe Do." In Advances Social Psychology, 19, ed. Explaining FL: Press. Orlando, Academic Leonard Berkowitz. as Constructs." The ConstrucIn and Wilson,Timothy, SarahHodges. 1991. "Attitudes Temporary and NJ: Hillsdale, Erlbaum. Tesser L. Martin. ed. tion Social Judgment, Abraham of as Judg1984. "The RecallandUse ofBeliefStatements Bases for Hartwick. Robert, Jon and Wyer, 20:65-85. of Social Psychology ments." Journal Experimental Hillsdale,NJ: in and Robert, ThomasSrull. 1989.Memory Cognition Its Social Context. and Wyer, Erlbaum. at of of Presented theannualmeeting a Response." Zaller,John.1984. "Toward Theory theSurvey DC. ScienceAssociation, Washington, Political theAmerican Response." and Elite OpinionLeadership, theMass Survey . 1990. "PoliticalAwareness, 8:125-53. Social Cognition Press. University Cambridge of and Origins Mass Opinion.Cambridge: . 1992. TheNature Forthcoming.

Zaller Feldman - A Simple Theory of Survey Response

Încărcat de

Informații document

Descriere originală:

Titlu original

Drepturi de autor

Formate disponibile

Partajați acest document

Partajați sau inserați document

Opțiuni de partajare

Vi se pare util acest document?

Este necorespunzător acest conținut?

Drepturi de autor:

Formate disponibile

Zaller Feldman - A Simple Theory of Survey Response

Încărcat de

Drepturi de autor:

Formate disponibile

A Simple Theory of the Survey Response: Answering Questions versus Revealing Preferences Author(s): John Zaller and Stanley

A SimpleTheory theSurvey of Response: Answering versus Questions Revealing Preferences*

Feldman John Zallerand Stanley

A THEORY OF SURVEY RESPONSE

overRepeatedInterviews: Table 1. ResponseStability TWoExamples

Feldman Zallerand Stanley John

A THEORY OF SURVEY RESPONSE

John Zallerand Stanley Feldman

A THEORY OF SURVEY RESPONSE

John Zallerand Stanley Feldman

rateor consensus information logically, influenced a peopleare moreoften by

. [I]nsteadof employing base

. Instead of reviewingall the

A THEORY OF SURVEY RESPONSE

John Zallerand Stanley Feldman

A THEORY OF SURVEY RESPONSE

Feldman Zallerand Stanley John

in interest an issueto whohavegreater we persons Similarly, wouldexpect

Services 57.8 33.6 5.2 3.4

Aid to Blacks 73.4 22.6 4.0 0.0

73.9%a 22.6 3.5 0.0

of Expressions Ambivalence 0 1+ 76.9 23.1 83.5 15.5 78.8 27.8

of Expressions Ambivalence 63.2 36.8 72.2 27.8 71.1 28.9

(StarCodes) Two-SidedRemarks 0 1+ 75.0 25.0 91.7 8.3 81.4 18.6

(StarCodes) Remarks Two-Sided 64.9 35.1 85.2 14.8 72.3 27.7

TotalIndications 24.3 50.9 14.8 10.1 24.7 44.0 12.0 19.3

in are Note:aMeasures described text. Source: 1987NES PilotStudy.

John Zallerand Stanley Feldman

A THEORY OF SURVEY RESPONSE

Table 4. Relationship between AvailableThoughts and Closed-EndedItems Wave 1 Wave2 Combined

Feldman Zallerand Stanley John

Table 5. Contentof Open-EndedResponseson Job GuaranteesQuestion

344. Program/educa- 140. Individualism/ work ethic tion(Lib.) (ETU)

John Zallerand Stanley Feldman Table5 (continued)

. . . letting eachperson getahead on their own?

. . . letting eachpersongetahead on their own?

A THEORY OF SURVEY RESPONSE

Feldman Zallerand Stanley John

A THEORY OF SURVEY RESPONSE

of and Consistency Considerations Table 6. ResponseStability

Consistency ofconsiderations: .00 .01 to .50

N .50 (7) .80 (20)

N .59 (11) .70 (25)

N .57 (7) .71 (19)

Consistency ofconsiderations: .00 .01 to .50 .51 to .99 1.00

N .63a (16) .68 (74) .73 (37) .88 (45)

.54 .77 .80 .73

(14) (63) (50) (37)

.57 .83 .84 .88

(14) (66) (44) (41)

John Zallerand Stanley Feldman

A THEORY OF SURVEY RESPONSE

Feldman John Zallerand Stanley

A THEORY OF SURVEY RESPONSE

Zallerand Stanley Feldman John

Ideo. + b3 x Form x Ideo.

A THEORY OF SURVEY RESPONSE

Jobguarantees services Government aid Minority

Zallerand Stanley Feldman John

A THEORY OF SURVEY RESPONSE

Feldman Zallerand Stanley John

A THEORY OF SURVEY RESPONSE