Cognitive Task Analysis Running head: COGNITIVE TASK ANAL SIS

Cognitive Task Analysis !enni"e# $add#ell Old %o&inion 'nive#sity I%T ()* Advan+ed Inst#u+tional %esign Te+hni,ues %#- Ga#y $o##ison O+to.e# 1/0 122(

Cognitive Task Analysis T#aditional Task Analysis A t#aditional 3#o+edu#al task analysis des+#i.es a task as a se#ies o" dis+#ete a+tions 4!onassen0 Tess&e#0 5 6annu&0 17778- 9igu#e 1 diag#a&s a 3#o+edu#al task analysis "o# the insu#an+e unde#:#iting su.&ission #evie: task- ;ithin this t#iage task0 the unde#:#ite# &ust evaluate va#ious as3e+ts o" the ne: su.&ission and de+ide :hethe# to ,uote o# de+line the su.&issionFigure 1. <#o+edu#al Analysis o" Insu#an+e 'nde#:#iting Su.&ission Revie: Task-

Guide to symbols: = Input and exit point; = Mental operation; = Decision Point; = Direction in Step

Cognitive Task Analysis

As de3i+ted in 9igu#e 10 in +o&3leting the su.&ission #evie: task0 the unde#:#ite# &ust &ake a se#ies o" &ental o3e#ations and de+isions in #oute to a +on+lusion to eithe# a8 de+line the su.&ission o# .8 ,uote the su.&ission- These &ental o3e#ations and su.se,uent de+isions in+lude the "ollo:ing: Assessing the viability of the opportunity- '3on #e+ei3t o" the su.&ission0 the unde#:#ite# &ust &ake a ,ui+k #evie: o" the in"o#&ation 3#ovided to assess the via.ility o" the o33o#tunityGiven the in"o#&ation 3#esented :ithin the su.&ission and dis+ussions :ith the .#oke#0 the unde#:#ite# &ust =udge the likelihood the a++ount :ill a+tually leave the in+u&.ent +a##ie#C#iti+al +ues to +onside# in+lude 3#io# se#vi+e and +lai&s handling 3#o.le&s :ith the in+u&.ent +a##ie#0 ti&e to t#ansition the a++ount0 and +o&3leteness o" the su.&ission- I" the #elationshi3 :ith the 3#io# +a##ie# has .een good0 the#e is little ti&e to t#ansition the a++ount0 o# the .#oke# only 3#ovided enough in"o#&ation to 3#ovide a 3#i+e 4not se#vi+e8 ,uote0 it is likely the insu#ed is not se#ious a.out &oving "#o& the in+u&.ent +a##ie# and the .#oke# is =ust seeking +o&3a#ative 3#i+e ,uotes- 6o:eve#0 i" the insu#ed is dissatis"ied :ith the in+u&.ent +a##ie#>s se#vi+e0 the#e is a&3le ti&e to t#ansition the se#vi+ing o" the a++ount0 o# the su.&ission 3#ovides a +o&3#ehensive ove#vie: o" .oth 3#i+e and se#vi+e #e,ui#e&ents0 it is likely the o33o#tunity is via.leI" the assess&ent o" the in"o#&ation leads to a +on+lusion that the +han+es a#e sli& the a++ount :ill &ove0 the unde#:#ite# &akes the de+ision to de+line the a++ount- 6o:eve#0 i" the assess&ents leads to a +on+lusion that the#e is a good +han+e o" :#iting the a++ount0 the unde#:#ite# &akes the de+ision to +ontinue :o#king on the a++ountExamining the employee concentrations. Given the 3otentially +atast#o3hi+ e?3osu#e o" 3#oviding +asualty insu#an+e at lo+ations :ith high e&3loyee +on+ent#ations0 the unde#:#ite#>s t#iage o" the su.&ission in+ludes an e?a&ination o" e&3loyee +on+ent#ations- I" the insu#ed has e&3loyee +on+ent#ations at any one lo+ation a.ove +o&3any guidelines0 the unde#:#ite# &akes the de+ision to de+line the a++ount- Othe#:ise0 the unde#:#ite# &akes the de+ision to +ontinue :o#king on the a++ountComparing the accounts exposures to the companys underwriting guidelines. '3on #e+ei3t o" the su.&ission0 the unde#:#ite# &ust +o&3a#e the 3#os3e+tive a++ount>s e?3osu#es to the insu#an+e +o&3any>s unde#:#iting guidelines- C#iti+al to this +o&3a#ison is a #evie: o" the insu#ed>s +u##ent and 3#io# o3e#ations- I" the +o&3any is involved in any o3e#ations :hi+h #esult in e?3osu#es that a#e against the unde#:#iting guidelines0 the unde#:#ite# &akes the de+ision to de+line the a++ount- Othe#:ise0 the unde#:#ite# &akes the de+ision to &ove "o#:a#d :ith the ,uotation task 4.eyond the s+o3e o" this su.&ission t#iage task analysis8Cognitive Task Analysis A +ognitive task analysis 4CTA8 o""e#s an alte#native &eans o" des+#i.ing the +ognitive ele&ents o" the evaluation and de+ision &aking 3#o+esses involved in the task- The "ollo:ing 3#ovides the #esults o" an A33lied Cognitive Task Analysis 4ACTA8 .ased on inte#vie:s +ondu+ted :ith an unde#:#iting su.=e+t &atte# e?3e#t 4S$E8 to gain in"o#&ation a.out +ognitive st#ategies used to +o&3lete the su.&ission t#iage task 4$ilitello 5 6utton0 177(8- The ACTA in+ludes a task diag#a&0 kno:ledge audit ta.le0 si&ulation inte#vie:0 and +ognitive de&ands ta.le-

Cognitive Task Analysis

Tas diagram 9igu#e 1 is the task diag#a& gene#ated a"te# an initial inte#vie: :ith the unde#:#iting S$E- The task diag#a& o""e#s a high level ove#vie: o" the su.&ission t#iage task :hi+h "o+uses on the &ost di""i+ult +ognitive as3e+ts- The S$E :as asked0 AThink a.out :hat you do :hen you t#iage a ne: 3#os3e+t- Can you .#eak this task do:n into less than si?0 .ut &o#e than th#ee ste3sBC The S$E &entioned "ive ste3s0 .ut one :as eli&inated 4"inan+ial a33#oval8 as it is not task 3e#"o#&ed .y unde#:#ite#Figure !. Task %iag#a& "o# Ne: A++ount <#os3e+t T#iage-

"nowledge Audit Table %u#ing inte#vie:s :ith the S$E0 the inte#vie:e# 3#o.ed "o# +on+#ete e?a&3les0 +ues and st#ategies0 and #easons :hy the task is o"ten di""i+ult "o# novi+es- The inte#vie:e# asked the S$E to "o+us on s3e+i"i+ e?a&3les "o# ea+h as3e+t o" e?3e#tise- Ta.le 1 su&&a#iDes the #esults o" the kno:ledge audit "o# the su.&ission t#iage task#imulation $nterview %u#ing a si&ulation inte#vie: :ith the S$E0 the inte#vie:e# asked the S$E to "o+us on the +hallenging as3e+ts o" a s3e+i"i+ #e3#esentative s+ena#io asso+iated :ith ne: su.&ission t#iage- Ta.le 1 su&&a#iDes the #esults o" the si&ulation inte#vie:0 in+luding the a+tions0 assess&ents0 +ues0 and 3otential e##o#s identi"ied "o# ea+h +ent#al eventCognitive %emands Table Ta.le * +onsolidates and synthesiDes the data +olle+ted du#ing the inte#vie: 3#o+ess- The +ognitive de&ands ta.le +ente#s on the +o&&on the&es that +a&e "#o& the inte#vie:s and identi"ies the di""i+ult +ognitive ele&ents0 +o&&on e##o#s0 and +ues o# st#ategies used .y e?3e#ts to ove#+o&e these +hallenges-

Cognitive Task Analysis Table 1. Kno:ledge Audit Ta.leAspect of expertise Past and future E?a&3le: Call "#o& .#oke# a.out a++ount :he#e in+u&.ent +a##ie# &essed u3 on +lai& and insu#ed>s legal de3a#t&ent insisting the a++ount &ust &oveCues and strategies 6igh level natu#e o" in+u&.ent &ess u3 Level o" 3eo3le involved in de+ision 4lo: level ve#sus high level8 Big picture E?a&3le: Ste3s .a+k "#o& all the "a+ts a.out the a++ount 3#esented .y the .#oke# to +onside# :hat is the A#ealC &otivation .ehind looking "o# a ,uoteB Is this 3#os3e+t a t#ue o33o#tunity o# does the .#oke# =ust need a +o&3eting 3#i+e ,uoteB I" it is only a need to get +o&3eting 3#i+e ,uotes0 highly unlikely the a++ount :ill &oveNoticing E?a&3le: E#oke# not soli+iting T<A ,uotes "o# +lai& handling :hi+h :ould .e a F1 +ondition o" a+tually &oving the a++ountJob Smarts E?a&3le: 9o+us on :hat .#oke# said in +onve#sation ve#sus 3u#ely :hat is 3#esented in the ,uoteOpportunities E?a&3le: Ou# unit +an>t :o#k on this a++ount0 .ut othe# units in +o&3any +anAnomalies E?a&3le: E#oke# doesn>t #etu#n 3hone +alls- Sho:s a la+k o" inte#est Eeyond 3#i+e0 the#e se#vi+e issues :ith 3#io# +a##ie# ou# 3e#sonal histo#y :ith that .#oke# Ti&e "#a&e to #elease ,uote ;hat othe# +a##ie#s a#e ,uoting

Going .eyond unde#:#iting in"o#&ation 3#esented in the su.&ission Conside#ing +onditional things that i&3a+t you# ,uote Ti&e"#a&es Othe#s +a##ie#s .eing asked to ,uote Reasons "o# leaving 'nde#standing o" unde#:#iting a33etite o" othe# units Kno:ing ho: to a++ess those 3eo3le Ti&ing o" #etu#ned 3hone +alls E?tent o" #es3onse to ,uestions 4eithe# la+king o# detailed8

Why Difficult? Novi+e &ay not #e+ogniDe signi"i+an+e o" &essed u3 +lai& handling Novi+e &ay not link level o" insu#ed to seve#ity o" 3#o.le& Novi+e &ay not link seve#ity o" 3#o.le& to in+#eased +han+e o" :#iting a++ountNovi+es &ay not +onside# othe# issues .eyond 3#i+e that in"luen+e .uying de+ision Novi+es do not have #elationshi3 :ith .#oke# to kno: :hen you a#e getting the Ast#aightC "a+ts ve#sus a Asales 3it+hC Novi+es &ay get into the &inutia o" the a++ount s3e+i"i+s and not ste3 .a+k and #ealiDe the ti&e"#a&e is not "easi.le to a+tually &ove the a++ount Novi+es a#e "o+used on details :ithin su.&ission Novi+es a#e "a&ilia# :ith AoutsideC +onside#ations that a""e+t the likelihood o" :#iting the a++ount

Novi+es tend to .e 3#eo++u3ied :ith ve#i"ying details :ithin su.&ission Novi+es not a:a#e o" situational issues that +an .e Adeal .#eake#sC o# Adeal &ake#sC Novi+es don>t kno: unde#:#iting a33etite o" othe# units Novi+es don>t kno: 3eo3le outside o" the unit Novi+es &ay not #e+ogniDe they a#e Agetting .lo:n o""C and they +ontinue :o#king on su.&ission Novi+es don>t #e+ogniDe signi"i+an+e o" Aout o" sight G out o" &indC :hi+h is signal i" you a#e alive o# dead

Cognitive Task Analysis Table !. Si&ulation Inte#vie:Events %is+ussion a.out 3#os3e+t :ith .#oke# Actions Ask 3#o.ing ,uestions a.out o33o#tunity Sensing tone "#o& .#oke# o" u#gen+y and desi#e to have you ,uoteAssessment Ans:e#s to ,uestion &ake sense o# not :ith :hat is in the su.&ission E#oke# :ants to :o#k :ith you o# =ust :ants a ,uote "o# +o&3a#ison 3u#3oses 6o: &u+h ti&e is the#e .et:een no: and e""e+tive dateB A#e the e?3osu#es inhe#ent in #isk a++e3ta.le unde# ou# unde#:#iting guidelinesB Critical Cues Can you &eet the issued The#e is disa""e+tion :ith in+u&.ent O3enness o" the .#oke# ;illingness to 3#ovide additional in"o#&ation

Potential Errors Eeing ove#ly o3ti&isti+ a.out any o33o#tunity Not 3#o.ing dee3ly "o# hidden "a+ts a.out situation Not #eading the ve#.al and nonve#.al +ues the .#oke# is giving you-

%e+iding :hethe# to ,uote

Evaluating ti&e "#a&e .et:een ,uote deadline and e""e+tive date Assessing i" a++ount &eets unde#:#iting guidelines

Too &u+h ti&e signals the .#oke# is Asho33ingC "o# an ea#ly ,uote Too little ti&e signals that .#oke# =ust :ants to kee3 +u##ent +a##ie# AhonestC ARed "lagC e?3osu#es that :e +annot :#ite AGoC +lasses o" .usiness that :e a#e ta#geting

Eeing so e?+ited a.out the o33o#tunity that you #ush to =udg&ent S3in :heels on a++ounts :he#e the#e isn>t a t#ue o33o#tunity %on>t dig dee3ly enough into :hat the a++ount #eally does o# did in the 3ast that +ould #e3#esent AhiddenC e?3osu#es

Cognitive Task Analysis

Table &. Cognitive %e&ands Ta.leDifficult cognitive elements Assessing :hethe# .#oke#>s ans:e#s &ake sense o# not :ith :hat is in the su.&ission Conside#ing the A#ealC o33o#tunity and e?3osu#es .eyond the o.vious in"o#&ation given in the su.&ission Co&3a#ing a++ount>s e?3osu#e in"o#&ation :ith unde#:#iting guidelines Why difficult Novi+e unde#:#ite#s tend to "o+us on .asi+ "a+ts in the su.&ission ve#sus :hat the .#oke# is telling the& E#oke#s #elu+tant to volunta#ily ai# di#ty laund#y a.out a++ount Novi+es unde#:#ite#s tend to "o+us on in"o#&ation given ve#sus in"o#&ation needed to &ake de+ision Can .e un+o&"o#ta.le situation "o# novi+e unde#:#ite#s to 3#o.e "o# ans:e#s Co&3anies o"ten have &any ty3es o" o3e#ations :hi+h +#oss seve#al +lasses o" .usiness Novi+e unde#:#ite#s tend to "o+us on the 3#i&a#y .usiness o3e#ations Novi+e unde#:#ite#s o"ten have di""i+ult assigning an a++ount to the a33#o3#iate .usiness +lassi"i+ation :ithin the guidelines Novi+e unde#:#ite#s tend to "o+us on :hat .#oke# is asking you to do Novi+e unde#:#ite#s o"ten "ail to identi"y :ays to ad=ust ,uotation o3tions to &eet guidelines Common errors %on>t #e+ogniDe o# 3#o.e "o# hidden A#ed "lagsC 9o+us e?+lusively on in"o#&ation in su.&ission Taking the su.&ission at A"a+e valueC 9ailing to engage in un+o&"o#ta.le 3#o.ing +onve#sations :ith the .#oke# 9ailing to "ully +a3tu#e e?3osu#es Getting lost in the details $isinte#3#eting unde#:#iting data $isinte#3#eting the unde#:#iting guidelines Cues and strategies used Conside# i" you #eally kno: the sto#y .ehind the sto#y Get and kee3 the .#oke# talking to eli+it in"o#&ation .eyond the su.&ission Ask a.out #easons :hy a++ount :ould &ove Conside# :hethe# ti&e"#a&e to &ove a++ount is #ealisti+

Revie: a++ount :ith senio# unde#:#ite# Che+k &ulti3le sou#+es to evaluate e?3osu#es

Conside#ing and suggesting alte#natives

Iuote only :hat is asked .y .#oke# 9ailing to 3#o.e "o# alte#nate o33o#tunities :ith the .#oke#

Conside# :ays to ad=ust ,uotation o3tions to "it :ithin unde#:#iting guidelines Conside# othe# +ove#ages and li&its that you o# othe# de3a#t&ents +ould ,uote

Cognitive Task Analysis Co&3a#ison o" A33#oa+hes

Analysis Comparison In +o&3a#ing the #esults o" the t#aditional task analysis :ith the +ognitive task analysis0 signi"i+ant di""e#en+es e&e#ge in "ollo:ing a#eas: a8 the identi"i+ation and analysis o" hidden +ognitive 3#o+esses0 .8 the #elative level o" ela.o#ation #ega#ding the +ent#al task ele&ents0 +8 the "o+us on e?3e#t and novi+e di""e#en+es'vert behaviors versus cognitive processes. The key st#ength o" the t#aditional task analysis is the a.ility to e?a&ine ove#t .ehavio#s #e,ui#ed to +o&3lete a task- 6o:eve#0 as seen in this e?a&3le0 additional +#iti+al +ognitive 3#o+esses and a+tions :e#e un+ove#ed :ithin the ACTA- 9u#the#0 the ACTA o""e#ed a &eans o" analyDing the #elative signi"i+an+e and di""i+ulty o" the #e,ui#ed task ele&ents(evel of elaboration. The t#aditional task analysis identi"ied the #elevant 3#o+esses and de+ision 3oints in the su.&ission t#iage task- 6o:eve#0 .y "o+using on the di""i+ult +ognitive as3e+ts o" the task0 the ACTA 3#ovided g#eate# ela.o#ation :ith #ega#d to the kno:ledge and +ognitive 3#o+esses #e,ui#ed to 3e#"o#& the task- As the +ognitive de&ands ta.le highlights0 the ACTA "o+used attention on the di""i+ult +ognitive ele&ents0 +o&&on e##o#s0 and st#ategies to ove#+o&e those di""i+ulties and e##o#s- 'n"o#tunately0 these ele&ents :e#e not unea#thed :ithin the t#aditional task analysisFocus on expert and novice differences. 'nlike the t#aditional task analysis0 the ACTA analysis "o+used on the +ent#al di""e#en+es .et:een ho: an e?3e#t and a novi+e 3e#"o#& the su.&ission t#iage task- The #esult is a +o&3a#ison o" +u##ent state 4novi+es8 and desi#ed state 4e?3e#ts80 as :ell as st#ategies to take the novi+e to an e?3e#t level$mplications for )ractice T#aditional task analysis allo:s 3#a+titione#s to ta#get the in3uts0 +ent#al o3e#ations0 and de+ision 3oints involved in +a##ying out a task- ;hile this 3#ovides a good ove#vie: o" :hat ha33ens as the task is +a##ied out0 it does not 3#ovide the designe# :ith an unde#standing o" the nature o" the +ognitive 3#o+esses #e,ui#ed to +o&3lete the task- 9u#the#0 "ollo:ing a t#aditional task analysis0 the 3#a+titione# +annot gage the #elative i&3o#tan+e o" the va#ious tasks ele&ents o# :hi+h as3e+t4s8 o" the task a#e ha#de# "o# the novi+e- As seen in the #esults .et:een the t:o analyses0 the +ognitive task analysis 3#ovides 3#a+titione#s :ith a .ette# unde#standing o" the di""i+ult and +#iti+al +ognitive 3#o+esses0 as :ell as the and +ues and st#ategies0 :hi+h a#e +ent#al to su++ess"ul +o&3letion o" the task*hen to use Traditional Tas Analysis versus Cognitive Tas Analysis Eoth a t#aditional task analysis and +ognitive task analysis highlight key as3e+ts o" the task- 6o:eve#0 as seen in the t:o analyses a.ove0 ea+h 3#odu+es di""e#ent #esults- As noted0 the +ognitive task analysis o""e#s a .ette# analysis o" the +ent#al kno:ledge and de+ision &aking +ognitive 3#o+esses- Given that ea+h task is di""e#ent0 the "ollo:ing 3#ovides a +o&3a#ison o" :hi+h analysis is &o#e a33#o3#iate .ased on the deg#ee o" o.se#va.le .ehavio#s0 the deg#ee o" #e,ui#ed e?3e#tise0 and the #elative +ognitive di""i+ulty o" the task-

Cognitive Task Analysis

%egree of observable behaviors. The di""e#en+e in out+o&es .et:een the t:o a33#oa+hes is likely less signi"i+ant :hen the task involves 3#i&a#ily o.se#va.le .ehavio#s- 6o:eve#0 i" the task involves 3#i&a#ily &ental a+tions that #esult in less o.se#va.le .ehavio#s0 a +ognitive task analysis is the &o#e a33#o3#iate o3tionExpert versus novice differences. ;hen little task #elated e?3e#tise is #e,ui#ed to 3e#"o#& the task0 the #esults o" .oth analyses :ould likely .e si&ila#- 6o:eve#0 i" su++ess"ul +o&3letion o" the task #e,ui#es kno:ledge that a novi+e :ould not 3ossess0 a +ognitive task analysis allo:s the 3#a+titione# to un+ove# o# d#ill do:n on the di""i+ult +ognitive ele&ents- As noted0 these +ognitive ele&ents a#e less likely to .e ade,uately analyDed in a t#aditional task analysis+elative cognitive difficulty. ;hile a t#aditional task analysis 3#ovides a +o&3#ehensive outline o" the ste3s in the task0 it does not o""e# a relative assessment o" :hi+h ste3s a#e ha#de# o# &o#e +#iti+al to su++ess"ul +o&3letion- Instead0 ea+h ste3 in the task is +onside#ed e,ually6o:eve#0 as seen in the +ognitive de&ands ta.le0 so&e tasks hinge on a s&alle# nu&.e# o" +#iti+al o# di""i+ult ele&ents- The#e"o#e0 the ACTA is &o#e a33#o3#iate :hen su++ess"ul task out+o&es de3end u3on +ognitively di""i+ult =udg&ents o# de+ision-

Cognitive Task Analysis

12

Re"e#en+es !onassen0 %- 6-0 Tess&e#0 $-0 5 6annu&0 ;- 6- 417778- Tas analysis methods for instructional design- $ah:ah0 N-!-: L- E#l.au& Asso+iates$ilitello0 L- G-0 5 6utton0 R- !- E- 4177(8- A33lied +ognitive task analysis 4ACTA8: a 3#a+titione#>s toolkit "o# unde#standing +ognitive task de&ands- Ergonomics0 ,141180 1H1(J1H@1-