Center for

Early Care and Education Research

Dual Language Learners

RESEARCH BRIEF #9
Assessment Tools for Language and Literacy Development of
Young Dual Language Learners (DLLs)

I
N  THE  PAST  DECADE,  a  dramatic  increase  in  invest-­ PHDVXULQJZKDWLWLVVXSSRVHGWREHPHDVXULQJDQG
ments  in  early  childhood  programs  (Barnett  et  al.,   ZKDWW\SHVRILQIHUHQFHVFDQEHGUDZQIURPWKHUHVXOWV
2011)  has  been  accompanied  by  an  increase  in  the   %RWKUHOLDELOLW\DQGYDOLGLW\LQIRUPZKDWW\SHVRILQIHU-­
number  of  dual  language  learners  (DLLs)—chil-­ HQFHVFDQEHGUDZQIURPWKHUHVXOWV
GUHQIURPELUWKWRDJH¿YH\HDUVZKRDUHJURZLQJXS
Typically,  information  about  evidence  of  the  reliability  and  
LQKRPHVZKHUHODQJXDJHVRWKHUWKDQRULQDGGLWLRQWR
validity  of  particular  assessments  is  found  in  the  technical  
English  are  spoken  (Aud  et  al.,  2012;  Aikens  et  al.,  2011;  
manuals  that  accompany  them  (see  Table  1  for  examples).  
Vogel  et  al.,  2011).  For  example,  recent  reports  from  the  
2WKHUWKDQDIHZGHVLJQHGVSHFL¿FDOO\IRU'//VPRVWDV-­
Family  and  Child  Experiences  Survey  (FACES)  2009  
sessments  report  about  the  reliability  and  validity  evidence  
cohort,  the  ongoing  national  study  of  children  in  Head  
based  on  samples  that  are  more  representative  of  children  
Start,  indicate  that  more  than  31  percent  of  preschool-­
ZKRDUHPRQROLQJXDODQGPDNHXSWKHPDMRULW\RIWKH
HUVLQWKHSURJUDPOLYHLQKRPHVZKHUHDODQJXDJHRWKHU
standardization  samples.  Test  development  is  expensive,  
than  English  is  spoken  (Aikens  et  al.,  2011).  This  is  also  
and  publishers  usually  do  not  invest  in  seeking  evidence  of  
true  of  almost  one-­third  of  children  in  Early  Head  Start  
validity  for  subgroups  such  as  DLLs.
nationally  (Vogel  et  al.,  2011).  Reliable  and  valid  assess-­
ments  for  this  population  are  needed  for  evaluations  of   Assessments  are  validated  by  accumulating  evidence  in  
HDUO\FKLOGKRRGSURJUDPVDQGKRZZHOOWKH\DUHPHHW-­ relation  to  different  types  of  inferences.  An  assessment  
ing  the  needs  of  DLLs. may  be  valid  for  a  particular  purpose  or  representative  
of  the  skills  of  one  group  of  children  but  not  another.  
All  children  need  assessments  that  are  fair,  equitable,  
7KHNH\LVVXHLVZKHWKHUDQDVVHVVPHQWUHDOO\PHDVXUHV
DQGZHOOFRQVWUXFWHG7KHUHVXOWVVKRXOGUHSUHVHQW
ZKDWLWGHFODUHVWRPHDVXUHZKHQXVHGZLWKDJURXS
VNLOOVNQRZOHGJHDQGEHKDYLRUVRI'//VWKDWDUHLP-­
of  children  similar  to  those  in  the  study  and  for  that  
portant  for  their  development  and  later  school  success.  
particular  purpose  (for  example,  program  evaluation,  
Since  the  questions  and  probes  for  most  assessments  re-­
UHVHDUFKDERXWFKLOGUHQ¶VGHYHORSPHQWRULGHQWL¿FD-­
quire  the  use  of  a  single  language,  either  by  the  assessor  
tion  of  a  child’s  need  for  more  intensive  instruction  or  
or  the  child,  the  results  from  an  assessment  of  a  child  
specialized  intervention).  
ZKRLVOHDUQLQJWZRODQJXDJHVPD\QRWUHSUHVHQWIXOO\
KLVRUKHUNQRZOHGJHDQGVNLOOV Different  types  of  evidence  can  be  collected  to  support  
interpretation  of  an  assessment  and  the  results  ob-­
Examination  of  reliability  and  validity  evidence  helps  us  
tained  from  it.  The  normative  group  (the  sample  used  to  
VHOHFWPHDVXUHVE\SURYLGLQJLQIRUPDWLRQDERXWZKHQ
develop  comparison  scores  on  the  assessment)  is  a  key  
DQGZLWKZKRPWKHDVVHVVPHQWZLOOSURYLGHWUXVWZRUWK\
component.  An  assessment  may  be  valid  for  comparing  
and  meaningful  results.  Information  about  reliability  
FKLOGUHQIURPWKHVDPHOLQJXLVWLFJURXSEXWXQIDLUZKHQ
FDQWHOOXVKRZGHSHQGDEOHWKHUHVXOWVDUHZKLOHHYL-­
comparing  the  skills  of  children  across  linguistic  groups.  
GHQFHRIYDOLGLW\LQGLFDWHVZKHWKHUWKHDVVHVVPHQWLV
A  measure  may  include  items  that  function  in  differen   can  differ  by  sample  and  study.  Poor  reliability  can  limit  
ZD\VDFURVVOLQJXLVWLFJURXSVLQFOXGLQJGLIIHUHQFHVLQ the  ability  to  detect  change  over  time  or  associations  
GLI¿FXOW\GLVFULPLQDWLRQDQGRUIDFWRUORDGLQJV7KH EHWZHHQFRQVWUXFWVIRUH[DPSOHEHWZHHQYRFDEXODU\
abilities  of  one  group  may  not  be  validly  represented  by   DQGHDUO\OLWHUDF\²SDUWLFXODUO\ZKHQWKHVDPSOHVL]HLV
the  scores  derived  from  the  items,  redereing  cross-­group   VPDOO²DQGFDQOHDGWRÀDZHGFRQFOXVLRQV0DQ\VWXGLHV
FRPSDULVRQVXQIDLU,QDVLPLODUZD\DQDVVHVVPHQW LQRXUUHYLHZGLGQRWSURYLGHDQ\VWXG\VSHFL¿FHYL-­
may  not  be  valid  for  making  critical  decisions  about   dence  of  reliability  but  reported  only  the  published  evi-­
children  or  the  programs  serving  those  children.  For   GHQFHIURPWKHDVVHVVPHQWPDQXDO2XUUHYLHZVXJJHVWV
example,  an  English  vocabulary  or  language  assessment   that  more  information  is  needed  about  the  performance  
ZRXOGEHDYDOLGLQGLFDWRURIZKHWKHUDFKLOGLVOHDUQLQJ RIPHDVXUHVZLWK'//V
English,  but  using  standard  scores  based  on  children  
Internal Consistency
ZKRVSHDNRQO\(QJOLVKWRGHWHUPLQHLID'//KDVD
ODQJXDJHGLVDELOLW\ZRXOGQRW The  most  commonly  reported  indicator  of  reliability  is  
LQWHUQDOFRQVLVWHQF\²WKDWLVKRZFRQVLVWHQWO\WKHLWHPV
ZLWKLQDQDVVHVVPHQWPHD-­
Table 1.Common Types of Reliability and Validity Evidence sure  the  construct  of  inter-­
HVW+RZHYHUPRVWRIWKH
Term 'HÀQLWLRQ
SHHUUHYLHZHGVWXGLHVGLG
Internal  consistency     Indicates  whether  the  items  within  a  measure  are  measuring  
reliability the  same  underlying  concept QRWLQFOXGHVWXG\VSHFL¿F
Test–retest  reliability Indicates  whether  assessment  of  a  particular  construct  (what   estimates  (reporting  only  
is  being  measured)  would  result  in  the  same  score  if  repeat-­ the  published  reliability  
ed  later estimates,  if  at  all)  or  sepa-­
Differential  item  functioning Notes  whether  the  items  within  a  measure  function  in  the   rate  estimates  by  language  
same  way  for  different  groups  of  children
group.  Large-­scale  studies  
Validity Indicates  whether  an  assessment  is  measuring  what  it  pur-­
ports  to  measure  and  under  what  conditions;;  assessments   ZHUHPRUHOLNHO\WRLQFOXGH
can  be  valid  for  some  uses  and  not  for  others VDPSOHVSHFL¿FHVWLPDWHV
Predictive  validity Indicates  whether  the  assessment  is  related  in  expected   DQGKDGVXI¿FLHQWVDPSOH
ways  to  similar  or  related  outcomes  measured  at  a  later  time sizes  to  report  estimates  by  
subgroups.  When  reported,  
the  estimates  of  internal  
To  extend  the  available  information  about  assessments  
FRQVLVWHQF\LQWKHVWXGLHVUHYLHZHGZHUHJHQHUDOO\ID-­
XVHGZLWK'//VZHH[DPLQHGUHVHDUFKVWXGLHVSXE-­
YRUDEOHWKRXJKQRWDOZD\VZLWKLQDFFHSWDEOHUDQJHVIRU
lished  in  the  last  10  years  that  included  young  DLLs.  
Spanish  assessments.    For  example,  the  reported  reli-­
7KLVEULHIGUDZVIURPDPRUHGHWDLOHGUHSRUWH[DPLQLQJ
ability  estimates  for  the  Spanish  version  of  the  Story  and  
language  and  literacy  measures  used  in  seven  large-­
Print  Concepts  used  in  the  FACES  2000    and  FACES  
scale  government  studies  and  thirty  research  studies  
VWXGLHVZHUHPXFKORZHUWKDQWKHUHOLDELOLW\HVWL-­
conducted  in  the  past  decade  (Bandel  et  al.,  2012).1  This  
mates  for  the  English  version.
brief  highlights  key  issues  noted  in  the  report  and  em-­
SKDVL]HVTXHVWLRQVWRFRQVLGHUZKHQHYDOXDWLQJWKHDS-­ Different  factors  can  affect  the  reliability  of  measures  
SURSULDWHQHVVRIDVVHVVPHQWWRROVXVHGZLWK'//VDQG LQFOXGLQJKRZWDUJHWHGWKHLWHPVDUHWRWKHFRQVWUXFW
interpreting  research  that  involves  such  assessments.   being  measured  and  to  the  sample  of  children  taking  the  
DVVHVVPHQWKRZZHOOLWHPVGLVFULPLQDWHDPRQJFKLO-­
How reliable is the assessment? GUHQDQGWKHQXPEHURILWHPV8VXDOO\PHDVXUHVZLWK
Reliability  information  indicates  to  consumers  of  re-­ PRUHLWHPVZLOOGHPRQVWUDWHJUHDWHUUHOLDELOLW\WKRXJK
VHDUFKWKHWUXVWZRUWKLQHVVRIWKH¿QGLQJVDQGUHVXOWV ZHOOFRQVWUXFWHGDGDSWLYHDVVHVVPHQWVFDQDWWDLQKLJKO\

2
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
UHOLDEOHHVWLPDWHVZLWKDOLPLWHGQXPEHURILWHPV0DQ\ Evidence  of  stability  across  longer  periods  of  time  
RIWKHDVVHVVPHQWVZHUHDGDSWLYHDVVHVVPHQWVZLWK PRUHWKDQWKUHHPRQWKVZDVDYDLODEOHIRUVHYHUDO
items  targeted  to  the  children’s  developmental  levels.   PHDVXUHVRIYRFDEXODU\DQGOLWHUDF\LQWKHUHYLHZHG
+RZHYHUZKHQFRPSDUHGZLWKDQ(QJOLVKGRPLQDQW studies,  but  these  may  not  be  a  good  measure  of  stabil-­
sample,  there  may  be  greater  differences  in  experiences,   LW\SDUWLFXODUO\GXULQJWKHSUHVFKRRO\HDUVZKHQ'//V
languages,  and  dialects  among  DLLs  that  affect  their   are  likely  to  have  increased  exposure  to  English  via  ed-­
UHVSRQVHVWRLWHPVLQZD\VXQUHODWHGWRZKDWLVEHLQJ ucational  opportunities.  For  some  children,  preschool  
measured.  In  those  cases,  more  items  may  be  needed   PD\EHWKH¿UVWHGXFDWLRQDOH[SHULHQFHDQGWKH¿UVW
WRREWDLQUHOLDEOHHVWLPDWHV7KRXJKRIWHQZHDNHUIRU exposure  for  some  DLLs  to  English.    This  makes  it  very  
Spanish  assessments,  reported  internal  consistency   GLI¿FXOWWRGLVHQWDQJOHGLIIHUHQWLDWHWKHVWDELOLW\RIWKH
HVWLPDWHVZHUHXVXDOO\ZLWKLQDFFHSWDEOHUDQJHVZKHQ assessment  from  the  responsiveness  of  different  chil-­
DJUHDWHUQXPEHURILWHPVZDVDGPLQLVWHUHG7KH dren  to  the  educational  opportunities  and  the  sensitiv-­
ZHDNHUHVWLPDWHVRIUHOLDELOLW\RQVRPH6SDQLVKDV-­ LW\RIWKHPHDVXUHWRLQWHUYHQWLRQ&KLOGUHQZKRDUH
sessments  relative  to  the  English  assessments  may  be   OHDUQLQJ(QJOLVKIRUWKH¿UVWWLPHPLJKWEHH[SHFWHG
due  in  part  to  the  number  of  items  administered.  For   WRVKRZJUHDWHUFKDQJHLQWKHLUNQRZOHGJHRI(QJOLVK
example,  in  FACES  2009,  the  reliability  estimates  for   ZRUGVWKDQFKLOGUHQZKRKDYHEHHQOHDUQLQJLWIRUDQ
WKH6SDQLVKOLWHUDF\DVVHVVPHQWVLQWKHIDOOZHUHZHDN extended  period  of  time.  Thus,  the  stability  estimates  
OHVVWKDQEXWWKH\ZHUHEDVHGRQDQDYHUDJHRI RYHUDORQJSHULRGRIWLPHZRXOGEHORZHUWKDQRYHUD
only  14  administered  items,  compared  to  the  average   WZRZHHNSHULRGSDUWLFXODUO\ZKHQVDPSOHVKDYHERWK
of  17–26  items  administered  to  the  children  (including   simultaneous  and  sequential  DLLs.  With  that  caveat  
VRPH'//VWDNLQJWKHDVVHVVPHQWVLQ(QJOLVKZKLFK in  mind,  examination  of  correlations  suggested  that  
had  stronger  internal  consistency  estimates  (about  .80).   literacy  measures  might  be  less  stable  than  vocabulary  
Because  many  assessments  used  in  early  childhood   PHDVXUHV'LFNLQVRQ0F&DEH&ODUN&KLDUHOOL :ROI
research  are  adaptive  measures—that  is,  they  present   +DPPHU'DYLVRQ/DZUHQFH 0LFFLR
LWHPVRILQFUHDVLQJGLI¿FXOW\XQWLOWKHFKLOGLQFRUUHFWO\ DOWKRXJKWKLVZDVQRWDOZD\VWKHFDVH$QWKRQ\HWDO
UHVSRQGVWRDVSHFL¿HGQXPEHURILWHPV²WKHQXPEHU +RZHYHUWKHVDPSOHVDQGWKHOHQJWKRIWLPH
of  items  administered  differs  according  to  the  child’s   EHWZHHQDVVHVVPHQWVYDULHGPDNLQJLWPRUHGLI¿FXOWWR
responses  and  the  stop  rules  of  the  assessment  (this  is   GUDZYDOLGLQIHUHQFHVIURPWKHHVWLPDWHV
HODERUDWHGRQEHORZ0RUHUHVHDUFKLVQHHGHGVSHFL¿-­
cally  about  the  validity  of  stop  rules  themselves  for  dif-­ How fair is the assessment?
IHULQJSRSXODWLRQV²LQSDUWLFXODUKRZUHOLDEOHDQGYDOLG Bias  from  a  number  of  sources  may  render  an  assess-­
WKH\DUHZKHQDGPLQLVWHULQJDVVHVVPHQWVWR'//V ment  unfair  or  invalid  for  a  particular  group  of  children.  
7KHTXHVWLRQVRUSUREHVPD\XVHIRUPDWVRUZRUGVWKDW
Stability
are  unfamiliar  in  some  cultures  or  the  questions  may  
The  stability  of  a  score,  or  test–retest  reliability,  indi-­ VDPSOHNQRZOHGJHWKDWLVVSHFL¿FWRDFHUWDLQFXOWXUH
FDWHVZKHWKHUDQDVVHVVPHQWRIDSDUWLFXODUFRQVWUXFW or  linguistic  group.  Under  these  conditions,  the  items  
ZRXOGUHVXOWLQWKHVDPHVFRUHLIUHSHDWHGDZHHNODWHU ZRXOGQRWEHUHSUHVHQWDWLYHRIWKHH[SHULHQFHVDQG
or,  for  more  stable  constructs,  months  or  years  later.   NQRZOHGJHRIVRPHRIWKHFKLOGUHQDVVHVVHG,QVKRUW
$PRQJWKHVWXGLHVZHUHYLHZHGWHVW±UHWHVWUHOLDELOLW\ the  evidence  suggests  that  some  approaches  to  assess-­
ZLWKLQDIRXUZHHNWLPHSHULRGZDVUHSRUWHGIRURQO\ ing  young  DLLs  may  not  result  in  a  fair  representation  
one  bilingual  measure,  the  Bilingual  English  Spanish   RIWKHFKLOGUHQ¶VNQRZOHGJH)RUH[DPSOHDVVHVVLQJWKH
Oral  Language  Screener  (BESOS;  Peña,  Bedore,  Gutier-­ FRQFHSWXDOYRFDEXODU\RID'//²WKDWLVZKHWKHUDFKLOG
UH]&OHOOHQ,JOHVLDV *ROGVWHLQLQSUHSDUDWLRQZLWK KDVZRUGVIRUGLIIHUHQWREMHFWVDFWLRQVDQGFRQFHSWV²
slightly  stronger  reliability  indicated  for  the  Spanish   in  a  single  language  is  likely  to  under-­represent  the  
than  for  the  English  version.  

3
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
ZRUGVWKHFKLOGNQRZV&KLOGUHQW\SLFDOO\DFTXLUHZRUGV UHVHDUFKVWXGLHVLQFOXGHGFKLOGUHQZKRVSRNHSULPDULO\
IRUREMHFWVDQGDFWLYLWLHVH[SHULHQFHGDWKRPHLQWKHODQ-­ (QJOLVKDQGFKLOGUHQZKRVSRNHSULPDULO\6SDQLVKRU
JXDJHXVHGPRVWRIWHQDWKRPHDQGZRUGVIRUDFDGHPLF another  language.  Although  74  percent  of  the  studies  
concepts  in  the  language  used  in  school  (Bialystok  et   assessed  children  in  both  languages,  only  one  (Bialys-­
al.  2010).  Using  one  of  the  most  commonly  applied   tok,  Luk,  Peets,  &  Yang,  2010)  reported  examination  of  
measures  of  English  vocabulary  (PPVT-­III),  Bialystok   KRZLWHPVIXQFWLRQHGIRUGLIIHUHQWJURXSV3DUWLFXODUO\
and  colleagues  (2010)  noted  that  test  items  referring   ZKHQUHVHDUFKHUVWUDQVODWHLWHPVRUGHYHORSWKHLURZQ
WRKRPHREMHFWVDQGDFWLYLWLHVZHUHPRUHGLI¿FXOWIRU assessments,  attention  is  needed  to  assure  that  the  as-­
Spanish-­dominant  children  than  for  English-­dominant   sessments  are  fair  representations  of  children’s  skills,  
FKLOGUHQZKLOHVFKRROUHODWHGZRUGVSRVHGVLPLODUGLI-­ NQRZOHGJHDQGDELOLW\7HVWGHYHORSHUVDQGUHVHDUFKHUV
¿FXOW\DFURVVJURXSV should  evaluate  the  items  in  tests  to  determine  if  they  
have  the  same  meaning  across  groups—that  is,  indicat-­
Basal and Ceiling Rules
LQJZKHWKHUWKHLWHPVIXQFWLRQLQWKHVDPHZD\IRUGLI-­
Implications  of  this  phenomenon  may  go  beyond  the   ferent  groups  of  children.
FRUUHFWQHVVRILQGLYLGXDOLWHPVZKHQDVVHVVPHQWVRI
Parent and Teacher Reports
young  children  are  adaptive,  using  basal  and  ceiling  
(or  start  and  stop)  rules.    The  items  in  adaptive  assess-­ Some  studies,  particularly  for  children  younger  than  
ments,  as  mentioned  earlier,  are  generally  ordered  in   WZR\HDUVXVHGDVVHVVPHQWVEDVHGRQSDUHQWRUWHDFKHU
WHUPVRIGLI¿FXOW\EDVHGRQWKHUHVSRQVHVRIWKHQRUPD-­ reports.  Early  childhood  researchers  often  rely  more  on  
tive  sample,  and  the  stop  rules  are  designed  so  children   WHDFKHUUHSRUWVWKDQSDUHQWUHSRUWVGXHWRHI¿FLHQFLHV
ZLOOKDYHDYHU\ORZSUREDELOLW\RIJHWWLQJDQ\LWHPV LQGDWDFROOHFWLRQDQGSRWHQWLDOV\VWHPDWLFELDVZLWKSDU-­
beyond  the  stopping  point  correct;  the  scoring  assumes   HQWUHSRUWLQJ3DQ5RZH6SLHU7DPLV/H0RQGD
responses  to  all  subsequent  items  are  incorrect.  Because   Rescorla,  Ratner,  &  Jusczyk,  &  Jusczyk,  2005;  Reese  &  
LWHPVFRQFHUQLQJWKHQDPHVRIREMHFWVIRXQGLQWKH Read,  2000;  Roberts  Burchinal,  &  Durham,  1999).  With  
home  are  easier  for  young  children  to  correctly  name,   English  monolingual  samples,  correlations  of  teacher  
they  are  presented  early  in  assessments  of  vocabulary.   UHSRUWZLWKGLUHFWDVVHVVPHQWVUDQJHIURPPRGHUDWHWR
&KLOGUHQZKRDUH'//VKRZHYHUPD\NQRZWKH(QJOLVK high  during  the  elementary  school  years,  but  suggest  
QDPHVIRUREMHFWVDQGDFWLYLWLHVUHODWHGWRVFKRRODQG SRWHQWLDOELDVHVZLWKWHDFKHUUHSRUWVSDUWLFXODUO\IRU
DFDGHPLFVEXWQRWIRUWKRVHDVVRFLDWHGZLWKWKHKRPH LWHPVWKDWKDYHKLJKHULQIHUHQFHOHYHOV3HUU\ 0HLVHOV
Using  the  published  ceiling  rules  could,  therefore,  result   1996).  Less  research  is  available  for  the  infant-­toddler  
in  underestimates  of  their  English  vocabulary.  None  of   \HDUVDQGWKHUHVHDUFKZLWK'//VLVSDUWLFXODUO\OLPLWHG
WKHUHYLHZHGVWXGLHVH[DPLQHGWKHDSSURSULDWHQHVVRI LQWKLVUHJDUG+RZHYHUDVWXG\E\9DJKDQGFROOHDJXHV
basal  and  ceiling  rules  for  young  DLLs.   9DJK3DQ 0DQFLOOD0DUWLQH]VXJJHVWVWKDW
parents  (rather  than  teachers)  may  provide  the  most  
'LIIHUHQWLDO,WHP'LIÀFXOW\
valid  information  about  DLLs’  vocabulary  development.
Greater  attention  is  needed  for  evaluating  the  fairness   $PRQJ'//VVWURQJHUUHODWLRQVKLSVZHUHIRXQGEH-­
of  individual  items  and  tasks  across  subgroups.  Some   WZHHQSDUHQWUHSRUWVRIFKLOGUHQ¶VYRFDEXODU\DQGGLUHFW
tasks  may  not  have  the  same  meaning  in  development   DVVHVVPHQWRIYRFDEXODU\WKDQEHWZHHQWHDFKHUUHSRUWV
DFURVVVXEJURXSVRUWKHGLI¿FXOW\PD\YDU\DFURVV of  those  children’s  vocabulary  and  direct  assessment  of  
languages.  For  example,  rhyming  is  usually  easier  for   WKHLUVNLOOV9DJK3DQ 0DQFLOOD0DUWLQH]
\RXQJFKLOGUHQZKHQWKHZRUGVKDYHRQO\RQHV\OODEOH
Conceptual Scoring
but  English  has  many  more  one-­syllable  rhymes  than  
6SDQLVK/LPLWHGHYLGHQFHZDVSURYLGHGLQWKHUHYLHZHG Other  studies  used  measures  that  are  conceptually  
studies  for  the  congruence  of  estimates  of  item-­  dif-­ VFRUHG&RQFHSWXDOVFRULQJDOORZVSUREHVRUUHVSRQVHV
¿FXOW\DFURVVODQJXDJHVWKRXJKPDQ\RIWKHUHYLHZHG LQPRUHWKDQRQHODQJXDJHDQGZLOOXVXDOO\SUHVHQW

4
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
DPRUHYDOLGDVVHVVPHQWZKHQPHDVXULQJFKLOGUHQ¶V WKHTXHVWLRQWKH\ZRXOGGRSRRUO\RQWKRVHLWHPVDW
NQRZOHGJHRIFRQFHSWVUDWKHUWKDQYRFDEXODU\LQD EDVHOLQHEXWGRZHOORQFHWKH\XQGHUVWDQGWKHTXHVWLRQ
SDUWLFXODUODQJXDJH)RUUHFHSWLYHYRFDEXODU\KRZ-­ LQ(QJOLVK7KHFKDQJHZDVLQWKHLUXQGHUVWDQGLQJRI
ever,  comparisons  across  linguistic  groups  can  be  more   English  rather  than  their  understanding  of  seriation.  In  
FKDOOHQJLQJ,I\RXSUHVHQWIRXUSLFWXUHVRIREMHFWVDQG DQRWKHUSURJUDPFKLOGUHQPD\HQWHUZLWKRXWXQGHU-­
the  name  of  one  of  them  in  one  language  (giving  a  1  in   VWDQGLQJRIVHULDWLRQDQGVRZRXOGQHHGWROHDUQERWK
4  chance  of  selecting  the  correct  picture)  and  the  child   WKH(QJOLVKDQGVHULDWLRQ$WWKHHQGRIWKH\HDUZKHQ
LVXQVXFFHVVIXOWKHQIROORZZLWKDSURPSWLQWKHRWKHU children  in  both  groups  successfully  responded  to  the  
ODQJXDJHQRZJLYLQJDLQFKDQFHRIVXFFHVVWKH seriation  tasks,  the  program  that  taught  the  children  
SUREDELOLW\RIVXFFHVVIDYRUVFKLOGUHQZKRNQRZWZR the  English,  but  did  not  extend  mathematical  thinking,  
ODQJXDJHVRYHUFKLOGUHQZKRNQRZRQO\RQH7KLVLVQRW ZRXOGORRNDVWKRXJKLWKDGDJUHDWHUHIIHFWRQFRJQLWLYH
a  problem  for  expressive  vocabulary  because  a  child   development  than  it  actually  did.    
FRXOGGUDZXSRQWKRXVDQGVRIGLIIHUHQWZRUGVWRQDPH
a  picture.   What Levels of Performance and
Progress Are Expected for DLLs
It  is  important  that  researchers  consider  all  sources  of   on Different Assessments?
SRWHQWLDOELDVZKHQSUHVHQWLQJDQGLQWHUSUHWLQJWKHLU
0RUHLQIRUPDWLRQLVQHHGHGDERXWH[SHFWHGSHUIRU-­
¿QGLQJVDQGZKHQIHDVLEOHFRQWULEXWHWRWKHNQRZO-­
mance  of  DLLs  on  assessments.  To  provide  information  
HGJHEDVHDERXWYDOLGLW\RIPHDVXUHVZLWK'//V
about  their  expected  performance  on  language  mea-­
Does the Assessment sures,  developers  should  provide  supplemental  norms  
Answer the Questions of Interest? for  DLLs  or  estimates  of  the  mean  and  standard  devia-­
WLRQIRUWKHVXEVDPSOHRI'//V0DQ\VDPSOHVRI'//V
If  assessments  of  DLLs  are  to  be  valid  for  the  goals  of  
LQWKHVWXGLHVZHUHYLHZHGZHUHUHODWLYHO\VPDOODQG
the  research,  care  must  be  taken  to  select  instruments  
often  the  only  norms  the  researchers  had  available  for  
DQGPHWKRGVWKDWPDWFKWKHTXHVWLRQWREHDQVZHUHG
LQWHUSUHWLQJWKHVFRUHVZHUHEDVHGRQVWDQGDUGL]DWLRQ
)RULQVWDQFHLWLVLPSRUWDQWWRFRQVLGHUZKHWKHUWKHUH-­
VDPSOHVLQZKLFKPRVWRIWKHFKLOGUHQZHUHPRQROLQ-­
VHDUFKTXHVWLRQUHTXLUHVPHDVXUHPHQWRIVNLOOVNQRZO-­
gual  in  either  English  or  Spanish.  
edge,  or  behavior  at  a  single  point  in  time  or  if  the  focus  
LVRQFKDQJHSDUWLFXODUO\LIWKHFKDQJHPD\UHÀHFWWKH 0RVWUHVHDUFKVWXGLHVFRQGXFWHGZLWK'//VLQWKH
effectiveness  of  a  program  or  intervention.  In  the  latter   United  States  in  the  last  10  years  included  primarily  or  
case,  researchers  should  consider  the  language  used  in   RQO\ORZLQFRPH6SDQLVK±(QJOLVK'//VOLPLWLQJWKH
the  program  or  intervention  and  the  possibility  that  they   JHQHUDOL]DELOLW\RIWKH¿QGLQJVWRRWKHUJURXSVRI'//V
ZLOOQHHGWRDVVHVVWKHFRQVWUXFWVRILQWHUHVWDWEDVHOLQH 1DWLRQDOO\KRZHYHUWKHPDMRULW\RI\RXQJ'//VUHVLGH
in  both  the  child’s  home  language  and  the  language  used   LQKRPHVZLWKOLPLWHGLQFRPHDQGKDYH6SDQLVKDVD
in  the  program  to  have  a  valid  estimate  of  the  program’s   home  language  (Shin  and  Kominski,  2010).
effects.  Without  an  assessment  in  the  home  language,  
,WZRXOGEHKHOSIXOLIUHVHDUFKHUVSDUWLFXODUO\IRUODUJH
FKLOGUHQPD\DSSHDUWRODFNVNLOOVDWEDVHOLQHDQGVKRZ
scale  studies)  provided  separate  information  about  
JDLQVLQPDQ\VNLOOVZKHQDVVHVVHGDWDODWHUSRLQWLQ
the  mean  scores  and  study  characteristics  of  their  DLL  
WLPHZKHQWKHRQO\NQRZOHGJHWKDWWKH\DFTXLUHGZDV
samples  (including  socioeconomic  background,  range  of  
comprehension  of  the  questions  in  English.  A  program  
dialects  represented,  and  age  range)  and  their  skills.  
ZLWKPDQ\'//VFRXOGDSSHDUWREHPDNLQJJUHDWHU
JDLQVWKDQDSURJUDPLQZKLFKFKLOGUHQHQWHUZLWKRXW
any  skills  at  all.  For  example,  if  the  children  already  pos-­
sess  understanding  of  a  concept  such  as  seriation  (for  
example,  tall,  taller,  and  tallest),    but  cannot  understand  

5
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
Vocabulary Assessments Used with DLLs in

Reviewed Studies
Assessment Abrieviations Title, Author, Date
English  Vocabulary
MacArthur-­Bates  Communicative  Development  Inventories  (Fenson  et  al.,  
  CDI
1993)
  EOWPVT Expressive  One-­Word  Picture  Vocabulary  Test  (Brownell,  2000a)
  ROWPVT Receptive  One-­Word  Picture  Vocabulary  Test  (Brownell,  2000b)
  PPVT
    PPVT–R Peabody  Picture  Vocabulary  Test–Revised  (Dunn  &  Dunn,  1981)
    PPVT–III Peabody  Picture  Vocabulary  Test–III  (Dunn  &  Dunn,  1997)
    PPVT–4 Peabody  Picture  Vocabulary  Test–4  (Dunn  &  Dunn,  2007
:RRGFRFN/DQJXDJH3UR¿FLHQF\%DWWHU\±5HYLVHG(QJOLVK)RUP:RRGFRFN
  WJ–III  (Picture  Vocabulary)
1995)
Spanish  Vocabulary
El  Inventario  del  Desarrollo  de  Habilidades  Comunicativas  (Jackson-­Maldona-­
  Inventario  (CDI  Spaish  Edition)
do,  Thal,  Marchman,  Newton,  Fenson,  &  Conboy,  2003)
  TVIP Test  de  Vocabulario  en  Imágenes  Peabody  (Dunn,  Padilla,  Lugo,  &  Dunn,  1986)
  WM–II  (Vocabulario  Sobre   :RRGFRFN/DQJXDJH3UR¿FLHQF\%DWWHU\±5HYLVHG6SDQLVK)RUP:RRGFRFN 
Dibujos) Muñoz-­Sandoval,  1995)  
Conceptuall  Scored  (English  &  Spanish)
Expressive  One-­Word  Picture  Vocabulary  Test:  Spanish  Bilingual  Edition  
  EOWPVT-­SBE
(Brownell,  2001a)
Receptive  One-­Word  Picture  Vocabulary  Test  :  Spanish  Bilingual  Edition  
  ROWPVT-­SBE
(Brownell,  2001b)
  SEVC Spanish–English  Vocabulary  Checklist  (Patterson,  1998)
Chiese  Vocabulary
  PPVT–R  Chinese Peabody  Picture  Vocabulary  Test—Revised:  Chinese  (Lu  &  Liu,  1998)

What Does the Score Mean and

The  baseline  information  in  FACES  2009  (Aikens  et  al.,  
How Can We Interpret the Results?
2011),  as  one  example,  provides  potentially  different  
6WXG\UHVXOWVVKRXOGEHLQWHUSUHWHGDQGGLVFXVVHGZLWK LQWHUSUHWDWLRQVRI'//V¶NQRZOHGJHRIYRFDEXODU\DQG
reference  to  the  characteristics  of  the  children  in  the   concepts  depending  on  the  dimensions  assessed  and  the  
study.  When  comparing  differences  in  the  performance   norm  group  used  for  deriving  standard  scores.  FACES  
and  progress  of  groups  of  children,  information  about   2009  assessed  vocabulary—the  area  most  commonly  as-­
differences  in  opportunity  to  learn  (due  to  different  so-­ sessed  among  young  DLLs  (see  Table  2)—using  multiple  
cioeconomic,  cultural,  or  linguistic  backgrounds)  should   measures  and  provided  standard  scores  for  4-­year-­old  
be  included.  When  discussing  standard  scores,  research-­ DLLs  based  on  the  PPVT-­4  (receptive  English  vocabu-­
ers  should  help  readers  understand  the  similarities  and   lary),  the  English  version  of  the  EOWPVT  (expressive  
GLIIHUHQFHVEHWZHHQWKHQRUPDWLYHJURXSDQGWKHVWXG\ vocabulary),  and  the  Spanish  bilingual  version  (EOW-­
sample.  Even  for  the  same  sample  of  children,  mean   PVT-­SBE),  in  addition  to  scores  on  the  TVIP,  a  Spanish  
standard  scores  vary  across  different  measures  of  the   receptive  vocabulary  measure.  Although  mean  scores  
same  construct.  

6
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
IRUWKHSUHGRPLQDQWO\(QJOLVKVSHDNLQJ'//VZHUHLQD (QJOLVKRQO\VSHDNHUV7REHWWHUXQGHUVWDQGKRZWRVXS-­
similar  range  across  the  English  assessments,  the  scores   SRUW'//VDQGPRQLWRUSURJUHVVWRZDUGVFKRROUHDGLQHVV
IRUWKH6SDQLVKVSHDNLQJ'//VYDULHGPRUHZLGHO\RQWKH VHSDUDWHYDOLGLW\DQDO\VHVZLWK'//VDUHQHHGHGDVZHOO
(QJOLVKDQGELOLQJXDODVVHVVPHQWVZLWKPHDQVWDQGDUG DVPRUHLQIRUPDWLRQDERXWKRZWKH6SDQLVKYHUVLRQVRI
scores  of  56,  67,  and  86  on  the  PPVT-­4,  and  the  English   DVVHVVPHQWVLQIRUPRXUXQGHUVWDQGLQJRIKRZFKLOGUHQ
and  bilingual  norms  for  the  EOWPVT-­SBE,  respectively.   DUHSURJUHVVLQJWRZDUGVXFFHVVLQVFKRRO
6WDQGDUGVFRUHVIRU6SDQLVKVSHDNLQJ'//VZHUHPRUH
1RQHRIWKHUHYLHZHGVWXGLHVSURYLGHGDQ\HVWLPDWHVRI
VLPLODUEHWZHHQWKH79,3DQGWKH(2:3976%(
concurrent  or  predictive  validity  separately  for  monolin-­
DQGUHVSHFWLYHO\7KHVHODWWHUWZRPHDVXUHVKDYH
standard  scores  based  on  Spanish-­speaking  or  bilingual  
VDPSOHVRIFKLOGUHQZLWKKDOIRUPRUHRIWKHVDPSOHIURP
KRPHVZLWKOLPLWHGPDWHUQDOHGXFDWLRQ7KHVHGLIIHUHQF-­
HVDQGVLPLODULWLHVKLJKOLJKWWKHGLI¿FXOW\LQLQWHUSUHWLQJ
FKLOGUHQ¶VVNLOOVZLWKRXWLQIRUPDWLRQDERXWWKHDVVHVV-­
ments  and  the  normative  samples  used  to  generate  the  
standard  scores.
Do the Assessments Measure What They
Are Supposed to Measure?
The  studies  provided  some  additional  evidence  of  valid-­
LW\²WKDWLVWKDWWKHDVVHVVPHQWVPHDVXUHGZKDWWKH\
ZHUHVXSSRVHGWRPHDVXUH7KHW\SHRIHYLGHQFHPRVW
IUHTXHQWO\LGHQWL¿HGIRUODQJXDJHDQGOLWHUDF\DVVHVV-­
PHQWVRI'//VZDVDQDVVRFLDWLRQZLWKFKLOGUHQ¶VDJH
or  exposure  to  English,  including  moderate  correlations   of  them  had  Spanish  as  a  home  language.  One  study  of
RIYRFDEXODU\DQGODQJXDJHDVVHVVPHQWVZLWKDJHDQG
SDUHQWUHSRUWHGH[SRVXUHWR(QJOLVKDVZHOODVHYLGHQFH
of  an  increase  in  assessment  scores  across  time.   sample  and  did  not  estimate  differences  by  language
gual  and  DLL  children.  The  sample  sizes  of  the  peer-­re-­
YLHZHGUHVHDUFKVWXGLHVZHUHRIWHQWRRVPDOOIRUVHSDUDWH
subgroup  analyses  of  validity.  Other  than  the  large-­scale  
government  studies,  only  three  of  the  research  studies  
had  sample  sizes  greater  than  200.  Predictive  validity  
HYLGHQFHLQWKHVHVWXGLHVXVXDOO\H[DPLQHGZKHWKHUFKLO-­
GUHQ¶VVFRUHVLQFUHDVHGDFURVVWLPHSRLQWVDQGZKHWKHU
earlier  vocabulary  and  language  assessments  predicted  
later  literacy.    
The  studies  that  provided  information  about  the  predic-­
tive  validity  of  the  measures  in  relation  to  school  readi-­
QHVVGLGQRWH[DPLQH¿QGLQJVIRUPHDVXUHVVHSDUDWHO\E\
subgroup.  The  report  for  FACES  1997  (Zill  et  al.,  2003)  
RQO\LQFOXGHGFKLOGUHQZKRWRRNWKH(QJOLVKDVVHVVPHQW
DWHDFKWLPHSRLQWDQGGLGQRWFOHDUO\LQGLFDWHKRZPDQ\
 
DLLs  reported  analysis  of  predictive  relationships  from  
SUHVFKRROWR¿UVWJUDGHUHDGLQJLQ(QJOLVKZLWKWKHIXOO
 
SUR¿FLHQF\5LQDOGL 3iH]:LWKWKHIXOOVDPSOH
6LPLODUWRUHOLDELOLW\FRHI¿FLHQWVWKHUDQJHRIUHSRUWHG
WKDWVWXG\LQGLFDWHGZHDNUHODWLRQVKLSVIRULQGLYLGXDO
FRQFXUUHQWYDOLGLW\FRHI¿FLHQWVZDVZHDNHUZKHQFRP-­
subtests,  although  a  combination  of  several  subtests  
SDUHGWRFRHI¿FLHQWVIRXQGDFURVVPHDVXUHVLQVWXGLHVRI
across  both  Spanish  and  English  increased  the  amount  of  
young  monolingual  English  samples.4    They  ranged  from  
explained  variance  in  English  reading.  Hammer  and  col-­
ORZPRGHUDWHWRVWURQJZLWKVWURQJHUHVWLPDWHVEHWZHHQ
leagues  (2007)  used  a  combination  of  language  measures  
YRFDEXODU\DQGODQJXDJHPHDVXUHVDQGZHDNHUHVWLPDWHV
WRH[DPLQHWKHUHODWLRQVKLSEHWZHHQSUHVFKRROODQJXDJH
EHWZHHQYRFDEXODU\DQGOLWHUDF\PHDVXUHV
development  and  spring  kindergarten  reading,  compar-­
LQJWKHVNLOOVRI'//VZKROHDUQHG(QJOLVKDWKRPHSULRU
Do the Assessments Support Understanding
of the School Readiness of DLLs?
9HU\OLPLWHGHYLGHQFHZDVDYDLODEOHIRUWKHSUHGLFWLYH
YDOLGLW\RIHDUO\PHDVXUHVIRUODWHURXWFRPHVZKHQXVHG
ZLWK'//V0RVWHYLGHQFHRIYDOLGLW\RIWKHPHDVXUHV
XVHGLQWKHUHYLHZHGVWXGLHVZDVIRXQGZLWKVDPSOHV
RIFKLOGUHQZKRZHUHDEOHWRWDNHDVVHVVPHQWVLQ(QJ-­
OLVKDQGLQWKRVHVWXGLHVWKH'//VZHUHFRPELQHGZLWK
WRVWDUWLQJSUHVFKRRO+HDG6WDUWWRWKRVHRI'//VZKR
ZHUHLQWURGXFHGWR(QJOLVKDW+HDG6WDUW$OWKRXJKWKH
analyses  provided  estimates  for  each  of  the  subgroups,  
the  evidence  cannot  be  attributed  to  a  single  measure,  but  
rather  to  the  component  measures.5    
0DQ\IDFWRUVFDQDIIHFWWKHVWUHQJWKRIYDOLGLW\FRHI-­
¿FLHQWVLQHDUO\FKLOGKRRG7\SLFDOO\WKHVWUHQJWKRIWKH

7
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
UHODWLRQVKLSEHWZHHQDVVHVVPHQWVZLOOGHSHQGRQPDQ\ Conclusions
different  factors,  including  the  reliability  of  the  assess-­
0RUHLQIRUPDWLRQLVQHHGHGDERXWRXUFXUUHQWDVVHVV-­
ment,  similarity  in  mode  of  assessment,  similarity  in  
PHQWVDQGZKDWLQIHUHQFHVFDQEHGUDZQIURPWKHLUUH-­
WKHGLPHQVLRQVDQGFRQVWUXFWVDVVHVVHGWLPHEHWZHHQ
VXOWV$OVRQHHGHGDUHDVVHVVPHQWVIRUXVHZLWK'//VZKR
assessments,  and  age  of  the  child  (the  younger  the  child  
speak  languages  other  than  English.  Researchers  should  
DW¿UVWDVVHVVPHQWWKHZHDNHUWKHUHODWLRQVKLS7KH
FRQVLGHUFDUHIXOO\ZKDWDVVHVVPHQWZLOODQVZHUWKHTXHV-­
predictive  validity  evidence  of  early  childhood  assess-­
tions  of  interest  and  report  more  information  about  the  
ments  among  English  monolingual  samples  typically  
samples  and  measures.  
GHPRQVWUDWHVORZWRPRGHUDWHSUHGLFWLYHFRUUHODWLRQ
FRHI¿FLHQWVZLWKOHVVWKDQSHUFHQWRIWKHRYHUDOOYDUL-­ ‡ Researchers  need  to  consider  if  assessments  are  valid  
ance  in  early  academic  performance  predicted  from  any   for  the  children  in  their  samples.  When  samples  
single  preschool  measure  (LaParo  &  Pianta,  2000).  When   include  children  from  multiple  linguistic  back-­
samples  include  DLLs,  the  number  of  additional  variables   JURXQGVZLOOWKHPHWKRGVDQGLWHPVIDLUO\UHSUHVHQW
WKDWFDQDIIHFWWKHVWUHQJWKRIWKHFRHI¿FLHQWLQFUHDVHV² DOOFKLOGUHQ¶VNQRZOHGJHVNLOOVDQGEHKDYLRUV"$UH
for  example,  the  age  of  introduction  to  the  language  used,   cultural  or  linguistic  biases  inherent  in  the  use  of  the  
the  amount  of  exposure  to  the  language  of  assessment,   DVVHVVPHQWZLWKDSDUWLFXODUJURXSRIFKLOGUHQ"(YHQ
and  intervention  or  preschool  experiences.  Of  the  stud-­ ZLWKLQDVLQJOHOLQJXLVWLFJURXSGLIIHUHQFHVLQGLDOHFW
LHVUHYLHZHGYRFDEXODU\ZDVDVVHVVHGPRVWRIWHQ\HW may  bias  results  unless  the  assessment  accounts  for  
HYLGHQFHRIDUHODWLRQVKLSWRVFKRRORXWFRPHVZDVIRXQG them.  
RQO\ZLWKODWHQWWUDLWVFRPELQLQJPXOWLSOHDVSHFWVRI
ODQJXDJH+DPPHU/DZUHQFHDQG0LFFLR5LQDOGL ‡ 5HVHDUFKHUVVKRXOGFRQVLGHUZKHWKHUWKHVHOHFWHG
DQG3iH]0RUHUHVHDUFKLVQHHGHGWRGHWHUPLQH DVVHVVPHQWVDUHELDVHGLQDQ\ZD\'RWKHWDVNVRU
KRZWRPRQLWRUWKHGHYHORSPHQWRI'//VWRHQVXUHODWHU items  require  similar  levels  of  skill  across  languages  
success  in  school.   DQGFXOWXUHV",VWKHWDVNHTXDOO\UHSUHVHQWDWLYHRI
VNLOOVDFURVVGLIIHUHQWJURXSV"

‡ 7HVWGHYHORSHUVLQFOXGLQJUHVHDUFKHUVZKRGHYHORS
forms  in  other  languages)  should  provide  evidence  
that  the  items  are  equivalent  for  children  from  differ-­
ent  groups,  that  is,  that  the  items  contribute  to  mea-­
VXUHPHQWRIWKHFRQVWUXFWLQWKHVDPHZD\VDFURVV
JURXSVDQGWKDWWKHGLI¿FXOW\RIHDFKRIWKHLWHPVRU
tasks  is  the  similar  across  groups.  

‡ 0RUHSUHGLFWLYHYDOLGLW\HYLGHQFHLVQHHGHGSDUWLFX-­
larly  for  infant-­toddler  measures.  While  this  is  true  
for  early  childhood  assessments  in  general,  measures  
and  evidence  of  predictive  validity  are  particularly  
sparse  for  young  DLLs.   ‡

8
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
(Endnotes)
References
1.   7KHUHYLHZLQFOXGHGSHHUUHYLHZHGMRXUQDODUWLFOHVSXE-­
$LNHQV1+XOVH\/.0RLGXGGLQ(.RSDFN$7DN\L
OLVKHGEHWZHHQDQGZLWKVDPSOHVIURPWKH
Laryea,  A.,  Tarullo,  L.,  &  West,  J.  (2011).  Data  tables  for  
United  States  or  its  territories  and  Canada  that  included  
FACES  2009  Head  Start  children,  families,  and  pro-­
at  least  one  direct  child  assessment  or  standardized  rating  
JUDPV3UHVHQWDQGSDVWGDWDIURP)$&(6UHSRUW$&)
of  the  language  or  literacy  development  of  DLL  children  
OPRE  2011-­33b).  U.S.  Department  of  Health  and  Human  
prior  to  age  6  or  kindergarten  entry.  To  examine  psycho-­
6HUYLFHV2I¿FHRI3ODQQLQJ5HVHDUFKDQG(YDOXDWLRQ
PHWULFSURSHUWLHVZHH[FOXGHGVWXGLHVZLWKOHVVWKDQ
Administration  for  Children  and  Families.  Washington,  
DLLs  and  those  that  only  analyzed  language  samples,  
'&86*RYHUQPHQW3ULQWLQJ2I¿FH
UHVXOWLQJLQSHHUUHYLHZHGUHVHDUFKDUWLFOHV6SDQLVK
DQG(QJOLVKZHUHWKHPRVWFRPPRQO\UHSRUWHGODQJXDJHV $QWKRQ\-/6RODUL(-:LOOLDPV-06FKRJHU.'
in  the  samples,  and  more  studies  focused  on  preschoolers   =KDQJ=%UDQXP0DUWLQ/ )UDQFLV'-
WKDQRQLQIDQWVDQGWRGGOHUV:HDOVRUHYLHZHGJRYHUQ-­ 'HYHORSPHQWRIELOLQJXDOSKRQRORJLFDODZDUHQHVVLQ
ment  reports  of  large-­scale  studies  of  early  childhood  that   6SDQLVKVSHDNLQJ(QJOLVKODQJXDJHOHDUQHUV7KHUROHV
included  at  least  one  direct  child  assessment  of  language   RIYRFDEXODU\OHWWHUNQRZOHGJHDQGSULRUSKRQRORJLFDO
DQGRUOLWHUDF\$PRQJJRYHUQPHQWUHSRUWVSXEOLVKHG DZDUHQHVV6FLHQWL¿F6WXGLHVRI5HDGLQJ(6),  535–564.  
LQWKHODVW\HDUVZHORFDWHGRQO\VHYHQODUJHVFDOH
national  studies  that  examined  children’s  language  or   $XG6+XVVDU:-RKQVRQ).HQD*5RWK(0DQ-­
literacy  development  prior  to  kindergarten  entry  and   ning,  E.,  …  Zhang,  J.  (2012).  7KHFRQGLWLRQRIHGXFDWLRQ
included  DLLs  in  any  of  these  assessments.  Description     (NCES  2012-­045).  U.S.  Department  of  Education,  
of  the  sample  characteristics  and  study  purpose  for  each   National  Center  for  Education  Statistics.  Washington,  
of  the  studies  can  be  found  in  the  full  report  (Bandel  et   '&86*RYHUQPHQW3ULQWLQJ2I¿FH
DOZKLFKLVDYDLODEOHRQWKH&(&(5'//ZHEVLWH
>KWWSFHFHUGOOISJXQFHGX@ Bacon,  D.  (2004).  The  contributions  of  reliability  and  pretests  
to  effective  assessment.  3UDFWLFDO$VVHVVPHQW5HVHDUFK
2.    Researchers  and  assessment  developers  often  require   (YDOXDWLRQ(3).  
that  assessment  tools  have  evidence  of  reliability  values  
of  0.70  or  higher  to  support  inferences  about  the  measure   Bandel,  E.,  Atkins-­Burnett,  S.,  Castro,  D.  C.,  Wulsin,  C.  S.,  &  
%DFRQ&RKHQ/LWZLQ1XQQDOO\ 3XWQDP0([DPLQLQJWKHXVHRIODQJXDJHDQG
KRZHYHUWKHPLQLPDOUHFRPPHQGHGOHYHORILQWHUQDO OLWHUDF\DVVHVVPHQWVZLWK\RXQJGXDOODQJXDJHOHDUQHUV  
consistency  differs  according  to  the  type  of  inference  that   Research  report  no.  1.  Center  for  Early  Care  and  Educa-­
ZLOOEHPDGHDERXWWKHUHVXOWV tion  Research–Dual  Language  Learners  (CECER-­DLL).  
&KDSHO+LOO8QLYHUVLW\RI1RUWK&DUROLQD)UDQN3RUWHU
3.    The  reported  reliability  estimates  for  the  Spanish  version   Graham  Child  Development  Institute.  
of  the  Story  and  Print  Concepts  used  in  the  FACES  2000  
DQG)$&(6VWXGLHVZHUHOHVVWKDQZKLOHUHOL-­ %DUQHWW:6&DURODQ0()LW]JHUDOG- 6TXLUHV-+
DELOLW\HVWLPDWHVIRUWKH(QJOLVKYHUVLRQZHUHJUHDWHUWKDQ (2011).  7KHVWDWHRISUHVFKRRO6WDWHSUHVFKRRO\HDU-­
.70.   ERRN1HZ%UXQVZLFN1-1DWLRQDO,QVWLWXWHIRU(DUO\
Education  Research.
4.   &RUUHODWLRQVZLWKWKH'//VDPSOHVUDQJHGIURPWR
.79.  Among  English-­only  samples,  estimates  are  often   Bialystok,  E.,  Luk,  G.,  Peets,  K.  F.,  &  Yang,  S.  (2010).  Recep-­
stronger.  Some  examples  of  correlations  among  English   tive  vocabulary  differences  in  monolingual  and  bilingual  
DVVHVVPHQWVLQFOXGHWKH3/6DQGWKH&(/)3ZLWKU  children.  %LOLQJXDOLVP/DQJXDJHDQG&RJQLWLRQ(4),  
IRUWRWDOVFRUHVU IRUH[SUHVVLYHODQJXDJHVFRUHV 525–531.  
from  each  of  these  assessments.  Scores  on  the  PPVT-­4  
UHFHSWLYHYRFDEXODU\ZLWKWKH([SUHVVLYH9RFDEXODU\ %URZQHOO5D([SUHVVLYHRQHZRUGSLFWXUHYRFDEX-­
Test-­Second  Edition  (EVT-­2)  ranged  from  .80  to  .84. ODU\WHVW1RYDWR&$$FDGHPLF7KHUDS\3XEOLFDWLRQV

5.   7KH(QJOLVKODQJXDJHFRPSRQHQWPHDVXUHZDVEDVHGRQ %URZQHOO5E5HFHSWLYHRQHZRUGSLFWXUHYRFDEX-­

the  PPVT-­III  and  the  Receptive  Language  subtest  of  the   ODU\WHVW1RYDWR&$$FDGHPLF7KHUDS\3XEOLFDWLRQV
7(/'7KH6SDQLVKODQJXDJHFRPSRQHQWPHDVXUHZDV
based  on  the  TVIP  and  the  Auditory  Comprehension  sub-­ %URZQHOO5D([SUHVVLYHRQHZRUGSLFWXUHYRFDEX-­
WHVWRIWKH3/67KHVWXG\IRXQGWKDWWKHJURZWKRQWKH lary  test6SDQLVK%LOLQJXDO(GLWLRQ0DQXDO1RYDWR
component  measure  during  the  preschool  year  predicted   &$$FDGHPLF7KHUDS\3XEOLFDWLRQV
literacy  scores  at  the  end  of  kindergarten.  
%URZQHOO5E5HFHSWLYHRQHZRUGSLFWXUHYRFDEXODU\
test6SDQLVK%LOLQJXDO(GLWLRQ0DQXDO1RYDWR&$
Academic  Therapy  Publications.

9
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
Cohen,  J.  (1977).6WDWLVWLFDOSRZHUDQDO\VLVIRUWKHEHKDYLRUDO 3DQ%$5RZH0/6SLHU( 7DPLV/HPRQGD&
sciencesUHYHG1HZ<RUN$FDGHPLF3UHVV 0HDVXULQJSURGXFWLYHYRFDEXODU\RIWRGGOHUVLQ
ORZLQFRPHIDPLOLHVFRQFXUUHQWDQGSUHGLFWLYHYDOLGLW\
'LFNLQVRQ'.0F&DEH$&ODUN±&KLDUHOOL1 :ROI of  three  sources  of  data.  -RXUQDORI&KLOG/DQJXDJH  
A.  (2004).  Cross-­language  transfer  of  phonological   587-­608.
DZDUHQHVVLQORZLQFRPH6SDQLVKDQG(QJOLVKELOLQJXDO
preschool  children.  $SSOLHG3V\FKROLQJXLVWLFV(03),   Patterson,  J.  (1998).  Expressive  vocabulary  development  and  
323–347.   ZRUGFRPELQDWLRQVRI6SDQLVK±(QJOLVKELOLQJXDOWRGGOHUV
$PHULFDQ-RXUQDORI6SHHFK±/DQJXDJH3DWKRORJ\(4),  
Dunn,  L.,  &  Dunn,  L.  (1981).  3HDERG\SLFWXUHYRFDEXODU\ 46–56.
WHVW²UHYLVHG&LUFOH3LQHV01$PHULFDQ*XLGDQFH
Service. Peña,  E.,  Bedore,  L.,  Gutierrez-­Clellen,  V.,  Iglesias,  A.,  &  
Goldstein,  B.  (in  preparation).  %LOLQJXDO(QJOLVK±6SDQLVK
Dunn,  L.,  &  Dunn,  L.  (1997).  3HDERG\SLFWXUHYRFDEXODU\WHVW   DVVHVVPHQW
UGHG&LUFOH3LQHV01$PHULFDQ*XLGDQFH6HUYLFH
3HUU\1( 0HLVHOV6-+RZDFFXUDWHDUHWHDFKHU
Dunn,  D.,  &  Dunn,  L.  (2007).  3HDERG\SLFWXUHYRFDEXODU\WHVW   MXGJPHQWVRIVWXGHQWV¶DFDGHPLFSHUIRUPDQFH"(NCES  
WKHG0LQQHDSROLV011&63HDUVRQ,QF :DVKLQJWRQ'&1DWLRQDO&HQWHUIRU(GXFDWLRQ
Statistics.  
Dunn,  L.,  Padilla,  E.  R.,  Lugo,  D.  E.,  &  Dunn,  L.  (1986).  Test  
GHYRFDEXODULRHQLPDJHQHV3HDERG\&LUFOH3LQHV01
American  Guidance  Service.
Fenson,  L.,  Dale,  P.,  Reznick,  S.,  Thal,  D.,  Bates,  E.,  Hartung,  
J.,  …  Reilly,  J.  (1993).  0DF$UWKXUFRPPXQLFDWLYHGHYHO-­
RSPHQWLQYHQWRULHV6DQ'LHJR&$6LQJXODU3XEOLVKLQJ
5LQDOGL& 3iH]03UHVFKRROPDWWHUV3UHGLFWLQJ
UHDGLQJGLI¿FXOWLHVIRU6SDQLVKVSHDNLQJELOLQJXDOVWX-­
GHQWVLQ¿UVWJUDGH/HDUQLQJ'LVDELOLWLHV(1),  71.  
5HVFRUOD/5DWQHU1%-XVF]\N3 -XVF]\N$0
(2005).    Concurrent  validity  of  the  language  development  
VXUYH\DVVRFLDWLRQVZLWKWKH0DF$UWKXU%DWHVFRPPX-­
+DPPHU&6/DZUHQFH)5 0LFFLR$:
QLFDWLYHGHYHORSPHQWLQYHQWRULHV:RUGVDQGVHQWHQFHV
Bilingual  children’s  language  abilities  and  early  reading  
$PHULFDQ-RXUQDORI6SHHFKDQG/DQJXDJH3DWKRORJ\
outcomes  in  Head  Start  and  kindergarten.  /DQJXDJH
(2),  156-­163.
6SHHFKDQG+HDULQJ6HUYLFHVLQ6FKRROV(3),  237.  
5HHVH( 5HDG63UHGLFWLYHYDOLGLW\RIWKH1HZ
+DPPHU&6'DYLVRQ0'/DZUHQFH)5 0LFFLR$
=HDODQG0DF$UWKXU&RPPXQLFDWLYH'HYHORSPHQW,QYHQ-­
W.  (2009).  The  effect  of  maternal  language  on  bilingual  
WRUW\:RUGVDQG6HQWHQFHV-RXUQDORI&KLOG/DQJXDJH
children’s  vocabulary  and  emergent  literacy  development  
(2),  255-­266.
during  Head  Start  and  kindergarten.6FLHQWL¿F6WXGLHVRI
5HDGLQJ(2),  99–121.   5REHUWV-(%XUFKLQDO0 'XUKDP03DUHQWV¶
report  of  vocabulary  and  grammatical  development  of  
-DFNVRQ0DOGRQDGR'7KDO'-0DUFKPDQ91HZWRQ
$IULFDQ$PHULFDQSUHVFKRROHUV&KLOGDQGHQYLURQPHQWDO
T.,  Fenson,  L.,  &  Conboy,  B.  (2003).  (OLQYHQWDULRGHOGH-­
associations.  &KLOG'HYHORSPHQW92–106.
VDUUROORGHKDELOLGDGHVFRPXQLFDWLYDV8VHU¶VJXLGHDQG
WHFKQLFDOPDQXDO%DOWLPRUH0'%URRNHV3XEOLVKLQJ&R Shin,  H.  B.,  &  Kominski,  R.  A.  (2010).  Language  use  in  the  
8QLWHG6WDWHVAmerican  Community  Survey  
/D3DUR.0 3LDQWD5&3UHGLFWLQJFKLOGUHQ¶V
5HSRUWV$&686&HQVXV%XUHDX:DVKLQJWRQ'&
FRPSHWHQFHLQWKHHDUO\VFKRRO\HDUV$PHWDDQDO\WLF
86*RYHUQPHQW3ULQWLQJ2I¿FH
UHYLHZ5HYLHZRI(GXFDWLRQDO5HVHDUFK(4),  443–484.  
9DJK6%3DQ%$ 0DQFLOOD0DUWLQH]-0HD-­
/LWZLQ06+RZWRDVVHVVDQGLQWHUSUHWVXUYH\SV\-­
VXULQJJURZWKLQELOLQJXDODQGPRQROLQJXDOFKLOGUHQ¶V
FKRPHWULFVQGHG7KRXVDQG2DNV&$6DJH3XEOLFD-­
tions,  2003.  
Lu,  L.,  &  Liu,  H.  S.  (1998).  7KH3HDERG\SLFWXUHYRFDEXODU\
HQJOLVKSURGXFWLYHYRFDEXODU\GHYHORSPHQW7KHXWLOLW\RI
combining  parent  and  teacher  report.&KLOG'HYHORSPHQW
(5),  1545-­1563.  
WHVW²UHYLVHG&KLQHVH7DLSHL7DLZDQ3V\FKRORJLFDO
9RJHO&$%ROOHU.;XH<%ODLU5$LNHQV1%XUZLFN
Publishing.
A.,  &  Stein,  J.  (2011).  /HDUQLQJDVZHJR$¿UVWVQDS-­
Nunnally,  J.  C.  (1978).  3V\FKRPHWULFWKHRU\QGHG1HZ
<RUN1<0F*UDZ+LOO
VKRWRI(DUO\+HDG6WDUWSURJUDPVVWDIIIDPLOLHVDQG
FKLOGUHQ  (ACF-­OPRE  2011-­7).  U.S.  Department  of  Health  
DQG+XPDQ6HUYLFHV2I¿FHRI3ODQQLQJ5HVHDUFKDQG
Evaluation,  Administration  for  Children  and  Families.  
:DVKLQJWRQ'&86*RYHUQPHQW3ULQWLQJ2I¿FH

10
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill
Woodcock,  R.  W.  (1995).  :RRGFRFNODQJXDJHSUR¿FLHQF\EDW-­ Zill,  N.,  Resnick,  G.,  Kim,  K.,  O’Donnell,  K.,  Sorongon,  A.,  
WHU\²UHYLVHG,WDVFD,/5LYHUVLGH3XEOLVKLQJ +XEEHOO0F.H\5ZLWKRWKHUV+HDG6WDUW
)$&(6$ZKROHFKLOGSHUVSHFWLYHRQSURJUDP
:RRGFRFN5: 0XxR]6DQGRYDO$):RRGFRFN SHUIRUPDQFHU.S.  Department  of  Health  and  Human  
ODQJXDJHSUR¿FLHQF\EDWWHU\²UHYLVHG²6SDQLVKIRUP   6HUYLFHV2I¿FHRI3ODQQLQJ5HVHDUFKDQG(YDOXDWLRQ
,WDVFD,/5LYHUVLGH3XEOLVKLQJ Administration  for  Children  and  Families.  Washington,  
'&86*RYHUQPHQW3ULQWLQJ2I¿FH

About CECER-DLL
&(&(5'//LVDQDWLRQDOFHQWHUWKDWLVEXLOGLQJFDSDFLW\IRUUHVHDUFKZLWKGXDOODQJXDJHOHDUQHUV'//VDJHVELUWK
WKURXJK¿YH\HDUV&(&(5'//DLPVWRLPSURYHWKHVWDWHRINQRZOHGJHDQGPHDVXUHPHQWLQHDUO\FKLOGKRRGUHVHDUFK
on  DLLs,  identify  and  advance  research  on  best  practices  for  early  care  and  education  programming,  and  develop  and  
GLVVHPLQDWHSURGXFWVWRLPSURYHUHVHDUFKRQ'//V&(&(5'//LVDFRRSHUDWLYHDJUHHPHQWEHWZHHQWKH)UDQN3RUWHU
*UDKDP)3*&KLOG'HYHORSPHQW,QVWLWXWHDW7KH8QLYHUVLW\RI1RUWK&DUROLQDDW&KDSHO+LOODQGWKH2I¿FHRI3ODQQLQJ
5HVHDUFK (YDOXDWLRQ235(LQWKH$GPLQLVWUDWLRQIRU&KLOGUHQ )DPLOLHV$&)LQFROODERUDWLRQZLWKWKH2I¿FHRI
+HDG6WDUWDQGWKH2I¿FHRI&KLOG&DUH

Suggested  citation
Atkins-­Burnett,  S.,  Bandel,  E.,  &  Aikens,  N.  (2012).  5HVHDUFKEULHI$VVHVVPHQWWRROVIRUWKHODQJXDJHDQGOLWHUDF\GH-­
YHORSPHQWRI\RXQJGXDOODQJXDJHOHDUQHUV'//V&KDSHO+LOO7KH8QLYHUVLW\RI1RUWK&DUROLQD)3*&KLOG'HYHORSPHQW
Institute,  CECER-­DLL.

7KLVEULHIVXPPDUL]HVUHVXOWVIURPDUHYLHZRIWKHOLWHUDWXUHVSRQVRUHGE\&(&(5'//FRQGXFWHGE\DUHVHDUFKWHDP
FRQVLVWLQJRI(LOHHQ%DQGHO6DOO\$WNLQV%XUQHWW'LQD&&DVWUR&ODLUH6PLWKHU:XOVLQDQG0DULVD3XWQDPZLWK0DUJDUHW
%XUFKLQDO/LVD/RSp]9HUD*XWLpUUH]&OHOOHQDQG(OOHQ3HLVQHU)HLQEHUJDVUHVHDUFKSDUWQHUV7KHZRUNZDVVXSSRUWHGE\
DFRRSHUDWLYHDJUHHPHQWIXQGHGE\WKH2I¿FHRI3ODQQLQJ5HVHDUFKDQG(YDOXDWLRQ235(86'HSDUWPHQWRI+HDOWK
DQG+XPDQ6HUYLFHV3HUPLVVLRQWRFRS\GLVVHPLQDWHRURWKHUZLVHXVHLQIRUPDWLRQIURPWKLVGRFXPHQWIRUHGXFDWLRQDO
purposes  is  granted,  provided  that  appropriate  credit  is  given.

$GGLWLRQDO5HVRXUFHV)RUDGGLWLRQDOLQIRUPDWLRQUHJDUGLQJWKLVUHVHDUFKEULHIVHHKWWSFHFHUGOOISJXQFHGX

11
RESEARCH BRIEF #9
CECER—DLL | FPG Child Development Institute, UNC-Chapel Hill