School Psychology Review,

2012, Volume 41, No. 3, pp. 315-325

Commentary

Connecting Cognitive Theory and Assessment:

Measuring Individual Differences in Reading
Comprehens ion

Paul van den Broek and Christine A. Espin

Leiden University and University of Minnesota

The ability to read and comprehend
texts is an essential component of successful
functioning in our world. A substantial
amount of information comes to us through
written means, whether it is through regular
print, Internet, or other media. Part of this
information is for our enjoyment, part of it is
vital for our basic functioningapplication
forms and tax forms that need to be read and
filled out, instructions for operating a new car,
prescription instructions, food labels, and so
forth. The importance of reading is reflected in
school settings, both as a primary means of
conveying knowledge and as main target of
instruction. Accordingly, the assessment of
children's (and adults') ability to read and
comprehend texts receives considerable atten-
tion, in both school and research settings.
In this article, we describe the complex
nature of reading comprehension, review re-
cent insights from cognitive-psychological re-
search into this complexity, and draw impli-
cations from this research for assessment. Our
purpose is to provide a theoretical foundation
for the development of new and the adaptation
of existing tools for assessing reading compre-
hension. In addition, we wish to illustrate
more generally how advances in theories
about psychological constructs (in this case,
reading comprehension) can contribute to the
development of assessment tools.
The Integrated Model of Reading
Comprehension (IMRED): Combining
Key Elements of Theoretical Models of
Reading Comprehension
Reading and comprehending written
language is a complex and uniquely human
activity. Extensive psychological research
over the past 2 decades has resulted in detailed
and comprehensive theoretical models of read-
ing comprehension that allow us to understand
what reading comprehension is and how indi-
viduals differ in their reading comprehension
abilities. Although these models differ in their
details, they agree to a considerable degree on
the main components (e.g., Sabatini, Albro &
O'Reilly, 2012). The remainder of this section
presents what we refer to as the Integrated
Model of Reading Comprehension (IMREC),

Correspondence regarding this article should be addressed to Paul van den Broek, Department of Education
and Child Studies, Leiden University, Wassenaarseweg 52, 2333 AK Leiden, The Netherlands; e-mail:
broekpwvanden@fsw.leidenuniv.nl
Copyright 2012 by the National Association of School Psychologists, ISSN 0279-6015

315
School Psychology Review, 2012, Volume 41, No. 3

in which the main components of the various the meaning of the text, abstracted from the
models are combined into a single account. By textual information itself and supplemented by
providing a single account, the integrated background knowledge and semantic relations
model addresses a major challenge in translat- constructed by the reader. The interpretation
ing theory to assessmentnamely, that differ- of the text based on information within the text
ent theories can lead to different types of as- and background knowledge is commonly re-
sessments (Messick, 1989). ferred to as the reader's situation model of the
text (Kintsch, 1988a), to distinguish it from a
The Product of Reading Comprehension representation of the text information only (the
textbase) and from a visual-perceptual repre-
In considering reading comprehension sentation of the text (the surface model).
and its assessment, it is useful to distinguish The mental representation of a text cre-
between the product and the process of com- ated by the reader is the basis for subsequent
prehending a text (Pearson & Hamm, 2005; performance such as retelling, question an-
van den Broek, Bohn-Gettler, Kendeou, Carl-, swering, or applying the textual information.
son, & White, 2011). The product of compre- Thus, the coherence of a reader's mental rep-
hension is the mental representation of the resentation of a text is an indicator of his or
textual information in the reader's mind after her comprehension of that text. For example,
he or she has completed reading the text good readers consistently remember text ele-
(Kintsch, 1988a, 2012; Cain & Oakhill, 2012; ments with many connections to other ele-
van den Broek, White, Kendeou, & Carlson, ments better than text elements with few con-
2009). In successful comprehension, this rep- nections. Poor readers show less selective
resentation is coherentthat is, the text ele- memory for this structurally central informa-
ments (events, facts, and so on) are intercon- tion (van den Broek, Helder, & van Leijen-
nected through semantic relations and form an horst, in press). As a second example, system-
integrated whole. Different types of relations atic developmental differences in the ability to
may contribute to this coherence but in most identify various types of relations and to con-
cases referential and causal/logical relations stmct coherent representations of text have
are central. Referential relations establish co- been identified (e.g., Oakhill & Cain, 2011;
herence by capturing the identity of objects, van den Broek et al., 2009).
persons, and so on across text elements (e.g.,
the pronoun she in one sentence refers to a
The Process of Reading Comprehension
particular female character mentioned earlier
in the text), whereas causal and logical rela- The mental representation of a text is the
tions establish coherence by capturing depen- puteme of comprehension processes that take
dencies between text elements (e.g., that the place as the reader proceeds through the text.
action of a character is brought about by his or Success and failure in executing these pro-
her motivation to achieve a particular goal). cesses determine the quality of the outcome,
Many of the relations that contribute to thus consideration of these processes is of
the coherent mental representation of a text prime interest in the diagnosis and remediation
must be inferred; to make these inferences the of reading comprehension problems (Pearson
reader activates relevant background knowl- & Hamm, 2005; Perfetti & Adlof, 2012; van
edge. Together, the elements from the text, the den Broek, 2010).
elements activated from background knowl- Before considering the comprehension
edge, and the interconnections between these processes that take place during reading in
elements form a semantic network that repre- detail, three general observations are relevant.
sents the reader's comprehension of the text First, coherence-building processes during
(Graesser & Clark, 1985; Kintsch & van Dijk, reading reflect a balancing act between the
1978; Trabasso & van den Broek, 1985). This reader's attempt to create coherence on the
representation is an interpreted rendering of one hand, and his or her limited attentional or
316

working memory resources on the other hand.
Thus, at any one point in the text, the reader
can only attend to a limited amount of poten-
tially relevant information and execute only a
subset of all processes in his or her cognitive
"toolbox" (Kintsch & van Dijk, 1978; Oakhill
& Cain, 2011; van den Broek, 2012). It fol-
lows that if lower level processes such as letter
recognition or word decoding are not fluent
and thus are consuming precious working-
memory resources, then comprehension pro-
cesses will be compromised (Perfetti, Yang, &
Schmalhofer, 2008; Whitehurst & Lonigan,
1998). Second, the comprehension processes
that take place during reading are partly auto-
matic and partly strategic (Rapp & van den
Broek, 2005; van den Broek, Rapp, & Kend-
eou, 2005). Strategic processes must be
learned and therefore are of primary interest
for instruction and intervention. With practice,
these processes can become routinized to the
point of being virtually automatic in many
situations (LaBerge & Samuels, 1974). As de-
scribed below, the execution of strategic pro-
cesses depends on the efficiency of the auto-
matic processes. Third, there is considerable
agreement that reading comprehension is not a
singular activity but an activity that consists of
multiple component processes that are applied
dynamically and in varying combinations
throughout the process of reading a particular
text (Kintsch 2012; Oakhill & Cain, 2011;
Perfetti & Adlof, 2012; van den Broek, 2012;
van den Broek, Young, Tzeng, & Linderholm,
1998/2004). This observation is supported by
research that found that multiple brain regions
are involved in reading comprehension and
that these regions are recruited to various de-
grees at different points in the text (e.g., Ferstl,
2007; Ferstl, Neumann, Bogler, & von Cra-
mon, 2008; Perfetti & Frishkoff, 2008; Prat,
Mason, & Just, 2011).
Turning to the details of reading com-
prehension processes, the general consensus is
that the processing of a text is cyclical, with
each new reading input (e.g., sentence) pro-
viding new information that triggers a new
combination of automatic and, possibly, stra-
tegic processes.
Connecting Cognitive Theory and Assessment
Automatic processes. The automatic
processes occur at every cycle and are not
under the control of the reader. The most
important of these is spread of activation:
The concepts in each new sentence trigger
activation of other, related concepts from
the representation that the reader has con-
structed during processing of the earlier text
(O'Brien & Albrecht, 1992) and from the
reader's background knowledge (Kintsch,
1988b; van den Broek, Young, Tzeng, &
Linderholm, 1998/2004). In addition, the
reader may have information from recently
processed sentences available in his or her
working memory. Together, information
from these four sources (current sentence,
both prior text and background knowledge
activated through spread of activation, and
working memory) is available "for free" to
the reader to use to comprehend the current
sentence.
Strategic processes. A second set of
processes concerns strategic processes, initi-
ated by the reader to construct coherence. Al-
though strategic processes may become more
and more automated with practice ahd ex-
perience, they demand effort even on the
part of skilled readers (as evidenced, for ex-
ample, by slower reading and/or additional,
eye movements). These strategic processes
include looking back at preceding text for
relevant information, searching one's repre-
sentation of earlier text, searching one's back-
ground knowledge to explain the informa-
tion in the current sentence, and so on (cf.
van den Broek, 1994, 2010). Strategic pro-
cesses also include specific processes that
the reader executes in response to text-struc-
tural features such as text signals (italics,
headers; see Lemari, Lorch, Eyrolle, & Vir-
bel, 2008), integrative processes elicited by
paragraph or section structures (e.g., Lorch,
Lemari, & Grant, 2011; Lorch & Lorch,
1995), and so forth. Individuals differ in the
toolbox of strategies at their disposal, both
in terms of which strategies they have mas-
tered and of how proficient they are in ap-
plying those strategies.
317
School Psychology Review, 2012, Volume 41, No. 3
Balancing automatic and strategic
processes: Standards of coherence.
Whether a sentence triggers strategic pro-
cesses and, if so, what combination of pro-
cesses is triggered depends on the information
available to the reader via automatie processes
at that sentence and on the reader's standards
of coherence in reading that particular text.
Standards of eoherenee refer to the type and
degree of semantic coherence that the reader
aims to maintain while reading (van den Broek
et al., 2011; van den Broek, Risden, & Huse-
bye-Hartman, 1995). These standards vary
among readers and among text types (Linder-
holm & van den Broek, 2002; Narvaez, van
den Broek, Barron-Ruiz, 1999; Wolfe, 2005),
but they ean also vary for a particular reader
depending on, for example, reading instruc-
tions and reading goals (e.g., van den Broek,
Lorch, Linderholm, & Gustafson, 2001) or as
a result of general cognitive functioning (e.g.,
fatigue, effects of distractors). If the automatic
processes meet the reader's standards, then
few or no additional strategic processes will
take plaee. Standards may be met on the basis
of automatic processes alone, for example,
when the text at hand is simple or when the
reader has little motivation or reason to engage
in deep processing. If the automatic processes
do not meet the reader's standards, then the
reader may engage in strategic processes.
To summarize which, if any, strategic
processes occur at each particular point in the
reading of a text depends on the information
available from the current sentence, on the
automatic processes that take place while
reading that sentence, on the reader's stan-
dards of coherence, and on the quality of his or
her toolbox of strategies. Thus, as a reader
proceeds through a text he or she engages in a
dynamically changing mix of automatic and
strategic processes.
Individual and Developmental
Differences
Significant individual and developmen-
tal differences exist in the execution of these
processes and, hence, in the quality of the
resulting mental representation of the text. As
318
described, each new input automatically trig-
gers activation of semantically related infor-
mation from earlier parts of the text and from
background knowledge. Furthermore, a small
buffer of immediately preceding information
is available in working memory. The informa-
tion that becomes available through spread of
activation depends on the depth and breadth of
the reader' background knowledge (McNamara
& Kintsch, 1996; McNamara, Kintsch,
Songer, & Kintsch, 1996; Schneider, Krkel,
& Weinert, 1989; Wolfe, 2005) and on the
quality of the representation that the reader has
constructed of the text to that point (which
itself is the outcome of the automatic and
strategic processes applied to the preceding
text, reflecting the eyclical nature of text pro-
cessing; van den Broek et al., 1998/2004).
Differences also occur in what information is
available from the immediately preceding
textfor example, as a function of the read-
er's working memory capacity (Linderholm &
van den Broek, 2002) and the extent to which
this capacity is consumed by lower level pro-
cessing such as lexical decoding (e.g., Perfetti
& Hart, 2002).
Individual and developmental differ-
ences also emerge as a function of the readers'
ability to employ strategic processes during
reading of a text. For example, readers differ
in the standards of coherence that they employ
and in their ability to adjust their standards to
different reading goals (van den Broek et al.,
2011; Kendeou, Bohn-Gettler, & Fulton,
2011; McCrudden, Magliano, & Schraw,
2010; McCrudden, Schraw, & Kambe, 2005).
These standards are a major factor in deter-
mining whether a reader will engage in stra-
tegic processing beyond the processing that
takes place automatically. As a second exam-
ple, there are substantial differences between
individual readers and between different age
groups in the range and efficiency of the strat-
egies available for comprehension (e.g., Oak-
hill & Cain, 2011; van den Broek et al., 2009;
van den Broek et al., in press). These strate-
gies are in part general to comprehension and
reasoning in any domain, in part specific to
reading and even to specific text genres
(Oakhill & Cain, 2011). The effectiveness of

all of these strategies is strongly influenced by
practice and instruction. Thus, facility in exe-
cuting inferential processes and strategies,
knowledge about text genres and about text
schmas, knowledge and application of stan-
dards of coherence, and the ability to allocate
attention to stmcturally central aspects of the
text are among the major factors that deter-
mine a reader's ability to engage in strategic
processes, and that allow him or her to identify
important relations while reading.
Together, individual and developmen-
tal differences in automatic and strategic
processing result in the identification of dif-
ferent sets of relations during reading and,
as a consequence, in different representa-
tions. In so doing, they form the basis for
systematic variations in the quality and type
of comprehension.
Toward Assessment
The insights into the process and prod-
uct of reading comprehension, as summarized
in the IMREC, may inform the development
of new tests or the adaptations of existing tests
for assessing reading comprehension. Specif-
ically, the IMREC model emphasizes the mul-
tidimensionality of reading comprehension
and provides a framework for more precisely
capturing individual differences and tracking
development in reading comprehension skills.
The IMREC model (and theoretical models in
general) lay out components skills and pro-
cesses that contribute to reading comprehen-
sion. Such models can potentially lead to im-
provements in the diagnostic power of tests
and hence inform intervention and instruction.
Furthermore, as interventions increasingly are
aimed at fostering deep comprehension, it be-
comes important to have instruments with
which that comprehension can be measured
and with which the effectiveness of the inter-
ventions can be evaluated (Kintsch, 2012).
The assessment of reading comprehen-
sion has received considerable attention in re-
cent years, as reading comprehension re-
searchers have raised questions about existing
methods for assessing reading comprehension
(e.g.. Cutting & Scarborough, 2006; Nation &
Connecting Cognitive Theory and Assessment
Snowling, 1997; Keenan, 2012). The ques-
tions originate from research that reveals,
among other things, a lack of consistency in
the skill sets sampled by various reading com-
prehension tests (e.g., some tests rely more on
word-level skills than others), the fact that
comprehension questions in these tests often
can be answered without reading the text, and
a lack of consistency in the rank ordering of
students' scores among different tests (e.g..
Cutting & Scarborough, 2006; Keenan &
Betjemann, 2006; Keenan, Betjemann, &
Olson, 2008; Keenan & Meenan, in press;
Nation & Snowling, 1997).
Given recent increases in our under-
standing of reading comprehension and its
components, and the questions raised about
existing measurement tools, it is timely to
consider whether theoretical advances could
be used to improve existing assessments or to
develop new assessments. However, the trans-
lation of theory to assessment may not be as
straightforward as it seems.
Connecting Theory and Assessment
The question of how theory and assess-
ment interrelate is not a new one. In 1955,
Cronbach and Meehl discussed the relation
between theory and assessment in their formu-
lation of the concept of construct validity.
Construct validity was presented as a method
for validating assessments of constructs for
which no operational definition existed, for
example, intelligence. The then existing ap-
proaches for validationpredictive, concur-
rent, and content validitywere insufficient
because no clear criteria or definable content
existed for such constructs. In Cronbach and
Meehl's (1955) formulation, construct valid-
ity, and theory development were nearly syn-
onymous. Building an assessment would be
building a theory:
We do notfirst"prove" the theory, and then
validate the test, nor conversely. In any prob-
able inductive type of inference from a pat-
tern of observations, we examine the relation
between the total network of theory and ob-
servations. The system involves propositions
relating test to construct, construct to other
319
School Psychology Review, 2012, Volume 41, No. 3

constructs, andfinallyrelating some of these
constructs to observables, (p. 294)
If we were to adopt Cronbach and
Meehl's (1955) conceptualization of construct
validity, the building of a well-developed,
well-supported theory would be the building
of a well-developed, well-supported assess-
ment. Thus, the task of creating a valid assess-
ment for a given construct would require only
that the theory be translated into a reasonable
set of measurement tasks. Specific to our dis-
cussion here, we might then argue that, be-
cause theories of reading comprehension are
fairly well developed and well supported, all
that remains is to translate the information
from these theories into a set of measurement
tasks, and our work on the improvement of
assessments for reading comprehension is
complete. However, such a direct translation
would likely not be effective and would not
reflect more recent theories of validity.
In recent theories of validity, construct
validity is seen as a unifying factor in validi-
tythat is to say, all validity is construct
validity (Messick, 1989, 1994, 1995). In this
conceptualization, validity is established via
the generation of hypotheses about the mean-
ing of test scores for particular uses. These
hypotheses are then examined empirically.
Messick (1989) describes validity as "an inte-
grated evaluative judgment of the degree to
which empirical evidence and theoretical ra-
tionales support the adequacy and appropriate-
ness of inferences and actions based on test
scores or other modes of assessment" (p. 13).
It is not tests or measures, then, that are vali-
dated, but the interpretation of the scores gen-
erated by those tests or measures for a partic-
ular use that are validated.
This view of validity emphasizes that
theories lead to hypotheses about the improve-
ment of existing measures or the development
of new measures. Such hypotheses must then
be empirically tested; that is, studies must be
conducted to examine whether the scores gen-
erated from these new or revised tests improve
our ability to make the decisions we wish to
make. This approach is captured in Kane's
(1992, 2001, 2011) argument-based approach
to validation, in which researchers must first
320
detail the proposed interpretations and uses of
test scores, and then provide evidence support-
ing the plausibility of such interpretations and
uses. The approach is also reflected in Rupp's
(2012) and Mislevy & Sabatini's (2012) de-
scriptions of building reading comprehension
assessments. In the concluding sections, we
outline examples of hypotheses that might be
generated from the IMREC model for the de-
velopment of new or adapted reading compre-
hension assessments.
Implications of Cognitive Theory for
Assessing Reading Comprehension
The conceptual framework of inference
making and coherence building during reading
summarized in the IMREC model has poten-
tial implications for the design of assessment
tools. In this section, we elaborate on several
implications that appear particularly promis-
ing and worth testing.
A general implication of the model is
that reading comprehension is a multidimen-
sional, multicomponential construct (Sabatini
et al., 2012). In comprehension, multiple pro-
cesses come together in combinations that
shift constantly during reading of the text as a
function of properties of the text and the goals
and skills of the text and reader. The various
components of reading comprehension are
likely to interact. They may support, compete
with, or compensate each other. As a conse-
quence of its multidimensional and multicom-
ponential character, the development of a sin-
gle, unidimensional indicator of reading com-
prehension that would be apphcable across all
testing purposes is unlikely and potentially
misleading. Decisions about test development
should be made with the complex nature of
reading comprehension and the specific pur-
pose for testing in mind.
Screening
One important purpose for testing is
screening, which involves comparing individ-
uals on their overall reading comprehension
ability or to determine whether individuals
surpass or fall below a benchmark. Sensitivity
to Structural Centrality (SSC) could be a guid-

ing principle to develop an overall test that
reflects the complex nature of reading compre-
hension processes and the resulting represen-
tations. SSC refers to the extent to which a
reader processes and represents information
that is central to the semantic structure of the
text (van den Broek et al., in press).
SSC can be measured after the reader
has finished reading a text (off-line) as well as
during reading (online). With respect to off-
line measurement, one possible indicator of
SSC is the degree to which a reader selectively
recalls or summarizes elements with many
connections. Selectivity can be expressed, for
example, by the correlation between the ele-
ments' number of connections and frequency
of their being recalled/summarized by the
reader or by the ratio of recall/summary of the
highly connected versus less connected ele-
ments. Such measures have been found to
capture individual and developmental differ-
ences (Rapp et al., 2007; McMaster et al.,
2012; for review, see van den Broek et al., in
press). Other off-line measures of comprehen-
sion, such as question answering, likewise can
be adapted to gauge SSC.
With respect to online measures, one
possible indicator is the degree of selective
attention during reading to text elements that
are central in the text's structure. Examples
are the ratio of reading times for central com-
pared to peripheral elements, the ratio of fre-
quency and duration of eye fixations on central
elements relative to peripheral elements, or
focus on central elements in think-aloud tasks
(van den Broek et al., in press). Of course,
such online measures may be difficult to im-
plement in practical settings, such as the class-
room, but they can be used in educational
research studies.
Monitoring Progress
A second purpose for testing is the mea-
surement of student progress over time with
approaches such as curriculum-based mea-
surement (CBM). CBM is an assessment
method designed to monitor progress of stu-
dents over time and to help educators make
instructional decisions regarding the effective-
Connecting Cognitive Theory and Assessment
ness of instructional programs for those stu-
dents (Deno, 1985). One issue of concern in
CBM progress monitoring is the unreliability
associated with growth rates generated by
CBM measures (e.g., Christ, 2006; Christ &
Ardoin, 2009a, 2009b; Hintz & Christ, 2004;
Tolar, Barth, Francis, Fletcher, Stuebing, &
Vaughn, 2012; Wayman, Wallace, Wiley,
Ticha, & Espin, 2007). The "bounce" or
amount of error associated with these growth
rates is of major concern if one wishes to
reliably and validly track students' progress
over time. A likely contributor to unstable
growth rates is the difficulty in creating paral-
lel forms of the reading measures. For a stu-
dent, scores may vary from passage to pas-
sage. Various methods have been proposed for
improvihg form equivalence, and proposals
have been made for methods that equate
scores "after the fact" (e.g., Ardoin, Suido,
Witt, Aldrich, & McDonald, 2005; Poney,
Skinner, & Axtell, 2005; Tolar et al., 2012;
Wayman et al., 2007). The presented models
of reading comprehension could be used to
create parallel texts that are similar in form
and relational complexity. Such an approach
might serve to improve the equivalence of
passages within students.
Identifying Strengths and Weaknesses
A third purpose for assessment is to
obtain a profile of an individual's strength and
weaknesses in the component skills and pro-
cesses that contribute to his or her reading
comprehension. Such profiles may be used, for
example, to diagnose the source of a reader's
comprehension problems and to guide instruc-
tional interventions. In preceding sections, we
have described several skills and processes
that are major factors in determining individ-
ual and developmental differences in reading
comprehension. These factors roughly fall into
three categories (van den Broek, 2012), per-
taining to (a) general cognitive characteristics
of the reader, (b) the efficacy and efficiency of
general comprehension-related processes, and
(c) language and text-specific skills and pro-
cesses, respectively. Table 1 lists the factors
discussed in this article, grouped by their pri-
321
School Psychology Review, 2012, Volume 41, No. 3
Table 1
Factors underlying individual and
developmental differences in reading
comprehension, grouped by their
primary category.
General cognitive reader eharaeteristics
Verbal working-memory capacity
Background knowledge relevant to text topic
Efficiency/efficacy of comprehension-related
processes
Inferential skills
Knowledge and application of appropriate
standards of coherence
Attention allocation to central information
Language- and text-related skills and processes
Basic language and reading skills (e.g.,
decoding)
Breadth and depth of vocabulary
On-line and off-line sensitivity to structural
centrality
Knowledge about text genres and text schmas
Text-specific coherence-building and inference-
generation skills
Motivation for reading
mary category. Note that category boundaries
are not strict: some factors may fall into mul-
tiple categories. For example, general compre-
hension skills and text-specific comprehension
skills are likely to overlap.
Based on cognitive models, these factors
are prime candidates for inclusion in a com-
prehensive test of reading comprehension. It
would be worthwhile to explore how the ex-
perimental paradigms of research on these
cognitive processes could be adapted to update
current tests of reading comprehension or to
develop new tests.
Conclusions
The aim of this commentary is to pro-
vide a theoretical foundation for the develop-
ment of new and the adaptation of existing
tools for assessing reading comprehension.
This foundation is based on recent advances in
our understanding of what it means to com-
prehend a text and of what factors influence
the quality of comprehension processes and
322
their outcomes, the mental representation of
texts. We have sketched a general model of
reading comprehension, IMREC, in which
major components across current theoretical
models are combined. With current theories
of assessment in mind, we proposed several
ways in which cognitive theories might lead to
the generation of empirically testable implica-
tions for constructing measures of reading
comprehension.
Most cognitive theories of reading com-
prehension focus on the construction of a co-
herent mental representation of a text. How-
ever, representations are not the end product in
terms of using texts in our interactions with
the world. Children may comprehend what
they read in a textbook in school, but be un-
able to apply what they have comprehended.
Likewise, an adult may comprehend the in-
structions of tax forms, but may not be able to
take the appropriate actions based on that un-
derstanding. Thus, in addition to comprehen-
sion of texts, it may be useful to also consider
assessing the range of uses to which readers
can put their representations of text. Cognitive
theories are in their infancy on this topic.
In this commentary, we have concen-
trated on the potential implications of cogni-
tive theories of reading comprehension for
assessment development. In doing so, we have
ignored potential implications of assessment
on theory development. Such implications un-
doubtedly exist. One can imagine, for exam-
ple, that the need to specify a certain process
targeted by a test under development (e.g.,
inference making) brings to light weaknesses
or imprcisions in the theoretical accounts of
that process. Similar application-to-theory im-
plications have been observed for computa-
tional simulations of the reading process
(Goldman, Golden, & van den Broek, 2007).
We hope we have illustrated how theo-
retical grounding can support the development
of valid reading comprehension tests. Of
course, many hurdles would need to be over-
come, including the generation of items, bal-
ancing the need for long texts to accommodate
the complexity of reading comprehension with
practical constraints on test length, and so on.
Moreover, combining the various models into

a single conceptualization such as the IMREC
omits important details of the various pro-
cesses, including interactions and boundary
conditions. We strongly recommend consulta-
tion of the original, more detailed models
when developing an instrument to measure
any of the indicated skills and processes. The
development of sophisticated item-response
techniques is likely to be helpful in elearing
these hurdles (e.g.. De Boeck & Wilson,
2004). Together with such techniques, in-
sights from cognitive-theoretical models
will lead to the next generation of tests of
reading comprehension.
References
Ardoin, S. P., Suido, S. M., Witt, J., Aldrich, S., &
McDonald, E. (2005). Aecuracy of readability esti-
mates' predictions of CBM performance. School Psy-
chology Review, 20, 1-22.
Cain, K., & Oakhill, J. (2012). Reading comprehension
development from seven to fourteen years: Implica-
tions for assessment. In J. P. Sabatini, E. R. Albro, &
T. O'Reilly (Eds.), Measuring up: Advances in how we
assess reading ability (pp. 59-76). Lanham, MD:
Rowman & Littlefield Education.
Christ, T. J. (2006). Short-term estimates of growth using
curriculum-based measurement of oral reading fluen-
cy: Estimates of standard error of slope to construct
confidence intervals. School Psychology Review, 35,
128-133.
Christ, T. J., & Ardoin, S. P. (2009a). Curriculum-based
measurement of oral reading: Standard errors associ-
ated with progress monitoring outcomes from DI-
BELS, AIMSweb, and an experimental passage set.
School Psychology Review, 38, 266-283.
Christ, T. J., & Ardoin, S. P. (2009b). Curriculum-based
measurement of reading: Passage equivalence and se-
lection. Journal of School Psychology, 47, 55-75.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity
in psychological tests. Psychological Bulletin, 52,
281-302.
Cutting, L. E., & Scarborough, H. S. (2006). Prediction of
reading comprehension: Relative contributions of word
recognition, language proficiency, and other cognitive
skills can depend on how comprehension is measured.
Scientific Studies of Reading, 10, 277-299.
De Boeck, P., & Wilson, M. (Eds.). (2004). Explanatory
item response models. A generalized linear and non-
linear approach. New York, NY: Springer.
Deno, S. L. (1985). Curriculum-based measurement: The
emerging alternative. Exceptional Children, 52, 219-
232.
Ferstl, E. C. (2007). The functional neuroanatomy of text
comprehension: What's the story so far? In F. Schmal-
hofer SL C. Perfetti (Eds.), Higher level language pro-
cesses (53-102). Mahwah, NJ: Erlbaum.
Ferstl, E. C , Neumann, J., Bogler, C , & von Cramon,
D. Y. (2008). The extended language network: A meta-
Connecting Cognitive Theory and Assessment
analysis of neuroimaging studies on text comprehen-
sion. Human Brain Mapping, 29, 581-593.
Goldman, S. R., Golden, R., SL van den Broek, P. (2007).
Why are computational models of text comprehension
useful? In F. Schmalhofer & C. Perfetti (Eds.), Higher-
level language processes in the brain (pp. 27-52).
Mahwah, NJ: Erlbaum.
Graesser, A. C , & Clark, L. F. (1985). Structures and
procedures of implicit knowledge. Norwood, NJ:
Ablex.
Hintze, J. M., & Christ, T. J. (2004). An examination of
variability as a function of passage variance in CBM
progress monitoring. School Psychology Review, 33,
204-217.
Kane, M. T. (1992). An argument-based approach to
validity. Psychological Bulletin, 112, 527-535.
Kane, M. T. (2001). Concerns in validity theory. Journal
of Educational Measurement, 38, 319-342.
Kane, M. T. (2011). Validating score interpretations and
uses: Messick Lecture, Language Testing Research
Colloquium, Cambridge, 2010. Language Testing, 29,
3-17. doi: 10.1177/0265532211417210
Keenan, J. M. (2012). Measure for measure: Challenges in
assessing reading comprehension. In J. P. Sabatini,
E. R. Albro, & T. O'Reilly (Eds.), Measuring up:
Advances in how we assess reading ability (pp. 77-
87). Lanham, MD: Rowman & Littlefield Education.
Keenan, J. M., & Betjemann, R. S. (2006). Comprehend-
ing the Gray Oral Reading Test without reading it:
Why comprehension tests should not include passage-
independent items. Scientific Studies of Reading, 10,
363-380.
Keenan, J. M., Betjemann, R. S., & Olson, R. K. (2008).
Reading comprehension tests vary in the skills they
assess: Differential dependence on decoding and oral
comprehension. Scientific Studies of Reading, 12, 281-
300.
Keenan, J. M., & Meenan C. E. (in press). Test differences
in diagnosing comprehension deficits. Journal of
Leaming Disabilities.
Kendeou, P., Bohn-Geter, C , & Fulton, S. (2011). What
we have been missing: The role of goals in reading
comprehension. In M. T. McCrudden, J. Magliano, &
G. Schraw (Eds.), Relevance instructions and goal-
focusing in text leaming (pp. 375-394). Charlotte, NC:
Infonnation Age Publishing.
Kintsch, W. (1988a). Comprehension: A paradigm for
cognition. New York, NY: Cambridge University
Press.
Kintsch, W. (1988b). The use of knowledge in discourse
processing: A construction-integration model. Psycho-
logical Review, 95, 163-182.
Kintsch, W. (2012). Psychological models of reading
comprehension and their implications for assessment.
In J. P. Sabatini, E. R. Albro, & T. O'Reilly (Eds.),
Measuring up: Advances in how we assess reading
ability (pp. 21-38). Lanham, MD: Rowman & Little-
field Education.
Kintsch, W., & van Dijk, T. A. (1978). Toward a model of
text comprehension and production. Psychological Re-
view, 85, 363-394.
LaBerge, D., & Samuels. S. J. (1974). Toward a theory of
automatic information processing in reading. Cognitive
Psychology, 6, 293-323.
323
School Psychology Review, 2012, Volume 41, No. 3
Lemari, J., Lorch, R. F., Jr., Eyrolle, H., & Virbel, J.
(2008). A text-based and reader-based theory of sig-
naling. Educational Psychologist, 43, 27-48.
Linderholm, T., & van den Broek, P. (2002). The effects
of reading purpose and working memory capacity on
the processing of expository text. Journal of Educa-
tional Psychology, 94, 778-784.
Lorch, R. F., Jr., Lemari, J., & Grant, R. A. (2011).
Signaling hierarchical and sequential organization in
expository text. Scientific Studies of Reading, 15, 267-
284.
Lorch, R. F., Jr., & Lorch, E. P. (1995). Effects of orga-
nizational signals on text processing strategies. Journal
of Educational Psychology, 87, 537-544.
McCrudden, M. T., Magliano, J. P., & Schraw, G. (2010).
Exploring how relevancy instructions affect personal
reading intentions, reading goals, and text processing:
A mixed-methods study. Contemporary Educational
Psychology, 35, 229-241.
McGrudden, M. T., Schraw, G., & Kambe, G. (2005). The
effect of relevance instructions on reading time and
learning. Journal of Educational Psychology, 97, 8 8 -
102.
McMaster, K. L., van den Broek, P. W., Espin, C. A.,
White, M. J., Rapp, D. N., Kendeou, P., Garlson, S.
(2012). Making the right connections: Differential ef-
fects of reading intervention for subgroups of compre-
henders. Learning and Individual Differences, 22(1),
100-111.
McNamara, D. S., & Kintsch, W. (1996). Learning from
text: Effects of prior knowledge and text coherence.
Discourse Processes, 22, 247-287.
McNamara, D. S., Kintsch, E., Songer, N. B., & Kintsch,
W. (1996). Are good texts always better? Interactions
of text coherence, background knowledge, and levels
of understanding in leaming from text. Cognition and
Instruction, 14, 1-43.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educa-
tional measurement (3rd ed., pp. 13-103). National
Council on Measurement in Education, American
Council on Education. MacMillan: New York.
Messick, S. (1994). The interplay of evidence and conse-
quences in the validation of performance assessments.
Educational Researcher, 23, 13-23.
Messick, S. (1995). Validity of psychological assessment:
Validation of inferences from persons' responses and
performances as scientific inquiry into score meaning.
American Psychologist, 50, 741-749.
Mislevy, R. J., & Sabatini, J. P. (2012). How research on
reading and research on assessment are transforming
reading assessment (or if they aren't, how they should).
In J. P. Sabatini, E. R. Albro, & T. O'Reilly (Eds.),
Measuring up: Advances in how we assess reading
ability (pp. 119-134). Lanham, MD: Rowman &
Littlefield Education.
Narvaez, D., van den Broek, P., & Barron-Ruiz, A.
(1999). The influence of reading purpose on inference
generation and comprehension in reading. Journal of
Educational Psychology, 91, 488-496.
Nation, K., & Snowling, M. (1997). Assessing reading
difculties: The validity and utility of current measures
of reading skill. British Journal of Educational Psy-
chology, 67, 359-370.
Oakhill, J., & Cain, K. (2011). The precursors of reading
ability in young readers: Evidence from a four-year
324
longitudinal study. Scientific Studies of Reading, 16,
91-121.
O'Brien, E. J., & Albrecht, J. E. (1992). Comprehension
strategies in the development of a mental model. Jour-
nal of Experimental Psychology: Learning, Memory,
and Cognition, 18, Ill-IM.
Pearson P., & Hamm, D. (2005). The assessment of read-
ing comprehension: A review of practices-past, pres-
ent, and future. In S. G. Paris & S. A. Stahl (Eds.),
Children's reading comprehension and assessment
(pp. 13-69). Mahwah, NJ: Lawrence Erlbaum Associ-
ates.
Perfetti, C. A., & Adlof, S. M. (2012). Reading compre-
hension: A conceptual framework from word meaning
to text meaning. In J. P. Sabatini, E. R. Albro, & T.
O'Reilly (Eds.), Measuring up: Advances in how we
assess reading ability (pp. 3-20). Lanham, MD: Row-
man & Littlefleld Education.
Perfetti, C. A., & Frishkoff, G. A. (2008). The neural
bases of text and discourse processing. In B. Stemmer
& H. A. Whitaker (Eds.), Handbook of the neurosci-
ence of language (pp. 165-174). Cambridge, MA:
Elsevier.
Perfetti, C. A., & Hart, L. (2002). The lexical quality
hypothesis. In L. Verhoeven, C. Elbro, & P. Reitsma
(Eds.), Precursors of functional literacy (pp. 189-
214). Amsterdam, The Netherlands: John Benjamins.
Perfetti, C. A., Yang, C.-L., & Schmalhofer, F. (2008).
Comprehension skill and word-to-text processes. Ap-
plied Cognitive Psychology, 22, 303-318.
Poney, B. C , Skinner, C. H., & Axtell, P. K. (2005). An
investigation of the reliability and standard error of
measurement of words read correctly per minute using
curriculum based measurement. Journal of Psychoedu-
cational Assessment, 23, 326-338.
Prat, C. S., Mason, R. A., & Just, M. A. (2011). Individual
differences in the neural basis of causal inferencing.
Brain and Language, 116, 1-13.
Rapp. D. N., & van den Broek, P. (2005). Dynamic text
comprehension: An integrative view of reading. Cur-
rent Directions in Psychological Science, 14, 276-
279.
Rapp, D. N., van den Broek, P., McMaster, K. L., Ken-
deou, P., & Espin, C. A. (2007). Higher-order compre-
hension processes in struggling readers: A perspective
for research and intervention. Scientific Studies of
Reading, 11, 289-312.
Rupp, A. A. (2012). Psychological versus psychometric
dimensionality in reading assessment. In J. P. Saba-
tinin, E. R. Albro, & T. O'Reilly (Eds.), Measuring up:
Advances in how to assess reading ability (pp. 135-
151). Lanham, MD: Rowman & Litdefield.
Sabatini, J. P., Albro, E. R., & O'Reilly, T. (Eds.). (2012).
Measuring up: Advances in how we assess reading
ability. Lanham, MD: Rowman & Littleeld Educa-
tion.
Schneider, W., Krkel, J., & Weinert, F. E. (1989). Do-
main-specific knowledge and memory performance: A
comparison of high- and low-aptitude children. Jour-
nal of Educational Psychology, 81, 306-312.
Tolar, T. D., Barth, A. E., Francis, D. J., Fletcher, J. M.,
Stuebing, K. K., & Vaughn, S. (2012). Psychometric
properties of maze tasks in middle school students.
Assessment for Effective Intervention, 37, 131-146.
 Connecting Cognitive Theory and Assessment

Trabasso, T., & van den Broek, P. W. (1985). Causal
thinking and the representation of narrative events.
Journal of Memory and Language, 24, 612-630.
van den Broek, P. (1994). Comprehension and memory of
narrative texts: Inferences and coherence. In M. A.
Gernsbacher (Ed.), Handbook of psycholinguistics (pp.
539-588). New York, NY: Academic Press.
van den Broek, P. (2010). Using texts in science educa-
tion: Cognitive processes and knowledge representa-
tion. Science, 328, 453-456.
van den Broek, P. W. (2012). Individual and developmen-
tal differences in reading comprehension: Assessing
cognitive processes and outcomes. In J. P. Sabatini,
E. R. Albro, & T. O'Reilly (Eds.), Measuring up:
Advances in how we assess reading ability (pp. 3 9 -
58). Lanham, MD: Rowman & Littlefield Education.
van den Broek, P., Bohn-Gettler, C , Kendeou, P., Carl-
son, S., & White, M. J. (2011). When a reader meets a.
text: The role of standards of coherence in reading
comprehension. In M. McCrudden, J. Magliano, & G.
Schraw (Eds.), Text relevance and learning from text
(pp. 123-140). Greenwich, CT: Information Age Pub-
lishing.
van den Broek, P. W., Helder, A., & Van Leijenhorst, L.
(in press). Sensitivity to Structural Centrality: Devel-
opmental and individual differences in reading com-
prehension skills. In M. A. Britt, S. R. Goldman, &
J.-F. Rouet (Eds.), Reading: Erom words to multiple
texts. New York, NY: Routledge, Taylor & Francis
Group.
van den Broek, P., Lorch, R. F., Jr., Linderholm, T., &
Gustafson, M. (2001). The effects of readers' goals on
inference generation and memory for texts. Memory &
Cognition, 29, 1081-1087.
van den Broek, P., Rapp, D. N., & Kendeou, P. (2005).
Integrating memory-based and constructionist pro-
cesses in accounts of reading comprehension. Dis-
course Processes, 39, 299-316.
van den Broek, P., Risden, K., & Husebye-Hartman, E.
(1995). The role of readers' standards for coherence in
the generation of inferences during reading. In R. F.
Lorch, Jr., & E. J. O'Brien (Eds.), Sources of coher-
ence in text comprehension (pp. 353-373). Hillsdale,
NJ: Erlbaum.
van den Broek, P., White, M. J., Kendeou, P., & Carlson,
S. (2009). Reading between the lines: Developmental
and individual differences in cognitive processes in
reading comprehension. In R. K. Wagner, C. Schatsch-
neider, & C. Phythian-Sence (Eds.), Beyond decoding:
The behavioral and biological foundations of reading
comprehension (pp. 107-123). New York, NY: Guil-
ford Press.
van den Broek, P., Young, M., Tzeng, Y., & Linderholm,
T. (1998/2004). The landscape model of reading: In-
ferences and the on-line construction of a memory
representation. In H. van Oostendorp & S. R. Goldman
(Eds.), The construction of mental representations dur-
ing reading (pp. 71-98). Mahwah, NJ: Erlbaum. (Re-
printed in R. B. Ruddell & N. J. Unrau [Eds.], Theo-
retical models and processes of reading [1244-1269].
Newark, NJ: International Reading Association)
Wayman, M., Wallace, T., Wiley, H. I., Ticha, R., &
Espin, C. A. (2007). Literature synthesis on curricu-
lum-based measurement in reading. Journal of Special
Education, 41, 85-120.
Whitehurst, G. J., & Lonigan, C. J. (1998). Child devel-
opment and emergent literacy. Child Development, 69,
848-872.
Wolfe, M. B. W. (2005). Memory for narrative and ex-
pository text: Independent influences of semantic as-
sociations and text organization. Journal of E.xperi-
mental Psychology: Learning, Memory, and Cogni-
tion, 31, 359-364.
Date Received: August 4, 2012
Date Accepted: August 4, 2012
Action Editor: Matthew K. Bums

Paul van den Broek is a professor in education and child studies and director of the Brain
and Education laboratory at the University of Leiden (The Netherlands), and adjunct
professor in cognitive sciences at the University of Minnesota. His expertise is on
cognitive processes in reading comprehension, on the development of these processes,
and on the application to reading comprehension interventions.

Christine Espin is a professor at Leiden University in the Department of Education and

Child Studies, Special Education and an adjunct professor in cognitive sciences at the
University of Minnesota. At Leiden, she holds the Leiden University Fund Chair for
Teachers' Assessment of Learning and Behavioral Problems in Adolescents. Her research
has focused on the development of CBM progress-monitoring procedures for secondary
school students in reading, written expression, language learning, and content-area learn-
ing. Most recently, her research has focused on teachers' interpretation and use of data in
educational decision making.

325
Copyright of School Psychology Review is the property of National Association of School Psychologists and its
content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's
express written permission. However, users may print, download, or email articles for individual use.