Nikolaraizi, M., Vekiri, I., & Easterbrooks, S. (2013).
Investigating deaf students' use of visual multimedia resources
in reading comprehension. American Annals ofthe Deaf, 157 {5), 458-473.
INVESTIGATING DEAF STUDENTS' USE OF
VISUAL MULTIMEDIA RESOURCES IN READING
COMPREHENSION
MAGDA NIKOLARAIZI,
IoANNA VEKIRI, AND
SUSAN R . EASTERBROOKS
NIKOLARAIZI IS AN ASSISTANT PROFESSOR.
DEPARTMENT OF SPECIAL EDUCATION,
UNIVERSITY OFTHESSALY, VOLOS, GREECE.
AT THE TIME OF THE PRESENT STUDY, VEKIRI
WAS AN ADJUNCT LECTURER, DEPARTMENT OF
ELEMENTARY EDUCATION, UNIVERSITY OF
THESSALY SHE IS NOW AN INDEPENDENT
RESEARCHER SPECIALIZING IN EDUCATIONAL
TECHNOLOGY EASTERBROOKS IS A PROFESSOR,
DEPARTMENT OF EDUCATIONAL PSYCHOLOGY
AND SPECIAL EDUCATION, GEORGIA STATE
UNIVERSITY, ATLANTA.
VOLUME 157, No. 5, 2013
MIXED RESEARCH D E SIG N was used to examine how deaf Students used
the visual resources of a multimedia software package that was designed
to support reading comprehension. The viewing behavior of 8 deaf stu-
dents, ages 8-12 years, was recorded during their interaction with mul-
timedia software that included narrative texts enriched with Greek Sign
Language videos, pictures, and concept maps. Also, students' reading
comprehension was assessed through reading comprehension ques-
tions and retelling. Analysis ofthe students' viewing behavior data, their
answers to reading comprehension questions, their "think alouds," and
their story retells indicated that they used visual resources, but they did
not exploit them in a strategic manner to aid their reading comprehen-
sion. The study underscores the important role of mediated instruction
in "visual literacy" skills that enable students to learn how to process
visual aids in a way that supports their reading comprehension.
Keywords: deaf, reading, multimedia, tion, instructional, and experiential fac-
visual resources, pictures, concept tors (Wilson & Hyde, 1997), and relate
maps to cognitive processing, language com-
prehension, and learning (Marschark
One of tbe major concerns and chal- et al., 2009). Furthermore, these chal-
lenging tasks of educators of deaf and lenges are associated with many vari-
hard of hearing students is to enhance ables, namely the text (e.g., word
the reading comprehension perform- identification, syntax), the reader (e.g.,
ance of their students, which, despite prior knowledge, metacognition), and
the application of years' worth of re- the context (e.g., the purpose of read-
search, appears to remain very poor ing; Paul, 1998; Wang & Paul, 2011),
(Marschark et al., 2009; Paul, 1998; and concern difficulties with lower- and
Schirmer, 2000; Traxler, 2000). Good higher-level reading skills, such as poor
reading comprehension is a key to later vocabulary understanding, problems
ability to access academic information with syntax processing (Kelly, 2007;
through print. The challenges facing Paul, 1998), and low-quality or inappro-
deaf and hard of hearing students are priate use of metacognitive reading
attributed to linguistic, communica- strategies (Andrews & Mason, 1991;
AMERICAN ANNALS OF THE DEAF
Kelly, Albertini, & Shannon, 2001;
Schirmer, 2003; Strassman, 1997).
In order to respond to the reading
comprehension needs of deaf and hard
of hearing students, enhance their
access to the curriculum, and maxi-
mize their learning capacity, teachers
need to design and differentiate
instruction according to these stu-
dents' needs. Differentiated instruc-
tion requires teachers to be flexible in
planning, decision making, and cur-
riculum selection while considering
the diverse needs of students in a class-
room (Tobin, 2008; Tomlinson, 1999;
Van Garderen & Whittaker, 2006). Also,
based on the principles of universal
design for learning, the diverse needs
of learners need to be considered at
the veiy beginning stages of planning
and organizing for instruction, so that
as many students as possible benefit
from the learning environment with-
out any further modifications beyond
those incorporated into the original
design (Tobin, 2008). Differentiated
instruction and universal design for
learning are challenging approaches,
given that deaf and hard of hearing stu-
dents constitute a heterogeneous
school population with diverse lan-
guage needs (Easterbrooks & Baker,
2002). One of the key elements of dif-
ferentiated instruction and universal
design for learning with regard to deaf
and hard of hearing students is the cre-
ation of a visual learning environment.
The significance of this element relates
to the fact that all deaf and hard of
hearing students, including ones with
hearing aids or cochlear implants, who
may communicate in sign or in a spo-
ken language, have full access to
visual information and therefore can
benefit from the use of visual tech-
niques, visual strategies, and visual
educational materials (Cannon, East-
erbrooks, Gagn, & Beal-Alvarez, 2001;
Dowaliby & Lang, 1999; Easterbrooks
& Baker, 2002; Luckner, Bowen, &
VOLUME 157, No. 5, 2013
Carter, 2001; Marschark, 2005; Paul,
1998; Schirmer, 2000).
Research on the role of visual re-
sources in the reading comprehension
of deaf and hard of hearing students
has pointed out these resources' ben-
efits. Reynolds and Rosen (1973) com-
pared the roles of three instructional
formats on 146 college students' text
comprehension and information re-
tention. The three formats were (a) a
narrative textbook style that included
text with vocabulary and syntax struc-
ture that were somewhat less complex
than that in a text for college-level stu-
dents, as well as eight line drawings;
(b) individualized instruction that
included learning objectives, text infor-
mation with sentence structure slightly
less difficult than that used in the first
instructional condition, and nine line
drawings, prior to which specific in-
structions and self-test questions were
also added; and (c) a picture format,
which entailed less and simpler text
information comparing with the two
first instructional conditions and 26
picture displays, each accompanied by
labels and a brief descriptive sentence.
The researchers found that the picto-
rial instructional format was the most
effective for facilitating students' com-
prehension and information retention.
Robbins (1983) investigated the read-
ing comprehension performance of 49
elementary and 32 secondary students
using texts with and without accompa-
nying sign language pictures placed
above English words. Results indicated
that students' reading comprehension
performance was better with the signed
versions of the texts. The role of Signed
English pictures in reading comprehen-
sion was also examined by Wson and
Hyde (1997), in a study with 16 stu-
dents 8-13 years old, who were divided
into two reading proficiency groups,
read a signed and a nonsigned English
text, then answered reading compre-
hension questions and retold the story.
The participants performed better
with texts with Signed English pictures,
and the poor readers derived greater
benefits compared with good readers.
Walker, Munro, and Richards (1998)
conducted an intervention study dur-
ing which they taught inferential read-
ing to 30 underachieving prelingually
deaf readers (ages 9-18 years) through
the use of pictorial material and printed
text. The intervention was effective in
improving students' ability to make
inferences and their overall reading
comprehension.
Apart from research investigating
the effects of static visual aids on read-
ing comprehension, a number of
studies have examined the role of edu-
cational multimedia applications,
which have the potential to create a
rich and interactive visual learning
environment. Texts can be displayed
in various forms and enriched with
many resources, including graphics,
pictures, movies, animation, and sign
language videos, creating a fiexible
learning environment that can offer
students the opportunity to explore
information at their own pace and
engage with learning materials in a way
that suits their needs. Therefore, mul-
timedia can be a valuable tool in the
implementation of differentiated in-
struction or universal design for learn-
ing (Boone & Higgins, 2007; Hitchcock,
Meyer, Rose, & Jackson, 2002; Rose,
Hasselbring, Stahl, & Zabala, 2005).
Eurthermore, those multimedia tools
that move children through a program
on the basis of each individual child's
performance can scaffold students'
learning, since these tools allow stu-
dents to make progress depending
on their mastery level (Cannon et al.,
2011). This finding is in line with
Vygotsky's theory (1978, 1962/1986)
that learning occurs within one's zone
of proximal development; therefore,
the level of difficulty of a learning task
should be within one's mastery level.
AMERICAN ANNAI.S OF THE DEAF
USE OF VISUAL MULTIMEDIA IN READING COMPREHENSION
not below or above tbat level (Byrnes,
1996; Tomlinson et al, 2003).
Research on multimedia technol-
ogy and reading comprehension of
deaf and hard of hearing students has
included applications with a variety
of accompanying visual resources and
adjunct instructional aids. More ana-
lytically Dowaliby and Lang (1999)
investigated tbe effecdveness of five
instructional conditions in facilitating
comprehension of a science text by
144 students enrolled in the National
Institute for the Deaf in Rochester, NY
Thefiveconditions were (a) text only,
(b) text and content movies, (c) text
and sign movies, (d) text and adjunct
questions, and (e) all of these together.
The researchers found that students
performed better in conditions that
involved the adjunct questions or all
the supports. Kelly (1998) explored
whether silent modon pictures facili-
tated reading comprehension of rela-
tive clause and passive voice printed
sentences. The meaning ofthe stories
was conveyed almost exclusively
through the acdons ofthe characters.
Eleven deaf adults watched short seg-
ments of video action and then chose
between two sentences that best
described the videos' action segments.
The results indicated that the videos'
context facilitated comprehension in
proficient readers, while less able read-
ers were not likely to benefit. Horney
and Anderson-Inman (1999) examined
tbe interactions of a 12-year-old stu-
dent within an electronic reading envi-
ronment that had been designed as
part of the Literacy-Hi project to fa-
cilitate the reading comprehension
of deaf readers. Several supportive
resources were included in this envi-
ronment, such as translational (e.g.,
American Sign Language), illustrative
(e.g., word illustrations), summarizing
(e.g., outlines) and evaluative resources
(e.g., comprehension questions). The
results of the study indicated that the
VOLUME 157, No. 5, 2013
student took advantage ofthe available
resources, mostly the pictures or ani-
mations. However, he did not use ASL
translations or self-monitoring ques-
tions because he did not know ASL or
how to benefit from the questions.
Within the Cornerstones approach,
Loeterman, Paul, and Donahue (2002)
developed multimedia educational
software to enhance students' word
identification, knowledge acquisition,
and story comprehension. Sample
units included captioning, hypertext
stories in which children could click
on words for further information, and
videotaped stories in ASL or Sign Sup-
ported English. Additional materials
were used to make lesson content
visual and engage children in an active
interaction with the story. The evalua-
tion of the program included eight
teachers who participated in a 2-week
field test with 32 deaf and hard of hear-
ing children 6-12 years old. Results
indicated that when teachers used
technology supports along with in-
structional techniques based on the
knowledge model of vocabulary devel-
opment, students' in-depth knowl-
edge of words increased.
Lang and Steely (2003) summarized
three empirical research studies show-
ing that multimedia science lessons
can enhance the learning of students
attending middle and high schools.
The researchers presented lessons
through a series of "triads," each con-
taining a short text screen, an anima-
tion corresponding to the text content,
and an ASL version of that text. Lang
and Steely found that the students who
used these multimedia materials gained
significantly greater knowledge com-
pared with students who experienced
standard teacher-led instruction.
In a study by Gentry, Chinn, and
Moulton (2004/2005), 28 students 9-18
years old were presented with digital
texts in four different formats: print
only, print plus pictures, print plus sign
language, and print plus pictures plus
sign. Reading comprehension of the
students was better when the stories
were presented in the print-with-pic-
tures format, while the worst perform-
ance was observed when the stories
were presented in a print-only format.
Wang and Paul (2011) compared
the effectiveness of the Cornerstones
approach, a literature-based, technol-
ogy-infused literacy project, with a
"Typical" instructional approach. Five
teachers with 22 students 7-11 years
old participated in the study; that is,
each teacher conducted both Corner-
stones and Typical lessons and each
student participated in both lessons.
There were three Typical and three
Cornerstones lessons, the order of
which was alternated over three exper-
iments and counterbalanced with a 1-
week break between the two lessons
in each experiment. In Experiment 1,
the Typical lesson came first, then the
Cornerstones lesson. In Experiment 2,
the Cornerstones lesson came first,
then the Typical lesson. In Experiment
3, the Typical lesson came first, then
the Cornerstones lesson. There were
significant differences in outcomes
between the Typical and Cornerstones
approaches in word identification and
story comprehension in two out ofthe
three experiments, though none in
word knowledge or story comprehen-
sion in the third experiment. The teach-
ers enjoyed the technology aspects of
the project, and the usefulness of the
instructional strategies and activities in
the accompanying materials.
Yoong and Kim (2011) examined
the effects of captions on 66 adult stu-
dents' content comprehension, cogni-
tive load, and motivation in online
learning, when captions were used in
conjunction with sign language. Partic-
ipants were randomly assigned to
either the control group or the ex-
perimental group. The results of the
experiment indicated a significant dif-
AMERICAN ANNALS OF THE DEAF
ference in content comprehension
between the two groups, but no statis-
tically significant difference in cogni-
tive load and motivation.
Findings from all the studies dis-
cussed above converge in the conclu-
sion that visual aids and multimedia
materials that include visual resources
can support the reading compre-
hension of deaf and hard of hearing
students. However, this conclusion
should be interpreted with some cau-
tion because relevant studies varied in
terms of the instructional conditions
in which visual resources were used,
the range and type of the resources
included in the materials, and the age
groups of the participants. In some
studies (Dowaliby & Lang, 1999; Kelly,
1998; Reynolds & Rosen, 1973; Yoong
& Kim 2011), visual materials were
used by adults. In studies involving
school-age students, visual resources
were used either in the context of
teacher-led instruction, where stu-
dents' learning with the resources was
guided (Lang & Steely, 2003; Loeter-
man et al., 2002; Walker et al., 1998;
Wang & Paul, 2011), or in the context
of an experiment in which presenta-
tion of the materials was controlled by
the experimenter (Lang & Steely, 2003)
and/or students had access to a lim-
ited number of resources at a time
(Gentry et al., 2004/2005; Robbins,
1983; Wilson & Hyde, 1997). Little is
known about how young deaf and
hard of hearing readers use visual aids
in the context of independent study
when they are not guided by a more
experienced peer or adult, particularly
when these visual resources are pro-
vided in an interactive multimedia
reading environment (e.g., Horney &
Anderson-Inman, 1999).
Using multimedia reading software
may be challenging for young deaf and
hard of hearing readers because they
must independently select from a vari-
ety of resources the ones that address
VOLUME 157, No. 5, 2013
their learning needs, choose the order
in which they are going to study them
relative to the text, and use cognitive
strategies that are appropriate for pro-
cessing each resource. Students are
not likely to allocate their attention to
particular visual resources if they are
not aware of these resources' value
(Peeck, 1993). In addition, encoding
and integrating information from the
text and visual aids is a complex task,
particularly for young and less skilled
readers who may not know appropri-
ate cognitive strategies (Filippatou &
Pumfrey, 1996; McTigue, 2009).
Research with hearing readers shows
that even when they decide to use
adjunct resources, poor readers do not
use them in a manner that can help
them better understand what they
read in the text (Hannus & Hyn,
1999; McTigue, 2009). Specifically, stud-
ies in which eye fixation data were col-
lected in an investigation of the role of
adjunct visual displays in expository
text comprehension of both adult read-
ers (Hegarty & Just, 1989; Schnotz,
Picard, & Hron, 1993) and young hear-
ing readers (Hannus & Hyn, 1999)
showed that although both low- and
high-ability readers spent time observ-
ing the displays, it was the high-ability
readers who could process them effec-
tively and benefited most from their
use. High-ability readers used better
processing strategies, such as focusing
on pertinent picture and text informa-
tion and looking back and forth many
times between the text and the illustra-
tions. The use of such strategies indi-
cated that good readers tended to
focus their attention on main text ideas
and picture elements and tried to inte-
grate picture and text information. The
viewing behavior of poor readers, who
paid attention to both important and
less important illustration segments
and did less back-and-forth looking,
indicated that they did not know what
picture information they needed to
attend to, and that they made fewer
attempts to integrate text and picture
information.
Currently, there is little research
examining how young deaf and hard of
hearing readers use visual resources
when they have access to multimodal
texts. Such investigations can help
researchers and educators understand
what type of instruction and assistance
deaf readers need in order to use mul-
timedia reading resources effectively
Also, relevant research can guide the
design of appropriate software scaf-
folds as well as teachers' efforts to help
students develop skills for using visual
resources in reading comprehension.
In the present article, we present
findings from a study in which we
examined young severely to pro-
foundly deaf students' use of visual
resources during independent reading
of narrative texts provided within a
specific package of multimedia soft-
ware. The software. See and See, was
developed to meet deaf students'
needs, as part of a project focusing on
the role of multimedia visual resources
in deaf children's reading comprehen-
sion. It included electronic texts en-
riched with Greek Sign Language
(GSL) videos, pictures, and multiple-
choice comprehension questions, as
well as concept maps, whose role in
deaf students' reading comprehension
had not been addressed in previous
multimedia research (Nikolaraizi &
Vekiri, 2012). (The project was imple-
mented by the University of Thessaly
and cofinanced by the European Social
Fund and the Greek Ministry of Na-
tional Education and Religious Affairs-
Operational Program for Education and
Initial Vocational Training.)
Specifically, we addressed three re-
search questions: ' :
1. Did students utilize the visual
resources of the See and See
software (i.e., GSL, pictures, and
AMERICAN ANNALS OF THE DEAF
USE OF VISUAL MULTI2VIEDL;\ IN READING COMPREHENSION
the concept map) during inde-
pendent reading and with mini-
mal prior instruction? Which
resources of the software did
they tend to use?
2. What information did the stu-
dents extract from the visual
resources of See and See during
reading? Did they integrate the
information from these resources
during reading?
3. Did use of the visual resources
of See and See aid reading com-
prehension?
Methodology
Participants
Eight deaf students enrolled in two pri-
mary special schools for the deaf par-
ticipated in the present study Five
students were male and three were
female. Their ages ranged from 7 to 12
years. We selected these eight prelin-
gually deaf students on the basis of six
criteria:
1. They had severe to profound
hearing losses.
2. They had no additional disabil-
ities.
3. They could communicate in
either Greek or GSL.
4. Their reading level allowed them
to participate in the test.
5. They attended primary schools
for deaf students.
6. They were allowed to participate
in the study.
Greece, a small country of about 11
million people, collects no official data
on the exact number of deaf and hard
of hearing students attending regular
schools and schools for the deaf, as
well as students' characteristics (e.g.,
home language, other disabilities,
reading ability level). So we had to rely
on a convenience sample of eight chil-
dren who met all of the study criteria.
Reading ability level, one of the criteria
VOLUME 157, No. 5, 2013
Table 1
Characteristics of Study Participants (N= 8)
participants had severe to profound hearing loss, and had hearing parents.
used in selecting study participants,
was estimated on the basis of perform-
ance on a Greek standardized reading
test and teachers' reports (see Table 1).
Materials: The See and See
Multimedia Softw^are
The educational software See and See
was based on an interdisciplinary the-
oretical framework, which incorpo-
rated research on deaf students'
reading comprehension, the role of
visual resources in text comprehen-
sion, and multimedia learning princi-
ples (Nikolaraizi & Vekiri, 2012).
See and See includes two user inter-
face components, one for students and
one for teachers. The student interface
component includes narrative texts,
visual resources, and reading compre-
hension questions. All reading texts
are from the state textbooks used in
Greek primary schools, and each text
corresponds to a specific grade and
reading level. Specifically, students
have access to (a) GSL videos, both for
each sentence and for the entire text;
(b) pictures for certain sentences or
paragraphs; and (c) one concept map,
which provides a visual representation
of the story structure of the narrative
text. The pictures have representation
and organization functions, which
means that they closely correspond
to text content and present relations
among pieces of information (Carney
& Levin, 2002; Levin, Anglin, & Carney,
1987). The concept maps illustrate the
main elements of the narrative text,
that is, the characters, the setting, the
problem or the plot of the story, and
the solution, which are presented
according to the story's structure. In
this way, a concept map can help learn-
ers identify the main elements of a
story and can aid story summarizing or
recalling (Castillo, Mosquera, & Pala-
cios, 2008; Chang, Sung, & Chen, 2001;
Chmielewski & Dansereau, 1998).
After students read the text, they
are forwarded to another screen to
reply to five multiple-choice reading
comprehension questions, which en-
gage them in cognitive processing
(Andre, 1979; Dowaliby, 1992; Dowal-
iby & Lang, 1999), and therefore can
foster mathemagenic behaviors, that is
"behaviors that give birth to learning"
and help students achieve instruc-
tional goals (Rothkopf, 1970, p. 325).
Four text-explicit questions and one
text-implicit question are included that
concern the most important elements
in the story. All these elements, and
therefore the answers to the ques-
tions, can be identified within the con-
cept map and the pictures that have
representation and organization func-
tions. Each reading comprehension
question and its available answers are
AMERICAN ANNALS OF THE DEAF
accompanied by GSL videos. When
users give an answer, they get a feed-
back message about wbether their
answer is right or wrong; if their
answer is wrong, the message encour-
ages them to return to the text and
search for the right answer.
The presentation of all visual
resources in See and See follows mul-
timedia design principles. In order to
avoid misdirecting students' atten-
tion and overloading their working
memory with redundant information
(Sweller, 2005), the software does not
permit students to access more than
one visual aid simultaneously in addi-
tion to the text. To facilitate integra-
tion of information between text and
visual aids, the spatial and the tempo-
ral contiguity principles are applied
(Mayer, 2005b). Specifically, the visual
aid window is placed right above the
text, and each visual aid is presented
concurrently with the text. Also, the
students can enlarge each visual aid,
but if they do so, they do not have
access to the text. Further, the text sen-
tence or paragraph corresponding to
each GSL video or picture, respec-
tively, is highlighted to signal its rela-
tionship with the video or the picture
(Mayer, 2005b). The design ofthe pic-
tures is based on tbe coherence prin-
ciple (Mayer, Bove, Bryman, Mars, &
Tapangco, 1996): Each picture is espe-
cially created to depict text informa-
tion that is essential for understanding
tbe main ideas of each paragraph or
section. Finally, the software follows
the segmenting principle (Mayer,
2005a) because it enables students to
control the pace of their reading and
choose when and how many times
tbey want to view each of the visual
aids.
The second user interface compo-
nent concerns teachers, and enables
them (a) to modify the existing read-
ing texts and resources or create new
ones, and (b) to monitor students' use
VOLUME 157, No. 5, 2013
of the software by viewing reports
regarding each student's interaction
with the texts and with the software's
resources (e.g., whether and wben
they used each visual aid, whether they
answered a question correctly). These
options serve the aims of differend-
ated instruction, because tbey allow
teachers to design activities and to scaf-
fold learning according to students'
mastery level and diverse needs (East-
erbrooks & Baker, 2002; Van Garderen
& Whittaker, 2001), and to make peri-
odic assessments of students' progress
(Marschark et al., 2002; Stewart &
Kluwin, 2001).
Instructional Conditions
and Procedure
A researcher fluent in GSL met each of
the eight students two times. The two
meetings took place over 4 days, and
each lasted approximately 1 hour. In
the first meeting, the researcher
demonstrated tbe software and ex-
plained what the student could expect,
and in the second meeting the testing
took place. Specifically, in the first
meeting the researcher selected a text
that corresponded to the student's
reading comprehension level, which
was assessed before the researcher
demonstrated the software. This eval-
uation was based on a Greek reading
test that was administered to each stu-
dent individually and on teachers'
judgments. During the demonstration
ofthe software, the researcher started
reading a section of the text and used
the visual resources (GSL videos and
pictures) within the section. While
viewing the GSL videos and the pic-
tures, the researcher explained that
these corresponded to the highlighted
sentences within the text. Also, the
researcher modeled "think-aloud"
behavior by describing in GSL what
he saw in the pictures and the con-
cept map, and related the information
that was illustrated in the pictures and
in the concept map to the text's con-
tent. After reading part ofthe text, the
researcher encouraged the student to
read the following part and use the
visual resources. He guided and
replied to each student's questions.
At the end of the text, the researcher
viewed the concept map and described
or expressed his thoughts regarding it
in GSL.
After the researcher and the stu-
dent read the text, they answered the
reading comprehension questions.
The researcher read tbefirsttwo ques-
tions and all the available answers. On
the first question, he intentionally
selected a wrong answer. An automatic
message appeared informing him that
his answer was wrong and encourag-
ing him to go back and search for the
right answer. The researcher followed
this instruction, re-read certain parts
of the text, used the available visual
resources, and then selected the right
answer. On the second question, the
researcher pretended tbat he did not
understand the question or a certain
answer, and he chose to view the
related GSL video. Then, the student
was asked to reply to the questions
that followed. If the student could not
understand a question, tbe researcher
encouraged viewing of the GSL video.
Also, if the student selected a wrong
answer, the automatic instruction
message appeared saying that the stu-
dent could go back to the text and
read part of it or view the visual
resources again. After answers had
been given to all the questions, the
researcher shut down the computer
and asked the student to recall in GSL
or in Greek everything he or she
remembered regarding the text. The
researcher gave the following prompts
to the students, only when the stu-
dents did not tell or sign anything:
"What happened at the beginning?"
"What happened then?" and "What
happened at the end?"
AMERICAN ANNALS OF THE DEAF
USE OF VISUAL MULTIMEDIA IN READING COVIPREHENSION

when the demonstration was over,
the researcher met each student once
more for the testing to take place. It is
important to note that there was no
systematic instruction on use of the
software, as the aim of the present
study was to record students' behavior
with minimal prior instruction. In the
second meeting, the researcher se-
lected a text on the basis of the stu-
dent's reading level and asked the
student to read the text, reply to five
multiple-choice reading comprehen-
sion questions, and recall the story.
Before reading the text, the researcher
reminded the student that he or she
could use all the visual resourcesthat
is, pictures, sentence GSL videos,
whole GSL videos, the concept map,
and the question GSL videos. Also, the
researcher encouraged the student to
"think aloud" his or her thoughts and
descriptions when viewing the pic-
tures and the concept map. The sec-
ond meeting, including the student's
picture "think-aloud" protocols, and
text recalls.
Software Reports
Software reports provided data regard-
ing each student's viewing behavior,
that is, which resources, how many
times, and when the student chose to
view them. Also, we checked students'
reading comprehension scores, that is,
how many questions they answered
correctly in their first and subsequent
trials. Then we created a graph for
each student (see Figures 1-8) show-
ing when, during the reading, he or
she selected each of the visual
resources, (i.e., the pictures; the sen-
tence GSL videos, which provided
Figure 1
Viewing Behavior of Student A
Wrong Answer
Right Answer
Question GSL Video
translations of each text sentence; the
text GSL video, which provided a trans-
lation of the entire text; and the ques-
tion GSL videos, which provided
translations of each of the multiple-
choice questions; and the concept
map) as well as when and how each
student responded to the multiple-
choice questions. For example. Figure 1
shows that Student A, who spent
approximately 800 seconds with the
software, chose to view a picture at the
75th second during reading, then con-
tinued reading without using any other
resource until about the 430th second,
when the student chose to view the
concept map. A few seconds later. Stu-
dent A selected the text GSL video and

interaction with the software, the Concept Map

"think aloud" during viewing of the
Text GSL Video
pictures, and the recalls after he or
she read the texts, was videotaped. Sentence GSL Video

Picture
Data Analysis
All videotapes were transcribed by 200 400 600 800
two researchers fluent in GSL. Tran-
Time in seconds
scripts contained student "think
alouds" and text recalls in GSL as well
as researchers' notes on students' Figure 2
other actions while they used the soft- Viewing Behavior of Student B
ware (i.e., their reading of the text VWong Answer
and answering of the questions, their
Right Answer
selection of software resources, and
any other comments they made while Question GSL Video
interacting with the software). A
Concept Map
mixed research design was imple-
mented in the present study includ- Text GSL Video

ing qualitative and quantitative data, Sentence GSL Video

that is, descriptive statistics, mostly
Picture
used to support the presentation and
understanding of the qualitative data.
Eollowing is an analytical description 200 400 600 800
of data analysis of software reports. Time in seconds

VOLUME 157, No. 5, 2013 AMERICAN ANNAI.S OF THE DEAF

Figure 3 then viewed another picture. Student
Viewing Behavior of Student C
A provided three right answers in a
Wrong Answer row, then a wrong answer, then two
Right Answer
right answers. Information obtained
from the software reports was cross-
Question GSL Video checked and interpreted with the use
Concept Map of video data that contained the stu-
dents' "think alouds" and their actions
Text GSL Video -- -
when they used the software.
Sentenoe GSL Video

Picture rv Picture "Think-Aloud" Protocols

In picture "think-aloud" protocols, we
200 400
looked for evidence regarding stu-
X) 800
dents' use of picture information. In
Time in seconds
particular, we examined whether stu-
dents focused on text-relevant picture
information and whether they inte-
Figure 4 grated picture and text information.
Viewing Behavior of Student D For each student's picture "think
Wrong Answer aloud," we recorded the number of
idea units (Mayer, 1985) that were (a)
R^ht Answer included in the text and the picture
Question GSL Video and (b) included only in the picture.
"An idea unit expresses one action or
Concept Map _
event or state, and usually corresponds
Text GSL Video to a single verb clause. Thus, each idea
Sentence GSL Video unit consists of a predicateeither a
verb or a location or a time marker
Picture
and one or more arguments" (Mayer,
1985, p-71). For example, in the sen-
200 400 600 800 tence 'All students went to the moun-
Time in seconds tain and played," there are two idea
units within the sentence, because of
the two main verbs went and played.
Examples of idea units in student pic-
Figure 5 ture "think alouds" are provided in
Viewing Behavior of Student E Table 2.
Wrong Answer
Text Recalls
Right Answer
The students' GSL recalls were scored
Question GSL Video for (a) the number of text idea units
Concept Map
that corresponded to text idea units
from the original text (Mayer, 1985)
Text GSL Video
and (b) for recall accuracy (Andrews,
Sentence GSL Video Winograd, & DeVille, 1994). We first
developed one list containing all the
Picture
idea units that were included in the
two original narrative texts (Text A and
250 soo 750 1000 Text B), as well as a second list with
Time in seconds the idea units that represented the

VOLUME 157, No. 5, 2013 AMERICAN ANNALS OF THE DEAF

USE OF VISUAL MULTIMEDIA IN READING COZVIPREHENSION

Figure 6 main story elements (information on

Viewing Behavior of Student F the characters, the setting, main events
Wrong Answer or actions, and story ending). Text A (a
grade-level 2 passage) contained 15
Right Answer
idea units (5 of which were main story
Question GSL Video elements) and Text B (a grade-level 4
passage) contained 31 idea units (of
Ctoncept Map-
which 10 were main story elements).
Text GSL Video- Then, each student's recall protocol
was divided into idea units that were
Sentence GSL Video -
compared to the idea units from the
Roture original text. When an idea unit in the
recall protocol was similar to the idea
300 600 900 1200 1500 unit from the original text, that idea
unit was scored as "present." The recall
Time in seconds
accuracy of each idea unit was scored
as follows:
Figure 7
Viewing Behavior of Student G 1. An idea unit in the recall proto-
col received 3 points if it was a
close paraphrase of the original
idea unit and included both the
main point and the details of the
original idea unit.
2. An idea unit in the recall proto-
col received 2 points if it fell
between a close and a far para-
phrase and included only the
main point and few or no details
of the original idea unit.
400 800 1200 1600 2000 3. An idea unit in the recall proto-
Time in seconds col was given 1 point if it was a
far paraphrase, partly correct, or
incomplete.
Figure 8
Viewing Behavior of Student H
Table 3 presents the scoring system
for the analysis of student recalls. For
each student we calculated the per-
centage of recalled text idea units, the
percentage of recall accuracy, and the
percentage of main text idea units
recalled, that is, the idea units that
included main story elements. Also,
we classified the idea units in each stu-
dent's recall protocol, based on
whether the units were (a) directly
drawn from the text, (b) drawn from
300 600 900 1200 1500 the pictures but not included in the
Time in seconds text, or (c) not included in the text or

VOLUME 157, No. 5, 2013 AMERICAN ANNAI^ OE THE DEAF

Table 2 that all students used the visual re-
Examples of Idea Units in Student Picture "Think Alouds" sources that were provided in the soft-
Idea units in student picture "thini< aiouds" ware during both text reading and
Text idea units included in the text Included only in the picture question answering, and they did so in
Red poppies, yellow and white There are yellow a similar manner. Figures 1-8 present
daisies have sprouted everywhere. flowers everywhere. information on each student's viewing
All trees are in full leaf. The tree branches are behavior. As the figures show, students
in fuli leaf. tended to follow the same viewing pat-
I can see a bird's nest. tern, which was partly demonstrated
The kids are biking happily. by the researcher during the demon-
stration of the software and implicitly
prompted by the software: Students
the pictures but inferred by the stu- Results read one section of the text at a time,
dent. Next, we calculated the percent- Did students use the visual resources and when they encountered a picture
age of idea units that were identified in of the See and See software during button, they stopped reading to view
each student's recall based on the lat- reading with minimal prior instruc- the corresponding picture. When stu-
ter classification. tion? Which resources of the software dents zoomed into a picture, they had
Scoring of all "think alouds" and did they tend to use? Students did use access only to the picture and not to
recalls was done independently by the the visual resources during reading the text, because the picture window
first two authors. Interrater reliability with minimal instruction, and they was enlarged, covering the whole com-
ranged from .87 to .93, and conflicting tended to use similar resources. Our puter screen. After zooming out of the
judgments were resolved through analysis of students' viewing behavior picture, students continued reading the
discussion. based on the software reports shows next text segment. Only three instances

Table 3
Coding System for Analyzing Student Recalls: Examples of Idea Units in the Text and in Student Recalis

Examples of idea units in student recalis

Drawn from Inferred by
Drawn from the text the pictures the students
Close paraphrase Near paraphrase Far paraphrase
Text idea units (3 points) (2 points) (1 point)

All the kids went to
the nearest mountain.
All trees are in full leaf.
Red poppies, yellow
and white daisies have
sprouted everywhere.
J1WE.
The kids are at the Some kids go to the
mountain, nearby. slope of the mountain.
In the meadow there are There are trees.
trees with thick leaves.
There are many flowers Some yellow flowers They looked at the
and they are yellow and
white, there are so many!
have sprouted, so nice! various flowers.
W
The kids are
biking.
The swallow
picked up a
worm with

I I1 its beak.
Their fathers
agreed to let .

I1, them play

whatever they
1 wanted.

VOLUME 157, No. 5, 2013 AMERICAN ANNAI^ OF THE DEAF

USE OF VISUAL MULTIMEDL^ IN READING COMPREHENSION

of the use of the sentence GSL videos See and See during reading? Did they was also idendfied while students were
during reading were identified: Stu- integrate the information from these viewing the concept map, which they
dents B, C, and F each chose to view resources during reading? While view- read after reading the entire text. They
the sentence GSL videos when tbey ing the pictures, six students "thought read the concept map, but they never
encountered unknown words or aloud"; that is, they described or went back and forth between the text
phrases. The rest ofthe students, who expressed their thoughts regarding and the concept map to integrate the
also could not identify the meaning of the pictures in GSL. Two students information from the two sources.
some words or phrases, did not chose viewed the pictures but did not "think They just read the concept map and
to watch the sentence GSL videos. aloud." Table 4 shows the total number then proceeded to the reading com-
However, after reading the entire text, of idea units produced by each stu- prehension questions.
all students viewed the GSL video for dent, as well as the percentages of text- Taken together, tbesefindingsindi-
the whole text and the concept map. related and picture-related idea units. cate that students did attend to the pic-
Following text reading, students Our analysis of students' picture "think- tures. Also, students' inspections did not
moved on to the multiple-choice com- aloud" protocols shows that the pro- appear to be guided only by text ideas.
prehension quesdons. While attempt- portion of idea units that were related Rather, their attention was also dis-
ing to understand the meaning of the to the text ranged from 39.5% to tracted by picture elements irrelevant to
questions and their available answers, 80.0%, which indicates that when stu- the text's content. Finally, students did
five students (A, B, C, E, and G) chose dents inspected the pictures they did not try to coordinate information from
to view at least one of the GSL videos locate text-relevant information. How- the text and the pictures.
that corresponded to the question and ever, students also attended to infor- Did the use ofthe visual resources
its answers. Also, Student C returned mation within the pictures that was of See and See aid reading compre-
to the text and reviewed a sentence not related to the text. The proportion hension? We gathered information
GSL video. In their second attempt of idea units that corresponded to the about children's reading comprehen-
and after giving a wrong answer, all stu- pictures' content but were not related sion performance after one training
dents except Student D, who replied to the text ranged from 20.0% to session to see if the use of visual
correctly in the first attempt, re-read 60.5%. Our data on students' viewing resources aided reading comprehen-
the text and reviewed the visual behavior, based on the software re- sion. Text comprehension evaluation
resources. Specifically, two students ports and the videotape transcripts, was based on students' text recalls and
(A and B) re-read part ofthe texts, one indicate that, regardless of whether their performance on the reading
student (C) reviewed a short sentence their descriptions were text related or comprehension questions (see Table
GSL video, three students (E, F, and picture related, students did not try to 5). We found that students replied cor-
H) reviewed the concept map, and compare picture and text information rectly to three (60%) or four (80%) out
one student (G) reviewed the pictures by going back to read the correspon- of the five reading comprehension
and the concept map. Students F, G, ding part of the text and then back questions in their first attempts. Our
and H, wbo read tbe longer passage, again to the pictures to review them in analysis of students' text recall proto-
revisited the concept map several relation to the text. A similar pattern cols shows that, overall, students were
times in their second attempts to
answer questions (see Figures 6-8).
Tabie 4
In summary, during reading, the stu- Picture Think-Aloud Protocol Data, by Student
dents viewed most visual resources one
Student Idea units (N) Text-related idea units (%) Picture-related idea units (%)
after another, in a sequence tbat was
Text A
implicitly prompted by the software. In
their attempts to reply to the reading
comprehension questions, they rarely
re-read text sections or reviewed the
visual resources, but they made more
selections of specific resources when
their trials were unsuccessful.
H 66.6 33.4
Wbat information did the students
Note. Students F and G did not think aloud.
extract from the visual resources of

VOLUME 157, No. 5, 2013 AMERICAN ANN.M.S OF THE DEAF

quite successful at comprehending the Table 5
main ideas of the text they had read, as Individual Students'Text Comprehension Soores
seven of them recalled 8096-100% of Correct Comprehension Comprehension
main story elements (see Table 5). In responses to of fuil-story of main-idea Recaii accuracy
addition, students recalled 53.3%-82.4% Student questions (%) idea units (%) units (%)
of the total amount of text idea units, Text A
and had a recall accuracy ranging from
50.0% to 83.3%. Our analysis showed
that students recalled not only text-
related information but also informa-
tion that was drawn exclusively from
the pictures or inferred by the stu-
dents themselves. The percentage of
picture-drawn idea units ranged from
5.5% to 27.3%, and the inferences con-
stituted 8.3% to 40.0% of the total
amount of idea units in students'
recalls (see Table 6). Table 6
Students demonstrated similar Student Text Recali Data
viewing behaviors and comparable Text-based Picture-based Inferred-idea
reading comprehension performance Student idea units (%) idea units (%) units (%)
in terms of the number of main-idea Text A
and full-story idea units they produced
in their GSL recalls, as well as the num-
ber of correctly answered questions.
Given the small number of study par-
ticipants and the lack of a comparison
group, it is hard to detect a striking
pattern indicating a relationship be-
tween students' use of the visual
resources and their text comprehen-
sion. However, a closer examination of
each individual student's viewing
behavior and reading performance systematic instruction took place, and, whether the selection of the resources
indicates that in some instances stu- as we discuss below, it is most likely was motivated by a general curiosity to
dents benefited from the use of the that the students would have bene- find out "what was there" or the stu-
resources. In particular, five students fited from multiple sessions of instruc- dents' need to monitor and augment
(C, E, F, G, and H) who gave wrong tion on the software to learn how to their text understanding. The beneficial
responses to some of the comprehen- use the feedback build into it to their role of the visual resources was more
sion questions appeared to benefit best advantage. evident when students replied to the
from their subsequent return to the reading comprehension questions. It is
text and the use of the visual re- Discussion possible that the limited instruction
sources, given that they were success- Regarding the two first research ques- regarding the software did not enable
ful in their following attempts. On the tions in the present study, our data indi- students to understand how to use the
other hand, in certain cases the inap- cated that students enjoyed and used visual resources strategically so that
propriate use of the resources, and most of the visual resources in the they could more effectively increase
specifically the emphasis on picture software, with which they interacted their reading comprehension.
elements that were not relevant to the independently without following spe- Specifically, during reading, stu-
text's content, affected their text recall- cific teacher guidelines. Regarding the dents chose to see the pictures, but
ing. It is worth noting, though, that no third question, it was not always clear when they inspected them, they did

VOLUME 157, No. 5, 2013 AMERICAN ANNALS OF THE DEAF

USE OF VISUAL MULTITVEDIA IN READING COMPREHENSION
not look only for specific text informa-
tion; rather, they tended to pay atten-
tion to everything that appeared
interesting to them, even if it was not
necessarily relevant to the text. These
particular pictures were designed so
as to correspond closely to text con-
tent, but they also included a few
"background" elements (e.g., sun and
clouds in the sky) that were not part of
the narrative text and not important
for understanding its meaning. Stu-
dents identified information that could
be identified within the text but also
paid attention to picture elements that
were irrelevant to the text. The per-
centages of text-unrelated idea units
in the protocols of Students C and E
were as high as 60.5 and 52.4, respec-
tively showing that these students dis-
tributed their attention nearly equally
among relevant and irrelevant picture
elements. Furthermore, students' be-
havior while viewing the visual re-
sources indicated that they did not
attempt to relate the information from
the visual resources to the text. Specifi-
cally, while viewing the pictures, stu-
dents did not go back to the text to
read the highlighted part correspon-
ding to the picture, and did not review
the picture again in order to compare
it to the information in the text. It is
possible that students remembered
text information and therefore could
relate picture to text information with-
out returning and re-reading the rele-
vant part of the text. Nevertheless,
their attention to elements both rele-
vant and irrelevant to the text picture
indicates that they treated pictures as
extensions of the story they read and
not as alternative representations of
the same text content. Furthermore,
their viewing behavior with pictures
affected their text recalls, which were
based on the text but also partly on
content drawn exclusively from the
pictures. A similar nonstrategic view-
ing behavior was also identified when
VOLUME 157, No. 5, 2013
students viewed the concept map.
Although all students viewed and read
the information within the concept
map, they never attempted to relate
the concept map to the text by going
back to the text, searching for and
identifying the information in the con-
cept map within the text, and then
returning to the concept map.
Among all the visual resources, the
GSL sentence videos were rarely used.
The students preferred to watch the
whole GSL video after reading the
entire text instead of using the sen-
tence GSL videos during reading to
repair possible misunderstandings of
text meaning. Their viewing behavior
might indicate that the students did
not monitor their reading comprehen-
sion and possibly were not aware of
their reading comprehension difficul-
ties. Another interpretation could be
that they found the whole GSL video
more helpful. Instead of interrupting
their reading, they preferred to read
the entire text and then watch the
whole GSL video tofillin possible gaps
in their understanding. While the
whole GSL video was in operation, the
corresponding part of the text was
highlighted on the basis ofthe tempo-
ral contiguity principle (Mayer, 2005b);
this helped them understand which
part of the text related to the video.
Students' behavior was somewhat
different when they replied to the
reading comprehension questions. In
theirfirstattempt to reply students did
not return to the text and its accompa-
nying visual resources to search for
information that could help them an-
swer the questions. Students used the
GSL question videos corresponding to
each question and its answers, which
helped them understand the meaning
ofthe questions and the answers, and
then replied immediately In their sec-
ond trial, though, after replying incor-
rectly, they returned to the text to
inspect several text sections and also
used the visual resources, mostly the
concept map. This behavior maybe an
indicator of the low levels of metacog-
nitive skills that deaf students often
appear to have (Strassman, 1997).
Although their preference to review
the concept map possibly indicates
that the students were aware that the
map included all the required informa-
tion to reply to the questions, they
decided to review the concept map
and re-read some sections in the text
in their second attempt to reply to the
questions. Wood Griffiths, and Webster
(1981) found that deaf children did not
use a feeling-of-knowing judgment
before answering. In a study by Davey
(1987), deaf students seemed unaware
of the role of the look-back strategy,
although they replied correctly to
more questions when they looked
back to the text. Also, in the same
study they used the look-back strategy
mostly to complete a task, while, in a
study by LaSasso (1986, 1987), deaf
students used the look-back strategy
as a visual matching technique, and
not as a metacognitive strategy. The
participants in our study probably
became aware of their comprehension
gaps after giving the wrong answer, or
became aware of the beneficial role of
the concept map when they were
prompted by the software to go back
to the text and search for the right
answer. The prompts provided by the
software encouraging students to go
back to the text and search for the
right answer might have encouraged
them to use the concept map and
helped them to realize that its role was
not to supplement or decorate the text
but to help them comprehend the
text.
We take this combined information
to mean that in addition to learning
how each of the resources in See and
See functions, children need to ana-
lyze the new information they have
gleaned with reference to their current
AMERICAN ANNALS OF THE DEAF
decisions about an answer, and either
to keep the answer they have or mod-
ify it on the basis of the new informa-
tion. Therefore, multiple instructional
sessions with scaffolded instruction
should be given before students are
asked to strategically implement and
exploit this software independently. In
the present study, such instructional
sessions did not take place, which was
a limitation. The information gleaned
in our study suggests that students
should receive multiple sessions of
instruction so that they may
1. determine whether visual infor-
mation is pertinent or extrane-
ous to the print information
2. carefully implement the use of
each visual resource (i.e., GSL,
pictures, concept maps)
3. compare and contrast informa-
tion between visual resources and
information in the text to accept
or reject their decisions about
information they have gleaned
Furthermore, sessions of instruction
need to focus on teaching visual strate-
gies regarding each visual aid so that
students become aware of the role of
visual resources and develop specific
strategies for using them to monitor
their text comprehension and repair
their misunderstandings. Future stud-
ies could explore the effectiveness of
teaching visual strategies, some of
which are presented below.
Regarding the use of pictures, strate-
gies might involve searching for text-
relevant information in the pictures,
examining whether information per-
ceived in the pictures is consistent with
what one has comprehended when
reading the text, forming hypotheses
about unknown vocabulary or expres-
sions based on picture content, and,
when pictures are viewed before print
information is read, making hypothe-
ses about text content. Similarly, future
VOLUME 157, No. 5, 2013
studies could examine the role of the
concept map in the comprehension of
a text by deaf and hard of hearing stu-
dents, either as an advance organizer
before the text is read, as a means of
forming hypotheses about text con-
tent, or, after the text is read, as a
means for students to evaluate their
comprehension.
GSL strategies might help deaf and
hard of hearing students who know
GSL to use GSL videos effectively GSL
sentence videos may help students who
know GSL to understand unknown
words or difficult sentences, or to eval-
uate their hypothesis about the mean-
ing of particular words or sentences,
while whole-text GSL videos might help
students comprehend the content of
the whole text. However, it is important
that students be taught GSL-specific
strategies so that they might use the
GSL videos effectively. For example,
strategies might involve forming hy-
potheses about unknown words or the
meaning of text parts and then evaluat-
ing these hypotheses using the GSL
videos.
According to Filippatou and Pum-
frey (1996), limited or inappropriate
use of visual resources is attributable
to low levels of student awareness of
the role of visual resources and their
use in reading comprehension. Visual
resources and adjunct instructional
aids may play a dynamic role in learning
for deaf students (Dowaliby & Lang,
1999; Lang & Steely, 2003); however,
all students need to know how to use
visual resources appropriately (Carney
& Levin, 2002), regardless of whether
they are deaf or hearing. Therefore,
there is a need for systematic instruc-
tion in "visual literacy" skills so that stu-
dents can learn how to process visual
aids and extract more and better infor-
mation from them, so as to increase
their reading comprehension (Hannus
& Hyn, 1999; Nikolaraizi & Vekiri,
2012; Peeck, 1993).
As we have already mentioned,
such instruction did not take place in
the present study, and the students
probably had not acquired any related
experience from their teachers. There-
fore, future intervention studies need
to examine whether systematic in-
struction in visual literacy skills and
strategies, some of them analytically
presented above, can help deaf stu-
dents learn to use visual resources
effectively to enhance their text com-
prehension. Also, additional research
is needed to investigate whether one
of the resources of the See and See
software is more effective than another,
or whether a single instruction package
consisting of all the components is
most effective. Research is also required
to determine how much instruction is
necessary for students to become effec-
tive and efficient users of the software.
Finally, research with a larger pool of
participants should be implemented
using pre- and post-study assessments
of intervention and control groups.
Note
Correspondence concerning the pres-
ent article should be addressed to
Magda Nikolaraizi, Assistant Professor
in Special Education-Education of the
Deaf, Department of Special Educa-
tion, University ofThessaly, Argonafton
& Philellinon, 38221, Volos, Greece.
E-mail: mnikolar@uth.gr
References
Andre, T. (1979). Does answering higher-level
questions while reading facilitate productive
learning? Review of Educational Research,
49, 280-319.
Andrews, J., & Mason, J. (1991). Strategy usage
among deaf and hearing readers. Excep-
tional Children, 57, 536-545.
Andrews, J., Winograd, E, & DeVille, G. (1994).
Deaf children reading fables: Using ASL sum-
maries to improve reading comprehension.
American Annals of the Deaf, 139,378-386.
Boone, R., & Higgins, K. (2007). The role of
instructional design in assistive technology
research and development. Reading Research
Quarterly,42,135-140. doi:10.1598/RRQ.42.1
AMERICAN ANNALS OF THE DEAF
USE OF VISUAL MULTIMEDIA IN READING COMPREHENSION
Byi'nes, J. (1996). Cognitive development and
/earning in instructional contexts. Need-
ham Heights, MA; Allyn & Bacon.
Cannon, J., Easterbrooks, S., Gagn, K, & Beal-
AlvarezJ. (2011). Improving DHH students'
grammar through an individualized software
program. Journal of Deaf Studies and Deaf
Education, 16, Ail-A5~l
Carney, R. N., & Levin, J. R. (2002). Pictorial illus-
trations still improve students' learning from
text. Educational Psychology Review, 14,
5-26.
Castillo, E., Mosquera, D., & Palacios, D. (2008).
Concept maps: A tool to enhance reading
comprehension skills of children with hear-
ing impairments. Retrieved from http://cmc
.ihmc.us/cmc2008papers/cmc2008-p247.pdf
Chang, K-E, Sung, Y-T, & Chen, I-D. (2002). The
effect of concept mapping to enhance text
comprehension and summarization./owrw/
of Experimental Education, 71, 5-23.
Chmielewski, T. C, & Dansereau, D. F. (1998).
Enhancing the recall of text: Knowledge map-
ping training promotes implicit transfer./owr-
nal of Educational Psychology, 90, 407-413.
Davey, B. (1987). Postpassage questions: Task
and reader effects on comprehension and
metacomprehension processes. Journal of
Reading Behavior, 19, 261-283.
Dowaliby, E (1992). The effects of adjunct ques-
tions in prose for deaf and hearing students
at different reading levels. American Annals
of the Deaf 137, 338-344.
Dowaliby, E, & Lang, H. (1999). Adjunct aids in
instructional prose: A multimedia study with
deaf college students. Journal of Deaf Stud-
ies and Deaf Education, 4, 270-282.
Easterbrooks, S., & Baker, S. (2002). Language
learning in children who are deaf and hard
of hearing. Needham Heights, MA: Allyn &
Bacon
Filippatou, D., & Pumfrey, P D. (1996). Pictures,
titles, reading accuracy, and reading compre-
hension. Educational Research, 38,259-291.
Gentry, M., Chinn, K, & Moulton, R. (2004/
2005). Effectiveness of multimedia reading
materials when used with children who are
deaf. American Annals of the Deaf, 149,
394-403.
Hannus, M., & Hyn, J. (1999). Utilization of
illustrations during learning of science text-
book passages among low- and high-ability
children. Contemporary Educational Psy-
chology, 24, 95-123.
Hegarty M., &Just, M. A. (1993). Constructing
mental models of machines from text and
digrms. Journal ofMemory and Language,
32, 717-742.
Hitchcock, C, Meyer, A., Rose, D., & Jackson, R.
(2002). Providing new access to the general
curriculum: Universal design for learning.
Teaching Exceptional Children, 35, 8-17.
Horney, M. A., & Anderson-Inman, L. (1999).
VOLUME 157, No. 5, 2013
Supported text in electronic reading envi-
ronments. Reading and Writing Quarterly,
15, 127-168.
Kelly L. (1998). Using silent motion pictures to
teach complex syntax to adult deaf learners.
Journal of Deaf Studies and Deaf Educa-
tion, 3, 217-230.
Kelly, L. (2007). The comprehension of skilled
deaf readers: The roles of word recognition
and other potentially critical aspects of com-
petence. In K. Cain &J. Oakhill (Eds.), Chil-
dren 's comprehension problems in oral and
written language: A cognitive perspective
(pp. 244-280). New York, NY: Guilford Press.
Kelly R., Albertini,J., & Shannon, N. (2001). Deaf
college students' comprehension and strat-
egy use. American Annals of the Deaf, 146,
385-399.
Lang, H., & Steely D. (2003). 'Web-based science
instruction for deaf students: 'What research
says to the teacher Instructional Science,
31, 277-298.
LaSasso, C. (1986). A comparison of visual
matching test-taking strategies of compara-
bly aged normal-hearing and hearing-
impaired subjects with comparable reading
levels. Volta Review, 88, 231-241.
LaSasso, C. (1987). Survey of reading instruction
for hearing-impaired students in the United
States. Volta Review, 89, 85-98.
Levin, J. R., Anglln, G. J., & Carney, R. N. (1987).
On empirically validating functions of pic-
tures in prose. In D. M. Willows cist H. A.
Houghton (FAs.), The psychology of illustra-
tion: Volume 1. Basic research (pp. 51-85).
New York, NY: Springer
Loeterman, M., Paul, P, & Donahue, S. (2002).
Reading and deaf children. Reading Online,
5. Retrieved from http://www.readingonline
.org/articles/loeterman/index.html
Luckner, J., Bowen, S., & Carter, K. (2001). Visual
teaching strategies for students who are deaf
or hard of hearing. Teaching Exceptional
Children, 33, 38-44.
Marschark, M. (2005). Classroom interpreting
and visual information processing in main-
stream education for deaf students: Live or
Memorex? American Educational Research
Journal, 42, 727-776.
Manschark, M., Lang, H., & Albertini, J. (2002).
Educating deaf students: Erom research to
practice. New York, NY Oxford University
Press.
Marschark, M., Sapere, R, Convertino, C, Mayer,
C, 'Wauters, L., & Sarchet, T. (2009). Are deaf
students' reading challenges really about
reading? ^mencfl Annals of the Deaf 154,
357-370. doi:10.1353/aad.0.0111
Mayer, R. E. (1985). Structural analysis of science
prose: Can we increase problem-solving per-
formance? In B. K. Britton &J. B. Black (Eds.),
Understanding expository text: A theoretical
and practical handbook for analyzing
explanatory text (pp. 65-87). Hillsdale, NJ:
Erlbaum.
Mayer, R. E. (2005a). Principles for managing
essential processing in multimedia learning:
Segmenting, pretraining, and modality prin-
ciples. In R. E. Mayer (Ed.), The Cambridge
handbook of multimedia learning (pp.
169-182). New York, NY Cambridge Univer-
sity Press.
Mayer, R. E. (2005b). Principles for reducing
extraneous processing in multimedia learn-
ing: Coherence, signaling, redundancy spa-
tial contiguity, and temporal contiguity
principles. In R. E. Mayer (Ed.), The Cam-
bridge handbook of multimedia learning
(pp. 183-200). New York, NY: Cambridge
University Press.
Mayer, R. E., Bove, 'W, Bryman, A., Mars, R., &
Tapangco, L. (1996). 'When less is more:
Meaningful learning from visual and verbal
summaries of science textbook lessons.
Journal of Educational Psychology, 88,
64-73.
McTigue, E. M. (2009). Does multimedia learn-
ing theory extend to middle-school students?
Contemporary Educational Psychology, 34,
143-153.
Nikolaraizi, M., & Vekiri, I. (2012). The design of
a software to enhance the reading compre-
hension of deaf children: An integration of
multiple theoretical perspectives. Educa-
tion and Information Technologies, 17,
167-185.
Paul, P (199S). Literacy and deafness: Thedevel-
opment of reading, writing, and literate
thought. Needham Heights, MA: Allyn &
Bacon.
Peeck, J. (1993). Increasing picture effects in
learning from illustrated text. Learning and
Instruction, 3, 227-238.
Reynolds, H., & Rosen, R. (1973, February/
March). The effectiveness of textbook, indi-
vidualized, and pictorial formats for hear-
ing impaired students. Paper presented at
the meeting of the American Educational
Research Association, New Orleans, LA.
(ERIC Document Reproduction Service No.
ED 075 968)
Robbins, N. (1983). The effects of signed text on
the reading comprehension of hearing
impaired children. American Annals of the
Deaf 128, 40-44.
Rose, D., Hasselbring, T. S., Stahl, S., & Zabala,
J. (2005). Assistive technology and universal
design for learning: Two sides of the same
coin. In D. Edyburn, K. Higgins, & R. Boone
(Eds.), Handbook of special education
technology research and practice (pp.
507-518). 'Whitefish Bay, WI: Knowledge by
Design.
Rothkopf, E. Z. (1970). The concept of math-
emagenic activities. Review of Educational
Research, 40, 325-336.
AMERICAN ANNALS OF THE DEAF
Schirmer, B. (2000). Language and literacy
development in children who are deaf. Need-
ham Heights, MA: Allyn & Bacon.
Schirmer, B. (2003). Using verbal protocols to
identify reading strategies of students who
are ds^t Journal of Deaf Studies and Deaf
Education, 8, 157-170.
Schnotz, W, Picard, E., & Hron, A. (1993). How
do successful and unsuccessful learners use
texts and graphics? Learning and Instruc-
tion, 3, 181-199.
Stewart, D., & Kluwin, T. (2001). Teaching deaf
and hard of hearing students. Needham
Heights, MA: A)lyn ec Bacon.
Strassman, B. (1997). Metacognition and reading
in children who are deaf: A review of the
reseas-ch. Journal of Deaf Studies and Deaf
Education, 2,140-141.
Sweller, J. (2005). The redundancy principle in
multimedia learning. In R. E. Mayer (Ed.),
Tbe Cambridge handbooiz of multimedia
learning (pp. 159-166). New York, NY: Cam-
bridge University Press.
Tobin, R. (2008). Conundrums in the differenti-
ated literacy classroom. Reading Improve-
ment,45, 159-169.
VOLUME 157, No. 5, 2013
Tomlinson, C. (1999). The differentiated class-
room: Responding to the needs of alt
learners. Alexandria, VA: Association for
Supervision and Curriculum Development.
Tomlinson, C, Brighton, C, Hertberg, H., Calla-
han, C, Moon, T, Brimijoin, K., et al. (2003).
Differentiating instruction in response to stu-
dent readiness, interest, and learning profile
in academically diverse classrooms: A review
of literature. 7oMma//or the Education of
the Gifted, 27, 119-145.
Traxler, C. (2000). The Stanford Achievement
Test, ninth edition: National norming and
performance standards for deaf and hard of
hearing students./oMrwa/ of Deaf Studies
and Deaf Education, 5,337-348. doi: 10.1093/
deafed/5.4.337
Van Garderen, D., & Whittaker, C. (2001). Plan-
ning differentiated, multicultural instruction
for secondary inclusive classrooms. Teach-
ing Exceptional Children, 38,12-20.
Vygotsky, L. (1978). Mind in society: The devel-
opment of higher psychological processes
(M. Cole, Y John-Steiner, S. Scribner, & E.
Souberman, Eds.). Cambridge, MA: Harvard
University Press.
Vygotsky, L. (1986). Thought and language.
Cambridge, MA: MIT Press. (Original work
published 1962)
Walker, L., Munro, J., & Richards, E (1998). Teach-
ing inferential reading strategies through pic-
tures. Volta Review, 100, 105-120.
Wang, Y, & Paul, E (2011). Integrating technol-
ogy and reading instruction with children
who are deaf or hard of hearing: The effec-
tiveness ofthe Cornerstones Project. Amer-
ican Annals ofthe Deaf 156, 56-68.
Wilson, T, & Hyde, M. (1997). The use of Signed
English pictures to facilitate reading compre-
hension by deaf students. American Annals
of the Deaf 142, 333-341.
Wood, D. J., Griffiths, A. J., & Webster, A.
(1981). Reading retardation or linguistic
deficit? II: Test-answering strategies in
hearing and hearing impaired schoolchild-
ren./Orna/ of Research in Reading, 4,
148-156.
Yoong,J-O, & Kim, M. (2011). The effects of cap-
tions on deaf students' content comprehen-
sion, cognitive load, and motivation in online
learning. American Annals ofthe Deaf 156,
283-289.
AMERICAN ANNALS OE THE DEAF
Copyright of American Annals of the Deaf is the property of American Annals of the Deaf and its content may
not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written
permission. However, users may print, download, or email articles for individual use.