NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

VOLUME 31, NUMBER 3, 2014

20
FACTORS AFFECTING THE
IMPLEMENTATION OF SHELTERED
INSTRUCTION OBSERVATION
PROTOCOLS FOR ENGLISH LANGUAGE
LEARNERS

Carlos Trevino Calderon
Ernest Zamora
Walden University


ABSTRACT

This article summarizes a sequential mixed methods case study of the influence
of teacher attitude on the implementation of Sheltered Instruction Observation
Protocols (SIOP). The study explored how those attitudes affect the programs
effectiveness. The qualitative component of the study was used to identify two
comparison groups of teachers and the quantitative component used
independent sample t tests of reading and math exams to compare the academic
performance of students across the two teacher groups. While the results of the
t tests, comparing mean academic performance, were not statistically
significant, the findings demonstrated a slight advantage to those students who
were taught by teachers with more positive attitudes toward the SIOP model.
The social change implications include improved academic performance of ELL
students by informing school leaders decisions about meaningful professional
development, customizing curricula, building awareness among course
developers in teacher preparation institutions about the needs of second
language learners, developing current and meaningful program offerings, and
promising delivery practices for teachers.

Introduction

he term English Language Learner (ELL) refers to students
who are in the process of acquiring English because they have
another language as their native and primary method of T
Carlos Trevino Calderon & Ernest Zamora 21
communication. Texas has the second largest number of ELL students
in the nation with only California surpassing it (Fry, 2007). In the
2007-2008 school year, 16.6 % of the total student population enrolled
in Texas public schools (775,432 students) were classified Limited
English Proficient (LEP) (Texas Education Agency, 2009). From 1997
to 2008, public school enrollment in Texas increased 19.8%, but the
number of LEP students grew by 49.1% (Texas Education Agency,
2009). In Texas, 78% of all students receive a regular high school
diploma, but only 39% of ELLs achieve the same (Zehr, 2010). The
repercussions that result from an individuals failure to attain a high
school diploma include an estimated $250 billion cost to the federal
government as a result of lost wages, lost tax revenue, and a greater
reliance on social services (Stanard, 2003). With 80% of ELLs being
Latino and the workforce predicted to be increasingly Latino, this data
is troubling for both Texas and for the nation (Echaveste & Edley,
2009). There is a diminished quality of life for the high school
dropout, which includes a lower standard of living and oftentimes
involvement in criminal activity. Stanard (2003) showed that 82% of
the prison population is comprised of high school dropouts. The
institutionalization rate for high school dropouts is 63 times greater
than that for college graduates (Sum, Khatiwada, McLaughlin, &
Palma, 2009).

This study delved into the issue of underperformance by ELL
students by looking at the effects of Sheltered Instruction Observation
Protocols (SIOP) implementation in the classroom (Echevarria, Vogt,
& Short, 2008a). More specifically this study attempted to discern how
teacher attitude towards the implementation of SIOP, and other similar
ELL interventions, affects the effectiveness of the strategies and in
turn ELL performance. The results of this study offer schools and
school districts information which they can use to plan staff
development, modify curricula, and identify instructional support
paradigms to better meet the needs of all students.



22 NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

Nature of the Study

This inquiry studied the students and teachers of a small school
district in the Lower Rio Grande Valley of Texas. According to the
latest information on the districts website, Study ISD has an
enrollment of 5,516 students. Of the total district population, 99.1% of
the students are of Hispanic descent and 29.4% are classified ELL.The
study included the 5
th
grade ELL students in the districts four
elementary schools and their teachers. The sample for this study was
nonprobability, purposive sampling and included 222 students and 12
teachers.

Looking both at causality and meaning, the study included both
qualitative and quantitative features. The methodology for this study
involved the collection and analysis of several sources of evidence
including interviews, surveys, documents, classroom observations, and
data from Data Management for Assessment and Curriculum (DMAC)
Solutions. DMAC Solutions is a web-based software suite used by the
district for the management of curriculum and assessment data,
including the 6 weeks Curriculum Based Assessments (CBA).


Research Questions

The guiding question for this sequential, mixed methods study
was whether or not a teachers attitude towards SIOP and other ELL
strategies affected the implementation of those programs and
consequently their effectiveness. In order to address this guiding
question, qualitative data was collected through surveys, observations,
and interviews. The qualitative questions addressed were:

1. What perceptions do teachers of ELL students have
regarding their ability to support the academic achievement
of ELLs?
2. What perception do teachers of ELL students have
Carlos Trevino Calderon & Ernest Zamora 23
concerning SIOP and other strategies or interventions
designed to address the needs of ELLs?
3. How does a teachers perception of SIOP affect
implementation of the program?

The goal of the qualitative component was to discover
teachers feelings and attitudes towards the SIOP intervention and
their having to use it as a strategy when working with ELL students.
This information assisted in the identification of teachers with a more
positive attitude towards the strategy and those who demonstrated
evidence of resistance and apprehension towards using it. The
qualitative component allowed for the identification of two distinct
groups for a comparison.

The second phase of the study was the quantitative component,
and it involved searching for a relationship between the two groups
using an analysis of performance data. This phase of the study
addressed the following question using test data from the districts 6
week CBA exams.

1. How does a teachers implementation of SIOP relate to the
performance of ELL students?

Information from the two components of the study was used to
deduce whether or not a significant relationship existed between the
academic performances of the student participants in each group. The
first stage, by way of its causal aspect, provided information on
teacher attitude: a factor that can influence teacher fidelity to the
research-based SIOP Model. The second part used assessment data to
substantiate if there was a link between a more positive teacher
attitude towards SIOP and greater numbers of ELL students meeting
the passing standard on the CBA exams. Together the two components
provided insight into reasons affecting ELL success or lack of it.
Using both qualitative and quantitative data provided the study with
triangulation to increase its credibility and validity.

24 NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

Literature Review

The key ideas and terms for this literature review included:
school reform, effective bilingual and ESL programs, teacher attitude
toward ELLs, program implementation resistance, change
management, leadership support, role of leadership in school change,
organizational health, and effective schools. These areas of interest
evolved into three distinct themes. The first of these themes is
organizational health and school climate, focusing on the particulars
that form the culture of a school including: norms, values, rituals,
beliefs, and traditions. School culture was further researched to find
how it impacts the dynamics of interacting and teaching ELLs. The
second theme was how teacher attitudes, perceptions, and expectations
can affect the delivery of instruction and the effectiveness of those
efforts on student performance and achievement. The third theme
related to instructional support and practices for ELL students and the
effectiveness of those strategies in promoting achievement and success
for the targeted population.

If teachers are unable to relate to the circumstances of their
ELL students they may not completely comprehend the need for some
of the interventions and strategies designed for them and fail to
implement them completely (Washburn, 2008). The failure in
complete implementation may result first and foremost in
underperformance by the ELL student and second in the intervention
being classified as inadequate. Once a teacher forms the perception
that a research based intervention is inadequate they can become
frustrated with it and this frustration can lead the teacher to utilize
other strategies. The methods they tend to turn to are those with which
they are more familiar such as the ones used with them when they
were students (Singer & Maher, 2007). If they find those strategies to
be ineffective with their students then they may seek to place the
blame on the student and not with the strategy (Darling-Hammond,
2001).


Carlos Trevino Calderon & Ernest Zamora 25
The Sheltered Instruction Observation Protocols

The SIOP Model includes features of high quality instruction
for all students, along with language objectives in every lesson that
specifically address the needs of the ELL (Echevarria, et al., 2008a).
Composed of 30 features in eight components, SIOP allows teachers
to accommodate the unique second language development needs of
ELLs. By pulling together many of the good teaching practices
teachers already use and adding targeted instruction for content and
language development, SIOP enhances learning for ELL students. The
programs eight components include:

Preparation
Building Background
Comprehensible Input
Strategies
Interaction
Practice/Application
Lesson Delivery
Review/Assessment

Each of the eight components has indicators that the teacher
can refer to when insuring that the lesson will adequately make the
content comprehensible for English learners. Each component has a
rubric called the Protocol that is used by teachers and administrators to
measure how well the lesson adheres to the certain component. As
explained by (Echevarria, Vogt, & Short, 2008b), SIOP has two
purposes: the first, which they refer to as the Model, serves as the
lesson planning and delivery system and the second which they refer
to as the Protocol serves to observe, rate and provide feedback on the
lessons. Lems, Miller, and Soro (2009) elaborated on the foundational
basis of SIOP pointing out that the model is soundly grounded in
Vygotskys Zone of Proximal Development (ZPD) theory and
considers the importance of differentiating instruction to meet student
needs as well as the dynamic process of learning.
26 NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

Methodology

The purpose of this sequential, mixed methods study was to
understand teacher perceptions and attitudes towards SIOP and other
interventions designed to assist ELL students. Furthermore, the goal of
the experiment was to explore if a relationship exists between teacher
attitudes and perceptions and the academic performance of language
minority students in the classroom. The intent of this study was to
explore a problem within the confines of a small school district located
in the Lower Rio Grande Valley of Texas and identified in this study
as Study ISD.

The results of the qualitative and quantitative components of
this study were summarized to address the purpose of this study. The
hypotheses considered were whether or not there was a significant
relationship between the academic performances of the students in the
comparison groups. The null hypothesis proposed that there is no
significant relationship between the performance on six weeks CBA
exams of ELL students who have teachers with a positive attitude
towards SIOP, and those who dont. The alternative hypothesis
proposed that there is a significant relationship between the
performance on six weeks CBA exams of ELL students who have
teachers with a positive attitude towards SIOP, and those who dont.
The resulting data were used to discern if teacher attitudes and
perceptions are possible factors which affect instructional traffic in
ELL classrooms.

Setting and Sample

According to data from the district website, Study ISD had an
enrollment of 5,516 students for the 2009-2010 school year including
398 fifth graders located in four elementary schools. Furthermore, data
on Study ISDs website revealed that over 99% of the students in the
district were of Hispanic descent and 29.4% were classified ELL. The
sample for this study was of a non-probability purposive nature. The
student participants selected were chosen because they were ELL and
Carlos Trevino Calderon & Ernest Zamora 27
receiving instruction in all English environments as opposed to being
taught in their primary language. The teachers in the study were
selected because they had been trained in SIOP and were utilizing the
intervention with their ELL students and also because they consented
to participate. The quantitative analysis of data for this case study
included the test scores of all the 5
th
grade ELL students enrolled in
the districts four elementary schools that were taught by teachers who
agreed to participate in the study.

The study was quasi-experimental in its sample, looking at
students in their current setting, and exploratory in nature, attempting
to discover a relationship between the performances of two
comparison groups of ELL students. The variable which was studied
was teacher attitude toward the SIOP Model and how that attitude
affected implementation of the strategy and student performance. The
study was sequential in that it collected qualitative data on teacher
attitudes and perceptions before looking at the quantitative data
produced by student performance.


Results

Qualitative data for the exploration into the teachers
perception regarding ELLs and SIOP were gathered using a written
survey, an oral interview, and a classroom observation. The written
teacher survey consisted of four parts labeled A, B, C, and D and
collectively they were used to discover teacher attitudes and
perceptions as they relate to ELL students, SIOP, and other strategies
or interventions designed to address the needs of ELLs. In an effort to
add validity to the outcomes of the qualitative component of this
study, interviews were used to triangulate the written surveys data.
The teachers responses yielded much rich data and insight into their
perceptions regarding ELL students. The following excerpts
demonstrate some of the contrasting points of view stated by the
teachers when asked to describe their perceptions about working with
28 NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

ELLs. An example of what can be viewed as positive perceptions
includes the following statement by Teacher 1 who said:

Like I said to me there is really no difference I am teaching a
child. Basically sometimes my ELLs can excel and sometimes
I get better results from them than my kids who have been here
from the beginning of time so for me its just the child and you
take the child and you work with them and you have to use
different things like pictures, drawings, music because they are
all gifted in a certain way and once you find that and you tap
on it and you develop it.

The teacher accepts that the LEP student is coming into the
educational arena with issues that are going to present obstacles but
that does not mean that the teacher will have a less positive perception
or lower expectation of the student. For example, Teacher 2 said,
They have special needs and I have to do whatever it takes to help
them be successful. Again, this statement demonstrates recognition
that there are going to be issues and difficulties when working with
this population but that those barriers are not impenetrable. That frame
of mind contrast completely with what can be surmised when Teacher
3 said that working with ELLs was, Very hard, harder I think than
working with regular students. Even the use of the term regular
students to refer to non ELL students implies that something is not
right with them and maybe thats why they cant achieve.

The third source of qualitative data used in this stage of the
study was the classroom observation. The observation, more than the
written survey or the interview, allowed the researchers the best
opportunity to see what really goes on in the classroom. In an effort to
minimize disruption by the presence of an outsider, the teacher and
students were observed from the rear of the class and a copy of the
SIOP Protocol was used for each observation to jot down field notes of
what was observed. As soon as possible, after each visit, and using the
field notes from the SIOP Protocol, details were filled in to compose a
more complete description of what was witnessed.
Carlos Trevino Calderon & Ernest Zamora 29
The results suggested that five of the teachers had attitudes and
perceptions that were not as positive, in regards to the SIOP Model
and ELL students ability to be successful, as the other seven teachers.
They were also not observed to implement the SIOP strategies as
consistently as the other teachers. These five teachers were grouped
and labeled ProALT. The other seven teachers demonstrated
themselves to have a more accepting attitude towards the SIOP Model
and were observed to implement it with more fidelity to design. They
also related a more positive and caring attitude towards ELL students
along with a sturdy belief in their ability to be a positive force in the
lives of their students. Consequently, they were included in the group
labeled ProSIOP.

In the quantitative component two independent-measures t test
were conducted using the same samples and the same hypotheses. The
reason for the two measures was to collect data from the two content
areas: reading and math. The CBA test, which is the local assessment
used to generate quantitative data, is administered on separate days for
the different content areas and some students may be present for one
test and absent for the other and vice versa. This information is
provided to explain the slight discrepancy in testers for the two content
areas. The null hypothesis for the comparison states that there is no
significant relationship between the performance on six weeks CBA
exams of students in ProALT and ProSIOP groups. The alternative
hypothesis states that there exists a significant relationship between
the performance on six weeks CBA exams of students in ProALT and
ProSIOP groups.

Data from the comparison for reading demonstrated that the
ProSIOP group had a mean which was slightly higher than the
ProALT group but the difference was not statistically significant. The
difference in scores for the ProSIOP group were (M=68.4, SD=16.4)
and for the ProALT group (M=67.6, SD=15.4) with conditions; t (216)
= .364, p = .716 and equal variances assumed. These results suggest
a small positive effect on the ProSIOP group but not statistically
significant so the alternative hypothesis is rejected and the null
30 NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

hypothesis is retained.

Analysis of the data for the Math CBA demonstrated once
again a slightly higher mean for the ProSIOP group than for the
ProALT group but once again this difference was not statistically
significant. The difference in scores for the ProSIOP group were
(M=91.7, SD=10.9) and for the ProALT group (M=90.8, SD=12.4)
with conditions; t (220) = .513, p=.608 and equal variances assumed.
Once again the results suggest a small positive effect on the ProSIOP
group but not statistically significant so the alternative hypothesis is
once again rejected and the null hypothesis is retained.

The results of the calculations showed a slight performance
advantage to those students who are taught by teachers with more
positive outlooks to ELLs and ELL strategies such as SIOP. However,
that advantage was not enough to be statistically significant. While
positive teacher attitudes and strategies may be desirable, in this study
they did not significantly improve student outcomes.


Recommendations

As an outcome of the overall findings of this study the
following implications for social change within teacher preparation
programs in universities and other educational entities as well as in
ongoing training at the school district level, are presented.

The current model of teacher preparation, especially for
elementary school teachers, which places an emphasis on
content as opposed to instructional delivery strategies,
needs to be redesigned.
Teachers need to be taught how to provide learning
environments that are empathetic to student needs and
culturally sensitive to the values and norms of all the
students in their classrooms.
Principals and superintendents, when planning professional
Carlos Trevino Calderon & Ernest Zamora 31
developments for their teachers, need to include offerings
that address how to make content more meaningful for
second language learners.

Conclusion

If ELLs are going to be taught predominately in English then it
is imperative that efforts be made to equip their teachers with tools
that will allow them to adequately reach more of the students.
Strategies such as SIOP have the purpose of minimizing the disparity
in content integration between those students who are dominant in
English and those who are not. In this study, the SIOP Model was
examined specifically whether factors such as teacher attitude
impacted its ability to promote successful academic outcomes for
language minority students. The findings demonstrated a slight
advantage to those students who were taught by teachers with more
positive attitudes toward the SIOP strategies as compared to the
students of teachers with less enthusiasm for the program.








32 NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

References

Darling-Hammond, L. (2001). The right to learn. San Francisco, CA:
Jossey-Bass.
Echaveste, M., & Edley, Jr., C. (2009). Americas changing
demographics. In Mark Green & Michele Jolin (Eds.) Change
for America: A progressive blueprint for the 44
th
president (pp.
204-212). Philadelphia, PA: Basic Books.
Echeverria, J., Vogt, M., & Short, D. (2008a). Making content
comprehensible for English language learners: The SIOP
model. Boston, MA:Pearson Education.
Echeverria, J., Vogt, M., & Short, D. (2008b). The SIOP model for
administrators. Boston, MA: Pearson Education.
Lems, K., Miller, L. D., & Soro, T. M. (2009).Teaching reading to
English language learners: Insights from linguistics. New
York, NY: Guilford Press.
Singer, J., & Maher, M. A. (2007). Preservice teachers and technology
integration: rethinking traditional roles. Journal of Science
Teacher Education, 18(6), 955-984. doi: 10:1007/s10972-007-
9072-5
Stanard, R. P. (2003). High school graduation rates in the United
States: Implications for the counseling profession. Journal of
Counseling & Development, 81(2), 217-221.
doi:10.1002/j.1556-6678.2003.tb00245.x
Sum, A., Khatiwada, I., McLaughlin, J., & Palma, S. (2009). The
consequences of dropping out of high school. Boston, MA:
Center for Labor Market Studies.
Texas Education Agency. (2009). Enrollment in Texas public schools
2007-08. Austin, TX: Texas Education Agency. Retrieved
from http://ritter.tea.state.tx.us/research/abs2.htm
Washburn, G. N. (2007). Alone, confused and frustrated: Developing
empathy and strategies for working with English Language
Learners. Clearing House: A Journal of Educational
Strategies, Issues and Ideas, 81(6), 247-250.
Zehr, M. A. (2010). Graduation rates on ELLs a mystery: Many states,
districts dont track Data, Education Week, 29(3), 20-21.