Program Evaluation:

Spanish I-IV Professional

Learning Community
Dr. Burke EDUC6130
April 19, 2015

WK7OGBURNDPROJ

Part I. Program Analysis for Spanish I-IV Professional Learning Community

The Spanish I-IV Professional Learning Community is run in accordance with the Enumclaw School Districts early release Fridays
for professional learning collaborative time, and is run by the Enumclaw High School with Assistant Principal Kevin Smith
overseeing and evaluating these PLC members. The program was approved, and then implemented in September of 2009 (Sexton,
2009). The goals and objectives of the ESDs professional learning communities include (e)ducators committed to working
collaboratively in ongoing processes of collective inquiry and action research to achieve better results for the students they
serve(and) under the assumption that the key to improved learning for students is continuous job-embedded learning for educators.
(ESD, n.d.). The Spanish I-IV specific PLC goal, presently, is to review the target for the following week, and ensure we have
student friendly success criteria. Furthermore, our weekly goal is to create weekly opportunities for students to assess their own
learning and monitor individual progress (Spanish I-IV PLC, n.d.) which is based on the UW 5D+ Framework dimension that all
teachers of the PLC wished to improve when self-evaluating on the rubric (UW, n.d.) in September of 2013.
A description of the operations of the PLC include checking the prior weeks progress to see where each teacher ended the week
instructionally, planning the following weeks instructional activities and assessments, editing assessments to better reflect what has
been taught, and what must be evaluated, deciding which assessments will be common assessments, and subject to a data analysis
process, and evaluating data from this process to evaluate which instructional practices yield greater student achievement and
engagement. If discrepancies between teacher data are noteworthy, further evaluation of probable causes are explored, including the

group of students served, the activities, and timing of activities so that highly successful techniques and methods can be replicated by
all team members. Support for new technologies like Data Director and ActivInspire, is also a key function of the PLC.
The current outcomes of the PLC include weekly learning targets, common summative and formative assessments for each unit,
continuity of instruction across teachers as far as pacing, and materials presented. While it is impossible to create completely identical
experiences for students of different teachers due to the natures and teaching styles of each teacher, and due to the learning needs of
diverse groups of learners, this PLC helps to ensure that all students have been exposed to the same vocabulary and grammatical
concepts at the end of their first and second years of Spanish at EHS, and strives to provide equitable learning experiences for students
of the same courses.
The performance history includes data from this PLC over a four year period. Informal observations, as well as student reporting,
and diagnostic tests show a high level of success in collaboration for the purpose of ensuring all teachers teach the same grammar and
vocabulary within a two week period. Student self-reporting, and diagnostic test results beginning in 2011-2014 show the PLC has
been successful in this regard. Prior to this period, incoming second year students came in with a variety of different backgrounds and
knowledge. The first two months of the second year had to be spent re-teaching concepts that should have been covered in their first
year. In the fall of 2014 only one week was necessary to review first year concepts. Diagnostic test scores have also improved from
the first year of implementation to the fall of 2014.

The stakeholders in this program include teachers of first and second year Spanish at EHS, students of first and second year
Spanish, as well as their family members, and friends, the community, the administrative team, the district office and more remotely,
OSPI. All stakeholders are interested in all students achieving at high levels, and earning credit for the courses without the need for
credit retrieval or retaking courses. Teacher stakeholders have additional interests as presented in the goals section of this paper.
Students, parents and community members are interested in the application of the skills to benefit the community and prepare learners
for future language experiences at the college level, in travel, in work experience and in volunteer positions. These interests were
concluded from parent, community and student input as well as from college admissions requirements.
The contextual factors include a highly supportive community with a high number of Spanish speaking families, a supportive
school board (all voted for the PLC, 2 did not because they wanted the time to be longer instead of shorter, not because they did not
want the PLCs approved). The district has adopted the UW 5D+ Framework for teacher evaluation, and PLCs are used to support this
work in teacher development and student engagement and achievement goals which dictate much of the required work in the PLCs.
Administrators used the PLC leads last year to mentor and prepare teachers for the new teacher evaluation process which includes
gathering information from students, parents and community members in evaluating individual teachers. Difficulties include one
teacher sharing professional learning community time with another professional learning community. This member is often absent
completely and has recently been up to two weeks ahead of the rest of the team because he is not using the culminating projects that
the others use. This results in the rest of the team having to rush.

The key, potential ethical challenge is that I am a member of the professional learning community being evaluated. I have been the
lead of the program for two years, and this year asked a new teacher to take over while finishing up my degree. I will need to be
careful to represent all stakeholders views impartially, compare input with other PLCs, and ensure that I have truly gathered
information from key, underserved program participants in this evaluation, as well as have an impartial evaluator review my work, to
ensure minimal bias is detectable (Fitzpatrick, Sanders & Worthen, 2011).

Part II. Evaluation Model Table

EVALUATION MODEL

ADVANTAGES

DISADVANTAGES

Expertise Oriented Approach

Consumer Oriented Approach

Program Oriented Evaluation

Approach
Theory based- Objectives based

Credibility - professional
Doesnt rely on content experts
but on experts on evaluation.
Based on evaluation logic and
quantitative methods
(Fitzpatrick et al. p. 150)
Objectives-oriented is simple
(explains the logic of the
program) Is relevant to their
mission (Fitzpatrick et al, p.
166)

EXAMPLES

Does not represent the

rest of the stakeholders
Evaluator bias
Evaluator is only decision
maker

Doctoral Exams
Staff Reviews

Angies List
Consumer Reports

The focus on objectives can

cause evaluators to ignore other
outcomes of program, neglects
program description and effects
of that context on program
success or failure (Fitzpatrick et
al, p. 166)

No Child Left Behind

-Logic Models (program)

Decision Oriented Evaluation

Approach

Participant Oriented Evaluation

(uses stakeholders p. 189)
-P-PE practical
participatory evaluation to
improve use of results
- T-PE-transformative
participatory evaluation
(p.201) (evaluator is consultant
and stakeholders hold control)

Inputs activities-outputsimmediate-intermediate-long
term-ultimate outcomes
Used for accountability
(Fitzpatrick et al, p. 184)

Involves program
participants actively in
decision making.
(Fitzpatrick et al, p.
199).
Increased participant
ownership results in
higher likelihood results
will be used (Fitzpatrick
et al, p. 200)
Different stakeholders
have knowledge and
perspectives that

Is part of program
oriented evaluation and
is not an evaluation
approach in and of itself
Often the data that are collected
are not relevant to particular
programs (Fitzpatrick et al, p.
184)
Not used for program
improvement (Fitzpatrick et al,
p. 185)

Creditability
Bias
Less acceptance of study
by external audiences
(Fitzpatrick et al, p. 228)
Stakeholders do not
have knowledge of
evaluation techniques
and principles
Greater cost and time
(Fitzpatrick et al, p. 228)

CIPP (focus on program

improvement lots of
stakeholders, with
evaluator in control
(Fitzpatrick et al, p.
176)
UFE (identify intended
users ) (Fitzpatrick et al,
p. 180)- primary
purpose is to inform
decisions, need to
identify stakeholders
who care and can use
results of evaluation
(Fitzpatrick et al,. p.
178)

Examples on (Fitzpatrick et al,

p 220-221)

My Choice of Model for program

evaluation is a

synthesis/blended model of
participant-oriented program
evaluation with elements of
the CIPP model, and elements
of a consumer-oriented
approach.

evaluators do not have

(Fitzpatrick et al, p. 223)
Evaluation is a
partnership (Fitzpatrick
et al, p. 225)
Provide more
valid/useful information
(Fitzpatrick et al, p. 228)

Includes participatory
elements including
representatives from
each of the identified
stakeholder groups, and
includes these
participants throughout
the process through a
variety of tasks and
activities, including
interviews, surveys,
reviews of
information/facts, and
reactions to findings
before finalization.
Participants would not
need formal training in
evaluation.
The evaluator would
maintain the
responsibility for
credibility and clear
communication of
methods and
procedures

The evaluator has not

used this blended
method of evaluation on
a similar program
before
Greater time in working
with so many
stakeholders and cost
More communication
and project
management skills will
be needed

Pending current
evaluation
CIPP (as stated above)

The evaluators role is

that of a learner
throughout the process.
Empowerment of
underserved
stakeholders
Anticipated program
improvement,
Useful as both
formative and
summative evaluation
Part III. Evaluative Criteria

5 proposed evaluation questions for evaluating the Spanish I-IV Professional Learning Community include: are student scores
higher after implementation of PLCs after 1 year, and after 4 years, is the time used to establish best practices based on experience and
data, do teachers adopt best practices that are shared by high achieving teachers, and thus improve their students scores, does data
analysis generate valid data that is serving its intended purpose, and is the teacher collaborative time responsible for affecting student
achievement?
The standards that are reflected in these evaluation questions are based on the 5 propositions from National Board Certification
that teachers are committed to students and their learning,know the subjects they teach and how to teach those subjects to students,
are responsible for managing and monitoring student learning, think systematically about their practice and learn from
experience, and are members of learning communities (NBPTS, n.d.). Furthermore, the Enumclaw School District implements

PLCs under the assumption that the key to improved learning for students is continuous job-embedded learning for educators (ESD,
n.d.). Standards of program evaluation are reflected in this evaluation, such as standard U2 attention to stakeholders, which states
evaluations should devote attention to the full range of individuals and groups invested in the program and affected by its evaluation
(JCSEE, n.d.).
A rationale for this focus includes the intended purpose of PLC time, the research behind it, and the outcomes demonstrated in
the Spanish I-IV Professional Learning Community. Shannon (2007) of OSPI states that PLCs promote identification and
commitment to common learning standards, common lessons and assessments, increased teacher efficacy (and) higher quality of
student learning (Shannon, 2007). The 5 questions chosen for this evaluation are based on the logic model developed for this
program evaluation which lead to stakeholders identifying the desire and need to review data collected over selected time periods as
well as the need to evaluate use of PLC time, and teacher and student outcomes. The Schaumburg teams findings show that since the
implementation of the PLC process in the district, schools have seen the highest gains in student achievement in the history of the
district (APQC, 2009).
What is not being evaluated is each years teacher data, and it should not be evaluated because these results depend on teacher
assignments, students assigned to particular teachers and factors outside of teachers control. The leadership of the PLC is also not
being evaluated, though the lead position has been held by 3 different program members, and the evaluating administrator has changed
3 times in a 4 year period. These facts may influence data and outcomes, but are not the focus of this particular program evaluation in

order to manage scope, time and budget constraints. Depending on the findings of this evaluation, this may be the topic of a future
evaluation.
The stakeholders that should be involved in determining these evaluative criteria are the students of first and second year
Spanish, teachers of first and second year Spanish, as primary stakeholders, family and friends of first and second year Spanish
students, administrators of first and second year teachers of Spanish classes, and community members who work with, teach in higher
education, and employ Spanish students from the program, as secondary stakeholders. Their roles should include filling in surveys,
answering interview questions, providing data from collections of evidence, and contributing to school, program or teacher rankings.

Part IV. Data Collection Design and Sampling Strategy

Part V. Reporting Strategy Table

Stakeholder

Reporting Strategy

Implications

Stakeholder Involvement

Primary: teacher in the Spanish IIV Professional Learning

Community (Shawn Tobius
Spenser Adam, Deb Ogburn)

Presentation, face to face, with

take away and time for
comments, next steps and
rejoinders. Information will have
been provided from start to finish
for stakeholder buy in and input.

Opportunity for differing

perspectives, buy in and next
steps to be sure these
stakeholders and the evaluator
are on the same page.

High stakeholder involvement

throughout the process for input
and decisions.

Face to face informal summary by

teachers of those students, with
access to the newsletter
describing results in more detail

Create understanding of goal of

the program, and gain/maintain
their support of the program if it is
deemed worthy of support by
these stakeholders.

Input is critical from these

stakeholders because they are
the intended beneficiaries of the
program.

Full written report, also present

form face to face delivery of
results to PLC team, but
previously advised of evaluation
results in writing and face to face.

Not only to inform, but to show

best practices across other PLCs.
To improve overall program
quality and efficacy.

Executive summary only, to be

communicated face to face by the
administrator who evaluates this
PLC (Kevin Smith).

Support for current practice.

Suggested improvements across
all school PLCs.

High involvement in giving and

sharing information, but in
implementing changes. This
stakeholder is key for
communications best practices
across groups and PLCs.
To gather historic data and use
evaluation information to better
school, district and benefit other
districts use of PLC time.

Executive summary in the form of

a written report presented by
building principal, Jill Burnes, with
evaluator present for further
questions as necessary.

The Superintendent is the

program champion, along with the
directors, and the Board
approved this program, so it
important to keep them informed
of its efficacy and use.

Primary: Students of teachers in

this professional learning
community(+/-420 students)

Primary: Administrator of
teachers in this PLC (Kevin
Smith).

Secondary: Building
Administration (Jill Burnes, Phil
Engebretson, Paul Iaccobazzi)

Secondary: Enumclaw School

District Superintendent and
School Board. (Mike Nelson, Tim
Madden, Terry Parker, Corey
Cassell, Tina McGann, Nancy
Merrill, April Schroeder, Bryan
Stanwood, Gia Landis, Katja
Barnhart).

To report to community and

OSPI, as well as to be informed in
order to continue/cancel or adjust
current practices.

Online publication
Secondary: Families/Community

Survey available by e-link or by

postal service if requested, to
allow optimal input and buy in of
this secondary stakeholder.

Benefit indirectly from students

skills and knowledge, as well as
being affected by early release,
and requirements due to PLC>

Values, Standards, and Criteria: To ensure that PLCs are meeting goals and objectives of program. To adjust practices to maximize
positive effects. To demonstrate positive correlation between higher student achievement and PLC time, as well as teacher and
administrator satisfaction with program.

Potential ethical issues: Evaluator as a member of the PLC being evaluated, with a high interest in its success. Limited historical
knowledge of pre-PLC data.

References
APQC, (2009). Best practices in evaluating professional learning communities (PLCs): Benchmarking report. Retrieved from
http://www.apqceducation.org/knowledge-base/download-documents/doc_download/113-01-9300-final-report-evaluating-professionallearning-communities

Enumclaw School District, (n.d.). Professional learning communities. Retrieved from

http://www.enumclaw.wednet.edu/board/goals/plc.aspx
Fitzpatrick, J., Sanders, J., & Worthen, B. (2011) Program evaluation: Alternative approaches and practical guidelines (4th ed.).
Boston, MA: Pearson.

Horde, S. (1997). The rationale for professional learning communities. Retrieved from
http://dnn.csd509j.net/Portals/1/Administration/Key%20Initiatives/PLCs/The%20Rationale%20for%20PLCs.pdf
JCSEE, (n.d.). The program evaluation standards: Summary form. Retrieved from http://www.eval.org/p/cm/ld/fid=103
National Board for Professional Teaching Standards, (n.d.). World Languages standards. Retrieved from
http://www.nbpts.org/sites/default/files/documents/certificates/nbpts-certificate-eaya-wl-standards.pdf
Sexton, B. (2009, March 23). Enumclaw schools to end early on Fridays. Enumclaw Courier-Herald. Retrieved from
http://www.courierherald.com/news/41730077.html#
Shannon, G.S. (2007). Nine characteristics of high-performing schools. OSPI. Retrieved from
http://www.k12.wa.us/research/pubdocs/ninecharacteristicsoverviewkeypoints.ppt
The New Teacher Project, (2010). Teacher evaluation 2.0. Retrieved from
http://tntp.org/assets/documents/Teacher-Evaluation-

Oct10F.pdf

University of Washington, (n.d.). 5 Dimensions of teaching and learning. Retrieved from

https://www.k-12leadership.org/5-dimensions-of-teaching-and-learning

Vassallo, P. (2004a). Getting started with evaluation reports: Answering the questions. ETC: A Review of General Semantics, 61(2),
277-286.

Vassallo, P. (2004b). Getting started with evaluation reports: Creating the structure. ETC: A Review of General Semantics, 61(3), 398403.