1:1 Chromebook Initiative

Evaluation Plan

Prepared by: David Foster

North Carolina State University

April 14, 2015

1. Introduction
In recent research studies, appropriately trained students and teachers in
technology has led to higher engagement and achievement. Based on this
data, Lead Academy, a public, charter middle school, will implement a 1:1
Chromebook initiative for each student in the 2014-2015 school year. The
purpose of this program is to increase student engagement, achievement,
and lower discipline problems at school. The stakeholders involved with this
evaluation are the administration, teachers, parents, students, and district
employees. Teachers will undergo appropriate professional development in
order to integrate and teach at high-levels with the new technology. They will
also learn new modes of discipline during this classroom change.

2. Overview of Evaluation
Being a public charter school that serves a large population of low-income,
minority students, the teachers and staff are continually committed to finding
new ways to increase student engagement and achievement. Students spend
so much of their days connected to their smart phones and tablets reading
information, watching videos, and talking to their peers, that the staff at Lead
Academy wants to meet our students where they are. Learning games,
chants, stories, effective lesson planning, and technology are all hooks that
raise student interest in class. In addition, research has proven that when
implemented effectively, technology raises both student engagement and
achievement.
In order for the 1:1 Chromebook Initiative to be possible, research began in
2013. Different individuals were invited to be involved in something called
the technology committee. This committee was in charge of creating a
technology plan for the school, its budget, determining appropriate
professional development, and what specific laptop or tablet would serve our
students and teachers best. The principal, technology facilitator, three
teachers, and the PTA president are members of this committee. The Google
Chromebooks was decided upon because of its affordability but mainly due to
the benefits of working with Google Classroom. In addition, the Google
Chromebook has options for filtering content for students and fewer
possibilities to stream television shows, sporting events, and games
compared to Apple devices.
At the beginning of the 2014 school year, every student and teacher in the
school will receive a Google Chromebook. Fifty extra laptops will be ordered
for students to rent if laptops are forgotten or being repaired. Teachers will
also be receiving or updating their SMARTboards in order to properly sync
with the Chromebooks. Updating of classrooms will take place during the
summer of 2014, so when the Chromebooks are passed out at the beginning
of the 2014 school year, technology integration will begin. Additionally,
during the summer and beginning of the school year, teachers will be
completing required professional development. These classes will be taken

online and in-person. 80% of professional development will be completed

once the school year begins, because teachers will undergo further training
as the year progresses to hone their skills as they become more comfortable
with technology.
As a result of the 1:1 Chromebook initiative, student discipline is hoped to
decrease by meeting students where they acquire their information and talk
to peers. In addition, technology provides ample opportunities for
differentiation. Now, it will be easier to meet students where they are
academically. Higher students can move ahead and be challenged (removing
boredom from class), and lower students will have more opportunities for
remediation and small-group instruction from the teacher. Student discipline
will be measured and compared to the number of referrals and detentions
written from 2013 to 2015. The programs goal is to decrease these issues by
20%. With the possibility of student discipline decreasing, the evaluation also
wants to determine the extent of raising student engagement from the 1:1
initiative. This will be determined through qualitative data through teacher
and student surveys and interviews. With increased engagement, student
achievement will also be measured, in hopes that it will increase by 10%
compared to the previous year. While it may be difficult to compare students
to the previous years students, measuring student achievement is necessary
to understanding if and how the 1:1 Chromebook Initiative is succeeding.
Also, measuring the programs progress helps to determine what tweaks
need to occur in order to increase its effectiveness.
As technology is continually integrated into the school, the role of the
technology facilitator is paramount to coaching teachers how to effectively
incorporate its use into the classroom. Pushback and difficulties are expected
at the beginning, but through professional development and trial and error,
the hope is that by the end of the year teachers feel comfortable with
implementing technology into their curriculums and classrooms. Stakeholders
also hope that students see technology as a tool that enhances the classroom
experience, elevating their attitude about learning. This would occur by
students using their Chromebooks within class multiple times a week. From
these outcomes, in the long-run, the staff at Lead Academy desire to be a
model for educational technology in the classroom that deeply engages
students though project-based learning to increase student achievement.
Integrated effectively, the Chromebook initiative will be the catalyst to
improving the schools reputation and enrollment rates in the coming years.

Logic Model

3. Evaluation Design
1) To what extent were classrooms updated before school began with

the appropriate technology?

2) How many professional development sessions are offered? What
topics were covered?
3) What knowledge/skills did teachers gain as a result of their
participation in PD? To what extent did they apply these new
knowledge and skills into the classroom?
4) To what extent did students feel prepared to use their Chromebooks
at the beginning of the year?
5) How often did students use their Chromebooks on a daily or weekly
basis?
6) To what extent has there been a decrease in student discipline
(based upon the number of detentions give and referrals written)
7) To what extent did student achievement increase (based on
assessments and homework completion grades)?

To understand if the 1:1 Chromebook Initiative is successful, there are

certain questions that need to be answered about the program. The
first is to what extent were classrooms updated before school
began with the appropriate technology? This question will
determine if the school year began with integrating technology from
day one. If classrooms were not updated until Christmas break, this
would definitely affect the evaluation of the 1:1 initiative and the
outcomes measured. The second question is how many professional
development sessions are offered? And what topics were
covered? This information links into the third question, which is also
about professional development. The third question asks what
knowledge/skills did teachers gain as a result of their
participation in PD? To what extent did they apply these new
knowledge and skills into the classroom? Good professional
development with the right amount of follow-up can tremendously aid
teachers in integrating new strategies and technology into the
classroom. Finding the right company and people to lead this process
can make or break how receptive teachers are to changing their craft.
It is also important to determine the barriers that caused teachers to
not implement this new knowledge. The answers to these questions
are paramount in understanding the success and evaluation of this
program. The fourth question is about students and their training to
learn about their laptops: to what extent did students feel
prepared to use their Chromebooks at the beginning of the
year? Before students could acquire their new laptops, they had to go
to an hour-long training session after school where the principal and
technology facilitator led them through an acceptable usage policy and
trained them with their device. Also, one parent or guardian had to be
there to hear and sign the usage policy. The fifth question is also
about the Chromebooks and how often they are used by
students on a daily or weekly basis. From this information, we are
able to determine how comfortable students are in using their devices
and how much they are a part of their educational experience. Finally,
the final two questions are about the ultimate outputs from the
program: student engagement and achievement. The sixth question is
about increased student discipline, which, hopefully, will increase
student engagement. The question asks to what extent has there
been a decrease in student discipline (based upon the number
of detentions give and referrals written) compared to last
year? The seventh question is to what extent did student
achievement increase (based on assessments and homework
completion grades)? Any time a new program or initiative is
introduced at a school, the focus of it should be on the students, and if
it is helping them focus and increase their cognition. The first five
questions are critically important to ascertain the effect of the last two.
Ultimately, though, the last two questions are the lynchpins of the
program and determine whether the program was a success.
The evaluation findings will be primarily for school administration,
teachers, and district employees. The school administration will use
the findings to tweak the programs professional development,

determine if teachers are integrating technology into their classrooms,

and if students feel comfortable using their laptops. From this
knowledge, the administration can give certain teachers and students
more attention and training, if needed. This knowledge will also effect
how future technologies will be rolled out and the most effective way
for doing so. Teachers will use the evaluation findings to see student
opinions of the program as well as if their engagement and
achievement scores increased. District employees will use the findings
to see how well this program began and to learn from it for future
integration projects in other schools. While all of these questions are
important to be answered, the most credible information from our
evaluation is the quantitative and qualitative results of how technology
changed student engagement and achievement. Numeric data that
illustrates rising achievement scores is something all parties hope for
when new programs are introduced. Numbers are hard to argue with
and do not lie.
The approach for this evaluation is two-fold; there will an internal,
objectives-oriented approach and an external, goal-free evaluation.
The objective-oriented approach has been outlined in this evaluation
plan. The internal worker is to focus on the extent to which our
purposes and objectives are achieved. The advantage of having
internal worker evaluating our program is the time not wasted
explaining the program model, its history, and various stakeholder
interests. Additionally, the internal worker can serve as an advocate for
our program and its findings after the evaluation is over. The second
part of the evaluation will be performed by an external evaluator that
is goal-free. The function of the goal-free evaluation is to reduce bias
that occurs from knowing a program and its goals. Often times, when
an evaluator is told the goals of program, he/she is immediately limited
in his/her perceptions. The goals can act like blinders, causing
evaluators to miss important outcomes that are not directly related to
what our program desires to achieve. Furthermore, having an external
evaluator gives more credibility to our Chromebook initiative, because
an outsider has an expertise beyond what our school knows. This
expertise can enlighten school administration if our 1:1 initiative is
comparable to other schools and what needs to be changed or
improved. In addition, teachers, students, and staff are, sometimes,
more willing to give honest answers to someone they will not answer
to after the evaluation is complete. This can lead to more
transparency, honest feedback, and, hopefully, better improvements to
our 1:1 initiative.

4. Data Collection
In order to collect data relating to our evaluation questions, we will
periodically hand out surveys to students. These surveys will ask
students about how often they use their laptops, their engagement

levels, and familiarity with their Chromebooks. These questions will be

measured through qualitative and quantitative answers in order to
articulate numeric data with explanations. Teachers will also receive
surveys that ask questions about their perceptions of the program, the
principals leadership of the initiative, their familiarity with the
technology, student engagement, and the effectiveness of their
professional development. Like the student surveys, answers are both
quantitative and qualitative to acquire different types of data.
Interviews will also performed with students and teachers to discern
their perceptions of the program, their comfort level with the
Chromebooks, and the level of student engagement in the classroom.
The 1:1 Implementation Rubric will be used to ascertain the
effectiveness of the professional development and how much it is
changing the school culture in terms of technology integration. It will
also serve as a benchmark for teachers collaborating together with
lesson planning, teacher proficiency in terms of teaching with
technology, and student proficiency when working with technology.
Two kinds of secondary data will be used outside of these collection
instruments to determine the effectiveness of the 1:1 Chromebook
Initiative. The first type of data will come from school discipline data
that records the number of referrals and detentions written from the
2013-2014 and 2014-2015 school years. The second type of secondary
data pulled will be from EOG assessment scores, assessment averages
from teachers, and homework completion scores from 2013-2015.
These two types of secondary data will help to determine if student
engagement and achievement has increased as a result from the 1:1
Initiative beginning in the fall of 2014.

Table 1: Evaluation Questions and Associated Data Collection Methods

Evaluation Question
1. To what extent were classrooms
updated with appropriate technology
on time?
2. How many PD sessions are
offered? What topics were covered?
3. What knowledge/skills did
teachers gain as a result of their
participation in PD? To what extent
did they apply these new knowledge
and skills into the classroom?
4. To what extent were students
prepared to use their Chromebooks at
the beginning of the year?
5. How often are students using their
Chromebooks on a daily or weekly

Data Collection Method

IT documentation

Technology committee
documentation
Teacher survey and interviews
1:1 Implementation Rubric

Student surveys

Student surveys and interviews

basis? (How proficient are students

with their laptops?)
6. To what extent has there been a
decrease in student discipline (based
on the number of detentions and
referrals written)?
7. To what extent did student
achievement increase (on
assessment and homework
completion grades)?

1:1 Implementation Rubric

Student discipline data from 20132015

EOG assessment data, mean

assessment grades, and homework
completion scores from 2013-2015

5. Data Analysis and Interpretation

Table 2. Indicators and Success
Evaluation Question

Criteria or Indicator

Standards
(What Constitutes
Success?)

1. To what extent were

classrooms updated
with the appropriate
technology by the
beginning of the 2014
school year?

# of classrooms
updated on-time

100% of classrooms
technology-ready
before the first day
of school of 2014.

2. How many PD
sessions are offered?
What topics are
covered?

# of PD sessions
scheduled

3 PD face-to-face
sessions completed
before start of
school
3 online PD sessions
completed before
start of school
4 PD sessions
offered during
school year
Teachers have
attended and
completed 80%
work for the PD
workshops

Types of PD offered

Types of PD cover a
variety of topics and
meet teacher
desires and needs to
integrate
technology into
classroom

3. What
knowledge/skills did
teachers gain as a
result of their
participation in PD? To
what extent did they
apply these new
knowledge and skills
into the classroom?

# of positive and
negative responses
to PD

80% of teacher
responses are
positive about PD

# of skills learned

80% of teachers
have applied new
knowledge/skills to
the classroom

4. To what extent were

students prepared to
use their Chrome books
at the beginning of the
year?

Types of behavior
changes

Students rate their

level of proficiency
as high and are
comfortable when
working with Google
Chromebooks and
software installed on
them. Questions
about how to work
devices are little to
non-existent.

5. How often are

students using their
Chromebooks on a
daily or weekly basis?

# of times students
use their devices on
a daily/weekly basis

Students are using

their laptops in class
or for homework at
least 3x a week.

6. To what extent has

there been a decrease
in student discipline (#
of detentions given and
referrals written)
compared to this time
last year?

% decrease in
referrals and
detentions over oneyear time span

20% decrease in the

number of referrals
and detentions
written from 20132014 school year to
2014-2015.

7. To what extent did

student achievement
increase (on
assessment and
homework completion
grades)?

% increase in EOG
scores, classroom
assessments, and
homework
completion grades

10% increase across

the board in EOG
assessment scores,
teacher
assessments, and
homework
completions grades

Student survey
responses