Houston County Schools

Bonaire Elementary
3rd Grade OAS

Program Evaluation by:

Stephanie Stone
Stacy Daly
Brian Trice

FRIT 7738-Practicum
Spring 2011
Georgia Southern University

1
 Table of Contents
Executive
Summary………………………………………………………………………………..
3

Introduction……………………………………………………………………………
……………………..5

Focus of the
Evaluation………………………………………………………………………..8

Brief Overview of Evaluation Plan and Procedures……………10

Presentation of Evaluation
Results……………………………………………….12

Conclusion and
Recommendations……………………………………………….16

Appendices………………………………………………………………………………
…………………18

2
 Executive Summary
This evaluation is to show that the OAS (Online Assessment System)
program is effective for all students that utilize the program prior to
taking the CRCT (Criterion Reference Competency Test)—focus on
reading. OAS is one of the most useful testing programs. Houston
County Schools has implemented its use in the past years after seeing
results from all over the state of Georgia. The students evaluated for
this report are third grade students. This report will inform you on how
the program works. There is detailed information as to when and
where students are able to access the quizzes online. Each classroom
has four computers, easily accessible to all students in the class, set up
with the program link. The school has two computer labs with 30
computers each, with the program installed. Students are also given
their access numbers and passwords to access the program at home.
Students have the opportunity to take quizzes during their weekly
computer lab time as well as in the classroom throughout the day as
designated by the classroom teacher. As previously mentioned,
students are also able to access the program outside of class at home.
This report will show through the OAS pre and post tests how OAS
improves students’ preparation for the CRCT in reading.
Several data sources were utilized in driving this evaluation. Parents
were surveyed both before and after the evaluation to see if they felt
their child(ren) was any more prepared for taking the reading portion
of the CRCT. A data analysis by standard for the class was collected
from the OAS system. This report gives the standards that were on the
reading test as well as what the child scored in that area. Each child
should show improvement on their reading readiness, vocabulary, and
ability to comprehend. These reports and information can be
monitored by teachers, as well as individuals, administration, and
parents. Teachers have documentation of student’s improvements

3
due to the OAS program. The evaluation proves that the program is
working. Data shows that the students who participate in using the
program both in school as well as at home are making positive gains
toward their reading percentages and comprehension. By the time
delivery of the CRCT Reading portion of the test is administered,
students should show major improvements in reading comprehension
and vocabulary due to the use of the OAS program.

4
 Introduction
This report focuses on third grade students during the 2010-2011
school year. It will show that the OAS program is working to increase
reading readiness, vocabulary, and overall reading comprehension on
the CRCT. The students should show an increase in reading,
vocabulary, and comprehension. The students are encouraged to
practice at home as well as practice during the school day in class.
The teacher is able to monitor whether or not the students are making
good use of the program. In an effort for all schools to meet AYP and
the NCLB act, schools are finding it necessary to utilize more and more
programs to boost reading readiness, vocabulary, and comprehension,
as well as other skills, to prepare students for standardized tests. The
OAS is one such program that is utilized in Georgia to prepare students
for the CRCT in all areas. Everyone from the Board of Education down
to the students themselves are working to increase student academic
achievement, particularly in reading. This program evaluation is
designed to show that the OAS program works and is effective.
Students are increasing their reading readiness, vocabulary, and
comprehension. Each of the five third grade teachers has four working
computers in their classroom with accessibility to the OAS program
online. The data that was collected is embedded in the report to help
confirm accuracy.
Purpose of the Evaluation
The purpose of this evaluation is to show that the OAS program is
effective for the students who use it. This report will show that the
program is in-fact increasing readiness, reading comprehension, and
vocabulary on the CRCT.

Audience for the Evaluation Report

5
This report is designed for the use of stakeholders to include members
of the Houston County Board of Education, administrators of the
school, classroom teachers, parents, and students. This program may
also benefit the program designers.

Limitations to the Evaluation

There are a few limitations that coincide with this report. This
evaluation was completed in a specific time frame and does not
include the entire school year. This could be a significant limitation.
Another significant limitation is that the school has not administered
the CRCT. Finally, the fact that there are students who did not or are
not able to participate fully participate in the program outside of school
could deter this evaluation. However, we feel that this evaluation is
adequate to show the purpose.
Overview of Report Contents
The information for this report is as current and accurate as possible
and is available through the school system. All data is accessible to
administration and the classroom teacher. Students receive
immediate feedback and scores to look at and reports are printable for
parents. All data has been thoroughly checked to make sure there are
no errors in reporting.
The stakeholders are very pleased with the program and its positive
results. Students are motivated to take tests as they know it helps to
prepare them for the CRCT and it is computer based, therefore, they
enjoy doing it since it is something more than paper and pencil.
Teachers are pleased with the program as well. They are able to run a
report that breaks the test down into standards and elements,
therefore being able to see which standard and/or element needs more
focus.

6
 Focus of the Evaluation
OAS or Online Assessment System allows teachers to create and
students to take practice CRCT tests to prepare for the real thing. It
provides immediate feedback to the student as to how well they did on
the test. Students are allowed to go back in and retake an assigned
test after the material has been reviewed by the teacher. The program
allows the teacher to run a report that shows an overview of how well
each individual student or the entire class is doing. It can be broken
down by each element of each standard taught.

The purpose of this evaluation is to investigate and answer the

question: How does student participation in the OAS relate to growth in
reading comprehension and preparedness for the CRCT? The third
graders in this report are able to access the OAS program at various
assigned times during the day in their classroom. They are also given
the opportunity to do so during their assigned computer lab time once
a week, as well as log in at home and practice. All classrooms are set
up with four working computers that have Internet access to the OAS
program. Through this evaluation, administrators, teachers, students,
and parents can make the necessary decisions on whether or not the
OAS program is truly effective for the students or how to make the
necessary changes that would make it more effective.

At the beginning of January, the teachers in the third grade created

numerous reading comprehension, reading readiness, and vocabulary
tests for students to access and complete. The OAS program provides
students and teachers with immediate feedback as to how the
students are meeting the learning objectives, for example, met the
learning objective, minimally met, or did not meet. The teachers can
then focus on students who need extra help on a specific standard or

7
element in small group settings, one-on-one, or whole group if most
students are still having trouble. Students are then able to retake the
test after the teacher reassigns it.

8
 Brief Overview of Evaluation
To be able to fully evaluate the effects of the OAS program on third
grade students’ and increased reading learning objectives in
preparation for the CRCT, information was collected and analyzed from
the third grade students’ OAS test reports.

To fully determine if students are utilizing the OAS program effectively,

data was collected from the OAS Test History report. The test history
report includes information on student performance, date and time the
test was taken, number of times the test was taken, and the minimum
and maximum percentage scores. In order for students to be making
satisfactory progress toward learning objectives, students should be
taking at least three tests, or as determined by their classroom teacher
and earn at least a 75% or better on each test. The pre and post OAS
test scores display a positive effect the program has on students and
the reading learning objectives.

The pre-test information and data from the Class List Report is used to
determine the students’ current progress toward reading learning
goals (Appendix A). The headings allow for the administrators and
teachers to see where each student is in their progress toward the
learning goals. Teachers are then able to print a report by standard or
element and adjust their instruction and focus on the reading learning
goals that most students are still struggling with. For those goals that
only a few are still struggling with, the teacher can pull strategy or
guided reading groups to focus on those areas with a smaller group of
children. The OAS post-test data (Appendix B) proves that all students’
reading learning goals have increased since the pre-test. A summary of
the 16 third graders who were assessed and their results are shown.
(Appendix C)

9
10
 Presentation of Evaluation Results
All 16 students in one third grade homeroom were given a pre test on
reading standards and learning goals utilizing the OAS online program.
OAS Pre-Test Data
Student Test Date Possible Points Earned Points % Progress
toward learning
goal
1 1/12/11 10 4 40% Did not meet
2 1/12/11 10 7 70% Minimally
met
3 1/12/11 10 6 60% Minimally
met
4 1/12/11 10 7 70% Minimally
met
5 1/12/11 10 10 100% Met
6 1/12/11 10 9 90% Met
7 1/12/11 10 10 100% Met
8 1/12/11 10 7 70% Minimally
met
9 1/12/11 10 8 80% Met
10 1/12/11 10 5 50% Did not meet
11 1/12/11 10 9 90% Met
12 1/12/11 10 6 60% Minimally
met
13 1/12/11 10 6 60% Minimally
met
14 1/12/11 10 5 50% Did not meet

15 1/12/11 10 7 70% Minimally

met
16 1/12/11 10 9 90% Met

Once the students have completed the pre-test, teachers are able to
adjust their focus and instruction toward the goals that most students
are needing remediation in. The OAS program keeps track of student
scores and provides immediate feedback to students so they know how
they are progressing toward the goal. If students complete the
assigned number of tests and meet 75% of the goal, they are
considered to be successfully and effectively participating and
progressing in the program. After a teacher-determined amount of

11
time and remediation, students are assigned the same test, post-test,
using the OAS program. The teacher then compares the scores from
the students’ pre-test and post-test to determine if students are
progressing toward their learning goals and if they are prepared to
take the CRCT.

OAS Post-Test Data

Student Test Date Possible Points Earned Points % Progress
toward learning
goal
1 3/28/11 10 8 80% Did not meet
2 3/28/11 10 9 90% Met
3 3/28/11 10 8 80% Met
4 3/28/11 10 8 80% Met
5 3/28/11 10 10 100% Met
6 3/28/11 10 10 100% Met
7 3/28/11 10 10 100% Met
8 3/28/11 10 9 90% Met
9 3/28/11 10 9 90% Met
10 3/28/11 10 7 70% Minimally
met
11 3/28/11 10 10 100% Met
12 3/28/11 10 9 90% Met
13 3/28/11 10 8 80% Met

14 3/28/11 10 8 80% Met

15 3/28/11 10 7 70% Minimally

met
16 3/28/11 10 8 80% Met

Summary of Pre/Post-Test Results (Appendix C)

12
 Scores Percentage
Pre-test Mean 7.188 72%
Post-test Mean 9.125 86%
Change +1.937 +14%
*Numbers based on the 16 students in the class who were assessed.

Stakeholder Perceptions
All stakeholders that are part of this evaluation reflect a positive
response to Houston County Schools utilizing the OAS program with
their students in order to prepare them for the CRCT. Only positive
comments were heard when interviewing the stakeholders. Positive
trends on the CRCT have been seen each year since the schools
started implementing the use of OAS. When all students maximize the
use of the OAS program, all standards in reading, reading
comprehension, vocabulary, and reading readiness conclude that the
OAS program is effective. A few negative comments that came from
teachers, students, and parents were time restraints during the day,
minimal computers in the classroom, and some students not having
access to a computer or Internet at home could cause some students
to not utilize the program to its fullest.

13
 Conclusion and Recommendations
After careful examination of the data for this evaluation, the
effectiveness of the OAS program on all third grade students, this
evaluation study concludes:
Program Strengths:
• Increase in progress toward mastering the reading
standards
• Increase in CRCT preparedness
• Increase in CRCT scores
Program Weaknesses:
• Lack of available computers in classroom
• Time restrictions
• Lack of computer in students’ home
• Lack of Internet in students’ home
• Students choose not to fully participate in using the
program
Recommendations:
• Designate a specific time during the day for testing
• Utilize designated computer lab time and complete testing
before allowing students to use other programs
• Create a “sign-up sheet” to maximize student testing time
• Allow students to access the program during “down time”
when they are finished with assignments
• Create a reward system for students who are progressing
successfully toward their reading learning goals.
It is the recommendation of this study team, based upon the results of
this evaluation, to continue to use the OAS program to prepare
students for the CRCT. If the teacher wishes and it is feasible, the
implementation of a reward system may encourage and increase the

14
students’ desire to progress positively toward their reading learning
goals and to participate more fully. It is also recommended to allow
students to access the program during “down times” during the day,
particularly those who do not have access to a computer/Internet at
home. Finally, setting a specific time in the class daily schedule and
creating a “sign-up sheet” could potentially maximize the program’s
effectiveness and participation.

15
 Appendix A
OAS Pre-Test Data
Student Test Date Possible Points Earned Points % Progress
toward learning
goal
1 1/12/11 10 4 40% Did not meet
2 1/12/11 10 7 70% Minimally
met
3 1/12/11 10 6 60% Minimally
met
4 1/12/11 10 7 70% Minimally
met
5 1/12/11 10 10 100% Met
6 1/12/11 10 9 90% Met
7 1/12/11 10 10 100% Met
8 1/12/11 10 7 70% Minimally
met
9 1/12/11 10 8 80% Met
10 1/12/11 10 5 50% Did not meet
11 1/12/11 10 9 90% Met
12 1/12/11 10 6 60% Minimally
met
13 1/12/11 10 6 60% Minimally
met
14 1/12/11 10 5 50% Did not meet

15 1/12/11 10 7 70% Minimally

met
16 1/12/11 10 9 90% Met

16
 Appendix B
OAS Post-Test Data
Student Test Date Possible Points Earned Points % Progress
toward learning
goal
1 3/28/11 10 8 80% Did not meet
2 3/28/11 10 9 90% Met
3 3/28/11 10 8 80% Met
4 3/28/11 10 8 80% Met
5 3/28/11 10 10 100% Met
6 3/28/11 10 10 100% Met
7 3/28/11 10 10 100% Met
8 3/28/11 10 9 90% Met
9 3/28/11 10 9 90% Met
10 3/28/11 10 7 70% Minimally
met
11 3/28/11 10 10 100% Met
12 3/28/11 10 9 90% Met
13 3/28/11 10 8 80% Met

14 3/28/11 10 8 80% Met

15 3/28/11 10 7 70% Minimally

met
16 3/28/11 10 8 80% Met

17
 Appendix C
Scores Percentage
Pre-test Mean 7.188 72%
Post-test Mean 9.125 86%
Change +1.937 +14%

18
 Technology Center Monthly Report
The Instructional Technology Coordinators that are part of the Houston
County technology department conduct many activities throughout the
month. This information is based on an interview conducted with one
of the ITCs.

• Nine Promethean Board training sessions with Special Education

teachers– These training sessions are completed in a series of
three. The ITCs go to the schools that have Special Education
teachers with Promethean Boards in their room to conducted
hands-on training. Some teachers are “technologically savvy”
and were already utilizing the program/board in their classroom.
Others were at the beginning stages of just setting up the
board/computer. Based upon the number of Special Education
teachers at the specific school determines the number of
teachers at each session.

• Daily visits to the different schools in Houston County to assist

with problems with software. Online logs confirm the number of
visits made to each school, the specific classrooms, and the
assistance provided.

• The ITCs attend the Georgia Technology Conference – They look

at the new technology that is becoming available and ways to
fund getting it.

• Daily fielding of emails and student transfers—ITCs take care of

student transfers within the county and making sure student
information is moved from one school to the next correctly. They
also field email questions about different programs utilized within
the county.

• Weekly meetings with the Student Services department keep the

ITCs up-to-date with what teachers/schools need their services
most.

19
• Attend Webinars—ITCs like to keep abreast of the current
technology and see what they can bring to the schools and how
they can fund the programs without money due to budget cuts.

20