Final Evaluation Report Brittany Thomas and Luis Del Aguila FRIT 8435 April 25, 2011

II. Executive Summary The purpose of this evaluation is to determine whether the educational, online program called, Classworks provides efficient assistance to improve student achievement in reading in the classrooms for the school A. For this evaluation, the audiences will include program managers, faculty, staff, and administrators utilizing the program. For this evaluation, there are limitations that evaluators have come across throughout the process. For instance, the challenge of not being able to observe every student who uses the program, not being able to interview every teacher due to time constrain, and distance between the location of the school and the location where one of the evaluators resides. This report contains several parts that may include interviews, surveys, and any other mediums of information that relates to the findings, and evaluation of Classworks. The conclusion of this report will demonstrate the outcomes of the evaluation, and any supporting evidences that provide whether; Classworks provides sufficient support for students to achieve their academic goals in reading. Data for this evaluation gathered and obtained from the program manager, administration, educators, and students. The methods used, for this evaluation, to gather information included a focus group of twenty students, focus group of six educators, interviews, classroom observations, educator surveys, and data reports gathered from the software program, Classworks. The online software program, Classworks provides several educational tools that enable learners to achieve their academic goals. Curriculum Advantage is the technology learning company that offers programs, such as Classworks, as an engaging, creative, interactive, online tool that strengthens core subjects taught in schools, such as language arts, reading, and math. Classworks main goals are to individualize lessons for learners

that are in accordance with state required assessments to reinforce, and improve student achievement on state level tests. Classworks is highly designed to provide support to all students from kindergarten level to seniors in high school. It is important to note that ESL(English as a second language) students, and SPED (Special Education) students will find this online program to be most helpful for achieving their academic goals. Classworks can also be implemented as a Response to Intervention for students who may fall under the category of remedial students or it can also be utilized as an additional learning tool for all students. This interactive tool uses student data from highstakes tests to develop an individualized learning plan to cater the needs of each student. This individualized learning plan provides such an array of learning tools such as, instructional mini-lessons, instructional activities in multiple teaching modalities, post assessment, remedial lessons (as necessary), and extension projects for each concept or lesson. Instructors will receive progress reports on each student providing critical information of the standards and skills that are being mastered by the student, as well as standards and skills where the student needs support. The following are evaluation questions used for this evaluation process: What are the goals of this program? Do students participate in this program in the intended manner? What is the program designed to accomplish? Are the students engaged in the programs activities? What evidence is there that the program is achieving its goals? Are the teachers well-trained for this program?

How are the activities in this program changing the students achievement on target areas? Is the program being implemented by teachers the way it was designed to? In conclusion, we feel that the online program, Classworks, promotes student achievement in reading. Every student using the program Classworks showed improvement in at least one area of the reading program. The teachers showed that they are utilizing the program Classworks in their classrooms; however, they do not utilize all of the features that Classworks suggests in order to maximize student learning and achievement. Most of the teachers do not utilize the reports, assessments, and data provided by the program to enhance the performance of their students. Therefore, it is our recommendation that teachers receive additional professional development in these areas of the program. Our recommendations are based on the teacher interviews and surveys that were conducted during this study. One recommendation is to provide additional professional development for teachers that are interested in learning more about Classworks. Another recommendation is for the teachers to use the data provided for them by the program to monitor student progress, and make adjustments to the students individualized learning plan as needed based on the data collected for each student.

III. Introduction to the Report The following report provides accurate information regarding the program being evaluated. It is important to note that this is a first-time assessment for the evaluators and authors of this report. Therefore, this report can only be used as an example and for

educational purposes only. The evaluators are not responsible for providing sensitive information, and any other sources of information that could damage the reputation of neither the clients nor the program being assessed during and after the evaluation is over. The sole purpose of the following evaluation report is to determine if the online program, Classworks, assist to improve student achievement in reading for the School A. The audiences who will be involved throughout the process of the evaluation will include, program managers, faculty, staff, and administrators utilizing the program. The limitations that evaluators have come across are the challenge of not being able to observe every student who uses the program, the challenge of not being able to interview every teacher due to time constrain, and distance between the location of the school and the location where one of the evaluators resides. This report contains an overall report that may include interviews, surveys, and any other mediums of information that relates to the findings, and evaluation of the program, Classworks. The conclusion of this report will demonstrate the outcomes of the evaluation, and any supporting evidences that provide whether such program provides sufficient support for students to achieve their academic goals. IV. Focus of the Evaluation Classworks is an instructional online software program that provides an array of educational tools to assist student achievement in the classroom. The company offers Classworks as a technological venue to provide engaging, creative, and reinforcing web tools to assist faculty and students by enhancing learning, and offering solutions in the core areas of mathematics, reading, and language arts. The goals of this program are to individualize instruction for students that are aligned with state assessments to improve student achievement on state standardized tests. The program is designed to assist all

students in grades kindergarten through twelfth grade. The program is especially helpful for students with disabilities and limited English proficiency. The program can be implemented as a Response to Intervention for struggling students or as an extra learning tool for all students. Classworks uses student data from high-stakes tests to create an individualized learning plan for each student. The individualized learning plan includes, instructional mini-lessons, instructional activities in multiple teaching modalities, post assessment, remedial lessons (as necessary), and extension projects for each concept. Teachers receive progress reports on each student providing information of standards and skills that are being mastered, as well as standards and skills the in which the student needs remediation. Evaluation Questions What is the program designed to accomplish? What are the goals of this program? What evidence is there that the program is achieving its goals? Do students participate in this program in the intended manner? Are the students engaged in the programs activities? How are the activities in this program changing the students achievement on target areas? Is the program being implemented by teachers the way it was designed to? Are the teachers well-trained for this program?

V. Brief Overview of evaluation plan and procedures

Data for this evaluation was obtained from the program manager, administration, educators, and students. The methods used to gather data included a focus group of twenty students, focus group of six educators, interviews, classroom observations, educator surveys, and data reports from the computer program, Classworks. Please refer to the appendix for a detailed explanation of the evaluation plan and procedures.

VI. Presentation of Evaluation Results The reading program in Classworks, is divided among four major components; grammar and mechanics, reading, study skills, and word analysis. We used a focus group of 20 students with various academic backgrounds to determine the overall student growth in achievement in reading after using the program Classworks for reading acceleration and remediation. For the pre-test we found that 80% of the students in this focus group needed intervention for at least one of the four areas. 60% of the students needed intervention for at least two areas, 40% needed intervention in three or more areas, and 25% of the students needed intervention for all four areas of the reading program. The post test for the reading program shows growth in the students reading skills. 45% of the students mastered all four areas of the reading program, 25% of the students still need intervention in one area, 15% of the students need intervention in two areas, 10% of the students still need intervention in three areas, and 5% of the students still need intervention in all four areas of the reading program. Below are two graphs showing the results of the pre and post test for the reading program.

Reading Pre-Test
16 14 12 10 8 6 4 2 0
m ar /M ec ha ni cs Re ad in g s An ay ls i Sk ills

Number of Students

Mastery Needs Intervention

G ra m

W or d

Reading Post Test

18 16 14 12 10 8 6 4 2 0
m ar /M ec ha ni cs Re ad in g is An ay ls Sk ills

Number of Students

St ud y

Mastery Needs Intervention

There are 10 fourth grade teachers, and 11 fifth grade teachers. Of the twenty one surveys we received only twenty in return. In the surveys filled out by the educators we found that

G ra m

We conducted a survey that was given to all twenty one educators in the school.

W or d

St ud y

100% of them use the program Classworks in the classroom on a weekly basis. Aside from the survey we also conducted a teacher focus group, where we had six educators that we interviewed. When asked the question, How do you incorporate the program Classworks into your classroom? one teacher explained that her, students use Classworks during RTI (response to intervention) time and rotate daily so that each student receives 45 minutes of Classworks per week. Another educator states that she uses, centers as part of my weekly reading instruction, and one of the centers is the computer center. So, each student has 30 minutes per week on Classworks during center time. Along with the interviews and the survey, we also held observations in the classrooms weekly. While in the classroom we observed the students using Classworks during their center time. For the thirty minutes allotted during computer time, the six students using the program Classworks were actively engaged in their activities. One student asked to teacher to clarify the instructions for the game, but the other students seem to understand the expectations for the assignments. From the surveys we concluded that most of the teachers do not use the data from the program Classworks to share with parents or to monitor the students progress in reading. Out of the teachers that returned their survey, 70% of them said that they feel that they have moderate to high knowledge of the program Classworks. However, 60% of the educators using the program would like additional professional development in the program Classworks. For more detailed data on the pre and post tests and surveys please refer to the appendix.

VII. Conclusions and recommendations

In conclusion, we feel that the online program, Classworks, promotes student achievement in reading. Every student using the program Classworks showed improvement in at least one area of the reading program. The teachers showed that they are utilizing the program Classworks in their classrooms; however, they do not utilize all of the features that Classworks suggests in order to maximize student learning and achievement. Most of the teachers do not utilize the reports, assessments, and data provided by the program to enhance the performance of their students. Therefore, it is our recommendation that teachers receive additional professional development in these areas of the program. Our recommendations are based on the teacher interviews and surveys that were conducted during this study. One recommendation is to provide additional professional development for teachers that are interested in learning more about Classworks. Another recommendation is for the teachers to use the data provided for them by the program to monitor student progress, and make adjustments to the students individualized learning plan as needed based on the data collected for each student.

VIII. Appendices 1.1 Teacher Survey Questions

Surveys given out to all twenty one teachers, only twenty teachers returned the survey completely filled out. 1.2 Student Achievement Data: Reading Pre-Test M = Mastery Survey Questions 1. How often do you use the program Classworks in the classroom? 2. What is your students level of engagement when using the program Classworks? 3. What level of interest do your students have in using the program Classworks? 4. How often do your students need outside help completing the activities on Classworks? 5. How would you rate your knowledge of the program Classworks? 6. How often do you use the assessments to monitor your students progress in Reading? 7. How often do you generate reports to convey students reading abilities to parents? 8. How would you rate the support you receive from your program manager in using the program Classworks? 9. How often do you check the status of your students progress in Classworks? 10. Would you like more professional development or training on the program Classworks? 1 (very low) Never 0 0 2 (low) 3 (average) 4 (high) 5 (very high) Daily 10 16

1-3 times per 1-2 times per 3-4 times per month week week 0 3 7 1 1 2

17

12

2 6

4 7

9 6

4 1

1 0

11

Yes 12

No 8

NI = Needs Intervention Target Proficiency for 5th Grade = 1530 1.3 Student Achievement Data: Reading Post-Test M = Mastery NI = Needs Intervention Target Proficiency for 5th Grade = 1530 Reading Post-Test Results Reading Study Skills M NI M M M M M NI M NI M M M M M M M M Reading Pre-Test Results M M Reading Study Skills NI NI NI M M NI M M M M M NI NI NI NI M M NI NI M M M M NI M NI M NI M M NI M M M NI NI NI NI M M M NI NI NI NI NI M M NI NI M M M M

Student Student A Student B Student C Student D Student E Student F Student G Student H Student I Student J Student K Student L Student A M Student B N Student C O Student D P Student E Q Student F R Student G S Student H T Student I Student J Student K Student L Student M Student N Student O Student P Student Q Student R Student S Student T

Grammar/Mechanics M NI M M M M M M M M Grammar/Mechanics M NI M NI M M NI M NI M M NI NI M M M NI NI NI M M NI NI M NI M M

Word Analysis M M M NI M M NI NI M M Word NI Analysis M NI M NI M M NI NI NI M NI NI M NI M NI M NI NI NI M NI NI M NI M M

Average Proficiency 1520 1540 1580 1500 1510 1540 1520 1500 1520 1560 Average 1440 Proficiency 1480 1500 1560 1490 1560 1540 1380 1480 1410 1580 1520 1400 1400 1600 1410 1580 1420 1510 1400 1420 1520 1530 1320 1380 1540 1340 1570 1540

Evaluation Questions or Objectives

Information Required

Information Source

Method for Collecting Information

Information Collection Arrangements By Whom Conditions When

Analysis Procedures

Interpretation Procedures And Criteria

Reporting of Information To Whom H o w When

1. What is the program designed to accomplish ?

Description of the programs purpose, how the program is to be used, who the program is designed for, and what the program is designed to achieve

Program manager

Interview

Interview with program manager at school in March, use literary resources found on the program to guide the interview, conducted by evaluator

Use results from interview to form focus group for teacher and focus group for students using the program.

Students pulled for focus group need to be remediated in a certain academic area, teachers in focus group must utilize the program

Report to teachers in meeting during their planning in March 21st

2. What are the goals of this program?

Information on the programs objectives and accomplishme nts

Program manager and administra tion

Interview Interview with program manager the same time as evaluation question number 1, conducted by evaluator

Use results from interview to form focus group for teacher and focus group for students using the program

The goals will be used to guide the evaluation to determine the effectiveness of the program

Report to teachers in meeting during their planning in March 21st

3. What evidence is there that the program is achieving its goals?

Student performance data before using the program and after using the program

Teachers, administra tion, and program manager

Focus Group, Data Report on student progress

Focus group of 20 students, data report on the students progress from the beginning of the school year to mid April, data reports collected by the program manager on each student

Use results of data report to determine the achievem ent of the programs goals Use results from the interview and observatio ns to determine how often students participate in the

95% of the students selected complete the program, 85% of the students showed growth in target learning areas

Presentation to program manager, administration, and selected teachers mid April

4. Do students participate in this program in the intended manner?

Findings of how the program was designed to be used by the students, how the educators are implementing the program in their

Teachers and students

Interview, focus group, and observation

Focus group with 6 teachers, interview conducted by the evaluator during teacher planning, beginning of April, classroom observations once a week beginning March 21st

95% of the students selected complete the program, 90% of the students actively participate in the program as intended

Presentation to program manager, administration, and selected teachers mid April

References: www.classworks.com