Evaluation of iSchool Campus 1:1 Device Training Program By Tiffany Galloway

A report submitted to Minidoka County School District August 1, 2013

Tiffany Galloway EDTECH 505 Boise State University

Tiffany Galloway Evaluation Report Summary In fall of 2012, Minidoka County School District joined with iSchool Campus to create a pilot program for Paul Elementary. This program consists of 1:1 iPad devices for each student in the school building. iSchool Campus goal is to have an effective, relevant, replicable approach to classroom teaching and learning resulting in measureable gains in student achievement and digital competency. iSchool has been implemented in nine states and is demonstrating school improvements in a variety of areas including: test scores, student engagement, and attendance.

iSchool Campus believes the key to the success of this program is to equip teachers with everything they need to feel comfortable and supported. This evaluation is focused on the training program this staff received prior to implementing this technology in the classroom. The purpose of this report is to evaluate the efficiency, effectiveness, and impact of the training. The training includes educators experiencing how the iPad can be used in their own individual classrooms by watching demonstrations and each having their own iPad and MacBook at hand. The personalized training program is broken down into three days from 8:00 a.m.3:00 p.m. The first day is focused on the devices themselves. An overview is conducted on how to utilize each and every button externally and then how the settings are configured. The following two days incorporate apps, programs and examples of classroom experience, safety measures and critical thinking for use at various age levels. The goal-based evaluation model is used in this evaluation through a survey that was created to measure the objectives of the iSchool training for educators. The end result demonstrates what changes occur with the educators after the three-day training and offers suggestions for the following trainings. The data was collected with this survey from a sample population of two teachers at each grade level K-5, resulting in a data collection from 12 participants. The training was delivered; the staff received the equipment involved, and had the time to use what they learned from the training in their classroom. The schools administrator can use thee findings to further benefit the faculty. iSchool Campus has specific objectives they set out to meet in this training. There is a range of age levels, subjects taught, experience with technology, and comfort levels among the staff at Paul Elementary. Based on the results, four out of the five major objectives were met. These educators gained valuable knowledge in core skills and concepts of using an iPad connected with a MacBook and their comfort level with technology in the classroom grew as a result of the training. Improvement areas for the training that the school administration should consider for upcoming iSchool trainings include: deeper critical thinking skills for all grade level instructors and a link between apps and daily subjects covered. The participants want and need to be able to use the information learned directly with their students. As a result of this evaluation, recommendations are included for the benefit of this training program and the participants.

Tiffany Galloway Evaluation Report

Description of the Program Evaluated Program Objectives A highly skilled teacher is what iSchool entails as the single greatest element of a 21 century classroom, and technology plays a role through facilitation and as a tool in the learning process. The objectives for the iSchool Campus training program are to:
st

Enhance the user experience by developing core skills and techniques with iPads and MacBooks. Practice and implement basic apps for increased student engagement. Demonstrate programs that create a teaching and learning environment that goes beyond the traditional classroom. Create critical thinking through the use of interactive teaching strategies. Apply skills and brainstorming techniques to utilize iPad implementation with current curriculum.

The evaluation purpose measures the impact of these objectives and how the participants react to them. If these goals are met successfully, there can be new objectives developed for the next training. If these are not achieved, further recommendation will be made for the following trainings to offer the best possible professional development. Program Components The iSchool Campus training program is designed solely to the specific schools needs. At Paul Elementary, the objectives are clear to enhance and develop the comfort level of iPad use among each participant and feel supported enough to take what is learned and use it as a teaching mechanism. Two trainers from iSchool present information and coach educators in teaching essential 21st century skills. For this first training, iSchool Campus provides the funding for their trainers for 3 full days and accommodations. A personnel from iKeepSafe is also provided for 1 day of the training. This Generation Safe provides resources for a positive media environment and its goal is to minimize negative effects of online incidents. The overall training consisted of 3 days, a Wednesday, Thursday, and Friday from 8:00a.m. 3:00p.m. Each day is broken up to cover the devices externally and internally, meaning every button is discussed and then the focus goes deeper. The above five objectives are covered in order beginning with the first two on Day 1, the third on Day 2, and the last two objectives on Day 3. Creativity is constant throughout the training along with problem solving, critical thinking, and communication. Grade levels have the opportunity to work and collaborate together to meet their needs and encourage thinking of ways for classroom motivation with the curriculum and devices. The training is set up in tables for groups to work

Tiffany Galloway Evaluation Report together. The materials provided include an iPad for each participant, a MacBook Pro, and the use of an HD television. Evaluation Method Participants Participants are both female and male that make up the faculty at Paul Elementary, with the majority being female. All of the elementary staff has at least an undergraduate degree in Elementary Education. The participants in the survey are a sample from each grade level. Two teachers from every grade K-5 were surveyed. There are 12 total participants with the age range of 26 to 61 years old. 91 percent are Caucasian and 9 percent are Hispanic. Personnel Involved Brayden W. - Served as an IT Director and Technology Integration Consultant and integrates the deployment of the devices. He is the key presenter at the professional development trainings and has worked with private corporations in developing technology plans.

Phillip B. Apple Certified Systems Administrator, he is the Lead Systems Administrator at iSchool Campus and holds many coordinator certifications. Colleen J. Administrator for the school and is fundamental in addressing the needs of the training. She will review suggestions and lead the discussion for future trainings.

Procedures After the training was conducted and the educators had experience with the technology and what they learned from the professional development, a survey was sent out to determine if the goals/objectives were actually met. The evaluator used SurveyMonkey and emailed it to the sample participants. There is a 100 percent response rate and the online survey consisted of 10 questions. Please look at Appendix A for the full list of questions and online link to the survey. Data Sources Data was collected from the participants surveys with a formative evaluation approach. This includes aspect such as: how important the training has been in implementation, if there has been a change in the vision of teaching, comfort level, and relation to curriculum. SurveyMonkey automatically calculates findings and makes a clear portrayal of responses for each question as well as the number of respondents. The survey included ranking responses as well as short answer responses. Each

Tiffany Galloway Evaluation Report

question required an answer before it could be submitted to record results. The attitude of the participants was positive as the evaluator was greeted with many questions about why and how a program evaluation should take place for professional development training. Results The evaluator of the iSchool Campus training that was provided received feedback through an online survey from the 12 participants that were chosen to complete it. The first two questions were asked to identify comfort level of using technology in the classroom before and after the training. One of the objectives was to ensure a greater comfort level among the staff and that indeed did occur. Before the training, 75% of the educators felt average or below with their comfort level of using technology in their own personal classroom and after the training, 91% felt average or above with the use of technology. The short answer questions, made it clear that these educators had a support system including the rest of the staff and iSchool Campus personnel.

4.5 4 3.5 3 2.5 2 1.5 1 0.5 0

Comfort Level with Technology before the Training

Comfort Level with Technology before the Training

Tiffany Galloway Evaluation Report

Comfort Level with Technology in the Classroom after the Training

6 5 4 3 2 1 0 Comfort Level with Technology in the Classroom after the Training

The importance of future professional development trainings is critical to this evaluation. The evaluators goal is to provide feedback on this training and suggest criteria for following trainings based on the experiences of the participants. The results of this question appear to represent that the staff feels as though future trainings are very important and the first one may not have been as important as the upcoming trainings.
8 7 6 5 4 3 2 1 0 Not at all 1 2 3 4 Very 5 How important was this first training How important will future iSchool trainings be

Tiffany Galloway Evaluation Report

The time frame the trainers used seemed to be appropriate for the programs they presented. 11 out of 12 participants felt it was very appropriate and 1 out of the 12 said the time spent to utilize the programs was just average. Overall, the training met one if its main objectives by helping to develop skills on the devices used, an iPad and MacBook. 91% of the participants felt that the training very much helped to develop some skills with these devices.

Development of Skills with iPad and MacBook

Not at all Somewhat Average Very Much

There were two short answer responses on the survey. These two questions were addressed to see if the objectives are being met. 9 out of the 12 participants demonstrated that they did think critical thinking skills were involved in this training. For example, two participants discussed the creativity of given tasks to create movies/trailers in an app within a small group. Time was given to critically think how students would use this app and then demonstrate their thinking by actually going around the school to make a short clip. Another participant discussed how brainstorming ideas for grade levels and school wide for the use of iPads is a form of critical thinking. The average response was just a couple sentences. The remaining 3 out of the 12 felt that little to no critical thinking aspects occurred during the training. One in particular felt that information was just given without the opportunity to explore first. The last question pertained to the apps involved in the training and how they could be related to the current curriculum these participants are using. 6 out of the 12 participants expressed how particular apps shown and explored would relate to what they were already doing. It was described as excitement in these responses and how enrichment could take place in lessons, especially math. The remaining half could describe some apps they liked and yet still expressed confusion or disbelief on how they

Tiffany Galloway Evaluation Report could tie it in with their curriculum without completely shying away from it. The evaluators review of the short responses demonstrates the similarity felt from the staff. Discussion

The purpose of the evaluation was to determine the efficiency, effectiveness, and impact of the training provided for an elementary school by iSchool Campus. To measure these aspects, the evaluator looked at the objectives to identify their goals and if indeed they were met. Analysis of the data portrayed that the objective of developing core skills with the devices was definitely achieved. The walk through of a step-by-step process on how they worked externally was important for many of the beginning learners. The objective to increase student engagement was also met with 83% of the participant surveys feeling very confident that specifically the training provided ways to increase motivation and engagement with their own students. Based on the results, the objectives of demonstrating programs and critical thinking were met. The objective of relating and brainstorming the integration of the devices with current curriculum was not identified as being sufficient enough based on the data collection. The first training may also not have been deemed as important as the following trainings provided by iSchool Campus because the staff was not exactly sure what to expect. A short response from an educators survey states, I have now had the opportunity to utilize and experience technology in my classroom and with my students. I have a lot more questions now than I ever dreamed of having before! As the evaluator of the program, it is recommended that the training continue with the time frame that was used on each topic. It is crucial that the participants have enough time to explore the topic at hand and be able to collaborate with peers. The majority of the objectives were met and future training is desired. Brainstorming ways of how a variety of apps and programs will relate and improve lessons in individual classrooms is encouraged. At the next training, it is recommended to reserve an amount of time for questions and answers to take place for those who need or want it. The main suggestion for this training is to hire someone who has experience dealing with new technological devices actually in the classroom. Based on some of the short answer responses, classroom management was brought up. Only providing the devices and unique ways of implementation is not completely enough for an educator to feel comfortable in delivering and changing their ways of teaching. Another objective of how to handle the devices in the classroom and still allow the exploration of learning and teaching should be a key component in the next training. In summary, the training was successful and the faculty is pleased with it. My recommendations should help improve the objectives for the same training taught again and encourage changes for the future training of this staff and their needs to relate the technology integration with their curriculum and new standards of teaching and learning. The next step is for these educators to feel more comfortable in how the devices are

Tiffany Galloway Evaluation Report being used in their particular classroom, it is not enough to have the great ideas they need to be utilized. Project Cost The budget of the evaluation is focused on the evaluator and the survey creator. The program itself would be rather expensive if the funds covered the trainers and catered services at the training itself. The evaluators daily rate includes 3 days at the training and 3 days for data collection and analyzing into a final report. The survey creator worked for 3 days at a rate of $220 per day. Evaluator 6 days x $500/day (Trainers provided and funded by iSchool Campus for this program evaluation)

$3,000

Survey Creator 3 days @ $220/day

$660

Supplies and Materials

$500

Total Budget $ 4,160

Tiffany Galloway Evaluation Report Appendix A: http://www.surveymonkey.com/s/9L6L7RL

10