Formative Evaluation Report: Implementation of Interactive Boards at Burrell High School

Written for: Teachers and Administration of Burrell School District

Written by: Cybil Hill December 11, 2012

Summary The program and tool that was evaluated is the implementation of the interactive boards into classrooms at the high school level. Every teacher received a Smart Board or Promethean Board and necessary tools at various times over the past five years. There was no specified length of time that the implementation would take place. Rather, it is expected that each year, the teachers will continue to develop and adjust their lessons for the classroom to incorporate this particular tool. The general goals include the following: to promote active student learning within the classroom to provide a more visual and multifaceted learning environment for teachers and students to infuse/encourage teachers to update content and teaching techniques to current trends The purpose of this evaluation is to determine whether the goals are being achieved and possible ways to make the implementation stronger. First, the program evaluator had to discuss with the schools Instructional Technologist the goals for the program and to actually write them down (although they were previously unwritten, they were agreed upon by administration). Next, the program evaluator sent out a survey to the teachers to find out their comfort level with the interactive boards, the student engagement because of the boards, the program use, and the training. Finally, the program evaluator also contacted the IT Department to find out additional information regarding challenges experienced with the interactive board tools. The evaluation found that, while all of the goals are being met, follow-up might be necessary to help teachers implement the interactive boards better and become more comfortable with them. Student engagement and interaction is more frequent, according to the teachers, because of the variety of media that can now be shown to the students in order to demonstrate more difficult and in-depth concepts. Students are provided with a more visual and kinesthetic experience in the classes because of the boards. Lastly, teachers are encouraged to infuse more current technologies in the classrooms. However, teachers do not feel very confident in their use of the interactive boards. More hands-on training might be necessary to fix this issue. Also, the teachers also would like time to practice implementing the boards in new and more advanced ways.

Description of the program evaluated The program and tool that was evaluated is the implementation of the Smart Boards and Promethean Boards into classrooms at the high school level. Every teacher received a Smart Board or Promethean Board and necessary tools at various times over the past five years. There was no specified length of time that the implementation would take place. Rather, it is expected that each year, the teachers will continue to develop and adjust their lessons for the classroom to incorporate this particular tool. The intention of the tool was to help the classes become part of the 21 st Century. It was to help the teachers by allowing them more freedom to project different information when presenting lessons, and it was meant to help students by encouraging more student participation, therefore creating a student-centered environment. All teachers at the high school level have access to one of the aforementioned boards. . Program Objectives When the tool was originally put into place, no objectives were recorded. However, prior to the evaluation, objectives were then written down. This is not to say that there were no objectives for the program; they were discussed but never officially recorded in a document. to promote active student learning within the classroom to provide a more visual and multifaceted learning environment for teachers and students to infuse/encourage teachers to update content and teaching techniques to current trends

Program Components The tools that are provided with the board include a projector, surround-sound system that is wired into the ceiling, and Activinspire, a computer program that allows the users to write directly on the computer screen with a pen. The interactive boards were purchased and implemented over three years, which began with the Social Studies department for a Classrooms for the Future (CFF) grant. It continued to most other departments the following year. The last implementation was just to finish up the few teachers who had not received the interactive boards. The program consisted of various small training sessions on how to use the interactive board. At this time, teachers were shown some ways of using the boards, but it mostly focused on how to use ActivInspire, as instructed by the Instructional Technologist. The ActivInspire program consists of an on-screen board that allows users to write directly on the board. There are many tools built within this program, which can be used to supplement many different types of classes. After this initial training, teachers were encouraged to request assistance from the Instructional Technologist on finding ways to implement the interactive boards into their lessons.

They were also encouraged to participate in the trainings found on Promethean Planet on their own time. Evaluation Method The participants chosen for this evaluation included the teachers at the high school in the school district, the Instructional Technologist, the IT Department of the school district, and Ms. Hill, who served as the program evaluator. From the thirty four teachers at the high school, twenty-five participated in the evaluation and responded to the survey. They represented many different content areas, including Math, English, Science, Social Studies, Foreign Language, Physical Education, Art, and Shop. At one point, the program evaluator was going to include the entire school district in the evaluation; however, due to time constraints, she decided to only include the high school as a representation of the school district. Dawn Lovic, the Instructional Technologist for the district, has been in this role for six years. She is also an English teacher in the district and had previous experience with implementing technology into the classrooms prior to becoming the Instructional Technologist for the district. Lastly,the IT Department consists of three members and is responsible for overseeing the purchase of the interactive boards and the installation of the boards. The evaluation began with an interview with the Instructional Technologist to determine the program objectives. Since no objectives were recorded at the time of the implementation, the Instructional Technologist and Ms. Hill discussed and documented possible objectives from what was known about the school districts plans for implementing technology. At this time, three objectives were determined as being viable for implementing the interactive boards into the classroom. Ms. Lovic stated that, although no objectives were formally written down, the objectives were discussed in length and were basically conveyed to and understood by all members of the district who would be a participant in some way for the implementation of the boards. Once the objectives were determined, the program evaluator determined appropriate survey questions that were to be sent to teachers of the district. After writing the survey, Ms. Hill requested permission to proceed with administering the surveys to the teachers of the school district. She obtained permission from the Superintendent Mrs. Wagner. Once permission was obtained, Ms. Hill decided it would be better to only survey the teachers at the high school. The surveys were sent to the teachers, who had approximately two weeks to complete it. During this time, Ms. Hill also requested information from the IT Department regarding the complete cost of each interactive board and the number of boards installed at the high school. During the evaluation process, no changes were made to the use of the interactive boards. All teachers continued to use the boards as normal, which meant that normal issues occurred during the process. This included a noticeable issue with screen quality and the life of the bulbs used in the projector. The teachers were not

aware that the evaluation was occurring prior to receiving the survey, which allowed for them to continue using the interactive boards as usual in their classes. No interventions were used as part of the evaluation. Participant Attitudes Various questions on the survey focused on participant attitudes while using the interactive boards. Teachers were asked to reflect on their and their students experiences using the interactive boards since they were installed. They were also asked to reflect upon their training and provide their thoughts on the training procedures and preparation prior to and during implementation. Program Use The teachers were also asked on the survey to reflect on their usage of the interactive boards per week. Not only were they asked to report the number of times they use the board during a regular week, but they were also asked to report how they use the boards, albeit solely as a whiteboard or to show other online resources and to provide the students with a 21st Century learning experience. Training On the survey, explanations of training were requested. The teachers were asked to describe the training they received for integrating interactive boards, encouragement to update content and teaching techniques to include the use of interactive boards in the classroom, and any challenges they have found during the process. Data Collection The teacher survey was provided through the use of an online survey tool called Survey Monkey. The survey was then e-mailed to all of the teachers via their school email addresses. The information was obtained from Ms. Lovic, the Instructional Technologist, through a face-to-face interview. The information about the interactive board installation was provided by e-mail discussions with the IT Department, and more specifically, Kellie Speer.

Results Participant Attitudes- Teachers Levels of Comfort This section summarizes the participant attitudes and comfort level with the interactive boards as reported in the survey, found on Survey Monkey.

Survey Results Extremely Mostly comfortable comfortable How comfortable are you with implementation of the interactive boards in your lessons? 28% 28%

Somewhat comfortable 32%

Not comfortable 12%

As shown in the table above, only 12% of the teachers (three out of twenty-five teachers) do not feel comfortable at all with the interactive boards. Interestingly enough, there is only a small difference in percentage between extremely comfortable, mostly comfortable, and somewhat comfortable. While 28% of the teachers feel extremely comfortable and mostly comfortable, 32% of the teachers are somewhat comfortable. It is important to note that, while there seems to be a 4% difference between extremely comfortable/mostly comfortable ratings and the somewhat comfortable rating, in actuality this is only a one person difference; seven teachers are extremely comfortable and seven are mostly comfortable, while eight teachers are somewhat comfortable. Later in the survey, multiple teachers reported that in order to feel more confident and comfortable with the implementation of the interactive boards, the school district should provide hands-on training and time during in-service days and throughout the school year to either train or receive a refresher course on the software. One teacher suggested that trainings on both the basic and more advanced things would be beneficial. Four teachers said that there is nothing that can be done to help with implementation, while one stated that Its an issue of time in most cases. Not resources. Three teachers said that they do not currently have access to one, which could relate to the three teachers in the previous question who reported to not being comfortable at all with the interactive boards. Lastly, four teachers skipped this question.

Participant Attitudes- Perceived Student Attitude and Engagement This section summarizes student engagement with the interactive boards as reported in the survey, found on Survey Monkey. Survey Results Yes Are your students more actively engaged in the content when you use interactive boards in your classroom?

No

No response

48%

24%

16%

The teachers were asked to describe the engagement of the students in the classrooms based on the use of the interactive boards. As reported above, almost half of the teachers surveyed determined that students are more engaged with the use of the interactive boards, while 24% did not see more engagement. 16% of teachers did not respond to this question on the survey or did not give a yes-or-no response. Some teachers also elaborated on this information by stating that it helps with engagement because there is less down time and it brings math to life. Other teachers explained that students arent impressed by the board and they are passive like they are watching tv. Program Use This section summarizes the use of the interactive boards as reported in the survey, found on Survey Monkey. Survey Results Every day, 3-4 days a multiple week times a day How often do you use the interactive board each week? 40% 32%

1-2 times a week 16%

Rarely 12%

As is shown in the above table, 40% of teachers use the board every day, multiple times a day, while 32% use it at least 3-4 days a week. Only 16% report using it 1-2 times a week and 12% of teachers rarely use it. It is important to note that eighteen teachers, or 72%, report using it more than half of the week. Also, only three teachers rarely use it; this number can be accounted for by the fact that three teachers reported not having an interactive board installed in their classrooms.

In what ways are interactive boards used in the classroom?

2, 8%

4, 16% Only Whiteboard/Notes Other N/A

19, 76%

The teachers were also asked to describe the ways they use the interactive boards in the classrooms. Two teachers reported that they do not have an interactive board, which is shown in the above pie chart as N/A (Not Applicable). Only 16% of the teachers use the interactive board for notes or as a whiteboard. Overwhelmingly, 76% of the teachers reported using the interactive board in ways other than solely as a whiteboard or to share notes. The teachers explained that they use the boards also to show Internet websites; to present interactive lessons utilizing games, Activoters, online graphing calculators, and other tools found in Activinspire; to demonstrate difficult concepts through videos, Powerpoints, and 3-D models; and to share documents and assignments with other students. Training Lastly, the teachers were asked to respond to multiple open-ended questions regarding the training they received, the support from their administration, and the challenges they have encountered. Three teachers stated that they did not receive any training on the board because they do not have them in their rooms. Most teachers reported that the training the received by the district was minimal and that it was mostly trial and error. A few in-service trainings covered a lot of information but offered little practice or mastery. Also, one teacher reported that he/she received help from Dawn Lovic on multiple occasions. There were multiple teachers, lastly, that stated that they received extensive training as part of the Classrooms for the Future (CFF) grant. When asked whether their administration encouraged them to update their lessons to implement the interactive boards, only two people responded with no and one person responded with Not Applicable. Everyone else stated that they were encouraged to update their lessons to incorporate the interactive board into their

lessons. One teacher stated that the district has really pushed us to use more technology in the classroom. Lastly, when asked to describe any challenges that they encountered, technical issues were reported that include calibration of the board with the pen frequently; connection breakdowns, including Internet usage; and the time consuming nature of preparing an interactive lesson. One other problem that was not reported on the survey, but was experienced by the program evaluator, is that often, the light bulb will flicker different colors because of problems within the projector itself. Kevin Pasko of the IT Department stated that this is an ongoing problem and that it deals with the quality of the projector purchased. If a higher quality projector had been purchased, then this problem would not occur as often, if at all. Discussion The purpose of this evaluation is to determine if the tools are being used as expected by the administration. The Instructional Technologist stated that the goals for the interactive boards are as follows: to promote active student learning within the classroom to provide a more visual and multifaceted learning environment for teachers and students to infuse/encourage teachers to update content and teaching techniques to current trends

As is shown from the data, it appears that the goals are being achieved. Active student learning is displayed through the use of the interactive boards by the majority of teachers as more than just a whiteboard; most teachers are providing the students with other ways of understanding the information through the use of online and other computer tools. The interactive boards provide more appeal visually than do chalkboards, which will assist the learners who learn information best visually in comprehending and processing the information. Lastly, by providing teachers with the interactive boards, administration is encouraging an increase in technology and, therefore, an update to content and teaching techniques to fit with the current trends of the 21st Century. Participant Attitudes- Teachers Levels of Comfort As displayed in the results, the teachers are all at least somewhat comfortable with the interactive boards. However, according to these results, the majority are only somewhat comfortable. However, as noted in the results, eight people are somewhat comfortable, seven people are mostly comfortable, and seven people are extremely comfortable. This data is very close and shows that comfort levels are varying. This can be for various reasons, including the amount of training that each teacher received,

comfort with other technological tools, and willingness to implement 21st Century tools in the classroom. According to the responses given by teachers, in order to increase the confidence in using the interactive boards, more time to practice and more guided practice throughout the school year might be useful. Participant Attitudes- Perceived Student Attitude and Engagement According to the data, almost half of the teachers believe that the interactive boards increase engagement from the students, which is a response to the first objective of promoting more student participation in the classroom. Although this is a subjective question and is based on perceptions, the teachers often can tell if students are engaged in a class or not based on their reactions. Program Use 72% of the teachers surveyed use the interactive whiteboards at least three times a week. Of the teachers surveyed, 76% admit that they use the interactive boards for other reasons than as a white board to present notes. This is an accomplished goal of the district. The district purchased the interactive boards in order to encourage interactive board implementation into the lessons. Overwhelmingly, the teachers are achieving this at the high school. Training Lastly, training is an area in which the district is lacking. Many teachers feel that they are self-taught when it comes to the interactive boards and have learned based on trial and error. Also, multiple teachers request more training throughout the year in order to share new ways of using the interactive boards in their lessons. However, there are teachers who feel that they have had plenty of training because of the CFF grant received by the school district. Standout Component According to this evaluation, many teachers are willing to incorporate technological tools in their lessons and implement the interactive boards. However, the lack of comfort with the interactive boards stands out as being a problem; when a person is not comfortable with a tool, it will not be used. The lack of hands-on training and practice, as stated by many teachers, seems to be a problem area that could help to sway the discomfort with the tools. If more hands-on training was provided (along with time to practice), this might help teachers incorporate the interactive boards into their lessons. Also, while many teachers have already accomplished the basic tasks with Activinspire, another possible training might be to show ways of using advanced tools provided by the software. Overall Evaluation As demonstrated by the data, the goals for the interactive boards are being achieved to some extent. The teachers are obviously using the technology to incorporate a variety of sources in their lessons, which then will engage students more

and help them to become 21st Century learners by building the necessary skills. Also, it assists with differentiating instruction for visual learners and kinesthetic learners, which is also an important concept currently in education. Lastly, because of the availability of the resources, teachers are able to easily integrate the interactive boards into their lessons. As stated above, one area that is lacking is with hands-on training. Without the proper training, teachers are not fully comfortable with the boards, which does not allow them to use the boards to their full potential. More attention needs to be provided to hands-on training and practice with the interactive boards throughout the year or during in-service days. By providing this practice, confidence and comfort levels will increase, which will give teachers an advantage when planning an interactive lesson. More time to practice is requested and should be considered for upcoming in-service training days. Also, more advanced applications for the software should also be considered.

Project Cost PERSONNEL Cybil Hill: 25 days x $500/day TOTAL PERSONNEL OPERATING Supplies Copies, printing, other paper supplies, etc. TOTAL OPERATING TOTAL PROJECT BUDGET $12,500 $12,500

$100 $100 $12,600

The total number of days includes the time it took to interview the Instructional Technologist and the IT Department. It also includes the time it took to receive completed surveys from the teachers. The supplies include printed surveys for those teachers who were not able to access the survey online or who did not complete it. The total cost of everything is $12,600.

Appendix A The survey can be viewed on Survey Monkey by following the link: http://www.surveymonkey.com/s/KMYDSKP The survey results are as follows: Question Responses 9-12 1. What grade level(s) 9 - 12, Health/Phys. Ed. and/or subject(s) do 9th - Biology 1 11th/12th - Biology 2 Honors you teach?
9-12 guidance High School Library Science 9-12 11/12 Language Arts, Journalism Secondary 9 Emerging Tech, 12 Personal Finance, 9-12 Programming and Accounting Special Education Grades 9-12 10, 11, 12 World Cultures Sociology Psychology 9th Grade American Cultures French 2, 3, 4, 5 6-12 Health/PE Physics I Physics II and Physical Science 10 and 12 English 10th and 11th Grade Social Studies 11-12 sophomore chemistry, physical science 9-12 Math 11 9 - 12 math American Cultures 9 Government 12 earth and space science 6th and 8th environmental science 11th and 12th grade elective 9-12

2. How often do you use the Smart Board/Promethean Board each week? Choose the response that works best for you. 3. In what ways do you use the Smart Board/Promethean Board? (ie. Are they only used as a white board or do you use other tools, gadgets,

I use the board every day, multiple times a day. 40%; 10 people. I use the board at least 3-4 days during the week. 32%; 8 people I use the board 1-2 times a week. 16%; 4 people I rarely use the board. 12%; 3 people.
Computer Labs Don't have Smart Board/Promethean boards, I only have a projector. Which I use daily to teach new techniques, show videos, give examples. white board TO run powerpoints, use activoters and annotate over the desktop Typically I will us e it for powerpoints or videos Internet, Interactive lessons, Interactive games, showing class

and Internet to enhance your lessons?)

notes Only as a white board web exploration, film study, document sharing, along with usual white board capabilities White board, Tools, Internet, and Videos Not at all, I don't have one Smart board, I pad I use it to project presentation software, film, etc. Mainly as a whiteboard or projection screen. From time to time, I create a more interactive lesson. Internet and white boards They are great at showing demo's that would be next to impossible to draw or set up White board and to show movies All of the above. show internet, show examples, show 3-D models, students play games on it to present information, a space for the kids and I to work out problems, presenting info. from the internet, showing videos, simulations, to record and present information White board only all of the above Internet, video, interactive games Flipcharts for notes and online graphing calculator Power points Student responses Interactive internet site Other interactive activities screen to show powr points and videos white board, games, wiki responses, etc

4. How comfortable are you with implementation of the boards in your lessons? Choose the best response.

I am extremely comfortable with the board. 28%; 7 people I am mostly comfortable with the board. 28%; 7 people I am somewhat comfortable with the board. 32%; 8 people I am not comfortable at all with the board. 12%; 3 people
Don't have the board, I've never been trained and if I were in a room with one I would probably just use it as a regular projector screen. Limited. 30 minute demonstration. Had a 3 hours training on it, but mostly trial and error. I had a 30 minute training during one of our professional development days. I have had several workshops on using the Smartboard in the classroom. None Little to none of use. I figured out how to use it largely by myself. In-house hands on training... 2 hour training before the boards were installed a few years ago. District trained Extensive. The social studies department received a Classrooms

5. Describe your training in the integration of Smart Boards in the classroom.

for the Future grant and were given a copious amount of training regarding the SmartBoard. We had a few in-service trainings. The software is not terribly user friendly. Limited and too fast. Here it is, figure it out A very quick 2-hour presentation that covered a lot of information but offered little practice or mastery. Several workshops. very little...pretty user friendly I've had no formal training - only picked it up by trial and error or asking other teachers for help Pretty extensive none Part of CFF Honestly, I don't really remember my training but I have figured out a lot of things on my own. In-service training at Burrell Help from Dawn Lovic Self-taught one inservice class inservice training + 1 conference no yes Yes Yes, the district has really pushed us to use more technology in the classroom. Yes Yes they only mention it. Yes NA Yes....Dawn is great. Yes. Yes. Yes yes Yes Sometimes. somewhat I don't remember them directly telling me to update for the Promethean Board, but since it covers up most of the only chalk board in my room, I assumed it was required yes all the time Somewhat I have never been "required" to use it for teaching. No due to the fact that I have used it since it became available. However they are always willing to assist when questions arise. no yes

6. Has your administration or your instructional technologist encouraged you to update your content and teaching techniques to include the use of Smart Boards in your teaching?

7. In what ways can the district help you to

none, I don't even have room for one. 11/29/2012 10:10 AMView Responses

implement the board more often in your classes? Explain.

I am a hands on person. Once someone shows me the information I would need to improve my skills and confidence with the board, I would need practice time with step by step instructions. None Provide teachers with tips on using the Smartboard. Workshops both informal and formal throughout the school year. Install one in my room clear expectations in writing as to exactly how they would like it utilized in ways that I am not currently using it. In-Service, morning meetings, and JIT (Just In Time ) Training. Get me one? More time for training during inservice. I do not believe that they can. They have been actively pursing the use of technology in the classrooms over the last five years. Offer more training with the software. More instruction - hands on would help. By giving us more time to practice and train. It's an issue of time in most cases.... not resources. none Trainings on both the basic and more advanced things that can be done with the board trainings Refresher courses during in-service days Continue support for training and upgrades. I am always willing and able to learn new things refresher inservices on the use or upgrades

Four people did not respond. 8. Are your students more actively engaged in the content when you use Smart Boards in your classroom? Explain why or why not.
Does not apply Not really. My use of the board is very limited, mostly for note taking. Yes, especially with use of Activoters I dont think so, they are passive like they watching tv. Yes. The students are excited to come up to the board and answer questions or play an interactive game. No, they prefer more hands-on activities. It probably helps visual learners more to some degree. Nevertheless, the device is nothing more than a more techno-

savvy way disperse information when comparing it to a chalkboard or overhead projector. Eventually there will come a time when administrators tell us not to use the "old and outdated whiteboard" and promote the use of some other new-fangled technological learning tool. Yes... it gives them something to concentrate on. Sometimes, I think adding a multimedia aspect to the classroom is also better. NA I think that they are engaged because it is interactive and we can do activities together. No. Apathy reigns supreme. The majority of the students would not be moved if a Zambelli fireworks display erupted in the middle of class. No. The SmartBoard is a great tool, but it is not much different than an overhead projector. Depends on the lesson. NO Not really---stuents aren't impressed by the board and sometimes it doesn't work very well (e.g., the pens, etc.) Depends on the exercise. a little, because the certain internet sites help teachers bring math to "life". It helps the student visualize the mathematics which leads to more understanding Sometimes. They especially like graphics and/or movies, and it's great to have them come to the board and work out problems. The Promethean Board enables me to carry out both of those activities more easily than with more traditional methods. I've been able to download some interactive Flipcharts, but I don't know how to set up those special effects in documents that I create myself No, they are so used to technology yes Yes, they participate and its just more appealing I like how I can have most diagrams and problems already on the board before class so I am not wasting time writing out every example. I would say there is less "down time" making students more engaged. Yes. The board acts to focus their attention particularly when using interactive activities. somewhat sometimes, depends on the lesson Does not apply I am very technology challenged. Sometimes Annotating over the Desktop does not work on my new laptop with ActivInspire. Technology glitches with sound or the board not responding to certain commands. It can be time consuming to create interactive presentations for daily use. It's an awkward set up. setting the thing up at the beginning of the year is always a hassle. I've also learned that if technology is mandatory for a given lesson it will fail--thereby wrecking the entire lesson. I never use the device as the focal point to a lesson for that reason, rather, only as a supplement. Sometimes the technology does not work flawlessly. It is always

9. Describe any challenges you have faced in using the Smart Board in your classroom.

good to have a back up plan. NA Sometimes it is slow to respond or doesn't respond to the pen. Technology issues (board won't respond to the pen, board needs calibrated daily, etc.) The software is difficult to use. Calibration. Stopping my student from taking every note they on the board. They must be overusing these board in other class because when I have notes or power point on the board the students copy it verbatim, when it does not need to be. It's sometimes difficult to write on and doesn't allow me much freedom of movement. There are times when it needs recalibration or other minor issues. none I've pretty much had to teach myself everything or ask another teacher for advice. I'm guessing there are possible bells and whistles for me to use that I don't know about Doesn't always work. pen not responding properly Internet is down, maintenance hits the projector & getting exact calibration can be difficult It's technology .... it does not always work. You have to be prepared in case something happens with your board. Switching from use of ActiveStudio to ActiveInspire. it seems to need calibrated frequently connection breakdowns - internet usage

Appendix B The following question was posed to the Instructional Technologist Dawn Lovic regarding the goals of the program: What are the goals of the program? Were they written prior to installing the interactive boards?