Evaluating Your 1:1 Learning Program: One District’s

Journey

Dr. Joseph Maruszczak

Mendon - Upton Regional School District
 Getting Started
• Get an index card
• Think about three core values or skills that shape your thinking
about 1:1 computing in your classroom/school/district
– MURSD’s 1:1 Learning Core Values: Communicate, Collaborate,
Innovate

• Record your three values and explain your reasoning to your

peers
 Historical Context: The Why

The Mendon-Upton Regional School District uses technology to communicate, collaborate,

and innovate, creating an exceptional, personalized learning experience so all will realize
their potential.
 Historical Context: The
Timeline
- Fall 2011: Pilot launched for Grade 7
students (90 total)

- Fall 2012 & 2013: 1:1 Learning for all

students in Grades 6 – 8

- Fall 2014: 1:1 Learning for all students in

Grades 5– 12

- Fall 2016: Proposals solicited for external 1:1

Learning program evaluation

- Fall 2017: Program evaluation released

Evaluation Process: What do We Want to
Know?
 Core Programmatic Questions
Communicate:
1. How are iPads being used by teachers and students to communicate with others?
2. How do our MURSD practices compare to other districts with 1:1 programs?
3. How might we capture and measure our teachers’ and our students’ growth in their ability to
communicate using devices?
4.What next steps are recommended to utilize the devices to enhance communication for
student learning?
Collaborate:
1. How are iPads being used by teachers and students to collaborate with others?
2. How do our MURSD collaboration practices compare to other districts with 1:1 programs?
3. How might we capture and measure our teachers’ and our students’ growth in their ability to
collaborate using devices?
4. What next steps are recommended to utilize the devices to enhance collaboration for
student learning?
 Core Programmatic Questions
Innovate:
1. To what extent do teacher and student uses of the iPad reflect innovation as compared to
Modification and Redefinition stages of technology integration?
2. How do our MURSD practices compare to other districts with 1:1 programs?
3. How might we capture and measure our teachers’ and our students’ growth in their ability to
innovate using devices ?
4. What next steps are recommended to utilize the devices to enhance innovation for student
learning?
Other:
1. To what extent does the MURSD use of technology reflect student-centered teaching practices?
2. To what extend does the MURSD use of technology reflect personalized learning opportunities?
3. To what extent does the 1:1 program evaluation impact other aspect of student learning outcomes
including, but not limited to, MCAS and academic achievement scores?
4. What other core values or opportunities might MURSD consider when planning the future of
technology integration and evolution in its schools?
 Selection Process
• Three proposals received
• Proposal teams have worked nationally on 1:1 program implementations and evaluations
• Applicants included
• the author of the U.S. Department of Education 2016 Tech plan
• Lynch School of Education/ developer of the SAMR Instructional Model
• the program evaluator for the Harvard / MIT edX Massive Open Online Courses (MOOCs)
• a former state-level ISTE Board President
• Selection Factors
• Multiple on-site visits over an extended period of time
• Clear understanding of our Communicate, Collaborate, Innovate goals
• Ability to provide concrete recommendations to advance teaching practices, particularly in
regards to SAMR
 MURSD Evaluation Team

Dr. Damian Bebell

Dr. Ruben Puentedura Dr. Steve Stemler

 The Process
• 5 month study of Grade 5-12
• Multiple Data Sources
• Classroom observations
• Teacher & Student Surveys
• Teacher, Student & Parent Focus Groups
• SAMR model used to provide descriptors of the type of learning
experience for student
• Regular contact between MURSD team & research team to navigate
logistics
• Final report presented in September of 2017
 Teacher Focus Group

Leadership Interview
Student Focus Group
Study Instruments Aligned Our

Class Observations

Teacher Survey
Student Survey
Questions
Communicate
How are iPads being used by teachers and students to communicate with others?
How do our MURSD practices compare to other districts with 1:1 programs?
How might we capture and measure our teachers’ and our students’ growth in their ability to
communicate using devices?
What next steps are recommended to utilize the devices to enhance communication for student
learning?
Collaborate

How are iPads being used by teachers and students to collaborate with others?
How do our MURSD collaboration practices compare to other districts with 1:1 programs?
How might we capture and measure our teachers’ and our students’ growth in their ability to collaborate
using devices?
What next steps are recommended to utilize the devices to enhance collaboration for student learning?

To what extent do teacher and student uses of the iPad reflect innovation as compared to Modification
and Redefinition stages of technology integration?
Innovate

How do our MURSD practices compare to other districts with 1:1 programs?
How might we capture and measure our teachers’ and our students’ growth in their ability to innovate
using devices ?
What next steps are recommended to utilize the devices to enhance collaboration for student learning?

To what extent does the MURSD use of technology reflect student-centered teaching practices?
To what extend does the MURSD use of technology reflect personalized learning opportunities?
Other

To what extent does the 1:1 program evaluation impact other aspect of student learning outcomes
including, but not limited to, MCAS and academic achievement scores?
What other core values or opportunities might MURSD consider when planning the future of technology
integration and evolution in its schools?

direct data source

KEY: indirect data source
 Classroom Observations
146 observations resulted in 3,510 minutes (58.5 hours) of classroom
observations across all major subject areas in grades 5-12:

3 components of MURSD observation rubric:

1. Running Narrative
2. Observation Summary Checklist
3. Time Interval Observation Checklist

Figure 2: Total classroom observations (# of minutes) conducted across grade level

 Student and Teacher Surveys

Surveys were designed to document and quantify: the variety and extent of students’ and
teachers’ technology use, students’ and teachers’ attitude toward technology, teaching,
and learning, as well as students’ and teachers’ beliefs on student motivation,
engagement, communication and collaboration

Teacher Survey n=102 Student Survey n=827

# of Total #of #ofIRB #ofcompleted
Total completed student completed eligible IRBelgible Response
eligible and eligible Response
enrollemt surveys students surveys Rate
teachers surveys Rate
Miscoe Hill 56 53 95% MiscoeHil 785 706 562 518 66%
Nipmuc High 51 49 96% NipmucHigh 619 509 345 309 50%
Total 107 102 95% Total 1404 1215 907 827 59%
 Focus Groups
Teacher Students Parents

n=36 n=31 n=10

Compliment and triangulate survey results and classroom observations

Provide more in-depth information on students’, teachers’ and parents’

experiences and perspectives

Document examples and perspectives towards new opportunities with

communication, collaboration, and innovation

Provide community members an opportunity to share experience, voice,

and perspective
All focus groups were 100% voluntary,
catered, and conducted in May & June 2017
Summary of Major Findings (Triangulated)
• Robust amount of student and teacher technology
use (comparable to international 1:1 study results)

• More frequent use and slightly “higher SAMR” use

@ in upper grade levels (despite shorter
implementation period)

• Large degree of positive attitudes and beliefs

towards technology in school

• Limited evidence of technology use beyond

“Substitution” & “Augmentation” levels

• Numerous and varied examples of Innovation,

Communication, & Collaboration
 General Recommendations and Next Steps

Use newly developed tools to survey students and teachers annually

Define and track success indicators against educational technology indicators

Self-evaluate the appropriateness of tablet devices for all learners

Enrich pedagogy with more Augmentation, Modification, and Redefinition

PD models with a more curricular and pedagogical focus (particularly in lower

grades)

Provide more technology rich offerings to a wider array of students

 Key Takeaways

• Be very transparent and explicit about the goals of your 1:1 learning
program
• Involve all stakeholders… and lay the political groundwork where you
have to
• Tying 1:1 learning to traditional metrics (e.g., MCAS, SATs, etc.) is a
fool’s errand
• Have a framework such as SAMR to use as a lens of your practices
• Use program evaluation as leverage to improve instruction!
Questions/Reflections

Thank you!