Learner Engagement: Working Definitions & Measurement Instruments

Definitions

Engagement at the University/Institution Level: The quality and quantity of participation in

educational activities, interaction with university systems, and utilization of university services at
every level of the university (institution level to course level), as well as the extent to which the
university facilitates this participation, interaction and utilization.

Engagement at the Course Level: The quality and quantity of a student’s cognitive, affective and
behavioral interaction with the course assignments and content, as well as interactions with the
instructor, other students, and the LMS.

● Affective engagement refers to learners’ emotional and attitudinal reactions to the tasks
(i.e., activities and assessments), people (i.e., instructions and students), and systems that
make up the learning environment. Affective engagement also influences cognitive
engagement.
● Cognitive engagement refers to learners’ inclination or motivation to spend time and effort
on learning activities. Cognitive engagement also includes the use of metacognitive learning
strategies and self-regulation.
● Behavioral engagement refers to the observable level of participation or interaction with
the learning environment, other students and faculty, learning activities and co-curricular
activities.

References
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept,
state of the evidence. Review of Educational Research, 74(1), 59-109. Retrieved from
http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edsj
sr&AN=edsjsr.3516061&site=eds-live&scope=site (UMUC login required)

Gunuc, S., & Kuzu, A. (2015). Student engagement scale: Development, reliability and validity.
Assessment & Evaluation in Higher Education, 40(4), 587-610. (Copy available in CILSS Resource
folder:
https://drive.google.com/a/umuc.edu/file/d/0B1DJjZQDZVjJWWFFdFRzNGRxYnc/view?usp=shari
ng)

Measurement Instruments

University Student Engagement Inventory (USEI)

● This recently developed instrument (Maroco, Maroco, Campos, & Fredricks, 2016) attempts
to integrate the four dominant research perspectives on student engagement -- behavioural,
 psychological, socio-cultural and holistic. It included questions intended to measure
affective, cognitive, and behavioral engagement at the university level.
● The instrument includes an adapted post-secondary version of Fredricks et al.’s 15-item
School Engagement questionnaire for upper elementary school students along with 17 new
items created from focus group analysis. Only one of the questions concerns
extracurriculars.
● The authors tested the reliability, content, construct and predictive validity of the 15-item
instrument using a sample of 609 Portuguese non-residential university students majoring
in STEM and social science disciplines. They reported the instrument demonstrated
predictive validity for self-reported academic achievement and intention to dropout.
● While the instrument is not as campus-centric as NSSE, it would still require additional
adaption for use with students who are predominately working adults pursuing their
degrees part-time online.
● The authors’ study, included in the references for this section, includes a copy of the actual
instrument.

Adapted Engagement Scale

● Sun, Martinez, and Seli (2014) also adapted the Engagement Scale developed by Fredricks
et al. (2004) for use with university students. Questions from the Engagement Scale
measuring affective, cognitive, and overall engagement were combined with adapted
questions from the Self-Efficacy for Learning and Performance scale from the Motivated
Strategies for Learning Questionnaire (MSLQ) (Pintrich, Smith, Garcia, & McKeachie, 1991).
● Unlike Maroco, Maroco, Campos, & Fredricks (2016), the authors measured engagement
at the learning intervention level, comparing engagement levels between students who
used clickers in the classroom (n = 95) to respond to conceptual questions and those who
used web-polling software to respond to concept questions outside of class (n=114).
(Students in the web-polling group had significantly higher levels of cognitive, emotional,
and total engagement compared to the clicker group). The mean age of participants, which
included undergraduates and graduates taking education classes was 23.18 years.
● The authors did not provide a copy of the adapted survey with their study, but from their
description in the methods section of the report, it appears as though the modifications
modifications to the Fredricks et al. (2014) questions were relatively minor. It does not
appear as though the authors adjusted questions specifically to address the online versus
classroom context of the interventions.

The Student Engagement Instrument (SEI)

● Developed by Appleton, Christenson, Kim, & Reschly (2006), the SEI is a survey instrument
designed to measure the affective and cognitive engagement of secondary students. The
instrument was initially tested in a random sample design involving more than 1900 9th
grade students.
● The instrument contains 35 items in six subscales: Control and Relevance of School Work;
Future goals and Aspirations; Extrinsic Motivation; Teacher-Student Relationships; Peer
Support for Learning; and Family Support for Learning. The authors reported that the
 factors were generally correlated with GPA, behavioral incidences and student achievement
in the pilot test.
● Subsequent studies involving students in grades 6-12 have found that the SEI functions
similarly across secondary grade levels. In another study, the SEI was also shown to be a
good fit for students with high-incidence disabilities (University of Minnesota Institute on
Community Integration, 2015).
● Grier-Reed, Appleton, Rodriguez, Ganuza, & Reschly (2012) adapted the SEI for use in a
university setting using simple modifications to the existing wording of the questions. Their
study involved 122 undergraduates (the majority were freshman and sophomores) at a
large midwestern university. They found that only the Peer Support factors stood out in
terms of predicting both GPA and career decision self-efficacy. Additionally, the Teacher-
Student Relationships subscale significantly predicted commitment anxiety and external
conflict. According to the authors, both of these subscales resemble several items on the
NSSE. However, they posited that the SEI was better positioned to address the disconnect
between how student engagement is operationalized and measurement across the student
lifecycle from secondary to post-secondary settings.

NSSE, CSSE & AUSSE: Institutional Measures of Engagement

● NSSE focuses on behavioral measures of engagement, including campus engagement
(participation in campus activities), institutional practices and learning behaviors.
● CSSE, the community college version of NSSE also focuses on behavioral measures,
including Active and Collaborative Learning, Student Effort, Academic Challenge, Student-
Faculty Interaction, and Support for Learners.
● AUSSE, the Australian version of NSSE, measures six facets of student engagement:
Academic Challenge (AC), Active Learning (AL), Student and Staff Interactions (SSI),
Enriching Educational Experiences (EEE), Supportive Learning Environment (SLE), and
Work Integrated Learning (WIL).

Additional Reading
Henrie, Halverson and Graham (2015) reviewed and categorized 113 studies that examined student
engagement in technology-mediated learning environments. The majority of these studies used …..
To evaluate the impact of technology-supported learning interventions on student engagement.
The technologies included clickers, Second Life, LMSs, mobile apps, video, audio, online discussion
boards, email, and social media. They found that engagement was not well defined in most studies.
Operationalization and measurement of engagement also varied widely and in some cases was not
consistent with the internal definition of engagement provided within a study (i.e., behavior
measures used to measure cognitive definitions of engagement). The majority of studies relied
primarily behavior indicators of engagement including participation, attendance, number of
assignments completed, viewing or login frequency and other time-on-task behaviors. Studies
measuring affective engagement were primarily conducted in k-12 settings. The authors identified
14 named surveys that were used or adapted to measure student engagement. The NSSE was
most often named. The engagement scale developed by Fredricks et al. (2004) was also used or
adapted as were constructs from Handelsman, Briggs, Sullivan, and Towler's (2005) Student
Course Engagement Questionnaire (SCEQ). Other methods of measuring engagement included self-
reported (by students) levels of engagement, physiological sensors, human observation, video
recordings, and computer-generated usage data.

References
Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and
psychological engagement: Validation of the Student Engagement Instrument. Journal of School
Psychology, 44(5), 427-445. Retrieved from
http://www.psychwiki.com/dms/other/labgroup/Measufsdfsdbger345resWeek1/Krisztina/Appel
ton2006.pdf

Fredricks, J. A., Blumenfeld, P., Friedel, J., & Paris, A. (2005). School engagement. In K. A. Moore & L.
Lippman (Eds.), What do children need to flourish? Conceptualizing and measuring indicators of
positive development (pp. 305-321). New York, NY: Springer.

Grier-Reed, T., Appleton, J., Rodriguez, M., Ganuza, Z., & Reschly, A. L. (2012). Exploring the student
engagement instrument and career perceptions with college students. Journal of Educational and
Developmental Psychology, 2(2), 85. Retrieved from
http://www.ccsenet.org/journal/index.php/jedp/article/download/15531/12898

Henrie, C. R., Halverson, L. R., & Graham, C. R. (2015). Measuring student engagement in technology-
mediated learning: A review. Computers & Education, 90, 36-53. Retrieved from
http://ezproxy.umuc.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=edse
lp&AN=S0360131515300427&site=eds-live&scope=site (UMUC Login Required)

Maroco, J., Maroco, A. L., Campos, J. A. D. B., & Fredricks, J. A. (2016). University student’s
engagement: development of the University Student Engagement Inventory (USEI). Psicologia:
Reflexão e Crítica, 29(1), 1. Retrieved from http://prc.springeropen.com/articles/10.1186/s41155-
016-0042-8

Sun, J. C. Y., Martinez, B., & Seli, H. (2014). Just-in-Time or Plenty-of-Time Teaching? Different
Electronic Feedback Devices and Their Effect on Student Engagement. Educational Technology &
Society, 17(2), 234-244. Retrieved from
http://www.ifets.info/others/download_pdf.php?j_id=63&a_id=1475

University of Minnesota Institute on Community Integration (2015) Check & Connect Student
Engagement Intervention. Retrieved from
http://checkandconnect.umn.edu/research/engagement.html