PROF.

NIKOS MATTHEOS (Orcid ID : 0000-0001-7358-7496)

Accepted Article
Article type : Original Article

Self-Regulated Learning strategies in world's first MOOC in Implant Dentistry.

Lan Min1, Hou Xiangyu,2 Qi Xinyu 3, Nikos Mattheos4

1. PhD Candidate, Information Technology in Studies, Faculty of Education, The University of Hong

Kong

2. PhD Candidate, Electrical and Electronic Engineering, Faculty of Engineering, The University of

Hong Kong

3. Lecturer, Technology-Enriched Learning Initiative, The University of Hong Kong

4. Clinical Associate Professor, Implant Dentistry, Prosthodontics, Faculty of Dentistry, The University

of Hong Kong

*Corresponding author:

Nikos Mattheos

Clinical Associate Professor

Prosthodontics, Prince Phillip Dental Hospital

This article has been accepted for publication and undergone full peer review but has not
been through the copyediting, typesetting, pagination and proofreading process, which may
lead to differences between this version and the Version of Record. Please cite this article as
doi: 10.1111/eje.12428

This article is protected by copyright. All rights reserved.

 34 Hospital Road, 3F, Bloc B,

Sai Ying Pun, Hong Kong SAR PR China

Accepted Article
nikos@mattheos.net

Tel. +852 5188 5893

Abstract

Background: Massive Open Online Courses (MOOCs) have been often described as a major

innovation Higher Education, but their application in the teaching of clinical disciplines is still very

limited, while there is a lack of scientific evaluations in this domain. The aim of this study was to

investigate learners' behaviors and correlate patterns of Self-Regulated Learning (SRL) with

performance and achievement during a MOOC in Implant Dentistry.

Methods: The clickstream data of learners from the first run of the MOOC Implant dentistry by The

University of Hong Kong, was modelled and quantified based on Zimmerman’s SRL model. The data

was quantitatively analysed by means of k-means clustering for evidence of five SRL behavioural

indicators of student activity. The clusters identified were then correlated with student engagement

and learning performance.

Results 7608 individuals enrolled, 5014 engaged (active learners 65.90 %), 1277 of them (25.47%)

completed the course and 1232 purchased a certificate. Two major groups of learners emerged:

Attentive (n=1433) who were more likely to follow the prescribed pathway in the MOOC and

Auditors (n=3581) who accessed content selectively. There was significantly higher engagement,

achievement and completion rates among Attentive than Auditors. Both groups included

subcategories (Browser, Digger, Test-driven, Sampler, Persistent) which might reflect different SRL

strategies.

This article is protected by copyright. All rights reserved.

 Conclusion: A MOOC in a clinical discipline can achieve high enrollment and completion rates as

compared to current benchmarks. There appears to be a wide diversity of learning behaviors among
Accepted Article
learners, with two however dominant patterns. Learners with a linear learning pathway achieved

significantly higher grades and completion rates than those who accessed content irregularly and

selectively. Such differences however might be influenced by the demographic and professional

background of the learner, as well as their motivation to attending the MOOC. Certain learning

behaviors, in particular how learners access content in relation to assessments might be closer related

to SRL.

Introduction

A Massive Open Online Course (MOOC) is an open online learning environment which can

simultaneously cater for an unlimited number of learners. The term is reported to have been

introduced in 2008 (1) and quickly gained traction, describing a learning instrument that offers

learners and tutors an open and continual form of networking beyond the limits of conventional online

courses (2,3). A large number of MOOCs are available on an ever-growing array of subjects, with the

most established online platforms reported to reach more than 101 million learners worldwide (2).

The recent development of MOOCs has been hailed by many educators as a significant innovation

within teaching and learning, with the potential to revolutionise the landscape of university education

worldwide (1-3).

Despite the rapid growth of MOOCs in most university disciplines, expansion in clinical sciences has

been slow. The role of open and online learning in the teaching of clinical disciplines, such as

Medicine and Dentistry, has been debated, with educators being divided when assessing the potential

of MOOCs (4,5). It is clear that MOOCs are not to replace dominant models of clinical education

such as residencies and face-to-face teaching environments. However, some educators believe

MOOCs could become the transformative pedagogy to help dental schools overcome many of the

This article is protected by copyright. All rights reserved.

 present challenges, such as increasing student tuition and debt, decreasing funds for faculty salaries

and associated faculty shortages, as well as the high cost of clinic operations (6).
Accepted Article
Implant Dentistry is one of the most recent and rapidly developing clinical subjects within Oral

Health Sciences, evolving around the replacement of missing teeth with surgically placed endosseous

titanium implants. The teaching of Implant Dentistry in the undergraduate curricula is limited, and it

varies significantly between regions and institutions, with universities having many difficulties to

adequately implement it (7,8). As a result, the majority of dental graduates worldwide start their

careers with a very limited understanding of dental implants as part of comprehensive care. Even

long-practicing dentists who were never taught implant dentistry in their curricula have little

opportunity to reach flexible, quality-assured and unbiased education in this discipline. The lack of

human and material resources and expertise is repeatedly cited as one of the obstacles for the

implementation of Implant Dentistry to the undergraduate curriculum to the required extend (9,10). In

that sense, sharing of teaching resources and content at a global level through open and online

learning has been often proposed as a possible remedy to address such deficiencies (11). As of the

time this project was launched (2015), there was no MOOC attempted for the teaching of clinical

disciplines in Dentistry, while few MOOCs had been launched within disciplines of clinical medicine

(12-13). The methodological challenge therefore was evident, to design a robust pedagogical

framework but also to produce evidence of the learning impact for the teaching of clinical sciences

with this medium. Under such a massive and autonomous learning environment, students’ self-

monitoring and organization become of paramount importance, demanding a higher self-regulated

learning ability. According to Zimmerman (14), the degree to which students are metacognitively,

motivationally, and behaviorally active in steering their own learning is described as self-regulated

learning (SRL). As a MOOC caters for large numbers of learners with diverse motivation,

professional background and learning needs, the SRL ability greatly varies among learners, some of

who are students, while others are practicing clinicians under different settings, expertise and

competence.

This article is protected by copyright. All rights reserved.

 Aims

The aim of this study was to evaluate learners’ dynamics and learning behaviours during the first run
Accepted Article
of a MOOC in Implant Dentistry. In particular, the study aimed to investigate patterns of SRL

behaviours through data mining and investigate correlations of such patterns with the learner’s

activity and performance.

Material and Methods

This study was approved by the HKU/HA West Cluster Institutional Review Board with reference

number UW 16-1005.

a) Pedagogic framework and structure

The authors used Zimmerman’s SRL model (14), in order to develop the pedagogical framework of

the MOOC. Zimmerman approached SRL from a cognitive point, where SRL is characterized by

three phases:

1. forethought phase (F): task analysis and self-motivation

2. performance phase (P): self-control and self-observation

3. self-reflection phase (S): self-judgment and self-reaction

Using the three phases of the SRL theory as scaffold, a multidisciplinary learning environment was

designed which could foster the growth of effective learning in three learning “pathways”, developing

a vertically and horizontally integrated matrix:

(A) Theoretical fundamentals and foundation knowledge, mainly presented through short seminars,

supported by discussion boards suggested readings and assessed by Multiple Choice Quizzes.

This article is protected by copyright. All rights reserved.

 (B) Clinical applications and procedures, mainly presented through practical tutorials, clinical videos

and case studies, supported by discussion boards suggested readings and assessed by Multiple Choice
Accepted Article
Quizzes.

(C) Application of knowledge, competences, decision making and guided reflection, by means of

three tailor-made “virtual” patients seeking help. This was also supported by discussions and

assessment was done through non-graded peer and self-assessment. Each patient is presented initially

at the start of the course and the problems are unfolding gradually as the course advances, while

certain tasks are requested by the students at each stage. The model was adopted from the “Interactive

Examination” concept (15), during which the students engage in a guided process with an actual

patient problem, define the parameters of the problem and propose solutions based on the theoretical

and experiential content. A peer solution is then utilized to kickstart a guided reflection, aiming to

help the students identify strengths and weaknesses in their own thinking and define new learning

objectives (15).

b) Learning content

The content and learning objectives of the MOOC were derived from the competencies defined by the

European Consensus Workshop in the university teaching of Implant Dentistry (16) for the graduating

dentist. Fifty-two distinct competencies (11 major and 41 supporting) were encoded into 28 learning

objectives, which were consequently mapped to the three learning pathways and organized in 5

modules composed of 17 lessons (Figure 1).

An evident but non-binding “prescribed” learning pathway was established, which encouraged the

learners to follow a sequential access to the different components of the course completing one

module per week. Gradual advancement on all three pathways through the 5 weeks of the course was

engineered through regular “checkpoints” and mentoring by experienced tutors and peers. In addition,

This article is protected by copyright. All rights reserved.

 the three pathways were color coded throughout the course's interface with distinct colors maintained

for each one e.g. in video backgrounds, notes etc. Learners remained however free at all times to
Accepted Article
access any content in their own preferred sequence and timing.

Twenty internationally acknowledged experts in respective fields of Implant Dentistry were invited to

contribute according to their expertise by developing content as directed by the map of the identified

competences and learning objectives. The MOOC ran for four rounds on Coursera

(www.coursera.org) during 2016-2017. The analyses in this study are based on the sample form the

first run which took place from 25th October to 5th December 2016.

c) Data collection

The numbers of learners enrolled in the course, as well as the number of course completers and

certificate holders were recorded. The clickstream data of all enrolled learners were collected,

representing user interactions with the content within the learning platform. Such data are matched

with the unique user ID and include any course item clicked or accessed by the user, in the sequence

and the time it occurred.

d) Data analysis

The course was organised in 5 modules, each of which has a number of lessons. Each lesson was

composed by a variety of resources such as lectures, clinical procedures videos, discussion boards and

a graded assessment. The way the students navigated the content was recorded in their individual

clickstream. The clickstream data was modelled and quantified based on the Zimmerman’s SRL

model The data was analysed for evidence of five SRL indicators of student activity performance,

pointing to student learning patterns: a) Time management (TM), b) Task strategy (comprehension)

This article is protected by copyright. All rights reserved.

 (TS), c) Self-instruction (assessment driven) (SI-ad) d) Self-instruction (systematic learning) (SI-sl)

and e) Application (AP). Definitions of the SRL indicators used can be seen in Table 1. Engagement
Accepted Article
(EN) was defined as the number of completed lessons out of the total number of lessons accessed and

was expressed in percentage from 0-100%. Scores on the basis of the completed graded course

assessments were calculated for each learner and expressed as a final grade (percentage passing grade:

75% - maximum possible 100%). A Python script was programmed in Jupyter web-based platform

(Project Jupyter, jupyter.org, version 5.4.0). Quantitatively, the toolkit of pandas, counter, sklearn

(e.g. k-mean clustering), and n-gram were used to identify clusters of learners based on the indicators.

The identified clusters were qualitatively classified using the constant-comparative method, a

technique that is typically recommended for systematic review pursuits (17). Students behavior for

each indicator was defined as “high level” if the clickstream suggested a score above the average and

“low level” if below the average, based on the defined metrics for each indicator. Pearson correlation

was used to investigate correlations between learners’ behaviors and performance, (grades,

engagement completion rates). Mann-Whitney U test and Chi-square test for independent samples

run on SPSS (IBM, version 24) were used to investigate significant differences in prevalence of

patterns among the different identified clusters of learners.

Results

a) Enrolment and completion

During the first run, 7608 individuals enrolled in the MOOC. Out of them, 5014 learners (65.90 %)

executed at least one action interacting with the content thus becoming “active learners”, and 2957 of

them (58.97 %) completed at least one assessment. 1277 learners completed the course, while 1232 of

them went further to purchase a certificate. Completion rate was 25.47% of active learners or 16.78 %

of the enrolled learners. A certificate was purchased by 96.48 % of the learners who qualified

(completers). The first certificate was acquired already on the first day after the start of the course,

This article is protected by copyright. All rights reserved.

 while 35 learners completed all course assessments by the end of the first week (Figure 2). The

majority of completions and certificates were achieved before the 5th and last week of the course.
Accepted Article
The results of the correlation of the indicators can be seen in table 2. Time management (TM) was

moderately and negatively correlated with engagement (EN) (r = -.303, p < 0.01) and also to the final

grade (r = -.300, p < 0.01). The self-instruction of systematic learning was highly and positively

correlated with the final grade (r = .841, p < 0.01), completion (r = .792, p < 0.01), earning certificate

(r = .781, p < 0.01), and engagement (r = .880, p < 0.01).

Patterns of Self-Regulated Behaviour

K-means clustering resulted in the identification of eight clusters among participants in the MOOC

(Table 2). The learners' types were further organised in two major patterns in terms of their learning

engagement on activities, namely “Auditors” (n= 3581, 71.42 %) and “Attentive” (n=1433, 28.58 %)

. Auditor describes participants who demonstrated interest in specific parts of the content and

typically did not follow the prescribed sequence of instructional design (clusters 2, 3, 4, 5, 7, 8 ).

Attentive describes participants who appeared to follow the prescribed pathway, typically including

passing the assessments in sequence and completion of the course. Based on the demonstrated

behaviors, Auditors included a) Browser, b) Test-driven, and c) Digger, while Attentive included e)

Sampler and f) Persistent learners.

Browser refers to participants who went through a few activities mainly at the initial modules of the

course, which they did not revisit such as the participants of clusters 2, 4 and 8. Test driven describes

participants who appeared to revisit activities especially after taking graded assessments, such as the

participants of cluster 7. Digger characterizes participants who were focused in few specific materials

such as video lectures, which they however accessed repeatedly multiple times, such as the

This article is protected by copyright. All rights reserved.

 participants of cluster 3 and 5. Among the Attentive learners, Samplers were students who did not

revisit already accessed content, even after taking assessments as in cluster 1. Persistent were learners
Accepted Article
who were more frequently revisiting content, in particular revisiting lectures after taking assessments,

such as cluster 6. In terms of achievement, significant differences were found in final grades,

completion rates as well as acquisition of certificates for learners of the different clusters (Table 3).

Discussion

MOOCs are no longer a novelty in the Higher Education landscape, with the most popular platforms

collectively reaching more than 81 million students already. Nevertheless, learners' behavior and

dynamics within these new learning environments remains poorly understood, as research remains

still scarce. With regards to healthcare, although there is at present a growing number of MOOCs,

published studies are very few and merely reporting expert opinions (4) or descriptive case studies

(18). Although peer-reviewed publications based on MOOCs in healthcare disciplines are emerging,

published studies so far discuss mainly non-clinical MOOCs in domains such as Pharmacology and

Allied Health (18), Medicine as Business (19, Dentistry for non-dentists (20), while there is still a

scarcity on a scientific evaluations for clinical disciplines. It becomes therefore imperative to

approach such learning environments in a scientific and systematic manner, in order to determine the

potential role of such learning platforms in the teaching framework of clinical disciplines.

It is difficult to benchmark enrolment and completion rates in a MOOC, as the figures vary

significantly among courses and disciplines and the drop out rated is typically very high. Completion

rates among MOOCs typically vary between 2-11%. In healthcare, A recent report on 4 non-clinical

MOOCs reported higher than the average completion rates, placed between 4.3-11 % (18).

Conclusively, one can note that in comparison to the current benchmarks, Implant Dentistry was well

attended and achieved exceptionally high completion rate, pointing to a favorable potential of this

learning platform for the teaching of clinical disciplines.

This article is protected by copyright. All rights reserved.

 Engagement, is term difficult to define and even more to assess in a MOOC environment, as the term

implies not only quantitative but also qualitative characteristics. In some reports engagement is
Accepted Article
defined as the percentage of learners that performed an "action" among the complete number of

students enrolled (18). As this action could however imply just a single click in a video lecture, what

this indicator mainly shows is rather how many students enrol without ever bothering to just click

once anything in the course. It is apparent that we need much better indicators in order to understand

and assess the actual engagement of students with the learning experience. At the same time there is a

limitation in what clickstream data can offer, without being supplemented with qualitative data from

each user. Unfortunately, in Coursera it proved to be impossible to collect background data from the

learners which could be analysed in parallel with the clickstream and help us pinpoint patterns to

actual learners and their demographic and professional background.

One of the major challenges today in evaluating MOOCs is to find the appropriate instruments,

metrics and indicators to assess the success of a such a course. Traditional metrics and assessments

based on completion and student grades do not completely approach such massive, student-driven

educational environments, as it becomes obvious that in particular non-novice professionals are

pursuing their own specific learning objectives in attending MOOCs, which are not necessarily met

by the teacher-defined pathways to completion. Rather than focusing on course completion or

percentage of resources accessed, the authors defined engagement as the amount of lessons completed

out of all lessons accessed. This was an attempt to capture a more qualitative aspect of engagement, as

each lesson included a variety of resources, videos, discussions boards and assessment. Attending

resources, discussions and assessment even if in one focused area of the course, might define a

different level of engagement than accessing multiple lessons and modules but without utilising all

possibilities of the learning environment, especially the more interactive ones such as discussions and

assessments.

It was obvious that learners approached the course in very different ways and there was a wide

diversity in all indicators that define learner's behavior. The analysis of the clickstream by clustering

certain indicators allowed the identification of some major patterns of navigation and interaction with

This article is protected by copyright. All rights reserved.

 the content. Significant correlations were found between certain learning patterns and student's

achievement in terms of grades and completion. At the same time, as the demographic and
Accepted Article
professional data of the students were not available, it was not possible to identify the types of

learners populating each cluster and investigate their individual objectives and motivation attending

the MOOC. There was a clear division between two patterns: a sequential navigation of the content as

directed by the instructional design (Attentive) and a more irregular and selective access of specific

areas (Auditors). Students with attentive patterns achieved significantly higher grades and were the

majority of the completers. In some studies, such distinct patterns have been related to the SRL

ability of the learner. Littlejohn, et al (21) interviewed 32 MOOC participants who were classified as

higher and lower self-regulated learner (SRLer) based on an SRL survey. They found that higher SR

Learners were less likely to follow a linear progression through the MOOC, while lower SR Learners

were more likely to follow the course in a structured way. Nevertheless, in the case of Implant

Dentistry Auditors and Attentive behaviors are also likely to reflect differences in the professional

background and experience of the learners, rather than SRL ability. As was obvious from the non-

identifiable demographics available, the course was attended by a diverse group of learners ranging

from undergraduate students, fresh graduates and junior clinicians to senior and experienced

practitioners. The first, being at a novice level and less familiar with the entire content were more

likely to follow the prescribed pathway, while the latter, being experienced were more likely to

selectively attend or interact in their areas of interest. For experienced clinicians such areas could be

topics related to their focus of practice, content of certain type (e.g. clinical procedure videos) or input

from certain lecturers. The motivation for completion and certification might have also been different

among learners with different professional background, as purchasing a verified MOOC certificate

might have been of much higher value to students and novice learners than experienced and already

established professionals. This might be one of the reasons why so many of the completers actually

completed the course and acquired the certificates much earlier than the prescribed time. Looking

closer at 31 learners who completed all assessments withon one day, we could find that 26 were

attentive and only 5 auditors. The percentage of certificate acquisitions was exceptionally high for a

MOOC, regardless of the background of those who purchased the certificates. This might point a

This article is protected by copyright. All rights reserved.

 significant potential of this learning platform in accreditation and credentialing and reflect a wider

need especially within clinical disciplines. Based on the available demographics offered from
Accepted Article
Coursera in August 2018, 64% of enrolled learners were male, while 51% of all learners were in the

age bracket between 25-34. Furthermore, 35% of them were full time students, while 40 % were

employed full time. In terms of education, 20 % already had a Masters degree and 6 % a Doctorate.

Unfortunately, this data is offered as a sample based on the more than 30,000 students enrolled up to

August 2018 and cannot be in anyway correlated to the actual sample used in this study, as

identification of users was not possible.

Even if Auditors and Attentive might reflect different professional backgrounds, the sub categories

might be closer to demonstrating actual SRL, as they describe more specific patterns of approaching

the content and assessments. A study based on an SRL survey and clickstream data from 4831 MOOC

participants showed that higher SR Learners were more likely to revisit previously studied course

materials, especially after the course assessment (22).

There is at present little knowledge as to the selection of the ideal indicators in order to study

engagement and learners' behaviors through clickstream data. The 5 indicators modeled for this study

were chosen among many possible combinations, in an attempt to identify certain behaviors based on

the performance phase in the Zimmerman’s model. Although there is a certain risk of bias in the

selection of the five indicators, the fact that some significant differences were finally identified among

the achievement and performance in each cluster suggests that the modeling was relevant to the aims

of the study. In future research, these indicators could be replicated, modified and enriched, hopefully

allowing a deeper understanding of how to best analyse clickstream data. Furthermore, the

availability of anonymised demographic and professional data or learners will empower clickstream

data to offer a much deeper understanding of learners' behaviors in a MOOC and platforms that offer

MOOCs need to seriously consider how to make such data available for research purposes.

This article is protected by copyright. All rights reserved.

 In conclusion, the experience with Implant Dentistry showed that a MOOC can be a favorable

learning environment for the teaching of clinical disciplines and can attract a wide diversity of
Accepted Article
learners, achieving much higher completion and certification rates than typically reported. There

appear two fundamental approaches in the navigation and interaction with the content, one of a

sequential engagement and one of more targeted and selective access of resources. The first learning

pattern is significantly correlated with higher engagement, performance and completion rates than the

latter in the MOOC context. These patterns might be influenced by the demographic and professional

background of the learners, as well as their motivation for attending the MOOC. Certain subcategories

however, in particular how learners access content in relation to assessments might be closer related

to SRL. In the future, research based on data from clickstreams combined with an analysis of the

demographic, professional and motivational background of the learners could help us better

understand learners' behaviors and the potential of MOOCs in the teaching of clinical disciplines in

particular.

References

1. Skiba DJ. Disruption in higher education: Massively open online courses (MOOCs). Nursing

education perspectives 2012: 33(6):416.

2. Shah D. By The Numbers: MOOCs in 2018: Class Central. 2018 [Available at: https://www.class-

central.com/report/mooc-stats-2018/. Retrieved 11 December 2018

3. Kaplan AM, Haenlein M. Higher education and the digital revolution: About MOOCs, SPOCs,

social media, and the Cookie Monster. Business Horizons. 2016 : 59(4):441-450.

4. Kearney RC, Premaraj S, Smith BM, Olson GW, Williamson AE, Romanos G. Massive open

online courses in dental education: two viewpoints: viewpoint 1: massive open online courses offer

transformative technology for dental education and viewpoint 2: massive open online courses are not

ready for primetime. Journal of dental education 2016 : 80(2):121-127.

This article is protected by copyright. All rights reserved.

 5. Walsh K. “Massive Open Online Courses on Health and Medicine: Will They Be Sustainable?”.

Journal of Medical Internet Research. 2014 :16 (8): e197. doi: 10.2196/jmir.3798.
Accepted Article
6. Walker MP, Duley SI, Beach MM, Deem L, Pileggi R, Samet N, et al. Dental education

economics: challenges and innovative strategies. J Dent Educ 2008: 72(12):1440-1449.

7. Mattheos N, de Bruyn H, Hultin M, Jepsen S, Klinge B, Koole S, et al. Developing implant

dentistry education in Europe: the continuum from undergraduate to postgraduate education and

continuing professional development. Eur J Dent Educ 2014 :18 Suppl 1:3-10.

8. Mattheos N, Albrektsson T, Buser D, De Bruyn H, Donos N, Hjørting Hansen E, et al. Teaching

and assessment of implant dentistry in undergraduate and postgraduate education: a European

consensus. Eur J Dent Educ 2009: 13 (Suppl. 1): 10–17.

9. Koole S, De Bruyn H. Contemporary undergraduate implant dentistry education, a systematic

review. Eur J Dent Educ 2014: 18 (Suppl. 1): 11–23.

10. Koole S, Vandeweghe S, Mattheos N, De Bruyn H. Implant dentistry in Europe: 5 years after the

ADEE consensus report. Eur J Dent Educ 2014: 18 (Suppl. 1): 43–51.

11. Mattheos N, Stefanovic N, Apse P, Attstrom R, Buchanan J, Brown P, et al. Potential of

information technology in dental education. Eur J Dent Educ 2008 : 12 Suppl 1:85-92.

12. Henningsohn L, Renstrom Koskela L. Introduction to Urology. Karolinska Institute, edX.

Available at: https://www.edx.org/course/introduction-to-urology

13. Du E. Introduction to cataract surgery. University of Michigan. Coursera. Available at:

https://www.coursera.org/learn/cataract-surgery

14. Zimmerman, Martinez-Pons. Construct validation of a strategy model of student self-regulated

learning. Journal of Educational Psychology 1988: 80(3), 284.

This article is protected by copyright. All rights reserved.

 15. Mattheos N, Nattestad A, Falk-Nilsson E, Attström R. The interactive examination: assessing

students' self-assessment ability. Med Educ 2004: 38:378-89.

Accepted Article
16. Sanz M, Shapira L. Competencies in implant therapy for the dental graduate. Appropriate

educational methods. Eur J Dent Educ 2009: 13 (Suppl. 1): 36–43.

17. Hew, K.F., & Cheung, W.S. Students’ and instructors’ use of massive open online courses

(MOOCs): Motivations and Challenges Educational Research Review 2014:12: 45-58.

18. Maxwell WD, Fabel PH, Diaz V, Walkow JC, Kwiek NC, Kanchanaraksa S, Wamsley M, et al.

Massive open online courses in U.S. healthcare education: Practical T considerations and lessons

learned from implementation. Currents in Pharmacy Teaching and Learning 2018: 10: 736–743.

19. Robinson R. Delivering a medical school elective with massive open online course (MOOC)

technology. PeerJ. 2016 Aug 11;4:e2343.

20. Stokes CW, Towers AC, Jinks PV, Symington A. Discover Dentistry: encouraging wider

participation in dentistry using a massive open online course (MOOC). British Dental Journal. 2015

Jul;219(2):81.

21. Littlejohn A, Hood N, Milligan C, Mustain P. Learning in MOOCs: Motivations and self-

regulated learning in MOOCs. The Internet and Higher Education 2016 : 29: 40-8.

22. Kizilcec RF, Pérez-Sanagustín M, Maldonado JJ. Self-regulated learning strategies predict learner

behavior and goal attainment in Massive Open Online Courses. Computers & education. 2017:

104:18-33.

This article is protected by copyright. All rights reserved.

 Table 1. Definitions of the SRL indicators
Accepted Article
Indicator Definition
Time management The number of modules
(TM) the participant accessed
per number of days logged
in
Task strategy for The ratio of replays of
comprehension (TS) individual lectures from
the total lectures the
participant accessed
Self-instruction The ratio of replays of the
(assessment driven) video lectures after a quiz
(SI-ad) out of in the total number
of the assessments the
participant took
Self-instruction of The ratio of the number of
systematic learning lessons followed in the
(SI-sl) prescribed sequence out
of total lessons the
participant accessed
Application (AP) The ratio of the
discussions accessed
related to the clinical
problems set, out of
number of clinical
problems attended
This article is protected by copyright. All rights reserved.
Quantification
The number of modules
accessed (1-5) / the number
of the days participant
logged in (1-35)
The times of video lectures
replayed / the total number
of the video lectures
accessed
The number of replays of
video lectures immediately
after an assessment / the
number of the assessments
the participant took
The number of lessons
accessed in the prescribed
sequence / the total
number of lessons the
participants accessed
The number of discussion
threads attended related to
the clinical problem/ the
number of clinical problems
attended
Significance
The higher the indicator,
the more likely is the
student to have spread
attendance over more
moduleson fewer days
logged in
The higher the indicator,
the more likely the
participant was to revisit
and replay video lectures
The higher the indicator,
the more likely the learner
to demonstrate
assessment driven learning
behaviours
The higher the indicator,
the more likely the
participant to have
followed the prescribed
sequence of the
instructional design
The higher the indicator,
the more likely the student
to be engaged in the
clinical problems and
related discussion
 Table 2. The eight clusters identified by the the constant-comparative analysis, further organised in
two overriding patterns “Auditors” and “Attentive”.
Accepted Article
cluster TM TS SI-ad SI-sl AP Types Types in general

1 low low low high low sampler Attentive

2 high low low low low browser Auditor

3 low high high low low digger Auditor

4 high low low low low browser Auditor

5 low high high low low digger Auditor

6 low low high high low lucubrator Attentive

7 high low high low low doer Auditor

8 high low low low low browser Auditor

This article is protected by copyright. All rights reserved.

 Table 3. Distribution of learners and engagement, performance and completion rates for each of the
clusters.
Accepted Article
Completed at EN Final
Certificate Grade
Types Sub-types N least one test Completers Grade 1 2 3 4 5
holders range
(mean)

1. Browser 3125 1176 (37.63%) 7 (0.2%) 6 (0.2%) 9.7% 0.07 0 – 0.93 -

Auditor 2. Digger 195 92 (47.18%) 40 (20.5%) 40 (20.5%) 25.5% 0.25 0–1 ** -

3. Test Driven 261 238 (91.19%) 37(14.2%) 36 (13.8%) 25.4% 0.26 0–1 ** ** -

Total 3581 1506 (42.06%) 84 (2.3%) 82 (2.3%) 11.7% 0.09 0–1 - - - - -

4. Sampler 973 940 (96.61%) 791 (81.3%) 766 (78.7%) 69.2% 0.87 0.40 – 1 ** ** ** -

Attentive 5. Persistent 460 451 (98.04%) 402 (87.4%) 384 (83.5%) 69.4% 0.88 0.33 – 1 ** ** ** .077 -

Total 1433 1391 (97.07%) 1193 (83.3%) 1150 (80.3%) 69.2% 0.88 0.33 – 1 - - - - **

EG = Engagement (%)

Final Grade= group's mean (%)

** means difference of final grade between types at the significant level (p < .01)

This article is protected by copyright. All rights reserved.

Accepted Article

This article is protected by copyright. All rights reserved.