Sunteți pe pagina 1din 21

ETEC 565A LMS EVALUATION RUBRIC

ETEC 565A LMS Evaluation Rubric


Anne Coustalin, Yuandi Du, Brandon Nam, Susan Roth
Mary Stephenson
University of British Columbia
ETEC 565A LMS EVALUATION RUBRIC
2

ETEC 565A LMS Evaluation Rubric

Precis
In 1974, the University of British Columbia (UBC) began its Rural Family Practice

Clerkship Program. It encouraged medical students to gain experience in traditionally

underserviced areas of BC, allowing them to participate in one-on-one training with a rural

physician (UBC, 2014). So successful was the program that in 1997 it became a curriculum

requirement, and in 2014 was fully incorporated into third-year medical school training.

This course apprentices students to a family physician, or team of preceptors, practicing

in a rural community. It is intended to give students an understanding of the total health

care of individuals and their community, including an understanding of the philosophy of

medical care that looks after patients from birth to death and from home to hospital and

back home (UBC, 2014).

Our task is to find an appropriate LMS to accommodate videos of specific clinical skills.

A student will be recorded performing a specific skillsuch as an injection, sutures, or biopsy

and that video will be viewed by two clinical faculty members and four peers. The reviewers

need to be able to provide feedback on the video and annotate their comments in some way. The

four peers will be in the same area as the video subject, as well one of the faculty members.

Other faculty will either be at UBC, or one of the cooperating universities, University of Victoria

or University of Northern BC; the learning shell needs to accommodate some distance. Thus, the

LMS needs to support high video functionality, while preserving privacy and minimizing

expense. It must support other multimedia artefacts, a clinical reasoning framework, and

educational discourse, as well as facilitate communication from different locations. We have


ETEC 565A LMS EVALUATION RUBRIC
3

divided our rubric into two main sections: Essential Elements and Preferred Elements. Each

section is further broken down into categories, including functional requirements, technical

requirements, data management, and costs. Proposed learning management systems that do not

meet the minimum essential elements score are not considered further.
ETEC 565A LMS EVALUATION RUBRIC
4

Rubric
Each criterion in the Essential section must achieve a minimum rating of 2 in order for the LMS to be considered further.

Where the rubric has multiple points under one criteria (e.g. Interaction) score 3 only if it addresses ALL points. Where the same

criteria are found under Acceptable and Recommended (e.g. Data Privacy) score 3.

FUNCTIONAL REQUIREMENTS- ESSENTIAL

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Theoretical basis underpins the design


elements. LMS provides access to
Theoretical basis underpins the design Theoretical basis underpins the design
organized content and multiple
elements. LMS provides access to basic elements. LMS provides access to
E- Pedagogical Design interactive tools to engage learners. New
organized material but includes few tools organized content and standard interactive
pedagogical tools are routinely added.
for interaction. tools to engage learners.
LMS provides a variety of modalities to
address different learning preferences.

Provides secure access to class


Provides secure access to class
Provides secure access to class participants email addresses (individually
participants email addresses.
participants email addresses. Provides or groups). Provides a variety of forms of
Provides only asynchronous
some forms of synchronous and synchronous and asynchronous
Communication Tools communication formats and has few
asynchronous communication (threaded communication (chat, instant messaging,
features to aid communication.
discussions, chat). Discussion threads load and threaded discussions).
Discussion threads load slowly and can
quickly and are easy to navigate. Discussion threads load quickly, are easy
be difficult to navigate.
to navigate, and subscribable.

Interface is functional but not intuitively Interface is obvious, self-explanatory, Interface is obvious, self-explanatory,
designed and aesthetically bland. aesthetically interesting. Content intuitive, aesthetically interesting and
Contains distracting elements, visually presented in a simple and straightforward inviting. Content presented in a simple
Design and Layout busy. Little use of design elements (text, manner. Good use of design elements and straightforward manner. Design
colour, font size) to create focal points. (text, colour, font size) to create focal elements (text, colour, font size) well
Media redundancy (visual + narration + points. Some evidence of media utilized to create focal points. No
text) evident. redundancy. redundancy of media.
ETEC 565A LMS EVALUATION RUBRIC
5

FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Students communicate freely with


instructor and peers via the
Students participate in online discussion,
Students can communicate freely with communication tools or forum. Ability to
and make comments to their peers
instructor and peers via the communication participate in online discussion and make
videos.
tools or forum. comments to peers videos.
Learning materials are available to
Learning materials cannot be associated Related textbooks or learning materials
students in LMS but cannot be connected
with LMS. are alongside of the video so that
to the video section.
students will have standards to refer to.
Faculty members communicate with
Faculty members are able to make
Interaction students freely via communication tools or Faculty members are able to make
comments to student videos.
forum, but is difficult to make interaction with comments to student videos and
learning materials. participate in their discussions.
Type of Interaction: teacher-student
interaction, student-student interaction,
Type of Interaction: teacher-student Type of Interaction: teacher-student
student-textbook interaction.
interaction and student-student interaction. interaction, student-student interaction,
student-textbook interaction.
Interaction characteristics: user-
Interaction characteristics: user-
generated interaction, designed
generated interaction. Interaction characteristics: inherent
interactivity.
interactivity, designed interactivity, user-
generated interaction.

Mobile Device Fully integrated with mobile devices; apps


No mobile version. Limited functioning on mobile devices.
Integration available for IOS, Android, and Windows.

Has native video upload/download


Upload of Video Relies on third-party video providers (e.g. Plug-ins available for video services. capabilities. Video available in multiple
YouTube). Video available in one size only. sizes to accommodate available
bandwidth.
ETEC 565A LMS EVALUATION RUBRIC
6

FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Offers only quantitative assessment Offers quantitative assessment Offers a diversity of qualitative and
opportunities. Basic database-generated opportunities as well as limited quantitative assessment opportunities.
quiz and assessment tool to evaluate qualitative assessment opportunities. Interactive assessment tools that allow
Testing and
students previous knowledge. Limited Multimedia enabled assessment tool students to reflect to previous knowledge.
Assessment Tools
range of questions types (multiple choice, (picture or attachments) to customize Separate tests to allow students to reflect
true/false, short answer and essay students reflection to previous to previous knowledge and enhance
questions). knowledge. skills. Has immediate feedback capability.

Testing and Reporting features limited to numerical Offers limited reporting features. Offers sophisticated reporting features,
Assessment - scores. Student progress is poorly Presentation of student progress is not presenting student progress in various,
Reporting Features communicated. customizable. customizable ways.

Instructor: LMS is difficult to use and Instructor LMS is moderately intuitive Instructor LMS is highly intuitive and
Ease of Use -Instructor requires considerable time investment to and requires a reasonable time requires limited time investment to
become proficient. investment to become proficient. become proficient.

Student: LMS is difficult to use and Student: LMS has an intuitive design
requires heavy time investment for Student: LMS has a functional design layout requiring little to no training in
Ease of Use - Student
proficiency. Appearance is bland and navigable with limited training. order to use. Appearance is inviting to
distracting. Appearance is appealing to user. user.

Video annotation tool is available and


Video annotation tool is not available and
Video annotation tool is available and supported by vendor. Viewers can add
cannot be integrated by third party.
Annotation Capabilities supported by third party. Viewers can video comment or text to video file.
Viewers can append/ comment at end of
add separate file to comment on video. Comment can be accessed in real time
video via webcam/camera.
on the video.

Video editing includes ability to add


Simple editing available. E.g. zoom,
Video Editing No video editing capabilities. slides, stills, and comments to initial
pan, transitions.
video.
ETEC 565A LMS EVALUATION RUBRIC
7

FUNCTIONAL REQUIREMENTS- ESSENTIAL contd

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

LMS supports a wide range of formats;


LMS supports a wide range of formats; graphics and pictures can be alt -tagged,
LMS has limited ability to provide a wide graphics and pictures can be alt -tagged, fonts can be enlarged, most text is
Accessibility for range of formats due to limited support fonts can be enlarged, most text is compatible with screen readers closed-
Diverse Learners for multiple file formats (text, audio, compatible with screen readers closed- captioning is available for video provided as
video, graphics, etc.). captioning is available for video provided learning objects. Closed-captioning can be
as learning objects. provided for annotated learner video
through voice-to text.

Available during business hours via email, Available with extended hours via email,
Available during business hours via email
telephone and live chat. Central IT support telephone and live chat. Both central IT
Technical Support with long response times. No central IT
is available but no department specific IT support and department specific IT support
support is available.
support is available. are available.

The LMS is compatible with Windows, Mac


The LMS is compatible with Windows, Mac
and Linux with IE, FF, Safari and Chrome
and Linux with IE, FF, Safari and Chrome
The LMS is compatible with Windows browsers supported for their current and
OSs and Browsers browsers supported for their current and
and Mac with IE, FF, Safari and Chrome previous versions
Compatibility previous versions.
browsers supported currently available.
Installation of plug-ins required for use with
No plug-ins required for video editing
video editing application.
application.

The LMS can create a Basic LTI link within


Basic Learning The LMS can access a video editing
The LMS can create an external link to a content page of the LMS so authenticated
Tools application within a content page of the
access a video editing application. users can access a full-fledged video
Interoperability (LTI) LMS.
editing application smoothly.
ETEC 565A LMS EVALUATION RUBRIC
8

TECHNICAL REQUIREMENTS - ESSENTIAL

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Integration with student information


Data Management Integrates well with student information Integrates seamlessly with student
system (SIS) is problematic and doesnt
- SIS Integration system (SIS), though functionality is limited. information system (SIS).
allow for full functionality.

Allows for export of data (with personally Allows for export of data ((with personally
Data Management identifiable information removed) to external identifiable information removed) to external
Data is not exportable.
- Exportability data warehouse for use with external data warehouse for use with external
analytics programs. analytics programs.

Basic Learning The LMS can create a Basic LTI link within
The LMS can access a video editing
Tools The LMS can create an external link to a content page of the LMS so
application within a content page of the
Interoperability access a video editing application. authenticated users can access a full-
LMS.
(LTI) fledged video editing application smoothly.

Does not comply fully with BC Freedom Full compliance with BC Freedom of Full compliance with BC Freedom of
Privacy Law of Information and Protection of Privacy Information and Protection of Privacy Act Information and Protection of Privacy Act
Compliance Act (FIPPA) and/or UBC Information (FIPPA) and UBC Information Security (FIPPA) and UBC Information Security
Security Standards. Standards. Standards.

Limited or no data encryption


Data Encryption Allows for encryption of data. Allows for easy encryption of data.
capabilities.

Data De- De-identification of data is possible.


De-identification of data is not possible. De-identification of data is possible.
identification Supports multiple authentications of data.
ETEC 565A LMS EVALUATION RUBRIC
9

DATA MANAGEMENT - ESSENTIAL

Not Acceptable Acceptable Score


Criteria Recommended (Rating of 3)
(Rating of 1) (Rating of 2)

Personally identifiable information (PII) and Personally identifiable information (PII) and
other sensitive information (personal health other sensitive information (personal health
Personally identifiable information
Data Security - information) is held on UBC servers behind information) is held on UBC servers behind
(PII) is housed in the cloud or on a
Storage UBC firewalls, in UBC data centres in UBC firewalls, in UBC data centres in
server outside of UBC.
compliance with UBC information security compliance with UBC information security
standards. standards.

No additional permissions required.


Data Security - Exhaustive additional permissions
Some additional permissions required.
Permissions needed. Includes tools for digital rights
management.

Automatic Backup No automatic backup system and/or Automatic backup system and disaster Automatic backup system and disaster
and Recovery disaster recovery plan recovery plan recovery plan

Compatible with database management


system. Large funding required but
Large funding required. Institution Cost is appropriate and institution is able to
institution is able to absorb within its
cannot absorb the cost. provide it within its capability.
capability. However, this means cut-offs of
some other projects.
Cost
(Score all three
parts) Students are required to pay Student LMS licenses are covered by There are no end user fees for the LMS and
additional fees to obtain user license. existing tuition. the school incurs no additional fees.

Ongoing maintenance costs covered with


Costly ongoing maintenance costs Affordable maintenance costs within
initial investment for a limited term.
beyond capability of existing budget. existing budget.

Essential Criteria Score Subtotal

LMS must score a minimum of two points on each assessment criteria to be considered further
ETEC 565A LMS EVALUATION RUBRIC
10

FUNCTIONAL REQUIREMENTS - PREFERRED

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Access to material on the LMS restricted to


currently enrolled students and links to
Copyright No copyright management tools Access to material on the LMS restricted to
external content are provided. Authors are
Management included. currently enrolled students.
given a choice of Creative Commons for
their content.

Basic calendar available with pop-up Collaborative calendar with pop-up


Calendar No calendar
announcements. announcements.

No file exchange capabilities beyond Supports exchange of smaller files (<10MB)


Supports unlimited file exchange.
simple email transactions. between instructors and students.
File Exchange
(Score both
Supports upload and download of HTML Supports upload and download of HTML
categories)
Does not support upload or download pages, images, flash objects, pdf, and MS pages, images, flash objects, pdf, and MS
of zip files. word files in a single Zip archive. word files in a single Zip archive.

Clinical faculty members and Clinical faculty members and students can
Clinical faculty members and students can
Advanced Search students can search uploaded search by keywords and metadata from
search by keywords or metadata from
Option students videos by title or author clinical faculty members uploaded students
uploaded students videos.
name. videos.as well as by comments

LMS unable to integrate multimedia LMS is able to seamlessly integrate


Other LMS is able to integrate multimedia and
and learning object libraries from multimedia and learning object libraries
Multimedia learning object libraries from limited major
major textbook publishers. from major textbook publishers.
Objects textbook publishers.
ETEC 565A LMS EVALUATION RUBRIC
11

FUNCTIONAL REQUIREMENTS PREFERRED contd

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

LMS is able to host third party interactive LMS supports interactive tools that can
Interactive
LMS does not support interactive tools. tools. Faculty can create an interactive be created easily within the LMS and
Functionality
clinical reasoning framework. seamlessly disseminated to the students.

The LMS can transfer teaching


activities to its platform. Though some The LMS can transfer teaching activities
Migration of The LMS can transfer teaching activities
limits in its functionality require faculty online. Faculty members need to make
Existing seamlessly. Minimal changes required to
members to adjust their teaching large structural changes in their materials in
Courses migrate existing courses.
activities according to LMS and use order to make the transition.
additional tools outside of LMS.

Training modules are included with LMS Training materials are included with LMS
Training Limited training materials available and
and include printed materials and online and include print, online tutorials, and
Materials not provided with LMS.
tutorials. face-to-face training sessions.

The LMS can add metadata of student


Addition of
The LMS can add a metadata of The LMS can add a set of metadata of video by time frame multiple times by
Metadata or
student's video as a whole. student video by time frame. authorized user (based on roles and
Keywords
permissions).

Single sign on is available with full


Single sign on is available with limited Single sign on is available with most attributes passed via protocols such as
attributes passed via protocols such as attributes passed via protocols such as Central Authentication Service (CAS) or
Single Sign On Central Authentication Service (CAS) Central Authentication Service (CAS) or Security Assertion Markup Language
(SSO) or Security Assertion Markup Language Security Assertion Markup Language (SAML). i.e. Student number, name,
(SAML). i.e. Student number, name (SAML). i.e. Student number, name, email, email, academic year, enrolled courses
and email. academic year and enrolled courses. and enrolled group associated with their
clerkship rotations.
ETEC 565A LMS EVALUATION RUBRIC
12

TECHNICAL REQUIREMENTS PREFERRED contd

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Institution level of learning object Department node of learning object


Course areas learning object
Learning Object repository is available to authorized user repository is available to authorized user to
repository of uploaded videos is only
Repository (LOR) to access global repository of uploaded access uploaded videos by their node
available.
videos. access rules and permissions.

The LMS has a cron task to automatically


Group Management The LMS has an advanced group
The LMS has a group builder tool that add students to a group and based on their
Tool builder tool that allows clinical faculty
allows clinical faculty members to add assigned group every 5 weeks.
** Year 3 student members to add students based on their
students based on their assigned LMS can automatically remove students
clerkships are rotated assigned group every 5 weeks.
group. from a previous group and assign them to a
every 5 weeks
new group after 5 weeks.

The LMS manages small to medium The LMS manages medium to large The LMS manages large throughput of
System Performance
throughput of users not affecting client throughput of users not affecting client users not affecting client to server
to server responses. to server responses. responses.

Students can track:


Students can track: Students can track: Completed assignments, progress and
Learning Analytics
completed assignments on a course- completed assignments as well as participation through course, grades across
&Tracking
Students by-course basis. LMS has no grades across courses. LMS has limited courses, relative success, LMS predicts
predictive features. predictive features of uncertain value. success in the course based on predictive
factors.

Instructors can track student engagement,


Instructors can track student learning outcomes, amount of
Instructors can track student
engagement, learning outcomes, student/student interaction, attention to
engagement, amount of
Learning Analytics amount of student/student interaction, feedback, time on task, participation,
student/student interaction,
& Tracking: time on task, participation, patterns of use, demographics, content
demographics, success on
Instructors demographics, content mastery/success mastery/success on assessments, course
assessments
on assessments, risk factors. quality (in relation to metrics), risk factors.
Includes ability to create specialized
queries.
ETEC 565A LMS EVALUATION RUBRIC
13

DATA MANAGEMENT- PREFERRED

Not Acceptable Acceptable Recommended Score


Criteria
(Rating of 1) (Rating of 2) (Rating of 3)

Institution can track: Institutions can track:


Learning Institutions can track: overall UBC Faculty of Medicine student
overall UBC Faculty of Medicine
Analytics & overall UBC Faculty of Medicine student retention, retention, students at risk of failure,
student retention, students at risk of
Tracking: students at risk of failure, student grades for each course quality (metrics), effects of training
failure, student grades for each
Institution course, course quality (metrics). and pro-d on overall course quality.
course.

Institutions can track:


Learning Institution can track: Institutions can track: overall UBC Faculty of Medicine student
Analytics & overall UBC Faculty of Medicine overall UBC Faculty of Medicine student retention, retention, students at risk of failure,
Tracking: student retention, students at risk of students at risk of failure, student grades for each course quality (metrics), effects of training
Institution failure, student grades for each course, course quality (metrics). and pro-d on overall course quality.
course.

Minimum standards met? (Did every criterion in the Essential section score two or higher?)
Yes No
If YES, proceed; If NO, LMS should be rejected

Preferred Subtotal
Criteria Score /54

Essential Insert Score


/84
Criteria Score

FINAL LMS
TOTAL /138
SCORE

Evaluation Analysis

Excellent- recommend: 120-138 points


Good - possibly recommended: 100-119 points
Not recommended: below 100 points
ETEC 565A LMS EVALUATION RUBRIC
14

Rationale

Learning management systems (LMS) are an indispensable technology for today's higher

education institutions and each institution has unique requirements to consider when selecting

one. Essential in the LMS is the functionality to support the teaching and learning needs of the

institutions students and faculty (Cavanaugh, 2014; Abdullateef, Elias, Mohamed, & Zaidan,

2016), as well as ensuring the institution is compliant with data privacy and security legislation.

Incorporated into the above rubric are Bates (2014) SECTIONS framework and

Chickering and Ehrmanns (1996) Seven Principles of Technology as Lever. The following

discussion outlines our rationale for the rubrics evaluation criteria.

E-pedagogy is learning and teaching using digital information and communication

technologies and requires a sound pedagogical foundation where teachers can select content and

strategies to meet specific learning objectives. An e-learning pedagogical foundation is critical to

an effective LMS and identified as essential criteria in our rubric.

In conjunction with a sound pedagogical foundation is a design and layout that facilitates

learning and teaching. Mayer (in Bates 2014) identified twelve principles of multimedia design.

The essence of these principles is that well-designed e-learning content is of little value if it

cannot be communicated effectively to the learner. Therefore, course interfaces should be

obvious, intuitive and self-explanatory so that the users time is spent thinking about the learning

and not about the interface (Gutierrez, 2014). According to Bates (2015), the affordance of

interaction is critically important when selecting a LMS, as students learn best when they are

active in their learning. For that reason, the LMS must be evaluated in terms of the

opportunities it affords for interaction between peers, instructors, and learning materials. In
ETEC 565A LMS EVALUATION RUBRIC
15

addition, good migration of materials and assessment tools must be in place for students to

further interact with learning materials and reflect on their learning experience.

An effective LMS should not require a heavy time investment in order for a user to

become proficient, thus ease of use is an essential criterion. Content is of little value if the user is

unable to access it or finds the system cumbersome to use. This includes access for users who

require special accommodations, typically due to physical or learning disabilities. Both Bates

(2015) and Brokup (2008) state that accessibility and access for diverse learners are important

factors to consider in ensuring the LMS adequately addresses different types of learners.

Communication tools that facilitate learner-learner and learner-teacher communication

are vital in a digital platform. Chickering and Ehrmann (1996) assert that communication

technologies increase access to faculty, afford resource sharing, and facilitate shared

learning. For that reason, email, computer conferencing, and instant chat, all of which serve to

increase total communication and the opportunities to share work and problem solve, were

deemed essential criteria.

Mobile devices are included under functional requirements as students increasing use of

mobile devices for educational and mobile access to an LMS is particularly valuable for students

who are relatively older and off-campus (Joo, Kim, & Kim, 2016). In addition, a survey by

Boruff and Storie (2014) found over 50% of the 3rd- and 4th-year students and residents

reported using their mobile devices to find drug information, find clinical practice guidelines,

read point-of-care information, do clinical calculations, or perform differential diagnoses

(Results, para 1), and 77% of students used them daily during their clinical clerkships. Mobile

access (always-anytime-anywhere connection) to the LMS is valuable for students who are

already accustomed to using their phones as medical tools.


ETEC 565A LMS EVALUATION RUBRIC
16

Learning analytics are becoming of increasing importance in online teaching and learning

In creating this rubric we have afforded high value to tracking capabilities and learning analytics

that provide students, faculty and the institution with the information they need to track and

analyze student progress, optimize teaching and learning, and promote student success. We

developed our analytics criteria based on the work of Johnson et al. (2016), Pappas (2014, 2016),

and Sclater (2014).

For the LMS technical requirements, technical supports, integration with SIS, OSs and

browser compatibility are essential criteria. Briefly, seamless integration of data among all

systems allows for efficient and optimal use of LMS. Other technical elements which contribute

to creating a seamless and enjoyable user experience were considered preferred, but non-

essential.

Data security was carefully considered in our rubric. Public post-secondary institutions

such as UBC fall under the BC Freedom of Information and Protection of Privacy Act (FIPPA).

Of particular relevance to our selection of an LMS is Part 3 Protection of Privacy (Assembly,

B. C. L. 1993). We determined that the Faculty of Medicine should under no circumstances

adopt a LMS that did not facilitate compliance with FIPPA regulations and that it must comply

with UBC Information Security Standards (2014). Students in the program record both actors

and real patients, so security and privacy are of paramount importance. The LMS must be able to

manage attempts from hackers to bypass authentication method and the external video annotation

editing tool.

Finally, cost is a major consideration in choosing an LMS. Initial purchase, maintenance,

and support costs all require careful consideration. Hidden costs and high maintenance fees can

significantly affect an institutions budget. Choosing wisely can prevent a costly mistake.
ETEC 565A LMS EVALUATION RUBRIC
17

Conclusion

Choosing an LMS can be a daunting task. The wide range of available LMS platforms

have different strengths and weaknesses, standard performance levels, infrastructure, and

licensing options. Choosing the wrong platform can be costly in terms of budget and time

commitment. Being clear about an LMS desired features, functionality, and cost can help guide

institutions in the selection process and prevent becoming distracted by an LMS bells and

whistles. It is crucial that a selection strategy or rubric be used to evaluate various systems being

considered. We feel we have created a comprehensive LMS evaluation rubric that is especially

suitable to UBCs Rural Family Practice Clerkship Program but could also be adapted by many

other institutions undertaking this task.


ETEC 565A LMS EVALUATION RUBRIC
18

References
Abdullateef, B. N., Elias, N. F., Mohamed, H., & Zaidan, A. A. (2016). An evaluation and

selection problems of OSS-LMS packages. SpringerPlus, 5. 248. doi:10.1186/s40064-

016-1828-y

Assembly, B. C. L. (1993). Freedom of Information and Protection of Privacy Act. Retrieved

from: http://www.bclaws.ca/Recon/document/ID/freeside/96165_00

Assembly, B. C. L. (2008). E-Health (Personal Health Information Access and Protection of

Privacy) Act. Retrieved from

http://www.bclaws.ca/Recon/document/ID/freeside/00_08038_01

Baer, L., & Campbell, J. (2012). From metrics to analytics, reporting to action: Analytics' role in

changing the learning environment. Game Changers: Education and Information

Technologies, Educause, 53-65.

Bates, A.W. (2015) Teaching in a digital age: Guidelines for designing teaching and learning.

Vancouver BC: Tony Bates Associates Ltd.

Bates, T. (2014). Choosing and using media in education: The SECTIONS model. In Teaching in

digital age. Retrieved from https://opentextbc.ca/teachinginadigitalage/part/9-

pedagogical-differences-between-media/

Brokop, F. (2008) Accessibility to e-learning for persons with disabilities: Strategies, guidelines,

andsStandards. Edmonton, AB: NorQuest College/eCampus Alberta.

Boruff, J.T., & Storie, D. (2014). Mobile devices in medicine: A survey of how medical

students, residents, and faculty use smartphones and other mobile devices to find

information. Journal of the Medical Library Association, 102(1), 22-30.

doi: 10.3163/1536-5050.102.1.006
ETEC 565A LMS EVALUATION RUBRIC
19

Cavanaugh, T. (2014). The lms selection process: Practices and considerations. Educause Center

for Analysis and Research. Available at

https://library.educause.edu/resources/2014/7/the-lms-selection-process-practices-and-

considerations

Chickering, A. W., & Ehrmann, S., C. (1996). Implementing the seven principles: Technology as

lever. American Association for Higher Education Bulletin, 49(2), 3-6. Retrieved from

http://www.aahea.org/articles/sevenprinciples.ht

CIHR. (2005). Best practices for protecting privacy in health research (GC Publication No.

MR221-63/2005E-PDF). Ottawa, ON: Public Works and Government Services Canada.

Retrieved September 27, 2016 from: http://www.cihr-

irsc.gc.ca/e/documents/et_pbp_nov05_sept2005_e.pdf

ECAR Working Group Paper. (2015). The predictive learning analytics revolution: Leveraging

learning data for student success. Retrieved September 23 from:

http://net.educause.edu/ir/library/pdf/ewg1510.pdf .

Gutierrez, K. (2014). If you confuse learners, you lose them: 4 steps to effective communication

in eLearning. Available at

http://info.shiftelearning.com/blog/if-you-confuse-learners-you-lose-them-4-steps-to-

effective-communication-in-elearning

Johnson, L., Adams Becker, S., Cummins, M., Estrada, V., Freeman, A., & Hall, C. (2016).

NMC Horizon Report: 2016 Higher Education Edition. Austin, Texas: The New Media

Consortium. Retrieved from

https://library.educause.edu/~/media/files/library/2016/2/hr2016.pdf
ETEC 565A LMS EVALUATION RUBRIC
20

Joo, Y.J., Kim, N. & Kim, N.H. (2016). Factors predicting online university students use of a

mobile learning management system (m-LMS). Education Technology Research

Development, 64, 611-630.

Pappas, C. (2014). How learning analytics can improve elearning design and development.

eLearning Industry. Retrieved from https://elearningindustry.com/5-reasons-why-

learning-analytics-are-important-for-elearning

Pappas, C. (2016). 8 types of elearning analytics to obtain from google and your lms. eLearning

Industry. Retrieved from https://elearningindustry.com/types-elearning-analytics-obtain-

google-lms

Sclater, N. (2014). Effective learning analytics: Using data and analytics to support students.

Joint Information Systems Committee. Retrieved from:

https://analytics.jiscinvolve.org/wp/2014/10/08/analytics-systems-centred-around-the-

vlelms-2/

University of British Columbia. (n.d.). Collaborative learning annotation system. Retrieved from

http://ets.educ.ubc.ca/clas/

University of British Columbia. (2014). Year 3 rural family practice clerkship: Preceptors

guide. Retrieved from http://undergrad.familymed.ubc.ca/files/2014/08/2014-

Preceptor_Guide.pdf

University of British Columbia. (2014). Information security standards document. Retrieved

from: https://cio.ubc.ca/sites/cio.ubc.ca/files/documents/standards/IS%20Standards%201-

21.pdf

Warwick Learning and Development Centre. (2007). Pedagogies for e-learning. Available

at http://www2.warwick.ac.uk/services/ldc/resource/eguides/pedagogies/
ETEC 565A LMS EVALUATION RUBRIC
21