Evaluating Educational Technology

Evaluating Educational Technology

Using Bates ACTIONS Model

Shaila Karim
2814351
University of Athabasca

MDDE 620: Advanced Technology for Distance Education and Training

Assignment 2: Option T (60 Points)
Dr. Avgoustos Tsinakos
July 26th, 2013

Evaluating Educational Technology

Table of Contents
Introduction ..................................................................................................................................... 3
The DA Continuing Education Program ......................................................................................... 3
Overview ..................................................................................................................................... 3
The Current DA CE Program...................................................................................................... 4
Needs Assessment Findings ........................................................................................................ 4
Envisioning the Future DA CE Program .................................................................................... 5
Developing Educational Criteria ................................................................................................. 6
Applying the ACTIONS Model to the DA CE Program ................................................................ 8
Overview ..................................................................................................................................... 8
Access ......................................................................................................................................... 8
Costs............................................................................................................................................ 9
Teaching and Learning .............................................................................................................. 12
Interactivity and User-Friendliness ........................................................................................... 13
Organizational Issues ................................................................................................................ 14
Novelty ...................................................................................................................................... 14
Speed ......................................................................................................................................... 15
Discussion & Conclusion .............................................................................................................. 15
References ..................................................................................................................................... 17

Evaluating Educational Technology

Introduction

We are living in a knowledge based society dominated by change societal, professional

and technological change. How do we keep up with all of this change? The answer is to
continually learn and become a life-long learner. Todays busy professionals often juggle worklife-study and therefore need flexible, accessible learning solutions. Educational technology can
help meet these demands by creating opportunities for high-quality learning, at a time and place
convenient to the learner. However, technology cannot simply be adopted in isolation, the use of
technology needs to be embedded within a wider strategy for teaching and learning (Bates,
2000, p.2). In other words, even though technology plays a crucial role in delivering education,
the technology plan must be guided by the organizations broader learning plan, which in turn is
driven by the organizations vision. In short, a plan will be only as good as the vision that drives
it (p.58).
This paper applies Bates (2000) ACTIONS model as a strategic framework to guide
technology selection for the Dental Association (DA) Continuing Education (CE) program. The
first part of this paper reviews the DA CE program and outlines educational criteria driven by the
CE program vision. The second part applies the ACTIONS model to evaluate the effectiveness of
various web-based technologies being considered for the DA CE program.
The DA Continuing Education Program
Overview
Before evaluating the technology requirements of the DA CE program, it is important to
assess the learning requirements of the program based on the needs of DA member dentists. This

Evaluating Educational Technology

section of the paper reviews the current DA CE program, envisions the future program and
outlines educational criteria.
The Current DA CE Program
According to the Canadian Dental Association (CDA, 2013), the scope of the dental
profession is now broader than ever before and the need to deliver the highest level of
professional care is also increasing.
The DA CE program aims to help dentists meet their CE requirements by delivering
clinical, practice management and personal development learning opportunities. A need has been
identified to expand the DA CE program beyond face-to-face offerings to include accessible
learning solutions.
Needs Assessment Findings
The DA CE program conducts an annual CE survey for all member dentists and uses the
results to inform the program.
Clinical versus practice management courses. The 2012 survey results indicate that
members are more interested in clinical learning opportunities with 77% of members expressing
a high level of interest, compared to practice management learning opportunities with 54%
expressing a high level of interest (Abate, 2012). Members are particularly interested in
information specific to their career stage (new, mid-career, transitioning dentist) and
demographic. In addition, members are seeking regulatory and legislative information to help
them navigate a complex government regulatory environment.
Resources. Overall, members are looking for resources that are:

relevant to the dental practice business model

Evaluating Educational Technology

easily accessible and convenient

up-to-date and current

unbiased and objective

of superior quality

specific to career stage

affordable

practical easy to adopt in their practice

Envisioning the Future DA CE Program

According to Bates (2000), in order to create a technology plan, an organization must first
have in place a vision for learning. In order to create a broader vision for learning, an
organization must go through a visioning process which Bates describes as a technique that
allows those working in an organization to understand the full range of possibilities for teaching
and learning that technology can facilitate and the possible outcomes, acceptable or otherwise,
that might result from its implementation (p.45).
The vision of the DA CE program is to be dentists lifelong learning partner. The DA
strives to be the number one CE provider of choice for dentists by delivering high-quality, unbiased, learner-centric education that is relevant to the practice of dentistry delivered both faceto-face and online to increase accessibility. The overall goal of the DA CE program is to deliver
quality education that helps dentists:

deliver the highest level of professional care

meet their CE needs

successfully run their practice while maintaining a work-life balance

Evaluating Educational Technology

Developing Educational Criteria

Bates (2005, p.48), highlights without a strategic approach, however, short-term tactical
decisions can lead to duplication and waste, as different parts of the organization start to build a
patchwork of different software and education applications. It is easy to see how without strong
educational criteria for technology selection decisions can be led by commercial or political
reasons resulting in unsustainable training programs.
According to Bates (2005), developing educational criteria involves two very different
levels of decision making strategic and tactical. Based on DA members needs for high-quality
CE delivered in an accessible format, the DA made the strategic decision to use the Internet as
the main technology of choice to enhance accessibility. In addition to being accessible, the
Internet also provides the opportunity for learners to interact both synchronously and
asynchronously. Now let us consider the tactical decisions regarding the mix of technologies to
be used to support the strategic decision.
Table 1
DA CE Program Educational Criteria
Educational Criteria

CE Program Goal

Technology

Accessible

Improve access to quality

CE enabling dentists to
learn at their convenience.

Deliver annual live webinar series (clinical

topic).

Offer self-paced online learning modules:

o E-learning modules (compliance and
regulatory)

Evaluating Educational Technology

o On-demand webinars
o Podcasts
Balanced

Offer a balanced CE

Offer a blend of face-to-face and online

Program to meet our

learning solutions to meet clinical, practice

members needs.

management and personal development CE

needs.

Learner-focused

Offer a superior learning

experience.

Pilot the DA Learning Management System

(LMS) to centralize and support all CE
learning activities (registration, access to
learning opportunities, assessments and
certificates).

Develop targeted learning solutions such as

multi-media e-learning modules, based on
adult learning principles.

Promote learner-learner interaction by

utilizing asynchronous discussion boards on
the LMS and enable chat feature for live
webinars.

Collaborative

Collaborate with strategic

partners.

Develop online learning solutions in

collaboration with strategic partners (dental
faculties, associations, funding or educational
partners) to cost share and leverage the

Evaluating Educational Technology

benefits of technology based learning.

Resourceful

Provide relevant practice

Develop targeted online practice management

management resources to

resources such as podcasts and online practice

help manage the business

toolkits (mini-briefs) to assist DA members

aspects of running a dental

manage the business aspects of running a

practice.

dental practice, based on career stage and

demographic.

Applying the ACTIONS Model to the DA CE Program

Overview
This part of the paper applies Bates (2000) ACTIONS model as a strategic framework to
guide technology selections for the DA CE program. The ACTIONS model outlines factors that
need to be considered when evaluating the effectiveness of educational technology: A
Accessibility and flexibility, C Costs, T Teaching and learning, I Interactivity and userfriendliness, O Organizational issues, N Novelty, S Speed.
Access
DA member research indicates that dentists find it challenging to take time away from their
practice to attend face-to-face seminars, and are in need of accessible CE (Abate, 2012). Since
the DA CE program utilizes the Internet as the main technology to deliver various online
offerings such as e-learning courses, podcasts, live and on-demand webinars and on-line
resources, evaluation of the accessibility factor requires an understanding of dentists access to
the Internet in their dental practice.

Evaluating Educational Technology

According to an DA member survey (Abate, 2010), 77% of all practice owners have at least
1 computer in their dental practice. Further, 80% of dentists who do have a computer in their
practice have access to the Internet. Not surprisingly, a significantly higher proportion of dentists
who do not have Internet in their practice have been in practice over 20 years. The results show
that nearly all (94%) of dental practices that have access to the Internet have access to high-speed
Internet. As a group, dentists have high level of access to the Internet.
Costs
As a non-profit association, fees for DA CE programs are calculated on a cost-recovery
basis. Even though it would be unwise to make a technology decision based on the cost alone; it
cannot be denied that cost is a crucial factor. In order to recover costs, a course fee would need to
be applied to CE programs that enable members to earn CE credits such live webinars, ondemand webinars and e-Learning courses; resources on the other hand would be offered for free.
The table below summarizes the cost of the various technology components. It should be noted
that the costs below are projections.
Table 2
DA CE program initial year 1 cost projections
Technical

Description

Cost

Unit cost/learner

DA CE LMS and

The LMS will

$12,500

Cost per learner =

admin costs

centralize learning and

$31.25

provide access to CE

(Assuming 400

component of the
CE program

Evaluating Educational Technology 10

courses and resources

learners access

such as e-Learning

the LMS in the

modules, webinars,

first year)

and podcasts. Dentists

will be able to register,
access courses,
complete quizzes and
print CE certificates,
all at their
convenience.
Live webinar

The DA will deliver a

Webinar

Cost per learner =

live clinical webinar

development cost =

$59.50

recorded and

$9,900

(Assuming 200

Speaker cost =

learners access

$2,000

the live webinar)

broadcasted from a

studio environment.
The recorded webinar
will be archived and
saved on the LMS for
those who are unable
to attend the live
webinar.

Total cost = $
11,900

Evaluating Educational Technology 11

On-demand webinar

The DA will pilot a

(recorded)

On-demand

Cost per learner =

recorded practice

webinar

$170

management webinar,

development cost

(Assuming 120

which will be archived

$18,900

learners access

Speaker cost =

the on-demand

$1,500

event in the first

Total cost = 20,400

year)

on the LMS.

e-Learning course

The DA will pilot a

$21,900

Cost per learner =

regulatory/legislative

$219

e-Learning course split

(Assuming 100

into three 20 minute

learners access

modules.

the on-demand in
the first year)

Podcast series

The DA will pilot a

$6, 200

Cost per learner =

podcast series for new

$41.30

dentists as a means to

(Assuming 150

deliver an accessible

learners access

resource.

the free podcast

per year)

Online resource

The DA will develop

$5,900

Cost per learner =

various online

$49.17

resources - briefs

(Assuming 120

Evaluating Educational Technology 12

specific to career stage

learners access

or demographic. A

the free online

collection of mini-

resource in the

briefs will constitute a

first year)

toolkit for example,

the New Dentist
toolkit.

Teaching and Learning

According to Bates (2000), web-based learning not only increases accessibility but has the
potential to improve the quality of learning by designing activities that support higher levels of
learning such as analysis, synthesis, problem-solving and decision making skills.
Given that the choice of technology is driven by the beliefs of how adults learn, the
assumptions about the learning process need to be made prior to selecting technology (Bates,
2005). The DA CE program is based on the assumption that learners construct knowledge
actively through dialogue and discussion and therefore aligns closely to the principles of the
constructivist learning theory (Bates, 2005). In addition, the program is based on the foundation
of adult learning and instructional design principles.
Well-designed learning identifies learning objectives and provides clear and relevant
examples (Jenkins, 1995). Jenkins explains that distance learners are often time-constrained,
therefore learning materials should be structured into manageable chunks (such as a series of
three 20 minute mini-e-learning modules instead of one hour long module). This allows for

Evaluating Educational Technology 13

flexibility and enables learners to study at their convenience. In addition, case-studies and
problem-based-learning should be integrated into learning in order to promote higher levels of
learning in accordance with Blooms Taxonomy - analysis, synthesis, evaluation (Orey, 2008).
Continuing Medical Education (CME) research has demonstrated that utilizing highly
interactive, case-based modules is positively associated with an increase in clinician knowledge
and competency and is also associated with a projected increase in performance, ultimately
resulting in not only a better learning experience but a better patient outcome (Drexel, Merlo,
Basile, Watkins, Whitfield, Katz, Pine and Sullivan, 2010). This research finding can be applied
to Continuing Dental Education (CDE) by developing interactive online case-based modules.
The DA CE program will utilize the multi-media capabilities of the web by offering welldesigned e-learning modules, podcasts and online resources chunked out into manageable units
to facilitate the learning process. Web-based learning can include text, graphics, animation or
video or audio. The use of audio or video can add to the learning experience but is currently
underexploited to the cost of production and often limitations in Internet bandwidth (Bates,
2005).
Interactivity and User-Friendliness
The DAs strategic decision to use the Internet as the main technology to expand the CE
program beyond face-to-face seminars is partly based on the interactive features of the Internet.
According to Bates (2005), there are three types of interaction learner-learner interaction,
learner-teacher interaction and learner-content interaction. Live webinars enable synchronous
communication allowing learner-learner and learner-teacher interaction via live chat messaging.
LMS discussion boards also provide the opportunity for learner-learner and learner-teacher
interaction but through asynchronous chats and discussions.

Evaluating Educational Technology 14

Learning can be designed to promote interaction with the content by building activities into
the learning (for example e-learning), by identifying learning objectives and measurable
outcomes, structuring the content into discrete chunks, providing explanations and illustrations,
including self-assessment questions and activities and also by providing summaries (Bates,
2005). Feedback to assessment questions should provide learner guidance and additional
information as opposed to just Thats right! or Sorry, that is incorrect response.
CE resources should be built for ease of use and portability. For example, podcasts are
highly portable and learners can listen at their convenience - while driving or commuting on the
train. Online resources should be available in printable PDF format.
Organizational Issues

According to Bates (2005), one of the key challenges faced by organizations shifting from
face-to-face to web-based learning is to find and adopt an appropriate organizational model while
controlling the cost of the project. Bates, describes several models of course development
including the lone ranger model where one individual works individually to create the course and
develop the material. Many issues can arise from this model such as quality of content and heavy
workload.
The project management model is a suitable approach for the DA CE program, since the
scale of the program demands a team of individuals each contributing different skills project
manager, subject matter expert, instructional designer, developer. This approach has defined
resources and clear project deadlines which would enable projects to stay on budget.
Novelty

Evaluating Educational Technology 15

As many project managers know all too well, gaining approval for funding for new novel
technologies can be easier that for older sometimes more reliable technologies (Bates, 2005).
Bates cautions that there is much risk in being too early in a new technology since the product
may still be unreliable and have glitches, in short better to be at the leading edge, just behind the
first wave of innovation, rather than at the bleeding edge (p.65).
Many corporations have moved into distance learning by utilizing LMS to promote a
culture of continuous learning and development (Bates, 2005). Learning Management Systems
(LMS) have been used for years by many organizations (corporate, government, universities, and
associations) as the backbone of distance education and are by no means a novel technology
todays LSM are highly reliable.
Speed
The speed factor considers how fast content can be added or updated (Bates, 2000, p.202).
Dental continuing education is ever changing with new medical procedures, practice guidelines
and government regulations. The web-based technologies being considered for the DA program
can be updated with relative ease. For example, using a project management model, instructional
designers would be responsible to ensure that the content is kept up-to-date for existing eLearning courses, while developers would make the actual updates.
Discussion & Conclusion
This paper has applied Bates ACTIONS model to evaluate technology for the DA CE
program. The various technology components of the CE program have been evaluated on a scale
of H high, M medium, L low. Note: High is allocated to the best option; Low reflects a
lack of advantage.

Evaluating Educational Technology 16

Table 3
Applying the ACTIONS model to evaluate web-based technologies
Technology
components of the DA

ACTIONS
A

LMS

Live webinar

On-demand webinar

e-Learning course

Podcast

Online resources

CE program

The ACTIONS model is an effective tool to compare various technologies. Based on

evaluation above, delivering a live webinar, creating a podcast, creating online resources are
ranked relatively high, while adopting an LMS, and creating an e-learning module are medium.
Finally on-demand webinar is scored low (undesirable) due to high costs and lack of interactivity.
Organizations should be cautioned not to rush into decisions regarding educational
technology selections and carefully first consider their learning goals and then assess the
technology goals as part of the overall goal. I would like to conclude with a quote by Bates who
stated that if organizations are successfully to adopt the use of technologies for teaching and
learning, much more than minor adjustments in current practice will be required. Indeed, the
effective use of technology requires a revolution in thinking about teaching and learning (2000,
p.xiii).

Evaluating Educational Technology 17

References

Abate, R. (2010). Dental office access to Internet survey Dental Association. Survey report.
Abate, R. (2012). Membership survey Dental Association. Nano Research. Final report. Project
2012-265.
Bates, A.W. (2000). Managing technological change: Strategies for college and university
leaders. San Francisco: Jossey-Bass.
Bates, A.W. (2005). Technology, e-learning and distance education. London New York:
Routledge.
Canadian Dental Association. (2013). Pursuing a Career in Dentistry. Retrieved July 25, 2013,
from http://www.cda-adc.ca/en/dental_profession/becoming/pursuing/
Drexel, C., Merlo, K., Basile, J. N., Watkins, B., Whitfield, B., Katz, J. M., Pine, B. and
Sullivan, T. (2011), Highly Interactive Multi-Session Programs Impact Physician Behavior
on Hypertension Management: Outcomes of a New CME Model. The Journal of Clinical
Hypertension, 13: 97105. doi: 10.1111/j.1751-7176.2010.00399.x. Retrieved July 22,
2013, from http://onlinelibrary.wiley.com/doi/10.1111/j.17517176.2010.00399.x/full?globalMessage=0
Jenkins, J. (1995). Producing Gender Sensitive Learning materials A Handbook for Educators.
The Commonwealth of Learning. Vancouver, BC.
Orey, M. (2008) Emerging perspectives on learning, teaching and technology. Department of
Educational Psychology and Instructional Technology, University of Georgia
Retrieved July 22, 2013, from http://epltt.coe.uga.edu/index.php?title=Main_Page&oldid=3225