ASSESSING THE ONLINE

LEARNER
Rena Palloff and Keith Pratt
Welcome

Rena Palloff
and Keith Pratt

Be inspired. Be inspiring.
Learning Outcomes for This Session

After reviewing this pre-recorded session, you should

be prepared to:
Align course design with assessment strategies.
Describe characteristics of online assessment.
Select appropriate assessment methods to
determine student competency levels.
Develop an online assessment activity and a rubric to
evaluate student performance.

Be inspired. Be inspiring.
3
Start with Your Learning Outcomes

What a student will be able to know or do at

the end of a learning unit or course
Measurable
Observable action or behavior

Be inspired. Be inspiring.
4
Then Move to Learning Objectives

Unit or activity basedcontribute to the

achievement of outcomes.
Describe cognitive skills achieved (i.e., what the
student will LEARN.
Once you know what you want your students to be
able to know, do, or learn, you can begin to design
activities and assessments.

Be inspired. Be inspiring.
5
Cycle of Course Design

Purpose of Course
(Course Outcomes
& Competencies)

Learner
Characteristics
Student Assessments

Course Activities Learning Objectives

Be inspired. Be inspiring.
Designing Courses That Align with
Assessment
Provide clear guidelines and objectives.
Tasks and assignments should be relevant not only to the
subject matter, but to students lives as well.
Students should understand what is expected of them.
Assessment should be in alignment with the course as a
whole and is not to be seen as a separate and cumbersome
task.
When the activities and assessment techniques are in
alignment, instructors and students alike are more satisfied
with the outcome of the learning process.

Be inspired. Be inspiring.
7
Assessment Basics
The following are some principles that should guide student
assessment in an online course:
Design learner-centered assessments that include self-
reflection.
Design and include grading rubrics for the assessment of
contributions to the discussion as well as for assignments,
projects, and the collaboration itself.
Include collaborative assessments through public posting of
papers along with comments from student to student.

Be inspired. Be inspiring.
8
Assessment Basics
The following are some principles that should guide student
assessment in an online course:
Encourage students to develop skills in providing feedback by
providing guidelines to good feedback and by modeling what
is expected.
Use assessment techniques that fit the context and align with
learning objectives.
Design assessments that are clear, easy to understand, and
that are likely to work in the online environment.
Ask for and incorporate student input into how assessment
should be conducted.
Be inspired. Be inspiring.
9
Assessment Basics
Angelo and Cross (1993) note that effective classroom
assessment should be:
Learner-centered
Teacher-directed
Mutually beneficial
Formative
Context-specific
Ongoing
Firmly rooted in good practice

Be inspired. Be inspiring.
10
Differences Between Online and Classroom
Assessment
Online
Involvement in online discussion
can improve reading/writing
skills.
Engagement in discussion can
deepen knowledge acquisition
and critical thinking skills.
Technology can be used to assist
in the assessment process.
Be inspired. Be inspiring.
Classroom
Can improve reading/writing
skills if assessment involves
written assignments beyond
tests/quizzes.
Active learning techniques can
deepen knowledge acquisition.
Technology is not often used for
assessment.
Assessment in Online Courses
Six key qualities for assessment of online students
(Morgan and OReilly, 1999):
A clear rationale and consistent pedagogical
approach
Explicit values, aims, criteria, and standards
Authentic and holistic tasks

Be inspired. Be inspiring.
12
Assessment in Online Courses
Six key qualities for assessment of online students
(Morgan and OReilly, 1999):
A facilitative degree of structure
Sufficient and timely formative assessment
Awareness of the learning context and perceptions

Be inspired. Be inspiring.
13
Ways to Measure Knowledge
Acquisition
Use good questioning techniques to review text and
other reading material
Rubrics
Self-assessments
Peer assessments
Authentic assessments

Be inspired. Be inspiring.
14
Tests and Quizzes Online
Generally used as the first resource for online
instructors because:
Course management systems contain test and quiz
functions.
Instructors may have limited time or resources
available to construct other forms of assessment
(Rocco, 2007).

Be inspired. Be inspiring.
15
Why Tests and Quizzes May Not Be the
Best Choice
May not align with course objectives or mode of
instruction.
The need to write good questionsnot all
instructors do this well!
Generally measure knowledge acquisition but not
application.
May actually encourage cheating.

Be inspired. Be inspiring.
16
Alignment of Objectives with Exam
Questions
Rocco (2007) provides the following alignment of test
question types with desired course outcomes:

Objective Multiple True/False Matching Completion Short Short Long

Level Choice Answer Essay Essay

Knowledge X X X X X
Comprehension X X

Application X X
Analysis X X
Synthesis X X
Evaluation X X

Be inspired. Be inspiring.
17
Use Quizzes as a Form of Self-
Assessment
Practice tests and quizzes align well with the use of
final exams online.
Helps students understand what will be expected of
them in terms of test questions and how to use the
technology for the test.
Practice tests and quizzes improve student
performance in the course (Michigan State
University, 2005).

Be inspired. Be inspiring.
18
Questioning Techniques
Use open-ended questions that encourage students to
talk to one another about what theyve read.
Use questions that encourage further research outside of
assigned reading.
Use questions that encourage application of material
read, such as case studies, scenarios, etc.
As the instructor, do not respond to every post.
Strategically jump in to link or extend learner discussion.
Do assess learner contributions to discussions!

Be inspired. Be inspiring.
19
Rubrics
Should support self-assessment, self-
monitoring, and instructor assessment
Should be based on Blooms taxonomy, and
criteria for each level of response assessed
should be clear

Be inspired. Be inspiring.
20
Rubrics
Define the performance levels for each
gradable activity element.
Provide students with a concrete way of
evaluating their own performance as well as
the performance of the members of their
team in collaborative activity.

Be inspired. Be inspiring.
21
Types of Rubrics
Holistic RubricsMeasure overall course
performance
Analytic RubricsBreak performance or product into
its component parts
Primary Trait RubricsSimilar to analytic rubrics with
the exception that they measure a common trait
across all categories (i.e., participation in the online
course)

Be inspired. Be inspiring.
22
Basic Discussion Rubric
B T
Summary Level Posts = C l a
o x
Analytic Level Posts = B o o
m n
Synthesis Level Posts s o
(i.e., You made me think!) = A m
y

Be inspired. Be inspiring.
23
Using Rubrics in Assessment
The following is an example of how to use a rubric for online
discussion:

Attached is a rubric for participation in online discussions. The way to

use this rubric is to look at the gradable items listed down the left-
hand side of the grid. Across the top are the performance levels, which
roughly (and I do mean roughly!) equate to grades. To earn an A, for
example, you would not need to fall in all of the boxes down the right-
hand columnyour performance in discussions might be extremely
strong with a combination of items from the TWO columns on the
right (proficient and distinguished) and maybe even one item that you
feel shows basic performance. The idea is to look at the total picture of
your discussion performance.

Be inspired. Be inspiring.
24
Using Rubrics in Assessment, cont.
So, at mid-term and again at the end of the term, what I want you to do is to
send me an e-mail (I'll nudge you at mid-term) telling me where you think you
fall on the rubric. For example, you might say that for including and applying
course concepts and theories, you feel that you fall into the proficient
category while you feel you are distinguished in applying real-world examples
and applications of the theories. You'll go through each category and tell me
where you think you fall on the grid. I don't generally give mid-term grades,
so what I'll be doing is responding to your e-mail with my own assessment
based on your self-assessment. I hope that's not too confusing! If there's a
discrepancy in our perceptions, we'll discuss it. If we agree, then great!

At the end of the term, I'm going to ask you to do this again. But, this time,
I'm going to ask you to tell me what grade you think you've earned as well.
Questions about this? Please ask!

Be inspired. Be inspiring.
25
Determining Performance Levels for
Activities
What evidence can students provide that would
show they have accomplished what you want them
to?
What are the highest expectations for student
performance?
What is the worst performance you can imagine
beyond not doing the assignment at all?

Be inspired. Be inspiring.
26
Determining Gradable Activities
Activities should directly relate to course learning
objectives.
All course activities should relate to one another.
Activities should be based on previous experience
with like assignments.
Activities should relate to the skill set of students
entering the course or should help develop new
skills.

Be inspired. Be inspiring.
27
Self-Assessments
Depending on the level of the learners, criteria can
be instructor-developed, peer-developed, or
independently developed.
Work best with some guiding questions.
Can be used effectively for individual activities as
well as overall performance in the course.

Be inspired. Be inspiring.
28
Peer Assessments
After collaborative activity, can be used to debrief
Can be in the form of a private e-mail to the
instructor including both self-assessment and
assessment of all other members of the group and
team
Can include suggested grades or simply be in
narrative form

Be inspired. Be inspiring.
29
Authentic Assessments
Generally take the form of application activities
such as simulations, role playing, or use of case
studies.
Portfolios can illustrate ongoing knowledge
acquisition and application.
Effectively demonstrate not only acquisition of
knowledge but ability to apply that knowledge in
professional or other settings.
Effectively assessed through the use of rubrics.

Be inspired. Be inspiring.
30
Using the Online Environment for
Assessment Advantage

Computer generated/scored tests and quizzes

Internet-based research projects
Peer review and assessment technologies
Internet-based case studies
Synchronous technologies to facilitate collaboration

Be inspired. Be inspiring.
31
Map Your Outcomes to Assessment
Activities!
Learning
outcomes
(include: knowledge, cognitive,
key and subject specific skills)
select core learning outcomes that
appear across unitsit is not
necessary to list all learning
outcomes.
Be inspired. Be inspiring.
Learning- Type of
teaching activity assessment
You will need this for the E.g., essay, group work,
learning-teaching-assessment presentation, work-based,
cycle partjust give idea of extended project, etc.
different teaching and learning
used to accommodate a
learning outcome.
 NEXT Develop an assessment activity
and a rubric you can use to evaluate it!

Be inspired. Be inspiring.
Thank you!

LIVE SESSION: TUESDAY, JUNE 26

12:00 2:30PM EST

WileyLearningInstitute@wiley.com
www.WileyLearningInstitute.com