Sunteți pe pagina 1din 20

Direct vs Indirect Assessment

of Student Learning:
An Introduction

Dr. Sheila Handy, Chair Business Management and Co-Chair University


Assessment Committee

Sponsored by University Assessment Committee

February 24, 2014


Assessment
• “The most important purpose of assessment should be not
improvement or accountability but their common aim: Everyone
wants students to get the best possible education. Everyone wants
them to learn what’s most important. A college’s mission statement
and goals are essentially promises that the college is making to its
students, their families, employers, and society. Today’s world needs
people with the attributes we promise. We need skilled writers,
thinkers, problem-solvers and leaders. We need people who are
prepared to act ethically, to help those in need, and to participate
meaningfully in an increasingly diverse and global society. Imagine
what the world would be like if every one of our graduates achieved
the goals we promise them! We need people with those traits, and
we need them now. Assessment is simply a vital tool to help us
make sure we fulfill the crucial promises we make to our students
and society. “
Linda Suskie, 2010
Continuous Improvement of Student Learning

Use Results
Articulate
for
Goals
Improvement

Offer
Collect
Programs,
Assessment
Services and
Information
Initiatives
Setting the Stage
• Middle States Commission on Higher Education

• “…each program should employ direct


measures of student learning…”

• “…each stated learning outcome should be


aligned to a measure, a performance goal, and
a finding. Measures can be used to evaluate
more than a single learning outcome.”
Assessment Measures - Definitions
Direct Indirect
• Direct examination or • Perceived extent or
observation of value of learning
student knowledge, experiences
skills, attitudes or
behaviors to provide
evidence of learning
outcomes.
Assessment Measures - Examples
Direct Indirect
• Exam • Surveys
▫ Standardized ▫ Students
▫ Locally developed ▫ Alumni
▫ Embedded questions • Focus Groups
• Juried review • Student Records
• Portfolios with rubrics ▫ Class Attendance
Short Table Activity
• Introductions

• Envelopes
▫ Match Assessment Measure and Description
▫ Sort into Direct and Indirect Measures
Standardized Tests
• Licensing Exams
▫ Education - Praxis, PAPA
▫ Nursing - NCLEX
▫ Accounting - CPA Exam

• Major Field Test


▫ Developed by ETS

▫ Administered in a proctored environment as paper


and pencil or online
MFT subjects
▫ Biology ▫ Math
▫ Business ▫ Physics
▫ Chemistry ▫ Political Science
▫ Computer Science ▫ Psychology
▫ Criminal Justice ▫ Sociology
▫ Literature in English
Standardized Tests
Advantages Disadvantages
 Carefully developed ▫ If tests don’t match LOs of
 Highly reliable program, scores will be low
 Professionally scored ▫ MC questions that test facts
 Nationally normed – faculty may focus on
higher order skills
▫ May be expensive
▫ Students may not take
seriously
Locally Developed Exam
Advantages Disadvantages
• Can be tied-in to specific LOs • Reliability and generalizability
• Can be administered as part of not known
regular semester testing • Time required to prepare
• Can use publisher developed questions
questions • Scoring may take a long time
• Norms not available
Embedded Questions
Advantages Disadvantages
• Can be conducted as part of • Faculty may think they are
regular student testing being assessed
• Does not require collection of • Takes time to develop and
additional data grade
• Can be used for both grading
and assessment
Embedded Questions
• One way to assess learning without faculty
“angst”
▫ Add assessment questions as a quiz administered
at the beginning of the next course in the
sequence
▫ Effective if different faculty teach multiple sections
of an introductory course
Juried Review
• Suitable for assessing
▫ Performing arts projects
▫ Written assignments
▫ Oral presentations
Juried Review
Advantages Disadvantages
• Provides experience and • Achieving consensus with
practice for student performers regard to the evaluation rubric
• Can be formative and • May not be sufficient to
summative – indicating “value determine if students have
added” achieved the LOs set by the
• Can also introduce students to faculty
the standards they will be
expected to achieve
Portfolios With Rubrics
• Suitable for many majors
▫ Art
▫ English
▫ Computer Science
▫ Business
▫ Education
Portfolios With Rubrics
Advantages Disadvantages
• Provides students with • Faculty must agree on
evidence to submit to potential assessment rubric
employers • Time consuming
• Tracks student work over time • Storage of evidence
• Can assess higher levels of
learning
• Allows students to reflect on
their learning
Case Study

Your goal is to develop a plan to assess this


student learning outcome for the Engineering
Program for the upcoming academic year. This
plan should include two direct measures.
Wrap Up
• Resources

▫ Materials in Folders

▫ University Assessment Committee

▫ Assessment Consulting Team

▫ Website