Sunteți pe pagina 1din 32

University of the Philippines

Open University

Assessment in
Open and Distance
July 10-August 12, 2017
Topic 1: Principles
of Assessing
Student Learning
University of the Philippines
Open University

Assessment in Open and Distance e-Learning

July 10-August 12, 2017

What are principles of assessment?
● The principles of assessment is a set of
statements that serves as the foundation of the
assessment practice of an educational
organisation and its members.
It is
● a declaration of belief
● an organisation's current and future actions,

● tools for policy formation, operations guide,

and plans fro changes

Principles provide a:
(REAP, n.d.)
1. “way of operationalizing an important idea in strategy or policy;”
2. “language for sharing and talking about good practice and about the
meaning behind proposed changes to curricula;”
3. “framework to make connections across examples of good and
innovative practice in assessment and feedback that already exist in
the institution;”
4. “reference point for evaluating change in the quality of educational
5. “tool for evaluating the benefits afforded by technology applications in
6. “summary and simplification of the research evidence [from the
research literature on assessment].”
The characteristics of principles are:
(REAP, n.d.)
1. “Principles capture an important idea while at the same time they point to implementation.
2. “Principles are action-oriented: they specify an action but do not over-specify it.”
3. “Principles can be written from different perspectives.”
4. “Reading the principles should have an immediate impact on the reader - the essential meaning
should be obvious.”
5. “Principles ... are different from procedures.”
6. “Principles should not state the obvious.”
7. Principles should be formulated “as independent and as non-overlapping as possible.”
8. There should be a balance between the general and specific application of principles.
9. “The more principles that can be identified in a learning design, normally the better the design is.”
10. A principle can be “strengthened by thinking of how another principle might be brought into play.”
11. “The number of principles should be limited – seven is a good number, ten or more is too many.”
In other word principles should be easy to remember so that stakeholders will be encouraged to
use them in discussing assessment of learning.
12. “The principles are essentially headline ideas/statements pointing to what needs to be done,”
David Nicol (2007) presented a shorter set of
characteristics that makes a good principle:
1. “A good principle should capture a core idea from the published research –
that is there should be research evidence to support its implementation.
2. A good principle should have broad relevance: it should guide practitioners
as they design learning or assessment tasks for students, but it should not
be too narrow or specific. In other words, there should be flexibility, that is,
there should be many ways of implementing a principle depending on the
discipline and the teaching and learning context.
3. Where there is a set of principles there should be minimal overlap across
them – as far as possible they should be defined independently.
4. The effectiveness of the principles should be higher when more principles
are operationalised in the same learning design.
5. Good principles should ... help those wishing to evaluate their assessment
designs or their implementations in practice.”
Nicol (2015) also presented the following key
criteria in constructing principles:
1. “Embody compelling vision – principles a way of operationalising that vision.
2. Informed by best available research – each captures a poweful
idea/aspiration from that research.
3. Succinct (6-13 words) and written in plain English – accessible, immediate
impact, face validity.
4. Action-oriented (verb-phrase): call to action.
5. Tight-loose in formulation: applicable to any disciplinary context, users
6. Complemented by other texts that enrich meaning – disciplinary examples,
research arguments, solutions to problems.
7. Relevant to wider educational agendas/concerns.
8. Number of principles important.”
And what educational principles are not (Nicol,
1. “Procedures (e.g. ensure faster turnaround time for
2. Vague statements with no obvious action (e.g. use every
opportunity to develop students as learners).
3. Things that are worthwhile but are taken for granted in
assessment (e.g. ensure assessment is valid or reliable or
4. Contentious issues (e.g. encourage competition in learning).
5. Ideas with no clear educational basis (e.g. maximise use of
available classroom/learning space).
6. Efficiency directives (e.g. reduce time spent marking).“
applied to
assessment of
learning in
Education (DE)

Examples of Principles of Assessment

Course design

view of DE Teaching and learning

Student support

Organisation and management

What can
you adopt/
Palloff & Pratt's Seven Principles of Effective
Online Assessments (2003; 2009)
Principles that should guide student assessment in an online course

1. Design learner-centered assessments that include self-reflection.

2. Design and include grading rubrics for the assessment of contributions to
the discussion as well as for assignments, projects, and collaboration itself.
3. Include collaborative assessments through public posting of papers,
along with comments from student to student.
4. Encourage students to develop skills in providing feedback by providing
guidelines to good feedback and by modeling what is expected.
5. Use assessment techniques that fit the context and align with learning
6. Design assessments that are clear, easy to understand, and likely to
work in the online environment.
7. Ask for and incorporate student input into how assessment should be
Nicol's Principles of Assessment and Feedback
Seven principles of good feedback Ten Principles of Good Assessment and Eleven Principles of good assessment and
(self-regulation & formative assessment) Feedback Practice feedback design
(Nicol & Macfarlane-Dick, 2006) (Nicol, 2007) (Nicol & Draper, 2009)
1. Good feedback practice helps clarify what good 1. Good assessment and feedback practices 1. Good feedback practice should help clarify what
performance is (goals, criteria, expected should help clarify what good performance is good performance is (goals, criteria, standards).
standards). (goals, criteria, standards).
2. Good feedback practice facilitates the 2. Good assessment and feedback practices 2. Good feedback practice should facilitate the
development of self-assessment (reflection) in should encourage ‘time and effort’ on challenging development of self-assessment and reflection in
learning. learning tasks. learning.
3. Good feedback practice delivers high quality 3. Good assessment and feedback practices 3. Good feedback practice should deliver high
information to students about their learning. should deliver high quality feedback information quality information to students about their learning
that helps learners self-correct. that helps them self-correct.
4. Good feedback practice encourages teacher and 4. Good assessment and feedback practices 4. Good feedback practice should encourage
peer dialogue around learning. should encourage positive motivational beliefs and teacher-student and peer dialogue around learning.
5. Good feedback practice encourages positive 5. Good assessment and feedback practices 5. Good feedback practice should encourage
motivational beliefs and self-esteem. should encourage interaction and dialogue around positive motivational beliefs and self-esteem.
learning (peer and teacher-student).
6. Good feedback practice provides opportunities to 6. Good assessment and feedback practices 6. Good feedback practice should provide
close the gap between current and desired should facilitate the development of self- opportunities to act on feedback.
performance. assessment and reflection in learning.
7. Good feedback practice provides information to 7. Good assessment and feedback practices 7. Good feedback practice should provide
teachers that can be used to help shape teaching. should give learners choice in assessment – information to teachers that can be used to help
content and processes shape their teaching (to student needs).
(See UofG 5-1, UoM 16-9, & NUS 10-1)
8. Good assessment and feedback practices 8. Effective assessment tasks should capture
should involve students in decision-making about sufficient study time in and out of class.
assessment policy and practice.
9. Good assessment and feedback practices 9. Effective assessment tasks should distribute
should support the development of learning student effort evenly across topics and weeks.
10. Good assessment and feedback practices 10. Effective assessment tasks should engage
should help teachers adapt teaching to student students in productive learning activity.
11. Effective assessment tasks should
communicate clear and high expectations to
Gibbs & Simpson's Ten Conditions under
which Assessment Supports Students'
Learning (2004)
Conditions notes
1. Sufficient assessed tasks are provided for students to capture sufficient Influences of assessment on the volume, focus
study time. and quality of studying
2. These tasks are engaged with by students, orienting them to allocate Influences of assessment on the volume, focus
appropriate amounts of time and effort to the most important aspects of the and quality of studying
3. Tackling the assessed task engages students in productive learning activity Influences of assessment on the volume, focus
of an appropriate kind. and quality of studying
4. Sufficient feedback is provided, both often enough and in enough detail. The influence of feedback on learning
5. The feedback focuses on students’ performance, on their learning and on The influence of feedback on learning
actions under the students’ control, rather than on the students themselves
and on their characteristics.
6. The feedback is timely in that it is received by students while it still matters The influence of feedback on learning
to them and in time for them to pay attention to further learning or receive
further assistance.
7. Feedback is appropriate to the purpose of the assignment and to its criteria The influence of feedback on learning
for success.
8. Feedback is appropriate, in relation to students’ understanding of what they The influence of feedback on learning
are supposed to be doing.
9. Feedback is received and attended to. The influence of feedback on learning
10. Feedback is acted upon by the student. The influence of feedback on learning
University of Glasgow's (UofG) Five Principles of
Assessment and Feedback for Learning (n.d._1)

1. Assessment and feedback for learning should facilitate flexibility and choice.
2. Assessment and feedback for learning should support development as a
3. Assessment and feedback for learning should foster interaction and dialogue
between students and staff.
4. Assessment and feedback for learning should develop skills and attributes in
addition to subject knowledge.
5. Assessment and feedback for learning should promote a positive impact on
student confidence and motivation.
UK National Union of Students' (NUS) Ten Principles of
Effective Feedback and Assessment (2015)

Principles Notes
1. There should be a range of assessment mechanisms that are linked to learning outcomes Diverse forms of assessment at a
and test competencies that graduates will need. Students should be involved in designing or variety of appropriate times
choosing these assessment mechanisms.
2. Assessment criteria should be clear, linked to learning outcomes and easily accessible to Assessment criteria
students. Students should be supported to understand them and to understand what
constitutes academic misconduct.
3. Submission processes should be simple for the student and electronic where possible. Submission processes
Processes should be appropriate to the assessment and accessible to all students.
4. Students should have their workload fairly distributed throughout the year, rather than Workload distribution
clustering deadlines together.
5. Approaches to anonymity should be decided in partnership between staff and students, with Anonymity and externality
the assumption that, unless decided otherwise, all summative assessments should be
anonymous (as far as is possible). Appropriate external input is sought during assessment to
ensure fairness and comparability.
6. Marks should be consistent across programmes, and the full range of marks should be used Marking consistency and distribution
across the institution.
7. Feedback should be given in time for students to act on it in their next piece of work. Feedback timeliness
8. Feedback should be constructive, helpful and detailed, to enable a student to understand Feedback quality
why they received the mark they got and what to do to improve for next time.
9. There should be opportunities for feedback on work that doesn’t contribute to the overall Formative assessment and
degree mark, in order to facilitate learning. feedback
10. Opportunities for peer learning and self-reflective exercises should be embedded in the Self-reflection and peer learning
The University of Melbourne's (UoM)
16 Indicators of Effective Assessment in Higher Education
(James, McInnis, & Devlin, 2002)
1. Assessment is treated by staff and students as an integral and prominent component of the entire teaching and learning process
rather than a final adjunct to it.
2. The multiple roles of assessment are recognised. The powerful motivating effect of assessment requirements on students is
understood and assessment tasks are designed to foster valued study habits.
3. There is a faculty/departmental policy that guides individuals' assessment practices. Subject assessment is integrated into an
overall plan for course assessment.
4. There is a clear alignment between expected learning outcomes, what is taught and learnt, and the knowledge and skills
assessed — there is a closed and coherent "curriculum loop".
5. Assessment tasks assess the capacity to analyse and synthesise new information and concepts rather than simply recall
information previously presented.
6. A variety of assessment methods is employed so that the limitations of particular methods are minimised.
7. Assessment tasks are designed to assess relevant generic skills as well as subject-specific knowledge and skills.
8. There is a steady progression in the complexity and demands of assessment requirements in the later years of courses.
9. There is provision for student choice in assessment tasks and weighting at certain times.
10. Student and staff workloads are considered in the scheduling and design of assessment tasks.
11. Excessive assessment is avoided. Assessment tasks are designed to sample student learning.
12. Assessment tasks are weighted to balance the developmental ("formative") and judgmental ("summative") roles of assessment.
Early low-stakes, low-weight assessment is used to provide students with feedback.
13. Grades are calculated and reported on the basis of clearly articulated learning outcomes and criteria for levels of achievement.
14. Students receive explanatory and diagnostic feedback as well as grades.
15. Assessment tasks are checked to ensure there are no inherent biases that may disadvantage particular student groups.
16. Plagiarism is minimised through careful task design, explicit education and appropriate monitoring of academic honesty.
University of Strathclyde's (UoS)
Four Assessment and Feedback Principles (2014)
Principles Notes
Assessment and 1.1 Assessment and feedback activities are designed to foster student engagement, to support students’
feedback practices attainment of knowledge, understanding, and transferable skills.
promote effective 1.2 A range of assessment methods are used, increasing in complexity across a programme,
student learning. and taking into consideration student and staff workloads.
1.3 Assessment and feedback practices align with intended learning outcomes and assessment criteria,
and provide opportunities for students to be active participants in the process, including opportunities for
students to discuss feedback.
1.4 Timely, constructive, and supportive feedback is required to help students understand the extent to
which they have fulfilled the assessment criteria, and supports future development of students’ work.
Assessment and 2.1 Assessment tasks are appropriate to disciplinary and/or professional contexts.
feedback practices are 2.2 Assessment applies rigorous academic standards related to and across the discipline(s) or
appropriate, fair, and professional context and is based on clearly defined assessment criteria.
transparent. 2.3 Assessment grading and feedback is based solely on students’ achievement against criteria and
2.4 Assessment and feedback practices are fair, inclusive and accessible to all students.
Assessment and 3.1 Students are made aware at the beginning of a class of the purpose, weighting, and timing of
feedback practices are assessment, and of the importance of academic honesty in assessment.
clearly communicated to 3.2 Students are made aware at the beginning of a class of the nature and timing of feedback.
students and staff. 3.3 All students and staff are aware of the criteria and standards used to assess and provide feedback on
students’ work.
3.4 There are opportunities for students and staff to engage in a dialogue around assessment and
feedback, including opportunities for students to clarify feedback.
3.5 In each programme the processes for marking, moderation, and feedback are appropriate and fair and
are explained to students and staff.
Assessment and 4.1 Assessment and feedback activities, and the outcomes of assessment, are reviewed via the class
feedback practices are review and external examining processes.
continuously reviewed. 4.2 Assessment and feedback activities are continuously reviewed to ensure effective alignment with a
programme’s intended learning outcomes and graduate attributes.
4.3 Opportunities to develop effective practice and innovation in assessment and feedback are available to
all staff involved in assessment.
Boud's Ten Principles of Great Assessment Design (2007)

1. The activity is a learning experience
2. It is seen as valid and worthwhile
3. It actively promotes learning and skills beyond the act itself
4. The student is an active agent throughout
5. It has a strong positive backwash effect on learning during the course
6. It enables students to celebrate and portray achievements
7. It is part of a sequence of great designs over the course as a whole
8. It arises from a great learning environment
9. It is not excessively resource-intensive
10. It requires and prompts informed judgement
Rowntree's 17 Proposals for Better Assessment
(1977 as cited in Draper, 2008)
1. Articulate the assessment criteria; including trying to express our implicit assessment-constructs.
2. Use more varied assessment methods. Make them educationally relevant.
3. Give credit for what learners learned, as well as whether they learned what we intended.
4. Assess "naturalistically" i.e. use assessment processes and products that are themselves educationally valuable.
5. Give learners maximum feedback (not just a grade or rank, but summative of their traits/qualities).
6. When criteria are judgmental, say (to learners) whether their performance is being compared to norms, criteria, our expectations,
or the learner's own previous performance.
7. Colleagues may have quite different perceptions.

Accept this, don't converge unnaturally; report divergence.

Give back exam scripts.
8. Resist drifting to criteria that attract consensus marks: stay with the educationally relevant ones.
9. Support portfolios: including both products and assessments from many including peers and self.
10. Report results only to learners (i.e. not made public).
11. Focus on eventual, not average or early, state (unless describing improvement rate).

Emphasise learners' strengths, but mention weaknesses.

12. Don't conflate i.e. no portmanteau grades. Prepare a multi-dimensional profile: with considerable narrative content.
13. No pass/fail except for professional competence certification.
14. No comments in confidential references that you wouldn't have learners read.
15. Be explicit in references that the assessment is about specific things; that it is not about permanent qualities; require that you are
given some understanding of how the reader will use the report; get the relevant qualities from the reference-requester.
16. If we predict learners' future qualities, follow up and see how right we were(n't).
17. Give health warnings on certificates (transcripts) i.e. about the limits on how much weight to give accreditations as a measure of
the person. E.g. "Relying too heavily on other people's opinions can damage your sense of reality."
Six Tenets of the “Assessment Standards: A Manifesto for Change”
(Price, O'Donovan, Rust, & Carroll, 2008)

1. The debate on standards needs to focus on how high standards of learning can be achieved through
assessment. This requires a greater emphasis on assessment for learning rather than assessment of learning.

2. When it comes to the assessment of learning, we need to move beyond systems focused on marks and
grades towards the valid assessment of the achievement of intended programme outcomes.

3. Limits to the extent that standards can be articulated explicitly must be recognised since ever more detailed
specificity and striving for reliability, all too frequently, diminish the learning experience and threaten its validity.
There are important benefits of higher education which are not amenable either to the precise specification of
standards or to objective assessment.

4. Assessment standards are socially constructed so there must be a greater emphasis on assessment and
feedback processes that actively engage both staff and students in dialogue about standards. It is when
learners share an understanding of academic and professional standards in an atmosphere of mutual trust that
learning works best.

5. Active engagement with assessment standards needs to be an integral and seamless part of course design
and the learning process in order to allow students to develop their own, internalised, conceptions of standards
and monitor and supervise their own learning.

6. Assessment is largely dependent upon professional judgement and confidence in such judgement requires
the establishment of appropriate forums for the development and sharing of standards within and between
disciplinary and professional communities.
Common Ground

G ibbs & Sim pson 10

Palloff & Pratt 7

Nicol 7-10-11

Row ntree 17

Tenets 6
Boud 10
UoM 16
NUS 10
UofG 5

UoS 4
1. Participation of students in the
development of assessment 1 1 1 1 1 1 6
2. Alignment with learning objectives,
outcomes, or competencies 1 1 1 1 1 1 6
3. Self-assessment 1 1 1 1 1 5
4. Workloads/distribution of assessment
in course 1 1 1 1 1 5
5. Choice of assessment 1 1 1 1 4
6. Formative assessment/formative
feedback 1 1 1 1 4
7. Timely feedback 1 1 1 1 4
8. Clear assessment criteria 1 1 1 1 4
9. Peer-assessment 1 1 1 3
10. Constructive feedback 1 1 1 3
11. Generic skills/backwash 1 1 1 3
12. Motivation for learning 1 1 1 3
13. Marks/grades 1 1 1 3
14. Inclusive assessment/no biases
against groups 1 1 1 3
15. Variety/mix methods/types 1 1 1 3
16. Integration with course design and
learning 1 1 2
Changing Assessment Practices

Sample projects which may serve as a

starting point in conducting similar large-
scale research projects on transforming
assessment practices in open and distance
Cases that may benefit from learning from efforts
to change institutional assessment practices

1. Teachers moving from classroom to online

distance learning.
2. Educational institutions or units thereof
planning to convert their current programmes
to open online distance learning.
3. Existing open online distance learning
institutions reviewing their assessment
practices and planning to change those
Re-Engineering Assessment
Practices in Higher Education
(REAP, n.d)
Watch the video Putting principles into practice - University of
Strathclyde (2014) at https://
Transforming the Experience of Students
through Assessment (TESTA)
(Jessop, El-Hakim, & Gibbs, 2011).

Research Methods

● Programme audit
● Assessment Experience Questionnaire

● Focus group with students
UPOU's Assessment Policies and
Practices Discourse

● Task Force on Assessment

● Round-Tables on Assessment
Assessment Principles within the
ODeL institution
consider gaps in
the principles
and the issues
learn from the in the projects
problems of
experience of assessment in
the projects massive courses

meaning making of
integrity and massive data
assessment Automation
transactional of
in MOOCs
distance assessment
variety and Learning
choices of analytics
Angelo, T.A., & Cross, K.P. (1993). Assessment techniques. A handbook for college teachers. 2nd ed. San Francisco: Jossey-Bass.

Berry, R. & Adamson, B. (Eds.). (2011). Assessment reform in education: Policy and practice. Dordrecht, Heidelberg, London, and New York:

Boud,D. (2007). Great designs: what should assessment do?. Presentation for the REAP International Online Conference on Assessment Design
for Learner Responsibility, 29th-31st May, 2007. Retrieved from Also
available at

Draper, S.W. (2007, May 27). A momentary review of assessment principles. Retrieved from

Gibbs, G. & Simpson, C. (2004) Conditions under which assessment supports student learning. Learning and teaching in higher education, 1, 3-31.

James, R., McInnis, C. & Devlin, M. (2002). Assessing Learning in Australian Universities. Ideas, strategies and resources for quality in student
assessment. Centre for the Study for Higher Education for the Australian Universities Teaching Committee, p.9. Retrieved from

Jessop, T., El-Hakim, Y., & Gibbs, G. (2011, December). The TESTA project: Research inspiring change. Educational developments: The magazine
of the Staff and Educational Development Association Ltd., 12.4, pp. 12-16.

LearnTeachLead. (2014). Rethinking classroom assessment with purpose in mind. [Digital video]. Retrieved from
Nicol, D. (2007). Principles of good assessment and feedback: Theory and practice. From
the REAP International Online Conference on Assessment Design for Learner Responsibility,
29th-31st May, 2007. Retrieved from

Nicol, D. (2015, September 15). Principles as discourse: A blueprint for transformational change in assessment. Presentation at Sheffield Hallam
University. Retrieved from .

Nicol, D., Draper, S., & Creanor, L. (2007). Re-Engineering Assessment Practices Project (REAP) completion report. Retrieved from

Nicol, D., & Draper, S. (2009). A blueprint for transformational organisational change in higher education: REAP as a case study. In M. Mayes, D.
Morrison, H. Mellar, P. Bullen & M. Oliver, Transforming higher education through technology-enhanced learning. York, U.K.: The Higher Education
Academy. Retrieved from

Nicol, D. J. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated

learning: A model and seven principles of good feedback practice. Studies in Higher
Education, 31(2), 199-218. Retrieved from

National Union of Students. (2015). Assessment and feedback benchmarking tool. Retrieved from
Palloff, R. M., & Pratt, K. (2003). The Virtual Student: A Profile and Guide. San Francisco: Jossey -
Bass, pp. 101 – 102.

Palloff, R. M., & Pratt, K. (2009). Assessing the online learner. Resources and strategies for faculty.
San Francisco, California: Jossey-Bass, p. 30-43.

Price, M., O'Donovan, B., Rust, C., & Carroll, J. (2008, December). Assessment standards: A
manifesto for change. Brookes eJournal of Learning and Teaching, 2(3). Retrieved from .

REAP. (n.d.). Assessment and feedback principles: Rationale and formulation. In Re-engineering
assessment practices in higher education. Retrieved from .

Re-engineering assessment practices in Scottish higher education. (n.d.). Retrieved from .
Rowntree, D. (1977) Assessing students: How shall we know them? London: Kogan Page.

Rust, C. (2007). Towards a scholarship of assessment. Assessment & Evaluation in Higher Education, 32(2), pp. 229-237.

Draper, S. (2008, April 9). Rowntree's 17 proposals for better assessment. Retrieved from Originally published
in Rowntree, D. (1977) Assessing students: How shall we know them? London: Kogan Page.

University of Glasgow. (n.d._1). The Principles of Assessment and Feedback for Learning. Assessment and Feedback Toolkit. Retrieved from

University of Glasgow. (n.d._2). What is LEAF. Retrieved from

University of Glasgow.(n.d._2). Leading enhancement in assessment and feedback. Assessment and Feedback Toolkit. Retrieved from h

University of Strathclyde. (2014). Assessment and feedback policy. Retrieved from

University of Strathclyde. (2007). Overview of Professor David Boud: Keynote theme 2. In the REAP07: Assessment design for learner responsibility [Website].
Retrieved from

University of Strathclyde. (2017). Re-engineering assessment practices in higher education. [Website]. Retrieved from

University of Strathclyde. (2017b). Assessment principles: Some possible candidates. In Re-engineering assessment practices in higher education. Retrieved from
This concludes topic
1. Kindly do the
related activity(ies)
in the MOOC site.
Thank you.