Student No.

13029903
Introduction
It is the intention of this assignment to explore the use of portfolio in
assessment.
Pertinent literature related to the use of portfolio as an assessment tool will be
utilised in the first part. The authors personal experience of portfolio will be
discussed in the latter.
The rationale that underpins the authors choice of assessment tool for
discussion relates to personal experience of portfolio which has been for the
purpose of revalidation and assessment. Reviewing literature and professional
forums lead the author to realise the heavily controversial nature of the use of
portfolio as assessment and indeed to one medical student who has put
together a petition to request that the production of portfolio as assessment
be banned. The author believes there are few methods of assessment which
attract more controversy than the use of portfolio and this assessment will
represent an exploration of this.
Self-directed production of a portfolio without an end date for submission or
review makes its use dubious however production of a portfolio for summative
assessment drives the learner to develop one that meets certain criteria as
defined by the assessing body.

Purpose of Assessment
Miller (1990) described the purpose of assessment via a simple pyramid
structure demonstrating the journey of the learner from the lowest point of
knowing or having knowledge, progressing to knows how or competence,
shows how through performance and finally at its pinnacle does therefore
producing an action. This simple concept of the purpose of assessment to
demonstrate the learner has progressed remains fundamental in assessment
and specifically in relation to methods such as the portfolio. Yudkowsky (2009)
accepts Millers pyramid as a useful model to guide health professionals.
The GMC(2009) believe it to be essential that assessments provide a valid and
reliable judgement of an individual students performance they further state
that methods of assessment must be fit for purpose in that they measure what
they set out to measure, this process must be fair and consistent and a range
of techniques utilised. The aim of this is to ensure students demonstrate the
knowledge skills and behaviour identified in the curriculum. Boud and
Falchikov (2006) further state that assessment is a method of providing
feedback and ultimately enables students to achieve certification. With specific
relation to medical assessment techniques Norcini (2010) claim that
assessment has developed dramatically over the last fifty years to cover more
competency based outcomes whilst not losing validity or reliability, although it
will be discussed that this is not a view held by all. Van der Vleuten (1996)
believes that for a method of assessment to be of value there are five essential
criteria- reliability, validity, acceptability to learners and faculty, cost and
impact of future learning and practice,. Each of these factors will be considered
with specific relation to the use of portfolio as a means to assessing a
candidate.
Reliability and Validity
Swanwick (2010) describes the reliability of an assessment technique as the
importance of providing similar results on different occasions with various
candidates. And the validity as how fit for purpose the assessment tool is in
that it measures exactly what it sets out to measure. Swanwick (2010) believes
it is difficult to say anything about the reliability of the portfolio, this is of
concern as reliability is considered essential. Downing and Yudkowsky (2009)
claim that for methods of assessment to be meaningful and successful there
must be a plan in place underpinned by a rational system to join the learning
outcomes to the considered important knowledge and skills required. They
criticise portfolios claiming the pervasive subjectivity can have a detrimental
effect on its validity and reliability in comparison to written exams. However
they describe this as a challenge as opposed to impossibility and recommend
portfolio is used as part of a comprehensive system to measure a learners
competence with specific learning objectives and examples of the types of
evidence required. Epstein (2007) supports this belief stating that indeed all
methods of assessment have strengths and weaknesses minimised by the use
of a variety of methods. The author has never seen the use of portfolio as a
means of assessment in isolation so it appears this is the general consensus
among educational bodies.
Pitts et al (2001) conducted a relatively small study that attempted to explore
the use of portfolio as a tool for assessment. They question the need for the use
of validity and reliability in the use of portfolio as one of the strengths of the
tool is that there is some validity in recognising what is wrong thus requiring
improvement as opposed to traditional methods of assessment which give
marks only for what is right. They criticise the use of portfolios in assessment
for commonly utilising the dichotomous pass or fail marking system and suggest
more work should be done in to recognising the qualitative nature of the
portfolio where creativity is valued.

Downing and Yudkowsky (2009) recognise however that portfolio is still a
popular assessment method and that to address the challenge related to
validity the portfolio should have learning objectives and offer examples of
desirable evidence. Would this theory however encourage the student to
produce a portfolio of evidence towards the specifications as opposed to their
own individual learning needs? Swanwick (2010) agrees with this further
stating that to achieve reliability a controlling format limits the value of
portfolio as it constricts student individuality however Biggs and Tang (2011)
argue identifying learner outcomes and favourable examples of evidence with
clear limits prevents the student going overboard. Tigelaar et al (2004) agree
that portfolios can cause controversy as they often combine both the
formative and summative types of assessment however they add that one can
actually guide the other. For example they suggest that for summative
assessment suggestions of desirable types of evidence are offered therefore
making the assessment uniform and enabling reliable and consistent rating
without hampering individual creativity ie partly open ended and partly
prescribed.
Gadbury-Amyot (2003) attempted to score portfolios quantitatively through a
relatively small study looking at twenty portfolios using seven sub scales eg
competencies lifelong learning etc across a four point likert scale. They claim
their methodology resulted in an acceptable phi coefficient of 0.83 which
increases the psychometric reliability but recognise this is at the expense of
the holistic development of the portfolio which is its ultimate strength. This
suggests they are trying to measure the portfolio using a restrictive method in
an effort to support its reliability but compromise its validity.
A small study by Driessen et al (2006) states that generally the portfolio is
considered to have high validity which allow insight in to learners clinical
competence via the production of evidence over a period of time. They do
recognise, however that studies directly related to validity are relatively few
whereas studies related to reliability are greater.
McCready (2007) recommends that quantitative approaches to traditional forms
of assessment, in the case of the use of portfolio should be swapped for
qualitative measures such as credibility, transferability, dependability and a
holistic approach which must take in to account the potential subjectivity of
the assessor. Swanwick (2010) states this is a paradox moving away from
objective and standardised methods of assessment towards a more subjective
non- standardised method. They concur achieving high reliability through
constricting and controlling format then scoring through a scale or checklist
limits the greatest value of the portfolio which is a reflective tool providing
evidence of personal professional practice identifying both strengths and
weaknesses.
Acceptability to learners and faculty
Pitts et al (2001) accepts the portfolio has become a popular choice, an
alternative to more traditional forms of assessment and agrees the use of
portfolio is beneficial for personal and professional development however as an
alternative method of assessment believes this should be explored further with
specific relation to the assessors subjectivity.
Crossley (2013) recognises that assessor based bias can be problematic.
McManus et al (2006) concur that assessing qualitative work has been
recognised as problematic for over a century with specific relation to rater bias
i.e. the Hawk v Dove effect whereby some examiners are more stringent and
others more lenient. Azer (2008) argues that layout and writing style can also
have a negative effect on rater bias. Driessen et al (2006) agree with this theory
claiming that raters may view portfolios in a negative way if they consider
irrelevant factors such as writing style and layout. They suggest a portfolio
scoring inventory which produced inter rater agreement to within an acceptable
range of moderate to excellent (0.46-0.87). They recommend that having a
scoring system in conjunction with assessor training can reduce rater bias with
specific relation to irrelevant factors.
Downing and Yudkowsky (2009) suggest using multiple raters as a way to
minimise rater bias however they do not comment on the time or cost
implications of this. Mcmanus et al (2006) state that often when considering
rater bias the pass or fail of the student can be purely down to luck depending
on which examiner their portfolio is assessed by.
This can be a potential cause of anxiety for the student however the practice of
multiple rates or second markers surely would reduce this.
Challis (1999) argues that the time implication for the production of a portfolio
may make it a less favourable choice to the learner specifically in comparison to
other methods of assessment however how many hours would a student spend
revising and memorising information that may or may not appear on a
traditional examination paper to then be completely forgotten and not utilised
in the future.
Azer (2008) concurs that one of the challenges related to the use of portfolio is
both learners and tutors often view the portfolio as labour intensive, they state
that this is born from a lack of knowledge and recommend workshops to
introduce the concept to its full potential which should also include examples of
previously constructed portfolios. However would this increase the cost?

Cost
McManus et al (2006) criticise portfolios as being a costly method of assessment
due to the time it takes to produce and assess them however Biggs and Tang
(2011) argue that even though portfolio can be time consuming to assess they
can be very interesting and preferable to assessing many very similar papers.
Clearly the cost in time both to the student and the assessor is relatively high in
comparison to more traditional methods of assessment.
Impact on future learning and practice
When used to its full potential the portfolio can greatly impact on future
learning and have a positive effect on practice which has to be one of its major
strengths. This is demonstrated below in the authors experience of the use of
portfolio. Challis (1999) believes the portfolio can drive and engender lifelong
learning within the autonomous real experience of the learner through
reflection. They describe this as integral to professional development via
identification through reflection of learning needs which ultimately should
improve practice.
The strengths of portfolios, the literature suggests, appears to be related to the
unique nature whereby the individual can create their own record of learning
needs and how this is addressed via reflection. Stark (2013) concurs that
portfolios encourage reflection and the practice of self-evaluation. Downing and
Yudkowsky (2009) aptly describe the portfolio as a vehicle which promotes
reflection and provides evidence of this. Benbow and Mooney (2013) support
this view and expand up on it by stating that portfolios enable the user to
identify their individual strengths and weaknesses in order identify areas of
improvement, they describe this as essential when creating an atmosphere of
openness for patient safety. Challis (1999) suggests the use of portfolios is a
move away from the traditional didactic form of teaching to place the emphasis
on the learner to identify and develop their individual learning needs. This will
allow unlimited progression and development.
Boud and Falchikov (2006) believe that a further function of assessment should
be to equip students to continue on their learning journey once the higher
education infrastructure is no longer present. They recognise that this additional
function of assessment has not traditionally existed with more commonly used
historical methods which they criticise encourage students to focus on marks
rather than the learning they represent. Surely on going reflection via portfolio
production is the perfect vessel for this
Authors Experience

NMC(2008) allude to the practice of portfolio as Documented evidence of learning
activities and feedback on nurse or midwifes practice.
From a personal perspective the author found this definition vague and
uninspiring causing a fairly negative perception of the use of portfolios born
from the notion that portfolios were a necessary process for revalidation which
was never policed and mostly comprised of a collection of certificates from
various study days. Rees (2005) terms this abuse of the term portfolio claiming
that to make portfolio of use it must record and reflect on key learning issues
and that this process is at the heart of the process of mandatory self-reflection
required to make portfolio of value. Rees (2005) further states that this
confusion regarding the correct application of portfolio is a common
misconception among health professionals.
Secondly the author has been asked to develop various portfolios of evidence
to demonstrate suitability for certification in various speciality areas such as
certification in contraception and education as part of the assessment process
whereby criteria would be set so the author would seek out evidence to
equate to this. This process introduced the author to considering portfolio to
be a valuable tool for personal development and indeed a tool for assessment.
Azer (2008) supports the use of portfolio for assessment as it can measure
previously considered difficult to measure critical aspects of students ability
such as attitudes, professionalism and teamwork
This view specifically supports Biggs and Tang (2011) argument regarding
constructive alignment and Boud and Falchikov (2006) who argue that portfolio
as assessment actually takes this process to another level whereby it drives
future personal development.
To demonstrate the authors experience of this when encouraging the value of
Socratic questioning to help non doctors challenge medical students to
engender critical thinking this lead the author to recognise the importance of
further research by gathering medical students feedback of time spent with
non-doctors in practice and its usefulness and potential barriers to this. Being
currently engaged in a university educational role this research will be
conducted by the author who considers it to be a relatively new area of research
of value to tutors and ultimately drive to improve the quality of the medical
student experience when placed with a variety of health care professionals. The
author has also produced an abstract related to these findings to potentially
present at an educational conference.
Secondly when the author reflected upon some teaching provided one of the
attendees commented on their difficulties with small group teaching. The
reflection caused the author to realise this had not been addressed within the
session and so then approached other team members for advice on small group
teaching skills which was then shared. The author also, as part of the new
academics program, enlisted to join a development group specifically related to
small and large group teaching skills.
This demonstrates constructive alignment perfectly and shows a direct relation
to relating theory to practice resulting from the positive use of portfolio. Indeed
Biggs and Tang (2011) describe that the whole concept of constructive
alignment was born from an investigation of portfolio as a means of assessment.
Constructive as the learner devises their own learning needs and aligned to the
specific learning outcomes in criterion referenced assessment. They further
state this empowers the learner to utilise a qualitative approach to reflect, self-
evaluate and ultimately apply the theory learnt to practice supporting a higher
level of learning as opposed to many traditional forms of assessment which
generate a lower level outcome (recognise, memorise describe)
However it would appear that the very strengths of the portfolio to promote
individual growth, in whatever direction the learner determines, some would
argue make it a weak tool to be used as assessment. Challis (1999) concurs that
this practice may make the learner produce a portfolio that meets the needs of
the assessment process and not the needs of the learner themselves.
Conclusion
The use of portfolio, absolutely, remains en vogue however the debate
regarding its use as an effective assessment tool is on-going.
Clearly there is evidence that when used to its full potential it can develop the
individual to be a reflective, critical thinking practitioner who identifies their
own learning needs and relates these to practice as described via constructive
alignment. As an assessment tool the use of portfolio can measure personal and
professional development whilst gaining evidence of performance and its
application to practice thus bridging the theory to practice gap. It could be
argued that the use of portfolio measures behaviour and professionalism which
historically have been largely ignored yet is critical.
Challis (1999) recognises however that one of the biggest threats to the use of
portfolio is with regard to confidence in its value from the user which they
claim, if it is not addressed, will undermine the value of the entire process. The
potential benefits of the use of portfolio as a tool to develop lifelong self-
assessing learners could indeed take the purpose of assessment to another level
however it would appear further recognition of this through increasing the
knowledge of its potential value to its user is very much in its infancy.

(Word count 2913)

References
Azer, S., (2008). Use of Portfolios By Medical Students: Significane of Critical Thinking. Journal of
Medical Science, Volume 24, pp. 361-366.
Benbow, E. & Mooney, J., (2013). Assessments and Examinations: A guide, Manchester: Manchester
Medical School.
Biggs, J. & Tang, C.,( 2011). Teaching for Quality Learning at University. 4th ed. London: Maidenhead
McGraw-Hill.
Boud, D. & Falchikov, N., (2006). Aligning assessment with long-term learning. Assessment &
Evaluation in Higher Education, 31(4), pp. 399-413.
Challis, M., (1999). AMEE Medical Education Guide No.11(revised):Portfolio-based learning and
assessment in medical education. Medical Teacher, 21(4), pp. 370-386.
Crossley, J., (2013). Validity and truth in assessment. Medical Education, Volume 47, pp. 1152-1161.
Driessen, E., Overeem, K., Tartwijk, J. & Van Der Vleuten, C., (2006). Validity of Portfolio Assessment:
Which qualities determine ratings?. Medical Education, 40(9), pp. 862-866.
Epstein, R.,( 2007). Assessment in Medical Education. The New England Journal of Medicine.
General Medical Council, (2009). Tomorrows Doctors, London: GMC.
Govaerts, M. & C., V. d. V., (2013). Validity in work-based assessment:expanding our horizons.
Medical Education, Volume 47, pp. 1164-1174.
Hrisos, S., Illing, J. & Burford, B., (2008). Portfolio learningfor foundation doctors:early feedback on
its use in the clinical workplace. Medical Education, Volume 42, pp. 214-223.
McCready, T., (2007). Portfolios and the assessment of competence in nursing: A literature review.
International Journal of Nursing Studies, Volume 44, pp. 143-151.
McManus, I., Thompson, M. & Mollon, J.,( 2006). Assessment of examiner leniency and stringency
('hawk-dove' effect) in the MRCP (UK) clinical examination (PACES) using multi-facet Rasch
modelling. BMC Medical Education, 42(6).
Miller, G., (1990). The Assessment of Clinical Skills/ Competence/ Performance. Academic Medicine,
65(9), pp. 63-67.
Norcin, J.; (2010) The Power of Feedback Medical Education, 2010, Vol.44(1), pp.16-17
Pitts, J., Coles, C. & Thomas, P., (2001). Enhancing reliability in portfolio assessment:'shaping' the
portfolio. Medical Teacher, 23(4), pp. 351-356.
Rees, C., (2005). The use (and abuse) of the term 'portfolio'. Medical Education, Volume 39, pp. 436-
437.
Schwirth, L. & Van der Vleuten, C., (2003). The use of clinical simulationsin assessment. Medical
Education, Volume 37, pp. 65-71.
Stark, P.; (2013) The Art of Portfolio, Scottish Universities Medical Journal Volume 1 P1
Swanwick T.; (2010). Understanding Medical Education Evidence, Theory and Practice. Wiley-
Blackwell, pp. 232-234.
Tigelaar, D., Dolmans, H., Wolfhagen, I. & Van der Vleuten, C.,( 2004). Using A Conceptual
Framework and the Opinions of Portfolio Experts to Develop a Teaching Portfolio Prototype. Studies
in Education Evaluation, Volume 30, pp. 305-321.
Van der Vleuten, C., (1996). The Assessment of Professional Competence: developments, research
and practical implications. Advanced Health Science Education.
Yudkowsky, R. & Downing, S., (2009). Assessment in Health Professionals Education. s.l.:Taylor and
Francis ltd.