EPTA webinar 29 July 2020: ‘Should theory exams be online or offline?

ABRSM introduction for EPTA members and guests

We are very pleased to be invited by EPTA to be panellists for this webinar to
address the question, ‘Should theory exams be online or offline?’ We believe the
learning of music theory is important and deserves review and modernisation as
for any of our qualifications. We seek to develop our exams in ways that meet the
needs of learners and provide better progression routes that prioritise access and
inclusion. The new situation that Coronavirus has created means that our teachers
and learners are seeing a pace of change that is bound to raise questions, ones that
we are pleased to answer. Our wish in participating as panellists is to benefit from
hearing the viewpoints raised and to provide more detail about what we are doing.

Penny Milsom, Executive Director, ABRSM

29 July 2020

ABRSM and Music Theory

ABRSM believes in the value that comes to learners from understanding how music
works. Through our Theory exams, supporting publications, practice materials and,
in recent years, our development of a learner-friendly app, we actively encourage
music theory learning and see this as part of our mission to inspire achievement in
music.

For many of those learning to play or sing, we know that the deeper understanding
and practical skills that flow from understanding the written language of music and
weaving this into practical learning can enhance a musician’s development and
enable greater freedom, independence and versatility as a player or singer. Our
position as the only music Awarding Organisation requiring Grade 5 Theory, or one
of our alternatives, for entry to the higher grades is a tangible demonstration of our
beliefs.

This is the context for our work over a number of years to develop and update our
theory exams and resources. We have long been working towards modernised
assessment approaches that represent best practice in the expert area of
assessment design. We want to offer assessments that give candidates the greatest
opportunity to demonstrate their musical understanding and that increase the
convenience of taking a theory exam in order to help smooth the learning pathway
for young people under many educational pressures and committed to progressing
with their music learning.

Coronavirus and sudden change

As with many other areas of life, Covid-19 has put music teaching and learning
under enormous pressure. We recognise the effect on teachers and learners of a
series of recent and rapid changes to the operation of our exams as we, too, have
worked hard to respond to the changed world.
We have been listening to teachers, learners and parents during this time as we
make adjustments to help the resumption of exams. We want to hear from people
directly and we welcome EPTA’s drawing together of voices and perspectives to
focus specifically on the online theory exams. We want to respond to feelings and
questions about the acceleration of our long-term plans and to continue to be as
adaptable as possible within the constraints we face as we resume activity. We
recognise that the pace of change and the short time-scales are not comfortable for
any of us. We are committed to doing everything we can to support these changes
in the months ahead, having prepared for them with great care during the period
while our theory exams have not been available.

Our live pilot for Grade 5 Theory in August is informing developments ahead of
Grades 1 to 5 in November and we shall actively seek feedback to make further
adjustments. The questions we have had already have been extremely helpful in
allowing us to clarify areas of question or concern and, in some cases, to correct
misconceptions. Our website is the source of up-to-date information, guidance and
answers to FAQs and the best place to go for answers.

Some central themes have emerged that we are addressing actively, both by
providing more information to evidence the care we have taken and our confidence
in the integrity of our arrangements and also by working to provide more
information and guidance to support preparation for the November exams. Where
issues arise that require further thought and preparation we are committed to
addressing these.

I would like to respond here to some of these key themes, with more detail first
around the important area of changes to the assessment itself that mean that we
will no longer ask candidates to write out music by hand.

A short history of our theory developments: Why change our

assessment approach?

We began a major review of our theory exams several years ago to explore how
best to support this learning in a changing environment. We saw growing evidence
of pressures on lesson time and a move away from lessons devoted exclusively to
theory in favour of learning within practical lessons where time allowed. We
discovered from long-term trends and research that a number of areas could be
developed to support music theory better. The most striking trend was the
increasing evidence of less music theory learning at the lower grades, with Grade 5
increasingly taken in isolation, often prepared for at relatively short notice and
with evidence in exam outcomes of ‘teaching to the test’ in ways that did not align
with the intentions of the syllabus.

Further research identified a number of areas to improve, with a focus on how we

could best encourage ongoing learning through our exams and resources. Headline
areas to address were the accessibility and convenience of exams, the design of
questions to ensure that they perform well in enabling candidates to demonstrate
what they know and are reliable and valid, the simplification of language to cater
for all ages, those with special educational needs and non-English speakers, and the
support we provide through our published and digital resources.

Our review identified that ABRSM’s support for music theory has impact in two
distinct, separate areas. The existence of graded exams is undoubtedly a stimulus to
achievement. However, the resources we, and many others, publish to support
music theory learning have an impact beyond what is tested in the exam and are a
vital component in encouraging this learning.

Since that period of review and research we have been working with a range of
musicians and educationalists to modernise aspects of the qualifications without
fundamentally changing the core knowledge and skills that are tested. Our drive is
towards objective questioning that allows learners to demonstrate their knowledge
more comprehensively, with less jeopardy on individual questions and more
emphasis on their having assimilated the underlying concepts.

We have long had a goal of increasing the convenience and accessibility of taking an
exam by moving to online individual assessment in favour of travelling to an exam
venue on a limited number of dates each year. We have been preparing for this
gradually, hand in hand with our careful review of exam question design to ensure
fairness and reliability. Our original plans were to introduce changes to the
question design in 2021 with a move to online exams to follow. Covid-19 has
disrupted this timeline and caused us to advance developments in order to enable
access to our exams once again at a time when doing so in person is not safe. We
fully recognise that enabling this access to exams comes with pressures on teachers
in adapting quickly to the new exam format.

We have also been developing our published and digital resources in ways that
encourage music theory learning and we will be publishing new workbooks in
October. As well as providing help in building the skills, knowledge and
understanding required for our exams, these workbooks encourage learners to
apply theory in a wider musical context and really see how it supports and
enhances practical skills. Each topic covered is followed by progressive and varied
step-by-step exercises, including many opportunities to practise writing out music,
a skill that continues to be valued and that we know and expect will be an
important part of learning for many.

One of the clear findings that emerged from our first phases of development after
our review was the difference between what is taught - which varies greatly across
teaching environments and from teacher to teacher - and what is tested by the
exam. This is felt most keenly in the area of writing out music by hand, an
important skill for many, but by no means all, learners and one that is not core to
the elements of knowledge and understanding that are specifically tested in the
exams at Grades 1 to 5. As with other areas of education we see a blend of
approaches and are confident that the skill of writing out music by hand will
continue to live alongside the ongoing increase in the use of music notation
software.

We made our first significant changes to question design in 2018. This included the
removal of the melody-writing question. This is an important area to highlight as
we received much feedback at that time, both in support of the change and from
those regretting it. We were very conscious that this would be felt as a loss by
some, especially for those who used preparation for this question as part of their
support for their students’ creative and compositional skills. However, there was in
reality a gap between this intention and the actuality both of how marks were
awarded for this question and how most candidates prepared for and answered it.
The question was intended to assess the ability to manipulate musical notation and
did not assess creativity. Exam answer papers showed that many students
answered according to a formula and also that this form of question did not
perform robustly in terms of consistency and spread of marks. Few candidates
performed well in this question.

I include this example in order to demonstrate that ABRSM has a responsibility not
only to support learning that is of value to a developing musician but, in the exam
context, to ensure that each question is valid and performs reliably in testing the
knowledge it sets out to assess. Objective test questioning is an area all responsible
exam boards address as they develop assessment approaches and benefit from the
significant research and expertise that exists in the world of question design.

Our approach to theory, represented in the changes we have accelerated in

response to Covid-19, is akin to our examiners’ motto in our Practical Grades: “I
will do my best to help you do your best”. The changes we have developed – now
introduced earlier than planned as we prioritise access to exams for thousands of
candidates – are made in this same spirit of enabling candidates to evidence their
knowledge and understanding. We remain committed to our important
responsibility as an Awarding Organisation to review and develop our assessments
so that each candidate can demonstrate what they know, in assessments that are
robust, resilient and fair.

Ensuring access to online exams

We have chosen the leading provider of remote assessments and remote

proctoring, PSI, to host our Theory exams because they have the credentials to
support our global customer base in a safe, reliable way, themselves handling over
15 million exams every year. These credentials are vital in two key areas, the
safeguarding of young people and provision for those with specific learning needs.

Safeguarding
The safeguarding of young people is of paramount importance to us, and we are
entirely confident that PSI’s policies and systems uphold this fundamental
principle. A change as radical as this to the way that exams are invigilated will
inevitably lead to reservations, and we will provide factual information to help our
customers come to a view. 43% of 71 AOs who responded to an Ofqual survey are
using remote invigilation, and the record and review system we have chosen is the
least intrusive because no candidate is invigilated ‘live’. The remote proctoring
arrangement is documented in guidance on our website. The salient points about
the system are:

 No candidate is watched ‘live’ during the exam. The exam is recorded and
viewed by trained proctors later for signs of malpractice.
 The room scan is necessary to ensure the exam environment.
 The remote proctoring software does not have access to any files on your
computer, but it does detect what processes are running to prevent
unauthorised applications (therefore potential cheating) being used during
an exam.
 Access to the videos is tightly controlled, restricted to the corporate VPN,
and monitored.
 All proctors are subject to background checks that are commensurate with
our own safeguarding policy, including criminal record checks.
 The Responsible Adult can be present for the duration of the exam if
desired.

Specific learning needs

No one with specific learning needs will be excluded. We have conducted
considerable research as to the suitability of a screen-based exam for all candidates,
and PSI’s exam system is WCAG2.21 compliant which means its overall content is
accessible to people with disabilities including visual, auditory, language, learning
and neurological disabilities. Again we intend to release more guidance on the
accessibility features and demo the question types on the PSI platform in the week
commencing 10 August.

Our intention is and always has been to adopt a personalised approach to all
individual candidates with access requirements and support them fully through the
August pilot exam. We will use the learnings to improve on the future product.

Access to computers
We recognise that an online exam presents different barriers to learners than face
to face exams in terms of access to technology and finding the quiet space in which
to sit the exam undisturbed. We will continue to monitor the impact of these and
design suitable adjustments and support for future exams. We will work in
partnership with others to understand where barriers exist and to identify
solutions. We are absolutely committed to making sure that no learner is left out
and are working to ensure that this is the case.

Why move from paper to online? Is it not possible to provide both?

Our priority is to provide a method of examining that generally is more resilient to
the local or country-level disruption we will continue to experience. Whilst we
might hope that it will be easier to manage larger gatherings safely in due course,
there is still a great deal of risk in backing that prospect. We will, however, keep
under review the practicality and need for paper-based exams in 2021.