Special Report
Measuring progress towards the SDGsa new vital science
A new statistical methodology has emerged to gauge progress towards reaching the 2030
deadline of the UNs Sustainable Development Goals. John Maurice reports.
With 193 member states to serve,
the UN system can hardly be simple.
Witness the 17 Sustainable Develop-
ment Goals (SDGs), which, after more
than 2 years of negotiations, the UN
launched in 2015 and which came
with a complement of 169 targets to
be met by 2030 and 231 indicators
for measuring progress in meeting
the targets. As one development
observer pointed out, the SDGs and
the assessment of their progress
are incredibly complex even for the
most astute development and health
experts. In comparison, with only
18 targets and 48 indicators, the
eight Millennium Development Goals
(MDGs) that preceded the SDGs were
childs play.
The complexity of the SDGs has
left many members of the global
health and development community
wondering whether assessing
progress towards reaching the SDGs is
doable. The work reported in todays
Lancet by a team of researchers
enlisted by the Institute for Health
Metrics and Evaluation (IHME), a
global health research centre housed
at the University of Washington in
Seattle, WA, USA, shows that the
task is doable, at least for most of the
health-related indicators chosen by
the UN.
New indices
The data used for this work come
from the 2015 edition of the Global
Burden of Diseases, Injuries, and
Risk Factors Study (GBD), which
covers 188 countries. The team used
these data to estimate how 33 of
the 47 health SDG indicators have
been performing between 1990 and
2015. The indicators include such
measurable parameters as maternal
mortality ratios, the proportion of
births attended by skilled personnel,
www.thelancet.com Vol 388 October 8, 2016
the incidence of several infectious
and non-communicable diseases,
the risk of ill health associated with
environmental, behavioural, and
metabolic factors, the proportion of
the population covered by essential
health services, death rates from road
trac injuries, and so on. To facilitate
comparisons between countries, the
IHME team gave each indicator an
index score, from zero, for countries
furthest from the SDG goals, to
100 for those closest to them.
...Users of our analysis need to
review the results with respect
to their specic needs and then
act on them. With such a
monitoring system that weve
constructed I think there is
more chance of action and of
creating accountability.
The team also developed a health-
related SDG index, which gives a
summary measure of the progress
being achieved on the 33 health-
related indicators. To measure the
influence the MDGs might have
had on current trends, the team
constructed two sub-indicesan
MDG index covering the health-
related SDG indicators that had also
been used to chart progress towards
the MDGs and a non-MDG index for
SDG health-related indicators that
had not been used in the MDG days.
To complete the SDG monitoring
toolbox the research team created
a Socio-demographic Index (SDI) to
judge to what extent the performance
of a country in progressing towards
the SDG health targets diers from
what might have been expected,
given its degree of development. The
SDI uses the countrys income per
capita, education levels, and total
fertility rates to define its overall See Editorial page 1447
degree of development. See Comment pages 1450 and
1453
See Articles page 1813
Creating accountability
For the SDG infographic see
I think this work is import- http://www.thelancet.com/
ant, IHME director Christopher infographics/sdg
Murray tells The Lancet, because it For more on GBD see
provides a baseline for all the health- http://www.thelancet.com/gbd
related indicators for which we have
data. That baseline allows us to know
where the world is and where each
country is with regard to development
goals and that knowledge is a critical
component of accountability.
Our work is saying that the health-
related SDGs are not some abstract
pie-in-the-sky idea. Were saying
they are important signposts that
can be measured and monitored and
with regular annual monitoring these
signposts will contribute to creating
national and global accountability
for progress. With this work we are
doing, we are making a rst critical
step to committing countries to act.
Users of our analysis need to review
the results with respect to their
specic needs and then act on them.
With such a monitoring system that
weve constructed I think there is
more chance of action and of creating
accountability.
Asked if the system has been eld
tested, Murray points out that it is
IHME
Christopher Murray, IHME director
1455
 Special Report

of the tool kit for measuring progress
andcriticallyholding governments
to account.
Im very impressed with this
work, says David Nabarro, Special
Adviser to the UN Secretary-General
for the 2030 Agenda for Sustainable
Development and Climate Change. It
is well crafted by a huge partnership
comprising national actors and also
independent groups. They have
produced a very full analysis of SDG
health-related measurements that will
IHME
measurable and allows countries to
understand where they stand on the
road towards achieving the goals.
In July, the SDSN, together with the
Bertelsmann Stiftung, a private non-
prot foundation based in Gtersloh,
Germany, published an SDG Index
covering all 17 SDGs. The IHME team,
Sachs notes, used variables that
are not otherwise readily available.
I look forward to incorporating
many of them into the revised and
updated 2017 SDSN SDG Index.

IHMEs oces, Seattle, WA, USA
the work of hundreds of researchers
from around the world. Weve had
about 1800 investigators from more
than 120 countries working on the
results of our analysis to see if they
make sense. Moreover, many of our
collaborators are in government or
government research institutions, so
theres been a lot of ground-proong
thats been going on even before we
got to the point of sending our report
for publication.
Whats the next big step for the
IHME monitoring work? We are
starting to work on the universal
health coverage indicators, which
are of great interest to many
organisations and governments and
which we will put a lot of eort into
improving. We are also trying to use
our system to forecast from current
trends all the gaps or shortcomings
that countries will face in 2030, the
deadline year for reaching the SDGs.
In a Comment published in todays
Lancet, Kevin Watkins, who was
executive director of the Overseas
Development Institute from 2013 to
September this year, and is now chief
executive of Save the Children UK,
sees the GDB 2015-based study as
a landmark event. It provides a
detailed snapshot of the state of global
health and an analytical approach to
tracking this dynamic picture. As the
international community embarks
on the transition from the MDGs to
the SDGs, GBD 2015 is a critical part
allow governments and civil society
decision makers to track progress and
confront any challenges in meeting
these SDG goals over time.
What also impresses me is the
robustness of the data this team is
using. Their data are grounded in
national statistics and supplemented
with material from independent
sources. Ill be interested to see how
easy its going to be to continue
producing this kind of comprehensive
data over time. But looking at it now,
I think it holds great progress. My
major concern is with the IHME teams
ranking of countries. I do not consider
that to be helpful.
What also impresses me is the
robustness of the data this
team is using. Their data are
generally grounded in national
statistics and supplemented
with material from
independent sources...
Jeffrey D Sachs, director of the
UN Sustainable Development
Solutions Network (SDSN) and Special
Adviser to UN Secretary-General
Ban Ki-moon, sees the IHME work
as very useful. The IHMEs health
SDG index is a major worldwide
eort to create and harmonise data
across health indicators. The IHME
researchers are to be praised for
having made large-scale efforts in
data collection, data management,
and data standardisation and
transformation. Their work helps
to make the health-related SDGs
Whereas the SDSNs SDG Index uses
easily accessible, published data that
anybody can look up from sources
such as WHO and the World Bank,
the IHME uses an augmented dataset
based on their own data processing
and statistical modelling to ll values
that are missing in published data.
This modelling work is very valuable
but I would like to see more clarity
concerning the condence we should
place on the imputed data.
The IHME should also indicate
where the worlds eorts should be
placed on collecting more detailed,
relevant, and timely health data.
A similar effort is of course very
important for many non-health-
related SDGs where data availability
tends to be far less complete,
notes Sachs.
Also commenting on the IMHE
work in todays Lancet, Devi Sridhar,
professor and chair of Global Public
Health at the University of Edinburgh
Medical School, Edinburgh, UK,
wonders whether the IHME SDG index
will be useful for governments in low-
income and middle-income countries.
The answer is not immediately
clear, she writes, especially since the
SDG index relies on GBD data, which
have been criticised for having limited
use at the national level.
One answer comes from
Isabella Maina, who heads the Health
Sector Monitoring & Evaluation Unit
of the Kenyan Ministry of Health in
Nairobi. Over the past decade, a
good number of low-income and
middle-income countries have made

1456 www.thelancet.com Vol 388 October 8, 2016


tremendous progress in developing
their health information systems,
she tells The Lancet. But many face
diculties that could hinder their use of
the GBDSDG index. Some have weak
data collection systems, inadequately
skilled staff, a shortage of tools and
techniques, and limited political
investment in data management. I am
convinced that the GBDSDG index work
will encourage and help these countries
to strengthen their data management
systems, says Maina. Countries are in
the process of refocusing on the lessons
they learnt during the MDG era and are
domesticating their SDG agendas for
their own specic needs over the coming
years of the SDG era. Using the IHME
SDG health index, they will be better
placed to prioritise the crucial areas of
their data management work that need
strengthening. And on an overarching
level, the SDG health index will, I am
sure, help countries to shape their
health policies so as to meet the SDGs.
However, this will call for concerted and
aligned eorts among all stakeholders
to assist countries in improving their
health information systems.
The Lancet heard a less positive
assessment of the IHME work from
Abul Kalam Azad, director general
for Health Services in the Bangladesh
Ministry of Health, Dhaka, Bangladesh.
The well-intentioned attempt to
analyse GBD data in order to measure
countries ratings on progress in
reaching SDG targets will not, to
my mind, reveal the real picture of a
countrys health outcomes, he says.
The IHME team defined health-
related SDG indicators as indicators
for health services, health outcomes,
and environmental, occupational,
behavioural, and metabolic risks with
well established causal connections
to health. Azad says a measurement
index or indicator should focus
primarily on only a small number of
burdens, such as mortality, morbidity,
economic problems, and health risks
that lead to these burdens. He is also
concerned that if a country is ranked
by a composite measurement index
www.thelancet.com Vol 388 October 8, 2016
that is not valid, the health ocials
of the country could suer negative
consequences. Moreover, some of
the 33 health-related SDG indicators
that the IHME team used for their
analysis were not strictly within the
health sector. This too may result in
the countrys health officials being
unjustly blamed for their failure to
attain these non-health targets.
As for the choice of SDG monitoring
tools, Bangladesh is using the WHOs
World Health Statistics 2016 and its
Global Reference List of 100 Core Health
Indicators. With the health-related
SDG indicators suggested by these
sources we are tracking every one
of our citizens in order to create a
population-based health summary.
...I am convinced that the
GBDSDG index work will
encourage and help these
countries to strengthen their
data management systems...
Progress in countries
Putting their monitoring tool to work
on the 188 countries, the IHME teams
SDG index gleaned from GBD 2015 a
profuse body of country data covering
the 19902015 period. Overall, the
SDG health indicators showed an
improvement since 2000. As health
indicators improved, however, more
people lived longer but had functional
losses associated with ageing.
Countries making the greatest gains
in progress to meet the development
goals since 2000, as measured by
the health-related SDGs, were in
southeast and central Asia, and also
in parts of Latin America. Timor-
Leste, Bhutan, and Colombia showed
the greatest improvements in health
indicators since 2000. Three countries,
Libya, Syria, and Chile, showed
signicant declines.
In 2015, Iceland topped the list of
development performers, followed by
Singapore and Sweden. At the bottom
of the list came the Central African
Republic, Somalia, and South Sudan.
Some ndings were unexpected. The
Special Report
Aj Photo/Hpr Bullion/Science Photo Library
The USA has made slow progress in tackling excessive weight in children
USA, for example, ranked only 28th,
mainly because of its relatively poor
performance on MDG indicators such
as alcohol consumption, childhood
overweight, deaths from interpersonal
violence, self-harm, and unintentional
poisoning. Surprising too was Indias
low ranking at 143rd, despite its rapid
economic growth over the decade. It
had made, however, a poor showing on
maternal mortality, malaria incidence,
mortality from poor access to water
and sanitation, and air pollution. Many
countries in western Europe, Latin
America, and parts of Asia, and also
Australia, attained unexpectedly high
levels. Interestingly, some countries
which had done well on MDG indicators
did badly on the SDG indicators.
Other SDG tracking initiatives
The publication in this issue of the
IHMEs indicator tracking toolbox
will add to a development landscape
already dotted with several initiatives
aimed at tracking the SDGs. Like the
IHMEs GBD-based tool, WHOs Global
Health Observatory operates an
exclusively health-related statistical
database that covers more than
1000 indicators and 194 countries. The
Observatory publishes a compendium,
World Health Statistics, of which the
latest 2016 edition is designed for
monitoring health for the SDGs. The
UN Statistics Division operates a global
SDG indicator database drawing on
data compiled through the UN system
1457
 Special Report
Hindustan Times/Contributor/Getty
Air pollution has contributed to Indias poor performance on SDG health indicators
in preparation for the Secretary-
Generals annual report on progress
towards the SDGs. The Organisation
for Economic Co-operation and
Development is also present in the
development arena. Its study of how its
35 member countries stand in reaching
the SDGs found that they had travelled
70% of the way to Goal 3 (health) and
Goal 6 (water and sanitation) and
50% of the way to Goal 1 (poverty),
Goal 2 (food), Goal 7 (energy),
Goal 11 (safe sustainable cities and
settlements), and Goal 14 (sustainable
use of oceans and seas). The World Bank
also maintains a monitoring database
with data from other organisations
and institutions (including the IHME),
and from statistical offices of more
than 200 countries. Implementing
the SDGs and measuring and
monitoring progress towards them
will require much more data than
are currently available, with more
accuracy, better timeliness, greater
disaggregation, and higher frequency,
notes Haishan Fu, who heads the
World Banks Development Economics
Data Group.
Some observers fear that a profusion
of differing monitoring systems
might cause confusion among
governments, donor institutions,
and other potential users looking
for clear answers to their concerns.
1458
Nabarro, who is one of six candidates
running in the election for the next
Director-General of WHO, recognises
that such a profusion is inevitable and
believes that users may need help to
avoid being confused. If the numbers
produced by the different tracking
Right now we have a complex
agenda but we have no choice.
Sustainable development has
become an urgent need and the
huge value of the SDGs is that
they give a point on the horizon
for us to aim at but they wont
make it easy.
systems turn out to be very dierent
that might be a bit frustrating but
what really matters is that there is
a clear explanation given at some
point of why there are differences.
Moreover, potential users should
always turn to standard-setting
bodies, such as WHO in the case of
health sector issues, for guidance in
choosing an appropriate system.
In March, WHO and its partner
development agencies, countries,
donors, and academics, launched a
Health Data Collaborative aimed at
boosting the capacity of countries
to collect, analyse, and use reliable
health data to be used for local
decision making and for tracking
progress towards the health-related
SDGs. The Collaborative produced the
Global Reference List of 100 Core Health
Indicators and hopes that by 2024 at
least 60 countries will be collecting
and using reliable health data. In
its introduction to its work plan for
201617, the Collaborative explains
its rationale. Many people are still
not being counted and important
aspects of their lives are not measured.
Recent disease outbreaks demonstrate
the urgent need for quality real time
data. Global leaders, national decision
makers and citizens are talking of a
data revolution and want to harness
the 21st century opportunities of
big and open data to address the
inequalities in access to quality
assured, disaggregated data and
information. The monitoring of the
SDGs provides an opportunity to take
this forward and to consider health
in a much more integral manner with
other development goals.
Sachs points to one issue that
complicates monitoring of progress
towards the SDGs. The actual 17 goals
are very good, very balanced and
very important, and will have lasting
power through the 2030 deadline.
The 169 targets are generally ne but
countries will have to make sure which
targets are relevant to their specific
needs and capabilities. But when it
comes to the 231 indicators, were in
a problematic zone. Indicators that are
well measured, accurate, timely, and
relevant for every part of the world are
certainly not at hand right now and its
going to take a number of years before
they will be.
Another hurdle to overcome,
he says, is the complexity of the
SDG agenda. Right now we have
a complex agenda but we have no
choice. Sustainable development has
become an urgent need and the huge
value of the SDGs is that they give a
point on the horizon for us to aim at
but they wont make it easy.
John Maurice
www.thelancet.com Vol 388 October 8, 2016