Research Article: The Imperfect World of Global Health Estimates

Date Published: November 30, 2010

Publisher: Public Library of Science

Author(s): Peter Byass

Abstract: Peter Byass provides an introduction to a PLoS Medicine cluster of articles on global health estimates, and argues why the “estimates debate” is so important.

Partial Text: Global measurement has long been contentious. Three hundred years ago, the exact size and shape of the world were a matter of scientific controversy and estimation. New ways of measurement were developed, and by the early 18th century that uncertainty ceased [1]. One hundred years ago, Sweden started to account for all deaths in its population, by cause, age, and sex [2], and annual summary tables based on individual death registration on a national basis have been published ever since. Today, despite increasing globalisation, there is still no similar universal individual registration of vital events in many countries, and consequently we find ourselves in an era of global estimates of population health. These global estimates are complex amalgams of detailed national measures from countries with universal registration and the best available data—which are often scanty—from other settings. Hopefully the long-term aim of the global health community is to move beyond this era of estimates, towards the relative certainty of accounting for individual health globally. Meanwhile, the purpose of this article is to explore issues and tensions around these currently necessary global estimates.

Current global estimates mainly come from one of two sources: (1) the United Nations (UN) and its specialised agencies (such as the World Health Organization and the United Nations Children’s Fund [UNICEF]) or (2) northern academic institutions. There are important underlying differences between the estimates from these sources, as shown in Figure 1. Why southern academic institutions are not more engaged in the process of developing global estimates, given that the major uncertainties within most estimates centre on southern data, is a further question of interest.

Some of the recent debate and contention around the source of global estimates emerged during 2010 when two separate estimates of global maternal mortality were published. One set of estimates originated from the Institute for Health Metrics and Evaluation (IHME) [12], and the other was the latest update of the UN inter-agency estimates of maternal mortality [13]. It is impossible to conclude which is the more “correct” set of estimates, because if that were measurable as a matter of fact, the estimates would be redundant anyway. However, it is interesting to compare some key issues in the approaches and conclusions of these different estimates. Headline figures were very similar and well within each other’s uncertainty intervals (342,900 and 358,000 maternal deaths worldwide and maternal mortality ratios of 251 and 260 per 100,000, respectively, for 2008). However, there may be important differences at the country level or for specific causes, depending on the data and methods used.

Because estimates are estimates, and not measurements, it is relatively easy for the proponents of particular estimates to claim high quality and reliability, and for the detractors to question the same, with little scope for objective adjudication. The key factor for robustness is the extent of available data, linked, of course, to sound methods. Transparency involves using all available data of quality and relevance, while usually imposing some explicit framework of rules as to what constitutes usable data. Methodological strategies then also need to be set out in a fully transparent manner. Any epidemiological interpretation depends on an understanding of the provenance and sampling basis of the underlying data, which leads to estimates of uncertainty. However, some of the data used in global estimates are so many stages removed from their origin that associated estimates of uncertainty themselves become very complex and hard to understand. Unfortunately, the gaps and uncertainties around data in many instances drive researchers to ever-increasing levels of methodological complexity in attempts to compensate, and transparency may be obscured by these complexities. This can rapidly lead to an “Emperor’s New Clothes” syndrome in which only the cognoscenti truly understand the underlying basis of complex estimates, while the vast majority may be reluctant to admit that the detail is beyond their comprehension.

The undeniable long-term aim must be to foster more and more production of high-quality and complete population data from locations that are as yet devoid of usable material. This implies a bottom-up philosophy emphasising the need to connect with population data at source [15]. If a gradual process of filling in such gaps in global data is realistic, then one would also hope that in parallel with increasing completeness of data there will be reductions in the complexity of appropriate estimation methods. This would lead towards the ideal situation, in which global estimates would become a thing of the past as the world’s population actually became measurable.