Date Published: May 30, 2018
Publisher: Public Library of Science
Author(s): Susan L. Norris, Veronica Ivey Sawin, Mauricio Ferri, Laura Raques Sastre, Teegwendé V. Porgo, Graciela Andrei.
The production of high-quality guidelines in response to public health emergencies poses challenges for the World Health Organization (WHO). The urgent need for guidance and the paucity of structured scientific data on emerging diseases hinder the formulation of evidence-informed recommendations using standard methods and procedures.
In the context of the response to recent public health emergencies, this project aimed to describe the information products produced by WHO and assess the quality and trustworthiness of a subset of these products classified as guidelines.
We selected four recent infectious disease emergencies: outbreaks of avian influenza A—H1N1 virus (2009) and H7N9 virus (2013), Middle East respiratory syndrome coronavirus (MERS-CoV) (2013), and Ebola virus disease (EVD) (2014 to 2016). We analyzed the development and publication processes and evaluated the quality of emergency guidelines using AGREE-II.
We included 175 information products of which 87 were guidelines. These products demonstrated variable adherence to WHO publication requirements including the listing of external contributors, management of declarations of interest, and entry into WHO’s public database of publications. For guidelines, the methods for development were incompletely reported; WHO’s quality assurance process was rarely used; systematic or other evidence reviews were infrequently referenced; external peer review was not performed; and they scored poorly with AGREE II, particularly for rigour of development and editorial independence.
Our study suggests that WHO guidelines produced in the context of a public health emergency can be improved upon, helping to assure the trustworthiness and utility of WHO information products in future emergencies.
The World Health Organization (WHO) shapes public health policy for United Nations Member States through the development of technical guidance. More than a decade ago, in response to harsh external criticism of its guidelines, WHO instituted the Guidelines Review Committee (GRC) with the mandate to develop rigorous methods and procedures ensuring that each WHO guideline is evidence-based and meets the highest international standards so that guidelines are credible, trustworthy and relevant to end-users . WHO quality standards dictate that guidelines must address a critical public health problem, use transparent and explicit processes minimizing potential sources of bias such as conflicts of interest, include diverse perspectives in the guideline development group, reflect the current state of the evidence, and provide a clear link between the evidence and recommendations taking into consideration the balance of benefits and harms of interventions and other important considerations .
We identified 175 information products of which 87 were classified as guidelines. Table 1 summarizes the characteristics of all information products. Table 2 outlines the methods and processes used in the development of guidelines specifically.
Information products issued by WHO during four recent infectious disease epidemics were difficult to find, were often published at long intervals after outbreaks were graded, and demonstrated variable adherence to WHO publication and reporting requirements. Few guidelines met international standards for the development and presentation of trustworthy and usable guidelines, as assessed with tools designed for standard (non-emergency) guidelines.
Despite our efforts to compile comprehensive lists of information products developed by WHO in response to H1N1, H7N9, MERS-CoV and EVD, we may not have captured all relevant documents. We examined only the latest version available: some information products may have been updates which may have different characteristics than previous versions that were developed in shorter timelines or earlier in the emergency response. For practical reasons, we evaluated only information reported in the documents. We may have misclassified derivative products as guidelines if they failed to reference the underlying guideline.
WHO guidelines produced in the context of a public health emergency can be improved upon, helping to assure the trustworthiness and utility of WHO information products in future emergencies. The principles of transparency, minimization of the risk of bias, and reliance on research and other evidence over expert opinion, apply to guidelines developed in any setting or context. With careful self-evaluation, thoughtful reflection on lessons learned in these emergencies, and a firm commitment to continuous improvement, WHO will continue to provide timely and high-quality guidance in the context of public health emergencies.