Date Published: August 7, 2007
Publisher: Public Library of Science
Author(s): Finlay A McAlister, Sean van Diepen, Rajdeep S Padwal, Jeffrey A Johnson, Sumit R Majumdar, Alessandro Liberati
Abstract: BackgroundTreatment recommendations for the same condition from different guideline bodies often disagree, even when the same randomized controlled trial (RCT) evidence is cited. Guideline appraisal tools focus on methodology and quality of reporting, but not on the nature of the supporting evidence. This study was done to evaluate the quality of the evidence (based on consideration of its internal validity, clinical relevance, and applicability) underlying therapy recommendations in evidence-based clinical practice guidelines.Methods and FindingsA cross-sectional analysis of cardiovascular risk management recommendations was performed for three different conditions (diabetes mellitus, dyslipidemia, and hypertension) from three pan-national guideline panels (from the United States, Canada, and Europe). Of the 338 treatment recommendations in these nine guidelines, 231 (68%) cited RCT evidence but only 105 (45%) of these RCT-based recommendations were based on high-quality evidence. RCT-based evidence was downgraded most often because of reservations about the applicability of the RCT to the populations specified in the guideline recommendation (64/126 cases, 51%) or because the RCT reported surrogate outcomes (59/126 cases, 47%).ConclusionsThe results of internally valid RCTs may not be applicable to the populations, interventions, or outcomes specified in a guideline recommendation and therefore should not always be assumed to provide high-quality evidence for therapy recommendations.
Partial Text: There has been a rapid expansion in the number of clinical practice guidelines over the past decade and, as a result, clinicians are frequently faced with several guidelines for treatment of the same condition. Unfortunately, recommendations may differ between guidelines [1,2], leaving the clinician with a decision to make about which guideline to follow. While it is easy to say that one should follow only those guidelines that are “evidence based,” very few guideline developers declare their documents to be non–evidence based, and there is ambiguity about what “evidence based” really means in the context of guidelines. The term may be interpreted differently depending on who is referring to the guideline—the developer, who creates the guidelines, or the clinician, who uses them. To their developers, “evidence-based guidelines” are defined as those that incorporate a systematic search for evidence, explicitly evaluate the quality of that evidence, and then espouse recommendations based on the best available evidence, even when that evidence is not high quality . However, to clinicians, “evidence based” is frequently misinterpreted as meaning that the recommendations are based solely on high-quality evidence (i.e., randomized clinical trials [RCTs]) . Previous studies of guidelines have focused almost exclusively on the elements embodied in the first definition of an evidence-based guideline. For example, guideline appraisal tools assess the methodology used in developing the guideline and the clarity with which recommendations and the type of underlying evidence are communicated in that guideline [5,6].
In summary, we found that while two-thirds of cardiovascular risk management therapy recommendations made in the nine different guidelines we examined were based on RCT evidence, less than half of these RCT-based recommendations were deemed “high quality” using an evidence-grading scheme that went beyond considerations of internal validity alone to take into account clinical relevance and direct applicability of the RCT to that recommendation. As a result, less than one-third of recommendations that advocated specific cardiovascular risk management therapies in these evidence-based guidelines were actually based on high-quality evidence.