Date Published: March 3, 2009
Publisher: Public Library of Science
Author(s): Gurdeep S Sagoo, Julian Little, Julian P. T Higgins
Abstract: Gurdeep S. Sagoo and colleagues describe key components of the methodology for undertaking systematic reviews and meta-analyses of genetic association studies.
Partial Text: The past decade has witnessed growing interest in genetic predisposition to common diseases, and along with rapid advancements in high-throughput genotyping technology, has resulted in a tremendous amount of published epidemiological evidence on gene-disease associations. Reported genetic associations with common diseases have become numerous and are mostly of small magnitude . With this growth in evidence has come an increasing need to collate and summarize the evidence in order to identify true genetic associations among the large volume of false positives . Convincing evidence of true association therefore requires careful control over potential biases and chance effects. Control over bias is important both in study design  and in considering the selective availability of data on associations that have been examined . Because most genetic associations are small, large sample sizes are necessary for their detection, especially when many genetic variants are investigated simultaneously, as in genome-wide association studies. Furthermore, replication of findings in independent data sets is now widely regarded as a prerequisite for convincing evidence of association. Thus, multiple studies from several independent groups exist.
Traditional (narrative) reviews are subjective, and as such have a number of disadvantages that lead them to being more prone to bias and error . Narrative reviews rarely state how studies were selected for inclusion, how (or whether) their quality was assessed, or how the findings from multiple studies were synthesized in order to draw conclusions. This can lead to a review that supports and reinforces the author’s view, which can be misleading. In contrast, systematic reviews are designed as rigorous research studies. They allow a more objective appraisal of the evidence than a narrative review by aiming to identify, critique, and synthesize evidence from all relevant existing studies on the topic in question using predefined methods. Figure 1 outlines the general processes involved in a systematic review.
Like other research studies, systematic reviews should ideally be carefully planned with a detailed protocol prepared in advance. Producing a protocol with criteria predefined for study selection will minimize any selection bias based on study results. Such a protocol should therefore clearly formulate the review question, explain the rationale for conducting such a review, define a priori eligibility criteria for study inclusion, describe the methods for conducting a comprehensive search for studies, indicate the methods for assessing study validity and relevance, and state whether a meta-analysis is planned and describe the methodology to be used for conducting such an analysis.
The objectives of a systematic review of genetic association studies will typically be (i) to identify all epidemiological investigations of the associations of interest; (ii) to assess the validity of the evidence; (iii) to determine whether an association exists; (iv) to assess whether associations are consistent across studies in magnitude and direction; and (v) to quantify the likely magnitude of an association if it exists. Associations of interest have in the past usually been identified through biological plausibility, although candidate associations are now frequently identified from genome-wide association studies.
A key characteristic of a systematic review is a comprehensive search. Limiting a review to studies identified only from MEDLINE (or PubMed) is usually insufficient, for two reasons. First, any review restricted to published literature is prone to publication bias, whereby only a subset comprising the most “publishable” findings are available from the totality of evidence on a particular genetic association. Thus, sources beyond the published literature should be examined. Second, even within the published literature, reporting bias may be present whereby studies with different conclusions are published in different types of journals. MEDLINE is just one of several major sources of bibliographic information. These biases may, to some extent, be reduced by searching comprehensively for studies available through other sources. Other bibliographic databases likely to be useful for genetic association studies include EMBASE and the Science Citation Index. Overlap between these various databases is far from complete; for example, of approximately 4,800 journals indexed in EMBASE, 1,700 are not indexed in MEDLINE (see Chapter 6 in ). The implications of this incomplete overlap for genetic association studies are not clear. Furthermore, bibliographic searches may still not retrieve all articles that are in the indexed journals .
All reasonable attempts should be made to prevent the introduction of errors and personal biases. An important attribute of systematic reviews is that criteria for study inclusion are clearly prespecified. This is perhaps the most notable difference between a narrative review and a systematic review. Independent duplication of steps in the review process, such as selection of studies, extraction of data, and critical assessment of methods used in the individual studies, can further reduce biases and minimize errors. Accidental omission of data, or accidental duplication of a study in a meta-analysis, may lead to spurious false-negative or false-positive findings.
The validity of a meta-analysis depends on the validity of the studies included within it, so it is important that each component study is appraised before being included. Because effect magnitudes in genetic association studies are generally small (i.e., odds ratio < 1.2), even small biases may be important. The most important sources of bias in genetic association studies are less well understood than those in other study designs such as randomized trials. Extensive discussions of potential biases are available, with the principal candidates being case definition, population stratification (confounding due to sub-populations in the sample that differ both in genotype prevalence and disease risk), and methods in the collection, handling, and processing of DNA and the determination of genotypes (including blinding to case-control status) [3,14]. Little empirical evidence associating study results with study characteristics for genetic association studies exists as yet, however. This evidence will perhaps be most reliably derived from meta-epidemiological studies, in which results of studies with different characteristics are compared within meta-analyses, and findings are synthesized across meta-analyses to enhance power [15,16]. In addition to potential biases in the individual studies, attention should be paid to bias in the collection of studies as a whole. Two particular considerations are reporting biases and bias in the selection of genetic variants to study. Meta-analysis is the statistical synthesis of results from multiple studies : an example is provided in Figure 3 . When implemented and interpreted appropriately, and applied to unbiased and correctly analyzed studies, it provides a powerful tool to understand both similarities and differences in results from multiple studies. By exploiting the totality of evidence, meta-analyses offer enhanced power to detect associations and increased precision in the estimation of their magnitude. Meta-analyses are encouraged in systematic reviews. However, attention should always be paid to the possibility of reporting biases in reviews based on published literature. Furthermore, it is important that each entry in a meta-analysis represents an independent sample of data. Thus for example, multiple reports of the same study need to be merged to obtain a single “best” answer for that study prior to inclusion in the meta-analysis. A systematic summary of all available evidence should allow the strengths and gaps in the evidence base to be identified and allow recommendations to be made in order to stimulate research to address such gaps. The magnitude of the association between the allelic variants and the clinical outcomes studied in terms of relative and attributable risks in different populations should be summarized in a systematic and concise way. Comments should also be provided on the quality and methodology of studies. Tables should summarize information on each gene-disease association study (possibly online as Web-based supplements, depending on size and journal formats). Table 2 lists some of the characteristics most likely to be relevant. If a meta-analysis is conducted, these should usually be presented as forest plots as in Figure 3. Statistics related to heterogeneity (e.g., between-study variance or I2 and confidence intervals) and investigations of bias should be presented. Multiple subgroup and sensitivity analyses are conveniently presented as “overview” forest plots without including the individual studies. The strengths and limitations of systematic reviews are well established for clinical trials, largely through the efforts of The Cochrane Collaboration . They are increasingly being applied to observational studies, and currently there are as many meta-analyses of observational data conducted as there are of clinical trials. The citation impact of both types of meta-analyses is equally high, the highest among all study designs in the health sciences . The principal value of a well-conducted systematic review of genetic association studies is in establishing reliably the presence and magnitude of individual gene-disease associations. By complementing both consortium-based pooled analyses and larger-scale attempts to collate genetic association evidence across whole fields, they play an important role alongside other research designs in the integration of evidence on genetic association . At their most ambitious, systematic reviews and meta-analyses can collate evidence across all studied genetic variants for a phenotype, with notable examples provided by three databases of genetic association evidence for Alzheimer disease, Parkinson disease, and schizophrenia that include ad hoc meta-analyses [38–40]. Source: http://doi.org/10.1371/journal.pmed.1000028