Research Article: When Things Are Not as They Appear: Assessing the Adequacy of Cluster Randomization When Outcome Events Are Rare at Baseline

Date Published: May 8, 2012

Publisher: Hindawi Publishing Corporation

Author(s): Veronica Dinaj-Koci, Nanika Brathwaite, Lynette Deveaux, Sonya Lunn, Lesley Cottrell, Carole Harris, Bonita Stanton, Xiaoming Li, Sharon Marshall, Perry Gomez, Xinguang Chen.


The present study randomly assigned 15 Bahamian elementary schools to one of three intervention conditions. To assess the adequacy of cluster randomization, we examined two concerns identified by the local research team: inequality of gender distribution and environmental risk among groups. Baseline significant differences in risk and protective behaviors were minimal. There were significantly more males in the intervention group. Males had higher rates of risk behavior at all assessments. Poor school performance was also higher among the intervention condition and was significantly associated with increased rates of many but not all risk behaviors. Prior to adjusting for gender and school performance, several risk behaviors appeared to be higher after intervention among intervention youth. Adjusting for gender and school performance eradicated the group differences in risk behavior rates. Results demonstrate the importance of adequate randomization where outcomes of interest are rare events at baseline or differ by gender and there is an unequal gender distribution and the importance of the local research team’s knowledge of potential inequalities in environmental risk (i.e., school performance). Not considering such individual differences could impact the integrity of trial outcomes.

Partial Text

As shown in Table 2, at baseline both intention to use a condom if having sex (a protective behavior) and intention to engage in sex in the next six months (a risk behavior) were significantly higher among FOYC+CImPACT youth compared to controls among the total sample and among the subsample of males. No significant differences in other sexual risk or protective perceptions, skills, or knowledge at baseline. Not shown in this table, GLAT scores were lower among FOYC+CImPACT schools (average 18.2 with three of the five schools having significantly lower mean scores of 11.2, 9.7 and 11.3 compared to the overall mean) compared to WW+GFI schools (average 24.2, with only one school having a significantly lower mean of 15.2 and one school a significantly higher mean of 31.7) (data available upon request).

The importance of using evidence-based approaches for disease prevention is well recognized; likewise, there is recognition of the need to reassess effectiveness of such interventions if they are significantly altered and/or applied in significantly different populations from that in which they were originally assessed [1]. School systems are often the site for “real-life” implementation of prevention programs targeting a broad audience [5, 10]. For a variety of logistic and methodological reasons, randomization will frequently be conducted at the level of the school and, as such, there may be relatively few units of randomization [8]. Because school populations are frequently geographic based, confounding due to possible homogeneity of underlying risk factors for the outcomes of interest may be problematic. If the outcomes and/or risk factors are relatively prevalent at baseline, adequacy of the randomization process could be assessed and/or baseline differences controlled for in the assessment of intervention impact. However, when the prevalence of outcomes of interest and/or known risk factors are low, baseline differences, which may impact outcomes, could be undetected. In such a case, utilization of available proxy measures would be of great importance in interpreting the results of such analyses. Likewise, if gender distribution is unequal across intervention groups and the outcomes of interest are of low frequency at baseline before the intervention, it may be difficult to disentangle the effects of gender and intervention assignment subsequently. Analysis by gender may be necessary to reveal actual (versus apparent) intervention impacts. In this study we show that both unequal gender distribution and inequality in baseline risk accounted for appearance of higher rates of sexual initiation and violence in the intervention group. By contrast, the higher rates of condom use, knowledge, and efficacy beliefs among FOYC+CImPACT youth appear to be true intervention effects.