Search | Search by Center | Search by Source | Keywords in Title
Lee ML, Sherman SE, Yano EM. Lack of clustering in a group randomized trial of smoking cessation guideline implementation (QUITS). Paper presented at: VA HSR&D National Meeting; 2001 Feb 15; Washington, DC.
Objectives: Smoking cessation has become an important focus within VHA, with a range of mandates, performance criteria, research funding initiatives and other activities designed to decrease smoking prevalence among VA patients. We have initiated a randomized intervention study, the Quality Improvement Trial for Smoking Cessation (QUITS), to evaluate whether evidence-based quality improvement strategies aimed at the institutional level lead to improved adherence to smoking cessation guidelines, and, in turn, to increased quit rates among smokers. A natural issue to consider in any experimental design where study sites are randomized, but assessment is made at the subject level, is whether a clustering effect on outcome variables is introduced. However, the rarity of published information on intraclass correlation coefficients (ICC's) for use in randomized trials has made appropriate cluster adjustment for organizational level and provider-level interventions problematic at best. In addition, the potential effect of clustering on the estimation of process vs. outcome differences may lead to alternative findings. After approximately 1,100 subjects had been enrolled in the QUITS trial, we decided to evaluate the appropriateness of clustering assumptions used to generate sample size estimates for experimental and control groups that may in turn be used to inform future VA randomized trials. Methods: Based on a group randomized design, we adjusted initial sample size calculations based on crude approximations of the ICC's for a 'reasonable amount of clustering' per study site with respect to the probability of smoking cessation. We used ICC estimates from a broad array of organizational 'best practice' interventions available from the Cochrane Center on Professional Practice. Using dat on the first 1,102 smokers from the initial 7 study sites, we calculated the proportion of individuals who had not smoked on a daily basis for at least the past 30 days up to 12 months for each site. This proportion was used as a baseline definition of smoking cessation. We also evaluated a key process of care variable, namely the extent to which physicians had spoken to the patients about quitting smoking in the past 12 months. The usual method of computing the ICC for dichotomous data was employed. Results: Based on preliminary QUITS trial data, we calculated an actual ICC of -0.002, which, although negative, was not significantly different from zero. A negative ICC is theoretically conceivable when the variation between sites is actually smaller than the variation within sites. It indicates a lack of clustering effect on this outcome variable induced by the use of a group randomized design and is in sharp contrast with initial assumptions about the nature of across-site and within-site variation in guideline implementation studies. On the other hand, the ICC for physician counseling on smoking cessation was 0.003, which, although positive, was not statistically significant either. Conclusions: Failure to adjust for cluster effects in randomized trials can result in underestimates of standard errors and erroneous statistical significance if standard analytic methods are employed. In this study of smoking cessation guideline implementation, it appears that such an effect on study outcomes has not been found. This may reflect a potential differential effect among care providers within sites that might increase intracluster variability. The influence of the design on the process variables also may not be substantial, although we would expect greater influence of study site on providers within sites compared to across sites. Impact: Health services researchers should publish explicit information about the extent of clustering in their provider- or practice-based interventions to facilitate appropriate design and sufficient sampling in randomized trials. Early reassessment of clustering assumptions permits investigators to adapt sample requirements to facilitate efficient recruitment and support appropriate statistical analysis.