Search | Search by Center | Search by Source | Keywords in Title
Olson MB, McCreedy EM, Baier RR, Shield RR, Zediker EE, Uth R, Thomas KS, Mor V, Gutman R, Rudolph JL. Measuring implementation fidelity in a cluster-randomized pragmatic trial: development and use of a quantitative multi-component approach. Trials. 2022 Jan 15; 23(1):43.
BACKGROUND: In pragmatic trials, on-site partners, rather than researchers, lead intervention delivery, which may result in implementation variation. There is a need to quantitatively measure this variation. Applying the Framework for Implementation Fidelity (FIF), we develop an approach for measuring variability in site-level implementation fidelity. This approach is then applied to measure site-level fidelity in a cluster-randomized pragmatic trial of Music and Memory (MandM), a personalized music intervention targeting agitated behaviors in residents living with dementia, in US nursing homes (NHs). METHODS: Intervention NHs (N = 27) implemented MandM using a standardized manual, utilizing provided staff trainings and iPods for participating residents. Quantitative implementation data, including iPod metadata (i.e., song title, duration, number of plays), were collected during baseline, 4-month, and 8-month site visits. Three researchers developed four FIF adherence dimension scores. For Details of Content, we independently reviewed the implementation manual and reached consensus on six core MandM components. Coverage was the total number of residents exposed to the music at each NH. Frequency was the percent of participating residents in each NH exposed to MandM at least weekly. Duration was the median minutes of music received per resident day exposed. Data elements were scaled and summed to generate dimension-level NH scores, which were then summed to create a Composite adherence score. NHs were grouped by tercile (low-, medium-, high-fidelity). RESULTS: The 27 NHs differed in size, resident composition, and publicly reported quality rating. The Composite score demonstrated significant variation across NHs, ranging from 4.0 to 12.0 [8.0, standard deviation (SD) 2.1]. Scaled dimension scores were significantly correlated with the Composite score. However, dimension scores were not highly correlated with each other; for example, the correlation of the Details of Content score with Coverage was t = 0.11 (p = 0.59) and with Duration was t = -?0.05 (p = 0.78). The Composite score correlated with CMS quality star rating and presence of an Alzheimer''s unit, suggesting face validity. CONCLUSIONS: Guided by the FIF, we developed and used an approach to quantitatively measure overall site-level fidelity in a multi-site pragmatic trial. Future pragmatic trials, particularly in the long-term care environment, may benefit from this approach. TRIAL REGISTRATION: Clinicaltrials.gov NCT03821844. Registered on 30 January 2019, https://clinicaltrials.gov/ct2/show/NCT03821844 .