Can Six Blind Men Find Apples & Oranges: Using Multiple Data Sources to Describe Variable Implementation of QI Interventions

Yano EM, Young AS, Cohen AN, Hamilton A, Rubenstein LV. Can Six Blind Men Find Apples & Oranges: Using Multiple Data Sources to Describe Variable Implementation of QI Interventions. Paper presented at: VA QUERI National Meeting; 2008 Dec 8; Phoenix, AZ.

Related HSR&D Project(s)

• RCS 05-195 – Research Career Scientist Award
• MNT 03-213 – Implementing Effective, Collaborative Care for Schizophrenia
• MNT 03-215 – Expanding and Testing VA Collaborative Care Models for Depression

Objectives: Implementation of quality improvement (QI) interventions requires a balance between retaining core elements of evidence-based practice (i.e., what we know about the key ingredients of an intervention or care model that "works"), while adapting to local contexts in ways that foster their adoption and implementation. The objectives of this workshop are to: 1) Describe ways in which QI interventions set us up for variable deployment; 2) Present evidence-based QI (EBQI) methods for intentional adaptation of evidence into the context of local practices; 3) Describe types of data sources and methods that may be brought to bear on measuring variable implementation of QI interventions, including derivation of fidelity scores; and, 4) Discuss triangulation of different data sources to help "tell the story" of variable implementation in ways that foster dissemination, publication and ongoing QI. Activities: Working group leaders will describe their experiences with multi-site QI intervention studies for different types of evidence-based practices using examples from the Mental Health and Substance Use Disorders QUERIs. Examples will include evaluations of the implementation of depression collaborative care in primary care (TIDES), and evidence-based practices for schizophrenia in specialty mental health (EQUIP-2) and smoking cessation in both primary care and mental health settings (QUITS). Methods for measuring variable implementation will include semi-structured interviews of clinical and administrative leaders, organizational structure site surveys or practice checklists, administrative data (e.g., workload changes), patient surveys (e.g., their recall of intervention exposures), and provider surveys (e.g., their recall of intervention activities, components observed). Methods for developing a fidelity score from qualitative interview data will also be presented. Working group participants will be engaged in describing experiences from other QUERIs, creating an initial catalog and classification of tools, followed by moderated discussion of methods for triangulating these different data sources and considering their application across different types of interventions in different settings. Integration of cross-QUERI approaches would yield content for a new Web-based resource. Target Audience: Implementers and evaluators interested in better understanding methods for documenting and analyzing variable implementation of QI interventions. Assumed Audience Familiarity with Topic: Participants with broad familiarity with QI interventions and implementation issues; in-depth methodological skills or experience will not be required.