Explaining Variable Outcomes from Implementation Trials: Studying Implementation Processes through Formative and Process Evaluation

Brown AH, Alkema GE, Yano EM. Explaining Variable Outcomes from Implementation Trials: Studying Implementation Processes through Formative and Process Evaluation. Presented at: VA HSR&D National Meeting; 2009 Feb 13; Baltimore, MD.

Related HSR&D Project(s)

• RCS 05-195 – Research Career Scientist Award

Workshop Objectives: Multi-site quality improvement and implementation trials typically result in variable impacts due to the strong influences of contextual factors and variations in the manner in which interventions are deployed and evolve. Understanding these variable impacts is critical for interpretation of trial results and preparation for further spread of interventions. Guidance and tools for conducting in-depth formative and process evaluations are needed to support implementation researchers' efforts to develop useful insights into implementation processes and the variable effectiveness of implementation programs. This workshop will present methods for planning, conducting, and evaluating process evaluations, including conceptual frameworks for designing implementation research process evaluations; descriptions of the range of qualitative methods suitable for process evaluations, including an overview of organizational change measures and data analysis issues related to these evaluations; and in-depth, grounded examples of mixed-methods process evaluations conducted within VA. Activities: Dr. Alkema will provide detailed guidance on the development and utilization of logic models as a conceptual foundation for these evaluations. Dr. Brown will provide an overview of qualitative methods that are well-suited for process evaluations; and will review available, valid organizational change measures and discuss the strengths and weaknesses of these measures. Examples from an implementation study in specialty mental health clinics across 4 VISNs will be utilized. Dr. Yano will describe the theoretical framework and mixed methods employed in process evaluations conducted as part of a large program of research evaluating implementation programs for improving depression in primary care. Drs. Brown and Yano will also discuss data analysis (particularly qualitative software-supported analysis) issues related to mixed methods studies. The audience will be invited to add their implementation research experiences to the discussion to maximize the potential for learning from multiple examples. Discussion will be co-facilitated by the workshop leaders. Target Audience: Researchers interested in implementation processes, process evaluation, multisite implementation research, organizational change, and qualitative methods. This is an advanced workshop in the CIPRS-supported "Implementation Research Methods Workshop Series." Assumed Audience Familiarity with Topic: It is assumed that the audience will be familiar with implementation research and concepts of organizational change.