BACKGROUND/RATIONALE:
Collaboratives and performance feedback have both been widely adopted within the VA as quality improvement strategies at the regional and national levels. Notably, neither strategy prescribes the means by which VA facilities should reach the goal of improved quality of care for veterans, as neither mandates prepackaged, standardized "solutions." Rather, the two strategies identify opportunities for improvement and then rely on individuals, teams and clinical microsystems to figure out how best to improve clinical performance within VHA.

Relatively little, however, is known about how individuals, teams, and clinical microsystems interact with one another to develop and implement these local solutions to improve patient care. Collaboratives provide formal structures to support the local development, implementation and assessment of solutions through the use of cross-functional teams, coaches, learning sessions, peer networks, and a shared improvement methodology over an extended period of time. Yet the impact of collaboratives on local change efforts is not well understood because collaboratives have rarely been rigorously evaluated with randomized-controlled trial (RCT) designs.

Classic RCT designs, though, present challenges of their own. While RCTs provide strong evidence of intervention effectiveness, they typically leave unexplained how and why an intervention succeeded or failed, a problem often referred to as "getting inside the black box." This particularly applies to RCT interventions with psychological, behavioral, educational, and/or social dimensions, where context can play an important mediating role in the way an intervention unfolds.

Understanding how and why an intervention works in context only increases in complexity when the target of an intervention is not an individual but instead a group of people working together as a team in a particular setting to improve care for Veterans. The issue gets further complicated with the use of cross-functional, multidisciplinary teams, where members bring different expertise, experiences, priorities, and social and peer networks to an improvement project. In determining how best to improve health care processes and outcomes for patients, diverse team members simultaneously face the challenge of how to integrate their different kinds of knowledge.

Albeit complex, the question of how individuals, teams, and clinical microsystems interact with one another to come up with local solutions to improving care is a crucial issue for the VA to address, given the VA's investment in team-based initiatives.

The RE-INSPIRE (Rich-context Evaluation of INSPIRE) study determined how and why local quality improvement initiatives within VA differ so widely in terms of success, spread, and sustainability by conducting a rigorous evaluation of the impact of context.

OBJECTIVE(S):
The RE-INSPIRE study has four specific research aims:

1. To evaluate the impact of context on the implementation mechanisms used at the sites randomly assigned to participate in a Systems Redesign-based intervention (including a Collaborative) in the INSPIRE randomized-controlled trial;

2. To evaluate the impact of context on the effectiveness of local initiatives to improve stroke care across all 11 intervention and control sites in the INSPIRE study;

3. To evaluate the impact of context on the spread and sustainability of local initiatives to improve stroke care over time across all 11 intervention and control sites in the INSPIRE study;

4. To pilot and disseminate a "context matrix" approach for integrating data from multi-site initiatives in a single cumulative project file that can be queried and analyzed in ways that support rapid-cycle reporting and real-time access to data.

METHODS:
RE-INSPIRE was a prospective, longitudinal, and mixed-methods study of context embedded within an existing randomized-controlled trial. The Consolidated Framework for Implementation Research served as the study's conceptual framework.

RE-INSPIRE collected context-related data through annual site visits at all 11 VA facilities, semi-structured phone interviews, and ongoing document/artifact capture.

Using this data, the RE-INSPIRE project team systematically quantified over 20 CFIR constructs for each site visit. Individual members of the project team rated interview transcripts for construct and valence (i.e., positive, neutral or negative). When all the interviews in a site visit had been rated, the entire team met to review these ratings and then score each VA medical center for each site visit with a facility-level valence (i.e., positive, neutral, negative, polarized, or minimally referenced) and magnitude (i.e., weak or strong). Each of the 11 VA medical centers participating in the RE-INSPIRE SDP had three site visits spaced a year apart, yielding a total of 33 site visits; by the time the scoring process is complete, the project team had assigned valence and (when appropriate) magnitude to CFIR constructs over 700 times.

Context-related data collected as part of the RE-INSPIRE study were imported and integrated on an ongoing basis in a single, cumulative project file using NVivo10 software. RE-INSPIRE investigators applied interpretive codes to qualitative data, analyzed coded material using case-ordered display matrices, and wrote analytic memos to document the evolution of the analyses, the emergence of early patterns and possible findings, the convergence of major themes, the challenges posed by the analysis, and ideas related to hypothesis testing. At the same time, quantitative and categorical descriptors were applied to the multi-level data as appropriate within the cumulative project file. Using the matrix coding query feature in NVivo10, qualitative, quantitative, categorical and CFIR codes were systematically, iteratively, and continuously mapped against one another at the individual, team and clinical microsystem levels to investigate patterns, associations, and relationships related to implementation mechanisms, implementation contexts, effective and ineffective improvement practices, spread, and sustainability.

Qualitative Comparative Analysis (QCA), a numerical method that uses set theory and Boolean algebra, was used to pinpoint specific elements of local context in RE-INSPIRE that mattered most to implementation outcomes.

The RE-INSPIRE study developed and refined a new measure of "group organization" level called the "GO Score" to reflect the high degree of observed variation in how VA staff organized themselves to provide or improve acute stroke care at their facility. Each score corresponded to a 10-point continuum ranging from 1 to 10 that spanned beginning, basic, intermediate and advanced group organization levels. A total of 66 GO Scores were assigned in the RE-INSPIRE study.

Throughout this process, RE-INSPIRE investigators used triangulation to support conclusions with multiple sources of evidence from the data.

FINDINGS/RESULTS:
Findings from the RE-INSPIRE study included new insights into how and why local improvement initiatives within VA differ so widely in terms of success, spread, and sustainability. RE-INSPIRE findings may help improvement initiatives within VA to reach more of their full potential by generating new scientific knowledge about the impact of local context on implementation.

Results included the following:

1. Wide variation was observed in how groups of VA staff organized themselves at different sites (1) to provide and (2) to improve acute stroke care. These differences translated conceptually into "program levels" that can be mapped along a continuum. At some sites, observable movement from one program level to another occurred during the course of the RE-INSPIRE project.

2. Providers engaging collectively in Reflecting & Evaluating was a key driver of implementation success in RE-INSPIRE. The influence of Reflecting & Evaluating on implementation success was roughly analogous to a dose-response curve.

3. Combinations of only four specific elements of context (Reflecting & Evaluating, Structural Characteristics, Relative Priority, and Champions) directly connected to the level (advanced, middle, low) at which local VA staff organized themselves to provide acute stroke care at their facility. Implementation success emerged directly from these conditions.

4. Longitudinal, positive change in the level at which local VA staff organized themselves to provide acute stroke care at their facility was directly connected to longitudinal, positive change in only three specific elements of context (Reflecting & Evaluating, Structural Characteristics, and Planning). In this gain analysis, each performance site served as its own control, which was made possible by the longitudinal design of the VA RE-INSPIRE SDP study.

5. Combinations of only four specific elements of context (Reflecting & Evaluating, Cosmopolitanism, Relative Priority, and Networks) directly connected to the level (advanced, middle, low) at which local VA staff organized themselves to improve acute stroke care at their facility. Two of these context-related conditions (Cosmopolitanism and Networks) were different from those conditions directly connected to level of providing acute stroke care (see Finding #3 above).

6. Longitudinal, positive change in the level at which local VA staff organized themselves to improve acute stroke care at their facility was directly connected to longitudinal, positive change in only three specific elements of context (Planning, Self-Efficacy, Additional Resources) plus a consistently strong Learning Climate over time.

7. In both the cross-sectional and longitudinal findings reported above, crisp-set Qualitative Comparative Analysis solutions accounted for all cases.

8. Participants generally did not communicate with staff at other VA medical centers, including VAMCs in their own VISN, about how acute stroke care was organized at different VA facilities.

9. In learning how groups of people at a local VA medical center moved from one way of organizing acute stroke care to another over time, this study discovered the role of "shared nodes." A shared node is a specific, tangible organizational mechanism that allows a group of people in a program to interact with one another in new ways that are dynamic, collaborative, information-rich, and productive, and ultimately binds them closer together as a group. A telltale sign of a shared node is that it provides a group with new capacity to capture, see, process, and act upon information that was largely invisible in the absence of that shared node. One example of a shared node, for instance, is an alphanumeric stroke pager worn by many staff across several different service lines at a local VA facility that goes off whenever a new patient with a suspected acute stroke presents to the ED or is admitted to the ICU, accompanied by a standardized set of data about that patient. Only a small number of staff may be on duty or on call at any given time, but every individual staff member wearing the stroke pager was now aware of the patient, has the opportunity to get involved in that patient's care if they wish, and can follow up on that patient as deemed appropriate. The "shared node" of the alphanumeric stroke pager allows a group of clinicians to perceive new things, to interact with each other in new ways, and to respond in new ways. Other examples of shared nodes include daily improvement huddles anchored visually by a huddle board; monthly improvement meetings where representatives from multiple service lines meet to review site-specific performance data in order to identify opportunities to improve care; and the ongoing monitoring of near-real time data by staff to ensure that all inpatients receive appropriate care and that all care is fully documented in the medical charts. The gradual aggregation of shared nodes over time had a cumulative effect in allowing a group of people in a program to move from one way of organizing acute stroke care to another. Certain shared nodes were more useful to a group at a more basic level of organizing acute stroke care, while others were more pertinent to groups at more advanced levels; groups largely discovered shared nodes on their own, with little to no sharing across VA facilities; new shared nodes connected with and built upon existing shared nodes at a local facility; and most shared nodes related to acute stroke care appear to be generalizable in terms of potential implementation across diverse VA settings.

IMPACT:
The quantification of CFIR constructs provides an important new analytic strategy for understanding how interventions, implementation strategies, and local contexts influence implementation success, and represents a major contribution to the VA and implementation science. PI Miech gave accepted oral presentations on this method at the 2015 HSR&D/QUERI National Meeting in Philadelphia and the 2015 Society for Implementation Research Collaboration (SIRC) Biennial Meeting in Seattle and is currently developing a manuscript on this approach.

The GO Score provides a new, standardized instrument to measure cross-site variation and within-site change over time in how local staff organize themselves to provide and improve care, a contextual element that is often important in implementation research but difficult to capture. This instrument was developed with generalizability in mind and can be applied to other clinical areas and contexts both inside and outside the VA. PI Miech gave an accepted oral presentation on the GO Score at the 2015 NIH/AcademyHealth Science of Dissemination & Implementation Conference in DC and is currently developing a manuscript about this measure.

Qualitative Comparative Analysis continues to grow as an approach within the VA research and implementation science communities. QCA is well-suited for studies of complex implementation processes and/or rigorous observational evaluations of VA initiatives and programs. PI Miech gave an accepted oral presentation on the GO Score at the 2015 NIH/AcademyHealth Science of Dissemination & Implementation Conference in DC and is currently developing a manuscript on the approach. PI Miech also leads the national VA Special Interest Group on Qualitative Comparative Analysis, co-presented a national HSR&D CyberSeminar on the use of QCA in VA research in July 2015, and will co-present a national HSR&D CyberSeminar on QCA again in January 2016.

Getting to "Reflecting & Evaluating" appears to be key for any VAMC trying to develop its facility-wide acute stroke care program. The new Precision Monitoring (PRIS-M) QUERI has been directly influenced by this finding and adopted a program-wide focus on assessing Reflecting & Evaluating in all of its projects.

The RE-INSPIRE study provides the VA with a new data integration strategy that can directly benefit both research and operations. This approach allows VA staff to integrate multiple types of data from multiple sources and sites into a single cumulative project file. This unified project file can then be explored, queried, and analyzed in interactive and flexible ways that support rapid-cycle reporting and real-time access to data. This method provides the VA with an option that goes beyond SharePoint: instead of an endpoint in which numerous documents and files get posted online in a shared folder, the VA has the ability to import and integrate disparate files in a single project file which VA staff can systematically query and mine in order to convert data into useful, actionable information.

---

External Links for this Project

NIH Reporter

Grant Number: I01HX000612-01
Link: https://reporter.nih.gov/project-details/8141743

---

PUBLICATIONS:

Journal Articles

1. Matthias MS, Chumbler NR, Bravata DM, Yaggi HK, Ferguson J, Austin C, McClain V, Dallas MI, Couch CD, Burrus N, Miech EJ. Challenges and motivating factors related to positive airway pressure therapy for post-TIA and stroke patients. Behavioral sleep medicine. 2014 Mar 4; 12(2):143-57. [view]
2. Rattray NA, Damush TM, Luckhurst C, Bauer-Martinez CJ, Homoya BJ, Miech EJ. Prime movers: Advanced practice professionals in the role of stroke coordinator. Journal of the American Association of Nurse Practitioners. 2017 Jul 1; 29(7):392-402. [view]

VA Cyberseminars

1. Damschroder L, Miech EJ. Qualitative Comparative Analysis and Implementation Research: An Introduction. QUERI Implementation Network [Cyberseminar]. QUERI. 2016 Jan 7. [view]

Conference Presentations

1. Miech EJ, Damush TM. Applying the CFIR Constructs Directly to Qualitative Data: The Power of Implementation Science in Action. Paper presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 9; Philadelphia, PA. [view]
2. Rattray N, Damush TM, Luckhurst C, Bauer-Martinez CJ, Miech EJ. Boundary Spanners: The Unique Role of the Nurse Practitioner in Developing Acute Stroke Care Programs within the VA. Poster session presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 9; Philadelphia, PA. [view]
3. Damush TM, Bauer-Martinez C, Rattray N, Williams LS, Miech EJ. Perceptions of a Stroke QI Collaborative on Acute Stroke Care within VA. Paper presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 9; Philadelphia, PA. [view]
4. Miech EJ. Pinpointing the Specific Elements of Local Context that Matter Most to Implementation Outcomes: Findings from Qualitative Comparative Analysis in the RE-INSPIRE Study of VA Acute Stroke Care. Paper presented at: National Institutes of Health / AcademyHealth Conference on the Science of Dissemination and Implementation; 2015 Dec 14; Washington, DC. [view]
5. Miech EJ, Damush TM. The Consolidated Framework for Implementation Research: Applying the CFIR Constructs Directly to Qualitative Data. Paper presented at: Society for Implementation Research Collaboration Biennial Conference on Advancing Efficient Methodologies Through Community Partnerships and Team Science; 2015 Sep 25; Seattle, WA. [view]
6. Miech EJ. The GO Score: A new context-sensitive instrument to measure group organization level for providing and improving care. Paper presented at: National Institutes of Health / AcademyHealth Conference on the Science of Dissemination and Implementation; 2015 Dec 14; Washington, DC. [view]

DRA: Cardiovascular Disease
DRE: none
Keywords: none
MeSH Terms: none