HSR&D Home » Research » IIR 02-097 – HSR&D Study
Meta-Analysis of the Efficacy of Computer-based Feedback Implementations
Charlene Raye Weir, PhD RN
VA Salt Lake City Health Care System, Salt Lake City, UT
Salt Lake City, UT
Funding Period: January 2004 - December 2006
Many recent reviews of the impact of Information Technology in healthcare have found the literature lacks sufficient specification of contextual and moderating variables to produce generalizable results. The purpose of this study was to conduct a systematic review of the literature of computerized interventions exploring two classes of theoretical variables to the pattern: 1) the intensity of social influence and; 2) the degree of task uncertainty.
I. Determine the overall averaged effect of computerized interventions on: a) patient outcomes, b) provider adoption and c) economic outcomes.
II. Determine the evidence that characteristics of social influence and/or task uncertainty directly or in interaction moderate the relationship between computerized interventions and: a) patient outcomes, b) provider adoption and c) economic outcomes.
Systematic standard procedures were used to develop a search strategy and relevance criteria. Multiple databases (Medline, CINAHL, Cochrane, PsychInfo, DARE, INSPEC, and EMBASE) were searched from 1976 through the end of 2005. The search criteria included all forms of computer*, alerts, reminders, decision-support, outcome and evaluation studies as well as English-only. Articles were judged relevant if they reported an intervention that occurred in a real clinical environment targeted at clinicians. References for articles published after 1990 were reviewed and authors contacted. The topic areas of Computerized Provider Order Entry, Lifestyle Counseling and Infectious Disease prevention and treatment were individually addressed. Research quality was assessed using an instrument developed for this study (QUASII). Coding protocols were developed, tested and utilized to assess: 1) the intensity of social influence (mandated use, role congruence, imbedded into workflow, and feedback related to performance assessment) and; 2) the degree of task uncertainty (complexity of decision, prevalence and/or frequency of decision, and ease of compliance).
The overall search yielded 63,731 references with 2,023 abstracts rated as meeting relevance criteria. CPOE: Thirty-two CPOE articles meeting the relevance criteria were extracted and reviewed. The Q test for heterogeneity was highly significant (X2 = 95.21; df=31) and the quality scores were very low, so no attempt was made to aggregate effects. The most significant quality problem was the lack of construct validity across studies. No two studies had comparable interventions, severely limiting generalizability. Across the entire literature, there was a failure to delineate relationships between variables and a limited number of dependent variables. Poor designs and measurement techniques were also prevalent. INFECTIOUS DISEASE: Forty-one articles meeting the relevance criteria were extracted and reviewed. The Q test for heterogeneity was highly significant (X2 = 102.34; df=40). Intensity of social influence and task uncertainty interacted such that social influence factors had more impact in studies with low uncertainty tasks (e.g. preventative care measures). Significant variance existed across studies after this analysis.
Effective implementation of alerts/reminders and other computerized interventions may depend on having a match between the implementation strategies and the type of decision task being supported.
External Links for this Project
Dimensions for VADimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.
If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/
VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project
DRA: Health Systems
Keywords: Behavior (provider), Decision support, Quality assurance, improvement
MeSH Terms: none