Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

HSR Citation Abstract

Search | Search by Center | Search by Source | Keywords in Title

Validating the Patient Safety Indicators in the Veterans Health Administration

Borzecki AM, Borzecki AM. Validating the Patient Safety Indicators in the Veterans Health Administration. Presented at: AcademyHealth Annual Research Meeting; 2009 Jun 30; Chicago, IL.




Abstract:

Research Objectives: The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) use administrative data to screen for potential inpatient adverse events. Several of these indicators have recently been endorsed by the National Quality Forum as hospital quality measures. However, the validity of such measures in different systems, including the Veterans Health Administration (VA), is unclear. As part of a comprehensive PSI validation study, we are currently evaluating the criterion validity of fifteen PSIs; this study reports on results from three PSIs: 1) postoperative pulmonary embolus or deep vein thrombosis (PE/DVT), 2) accidental puncture or laceration (APL) and 3) iatrogenic pneumothorax (IP). Study Design: This was a retrospective observational study using FY03-07 inpatient administrative and electronic medical record data from 28 VA hospitals. Hospitals were selected based on geographic diversity and observed PSI rates. We applied the AHRQ PSI software (v.3.1a) to administrative data to identify cases suspected of having a postoperative PE/DVT, APL or IP. To determine the positive predictive value (PPV) of these indicators, trained nurses conducted chart reviews of 112 flagged cases for each PSI from sample hospitals, using standardized chart abstraction tools and guidelines developed by AHRQ and modified for VA use. Based on previously reported PPV estimates, this number was selected to ensure reasonably narrow PPV confidence intervals (range = 10 to 20%). Physicians performed additional false positive analysis to determine the strengths and weaknesses of each PSI. Inter-rater reliability was also measured between two nurse abstractors. Population Studied: Veterans receiving VA inpatient acute care at 28 selected hospitals from FY03 through FY07. Principal Findings: The PPVs for postoperative PE/DVT, APL and IP were 44% (95% CI, 34-53%), 86% (78-91%) and 78.4% (70-85), respectively. Inter-rater reliabilities were > 90% for all indicators. For postoperative PE/DVT, 62% of false positives were related to a PE/DVT that occurred prior to admission or before the surgical procedure; the remaining 38% were attributed to coding issues due either to individual coder errors or to inherent limitations in available codes. For APL, 33% of false positives were due to events occurring prior to admission; 40% appeared to be due to coding inaccuracies, for example 2 cases were associated with oozing from a central venipuncture site. For IP, 35% of false positives were due to events occurring prior to admission; 22% were related to thoracic procedures known to breach the pleural cavity, 13% were non-procedure related (i.e., spontaneous) pneumothoraces. About 13% of false positives had no chart documentation of pneumothorax, while 9% had a radiology report suggesting a "possible pneumothorax", with later reference to the radiologic changes being more likely due to other pathology. Conclusions: While the PSI algorithms for APL and IP demonstrated high predictive values for detecting "true" events, the postoperative PE/DVT PSI did not accurately detect cases in which a postoperative PE or DVT occurred. Implications for Policy, Practice or Delivery: The accuracy and usefulness of these PSIs for quality measurement could be improved by relatively simple coding enhancements, such as adoption of "present on admission" codes, and by better training of coders in general adverse event coding and in specific post-procedure complication coding.





Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.