Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events?

Rosen AK, Itani KM, Cevasco M, Kaafarani HM, Hanchate A, Shin M, Shwartz M, Loveland S, Chen Q, Borzecki A. Validating the patient safety indicators in the Veterans Health Administration: do they accurately identify true safety events? Medical care. 2012 Jan 1; 50(1):74-85.

Related HSR&D Project(s)

• SDR 07-002 – Validating the Patient Safety Indicators in the VA: A Multi-faceted Approach

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

BACKGROUND: The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) use administrative data to detect potentially preventable in-hospital adverse events. However, few studies have determined how accurately the PSIs identify true safety events. OBJECTIVES: We examined the criterion validity, specifically the positive predictive value (PPV), of 12 selected PSIs using clinical data abstracted from the Veterans Health Administration (VA) electronic medical record as the gold standard. METHODS: We identified PSI-flagged cases from 28 representative hospitals by applying the AHRQ PSI software (v.3.1a) to VA fiscal year 2003 to 2007 administrative data. Trained nurse-abstractors used standardized abstraction tools to review a random sample of flagged medical records (112 records per PSI) for the presence of true adverse events. Interrater reliability was assessed. We evaluated PPVs and associated 95% confidence intervals of each PSI and examined false positive (FP) cases to determine why they were incorrectly flagged and gain insight into how each PSI might be improved. RESULTS: PPVs ranged from 28% (95% CI, 15%-43%) for Postoperative Hip Fracture to 87% (95% CI, 79%-92%) for Postoperative Wound Dehiscence. Common reasons for FPs included conditions that were present on admission (POA), coding errors, and lack of coding specificity. PSIs with the lowest PPVs had the highest proportion of FPs owing to POA. CONCLUSIONS: Overall, PPVs were moderate for most of the PSIs. Implementing POA codes and using more specific ICD-9-CM codes would improve their validity. Our results suggest that additional coding improvements are needed before the PSIs evaluated herein are used for hospital reporting or pay for performance.