» Back to Table of Contents

Research Highlight

The Use of Electronic Health Record Data for Adverse Event Detection to Promote Patient Safety

Surgical services are increasingly provided in outpatient settings of care both in the Veterans Health Administration (VHA) and the private sector, increasing from 13.4 million surgeries in 1995 to 19.2 million surgeries in 2018. Achieving optimal care and developing strategies to mitigate risks for outpatient procedures necessitates identifying systems-level vulnerabilities. Measuring and evaluating adverse events (AEs) or defects in a system is a critical step in this process. Although several approaches and tools are available for supporting improvements in the quality of care in surgical and non-surgical invasive procedures, AEs in outpatient procedures remain an area in need of programmatic tracking, monitoring, and subsequent quality improvement. Patient safety surveillance tools that rely on electronic health record (EHR) data offer a potential catalyst to support accurate retrospective AE detection with minimal manual review effort.

What are Electronic Surveillance Tools?

Surveillance tools were originally developed with coded data to facilitate identification of AEs. Trigger systems can then focus manual chart review to target additional investigation into modifiable contributing factors. Triggers have evolved to become a widely used way to retrospectively analyze EHRs to identify errors and AEs, measure the frequency with which such events occur, and track the progress of safety initiatives over time. Many automated AE triggers rely on administrative data; however, the development of EHR flags based on text mining of free-text clinical notes offers a powerful complement to administrative data that can further improve surveillance processes. Free-text clinical notes provide additional information not available through coded data alone that can be leveraged to enhance AE ascertainment and reduce burdensome manual chart review of cases without AEs. Over the last five years, we have developed, tested, and implemented AE surveillance tools that combine both administrative and EHR data for use in non-inpatient settings to automate and expand quality monitoring activities (Table 1).

Development of an Adverse Event Surveillance Model for Outpatient Surgery

Our earliest effort to use EHR data to improve detection of outpatient AEs focused on outpatient surgical care in VHA.¹ This was the first study to comprehensively evaluate patient safety after outpatient surgery across an entire healthcare system; previous work had been limited to a sample of surgeries or individual facilities. We adapted previously developed AE triggers from inpatient surgical care based on administrative data: 14-day emergency department visit, admission, and mortality, and developed unique triggers based on VHA clinic name data: multiple postoperative surgery or urology clinic visits, or a nurse telephone encounter. We applied these triggers to FY2012-2014 outpatient surgeries (n=744,355) and chart reviewed a selection of trigger-flagged and unflagged cases to develop a training dataset for our AE surveillance model. The trigger-based surveillance model performed well, accurately identifying outpatient surgeries with a high probability of an AE. In the summer 2020 FORUM, we presented how this model was successfully implemented in one VHA ambulatory surgery center.

Novel Method to Flag Cardiac Implantable Device Infections by Integrating Text Mining with Structured Data

Following the same informatics method, we applied electronic triggers to develop a surveillance model to detect specific AEs: infections following cardiac implantable device (CIED) placements in interventional cardiology.² These infection-targeted triggers included administrative data-based comorbidities and mortality data, as well as EHR structured and free text diagnostic and therapeutic data (e.g., vital signs, procedure notes, discharge summaries, and microbiology results). The study sample consisted of FY2016-2017 CIED procedures (n= 19,212). As before, we chart reviewed cases to build a training dataset to estimate our AE prediction model using half the CIED procedures; we used the other half for model validation. Again, the model demonstrated strong predictive value for measuring true infections with a PPV=44 percent when the AE predicted probability exceeded 10 percent. This infection prediction model is currently being tested for real-time use as part of a bundled implementation intervention at three VA sites to assess audit and feedback approaches to improve periprocedural CIED infection prevention and antimicrobial use.

Development of a Periprocedure Trigger for Outpatient Interventional Radiology Procedures

Most recently, we developed another clinical note text trigger to detect potential AEs before, during, or shortly after an outpatient interventional radiology procedure.3,4 We developed and tested this trigger with invasive outpatient interventional radiology procedures from FY2017-2019 (n=135,285). The periprocedure algorithm flagged 245 cases (0.18 percent), all of which underwent expert chart review, and 138 of these had ≥1 AE (PPV=56 percent). We also evaluated how well the periprocedure trigger identified AEs not detected by previously developed triggers: 43 of 138 (27 percent) AEs were flagged exclusively by the periprocedure trigger. These included allergic reactions, adverse drug events, ischemic events, bleeding events requiring blood transfusions, and cardiac arrest requiring cardiopulmonary resuscitation. The periprocedure trigger offers a complement to other electronic triggers developed for outpatient AE surveillance.

Our work encompasses several years of developing, validating, and implementing AE surveillance algorithms leveraging the rich data available in VHA’s EHR. Over time and across multiple specialty areas, we have improved and expanded electronic trigger flags both for retrospective and real-time applications. The combined results demonstrate the potential of automated administrative and text-based predictive algorithms to improve detection of AEs in the outpatient procedure setting. This is critical for ensuring high-quality surgical and non-surgical invasive procedure care in VHA.

References

1. Mull HJ, et al. “Development of an Adverse Event Surveillance Model for Outpatient Surgery in the Veterans Health Administration,” Health Services Research 2018 Dec;53(6):4507-452.8.
2. Mull HJ, et al. “Novel Method to Flag Cardiac Implantable Device Infections by Integrating Text Mining With Structured Data in the Veterans Health Administration’s Electronic Medical Record,” JAMA Network Open 2020 Sep 1;3(9):e2012264.
3. Bart N, et al. “Development of a Periprocedure Trigger for Outpatient Interventional Radiology Procedures in the Veterans Health Administration,” Journal of Patient Safety 2023 Apr 1;19(3):185-92.
4. Mull HJ, et al. “Development and Validation of an Electronic Adverse Event Model for Patient Safety Surveillance in Interventional Radiology,” Journal of the American College of Radiology 2023 Dec 27:S1546-1440(23)01041-4.