Lead/Presenter: Andrew Gawron, COIN - Salt Lake City
All Authors: Gawron AJ (Informatics, Decision-Enhancement, & Analytic Sciences Center, Salt Lake City VA Healthcare System), Yao Y (Salt Lake City VA Healthcare System & University of Utah), Thompson W (Northwestern University) Patterson OV (Salt Lake City VA Healthcare System & University of Utah) Cole G (Salt Lake City VA Healthcare System & University of Utah Gupta S (San Diego VA Healthcare System & University of California San Diego) Dominitz JA (National VA GI program office, VA Puget Sound Healthcare System & University of Washington) Mary Whooley (Measurement Science QUERI, San Francisco VA Healthcare System) Carmel Malvar (Measurement Science QUERI, San Francisco VA Healthcare System) Tiffany Nguyen—Vu (Measurement Science QUERI, San Francisco VA Healthcare System) Tonya Kaltenbach (Measurement Science QUERI, San Francisco VA Healthcare System)
Objectives:
Colorectal cancer (CRC) prevention is a top VA priority. CRC is commonly diagnosed in Veterans with a 35% 3-year mortality rate. In the VA, > 200,000 colonoscopies are performed each year, 50-60% of which are for screening. Colonoscopy quality benchmarks, especially the adenoma detection rate (ADR), have been strongly linked to CRC incidence and death. A recent Office of the Inspector General (OIG) report highlighted colonoscopy quality deficiencies in the VA. A primary objective of the Measurement Science QUERI was to build the infrastructure to measure and report colonoscopy quality at the site and provider level for the VA health care system.
Methods:
We built an operational database of colonoscopy procedures and linked pathology results with a combination of structured data (patient level), colonoscopy procedure and pathology notes from the existing VA Corporate Data Warehouse Text Integration Utility using data from January 1, 2013 through December 31, 2017. Using FY 2013-2016 data, we iteratively developed a scalable natural language processing (NLP) and reporting system to extract and visualize (1) procedure indication, (2) bowel preparation quality, (3) exam extent, and (4) adenoma detection. We applied the final system to FY 2017 data and conducted a random sample error analyses of 400 colonoscopy notes and linked pathology across 76 VA sites to determine information extraction performance for information retrieval, including precision, recall, and F measures.
Results:
The completed quality reporting infrastructure within VA includes a framework for scalability, prospective updates, and a secure website to visually present quality metrics for sites and providers. We identified and processed 432,486 colonoscopy and 267,027 linked pathology notes from 76 VA sites meeting inclusion criteria from 2013-2017. Final error analysis showed excellent performance for extracting colonoscopy quality metrics, including adenoma detection rate (F = 0.96), screening indication (F = 0.83), cecal intubation rate (F = 0.97), and bowel preparation quality (F = 0.90).
Implications:
We successfully developed robust, accurate NLP tools and the informatics infrastructure to measure and report colonoscopy quality for national VA implementation.
Impacts:
This system will serve as the foundation of the VA Endoscopy Quality Improvement Program to directly address VA's critical need to implement evidence-based colonoscopy quality measurement and reporting.