Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

RRP 11-428 – HSR Study

RRP 11-428
Guidelines to Performance Measures: Automating Quality Review for Heart Failure
Mary K. Goldstein, MD MS
VA Palo Alto Health Care System, Palo Alto, CA
Palo Alto, CA
Funding Period: January 2012 - December 2012
Chronic Heart failure (HF) is a leading cause for medical care in VA, causing frequent rehospitalizations and an increasing number of outpatient visits. Approximately 240,000 veterans receive care for HF in the VA. Additionally, HF is the number one reason for discharge for Veterans treated within the VA healthcare system. Accordingly, HF is an important target for quality assessment so that performance gaps between best recommended practices and actual practices may be identified and addressed. Evidence and recommendations regarding effective treatment for HF have been summarized in the American College of Cardiology/American Heart Association (ACC/AHA) guideline for HF, endorsed by VA HF experts. Several recommendations for HF management have complex clinical criteria, requiring data from multiple different components of patient data, including lab, pharmacy and diagnostic data.

The VA is in a position to use its advanced clinical information systems in order to develop and apply complex performance measures, thereby improving care for patients with HF.

The long-term goal across our HF projects is to improve health care and outcomes for Veterans with HF. The short-term objective of this one year project was to develop methods to process computable performance measures with patient data from the VA electronic health record, translating the evidence-based recommendations in the guideline into automated performance measures capturing the full complexity of the guideline recommendations.

Specific aims were as follows:

1.Refine the encoding of 4 ACC/AHA HF guideline recommendations in ATHENA-HF knowledge base (KB)
2.Define and apply the necessary patient data inputs to process the guideline recommendations
a.Link the output of the NLP EF software to become input to the HF guidelines system
b.Identify other patient data necessary for input and assemble sample data on VINCI
3.Extend the existing EON execution engine, developed for processing one case at a time, to process these recommendations with test data for multiple cases in a batch;
a.Establish a system on VINCI to process multiple VA records for patients with HF to generate a database recording applicability of the guidelines and adherence where applicable
4.Test the accuracy of the conclusions generated from the recommendations with sample VA data.

The overall method was iterative cycles of design, implementation, and testing of software on the VA HSR&D secure computing environment VINCI.

In discussion with heart failure stakeholders, we identified high priority measures from the ACC/AHA guidelines for the Diagnosis and Management of Heart Failure in Adults. The initial measures addressed appropriate usage of beta-blockers, ACE inhibitors/ARBs. We compared authoritative information regarding these heart failure recommendations from multiple sources. We operationalized these guideline recommendations into performance measures by defining the numerator, denominator, and exclusions. We used EON, an existing expert system coded in Java that can be used to compute recommendations from clinical guidelines with patient data input. EON was extended to evaluate performance measures (PM) for each patient. We also created the Heart Failure Performance Measurement (HFPM) Java system for processing whole patient populations. This system extracts and transforms raw patient data, invokes EON for each patient, and then saves EON's evaluations into database tables. The output of the system specifies (1) whether a patient is eligible to be considered for the PM and (2) whether the patient's medical regimen adheres to the recommendations. We used standardized data terminologies such as ICD9 codes for diagnoses and CPT codes for procedures to enhance transferability of the tools developed.
We tested the accuracy of the system by comparing system outputs with human review of the same patient data.
We defined numerator, denominator, and exclusions for six heart failure measures in detail, including specifications of the VA data source and temporal constraints. The performance measures addressing ACE-inhibitors/ARBs and beta-blocker usage were completely encoded in Prot g knowledge base(KB); the rest of the performance measures were partially encoded pending details on these measures that still need to be elaborated.

We created the automated HFPM system for handling whole patient populations The HFPM system extracts raw patient data from SQL server databases on VINCI, applying a filtering query to identify the patient population potentially eligible for the heart failure performance measures. The potentially eligible raw patient data is then transformed so that it can be processed by EON, patient by patient. After EON processes and evaluates the data, the detailed results are stored in a database and SQL queries can be run to display the results and analyses. We have evaluated the system performance using a small convenience sample of 340 VA patients, using actual structured patient data combined with synthetic ejection fraction data. Out of the 340 patients, 73 outpatient cases and 33 hospitalizations satisfy initial eligibility criteria. A preliminary validation of the accuracy of the system on 12 inpatient hospitalizations and 20 outpatient cases demonstrates that the system successfully generates conclusions for the ACE-inhibitors/ARB and beta-blockers performance measures in the majority of cases.

This project gives VA an opportunity to maintain its leadership position in health care quality measurement, by developing and applying complex measures that reflect clinical complexity more accurately than do simple measures. Performance measures that capture clinical details of the evidence-based recommendations allow a method to identify which patients are not being treated in keeping with the evidence, while accounting for clinical complexity, which provides an opportunity to improve quality of care for Veterans with HF.

External Links for this Project

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project


VA Cyberseminars

  1. Goldstein MK. Knowing and Doing: Automating Performance Measures and Clinical Decision Support. [Cyberseminar]. 2013 Jul 16. [view]

DRA: Health Systems, Cardiovascular Disease
DRE: Technology Development and Assessment
Keywords: none
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.