Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

Health Services Research & Development

Go to the ORD website
Go to the QUERI website

HSR&D Citation Abstract

Search | Search by Center | Search by Source | Keywords in Title

The Development and Validation of Clinical Performance Measures

Bravata DM, Myers L, Cheng E, Arling G, Damush TM, Miech EJ, Sico J, Phipps MJ, Johanning J, Williams LS. The Development and Validation of Clinical Performance Measures. Poster session presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 9; Philadelphia, PA.

Related HSR&D Project(s)


Objectives: The Veterans Health Administration (VHA) does not assess quality of care for patients with transient ischemic attack (TIA). The VHA currently uses self-report for three measures of stroke care quality and has begun to contract for chart review to assess the Joint Commission stroke metrics for a sample of stroke patients. Although patients with TIA and minor stroke are at very high risk of recurrent vascular events (25% of patients will have a recurrent vascular event or death in the first 90-days post-index event), evidence demonstrates that timely delivery of guideline concordant care dramatically reduces this risk. The first step toward improving quality of care, and thereby reducing recurrent vascular events, is to develop a robust measurement system that comprehensively evaluates quality of care for this high risk population. We sought to develop and validate process measures that could be assessed for all patients with TIA and minor stroke using readily available electronic data and that could be used for the evaluation of care at the facility-level. Specifically, we were interested in evaluating guideline-concordant care components that have been associated with improved patient outcomes and that could serve as targets for future quality improvement efforts. Methods: This project included two data components: (1) a VHA administrative data development cohort including all patients with a TIA or minor stroke who received care in any VHA Emergency Department (ED) or inpatient setting during fiscal year 2011; and (2) a chart review data validation cohort that consisted of a sample of TIA and minor stroke patients from large volume VHA hospitals. Process measures were included if they represented guideline-concordant elements of care, were currently being used in VHA quality assessment, or were part of the proposed American Heart Association/American Stroke Association quality metrics. The development cohort algorithms were iteratively refined on the basis of comparisons with the validation data. Agreement between the development (administrative) and validation (chart review) data was classified for both the numerators and denominators as: "excellent" if agreement exceeded 90%, "good" if agreement exceeded 80%, and "unacceptable" if agreement on either the numerator or denominator was < 80%. Process measures were considered reasonable for use in future implementation projects if: the agreement was at least good, and the mean national facility pass rate was < 85% or if variation existed across facilities such that at least 10% of facilities were outliers. Results: We developed 25 administrative data process measures describing 15 domains of care: 5 processes of care had unacceptable agreement between the development (administrative) data and the validation (chart review) data (oral hypoglycemic medication intensification, electrocardiography, telemetry, nicotine replacement, deep vein thrombosis prophylaxis); 3 processes had clinical concerns which limited their usefulness in performance measurement (dysphagia screening, Holter monitor, and lipid medication intensification); and 4 processes had performance rates above 85% (anticoagulation for atrial fibrillation and INR ordered, antithrombotics at discharge, and brain imaging). Among the remaining 13 processes, 9 had excellent agreement and 4 had good agreement, and therefore should be considered for use in future implementation projects (neurology consultation, carotid artery imaging and timely carotid stenosis intervention, anticoagulation quality, rehabilitation needs assessment, lipid measurement, HbA1c measurement, substance abuse treatment referral for alcohol dependency, antithrombotics by hospital day 2, hypertension control and antihypertensive intensification, lipid management, and coronary risk assessment). Implications: These results demonstrate that it is feasible to construct electronic measures of quality for key processes of acute TIA/minor stroke care and that for thirteen processes of care opportunities exist to improve care quality for the VHA system nationwide. Impacts: These data provide the first national benchmarking data for TIA/minor stroke care quality for the VHA. The electronic measures of quality of care will be used to design a future implementation program.

Questions about the HSR&D website? Email the Web Team.

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.