Session number: 1011
Abstract title: Benchmarking VAMCs in Delivery of Evidence-based Clinical Preventive Services
Author(s):
BN Doebbeling - Program in Health Services Research, Iowa City VAMC, Departments of Internal Medicine and Epidemiology, University of Iowa Colleges of Medicine and Public Health
TE Vaughn - Department of Health Management and Policy, University of Iowa College of Public Health
E Letuchy - Program in Health Services Research, Iowa City VAMC, Department of Internal Medicine, College of Medicine
LG Branch - The Duke University School of Medicine, Durham, NC
MM Ward - Department of Health Management and Policy, University of Iowa College of Public Health
RF Woolson - Department of Biostatistics, University of Iowa College of Public Health
BJ BootsMiller - Program in Health Services Research, Iowa City VAMC and Department of Internal Medicine, The University of Iowa College of Medicine
T Tripp-Reimer - The University of Iowa College of Nursing, Iowa City, IA
PM Peloso - Program in Health Services Research, Iowa City VAMC and Department of Internal Medicine, The University of Iowa College of Medicine
Objectives: To identify factors influencing consistent provision of clinical preventive services (CPS), we developed and compared methods to select VAMCs performing consistently higher or lower than expected.
Methods: The 1998 Veterans Health Survey (VHS) assessed evidence-based CPS delivery in a stratified sample of 450 outpatients from each VAMC. Proportions appropriately receiving each CPS within the recommended interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-90%) and above a range of lower cutoffs (eg, 40-55%). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall’s tau-b and chi-square quality-of-fit tests. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank-sum test all on 14 VHS and 11 EPRP outcomes.
Results: Data from 39 939 patients (67% response rate) were examined. In combination, cutoffs of > 50th percentile and > 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (N=17) were >= 15. The sum of outcomes ranked above the same cutoffs for CLA facilities (N=16) were >= 3. EPRP and 1999 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CHL facilities; differences were greater using the percentile rank approach.
Conclusions: The VA has successfully encouraged adoption of evidence-based CPS throughout its health care system. However, facilities vary in their levels of delivery.
Impact statement: Examining service delivery across multiple performance indicators allows identification of opportunities to improve the delivery of CPS. This approach identifies institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.