Diabetes healthcare quality report cards: how accurate are the grades?

Pogach L, Xie M, Shentue Y, Tseng CL, Maney M, Rajan M, Tiwari A, Kolassa J, Helmer D, Crystal S, Safford M. Diabetes healthcare quality report cards: how accurate are the grades? The American journal of managed care. 2005 Dec 1; 11(12):797-804.

Related HSR&D Project(s)

• IIR 00-072 – Risk Adjustment of HbA1c for Performance Measurement in the VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

If you have VA-Intranet access, click here for more information vaww.hsrd.research.va.gov/dimensions/

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.    Search Dimensions for VA for this citation
* Don't have VA-internal network access or a VA email address? Try searching the free-to-the-public version of Dimensions

Search for Abstract from PubMed

---

Abstract:

OBJECTIVE: To evaluate the accuracy and precision of random sampling in identifying healthcare system outliers in diabetes performance measures. STUDY DESIGN: Cross-sectional analysis of 79 Veterans Health Administration facilities serving 250 317 patients with diabetes mellitus between October 1, 1999, and September 30, 2000. METHODS: Primary outcome measures were poor glycosylated hemoglobin (A1C) control and good low-density lipoprotein cholesterol (LDL-C) and blood pressure (BP) control. Facility performance for each measure was calculated using 150 separate random samples and was compared with results using the bootstrap method as the criterion standard for determining outlier status (defined as a > / = 5% difference from the mean, within the 10th or 90th percentile, or > / = 2 SDs from the mean). RESULTS: The study population was largely male (97.4%), with 54.0% of subjects being 65 years or older. The facility-level mean performances were 22.8% for poor A1C control, 53.1% for good LDL-C control, and 55.3% for good BP control. Comparing the random sampling method with the bootstrap method, the sensitivity ranged between 0.64 and 0.83 for the 3 outcome measures, positive predictive values ranged between 0.55 and 0.88, and specificity and negative predictive values ranged between 0.88 and 0.99. CONCLUSIONS: The specificity and negative predictive value of the random sampling method in identifying nonoutliers in performance were generally high, while its sensitivity and positive predictive value were moderate. The use of random sampling to determine performance for individual outcome measures may be most appropriate for internal quality improvement rather than for public reporting.