Talk to the Veterans Crisis Line now
An official website of the United States government
Search | Search by Center | Search by Source | Keywords in Title
Kerr EA. Comparing Diabetes Quality Measures Derived from Different Data Sources. Paper presented at: Society of General Internal Medicine Annual Meeting; 2002 May 3; Atlanta, GA.
Background: Different data sources (survey, medical record, and administrative) are commonly used to assess diabetes quality of care. However, little is known about the relative reliability of these data sources. Our objectives were to 1) compare results of technical diabetes quality measures constructed from medical record and administrative data sources; and 2) examine whether facility level variation varies for different types of measures and method of construction. Further, we constructed hybrid quality measures, which require examining the medical record only if the administrative data suggested that a service had not been performed. Hybrid methodology is advocated to decrease the costs of medical record data collection, because fewer records need be reviewed, but is only a valid approach if concordance between the medical record and administrative data is high.Methods: We abstracted medical records of 1085 diabetic veterans who received care from 21 facilities in 4 Veterans Integrated Service Networks and who had answered the Diabetes Quality Improvement Program (DQIP) survey. Administrative data were obtained from a central VHA diabetes registry that contained information on laboratory tests and medication use. We constructed 6 DQIP-based quality measures (3 process, and 3 intermediate outcome) from each data source, and for the hybrid form, for the same time frame (1999-2000) and compared our results using success rate, agreement beyond chance (kappa), and variance attributable to the facility level (intra-class correlation coefficient). Results: Success rates were higher for process measures derived from medical record versus administrative data (e.g., 78% vs. 68% for percent with LDL measured; 84% vs. 78% for percent with A1c measured). This difference narrowed for intermediate outcome measures (e.g., 79% vs. 76% for percent with LDL < 130; 86% vs. 88% for A1c < 9.5%). Agreement for measures derived from the medical record compared to administrative data was moderate for process measures (e.g., LDL measured, kappa = 0.58; A1c measure, kappa = 0.61) but high for intermediate outcome measures (e.g., LDL < 130, kappa = 0.84; A1c < 9.5%, kappa = 0.92). Results were similar for checking and controlling blood pressure. Hybrid measures yielded success rates similar to those of medical record based measures, but would have required abstraction of less than 30% of the total records. All process measures showed significant variation attributable to the facility, regardless of the data source, but there was no facility level variation in the intermediate outcome measures. Using administrative data to rank facilities' scores on process measures, resulted in 15% of facilities changing rank by more than one quartile, compared to medical record data. In contrast, no facilities changed by more than one quartile rank when hybrid measures were used. Conclusions: We found that agreement between medical record and administrative laboratory and blood pressure data was generally high. Nonetheless, even in an integrated healthcare system with sophisticated information technology, administrative data tended to underestimate the success rate in technical process measures for diabetes and yielded different quartile rankings. Applying hybrid methodology yielded results consistent with the medical record but required much less data to come from medical record reviews. Despite the high rates in overall performance, further research should examine the underlying reasons for facility level variation in diabetes process measures in order to craft appropriate quality improvement programs.
