BACKGROUND/RATIONALE:
The VA has incorporated extensive diabetes measures in their External Peer Review Program (EPRP) quality measurement system, which is based on medical record abstraction.
While EPRP is VA’s gold standard and provides the most reliable diabetes quality information currently available, chart abstraction is very expensive. Certain process and intermediate outcome data (for example, whether a lab test was performed and its value) are available from the Veteran’s Integrated Health Systems Technology and Architecture (VISTA) system. However, the validity and reliability of these data are unknown.

OBJECTIVE(S):
The specific goals of the project include: (1) to examine the correlation of diabetes measures derived from VISTA measures with those derived from medical record and patient survey, and (2) to assess the ability of each of the measures to accurately identify facilities with higher or lower quality.

METHODS:
The project compares diabetic quality measures in the VA obtained from three existing data sources: VISTA, medical records and patient surveys (using data already collected by the VA Office of Quality and Performance (OQP) as part of a quality assurance activity). We have compare the quality measures derived from each data source to determine how they correlate and how each contributes to the variation in quality scores across facilities.

FINDINGS/RESULTS:
Success rates were higher for process measures derived from medical record versus automated data (e.g., 78% vs. 68% for LDL measured; 84% vs. 78% for A1c measured). This difference narrowed for intermediate outcome measures (e.g., 79% vs. 76% for LDL<130; 86% vs. 88% for A1c<9.5%). Agreement for measures derived from the medical record compared to automated data was moderate for process measures (e.g.,A1c measured, kappa=0.61) but high for intermediate outcome measures (e.g., A1c<9.5%, kappa=0.92). Hybrid measures, which use automated data supplemented with medical record data, yielded success rates similar to those of medical record based measures. Hybrid process measures would require medical record review in only 50% of cases. Process measures, but no intermediate outcome measures, showed significant variation attributable to the facility, regardless of the data source.

IMPACT:
We found that agreement between medical record and automated data was generally high. Nonetheless automated data tended to underestimate the success rate in process measures for diabetes. Applying hybrid methodology yielded results consistent with the medical record but required less data to come from medical record reviews. Despite the high rates in overall performance, further research should examine the underlying reasons for facility level variation in diabetes process measures in order to craft appropriate quality improvement programs.

---

External Links for this Project

---

PUBLICATIONS:

Journal Articles

1. Asch SM, Kerr EA, Lapuerta P, Law A, McGlynn EA. A new approach for measuring quality of care for women with hypertension. Archives of internal medicine. 2001 May 28; 161(10):1329-35. [view]
2. Kerr EA, Krein SL, Vijan S, Hofer TP, Hayward RA. Avoiding pitfalls in chronic disease quality measurement: a case for the next generation of technical quality measures. The American journal of managed care. 2001 Nov 1; 7(11):1033-43. [view]
3. Kerr EA, Smith DM, Hogan MM, Hofer TP, Krein SL, Bermann M, Hayward RA. Building a better quality measure: are some patients with 'poor quality' actually getting good care? Medical care. 2003 Oct 1; 41(10):1173-82. [view]
4. Kerr EA, Smith DM, Hogan MM, Krein SL, Pogach L, Hofer TP, Hayward RA. Comparing clinical automated, medical record, and hybrid data sources for diabetes quality measures. Joint Commission Journal on Quality Improvement. 2002 Oct 1; 28(10):555-65. [view]
5. Heisler M, Smith DM, Hayward RA, Krein SL, Kerr EA. How well do patients' assessments of their diabetes self-management correlate with actual glycemic control and receipt of recommended diabetes services? Diabetes Care. 2003 Mar 1; 26(3):738-43. [view]
6. Heisler M, Smith DM, Hayward RA, Krein SL, Kerr EA. Racial disparities in diabetes care processes, outcomes, and treatment intensity. Medical care. 2003 Nov 1; 41(11):1221-32. [view]
7. Kerr EA, Smith DM, Kaplan SH, Hayward RA. The association between three different measures of health status and satisfaction among patients with diabetes. Medical care research and review : MCRR. 2003 Jun 1; 60(2):158-77. [view]
8. Heisler M, Bouknight RR, Hayward RA, Smith DM, Kerr EA. The relative importance of physician communication, participatory decision making, and patient understanding in diabetes self-management. Journal of general internal medicine. 2002 Apr 1; 17(4):243-52. [view]

VA Cyberseminars

1. Kerr EA. Comparing Data Sources: Examining Diabetes Quality of Care. [Cyberseminar]. 2002 Dec 4. [view]

Conference Presentations

1. Kerr EA. Building a Better Quality Measure: Are some patients with 'poor quality' actually getting good care. Paper presented at: VA HSR&D National Meeting; 2003 Feb 13; Washington, DC. [view]
2. Kerr EA. Building a Better Quality Measure: How Can Measurement Help Us Close the Quality Gap? Paper presented at: AcademyHealth Annual Research Meeting; 2002 Jun 23; Washington, DC. [view]
3. Kerr EA. Comparing Diabetes Quality Measures Derived from Different Data Sources. Paper presented at: Society of General Internal Medicine Annual Meeting; 2002 May 3; Atlanta, GA. [view]
4. Kerr EA. The influence of health status of veterans with diabetes: Results from the VA Diabetes Quality Improvement Project. Paper presented at: VA HSR&D National Meeting; 2000 Feb 1; Washington, DC. [view]
5. Kerr EA. What Do Our Veterans Tell Us About Their Care. Paper presented at: Diabetes National Annual Symposium; 2002 Mar 21; Alexandria, VA. [view]

DRA: Health Systems Science
DRE: Technology Development and Assessment
Keywords: Diabetes, Quality assessment, Risk adjustment
MeSH Terms: none