BACKGROUND/RATIONALE:
VA performance monitoring makes extensive use of diagnosis-based quality measures that track delivery of care only among patients who have qualifying ICD-9 diagnosis codes. However, diagnosis-based measures can have critical validity problems if the targeted condition is under- or over-diagnosed to differing degrees across facilities. Use of diagnosis-based process measures can therefore undermine one of the primary purposes of quality measurement: The comparison of facilities and systems. In addition, diagnosis-based measures cannot be used to detect gaps in access to care for patients who have a targeted condition but no qualifying diagnosis code. Finally, when diagnosis rates vary across patient subgroups, diagnosis-based measures cannot be used to detect and act on healthcare disparities. Problems with diagnosis-based measures could be remedied if true prevalence data were available; however, for many conditions, the electronic health record (EHR) does not contain data on true prevalence.

OBJECTIVE(S):
We propose to build a model for predicting prevalence using multiple sources of existing data and to validate it through a one-time collection of survey-based SUD prevalence data. Focusing on substance use disorder (SUD) care as an example, the objectives of this study are to: (a) assess the degree of SUD under- or over-diagnosis; (b) refine and validate a model for predicting SUD prevalence among VA patients; and (c) assess disparities in SUD diagnosis.

METHODS:
We will collect data on SUD among VA patients using a validated instrument. We will compare observed diagnosis rates to survey-based prevalence estimates and will refine a prototype SUD prediction model and will compute facility performance rankings using diagnosis rates versus predicted prevalence to assess the extent to which variation in performance may reflect variation in diagnosis or coding. Finally, we will assess possible disparities in diagnosing by comparing the gap between diagnosis and estimated prevalence across patient groups.

FINDINGS/RESULTS:
This project has not yet produced findings.

IMPACT:
In this project, we will develop methods to improve diagnosis-based measures to ensure the integrity and effectiveness of VA performance monitoring. These methods can be readily adapted for other conditions where performance monitoring depends on diagnosis-based measures and gold-standard prevalence data are not readily available.

---

External Links for this Project

NIH Reporter

Grant Number: I01HX002128-01A1
Link: https://reporter.nih.gov/project-details/9189567

---

PUBLICATIONS:

Journal Articles

1. Williams EC, Fletcher OV, Frost MC, Harris AHS, Washington DL, Hoggatt KJ. Comparison of Substance Use Disorder Diagnosis Rates From Electronic Health Record Data With Substance Use Disorder Prevalence Rates Reported in Surveys Across Sociodemographic Groups in the Veterans Health Administration. JAMA Network Open. 2022 Jun 1; 5(6):e2219651. [view]
2. Hoggatt KJ, Harris AHS, Washington DL, Williams EC. Prevalence of substance use and substance-related disorders among US Veterans Health Administration patients. Drug and Alcohol Dependence. 2021 Aug 1; 225:108791. [view]
3. Boden MT, Hoggatt KJ. Substance Use Disorders Among Veterans in a Nationally Representative Sample: Prevalence and Associated Functioning and Treatment Utilization. Journal of studies on alcohol and drugs. 2018 Nov 1; 79(6):853-861. [view]

Journal Other

1. Hill J, Hoggatt KJ. The Tenability of Counterhypotheses: A comment on Bross' discussion of statistical criticism. Observational studies. 2018 Jan 1; 4(2018):34-41. [view]

DRA: Mental, Cognitive and Behavioral Disorders
DRE: TRL - Applied/Translational, Epidemiology, Diagnosis
Keywords: Disparities, Ethnicity/Race, Predictive Modeling, Quality Indicators, Research Measure Development, Substance Use and Abuse
MeSH Terms: none