Talk to the Veterans Crisis Line now
An official website of the United States government
CRE 10-370
Improving the Quality of Addiction Treatment Quality Measurement
Alexander H.S. Sox-Harris, PhD MS VA Palo Alto Health Care System, Palo Alto, CA Palo Alto, CA Funding Period: January 2012 - December 2013 |
|
BACKGROUND/RATIONALE:
Our recently completed HSR&D-funded research (IIR 07-092) revealed three serious problems with the validity of VHA and Health Plan Employer Data and Information Set (HEDIS) measures of addiction treatment quality: First, the presumed associations between established process quality measures and outcomes are weak or non-existent. Second, the validity of underlying strategies used at VHA and HEDIS for identifying health care encounters in the medical record is generally poor and varies substantially by setting and facility. Third, currently utilized addiction treatment quality measures focus exclusively on process and do not adequately capture other important domains of quality (e.g., structure, access, patient experiences, and outcomes). All VHA stakeholders benefit from the existence and proper use of valid healthcare quality measures. OBJECTIVE(S): Aim 1. Evaluate the associations between currently proposed, high-profile measures of addiction treatment quality and outcomes (predictive validity). Predictive validity refers to the strength of association between access or process quality measures and subsequent patient outcomes. We identified three sources of 41 newly developed, but unvalidated, addiction treatment quality measures: 18 VA Unform Mental Health Services Handbook Metrics, 18 from the recently completed Congressionally-mandated RAND/Altarum evaluation of VA mental health services; and 5 newly developed metrics from the Washington Circle policy group. Aim 2. Examine the associations among contemporaneously measured quality indictors, and investigate the validity of underlying care identification strategies (concurrent validity). According to Donabedian, quality indicators should be associated to other contemporaneously measured indicators of the underlying construct. If the association between theoretically linked quality indicators is strong, this is evidence that the method of assessment is strong and the subsequent inferences are valid. Conversely, if the expected relationships between indicators are weak, this suggests problems with the quality or measurement model. Therefore, we will assess the associations between contemporaneously measured, theoretically linked quality measures. Another aspect of concurrent validity is the extent to which quality measure specifications accurately identify the targeted processes and patients in available data (termed specification strategy). Secondary Aim 1. Evaluate the impact of diagnostic and setting factors that may moderate predicitve or concurrent validity. Given the finding by our team and others that relationships between patterns of care and outcomes vary by diagnostic and setting factors, we will evaluate these potential moderators of the predictive and concurrent validity examined in Aims 1 and 2. Secondary Aim 2. Evaluate methods to combine validated quality metrics into information that is clinically and operationally meaningful. We propose to explore various means of combining the quality metrics into composite measures, and then evaluate the association of these composites with outcomes. METHODS: Aim 1. To achieve this aim, we will utilize four samples of patients for which the quality measures can be calculated and for whom we have pre-existing outcome survey data (e.g., pre- and post-treatment symptom improvement data) and/or administrative outcome data (e.g., readmissions, ER utilization, number of detoxification episodes, etc.). Aim 2. Our previous HSR&D-funded research found several problems in the specification strategies used in both the VHA Continuity of Care measure (based on VHA clinic stop codes), and the 2006 version of HEDIS SUD treatment Initiation and Engagement measures (based on diagnosis/CPT code combinations). In this project, we propose to conduct a similar validation study of the substantially different 2010 HEDIS specification strategy. There are also specification strategies in the new OMHS metrics that have not been validated. For example, patients receiving a short course of certain medications are assumed to be undergoing detoxification, but this assumption has not been verified. After checking and possibly refining these specification strategies, we will test if various modifications of the specification strategies alter facilities' relative performance, and whether modifications can improve the links between the new access and process quality measures and outcomes. FINDINGS/RESULTS: Several process measures had consistent associations to diverse outcomes across the samples of patients. For example, patients who received at least one week of intensive SUD treatment, defined as 9 hours of treatment contact, had more improvement in alcohol and drug outcomes and 2-year mortality compared to patients not receiving intensive treatment. The number of weeks of intensive treatment was not linearly related to outcomes. Several quality measures, such as receiving an outpatient SUD visit within 14 days of discharge from a residential addiction program, had less reliable or weaker associations to various outcomes across samples. Other measures had no observable link to outcomes. Chart review analyses indicated that the 2009 HEDIS care identification strategy had excellent validity within addiction specialty settings. Most outpatient and inpatient records identified by 2009 HEDIS-qualified SUD diagnosis/procedure code combinations and SUD specialty settings contained qualifying chart documentation of SUD care: 585 (90.4%) of the 647 outpatient records and 667 (95.7%) of the 697 inpatient records. Our findings of high rates of documented SUD care in records identified by this record type were consistent with our initial HSR&D study using the 2006 HEDIS measure. In terms of non-SUD specialty settings, the 2009 HEDIS care identification strategy had weaker, but improved validity when compared to the results from our initial study. Of the 699 outpatient records identified by 2009 HEDIS-qualified SUD diagnosis/CPT combinations and non-SUD specialty clinic stop codes, 684 had an outpatient progress note on the day of care. Of these, 525 (76.8%) had SUD care documented. This rate is significantly higher than the rate (62.7%) found for the same record type using the 2006 HEDIS measure. Also, of the 700 inpatient records identified by 2009 HEDIS-qualified SUD diagnosis/procedure code combinations and non-SUD specialty bed section codes, 698 had a progress note on the day of care. Of these, 455 (65.2%) had SUD care documented, compared with 46.2% for the same record type identified by the 2006 HEDIS measure. Though the HEDIS addiction treatment quality measure continued to have validity in addiction specialty settings, it persisted as an unreliable indicator of SUD care on the facility (VISN) level. Among outpatient records identified by HEDIS-qualified SUD diagnosis/CPT code combinations and SUD-specialty clinic stop codes, the facility rates of concordance with chart documentation of SUD care ranged from 68% to 100%. For outpatient records identified by HEDIS-qualified SUD diagnosis/CPT code combinations and non-SUD specialty clinic stop codes, the facility rates of concordance ranged from 59% to 93%. Unlike our initial study, inpatient records identified by HEDIS-qualified SUD diagnosis/procedure code combinations and SUD-specialty bed section codes had substantial variability in facility rates of concordance with documentation of SUD care, ranging from 74% to 100%. (Previously, the facility range was 95% to 100%). Among inpatient records with HEDIS-qualified SUD diagnosis/procedure code combinations and non-SUD specialty bed section codes, the facility rates of concordance with chart documentation of SUD care ranged from 37% to 86%. IMPACT: Results from this study are being used by diverse stakeholders to focus on the addiction treatment quality measures and underlying processes that are most tightly linked to outcomes. Based on our results, changes to the SUD metrics in the VA Mental Health Information System are under consideration. External Links for this ProjectNIH ReporterGrant Number: I01HX000517-01A2Link: https://reporter.nih.gov/project-details/8269875 Dimensions for VA Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects. Learn more about Dimensions for VA. VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address. Search Dimensions for this project PUBLICATIONS:Journal Articles
DRA:
Mental, Cognitive and Behavioral Disorders, Substance Use Disorders, Health Systems
DRE: Diagnosis Keywords: none MeSH Terms: none |
