» Back to Table of Contents

Research Highlight

Using Artificial Intelligence to Improve Buprenorphine Retention for Veterans with Opioid Use Disorder

Opioid overdoses among Veterans have risen substantially over the past decade. Buprenorphine, one of the primary medication treatments for opioid use disorder (MOUD), saves lives and decreases OUD-related morbidity. Unfortunately, over half of Veterans initiating buprenorphine discontinue within six months – a minimum duration shown to be associated with the life-saving benefits of the medication.¹ New and innovative statistical strategies for identifying Veterans at risk for buprenorphine discontinuation and strategies that may help mitigate that risk are desperately needed. Can artificial intelligence help?

Can Machine Learning Predict Which Veterans are at Greatest Risk of Buprenorphine Discontinuation?

Machine learning is a branch of artificial intelligence and computer science that gives computers freedom to iteratively learn from large amounts of data without being explicitly programmed, imitating the way humans learn. This form of iterative learning gradually improves the accuracy of the decisions or predictions generated by the computer without the need for human direction. Unlike traditional programming, where a computer follows a set of predefined instructions, machine learning gives the computer a dataset and a task to perform and allows the computer to determine how best to accomplish the given task. This allows the computer to work without human assumptions or preconceived notions about the best way to approach a task. For example, if we want a computer to automatically recognize when an image of a horse appears, we give the computer thousands of images and tell it which of the images are horses. Then, we let the machine learning algorithms determine common patterns and features that are specific to an image of a horse. As thecomputer processes more images, it improves its ability to recognize horses, even among images it has never seen before.

Predictive modeling is one of the “real-world” applications of machine learning in healthcare. It involves teaching the computer to accurately make predictions about the future probability of a given outcome (e.g., buprenorphine discontinuation). VHA is currently using predictive modeling to decrease Veteran drug overdose and suicide rates. For example, VHA developed the Stratification Tool for Opioid Risk Mitigation (STORM) using population-level electronic health record data on Veterans. STORM is a clinical decision support tool, based on predictive modeling, that identifies Veterans at high risk for opioid overdose and suicide in real-time. For Veterans considered to be at high risk, STORM suggests additional care options that providers might deliver (e.g.,prescribing naloxone) to mitigate the patient’s health risk. In a recent trial, researchers found case reviews of Veterans at the highest risk of overdose or suicide per STORM were associated with 22 percent decreased odds of all-cause mortality in the four months following the review.²

Like STORM, predictive models could be developed to identify Veterans at high risk of buprenorphine discontinuation. As withpredicting drug overdose and suicide, predictive models for buprenorphine discontinuation could provide VHA clinicians with an opportunity to intervene with a Veteran before buprenorphine discontinuation occurs. For example, knowing a patient is at risk for this particular outcome, providers might address identified and modifiable risk factors including polysubstance use, mental health comorbidities, and transportation or financial challenges.

Development of a Predictive Model for Buprenorphine Discontinuation

Using national-level data from VHA’s Corporate Data Warehouse (CDW), we developed and validated predictive models, using machine learning algorithms, to predict buprenorphine discontinuation among Veterans. To do this, we identified nearly 60,000 buprenorphine treatment episodes among Veterans and identified 114 candidate predictors. For each episode, we determined whether the Veteran discontinued buprenorphine within six months of initiating the episode. We split the buprenorphine episodes into a training dataset (80 percent of episodes) and a testing dataset (20 percent of episodes) to ensure the training data for the models were not intermingled with the data on which we would test the models. We then instructed the computer to iteratively learn how best to predict buprenorphine discontinuation among the training dataset using machine learning models and the 114 candidate predictors. Finally, we evaluated the machine learning models on the testing dataset to determine how well the machine learning models could predict buprenorphine discontinuation on new, unseen Veteran episodes.

Overall, we found machine learning-based predictive models could accurately predict buprenorphine discontinuation.³ This finding demonstrates the feasibility of developing and validating models that can predict buprenorphine discontinuation before it occurs. Two important clinical implications can be drawn from our research thus far.

First, no candidate predictor contributed more than four percent to the model’s prediction of buprenorphine discontinuation. This empirically validates what many clinicians may already know – retaining Veterans with OUD on buprenorphine is a complex issue. Given no one predictor alone contributes a substantial portion to the model’s prediction indicates buprenorphine retention is complex and simple heuristics that may be used by clinicians to identify Veterans at high risk for discontinuation are likely to be inadequate. Second, because prediction of buprenorphine discontinuation is not driven by a small number of predictors, buprenorphine retention is a multi-faceted problem that will likely require a constellation of interventions targeting multiple predictive factors. For example, a suite of actions may be necessary to address the multiple factors affecting this outcome such as raising the buprenorphine dose, providing counseling, or case management that could help with transportation or financial challenges. Third, several of the most important predictors (i.e., those contributing 3-4 percent to the model’s predictive performance) of early buprenorphine discontinuation are structural in nature. For example, the social vulnerability of the geographic area in which the Veteran lives (i.e., the socioeconomic factors, such as county-level poverty, that adversely affect communities) and travel distance to the closest VA facility each contribute 3 percent to the model’s predictive performance. Overall, these findings suggest machine learning-based predictive modeling has potential to help clinicians identify Veterans at risk of buprenorphine discontinuation and the specific needs they may have. This identification can help direct targeted interventions designed to keep Veterans in treatment longer, thereby improving their outcomes.

Research Next Steps

With the development and validation of the predictive model complete, we are now working with the VHA Program Evaluation and Resource Center (PERC) to adapt the predictive model into a user-centered clinical decision support tool for use by VHA providers prescribing buprenorphine for Veterans. The aim is for this model to be used by providers to continuously monitor Veterans’ risk of buprenorphine discontinuation. The tool will give providers a Veteran’s real-time risk score for buprenorphine discontinuation, his/her risk category (e.g., low/medium/high probability of buprenorphine discontinuation), modifiable and non-modifiable risk factors drivingthe Veteran’s risk score, and suggestions for strategies (e.g., counseling, adjusting buprenorphine dose) that may be helpful for keeping the Veteran in treatment longer. Further, to enhance the tool’s usefulness and acceptability, we are conducting focus groups with key stakeholders (i.e., Veterans receiving buprenorphine, buprenorphine providers, and operational partners) to get their perspectives on how to optimally present model information to providers. Next steps in this line of research are to conduct a pilot randomized controlled trial to determine the tool’s acceptability among providers and to obtain preliminary data on its effectiveness in clinical settings.

Overall, this line of research translates knowledge gained from population-level data and predictive analytics, using artificial intelligence, into potentially actionable strategies to improve Veterans’ care and outcomes, particularly for this high-risk, vulnerable population of Veterans with OUD. By accurately identifying who is most likely to discontinue buprenorphine early, we can position interventions to hopefully prevent discontinuation before it occurs, thereby getting the right care at the right time to Veterans.

References

1. Williams AR, et al. “Opioid Use Disorder Cascade of Care Framework Design: A Roadmap,” Substance Abuse 2022;43(1):1207-14.
2. Strombotne KL, et al. “Effect of a Predictive Analytics-Tar- geted Program in Patients on Opioids: A Stepped-Wedge Cluster Randomized Controlled Trial,” Journal of General Internal Medicine 2023;38, 375-81.
3. Hayes CJ et al. “Development and Validation of Ma- chine-learning Algorithms Predicting Retention, Over-doses, and All-cause Mortality among US Military Veterans Treated with Buprenorphine for Opioid Use Disorder,” Journal of Addictive Diseases 2024;1-18.