» Back to Table of Contents

Response to Commentary

Realizing the Opportunities and Avoiding the Pitfalls of Artificial Intelligence to Achieve Improved Healthcare for Veterans

Artificial Intelligence (AI) has experienced explosive growth in the last 15 years, from foundational innovations in deep learning (DL) and then again in the last four years with layered-on advances in generative AI. With the success of AI for a variety of objectives and tasks, the academic, government, and tech industries have made massive AI investments. These investments have, in turn, created a mind-boggling boom in innovations and applications across almost every workforce sector.

Within health and healthcare, AI is poised to impact myriad business and clinical processes, from scheduling, chatbot support for patient care and prior care interpretation, and supporting ambient dictation documentation from both the patients’ and providers’ perspectives, to cognitive support in diagnostics and treatment, among many others. Some health AI technologies have flourished and are approaching stages of maturity that are already trending towards sustained use. For example, as of May 2024, 76 percent of all FDA AI-enabled medical devices were in AI imaging technologies in radiology, and the accuracies and performance of technologies in that domain are impressive. However, these technologies are the tip of the health AI iceberg, as many clinical AI applications are not considered FDA devices if they operate on information for clinical providers in a cognitive or workflow support capacity. In addition, AI classified as a medical device does not encompass business workflow applications that are already substantially impacting the healthcare industry.

In their commentary article, Drs. Alterovitz and Makridis point out existing strengths of the VA AI research community and provide recommendations for synergies within AI research and development. They highlight AI research areas in which it is exciting to see VA’s prominence, as many of these are strategic priorities for our care delivery system and research enterprises, such as population health and preventive maintenance, rehabilitation medicine, men’s health, oncology, cardiovascular disease, dermatology, genomic research, and mental health. To provide just a few examples, large investments in genomics research, such as the Million Veterans Program, have necessarily created innovations in leveraging AI at scale to generate inference and new discovery from massive multi-modal data resources. The Precision Oncology Program, which includes clinical genomics and cancer programs, has multiple AI projects across clinical domains in development and early use. Also, discriminative AI predictive models have been widely deployed to the field, such as REACH VET for suicide risk prediction and CAN for mortality and hospitalization risk prediction.

Within this larger AI R&D context, compelling opportunities and needs for health AI research exist for Health Systems Research (HSR). Health AI research aligns to multiple ORD strategic priorities, including most directly putting VA data to work for Veterans, but also increasing the real-world impact of VA research, which is central to HSR’s mission.

Critical gaps and limitations in many of these technologies remain, as also noted by Drs. Alterovitz and Makridis in their ethical AI synergies section. In addition to established challenges in discriminative AI, generative AI brings distinctly new capacities and failure modes to health AI. Challenges in both types of data-driven AI have been highlighted in recent literature across many clinical domains, such as issues in bias and fairness in racial, ethnic, and clinical sub-populations, performance degradation over time, as well as hallucinations and incorrect results. Some research and operational efforts, such as those led by the National AI Institute, and national clinical program offices, are underway to ensure fair and trustworthy AI applications in VA in alignment with the Executive Order for Safe, Secure, and Trustworthy AI, the OMB Memorandum, and HHS Plan for Promoting Responsible use of AI. Strong investments in translational and applied research into solving these issues for VA and Veteran healthcare are critical to long-term health AI success, and VA has the scope and breadth of clinical care coupled with a mature digital data infrastructure to provide one of the best integrated care environments to conduct this work.

However, this is not the only domain that is critical to the success of health AI in VA. Successful AI integration into clinical care has been very limited compared to the volume of research in developing health AI algorithms and models. Much of this gap stems from a lack of effective adaptation, implementation, and scaling due to challenges in sociotechnical issues, such as a lack of patient and user engagement, limited clinical needs alignment, workflow integration challenges, and misalignments with user cognitive preferences and needs.

For these reasons, it is useful to frame some of the health AI research opportunities within an AI Implementation Lifecycle (Matheny, et al NAM, 2022), which aligns learning health system (LHS) principles, implementation science, human-computer interaction, and user-centered design domains to support the conceptualization, development, implementation, and sustainment of AI applications in use.¹ The LHS is a core framework for HSR research, which uses a systematic and data-driven approach to generating and delivering evidence and care to Veterans.²;This AI lifecycle framework, such as the international Organization for Economic Cooperation and Development’s AI system classification and lifecycle, places emphasis on establishing the use case need, the gaps in the current state, and the desired state with measurable outcomes prior to deciding what technology is the most appropriate to solve the challenges. This framework also emphasizes the need for patient and user engagement and involvement throughout the development and implementation process, and a corresponding focus on issues of workflow integration, aligning to user preferences, and issues of both statistical and clinical performance monitoring and correction to sustain the solution safely as workflows, clinical practice, health system policies, and data collection practices change over time. Many AI applications fail to achieve the clinical targets even with strong underlying technical accuracy, and one of the largest impact opportunities for VA in Health AI is to develop and lead in methods, tools,and approaches to ensure safe, equitable, and trustworthy integration of AI into practice.

Lastly, this type of applied research is not feasible without strong partnerships with AI leadership in VA’s Office of Information and Technology and the newly organized VHA Digital Health Office, which includes responsibility for AI and emerging technologies, led by the National AI Institute. For research efforts to facilitate measurable improvements in VA care, AI must be safely and appropriately integrated into use, evaluated, and iteratively improved to meet the needs of patients, healthcare personnel, and the organization. This must also be done in the context of evolving regulations, requirements, and best practices, in a field moving at a tremendous pace.

Looking at the VA HSR-funded projects portfolio, several QUERI centers/projects and COINs have significant portfolios of investigator-initiated and operational- partnered work in health system facing or integrated AI, and the scope and number of these projects have grown rapidly in recent years. Our newly funded COIN, Veterans’ Wellbeing through Innovation Systems Science and Experience in Learning Health Systems (VETWISE-LHS) at Tennessee Valley Healthcare System VA (Nashville, Tennessee), is focused on developing and implementing approaches and interventions to improve Veteran care in the context of LHS, with informatics and applied AI as an area of focus.

In conclusion, health AI is positioned to create sustained and impactful improvements in Veteran health and healthcare, but the course is not yet clear, and the path is not well trodden. To realize the potential of health AI in VA, important work remains to be done in both translational research and applied operations- partnered implementations and evaluations. VA provides a world-class environment to do this work, and AI has the potential to support delivery of best-in-class care to our Veterans.

References

1. Matheny ME, Thadaney-Israni S, Ahmed M, Whicher D, Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril. National Academy of Medicine, 2022. Washington, DC: The National Academies Press.
2. Kilbourne AM, Borsky AE, O’Brien RW, et al.“The Foundational Science of Learning Health Systems,” Health Services Research 2024;1-7.