» Back to Table of Contents

Research Highlight

Measuring Quality of Care in the Era of Veterans' Choice

In the 1990s, the Veterans Health Administration (VHA) underwent a radical transformation, from a health system widely derided for its poor quality to "the best care anywhere."¹ Central to this transformation were efforts to measure and track quality of care.² Today, VHA has the most extensive tracking and reporting system of any healthcare system in the country, and perhaps the world. However, the Veterans Access, Choice, and Accountability Act of 2014 (also referred to as the Veterans Choice Act or VCA) has created substantial challenges for monitoring quality of care. Indeed, in the era of VCA, our healthcare system is undergoing yet another transformation, this time moving from being a provider of care to a purchaser of care. As providers of care, we had access to detailed electronic health record data and the ability to leverage these data to assess and promote quality through a robust performance management system. But how will we ensure that Veterans who receive services from non-VHA providers are getting the same high-quality care? In this new era, it is imperative that we develop methods to identify, prioritize, and track care in both VHA and non-VHA settings.

Studying the Implications of VCA

Recognizing the impending new challenges under VCA, HSR&D's Quality Enhancement Research Initiative (QUERI) issued a Request for Applications to study the potential effects of VCA on various aspects of healthcare delivery. The Center for Clinical Management Research, a VA HSR&D Center of Innovation, was funded to study the implications of VCA for quality of care. One of the key aims of this study was to develop a streamlined, transparent, and reproducible approach to identify and prioritize performance measures of underuse and overuse relevant to VCA.

Adapting the RAND/UCLA method used for the development of Quality of Care Assessment Tools, the project proceeded in three main steps: 1) identification of clinical areas, 2) an environmental scan, and 3) rating using modified Delphi panels.³ We first assembled an expert council comprising six national VHA clinical and policy leaders. Council members were provided with a list of the most prevalent diagnoses and procedures for Veterans receiving care through VHA. Using this information, they collaboratively identified clinical areas of potential importance to VCA participants. Following this initial meeting, the project team refined the list of clinical areas. Council members then individually rated the clinical areas based on improvement opportunity and feasibility of measurement. The top eight clinical areas were prioritized for a formal environmental scan. These included diagnosis, treatment, and screening or surveillance for: back pain, cardiac testing, diabetes, gastrointestinal procedures, headaches, hepatitis C, prostate cancer, and post-traumatic stress disorder.

After the selection of clinical areas, two team members conducted a rapid environmental scan to identify measures, guidelines, and recommendations related to the clinical areas. Team members reviewed particular highquality data sources, such as National Quality Forum-endorsed performance measures, American College of Physicians guidelines, VA guidelines, and Choosing Wisely recommendations. A collaborative process was used by the study team to select approximately five measures or recommendations per clinical area.

Role of Expert Council

In order to obtain ratings for each of the recommendations, we first expanded the expert council from 6 to 10 members to ensure expertise in each clinical area. Members of the council were then provided with information derived from our environmental scan for each measure or recommendation, including the data source, a brief description, and supporting evidence. After reviewing these materials, members pre-rated each measure or recommendation on validity, feasibility of measurement, and improvement opportunity. Using a modified Delphi panel process during a virtual meeting—facilitated through the use of a collaborative software platform—expert panel members reviewed, discussed, and then re-rated each recommendation. To support future measure development, we queried the council for suggestions on: 1) how each recommendation could be adapted and modified into a formal performance measure; and 2) whether quantitative data on improvement opportunity would be helpful for prioritizing measures in the future.

The expert council reviewed 35 measures and recommendations. The council identified 29 measures and recommendations with high validity (median panel rating ≥ 7 on a 1-9 scale), indicating that they should be prioritized for quality monitoring (see table at: www. annarbor.hsrd.research.va.gov/vcatable. asp). Of course, additional work is needed to implement the recommendations prioritized by the expert council. In some cases, the areas identified are important, but existing measures may fall short of being ready for implementation. Nonetheless, we believe that information obtained from this project will aid efforts to ensure that Veterans utilizing community care get the most appropriate care possible.

In summary, VHA is undergoing yet another transformation, one that will create new challenges for monitoring the quality of care for our Veterans. Using a methodical and reproducible process, we convened an expert panel to identify measures and recommendations that should be considered for assessing quality of care received in non-VHA settings. While implementation of tracking and monitoring systems from our findings will require additional work, these efforts can serve as a starting point for those who seek to assess and improve quality of care in this new era. Moreover, the process used in our work can be adapted to other contexts where rapidly and systematically identifying and prioritizing performance measures is of importance.

References

1. Jha AK, Perlin JB, Kizer KW, et al. "Effect of the Transformation of the Veterans Affairs Health Care System on the Quality of Care," New England Journal of Medicine 2003; 348(22):2218-27.

2. Kerr EA, Gerzoff RB, Krein SL, et al. "Diabetes Care Quality in the Veterans Affairs Health Care System and Commercial Managed Care: the TRIAD Study," Annals of Internal Medicine 2004; 141(4):272-81.

3. Survey tool available at http://www.rand.org/health/surveys_tools/qatools.html