Study Explores Unintended Consequences of Local Implementation of VA Performance Measures

BACKGROUND:
VA’s implementation of a national clinical performance measurement (PM) system was associated with dramatic improvements in clinical outcomes. In recent years, scores on many VA measures have stabilized at high levels, suggesting that the benefit of these measures has been mostly realized. Therefore, within VA and other healthcare systems there might be more potential for improvement by minimizing the negative effects of PMs than by further improving already high scores. This qualitative study explored the possible relationships between a centralized primary care clinical PM system, facility-level practices to implement the PM system into daily care, and unintended negative consequences for Veterans. Investigators conducted interviews with primary care staff and facility leaders (n=59) at four VA facilities between 2/09 and 7/09. Interviews focused on the effects of PM on patients, providers, clinics, and facilities.

FINDINGS:

• Primary care staff described several ways in which PMs may lead to inappropriate care (e.g., over-prescribing of medication), decrease focus on Veterans’ concerns and patient service (e.g., inconveniencing patients for little benefit), and may make it more difficult for Veterans to make informed, value-consistent decisions (e.g., performance system doesn’t acknowledge when a patient makes an informed refusal of a recommended intervention).
• Staff also described unintended consequences on primary care team dynamics, e.g., requiring nurses to check on providers to be sure they completed and documented PMs, and providing performance bonuses based on PMs to providers, but not to nurses.
• In many instances, problems originated from local implementation strategies developed in response to national PM definitions and policies.
• Some noted benefits of PMs included feedback from the system helping some clinic staff feel more confident that their care was thorough, and performance scores as a source of pride and positive competition.

UPDATE:

• VA is currently making changes to the national PM system based on this and other research, e.g., developing new PMs that reward clinically appropriate action, even if the patient has not achieved specific targets, and developing clinical reminders that facilitate patient-centered decisions.

LIMITATIONS:

• The methodology used in this study was not well-suited for assessing the frequency with which the identified effects occur or evaluating the balance of positive and negative effects of performance measures.
• Data consisted of facility staff self-reports – and, in some cases, second-hand reports on the behavior of others.

NOTE:
Kenneth Kizer, M.D., M.P.H., former VA Under Secretary for Health, has written a Commentary addressing this topic that also will appear in this issue of JGIM.

AUTHOR/FUNDING INFORMATION:
This study was funded by HSR&D (IIR 07-140). Drs. Powell and Partin are part of HSR&D’s Center for Chronic Disease Outcomes Research, Minneapolis, MN.

Powell A, White K, Partin M, et al. Making the Grade: Unintended Consequences of Implementing a National Performance Measurement System into Local Practice. Journal of General Internal Medicine October 13, 2011;E-pub ahead of print.