A Review of the Evidence: Public Presentation of Health System or Facility Data about Quality and Safety

VA's "Open Government Plan" outlines the agency's commitment to transparency, and defines transparency as both increasing access to public information and enabling better engagement and advocacy on behalf of Veterans. Key elements of the transparency initiative involve public presentation of health system and facility data about quality of care and safety. Examples include VA's Hospital Compare website, which provides outcomes and process data for selected diagnoses, and the ASPIRE dashboard, which reports quality and safety goals for all VA hospitals.

Investigators at the VA Evidence-based Synthesis Program in West Los Angeles, CA conducted a review of the literature through July 2011, updating a 2008 systematic review by Fung and colleagues. This review sought information on whether making performance data publically available leads to improvements in quality of care and safety, and to summarize current research about patients' and families' use of performance data. After screening 370 articles, 18 new articles were included, with 37 articles from the prior review, in order to address the following four key questions:

Question #1 and Question #2
What is the most effective way of displaying quality and service information so that it is understandable? And, how do patients prefer to receive or access this information?

• To make the information more relevant to what consumers already understand and care about, public reports should:
  • Give an overall definition of quality,
  • Define the elements of quality and use them as the reporting categories, and
  • Include information about the sponsor and methods.
• Test the report with patients during development to learn what does and doesn't work. Cognitive interviews are the gold standard for testing surveys; for example, asking patients to explain, in their own words, what a label or symbol means.
• Make it easier for patients to understand comparative information,. For example, summarize and interpret data, highlight its meaning, and narrow options.
• Features of reports that patients could find helpful include:
  • Order by level of performance rather than alphabetically,
  • Use meaningful symbols instead of numbers,
  • Provide an overall summary measure, and
  • Include fewer reporting categories.

Question #3
What is the evidence that patients or their families use publicly reported quality and safety information to make informed healthcare decisions?

• Most studies in this review found the use of publically available data to be modest, at best. Although patients may show interest in public reports, in most cases interest does not seem to translate to actual use. The studies that do show use suggest that patients may avoid low-performing facilities or clinicians, but high-performers may not reap comparable benefits of public reporting.

Question #4
What is the evidence that public reporting of quality and safety information leads to improved quality or safety?

• There were relatively few new studies since the 2008 review that addressed the impact of reporting on quality improvement activities.
• Two new studies, in addition to the evidence from the prior review, support the conclusion that public reporting stimulates quality improvement activities.
• Five new studies address the effect of public reporting on a variety of outcomes (e.g. patient experience, performance measures, mortality). They show that public reporting has a positive impact on quality or safety outcomes, but the effect was small.

Authors' note
New evidence continues to support the conclusion of the previous review by Fung and colleagues, which found that "the effect of public reporting on effectiveness, safety, and patient-centeredness remains uncertain."

Cyber Seminar
A Cyber Seminar session on this ESP Report is scheduled for January 26, 2012 at 12:00 pm ET. Register here.

This report is a product of the HSR&D Evidence-based Synthesis Program (ESP), which was established to provide timely and accurate synthesis of targeted healthcare topics of particular importance to VA managers and policymakers -- and to disseminate these reports throughout VA.

Reference
Totten A, Miake-Lye I, Vaiana M, Beroes J, and Shekelle P. Public Presentation of Health System or Facility Data about Quality and Safety: A Systematic Review. VA-ESP Project #05-226;2011.

View the full report online

Please feel free to forward this information to others!

Read past HSR&D Management e-Briefs on the HSR&D website.

This Management eBrief is a product of the HSR&D Evidence Synthesis Program (ESP). ESP is currently soliciting review topics from the broader VA community. Nominations will be accepted electronically using the online Topic Submission Form. If your topic is selected for a synthesis, you will be contacted by an ESP Center to refine the questions and determine a timeline for the report.

This Management e-Brief is provided to inform you about recent HSR&D findings that may be of interest. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. If you have any questions or comments about this Brief, please email CIDER. The Center for Information Dissemination and Education Resources (CIDER) is a VA HSR&D Resource Center charged with disseminating important HSR&D findings and information to policy makers, managers, clinicians, and researchers working to improve the health and care of Veterans.

This report is a product of the HSR&D Evidence-Based Synthesis Program (ESP), which was established to provide timely and accurate synthesis of targeted healthcare topics of particular importance to VA managers and policymakers - and to disseminate these reports throughout VA.

See the full reports online.