Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

CRE 12-313 – HSR Study

 
CRE 12-313
Cognitive Support Informatics for Antimicrobial Stewardship
Peter A Glassman, MBBS MSc
VA Greater Los Angeles Healthcare System, West Los Angeles, CA
West Los Angeles, CA
Funding Period: December 2013 - August 2018
BACKGROUND/RATIONALE:
Inappropriate use of antibiotics leads to increased antimicrobial resistance and healthcare costs. Antimicrobial stewardship programs (ASPs) encourage evidence-based decisions, ensure optimal dosing and limit unintended consequences. However, best practices for ASP interventions are uncertain as supporting evidence is limited and formal studies on decision-making processes of ASP teams and targeted users are lacking. Numerous surveys indicate variations in ASP structure and processes and suggest the need for better informatics tools.

OBJECTIVE(S):
We assessed antimicrobial stewardship activities at individual VA medical centers nationwide and developed decision support tools to promote and enhance antimicrobial stewardship activities. Our three aims were:

Aim 1: Characterize existing antimicrobial stewardship structural aspects and practices that are predictive of quality metrics for antimicrobial use.

Aim 2: Develop measures of antimicrobial use for visual analytic displays and summary reports that are clinically aligned for cognitive support of, and useful to, antimicrobial stewards.

Aim 3: Pilot test an implementation program and cognitive support tools (aka visual analytic displays) at eight VA facilities. Evaluate tools and efforts to improve antimicrobial stewardship.

METHODS:
In Aim 1, independent variables characterizing antimicrobial stewardship were derived primarily from the 2012 Healthcare Analysis & Information Group (HAIG) survey on ASPs at each VA site. Factor analysis was utilized as were four specific quality metrics for antimicrobial use (overall antimicrobial usage, antimicrobial usage among patients with a primary noninfectious etiology for admission, and missed opportunities for avoiding double anaerobic coverage and for converting intravenous to oral antibiotic therapy). We conducted exploratory multivariate analyses to assess associations between quality metrics and the model set of antimicrobial stewardship characteristics.

In Aims 2 and 3, we conceived, developed and improved two sets of visual display tools for antimicrobial stewards, using SSAS and Pyramid Analytics. The tools were implemented at eight pilot sites in 2016: Greater Los Angeles, Boise, Salt Lake City, Cincinnati, Madison, Boston, San Antonio, and Houston.

The first visual tool set was based on the Centers for Disease Control and Prevention's Antimicrobial Utilization Module of the National Health Safety Network (NHSN). This provides antimicrobial utilization benchmarking data (via Standardized Antimicrobial Administration Ratios or SAARs) whereby individual facilities are identified as relatively high or low users in several antimicrobial classes, stratified by intensive care units and acute care wards. Our visual analytic system expands on this, using visual reports of longitudinal and/or comparative antibiotic use to other VA facilities. Stewards can use pre-designed or customized summary reports and interactive displays. The reports are self-contained, displaying usage metrics (Days of Therapy per 1000 patient days), for monitoring use and/or tracking interventions over time.

The second visual tool set describes inpatient antibiotic utilization using a novel analytic framework in treating common infectious diseases: pneumonia, skin and soft tissue infection, and urinary tract infection. We examined antibiotic decisions at three branch points: empiric selection (Choice; days 0-2), de-escalation (Change, days 3-4), and discontinuation (Completion, days 5 onward). We used structured and unstructured data to identify the syndromes (also using admission and discharge diagnoses) and concordant antimicrobial utilization at each of the branch points. Dividing antibiotic usage into these discrete points allows stewards to assess and target specific patterns of potential antimicrobial overuse at different time points including total duration of antibiotics.

Methodologically, we took a multi-pronged approach in the testing and improving the visual tools: 1. qualitative (usability) assessments of stewards' tool use was done using cognitive task analysis; 2. individual meetings with site stewards regarding use of tools; and 3. collaborative calls with intervention sites to share tools and methods. Quantitative analyses are discussed below. Aim 3 also included a provider survey on antimicrobial experiences and attitudes.

FINDINGS/RESULTS:
Aim 1:

Five factors were associated with at least 3 potentially favorable outcomes (e.g., reduced antibiotic use and/or reduced missed chances): presence of postgraduate physician/pharmacy training programs, number of antibiotic-specific order sets that were present in the electronic medical record, degree to which ASPs perform systematic de-escalation review, presence of pharmacists and/or Infectious Diseases (ID) attendings on acute care ward teams, and formal ID training of the lead ASP pharmacist. We concluded that formalization and presence of ID expertise was associated with potentially favorable antibiotic-related outcomes.


Aim 2 & 3:

We used a generalized estimation equation with Poisson distribution to model the antibiotic use outcome for three metrics (total usage, anti-MRSA and anti-pseudomonal antibiotic usage, per 1000 patient days) as a function of the intervention phase and intervention site indicator. Overall, results suggest an approximately 5% decrease in antimicrobial use rates at intervention sites compared to other VA facilities.

1) For total antibiotic usage, non-intervention sites had an average 2.5% increase (p-value=0.0026) while intervention sites had an average 2.1% decrease (p-value=0.2529) from non-intervention phase (before Jan 2016) to intervention phase (after July 2016). The difference in the percentage change between non-intervention and intervention sites is statistically significant (p-value=0.025).

2) For total antibiotic use for MRSA, non-intervention sites had an average 6.6% decrease (p-value<0.0001) while intervention sites had an average 11.3% (p-value<0.0001) decrease from non-intervention phase to intervention phase. The difference in the percentage change between non-intervention and intervention sites shows a trend for significance (p-value=0.092).

3) For total anti-pseudomonal antibiotic use, non-intervention sites had an average 3.6% increase (p-value<0.0001) while intervention sites had an average 3.4% (p-value<0.0001) decrease from non-intervention phase to intervention phase. The difference in the percentage change between non-intervention and intervention sites is statistically significant (p-value=0.018).

IMPACT:
We developed simplified and standardized visual tools, based on our CREATE project, to allow any VA facility to access their antibiotic usage. To disseminate these, we have used VA Antimicrobial Stewardship Task Force webinars, list-serves, and our VA Pulse site. Webinar launch in July 2018 included over 200 attendees; subsequently, approximately 30 facilities have signed up for the automated reports. We are conducting quarterly conferences calls to support interested sites in their use of these reports.


External Links for this Project

NIH Reporter

Grant Number: I01HX001155-01
Link: https://reporter.nih.gov/project-details/8495192

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project

PUBLICATIONS:

Journal Articles

  1. Graber CJ, Jones MM, Chou AF, Zhang Y, Goetz MB, Madaras-Kelly K, Samore MH, Glassman PA. Association of Inpatient Antimicrobial Utilization Measures with Antimicrobial Stewardship Activities and Facility Characteristics of Veterans Affairs Medical Centers. Journal of hospital medicine. 2017 May 1; 12(5):301-309. [view]
  2. Chou AF, Graber CJ, Jones M, Zhang Y, Goetz MB, Madaras-Kelly K, Samore M, Kelly A, Glassman PA. Characteristics of Antimicrobial Stewardship Programs at Veterans Affairs Hospitals: Results of a Nationwide Survey. Infection control and hospital epidemiology. 2016 Jun 1; 37(6):647-54. [view]
  3. Graber CJ, Jones MM, Goetz MB, Madaras-Kelly K, Zhang Y, Butler JM, Weir C, Chou AF, Youn SY, Samore MH, Glassman PA. Decreases in Antimicrobial Use Associated With Multihospital Implementation of Electronic Antimicrobial Stewardship Tools. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2020 Aug 22; 71(5):1168-1176. [view]
  4. Howard-Anderson J, Schwab KE, Chang S, Wilhalme H, Graber CJ, Quinn R. Internal medicine residents' evaluation of fevers overnight. Diagnosis (Berlin, Germany). 2019 Jun 26; 6(2):157-163. [view]
  5. Graber CJ, Simon AR, Zhang Y, Goetz MB, Jones MM, Butler JM, Chou AF, Glassman PA. Performance of infectious diseases specialists, hospitalists, and other internal medicine physicians in antimicrobial case-based scenarios: Potential impact of antimicrobial stewardship programs at 16 Veterans' Affairs medical centers. Infection control and hospital epidemiology. 2023 Mar 1; 44(3):400-405. [view]
  6. Taber P, Weir C, Butler JM, Graber CJ, Jones MM, Madaras-Kelly K, Zhang Y, Chou AF, Samore MH, Goetz MB, Glassman PA. Social dynamics of a population-level dashboard for antimicrobial stewardship: A qualitative analysis. American journal of infection control. 2021 Jul 1; 49(7):862-867. [view]
  7. Chou AF, Graber CJ, Zhang Y, Jones M, Goetz MB, Madaras-Kelly K, Samore M, Glassman PA. Specifying an implementation framework for Veterans Affairs antimicrobial stewardship programmes: using a factor analysis approach. The Journal of antimicrobial chemotherapy. 2018 Sep 1; 73(9):2559-2566. [view]
Journal Other

  1. Goetz MB, Graber CJ, Jones M, Madaras-Kelly K, Samore MH, Glassman PA. Broad spectrum antibiotic use at Choice, Change, and Completion throughout VA: patterns of initiation and de-escalation. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(suppl_1):S250-S251. [view]
  2. Kean J, Butler JM, Bunker L, Goetz MB, Glassman PA, Jones M, Graber CJ, Madaras-Kelly K, Weir C. Causal conditions supporting antibiotic stewardship information dashboards. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(Supplemental 1):S326-S327. [view]
  3. Chou AF, Zhang Y, Jones M, Graber CJ, Goetz MB, Madaras-Kelly K, Samore M, Kelly A, Glassman PA. Effects of policy and resources on antimicrobial stewardship interventions in the VA: applying a transaction cost economics framework. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(Supplemental 1):S254. [view]
  4. Sutton JD, Graber CJ, Madaras-Kelly K, Jones M, Glassman PA, Spivak E, Goetz MB. Exploring visual analytic tools for antimicrobial stewardship intervention across 8 Veterans Affairs hospitals. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(Supplemental 1):S512. [view]
  5. Jones M, Khader K, Huttner B, Graber CJ, Zhang Y, Samore M, Madaras-Kelly K, Goetz MB, Glassman PA. How to use antimicrobial use data? A model to support decision-making and facilitate understanding. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(Supplemental 1):S249-S250. [view]
  6. Graber CJ, Goetz MB. Next steps for antimicrobial stewardship. The Lancet Infectious Diseases. 2016 Jul 1; 16(7):764-765. [view]
  7. Weir C, Butler J, Sutton JD, Graber CJ, Goetz MB, Madaras-Kelly KJ, Jones M, Samore M, Glassman PA. The experience of stewards in using a visual analytic tool to benchmark and track therapy duration for pneumonia, urinary tract infections, and skin and soft tissue infections. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(Supplemental 1):S279-S280. [view]
  8. Sutton JD, Graber CJ, Goetz MB, Madaras-Kelly K, Jones M, Glassman PA, Spivak E. Visual analytic tools for automated measurement and tracking of durations of therapy for pneumonia, urinary tract infections, and skin and soft tissue infections. [Abstract]. Open forum infectious diseases. 2017 Oct 4; 4(Supplemental 1):S511-S512. [view]
Conference Presentations

  1. Graber CJ, Chou AF, Jones MM, Zhang Y, Goetz MB, Madaras-Kelly K, Kelly AA, Samore MH, Glassman PA. Antimicrobial stewardship activities within VA in 2012. Poster session presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 8; Philadelphia, PA. [view]
  2. Graber CJ, Jones MM, Chou AF, Zhang Y, Goetz MB, Madaras-Kelly K, Samore MH, Glassman PA. Association of antimicrobial stewardship activities with acute care antimicrobial usage at Veterans Affairs (VA) Medical Centers, 2012. Paper presented at: ID Week Annual Conference; 2015 Oct 7; San Diego, CA. [view]
  3. Chou A, Graber C, Goetz MB, Jones MM, Madaras-Kelly K, Glassman P. Characteristics of antimicrobial stewardship programs at Veterans Affairs hospitals: Results of a nationwide survey. Poster session presented at: AcademyHealth Annual Research Meeting; 2015 Jun 14; Minneapolis, MN. [view]
  4. Bohan JG, Goetz MB, Graber CJ, Jones M, McClain S, Spivak E, Jahng M, Samore M, Chou A, Glassman P, Madaras-Kelly K. Development and impact of visual analytics to improve antimicrobial use. Poster session presented at: ID Week Annual Conference; 2016 Oct 26; New Orleans, LA. [view]
  5. Chou AF, Graber CJ, Jones MM, Zhang Y, Goetz MB, Madaras-Kelly K, Samore M, Glassman PA. Specifying an Implementation Framework for VA Antimicrobial Stewardship Programs. Paper presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 8; Philadelphia, PA. [view]


DRA: Health Systems Science
DRE: Pathology, Diagnosis, Treatment - Efficacy/Effectiveness Clinical Trial
Keywords: none
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.