BACKGROUND/RATIONALE:
Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, cross-hospital comparisons to identify under-performing hospitals are limited by differences in patient case-mix and illness severity. Clinicians consider the current approach to benchmarking with conventional regression to be unfair, unclear, and unhelpful. While substantial resources are devoted to hospital benchmarking, the current return on this investment is limited because clinicians do not understand or trust the methods. The National Academy of Medicine has recently called for investing into the science of performance measurement and for increasing its transparency and validity. Template matching has been proposed as an alternative methodological approach to benchmarking that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings.

OBJECTIVE(S):
To test the utility of template matching for comparing quality of care across VA's diverse acute care hospitals, this project will assess the feasibility, accuracy, and interpretability of this approach. Specifically, the project will: (A1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (A2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (A3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders.

METHODS:
Aim 1 will develop and refine the statistical methodology for benchmarking 30-day mortality in VA acute care hospitals using two template matching approaches: (1) a single template and (2) personalized templates for each hospital. Multiple statistical approaches to matching will be considered to achieve the fairest comparisons across hospitals. Aim 2 will use simulation and real patient data to measure when and why template matching and conventional regression approaches may yield discordant assessments of hospitals. Aim 3 will develop the presentation of template matching data, then survey VA Chiefs of Medicine to compare the interpretability and credibility of performance reports using template matching versus conventional regression for hospital benchmarking, with follow-up semi-structured interviews with a subset of Chiefs to further understand template matching performance reports from the perspective of Chiefs of Medicine.

FINDINGS/RESULTS:
We have developed initial SAS code necessary to run template matching using a single template, as well as the SAS code to automate the simulations in Aim 2. We have developed the data-generation process for 4 initial scenarios, in which we will test template matching versus conventional regression:
-Scenario #1: No differences in case-mix of quality of care across hospitals
-Scenario #2: Differences in case-mix, but not quality of care
-Scenario #3: Differences in quality of care, but not case-mix
-Scenario #4: Differences in both case-mix and quality of care.
Our preliminary findings indicate that template matching is equivalent to conventional regression for Scenario #1 (both methods have a low rate of false-positive outlier detection).

IMPACT:
The overarching goal of this proposal is to improve the care of hospitalized Veterans. This proposal will apply promising, new template matching approaches for benchmarking to the diverse VA healthcare system and complete a multi-faceted evaluation of these approaches. We will consider not only the feasibility and methodological rigor of template matching, but also the interpretability, credibility and accountability of the data. We expect that template matching will be feasible for benchmarking VA acute care hospitals, that it will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to end-users.

---

External Links for this Project

NIH Reporter

Grant Number: I01HX002546-01
Link: https://reporter.nih.gov/project-details/9502545

---

PUBLICATIONS:

Journal Articles

1. Hensley MK, Prescott HC. Bad Brains, Bad Outcomes: Acute Neurologic Dysfunction and Late Death After Sepsis. Critical care medicine. 2018 Jun 1; 46(6):1001-1002. [view]
2. Rogne T, Solligård E, Burgess S, Brumpton BM, Paulsen J, Prescott HC, Mohus RM, Gustad LT, Mehl A, Åsvold BO, DeWan AT, Damås JK. Body mass index and risk of dying from a bloodstream infection: A Mendelian randomization study. PLoS Medicine. 2020 Nov 1; 17(11):e1003413. [view]
3. Ketcham SW, Bolig TC, Molling DJ, Sjoding MW, Flanders SA, Prescott HC. Causes and Circumstances of Death among Patients Hospitalized with COVID-19: A Retrospective Cohort Study. Annals of the American Thoracic Society. 2021 Jun 1; 18(6):1076-1079. [view]
4. Miller HC, Liu VX, Prescott HC. Characteristics and Outcomes of Clinic Visits Immediately Preceding Sepsis Hospitalization. American Journal of Critical Care : An Official Publication, American Association of Critical-Care Nurses. 2021 Mar 1; 30(2):135-139. [view]
5. Hensley MK, Sjoding MW, Prescott HC. COUNTERPOINT: Should Corticosteroids Be Routine Treatment in Early ARDS? No. Chest. 2021 Jan 1; 159(1):29-33. [view]
6. Molling D, Vincent BM, Wiitala WL, Escobar GJ, Hofer TP, Liu VX, Rosen AK, Ryan AM, Seelye S, Prescott HC. Developing a template matching algorithm for benchmarking hospital performance in a diverse, integrated healthcare system. Medicine. 2020 Jun 12; 99(24):e20385. [view]
7. Prescott HC, Angus DC. Enhancing Recovery From Sepsis: A Review. JAMA. 2018 Jan 2; 319(1):62-75. [view]
8. Liu VX, Escobar GJ, Chaudhary R, Prescott HC. Healthcare Utilization and Infection in the Week Prior to Sepsis Hospitalization. Critical care medicine. 2018 Apr 1; 46(4):513-516. [view]
9. Vincent BM, Molling D, Escobar GJ, Hofer TP, Iwashyna TJ, Liu VX, Rosen AK, Ryan AM, Seelye S, Wiitala WL, Prescott HC. Hospital-specific Template Matching for Benchmarking Performance in a Diverse Multihospital System. Medical care. 2021 Dec 1; 59(12):1090-1098. [view]
10. Prescott HC, Costa DK. Improving Long-Term Outcomes After Sepsis. Critical care clinics. 2018 Jan 1; 34(1):175-188. [view]
11. McGrath BM, Takamine L, Hogan CK, Hofer TP, Rosen AK, Sussman JB, Wiitala WL, Ryan AM, Prescott HC. Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study. BMC health services research. 2022 Jun 3; 22(1):739. [view]
12. Munroe E, Prescott HC. Late Mortality From Sepsis: What We Know and What It Means. Critical care medicine. 2021 Feb 1; 49(2):353-355. [view]
13. Viglianti EM, Bagshaw SM, Bellomo R, McPeake J, Molling DJ, Wang XQ, Seelye S, Iwashyna TJ. Late Vasopressor Administration in Patients in the ICU: A Retrospective Cohort Study. Chest. 2020 Aug 1; 158(2):571-578. [view]
14. Wayne MT, Molling D, Wang XQ, Hogan CK, Seelye S, Liu VX, Prescott HC. Measurement of Sepsis in a National Cohort Using Three Different Methods to Define Baseline Organ Function. Annals of the American Thoracic Society. 2021 Apr 1; 18(4):648-655. [view]
15. Ketcham SW, Ice E, Molling DJ, Thompson AD, Sukul D, Thomas MP, Prescott HC. Noncardiac Organ System Dysfunction and Cause of Death Common Among Patients Admitted to the Cardiac Intensive Care Unit. Circulation. Cardiovascular quality and outcomes. 2020 Nov 12; 13(11):e007147. [view]
16. Prescott HC, Chang VW. Overweight or obese BMI is associated with earlier, but not later survival after common acute illnesses. BMC geriatrics. 2018 Feb 6; 18(1):42. [view]
17. Prescott HC, Carmichael AG, Langa KM, Gonzalez R, Iwashyna TJ. Paths into Sepsis: Trajectories of Presepsis Healthcare Use. Annals of the American Thoracic Society. 2019 Jan 1; 16(1):116-123. [view]
18. Prescott HC, Sussman JB, Wiersinga WJ. Postcritical illness vulnerability. Current opinion in critical care. 2020 Oct 1; 26(5):500-507. [view]
19. Prescott HC. Preventing Chronic Critical Illness and Rehospitalization: A Focus on Sepsis. Critical care clinics. 2018 Oct 1; 34(4):501-513. [view]
20. Govindan S, Prescott HC. Quick Sequential Organ Failure Assessment: Illness Severity Indicator, Clinical Decision Support Tool, or Both?. Critical care medicine. 2017 Nov 1; 45(11):1947-1949. [view]
21. Valbuena VSM, Seelye S, Sjoding MW, Valley TS, Dickson RP, Gay SE, Claar D, Prescott HC, Iwashyna TJ. Racial bias and reproducibility in pulse oximetry among medical and surgical inpatients in general care in the Veterans Health Administration 2013-19: multicenter, retrospective cohort study. BMJ (Clinical research ed.). 2022 Jul 6; 378:e069775. [view]
22. Royer S, Prescott HC. Reply to Azithromycin: Short Course with Long Duration. Journal of hospital medicine. 2018 Aug 1; 13(8):583. [view]
23. Prescott HC, Cope TM, Gesten FC, Ledneva TA, Friedrich ME, Iwashyna TJ, Osborn TM, Seymour CW, Levy MM. Reporting of Sepsis Cases for Performance Measurement Versus for Reimbursement in New York State. Critical care medicine. 2018 May 1; 46(5):666-673. [view]
24. Prescott HC, Kadel RP, Eyman JR, Freyberg R, Quarrick M, Brewer D, Hasselbeck R. Risk-Adjusting Mortality in the Nationwide Veterans Affairs Healthcare System. Journal of general internal medicine. 2022 Nov 1; 37(15):3877-3884. [view]
25. Royer S, DeMerle KM, Dickson RP, Prescott HC. Shorter Versus Longer Courses of Antibiotics for Infection in Hospitalized Patients: A Systematic Review and Meta-Analysis. Journal of hospital medicine. 2018 May 1; 13(5):336-342. [view]
26. Chopra V, Flanders SA, O'Malley M, Malani AN, Prescott HC. Sixty-Day Outcomes Among Patients Hospitalized With COVID-19. Annals of internal medicine. 2021 Apr 1; 174(4):576-578. [view]
27. Prescott HC, Levy MM. Survival From Severe Coronavirus Disease 2019: Is It Changing? Critical care medicine. 2021 Feb 1; 49(2):351-353. [view]
28. Vincent BM, Wiitala WL, Luginbill KA, Molling DJ, Hofer TP, Ryan AM, Prescott HC. Template matching for benchmarking hospital performance in the veterans affairs healthcare system. Medicine. 2019 May 1; 98(20):e15644. [view]
29. Wayne MT, Seelye S, Molling D, Wang XQ, Donnelly JP, Hogan CK, Jones MM, Iwashyna TJ, Liu VX, Prescott HC. Temporal Trends and Hospital Variation in Time-to-Antibiotics Among Veterans Hospitalized With Sepsis. JAMA Network Open. 2021 Sep 1; 4(9):e2123950. [view]
30. Meyer N, Harhay MO, Small DS, Prescott HC, Bowles KH, Gaieski DF, Mikkelsen ME. Temporal Trends in Incidence, Sepsis-Related Mortality, and Hospital-Based Acute Care After Sepsis. Critical care medicine. 2018 Mar 1; 46(3):354-360. [view]
31. Carlton EF, Prescott HC. The After Shock-Reduced Health-Related Quality of Life Following Sepsis. Pediatric critical care medicine : a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies. 2020 Oct 1; 21(10):899-901. [view]
32. Haak BW, Prescott HC, Wiersinga WJ. Therapeutic Potential of the Gut Microbiota in the Prevention and Treatment of Sepsis. Frontiers in immunology. 2018 Sep 10; 9:2042. [view]
33. Sedhai YR, Yuan M, Ketcham SW, Co I, Claar DD, McSparron JI, Prescott HC, Sjoding MW. Validating Measures of Disease Severity in Acute Respiratory Distress Syndrome. Annals of the American Thoracic Society. 2021 Jul 1; 18(7):1211-1218. [view]
34. Chopra V, Flanders SA, Vaughn V, Petty L, Gandhi T, McSparron JI, Malani A, O'Malley M, Kim T, McLaughlin E, Prescott H. Variation in COVID-19 characteristics, treatment and outcomes in Michigan: an observational study in 32 hospitals. BMJ open. 2021 Jul 23; 11(7):e044921. [view]
35. Wang XQ, Vincent BM, Wiitala WL, Luginbill KA, Viglianti EM, Prescott HC, Iwashyna TJ. Veterans Affairs patient database (VAPD 2014-2017): building nationwide granular data for clinical discovery. BMC medical research methodology. 2019 May 8; 19(1):94. [view]

Journal Other

1. Royer S, Prescott HC. Next Steps for Confirming Bronchoalveolar Lavage Amlyase as an Useful Biomarker for Ventilator-Associated Pneumonia. [Editorial]. Critical care medicine. 2018 Jan 1; 46(1):165-166. [view]
2. Prescott HC, Angus DC. Postsepsis Morbidity. JAMA. 2018 Jan 2; 319(1):91. [view]
3. Donnelly JP, Wang XQ, Iwashyna TJ, Prescott HC. Readmission and Death After Initial Hospital Discharge Among Patients With COVID-19 in a Large Multihospital System. [Letter to the Editor]. JAMA. 2021 Jan 19; 325(3):304-306. [view]
4. Hensley MK, Sjoding MW, Prescott HC. Rebuttal From Dr Hensley et al. Chest. 2021 Jan 1; 159(1):34-35. [view]
5. Prescott HC, Sussman JB. Smarter Use of Corticosteroids in Treating Patients with Septic Shock. JAMA Network Open. 2020 Dec 1; 3(12):e2029323. [view]

DRA: Health Systems
DRE: Treatment - Observational, Research Infrastructure, TRL - Applied/Translational
Keywords: Information Management, Quality Indicators
MeSH Terms: none