1150 — Clinical Supervision Index: Measuring Supervision of Physician Residents in VA Medical Centers

Perez EV, VA Loma Linda Healthcare System; Byrne JM, VA Loma Linda HCS and Loma Linda University School of Medicine; Durkin R, OI&T, Loma Linda, CA; Wicker AB, Office of Academic Affiliations, VA Central Office; Henley SS, Martingale Research Corporation, Plano, TX; Golden RM, University of Texas at Dallas, Richardson, TX; Hoffman KA, OI&T / Allocation Resource Center, Braintree, MA; Hinson RS, OI&T, VA Central Office; Aron DC, Louis Stokes Cleveland VA Medical Center; Baz S , Loma Linda University School of Medicine and VA Loma Linda HCS

Objectives:
We assessed the feasibility of VHA Office of Academic Affiliations (OAA) Clinical Supervision Index (CSI), a performance metric with 13 subscales quantifying the intensity by which clinical trainees involved in patient care were supervised at VA medical centers (IIR#08-164, IIR#15-071, OIandT-Greenfield#236). CSI is mathematically theory-driven (defining optimal patient-centered supervision) that specifies analytic models of algorithmically scored survey data electronically administered in CPRS by MS COM Object software to trainees and their supervisors whenever registered trainees enter progress notes (the unit of analyses) into health records of active patients (randomly and purposively selected). CSI scores are merged with trainee and supervisor registry information and patient's CPT, DRG, ICD9/10, demographic VISTA data, plus the progress note text.

Methods:
Feasibility was assessed qualitatively with focus groups and quantitatively by comparing CSI scores with trainee satisfaction ratings from OAA's Learners' Perceptions Survey and determining construct validity (testing theory-derived hypotheses) with data from VA Loma Linda HCS where CSI has been operational since August 2013.

Results:
We assessed CSI performance on 600 encounters (progress notes) involving 40 continuity care residents and 75 supervisors. Focus groups revealed trainees understood CSI purpose. Resident satisfaction with VA clinical learning and preceptor/faculty access was higher following CSI implementation. Supervision scores were associated with clinic workload (RVU/staff), care complexity (RVU/case), trainee PGY level, and supervisor experience (years supervising) in models testing positive for specification (equivalence of Outer Product Gradient and Hessian estimates of covariance matrix).

Implications:
CSI is feasible for VA residents in primary care clinics.

Impacts:
To ""¦ improve the access of veterans to medical services"¦" the veterans' Choice Act (PL 113-146) requires VA to increase within five years the number of residents who rotate through VA medical centers by 11% (1,500 new positions for 4,500 residents). By 2019, 38% (45k/120k) of all US residents will rotate through VA medical centers. CSI is a critical tool to help VHA leadership evaluate how health trainees are supervised, ensuring safe and effective care to patients and optimal clinical experiences to trainees. Favorable learning experiences are tied to VA's ability to recruit/retain clinical staff who understands veterans' care needs.