2038. Predicting the Quality of Preventive Care:  A Prospective Evaluation of Three Measurement Methods
TR Dresselhaus, HSR&D REAP-Center for Research in Patient-Center Care, VA San Diego Healthcare System, J Luck, Center for the Study of Healthcare Provider Behavior, School of Public Health, University of California, Los Angeles, VA Greater Los Angeles Healthcare System, DS Bertenthal, San Francisco VA Research Enhancement Award Program, JW Peabody, San Francisco VA Research Enhancement Award Program, Institute for Global Health, University of California, San Francisco and Los Angeles

Objectives: To prospectively evaluate variation in quality of preventive care at multiple sites using standardized patients chart abstraction and vignettes and to determine whether clinical vignettes predicts these variations.

Methods: 72 physicians in 2 VAMC primary care clinics and 2 community clinics were randomly selected among consenting physicians (95%).  We compared 3 measurement methods for 4 common conditions:  (1) standardized patients (SPs) presenting unannounced to physicians’ clinics; (2) abstraction of the SP medical records and; (3) clinical vignettes that exactly corresponded to the SPs.  Physicians completed 480 visits.  Scoring criteria were based on national guidelines for 12 prevention measures and categorized as Vaccines, Vascular, Cancer Screening, or Personal Habits.  We calculated the proportion of prevention items completed for the 3 methods.  We also developed a multiple regression model to predict performance using half the data; subsequently, this model was applied to the remaining data to determine if site, training level, or clinical condition predicted quality of preventive care.

Results:  Measurements of the quality of preventive care ranged from 57% (SPs) to 54% (vignette) to 46% (chart abstraction).  Vignettes matched or exceeded SP scores for 3 of 4 categories (Vaccine, Vascular, Cancer Screening); charts were lowest in all 4 categories.  We found significant variation in most sites (p <.05 for SPs and charts at 4 sites, and p <.05 for Vignettes at 3 sites), conditions (p <.05 for SPs and charts for 3 conditions, and p <.05 for vignettes for 4 conditions) but no difference by  training level. Quality of preventive care at VA sites was superior to community sites (vignettes: p < 0.05).

Conclusions: These data indicate overall poor quality of preventive care regardless of measurement method.  Chart abstraction, as a method, appears to be subject to recording bias.  As hypothesized, quality varies significantly depending upon site and condition, but not training level.  Clinical vignettes are comparable to SPs in measuring variation in quality and therefore may be useful for evaluating preventive care performance.

Impact: Quality of preventive care could be improved in the VA.  Clinical vignettes offer a comparatively inexpensive and case-mix controlled method to assess and predict preventive care quality.