BACKGROUND/RATIONALE:
Valid and cost effective measures of quality are needed to assess physician competency and systems effects on clinical practice.

OBJECTIVE(S):
To further develop a valid, responsive, case mix adjusted and inexpensive method to measure the quality of care.To further develop a valid, responsive, case mix-adjusted, and inexpensive method to measure the quality of care, physicians’ management of computerized clinical vignette scenarios were compared to the "gold standard" of standardized patients (SPs) and to abstracted medical records of the SP visits.

METHODS:
We used a randomized, controlled design to compare vignette to standardized patients measurement of the quality of care in 4 outpatient clinics. Measurements were done for four common acute and chronic diseases that are common in the primary care setting. Two different clinical scenarios are being developed for each of the four diseases. Clinical vignettes and actor patient scripts are also being developed to reflect these scenarios. The measurement criteria, appropriate to each scenario, are based on national guidelines and further reviewed by expert panels.
Subjects were general internal medicine residents (excluding interns) and faculty providing care in the primary care clinics at two sites in Los Angeles and two sites in San Francisco VA Medical Centers. After obtaining consent from over 95% of eligible physicians, 10 physicians are being randomly selected at each site to receive vignettes and see standardized patients. These "patients" are experienced actors who underwent a rigorous training protocol and ongoing quality assessment. The standardized patients are being scheduled into regular clinic slots without notification to the physician. Each participating physician will complete all eight vignettes and see eight standardized patients. To assure the accuracy of the gold standard, a random selection of visits for each actor will be record and then compared with patient reports.
Vignettes and chart abstractions are being scored by trained abstractors and compared to the quality criteria. The data analysis will compare provider scores for each encounter for each method. The complete analytical model will be a 3-way crossed, one way nested analysis of variance (ANOVA) model. The main effect will be methodology. Case type and facility were included as random effects.

FINDINGS/RESULTS:
10% of the SP visits, Transcription, 50% of the vignettes are pending and electronic word recognition programs are pending.

IMPACT:
Vignettes are an inexpensive method to measure the quality of outpatient care. Potential uses include evaluation of physician training, incentive programs, and comparative evaluation of health care systems.

---

External Links for this Project

---

PUBLICATIONS:

Journal Articles

1. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA : the journal of the American Medical Association. 2000 Apr 5; 283(13):1715-22. [view]
2. Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P. How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. The American journal of medicine. 2000 Jun 1; 108(8):642-9. [view]
3. Dresselhaus TR, Peabody JW, Lee M, Wang MM, Luck J. Measuring compliance with preventive care guidelines: standardized patients, clinical vignettes, and the medical record. Journal of general internal medicine. 2000 Nov 1; 15(11):782-8. [view]
4. Glassman PA, Luck J, O'Gara EM, Peabody JW. Using standardized patients to measure quality: evidence from the literature and a prospective study. Joint Commission Journal on Quality Improvement. 2000 Nov 1; 26(11):644-53. [view]
5. Peabody JW, Feachem RG. Why global health matters to US primary care physicians. Western Journal of Medicine. 2001 Sep 1; 175(3):153-4. [view]

Book Chapters

1. Peabody JW, Lee D, Yang B, Gertler P. Financing Health Care for the Elderly: Will an Aging Population End 'Health for All' in South Korea. In: Hu T, editor. Financing Health Care in Asia. 2002. [view]

Conference Presentations

1. Peabody JW, Luck J, Levis B, Yeretsian O. Development of a Cost Effective Automated Scoring System for Computerized Clinical Vignettes. Presented at: Society of General Internal Medicine Annual Meeting; 2002 May 1; Atlanta, GA. [view]
2. Luck J, Peabody J, Yeretsian O, Conti C, Bates M. Listening in: Assessing the validity of standardized patient ratings of physician performance. Paper presented at: VA HSR&D National Meeting; 2001 Feb 1; Washington, DC. [view]
3. Dresselhaus T, Peabody J, Lee M, Glassman P, Luck J. Measuring compliance with preventive care guidelines: A comparison of standardized patients, clinical vignettes and the medical record. Paper presented at: VA HSR&D National Meeting; 2000 Mar 1; Washington, DC. [view]
4. Peabody JW, Luck J, Glassman P. Measuring What We Want to Measure: Using Vignettes in Clinical Education. Paper presented at: Society of General Internal Medicine Annual Meeting; 2002 May 4; Atlanta, GA. [view]
5. Dresselhaus TR, Luck J, Peabody JW. The ethical problem of false positives: A comparison of standardized patients and the medical record. Paper presented at: VA HSR&D National Meeting; 2002 Feb 1; Washington, DC. [view]

DRA: Health Systems
DRE: Technology Development and Assessment
Keywords: Quality assessment, Research method
MeSH Terms: none