Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

HSR&D Citation Abstract

Search | Search by Center | Search by Source | Keywords in Title

How Many Reviews Do You Need? Measuring The Quality Of Care At Clinic Sites With Implicit Physician Review

Hofer TP, Asch S, Hayward RA, Hogan MM, Rubenstein LV, Adams JL, Kerr EM. How Many Reviews Do You Need? Measuring The Quality Of Care At Clinic Sites With Implicit Physician Review. Paper presented at: Society of General Internal Medicine Annual Meeting; 2004 May 14; Chicago, IL.




Abstract:

Background: We sought to develop a more reliable structured implicit chart review instrument for use in assessing the quality of care for chronic disease and to estimate how many reviews would be needed to provide a reliable estimate of quality of care for a clinic site.Methods: We conducted a reliability study with records of both outpatient and inpatient care as the objects of measurement. 12 reviewers conducted a total of 1666 reviews of 621 patient records selected from 26 VA clincal sites in in two regions of the country. Each patient had between one and four conditions specified as having a highly developed evidence base (diabetes and hypertension) or a less developed evidence base (chronic obstructive pulmonary disease or a collection of acute conditions). Multilevel analysis that accounts for the nested and cross-classified structure of the data was used to estimate the signal and noise components of the measurement of quality and the reliability of implicit review.Results: For COPD and a collection of acute conditions the reliability of a single physician review was low (intra-class correlation = .16-.26) but comparable to most previously published estimates for the use of this method. For diabetes and hypertension the reliability is significantly higher at 0.46. The higher reliability is a result of the reviewers collectively being able to distinguish more differences in the quality of care between patients (p < .007) and not due to less random noise or individual reviewer bias in the measurement. The variance at the level of the clinical site was one third of that at the patient level within site and implicit reviews of between 24 and 33 records would give a quality rating with a reliablity of 0.8 at the level of the clinical site.Conclusion: For conditions with a well-developed quality of care evidence base, such as hypertension and diabetes, structured implicit review to assess the quality of care over a period of time could reliably detect differences in quality across practice settings with as few as 25 reviews per site of care and could be a reasonable complement or alternative to explicit indicator approaches for assessing and comparing quality of care. Structured implicit review, as well as explicit quality measures, must be used more cautiously for illnesses for which the evidence base is less well developed, such as COPD and acute, short-course illnesses.





Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.