Session number: 1029
Abstract title: Do Different Modeling Techniques Affect Judgments of VISN Performance after Adjustment for Case-Mix?
Author(s):
S Loveland - Bedford VA, HSR&D Field Program
C Christiansen - Bedford VA, HSR&D Field Program
C Rakovski - Bedford VA, HSR&D Field Program
M Montez - Bedford VA, HSR&D Field Program
A Rosen - Bedford VA, HSR&D Field Program
Objectives: Efficiency ratios (ERs), a measure of the over- or under-utilization of a provider adjusted by its case-mix, are increasingly being used for profiling. Differences in how these ratios are calculated may affect judgments of provider performance. We examine whether the use of different modeling techniques results in different assessments of performance across VISNs using two case-mix measures, Adjusted Clinical Groups (ACGs) and Diagnostic Cost Groups (DCGs).
Methods: A 60% random sample of veteran patients (N=1,571,264) who received inpatient or outpatient care during FY’97 was obtained from VA databases. We developed and validated two regression models for each of the ACG and DCG systems (the basic models). To age and gender categories, we added 116 Hierarchical Condition Categories (for the DCG/HCC model) and 32 Adjusted Diagnostic Groups (for the ACG/ADG model) to explain FY'97 utilization (sum of inpatient and outpatient days). To predict “expected” days across VISNs, we used 3 regression modeling techniques: 1) the two basic models, 2) two basic models using only 21 VISNs to calculate the 22nd VISN’s predicted utilization (repeating this procedure 22 times), and 3) the basic models + 22 VISN dummies, subtracting out VISN-fixed effects. Models 2 and 3 decrease dependence between VISN expected and observed utilization. We calculated ERs (VISN mean actual days/ VISN mean expected days) using each of these 3 methods and compared them within each case-mix system.
Results: Compared to model 1, the ERs from models 2 and 3 were slightly lower for “efficient” VISNs (ERs<1) and higher for the less efficient VISNs (ERs>1) (e.g., one efficient VISN’s ERs were 0.884, 0.875 and 0.882 for models 1,2, and 3, respectively). VISN rank ordering was unchanged. These generalizations held regardless of which case-mix measure was used.
Conclusions: ERs resulting from the different modeling techniques of expected utilization were similar, suggesting that the basic ACG/DCG models are reliable and stable for this large sample application.
Impact statement: Case-mix adjusted profiling is essential for VISN-level comparisons. Because profiling indicators could be biased depending upon the modeling techniques used, the VA needs to ensure that selected indicators show consistent results across methodologies.