Sloan KL (Northwest HSR&C COE)
Rosen AK (CHQOER (Bedford))
Christiansen CL (CHQOER (Bedford))
Montez M (CHQOER (Bedford))
The performance of VA facilities is currently measured by ascertaining whether some proportion of their patients with ischemic heart disease (IHD) have low density lipoprotein (LDL) values below a goal based on clinical evidence. Little is known, though, about how sensitive the identification of facilities as “outliers” is to choice of target LDL value. Our goal is to examine the impact of LDL cut-point on whether facilities are identified as “high” or “low outliers”.
We included all individuals who had (a) both FY00 and FY01 primary care visits, and (b) at least one FY01 IHD-related diagnosis (ICD-9 410 to 414), and (c) available LDL values. Patients were assigned to the last facility seen in FY01. For each facility we calculated the proportion of target patients with LDL = 70mg/dL, 100 mg/dL, 130 mg/dL and 160 mg/dL along with the corresponding 99% confidence intervals (CI). Facilities whose 99% CI excluded the weighted group mean were considered outliers.
: Of the 116 facilities with available data, 10 (9%) were consistently low outliers, 6 (4%) were consistently high outliers, and 38 (24%) were never outliers. The remaining 72 (62%) had inconsistent outlier status (including one facility that was high at the 70 cut point and low at the 160 mg/dl cut point). Examining just the middle two cut points, of the 78 facilities which were outliers at either 100 or 130mg/dl, 44(56%) were outliers at both cuts, 28(36%) were outliers only at 100mg/dl, and 6(8%) were outliers only at 130mg/dl. Bayesian estimation modestly shrank the numbers of outliers but did not alter the overall pattern.
Given the sensitivity of outlier status to choice of cut point, it is critical that any LDL level(s) chosen for facility performance assessment be based on the best available evidence regarding optimal clinical outcomes.
One cannot assume that performing well at a given threshold implies superior performance at other values. Furthermore, investigation of the sensitivity of facility performance assessment on other profiled measures (e.g., A1c or blood pressure control) is warranted.