Practices such as inappropriately tight glycemic control which increase risk while adding little to no health benefits are considered "low value care". The Choosing Wisely Initiative (CWI) aims to reduce "low value care" and includes a recommendation to "not treat most persons over 65 years of age with medications to reduce the blood sugar levels to an A1c<7.5%." This is a departure from previous recommendations and for most health care providers this involves changes to their current clinical practice. However, this change is very important because in some patients tight glycemic control (A1C<7.5%) can cause undue risk for hypoglycemia (dangerously low blood sugar). Ensuring the safety of insulin and other drugs which are used to reduce blood sugar level for diabetes is an important focus of the VHA because of the potential adverse consequences of hypoglycemia. However, changing the way providers think about how to treat diabetes with these drugs is more involved than merely implementing a new guideline. Rather, the implementation of new guidelines requires multiple improvement strategies.
The purpose of this study is to understand what and how health care provider and health care facility characteristics affect the ability of providers to change patient care practices at VHA facilities. This study will observe how patient outcomes are affected after the release of CWI with three specific aims: (1) To assess the overall impact of the CWI on overtreatment rates in VHA facilities; (2) To assess the impact of facility level characteristics including: commitment to quality, teaching intensity, and safety culture on changes in overtreatment rates; and (3) To identify configurations of the implementation strategy, provider characteristics and organizational factors that are associated with successful reduction of overtreatment rates.
We will use a Type III Hybrid Design that focuses on study of implementation while at the same time observing and gathering information on clinical interventions and outcomes.
Data extracts from CDW including longitudinal patient data and aggregate facility data have been used to identify high performing sites (which greatly reduce overtreatment rates) and low performing sites (that do not reduce or increase their rate of overtreatment) based on changes in overtreatment rates pre- and post-initiative (rather than on absolute values).
Facility level data will be extracted from the CDW on gold star rating, patient safety data compiled by and sent to the CDW by the National Center for Patient Safety, and the total number of teaching slots compiled by and sent to the CDW by the Office of Academic Affiliations.
Providers at high and low performing sites will be surveyed to obtain information on participation in the CWI, provider type and other demographics. We will also determine whether and what practices were (or were not) implemented to reduce overtreatment, as well as what barriers and facilitators existed following the initiative.
Analyzing serial cross-sectional data from 2009-13, we observed trends towards lower rates of overtreatment and higher rates of undertreatment (Figure 1). Correlations between overtreatment rates and undertreatment rates were calculated for each year and threshold. For overtreatment thresholds 6.5% and 7%, significant negative correlations between overtreatment rates and undertreatment rates were observed across all years; using 6% as the overtreatment threshold, correlations were negative but of lower magnitude and not significant. Outlier values and performance deciles based on outlier values were found to be correlated and sustained over time. Correlations of outlier values between years for each comparator and threshold were calculated. Outlier values in adjacent years were highly correlated and the year-to-year correlation decreased with increased separation in time. The lowest correlations within each threshold and comparator were generally seen between the 2009 and 2013 outlier values. The VISN comparator correlations were generally lower than those using all VA facilities or same complexity comparators. The correlations were consistent across the three thresholds. Generally, we observed sustained high or sustained low overtreatment, with the 2009 high and low performers spreading towards each other increasingly across time. Looking across years and thresholds, facility complexity was not a significant predictor of overtreatment and inter-VISN variability consistently exceeded intra-VISN variability. There was modest overlap in facility rankings across A1c thresholds. For example, when the lowest decile was considered for each threshold, only half (7 of 14) of the facilities were in that decile for both of the other overtreatment thresholds. (data not shown.) When undertreatment rates over time were analyzed, several facilities flagged as high performers based on the overtreatment models exhibited a trend of increasing undertreatment over time that far exceeds the VA-wide average. Two facilities exceeded the nationwide average undertreatment rate by roughly 2 or more percentage points in each year.
This project will advance implementation science by using an innovative mixed methods multi-paradigm approach to examine potential mechanisms to explain the variation in reduction of rates of overtreatment and to contribute to a better understanding of implementation of national dissemination projects and multi-component interventions in complex systems.
Our results indicate the importance of having a balancing measure. While overtreatment in general fell over time and undertreatment rose, we found that those high performing facilities in overtreatment tended to exhibit a trend of increasing undertreatment over time that often exceeded the VA-wide average. Such facilities should not be considered positive deviants. A low rate of overtreatment may be observed under both desirable and undesirable conditions.
Our findings indicate that the statistical identification of high performing outliers is very sensitive to the specific criteria chosen. For example, the choice of three different A1c levels to define overtreatment resulted in three series of high performing outliers with only modest overlap. This variation has implications for the validity of conclusions drawn from league tables, particularly those based on a single measure, and highlights the need to understand the clinical differences of thresholds when interpreting differences in quality results seen across thresholds. Interestingly, there was little impact of facility complexity or VISN on the findings; limiting comparators to like facilities was not necessary in this particular circumstance. It may be that the lack of effects of facility characteristics reflects the fact that most patients with diabetes are managed by primary care providers even when a facility has a diabetes specialist. Primary care services are available at all facilities regardless of the overall scope of services provided by the facility. Thus, our results have implications for the increasingly popular "positive deviance" approach to improvement. Although there are numerous methods for outlier detection and differences in both criteria and comparator, our study suggests that considerable thought needs to be given to this issue at the outset, before attempts are made to identify performance outliers.
This project is integrated into the Choosing Wisely/Hypoglycemia Safety Initiative and its results are informing that initiative's activities including
External Links for this Project
Grant Number: I01HX001536-01A1
None at this time.