BACKGROUND/RATIONALE:
Depression is a major cause of morbidity and mortality in primary care patients, and depression affects approximately 12% of Veterans attending VA primary care practices. To address this heavy burden, a series of depression quality improvement efforts (Translating Initiatives in Depression into Effective Solution-TIDES and ReTIDES) based on collaborative care management (CCM) were implemented at the VA beginning in 2002. In 2007, the national primary care/mental health integration (PC-MHI) mandate (VA Uniform Mental Health Services Handbook) introduced both co-located mental health specialists and CCM into larger VA primary care sites. This mandate required implementation and spread of CCM to translate from a research innovation to the roll-out of a national program making evaluation strategies based on randomization not feasible. Also, electronic performance measures for assessing the level at which the population of Veterans with depression received appropriate care had not proven robust enough to support rigorous evaluation of VA's depression management. Therefore, there was a critical need for longitudinal electronic depression care quality measures capable of evaluating the implementation, spread and sustainability of CCM.

OBJECTIVE(S):
We developed longitudinal electronic population-based measures of depression quality of care and validated the measures conceptually and operationally using panel judgment by VA and non-VA experts; and demonstrated the feasibility of using a modified stepped wedge design approach with the new measures for evaluating TIDES/ReTIDES.

METHODS:
We used the VA National Patient Care Database and Pharmacy Benefits Management data for primary care patients from 1999-2011 from nine Veteran Integrated Service Networks (VISNs). We developed four population-based depression care quality metrics that incorporated a six-month look back and one-year follow-up: detection of a new episode of depression, 84 and 180 days follow-up, and minimally appropriate treatment one-year post-detection. The development of longitudinal population-based quality measures required systematic referencing of final measures to the full primary care population, attention to timing of the measures in relationship to a patient's index visit, establishing continuity of care, and determining evidence-based cut-points for health care visits, prescription medications and refills.

We developed a modified stepped wedge design based on a general linear model and published stepped wedge design methodology to improve the precision of comparisons between intervention and control periods when randomized designs are not feasible. Our model is multi-level, requiring inclusion of three levels of variables: time, site-level, and patient-level covariates. We fit the model for all primary care patients with a new episode of depression seen at the 34 sites within the nine VISNs that implemented TIDES or ReTIDES from 2000-2010. Implementation data on TIDES/ReTIDES start date, facility type (medical center vs. community-based clinic) and facility size, as well as patient-level covariates (age, gender, race/ethnicity, marital status, and PTSD) were used to test the utility and validity of a modified stepped wedge design for evaluating depression follow-up and treatment. We used a modified Delphi expert panel process with VA and non-VA experts in the area of depression care quality, PC-MHI, program evaluation, and performance measurement to assess overall acceptability of the measures and the modified stepped wedge design, future implications for VA's CCM program implementation, and optimal statistical features for broader use.

FINDINGS/RESULTS:
There was a substantial increase in the number of patients seen in primary care - 1.19 million in 2000 to over 2.26 million in 2010. Despite rapid growth, the detection of new episodes of depression (7-8%) and minimally appropriate treatment rates (82-84%) remained stable over the decade indicating that VA was able to maintain a standard of care while treating significantly more patients each year. Follow-up at 84 and 180 days were 37% and 45% respectively in 2000 and increased to 56% and 63% by 2010 indicating improvements in timely treatment initiation and follow-up.

In the modified stepped wedge model, 53% of the sites were medical centers, and 21% were the original TIDES sites. The primary care patient population was 9% women, 30% white and 10% black patients,18% less than 44 years, 51% ages 45-64, and 31% 65 years or older. We found improvements in depression care quality over time. However, after correcting for covariates, the interactions of TIDES and ReTIDES with time (year) were not significant, indicating that the effect for sites engaging in the programs over the ten years compared to the effect seen in control periods did not significantly vary by calendar year. Women had increased odds of treatment compared to men (OR=1.10; p<.05), whereas blacks had increased odds of follow-up (84 days OR=1.11 and 180 days OR=1.14; p<.05) compared to whites. Younger patients had higher odds of follow-up and treatment than older age groups, and patients with PTSD had increased odds of follow-up (84 days OR=2.79, 180 days OR=4.31; p<.01) and treatment (OR=3.91; p<.01) than patients without PTSD.

We convened an expert panel in March 2015. Prior to the meeting, our 14 panelists received a measures development monograph and an on-line survey to evaluate our methods. Pre-meeting survey results showed a high level of agreement on most cohort construction and measures development methods, including using antidepressants prescribed in primary care as a signal for depression detection. Experts reached consensus on our definition of "minimally appropriate treatment" as the lowest level of activity that would indicate treatment for depression in a 12-month period. For the modified stepped wedge, panelists agreed that additional measures with stricter treatment requirements would be useful, especially considering the mental health related complexity of the Veteran population. Panelists noted the importance of measuring non-face-to-face encounters for evaluating CCM; however, because reliable coding of telephone, tele-health, and secure messaging has only recently become available, inclusion of these was not feasible in these historical analyses.

IMPACT:
These electronic depression care quality measures meet a widely-recognized need for longitudinal metrics that span the course of depression clinical care from screening through treatment. By using these measures to demonstrate the feasibility of using a modified stepped wedge model to evaluate collaborative care roll-out and sustainability over a ten-year period, the results inform use of comparative evaluation designs for nonrandomized spread interventions and on-going national implementation of CCM as well as advance the science of depression quality measurement.

---

External Links for this Project

NIH Reporter

Grant Number: I01HX000821-01A1
Link: https://reporter.nih.gov/project-details/8391006

---

PUBLICATIONS:

Journal Articles

1. Huynh AK, Lee ML, Farmer MM, Rubenstein LV. Application of a nonrandomized stepped wedge design to evaluate an evidence-based quality improvement intervention: a proof of concept using simulated data on patient-centered medical homes. BMC medical research methodology. 2016 Oct 21; 16(1):143. [view]
2. Farmer MM, Rubenstein LV, Sherbourne CD, Huynh A, Chu K, Lam CA, Fickel JJ, Lee ML, Metzger ME, Verchinina L, Post EP, Chaney EF. Depression Quality of Care: Measuring Quality over Time Using VA Electronic Medical Record Data. Journal of general internal medicine. 2016 Apr 1; 31 Suppl 1:36-45. [view]
3. Lam CA, Sherbourne C, Gelberg L, Lee ML, Huynh AK, Chu K, Strauss JL, Metzger ME, Post EP, Rubenstein LV, Farmer MM. Differences in Depression Care for Men and Women among Veterans with and without Psychiatric Comorbidities. Women's health issues : official publication of the Jacobs Institute of Women's Health. 2017 Mar 1; 27(2):206-213. [view]

Conference Presentations

1. Lam CA, Sherbourne CD, Lee ML, Metzger M, Post EP, Rubenstein LV, Farmer Coste MM. Age Related Differences in Quality of Depression Care Among Older Veterans. Poster session presented at: VA Geriatrics and Extended Care Data Analysis Center National Meeting; 2016 May 6; Los Angeles, CA. [view]
2. Lam CA, Sherbourne C, Gelberg L, Lee M, Huynh A, Chu K, Strauss J, Metzher M, Post E, Rubenstein L, Farmer MM. Differences in Depression Care for Men and Women Among Veterans with and without Psychiatric Co-Morbidities. Poster session presented at: AcademyHealth Annual Research Meeting; 2016 Jun 26; Boston, MA. [view]
3. Farmer MM, Rubenstein LV, Sherbourne C, Huynh AK, Chu K, Lam C, Fickel J, Lee ML, Metzger M, Chaney E, Post E. Measuring a decade of depression detection and treatment at the Veterans Health Administration. Paper presented at: AcademyHealth Annual Research Meeting; 2015 Sep 22; Washington, DC. [view]
4. Huynh-Hohnbaum A, Lee ML, Farmer Coste MM, Rubenstein LV. Modified Stepped Wedge Analytical Model to Evaluate Evidence-Based Quality Improvement Interventions. Poster session presented at: Electronic Data Methods (EDM) Forum Annual Symposium; 2013 Jun 1; Baltimore, MD. [view]
5. Huynh A, Lee M, Farmer M, Rubenstein LV. Modified Stepped Wedge Analytical Model to Evaluate Evidence-Based Quality Improvement Interventions. Poster session presented at: AcademyHealth Electronic Data Methods Forum; 2013 Jun 22; Baltimore, MD. [view]
6. Tubbesing S, Fickel JJ, Ong M, Cowper-Ripley D, Lind J, Obal L, Elenes-Campos M, Weber M, Sadler A, Wilson L, Toy S. Research-Operations Partnership: Preparing for HBPC to Utilize GIS Mapping Tools. Paper presented at: VA Greater Los Angeles Healthcare System Annual Research Day; 2015 May 4; Los Angeles, CA. [view]

DRA: Mental, Cognitive and Behavioral Disorders, Health Systems Science
DRE: Treatment - Comparative Effectiveness
Keywords: none
MeSH Terms: none