Talk to the Veterans Crisis Line now
U.S. flag
An official website of the United States government

VA Health Systems Research

Go to the VA ORD website
Go to the QUERI website

PPO 13-122 – HSR Study

PPO 13-122
Enhancing Delivery of Problem Solving Therapy using SmartPhone Technology
Kathleen M Grubbs, PhD
Central Arkansas Veterans Healthcare System , Little Rock, AR
Little Rock, AR
Funding Period: October 2013 - September 2014
Increasing access to and engagement in evidence-based psychotherapies for Veterans is a high priority for the Veterans Health Administration (VHA), especially the Office of Rural Health (ORH) and the office of Mental Health Services (MHS). SmartPhone applications are an emerging technology with a vast potential to extend the reach of traditional in-person psychotherapy by allowing increased digital access to providers and self-management tools. Preliminary data on acceptability and satisfaction are promising, suggesting a need for further research. The National Center for PTSD in collaboration with the National Center for Telehealth & Technology (T2) recently completed the development of an app called Moving Forward, designed for an evidence-based psychotherapy - Problem Solving Therapy (PST). The evidence base for PST spans a range of Axis I diagnoses and is ideal for Primary Care-Mental Health Integration where prevalence estimates for anxiety and depressive disorders range from 10-20%. A major strength of the patient-centered Moving Forward app is that it was specifically designed to target Veteran needs. Due to the relatively recent development of this technology, there are no data on the effectiveness of SmartPhone augmented psychotherapy.

Specific Aim #1: To test whether the Moving Forward app as an augmentation to Problem Solving Therapy facilitates engagement compared to Problem Solving Therapy alone.
Specific Aim #2: To test whether the Moving Forward app as an augmentation to Problem Solving Therapy facilitates symptom improvement compared to Problem Solving Therapy alone.
Specific Aim #3: Elicit specific detailed feedback on the Moving Forward app.

We recruited 34 Veterans diagnosed with an anxiety or depressive disorder from PC-MHI clinics. At baseline the research assistants conducted in-person interviews. Measures included a demographics questionnaire, Depression Anxiety and Stress Scale, and the Short Form 12-Item Health Survey for Veterans. Participants were interviewed using select modules from the MINI International Neuropsychiatric Interview, the Social Problem Solving Inventory-Revised, Perceptions of Computerized Therapy Questionnaire-Patient Version. At 6-weeks, participants completed the DASS-21 and the SF-12V, the SPSI-R, the Client Satisfaction Questionnaire-8, and Problem Solving Skills Use measure (PSSU, designed for this study). At 12 weeks, participants completed the DASS-21, SF-12 V, the SPSI-R, the PSSU and a questionnaire on general SmartPhone use for accessing social support and health related information. Veterans randomized to receive the Moving Forward app also completed an additional semi-structured interview to solicit feedback about the Moving Forward intervention.
Veterans in both groups received a SmartPhone with basic instructions during their initial treatment session. Participants, were randomized to receive either PST alone (n=16) or PST plus the Moving Forward app (n=17). Randomization was stratified based on whether they owned a SmartPhone. Veterans randomized to the Moving Forward group were instructed on how to access and use the Moving Forward app and Veterans in the PST-only group learned how to access the internet. All Veterans received a standard 6-session administration of PST. Session 1 took place in-person during a scheduled appointment. Subsequent sessions took place over the telephone. As part of each PST session, participants completed the PHQ-9 to evaluate on-going treatment effects on depression symptoms. The PHQ-9 also contains a question inquiring about suicidality. This measure allowed therapists to monitor suicidality at each session. Any positive response will triggered our suicide risk protocol. The PST treatment is part of standard care in PC-MHI.

Eighty percent of participants were male and the sample had a mean age of 46.6 years (SD = 12.1). Ninety-four percent of participants completed all size sessions of PST therapy. Groups differed on their self-reported use of problem solving skills with a larger proportion of the app group reporting that they used they practiced skills at 6-weeks (93% versus 75%; O.R. = .21) and at 12-weeks (57% versus 30%; O.R. = 3.0). The groups did not differ substantially on treatment satisfaction (d = .06), or attendance (d = .16) (Aim 1). The app group reported moderately higher scores on the Social Problem Solving Inventory at 6-weeks (d = .39) but not at 12-weeks (d = -.22). Both groups showed a 1-category reduction (from moderate to mild) in depression, anxiety, and stress symptoms from baseline to 6-weeks, however there were no observed differences between groups for depression (d = .04, p = .90), anxiety (d = .01, p = .97) and stress (d = .07, p = .83). At 12-weeks, there were no between group differences for depression (d = .16, p = .67), anxiety (d = .34, p = .37) or stress (d = .51, p = .20). Qualitative data highlighted strengths of the app (e.g., empowerment associated with ease of accessing self-support tools, confidentiality and anonymity of using a SmartPhone to manage symptoms, and satisfaction with the app to target symptoms) and weaknesses (unavailable on some mobile platforms).

This research extends current scientific knowledge by (1) establishing preliminary effectiveness data for Moving Forward, (2) providing information on patient reactions to the app, and (3) giving direct feedback on multiple dimensions of app content. This study increases the potential for providers to deliver an effective PST treatment, and helps to inform recommendations for app use, and could inform changes to the app that could increase appeal of this product for consumers. This study also expands the overall knowledge of individual patient reactions to SmartPhone technology in general and informs other efforts to increase digital access within VA.

External Links for this Project

NIH Reporter

Grant Number: I21HX001349-01

Dimensions for VA

Dimensions for VA is a web-based tool available to VA staff that enables detailed searches of published research and research projects.

Learn more about Dimensions for VA.

VA staff not currently on the VA network can access Dimensions by registering for an account using their VA email address.
    Search Dimensions for this project


HSR&D or QUERI Articles

  1. Fortney JC, Grubbs KM. Improving Digital Access to Evidence-Based Psychotherapy. VA HSR&D FORUM: Translating research into quality health care for Veterans. 2014 May 1; 7-8. [view]
Conference Presentations

  1. Grubbs KM, Green CJ, Abraham TH, Pyne JM, Fortney JC. Enhancing Problem Solving Therapy in Primary Care-Mental Health Integration Using SmartPhone Technology. Paper presented at: VA HSR&D / QUERI National Meeting; 2015 Jul 8; Philadelphia, PA. [view]
  2. Teng E, Hiatt EL, Grubbs KM, Kunik ME, Stanley MA. Multiple channel exposure therapy for PTSD and panic. Paper presented at: International Society for Traumatic Stress Studies Annual Meeting; 2013 Nov 7; Philadelphia, PA. [view]
  3. VanKirk N, Grubbs KM, Teng EJ. Smartphone Applications in the Treatment of Anxiety and Depression: Important Considerations for Development and Testing. Paper presented at: Anxiety and Depression Association of America Anxiety Disorders and Depression Conference; 2015 Apr 10; Miami, FL. [view]
  4. Grubbs KM, Fortney JC. Using smartphone apps to increase access and engagement in psychotherapy among rural Veterans. Paper presented at: National Association for Rural Mental Health Annual Conference; 2014 Jul 18; Washington, DC. [view]

DRA: Mental, Cognitive and Behavioral Disorders
DRE: Treatment - Efficacy/Effectiveness Clinical Trial
Keywords: none
MeSH Terms: none

Questions about the HSR website? Email the Web Team

Any health information on this website is strictly for informational purposes and is not intended as medical advice. It should not be used to diagnose or treat any condition.