2023 HSR&D/QUERI National Conference

4013 — Veteran Perspectives on the Usability of VHA’s Mental Health Checkup Mobile Health Application

Lead/Presenter: Felicia Bixler,  COIN - Hines
All Authors: Bixler FR (Center of Innovation for Complex Chronic Healthcare (CINCCH), Hines VA Hospital), Etingen B (Center of Innovation for Complex Chronic Healthcare (CINCCH), Hines VA Hospital) Zocchi M (Center for Healthcare Organization and Implementation Research (CHOIR), VA Bedford Healthcare System) Higashi RT (Department of Population and Data Sciences, UT Southwestern Medical Center) Palmer JA (Center for Healthcare Organization and Implementation Research (CHOIR), VA Boston) Richardson E (Center for Healthcare Organization and Implementation Research (CHOIR), VA Boston) McMahon N (Center for Healthcare Organization and Implementation Research (CHOIR), VA Bedford Healthcare System) Patrianakos J (Center of Innovation for Complex Chronic Healthcare (CINCCH), Hines VA Hospital) Ndiwane N (Center for Healthcare Organization and Implementation Research (CHOIR), VA Bedford Healthcare System) Smith BM (Center of Innovation for Complex Chronic Healthcare (CINCCH), Hines VA Hospital) Hogan TP (Center for Healthcare Organization and Implementation Research (CHOIR), VA Bedford Healthcare System)

Objectives:
Measurement-based care (MBC) practices, wherein providers collect patient-reported outcome measures (PROMs) from Veterans and integrate PROM data into clinical practice, are a priority of the Veterans Health Administration (VHA). The Mental Health Checkup (MHC) mobile health application (app) was developed to support MBC in VHA, and is used to collect PROMs from Veterans remotely, in the context of their daily life. Our objective was to examine Veterans’ use of MHC, and perceptions of its usability.

Methods:
We completed a mixed-methods evaluation of MHC. We first fielded a mailed survey of Veterans who had logged into MHC at least twice, using a modified Dillman approach. We then completed semi-structured telephone interviews with a purposeful sample of survey respondents. Surveys and interviews asked Veterans about their use of MHC, including comfort using the app, sources of troubleshooting support, and challenges experienced with the app. Survey data were analyzed using descriptive statistics and interview data were analyzed using thematic analysis.

Results:
We invited 2,690 Veterans to complete a survey and received responses from n = 533 (20% response rate). Survey respondents were predominantly male (71.0%), non-Hispanic (80.6%) White (66.2%), aged 46-55 (23.9%) or 56-65 (25.6%), and had completed at least some college (90.3%). Most respondents reported being comfortable/extremely comfortable using MHC (78.4%), and 41.2% reported not having had trouble using the app. Among those who did experience trouble using MHC, Veterans reported that, when they encountered difficulties, they sent a secure message to (32.1%) or called (21.8%) their VA providers, or called the VA Technology Help Desk (11.1%). The most frequent issues related to MHC use that Veterans reported to be somewhat or seriously challenging included: the graphs of their assessment scores were hard to understand (41.5%); they did not receive enough training when they first started using MHC (28.1%); they cannot change an answer to an assessment question if they entered it erroneously (28.0%); their provider(s) do not talk to them about their assessment scores (26.8%); and they cannot access the assessments their provider(s) asked them to complete (26.0%). Interview respondents (n = 20) further described challenges they experienced, including technical issues (e.g., links to the assessments did not work, their assessment responses did not get recorded and sent to their providers) and feeling that they had limited resources to turn to for help using the app other than their providers, which could curtail therapy time during appointments.

Implications:
By and large, the Veterans in our sample found MHC to be easy to use. The challenges Veterans most frequently reported related to a lack of training on how to use MHC, its user interface, and limited discussions with their providers about their assessment scores.

Impacts:
MHC has potential to support nationwide efforts across VHA to integrate MBC into mental health services. Efforts to make training resources available and known to Veterans regarding how to use MHC and interact with their data in the app may bolster usability and in turn, sustained use.