1084 — Two Pilot Studies of a Computer-Based Relational Agent Compared to Human and Text-Based Screenings for Substance Misuse in Primary Care

Rubin A, VA Boston Healthcare System and Boston University; McNair SS, VA Boston Healthcare System and Harvard Medical School; Checchi K, VA Boston Healthcare System and Harvard Medical School; Marcello T, VA Boston Healthcare System; Bickmore T, Northeastern University; Simon SR, VA Boston Healthcare System and Harvard Medical School;

Objectives:
Primary Care clinicians are being asked to provide more behavioral health care while still meeting all of their other objectives for maintaining and improving Veterans' health; moreover, the quality and fidelity of these interactions are uncertain. Relational Agents are computer characters that simulate face-to-face conversation using voice, hand gesture, gaze cues and other nonverbal behavior to establish trust and therapeutic alliance with patients. We conducted two pilot studies to examine the feasibility of using Relational Agents to engage Veterans on sensitive behavioral health topics, such as substance misuse, in preparation for a larger trial of screening, brief counseling, and referral.

Methods:
Each pilot was a one-time within-subjects assessment survey with 30 Veterans randomized into receiving either the Relational Agent or the human Research Assistant assessment (Pilot 1) and the Relational Agent or text-based computer assessment (Pilot 2). We administered the NIDA-Modified Alcohol, Smoking, and Substance Involvement Screening Test plus brief feedback. Both qualitative and quantitative measures were used to assess participant reactions to the Relational Agent, satisfaction with the different methods of screening for substance use, and agreement of substance use severity scores between Relational Agent and human (Pilot 1) and between Relational Agent and text-based computer assessment (Pilot 2).

Results:
In general, participants reported greater satisfaction with a human interviewer than the Relational Agent (p < .01), but 77% reported their experience with the Relational Agent as positive. They also reported greater satisfaction with the Relational Agent compared to text-based screening (p < 0.05). A majority favored the use of Relational Agents to ask questions about other sensitive topics such as sexual history, and a number of Veterans reported that they appreciated the computer not being judgmental. Intraclass correlations between scores on drugs of abuse ranged from 0.83 - 0.94 for Pilot 1 but were lower for Pilot 2 (0.56 - 0.88).

Implications:
Relational Agents are a promising technology for providing consistent, high-quality screening and feedback for substance misuse.

Impacts:
Computerized solutions may be able to enhance the VHA's ability to provide high quality behavioral care within the real world limitations of the primary care clinic in a low cost, effective manner.