Original Investigation | Psychiatry
A Machine Learning Approach to Understanding Patterns of Engagement
With Internet-Delivered Mental Health Interventions
Isabel Chien, MEng; Angel Enrique, PhD; Jorge Palacios, MD, PhD; Tim Regan, PhD; Dessie Keegan, MSc; David Carter, PhD; Sebastian Tschiatschek, PhD; Aditya Nori, PhD;
Anja Thieme, PhD; Derek Richards, PhD; Gavin Doherty, DPhil; Danielle Belgrave, PhD
Abstract
IMPORTANCE The mechanisms by which engagement with internet-delivered psychological
interventions are associated with depression and anxiety symptoms are unclear.
OBJECTIVE To identify behavior types based on how people engage with an internet-based
cognitive behavioral therapy (iCBT) intervention for symptoms of depression and anxiety.
DESIGN, SETTING, AND PARTICIPANTS Deidentified data on 54 604 adult patients assigned to the
Space From Depression and Anxiety treatment program from January 31, 2015, to March 31, 2019,
were obtained for probabilistic latent variable modeling using machine learning techniques to infer
distinct patient subtypes, based on longitudinal heterogeneity of engagement patterns with iCBT.
INTERVENTIONS A clinician-supported iCBT-based program that follows clinical guidelines for
treating depression and anxiety, delivered on a web 2.0 platform.
MAIN OUTCOMES AND MEASURES Log data from user interactions with the iCBT program to
inform engagement patterns over time. Clinical outcomes included symptoms of depression (Patient
Health Questionnaire-9 [PHQ-9]) and anxiety (Generalized Anxiety Disorder-7 [GAD-7]); PHQ-9 cut
point greater than or equal to 10 and GAD-7 scores greater than or equal to 8 were used to define
depression and anxiety.
RESULTS Patients spent a mean (SD) of 111.33 (118.92) minutes on the platform and completed
230.60 (241.21) tools. At baseline, mean PHQ-9 score was 12.96 (5.81) and GAD-7 score was 11.85
(5.14). Five subtypes of engagement were identified based on patient interaction with different
program sections over 14 weeks: class 1 (low engagers, 19 930 [36.5%]), class 2 (late engagers, 11 674
[21.4%]), class 3 (high engagers with rapid disengagement, 13 936 [25.5%]), class 4 (high engagers
with moderate decrease, 3258 [6.0%]), and class 5 (highest engagers, 5799 [10.6%]). Estimated
mean decrease (SE) in PHQ-9 score was 6.65 (0.14) for class 3, 5.88 (0.14) for class 5, and 5.39 (0.14)
for class 4; class 2 had the lowest rate of decrease at −4.41 (0.13). Compared with PHQ-9 score
decrease in class 1, the Cohen d effect size (SE) was −0.46 (0.014) for class 2, −0.46 (0.014) for class
3, −0.61 (0.021) for class 4, and −0.73 (0.018) for class 5. Similar patterns were found across groups
for GAD-7.
CONCLUSIONS AND RELEVANCE The findings of this study may facilitate tailoring interventions
according to specific subtypes of engagement for individuals with depression and anxiety. Informing
clinical decision needs of supporters may be a route to successful adoption of machine learning
insights, thus improving clinical outcomes overall.
JAMA Network Open. 2020;3(7):e2010791. doi:10.1001/jamanetworkopen.2020.10791
Key Points
Question Can machine learning
techniques be used to identify
heterogeneity in patient engagement
with internet-based cognitive behavioral
therapy for symptoms of depression
and anxiety?
Findings In this cohort study using data
from 54 604 individuals, 5
heterogeneous subtypes were
identified based on patient engagement
with the online intervention. These
subtypes were associated with different
patterns of patient behavior and
different levels of improvement in
symptoms of depression and anxiety.
Meaning The findings of this study
suggest that patterns of patient
behavior may elucidate different
modalities of engagement, which can
help to conduct better triage for
patients to provide personalized
therapeutic activities, helping to
improve outcomes and reduce the
overall burden of mental health
disorders.
+
Invited Commentary
+
Supplemental content
Author affiliations and article information are
listed at the end of this article.
Open Access. This is an open access article distributed under the terms of the CC-BY-NC-ND License.
JAMA Network Open. 2020;3(7):e2010791. doi:10.1001/jamanetworkopen.2020.10791 (Reprinted) July 17, 2020 1/12
Downloaded From: https://jamanetwork.com/ on 07/07/2023
