TY  - JOUR
AU  - Bell, Lauren
AU  - Garnett, Claire
AU  - Bao, Yihan
AU  - Cheng, Zhaoxi
AU  - Qian, Tianchen
AU  - Perski, Olga
AU  - Potts, Henry W W
AU  - Williamson, Elizabeth
PY  - 2023
DA  - 2023/6/9
TI  - How Notifications Affect Engagement With a Behavior Change App: Results From a Micro-Randomized Trial
JO  - JMIR Mhealth Uhealth
SP  - e38342
VL  - 11
KW  - mobile health
KW  - mHealth
KW  - digital health
KW  - behavior change
KW  - digital behavior change
KW  - engagement
KW  - micro-randomized trial
KW  - randomized trial
KW  - randomization
KW  - just-in-time adaptive intervention
KW  - adaptive intervention
KW  - push notification
KW  - notification
KW  - excessive alcohol consumption
KW  - smartphone app
KW  - alcohol
KW  - drinking
KW  - drinker
KW  - mobile phone
AB  - Background: Drink Less is a behavior change app to help higher-risk drinkers in the United Kingdom reduce their alcohol consumption. The app includes a daily notification asking users to “Please complete your drinks and mood diary,” yet we did not understand the causal effect of the notification on engagement nor how to improve this component of Drink Less. We developed a new bank of 30 new messages to increase users’ reflective motivation to engage with Drink Less. This study aimed to determine how standard and new notifications affect engagement. Objective: Our objective was to estimate the causal effect of the notification on near-term engagement, to explore whether this effect changed over time, and to create an evidence base to further inform the optimization of the notification policy. Methods: We conducted a micro-randomized trial (MRT) with 2 additional parallel arms. Inclusion criteria were Drink Less users who consented to participate in the trial, self-reported a baseline Alcohol Use Disorders Identification Test score of ≥8, resided in the United Kingdom, were aged ≥18 years, and reported interest in drinking less alcohol. Our MRT randomized 350 new users to test whether receiving a notification, compared with receiving no notification, increased the probability of opening the app in the subsequent hour, over the first 30 days since downloading Drink Less. Each day at 8 PM, users were randomized with a 30% probability of receiving the standard message, a 30% probability of receiving a new message, or a 40% probability of receiving no message. We additionally explored time to disengagement, with the allocation of 60% of eligible users randomized to the MRT (n=350) and 40% of eligible users randomized in equal number to the 2 parallel arms, either receiving the no notification policy (n=98) or the standard notification policy (n=121). Ancillary analyses explored effect moderation by recent states of habituation and engagement. Results: Receiving a notification, compared with not receiving a notification, increased the probability of opening the app in the next hour by 3.5-fold (95% CI 2.91-4.25). Both types of messages were similarly effective. The effect of the notification did not change significantly over time. A user being in a state of already engaged lowered the new notification effect by 0.80 (95% CI 0.55-1.16), although not significantly. Across the 3 arms, time to disengagement was not significantly different. Conclusions: We found a strong near-term effect of engagement on the notification, but no overall difference in time to disengagement between users receiving the standard fixed notification, no notification at all, or the random sequence of notifications within the MRT. The strong near-term effect of the notification presents an opportunity to target notifications to increase “in-the-moment” engagement. Further optimization is required to improve the long-term engagement. International Registered Report Identifier (IRRID): RR2-10.2196/18690 
SN  - 2291-5222
UR  - https://mhealth.jmir.org/2023/1/e38342
UR  - https://doi.org/10.2196/38342
UR  - http://www.ncbi.nlm.nih.gov/pubmed/37294612
DO  - 10.2196/38342
ID  - info:doi/10.2196/38342
ER  -