Background: A growing number of psychological interventions are delivered via smartphones with the aim of increasing the efficacy and effectiveness of these treatments and providing scalable access to interventions for improving mental health. Most of the scientifically tested apps are based on cognitive behavioral therapy (CBT) principles, which are considered the gold standard for the treatment of most mental health problems.
Objective: This review investigates standalone smartphone-based ecological momentary interventions (EMIs) built on principles derived from CBT that aim to improve mental health.
Methods: We searched the MEDLINE, PsycINFO, EMBASE, and PubMed databases for peer-reviewed studies published between January 1, 2007, and January 15, 2020. We included studies focusing on standalone app-based approaches to improve mental health and their feasibility, efficacy, or effectiveness. Both within- and between-group designs and studies with both healthy and clinical samples were included. Blended interventions, for example, app-based treatments in combination with psychotherapy, were not included. Selected studies were evaluated in terms of their design, that is, choice of the control condition, sample characteristics, EMI content, EMI delivery characteristics, feasibility, efficacy, and effectiveness. The latter was defined in terms of improvement in the primary outcomes used in the studies.
Results: A total of 26 studies were selected. The results show that EMIs based on CBT principles can be successfully delivered, significantly increase well-being among users, and reduce mental health symptoms. Standalone EMIs were rated as helpful (mean 70.8%, SD 15.3; n=4 studies) and satisfying for users (mean 72.6%, SD 17.2; n=7 studies).
Conclusions: Study quality was heterogeneous, and feasibility was often not reported in the reviewed studies, thus limiting the conclusions that can be drawn from the existing data. Together, the studies show that EMIs may help increase mental health and thus support individuals in their daily lives. Such EMIs provide readily available, scalable, and evidence-based mental health support. These characteristics appear crucial in the context of a global crisis such as the COVID-19 pandemic but may also help reduce personal and economic costs of mental health impairment beyond this situation or in the context of potential future pandemics.
Prevalence of Psychiatric Disorders
Heightened prevalence of psychiatric disorders is one of the largest challenges for modern health care. In 2001, the World Health Organization (WHO) estimated that 1 in every 4 individuals worldwide was affected by one or more mental or neurological health problems during their lives . Prospective longitudinal studies suggest that the vast majority of people will have a mental health disorder at some point in their lives [ , ]. In 2017, >970 million people were diagnosed with mental or substance use disorders [ ]. These numbers likely represent a gross underestimate because of underreporting for fear of stigma or limited mental health literacy, limited access to therapy services, high costs of treatment, and additional reasons, such as recent conditions caused by the COVID-19 outbreak.
The global COVID-19 pandemic is likely to further increase the need for mental health care interventions and new ways of implementing them. Global pandemics cause high levels of stress and may lead to mental health problems, such as depression, anger, anxiety disorders, and posttraumatic stress disorder, as well as an increase in smoking and alcohol consumption [, ]. Current measures of physical distancing and quarantine aimed at curbing the spread of COVID-19 likely have additional detrimental psychological side effects, including loneliness [ ]. At the same time, people are considerably less likely to receive professional face-to-face psychological help to overcome their fears, lowered mood, and other mental health problems [ ].
Given the low detection rates of mental health problems, the WHO estimates that 76% to 85% of people in low- and middle-income countries and 35% to 50% of people in high-income countries receive no treatment for their mental health disorder . After the pandemic is over, these numbers will be even higher. Moreover, those who seek treatment often do so only years into the mental disease, meaning that comorbid disorders and difficulties in many domains of life, including family and work, have already developed [ ].
Ecological Momentary Interventions
The rapid growth in the use of smartphones has created a new branch of medicine—mobile health (mHealth). Currently, the most popular solutions in mHealth are mobile apps because they are easy to use and are widely available . Such apps typically use ecological momentary intervention (EMI) to deliver treatments provided to people in their everyday lives [ ]. This approach captures and modifies specific moment-to-moment situations that emerge in the real world rather than targeting problematic thoughts, emotions, and behaviors within therapy sessions or in the hospital [ , ].
Cognitive Behavioral Therapy
Most of the scientifically tested apps are based on cognitive behavioral therapy (CBT) principles. CBT was first created and established by Beck  and Ellis [ ] and is based on the theory that maladaptive cognitions, such as general beliefs and automatic thoughts about the self and the world, contribute to the maintenance of emotional distress and behavioral problems. Accordingly, CBT specifically targets these maladaptive cognitions and behaviors. CBT is one of the most extensively used and researched form of psychotherapy [ ] and is considered the gold standard for the treatment of many mental health problems [ ]. Owing to their strong empirical foundation and clear structure, CBT-based interventions are well suited for application in mHealth. Here, we examine mobile apps that have been designed using a rational app design. With this definition, we refer to apps that are based on a CBT rationale and implement established and empirically validated CBT tools and techniques.
Several reviews and meta-analyses reported the efficacy of EMIs for several mental health problems, including depression, anxiety, perceived stress, and eating disorders [, - ], but they were not focused solely on CBT-based interventions. Other reviews have focused on or have not excluded blended treatments such as face-to-face psychotherapy in combination with an app [ , ]. However, to the best of our knowledge, there are currently no reviews on the efficacy, effectiveness, and feasibility of standalone EMIs delivered via mobile apps and following a rational app design, in both healthy and clinical populations. However, the rapid development of this type of psychological support and recent pandemic-quarantine circumstances have created an urgent need for a comprehensive summary of the studies published so far.
To build a comprehensive overview of the existing literature, 4 search terms were used to identify articles: (1) mental health, (2) smartphones, (3) CBT, and (4) ecological momentary interventions (seefor the complete search strings) in the MEDLINE, PsycINFO, EMBASE, and PubMed databases on January 16, 2020.
The following criteria were used to select the studies: (1) peer-reviewed publications; (2) written in English; (3) published between January 1, 2007, and January 15, 2020; (4) standalone treatments (blended interventions were excluded); (5) explicitly aiming to increase mental health; and (6) focusing on feasibility, efficacy, and/or effectiveness of EMI. Both within- and between-group designs and studies with both healthy and clinical samples were included.
The time range was decided based on the launch date of the first App Store in 2007. This means that the review proposes an overview of presumably all the papers published since the development of the field started.
For selected studies, we assessed the methodological risk of bias of included studies in accordance with the Cochrane Handbook  and the guidelines of the Cochrane Consumers and Communication Review Group [ ], which recommends the explicit reporting of the following individual elements for the studies: random sequence generation, allocation sequence concealment, blinding (participants and personnel), blinding (outcome assessment), completeness of outcome data, selective outcome reporting, and other sources of bias. Overall, from the 14 randomized controlled trials (RCTs) included, 8 were rated as low risk and 6 as medium risk. From the remaining 12 non-RCT studies, 11 were rated as medium risk and 1 as high risk. The inter-rater reliability was 96%. Owing to the small number of studies fulfilling the inclusion criteria, we decided to keep the study with high risk in the review.
Data extraction included (1) study design; (2) choice of the control condition; (3) sample characteristics in terms of sample size and justification of the size in power analysis, gender ratio, age, and mental health problems; (4) content of EMI in terms of specific CBT strategies; (5) EMI delivery characteristics in terms of mode, duration, and frequency; (6) feasibility in terms of acceptance, satisfaction, and helpfulness rates; (6) efficacy or effectiveness in terms of improvement in primary outcomes; and (7) outcome measures used in the studies.
Owing to the heterogeneity of the designs and objectives of the included studies, quantitative analysis was possible only in a few cases, for instance, for sample characteristics, delivery characteristics, and feasibility measures. However, in many aspects, such as effectiveness and effectiveness measures, quantitative analysis was not possible and, instead, a narrative approach for the summary of the qualitative findings was chosen.
Inclusion criteria resulted in the identification of 26 articles (seefor the complete literature flow chart). Of those, 3 [ - ] described the outcomes of 1 trial. For this particular trial, we only included data from the latest publication to avoid bias or duplication. Thus, our results are based on n=24 studies.
Study Design and Control Conditions
A variety of study designs were employed to investigate the effects of EMIs: 2-, 3-, or 4-armed RCTs (12 studies [- ]) as well as a nonrandomized trial with a control group (1 study [ ]) for between-group comparisons or case studies and 1-group-only studies (11 studies [ - ]) for within-group or intraindividual comparisons ( ). Control conditions were selected based on the research questions of a given study. The control conditions in the studies reviewed here include (1) another app with different content (8 studies), (2) a waiting list (4 studies), (3) ecological momentary assessment (EMA)—participants were involved in self-monitoring only (2 studies), (4) encouragement to visit helpful websites or call hotlines (2 studies), (5) computer-based treatment (the same content of the intervention but delivered via internet browser; 1 study), (6) real-life training provided by a qualified person (eg, meditation training; 1 study), and (7) treatment as usual (1 study).
|Study||Names of apps||RCTa||Number of arms||Control condition||Type of control condition|
|Arevian et al, 2018 ||B-RESILIENT app||—b||—||—||—|
|Bakker et al, 2018 ||Mood Prism, Mood Mission, Mood Kit||✓c||4||✓||AAd, AA, WLe|
|Birney et al, 2016 ||Mood Hacker||✓||2||✓||Ef|
|Christoforou et al, 2017 ||Agoraphobia Free, Stress Free||✓||2||✓||AA|
|Dahne et al, 2019 ||¡Aptívate!, iCouch CBT||✓||3||✓||AA, TAUg|
|Donker et al, 2019 ||ZeroPhobia||✓||2||✓||WL|
|Donovan et al, 2016 ||BodiMojo||—||—||—||—|
|Dulin et al, 2014 ||Buddy Steps||—||—||—||—|
|Hidalgo-Mazzei et al, 2018 ||OpenSIMPLe||—||—||—||—|
|Horsch et al, 2017 ||Sleepcare app||✓||2||✓||WL|
|Hur et al, 2018 ||Todac Todac app||✓||2||✓||EMAh|
|Levin et al, 2018 ||—||✓||3||✓||AA, WL|
|Morris et al, 2010 ||Mood Map||—||—||—||—|
|Morrison Wylde et al, 2017 ||Headspace||—||2||✓||RLTi|
|Prada et al, 2016 ||EMOTEO||—||—||—||—|
|Pratap et al, 2018 ||iPST, EVO, Health Tips||✓||3||✓||AA, AA|
|Roncero et al, 2018 ||GGRO||—||—||—||—|
|Roy et al, 2015 ||GETSmart||✓||2||✓||E|
|Schlosser et al, 2018 ||PRIME-D||—||3||—||AA, AA|
|Shrier et al, 2017 ||—||—||—||—||—|
|Stoll et al, 2017 ||REACH app||—||—||—||—|
|Versluis et al, 2018 ||—||✓||3||✓||WL, EMA|
|Watts et al, 2013 ||Get Happy||✓||2||✓||Cj|
|Wen et al, 2017 ||Headspace||—||—||—||—|
aRCT: randomized controlled trial.
bNot applicable or no information.
cUsed in the study design.
dAA: another app with different content.
eWL: waiting list.
fE: encouragement to visit helpful websites or call hotline in case of a difficult situation.
gTAU: treatment as usual.
hEMA: app with ecological momentary assessment.
iRTL: real-life training with a qualified person.
jC: computer-based treatment.
Target Populations and Mental Health Problems
EMIs were implemented in nonclinical populations, including healthy participants, and participants with subthreshold symptoms of a mental disorder (10 studies). In these studies, EMIs aimed to tackle a variety of problems, including high self-criticism or high levels of stress in the workplace (eg, of medical doctors, novice nurses, and corporation workers), all of which were assumed to increase the risk of developing a mental health disorder. In additional studies with healthy populations, EMIs addressed subthreshold posttraumatic stress disorder. EMIs were also implemented for the treatment of mental health issues in clinical populations (14 studies), including patients with depression, insomnia, bipolar disorder, schizophrenia, alcohol addiction, borderline personality, agoraphobia, or obsessive-compulsive disorder ().
|Study||Population||Final sample size (n)||Power analysis||Age (years)||Age (years), mean (SD)||Gender (female %)|
|Arevian et al, 2018 ||Healthy||28||—a||18+||—||40 (11)|
|Bakker et al, 2018 ||Healthy and subthreshold depression and anxiety in screening||198||✓b||18-76||34.2 (14.2)||72.7 (144)|
|Birney et al, 2016 ||Clinical: depression||300||✓||18+||—||—|
|Christoforou et al, 2017 ||Clinical: agoraphobia||142||✓||18+||—||—|
|Dahne et al, 2019 ||Clinical: depression||42||—||18+||36.05 (11.44)||67 (28)|
|Donker et al, 2019 ||Clinical: acrophobia||193||✓||18-65||41.33 (13.68)||66.9 (129)|
|Donovan et al, 2016 ||Healthy||16||—||13-22||16.9 (1.3)||65 (10)|
|Dulin et al, 2014 ||Clinical: alcohol use disorder||28||—||22-45||33.6 (5.6)||46 (13)|
|Hidalgo-Mazzei et al, 2018 ||Clinical: bipolar disorder||103||—||18+||36.59 (11)||63.2 (65)|
|Horsch et al, 2017 ||Clinical: insomnia||151||✓||18-80||39.66 (13.44)||62.3 (94)|
|Hur et al, 2018 ||Clinical: depression||34||—||18-35||23.71 (3.26)||88 (30)|
|Levin et al, 2018 ||Healthy: high self-criticism||87||—||18-52||22.76 (7.02)||69 (60)|
|Morris et al, 2010 ||Healthy: high level of stress||8||—||30-48||37 (5.75)||—|
|Morrison Wylde et al, 2017 ||Healthy: medical workers||95||✓||23-50||—||92 (87)|
|Prada et al, 2016 ||Clinical: borderline||16||—||18-50||30.5 (93)||100 (16)|
|Pratap et al, 2018 ||Clinical: depression||345||—||18-70||34.9 (10.72)||77.1 (266)|
|Roncero et al, 2018 ||Clinical: obsessive-compulsive disorder||20||—||19-26||—||80 (16)|
|Roy et al, 2015 ||Subthreshold PTSDc||13||—||—||—||—|
|Schlosser et al, 2018 ||Clinical: depression||36||—||18+||31.33 (12.4)||78 (28)|
|Shrier et al, 2017 ||Clinical: depression||16||—||16-23||19.6 (—)||100 (16)|
|Stoll et al, 2017 ||Healthy||132||—||8-12||9.65 (0.82)||62.9 (83)|
|Versluis et al, 2018 ||Healthy: high level of stress||128||✓||18+||—||—|
|Watts et al, 2013 ||Clinical: depression||35||—||18-63||41 (12.38)||80 (28)|
|Wen et al, 2017 ||Healthy: medical workers||30||—||—||—||90 (27)|
bPower analysis was conducted.
cPTSD: posttraumatic stress disorder.
Sample sizes varied widely, from n=8 to n=348. Seven studies reported a power analysis to justify their sample size. The age range varied from 8 to 80 years. Over one-third of the studies did not report the age range of participants, informing only that they were aged 18 years or older. Five studies did not report the sex composition of their sample. Those that did included a majority (>70%) of female participants. This was due, in part, to the populations of interest. For instance, 1 study was conducted in novice nurses—a profession that is dominated by women. Two other studies included only women. In summary, the sample characteristics of current EMI studies are heterogeneous, and the quality of sample descriptions varies widely across studies.
Cognitive behavioral techniques provide a range of possible interventions that address diverse and complex patient needs . CBT comprises techniques addressing psychological mechanisms that underpin negative and potentially harmful thoughts and beliefs (eg, reflection and cognitive restructuring) and actions (eg, behavioral activation or social skills training) endorsed by patients.
The most frequently used CBT strategies were cognitive restructuring, including reappraisal (14 studies), self-monitoring (13 studies), reflection (10 studies), and relaxation (8 studies). Cognitive restructuring identifies and changes the maladaptive thoughts and beliefs and reevaluates a given situation .
Self-monitoring is usually implemented in apps that contain an EMA module and is often described as a key component of successful therapy for many mental health conditions [, ]. By getting access to their self-reported data, users become self-aware; manage their symptoms; better understand their mood, behavior, or illness; and work on these factors of mental well-being. Reflection is a metacognitive process that allows an individual to increase his or her psychological mindedness and another central process in CBT [ , ]. The reflection process requires the user to look at her or his experiences from a distance and often poses a starting point for many other therapeutic strategies (eg, reappraisal and gratitude). In addition, reflection was used, for instance, as an outcome of self-monitoring [ , ].
The use of digital technologies opens a new door to deliver CBT strategies. Apart from basic solutions, such as sending messages  or text-based scenarios [ , ], novel approaches were employed in the reviewed studies. For instance, reflection and cognitive restructuring were presented in a fabular comic story of the main character who had depression [ ]. A similar solution was employed in the Agoraphobia free app, which was game based and presented a virtual character who needed to meet the virtual therapist to work on reflection and cognitive restructuring [ ]. Relaxation and meditation exercises were delivered via audio and video tools [ , ]. Self-monitoring outcomes were presented, for instance, as mood cloud, providing a visual representation of the participant's self-reported mood [ ], or in a calendar view, to allow the user to track their behavior day-by-day or even hour-by-hour [ ]. One novel solution combines a mobile app with virtual reality (VR) to treat acrophobia with exposure [ ] or a platform allowing users to contact each other to provide social support that enhances behavioral activation [ ]. All apps in the studies reviewed here used more than one strategy to improve mental health ( ).
|Arevian et al, 2018 ||—n||✓o||—||—||—||—||—||—||—||—||—||—||✓|
|Bakker et al, 2018 p||✓||—||—||—||—||—||—||—||✓||—||—||✓||—|
|Birney et al, 2016 ||✓||✓||—||—||—||—||✓||—||—||—||—||✓||—|
|Christoforou et al, 2017 ||—||✓||—||—||✓||✓||—||—||✓||✓||✓||—||—|
|Dahne et al, 2019 ||✓||—||—||—||—||—||—||—||✓||✓||—||✓||—|
|Donker et al, 2019 p||—||—||—||—||—||✓||—||—||✓||—||—||—||—|
|Donovan et al, 2016 ||—||✓||—||—||—||—||✓||—||—||✓||—||✓||—|
|Dulin et al, 2014 ||✓||—||—||—||—||—||—||✓||✓||—||—||✓||✓|
|Hidalgo-Mazzei et al, 2018 ||—||—||—||—||—||—||—||—||✓||—||—||✓||—|
|Horsch et al, 2017 ||✓||—||—||—||—||—||—||—||✓||—||✓||✓||—|
|Hur et al, 2018 ||—||✓||—||✓||—||—||—||—||—||✓||—||✓||—|
|Levin et al, 2018 ||—||✓||✓||—||—||—||—||—||—||✓||—||—||—|
|Morris et al, 2010 ||—||✓||—||—||—||—||—||—||—||—||✓||✓||—|
|Morrison Wylde et al, 2017 ||—||—||—||—||—||—||✓||—||—||✓||✓||—||—|
|Prada et al, 2016 ||—||—||—||—||✓||—||✓||—||—||—||✓||✓||—|
|Pratap et al, 2018 ||—||✓||—||—||—||—||—||✓||✓||—||—||—||—|
|Roncero et al, 2018 ||✓||✓||—||—||—||—||—||—||—||—||—||—||—|
|Roy et al, 2015 ||—||✓||—||—||—||—||—||—||✓||✓||✓||—||✓|
|Schlosser et al, 2018 ||✓||—||—||—||—||—||✓||—||✓||—||—||—||✓|
|Shrier et al, 2017 ||—||✓||—||—||—||—||—||—||—||✓||—||✓||—|
|Stoll et al, 2017 ||—||✓||—||—||—||—||—||—||—||—||✓||✓||—|
|Versluis et al, 2018 ||—||✓||—||—||—||—||✓||—||—||—||—||✓||—|
|Watts et al, 2013 ||—||✓||—||—||—||—||—||—||✓||✓||—||—||✓|
|Wen et al, 2017 ||—||—||—||—||—||—||✓||—||—||✓||✓||—||—|
aBA: behavioral activation.
bCR: cognitive restructuring, including reappraisal.
hPS: problem solving.
mSST: social skills training.
nTechnique not implemented in the ecological momentary intervention.
oTechnique implemented in the ecological momentary intervention.
pInsufficient information was provided—probably more modules were implemented.
The most common way to deliver EMIs on a smartphone is a mobile app, but ever-growing possibilities are emerging with technological advances. For example, mobile apps can be combined with wristbands to track indicators of physiological function (eg, heartbeat), GPS to track geolocation, or VR.
The duration of interventions ranged from 2 weeks to 6 months, with a median of 30 days. A frequent option was to send prompts to remind participants either about using EMI or about completing the EMA (17 studies). The number of prompts with EMA or reminders varied from 5 per day to once every week (). In 4 apps, prompts were sent only when the participant forgot to use the app for a longer, predefined period. Another way to deliver EMIs was to allow participants to use them anytime they needed to or once per day.
|Study||Intervention duration in days||Prompts sent||Number of prompts per day||Trigger|
|Arevian et al, 2018 ||28||✓a||1||Time based|
|Bakker et al, 2018 ||30||✓ 1/3b||N/Ic||N/Ad|
|Birney et al, 2016 ||42||✓||1||N/I|
|Christoforou et al, 2017 ||84||✓||0.14||N/I|
|Dahne et al, 2019 ||56||✓||0.14||Participant|
|Donker et al, 2019 ||21||✓||0.14||Participant|
|Donovan et al, 2016 ||30||N/A||N/A||Participant|
|Dulin et al, 2014 ||42||✓||1||Random|
|Hidalgo-Mazzei et al, 2018 ||168||✓||1||EMAe|
|Horsch et al, 2017 ||84||✓||1 ALf||Depending on datag|
|Hur et al, 2018 ||21||✓||0.5 ANUh||EMA|
|Levin et al, 2018 ||14||✓||4||Participant and random|
|Morris et al, 2010 ||30||✓||N/I||Participant|
|Morrison Wylde et al, 2017 ||28||N/A||N/A||N/A|
|Prada et al, 2016 ||168||N/A||N/A||N/A|
|Pratap et al, 2018 ||84||✓||1||Participant|
|Roncero et al, 2018 ||15||✓||1||N/I|
|Roy et al, 2015 ||42||✓||1||Depending on data|
|Schlosser et al, 2018 ||56||N/A||N/A||Participant|
|Shrier et al, 2017 ||28||✓||5||Random|
|Stoll et al, 2017 ||1||N/A||N/A||N/A|
|Versluis et al, 2018 ||26||✓||5||Random|
|Watts et al, 2013 ||56||N/A||N/A||N/A|
|Wen et al, 2017 ||30||N/A||N/A||Participant|
aPrompts were sent by ecological momentary intervention.
b1/3: in 1 of the 3 apps introduced in the study (when not marked, all apps introduced in the study were sending beeps).
cN/I: no information.
dN/A: not applicable.
eEMA: ecological momentary assessment.
fAL: at least (depending on participants’ reports).
gDepending on data: depending on participants’ report other than ecological momentary assessment (eg, sleep hygiene).
hANU: beeps were sent only when app was not used.
Triggers can be divided into 3 categories: time based, event based, and randomized within a specific time frame. In 10 studies, event-based triggers were employed to allow participants to trigger EMI themselves. For example, participants were instructed to use the app anytime they felt they might benefit from the EMI (). Another trigger from this category was EMA (used in 2 studies): when problems or symptoms were reported (eg, low mood or above-threshold stress level), the participant received an EMI. This just-in-time adaptive intervention (JITAI) aims to provide the right type or amount of support, at the right time, by adapting to an individual's changing internal and contextual state [ ]. Four studies used randomized prompt-sending as an EMI trigger to avoid prepared answers from participants. One app sent prompts every morning.
The EMI frequency varied among studies. The range of the frequency of EMI delivery was from 3 times per day to once per week. Four studies did not report the number of delivered EMIs.
Taken together, the frequency, triggers, and duration of the intervention varied and depended on the target group, targeted problem, and CBT techniques implemented. A few studies did not report this information, including default or the average number of prompts per participant or duration of intervention for different participants.
For this review, feasibility was indexed by compliance rate and participants’ satisfaction with the intervention.
Compliance reporting was variable. First, the definition of compliance differed across studies. Some researchers reported how many training sessions participants completed, how many prompts participants answered, or the number of participants who did not drop out of the study. Others set thresholds, such as reacting to at least one or two prompts and subsequently excluded all participants with lower adherence. For the purpose of this review, we defined compliance rate as the number of reactions to prompts. Nineteen studies did not report compliance. None of the studies with healthy samples reported compliance rates.
In other studies, compliance ranged from 33.8% to 93.3% (7 studies; mean 64, SD 22). There was no relationship between compliance rate and duration as well as the overall time cost of the intervention. For instance, one study reported a compliance rate of 55.6% for a 15-day intervention with 1 prompt per day , and another reported a compliance rate of 33.8% for a 168-day intervention with 1 prompt per day [ ], whereas other studies reported a compliance rate of 90.5% for a 56-day intervention with 1 prompt per week [ ] or a compliance rate of 93.3% for a 42-day intervention with 1 prompt per day [ ]. All studies with higher than average compliance rates included participants with depression. Lower compliance rates were reported in a study of participants with bipolar disorder.
Satisfaction and helpfulness rates were underreported (satisfaction was reported in 7 studies and helpfulness in 4 studies) but rated positively (mean perceived satisfaction 72.6%, SD 17.2 and mean perceived helpfulness 70.8%, SD 15.3). Helpfulness was rated highest by individuals with borderline personality disorder and healthy samples with high self-criticism. Healthy populations, especially adolescents, were generally more satisfied with the EMIs compared with clinical populations (). The least satisfied were users with bipolar disorder; however, only 1 study was conducted on such a sample. Discrepancies were found in satisfaction rates reported by depressed samples, ranging from 46% and 54% (which were the lowest reported rates) to 91.8% (which was one of the highest rates). However, only 3 of 7 studies that delivered EMI to depressed populations reported these numbers.
|Study||Compliance in %||Helpfulness in %||Satisfaction in %||Primary outcome measure||Improvement in primary outcome||Secondary outcome measure|
|Arevian et al, 2018 ||—a||—||—||—||—||—|
|Bakker et al, 2018 ||—||—||—||PHQb, GADc, WEMWBSd||✓e||ESAS-Rf, CSESg, MHLQh|
|Birney et al, 2016 ||93||—||46||PHQ||—||BADSi, ATQ-Rj, KTk|
|Christoforou et al, 2017 ||52||—||—||PASl||✓||Fm|
|Dahne et al, 2019 ||91||—||—||PHQ||✓||F|
|Donker et al, 2019 ||—||—||—||AQn||✓||ATHQo, BDI-IIp, IPQq, PHQ, Mr, SUSs|
|Donovan et al, 2016 ||—||64||92||—||—||—|
|Dulin et al, 2014 ||—||—||—||TLFBt||✓||F|
|Hidalgo-Mazzei et al, 2018 u||33.8||—||62||WHO-5v, SF-36w||✓||F|
|Horsch et al, 2017 ||49.7||—||—||ISIx||✓||PSQIy, DBAS-16z, HADSaa, CES-Dab|
|Hur et al, 2018 ||—||—||—||DASac, BDI-II, STAIad, RSESae||✓||QoLaf|
|Levin et al, 2018 ||—||78 and 83||81 and 81||FSCRSag||✓||DASSah, WSASai|
|Morris et al, 2010 ||—||—||—||Iaj, EMASSak||✓||—|
|Morrison Wylde et al, 2017 ||—||—||—||CFSTal||✓||LECam, PCL-Can, FFMQao|
|Prada et al, 2016 ||—||82||—||EMAATap||✓||—|
|Pratap et al, 2018 u||—||—||—||PHQ, SDSaq||—||CRar|
|Roncero et al, 2018 ||56||—||—||OCI-Ras, OBQat||✓||ROCIau, PROCSIav, DASS|
|Roy et al, 2015 ||—||—||—||PCLaw||✓||PHQ, GAD|
|Schlosser et al, 2018 ||—||—||72||PHQ, SDS||✓||—|
|Shrier et al, 2017 ||—||—||—||—||—||—|
|Stoll et al, 2017 ||—||—||92||—||—||—|
|Versluis et al, 2018 ||—||47||—||HRax||—||IPANATay, IATaz, ERIba, PSWQbb, GAD, PHQ, FFMQ, CEQbc, EMASEbd, F|
|Watts et al, 2013 ||69||—||54||PHQ||✓||K-10be, BDI-II, CEQ, SDS, ERSbf, HRSbg|
|Wen et al, 2017 ||—||—||—||PANASbh, FMIbi||✓||—|
bPHQ: Patient Health Questionnaire.
cGAD: Generalized Anxiety Disorder Scale.
dWEMWBS: Warwick-Edinburgh Mental Well-Being Scale.
eStudy with improvement in the primary outcomes. Improvement in primary outcomes=results statistically significant in at least one measure of the primary outcomes.
fESAS-R: Emotional Self-Awareness Scale-Revised.
gCSES: Coping Self-Efficacy Scale.
hMHLQ: Mental Health Literacy Questionnaire.
iBADS: Behavioral Activation for Depression Scale.
jATQ-R: Automatic Thoughts Questionnaire-Revised.
kKT: knowledge test.
lPAS: Panic and Agoraphobia Scale.
mF: feasibility measures.
nAQ: Acrophobia Questionnaire.
oATHQ: Attitudes Toward Heights Questionnaire.
pBDI-II: Beck Depression Inventory II.
qIPQ: Igroup Presence Questionnaire.
sSUS: System Usability Scale.
tTLFB: The Timeline Followback.
uTo provide information about efficacy, we reversed the order of the outcomes.
vWHO-5: World Health Organization 5-point Well-Being Index.
wSF-36: Short Form Health Survey.
xISI: Insomnia Severity Index.
yPSQI: Pittsburgh Sleep Quality Index.
zDBAS-16: Dysfunctional Beliefs and Attitudes about Sleep.
aaHADS: Hospital Anxiety and Depression Scale.
abCES-D: Center of Epidemiologic Studies Depression Scale.
acDAS: Dysfunctional Attitude Scale.
adSTAI: State-Trait Anxiety Inventory.
aeRSES: Rosenberg Self-Esteem Scale.
afQoL: quality of life.
agFSCRS: Forms of Self-Criticism and Self-Reassurance Scale.
ahDASS: Depression, Anxiety, and Stress Scale.
aiWSAS: Work and Social Adjustment Scale.
akEMASS: Ecological Momentary Assessment-Stress Scale.
alCFST: Compassion Fatigue Self-Test.
amLEC: Life Events Checklist.
anPCL-C: Posttraumatic Stress Disorder Checklist-Civilian Version.
aoFFMQ: Five Facet Mindfulness Questionnaire.
apEMAAT: Ecological Momentary Assessment-Aversive Tension Scale.
aqSDS: Sheenan Disability Scale.
arCR: comparison of recruitment and engagement in a fully remote trial of individuals with depression who either self-identify as Hispanic/Latino or not.
asOCI-R: Obsessive-Compulsive Inventory-Revised.
atOBQ: Obsessive Beliefs Questionnaire.
auROCI: Relationship Obsessive-Compulsive Inventory.
avPROCSI: Partner-Related Obsessive-Compulsive Symptoms Inventory.
awPCL: Posttraumatic Stress Disorder Checklist.
axHR: heart rate.
ayIPANAT: Implicit Positive and Negative Affect Test.
azIAT: Implicit Association Test.
baERI: Effort-Reward Imbalance Questionnaire.
bbPSWQ: Penn State Worry Questionnaire.
bcCEQ: Credibility/Expectancy Questionnaire.
bdEMASE: Ecological Momentary Assessment-Stress and Explicit Affect.
beK-10: Kessler 10-Item Psychological Distress Scale.
bfERS: Environment Rating Scale.
bgHRS: Homework Rating Scale.
bhPANAS: Positive and Negative Affect Scale.
biFMI: Freiburg Mindfulness Scale.
Efficacy and Effectiveness
Efficacy or effectiveness was reported in 21 studies. In the 8 studies reviewed here, the authors used both these terms alternately to report the outcomes of EMI. Hence, both effectiveness and efficacy are reported in this review, and the particular term is used as indicated by the authors of the studies.
In total, 16 of the 21 interventions included in our review reported evidence of a significant reduction in mental disorder symptoms ().
Nine studies reported effect sizes (Cohen d). In the EMI study for insomnia, the authors found significant interaction effects between time and condition on the primary outcome measures (d=−0.66) and sleep efficiency (d=0.71) . A significant effect on insomnia severity was also recorded at the 3-month follow-up. In the study investigating high self-criticism, effect sizes for between-group comparison and for all primary outcome measures ranged from 0.46 to 0.72 (except inadequate self-criticism, which was nonsignificant) [ ]. Another study assessing the effectiveness of EMI in individuals with acrophobia found a between-group effect size d=1.14 [ ].
The within-group effect sizes varied from d=0.47 to d=0.64 and d=0.67 for depressed samples and changes in overall health, as indexed by the Patient Health Questionnaire. One study found effect sizes of d=0.77 for measures of mindfulness in a medical workers’ sample and a moderate effect size d=0.38 for positive affect . A large effect (d=1.0) was found in the study on a population with alcohol use disorder [ ]. The only effect sizes that were small or null were found in the study investigating EMI helping in obsessive-compulsive disorder [ ]. Taken together, the effect sizes for a reduction in primary outcomes mostly varied from moderate to large, with large effects found for the majority of the reviewed EMIs.
The most common method to assess efficacy and effectiveness was with questionnaires, and almost all the studies using this method found statistically significant differences (in both between- and within-group comparisons) in the primary outcomes (). With respect to other measures of primary outcomes, 1 study used heart rate [ ] and 2 other studies calculated effectiveness based on EMA data [ , ]. Although the outcomes based on EMA were statistically significant, there were no effects found based on physiological readouts.
As for secondary outcomes, 1 of the studies employed a knowledge test about depression and found statistically significant positive effects for the program on information and knowledge . In another study, the authors added an Implicit Positive and Negative Affect Test to assess the level of unconscious stress and found that implicit stress did decrease over time for all participants, independent of the intervention [ ].
We reviewed 26 studies that investigated the feasibility, efficacy, and effectiveness of standalone CBT-based EMIs to increase mental health. Results show that EMIs can be successfully delivered via smartphones, significantly increase well-being among users, and reduce symptoms of mental disorders. Designs and quality of the studies reviewed using a rational app design were heterogeneous. Across studies, EMIs were generally accepted by users with various age, sex, education background, and professions and were shown to be effective treatments for a broad range of psychological symptoms.
CBT assumptions constitute an evidence-based basis for EMIs, especially with the use of techniques involving self-monitoring, reflection, cognitive restructuring, and relaxation. Such techniques were mostly employed in combination, and users were sometimes provided with a personal choice of which approach they preferred.
Many studies investigated the efficacy of reducing symptoms of depression in clinical populations. The Todac Todac app, for instance, significantly reduced both depression and anxiety symptoms . The Get Happy program was associated with stable reduction of depressive symptoms, including a demonstration of stability at 3 months follow-up [ ]. Promising results were also shown in studies with healthy samples, where an increase in positive well-being and higher emotional self-awareness along with self-efficacy as a result of using EMI were reported. Here, the MoodKit, MoodMission, and MoodPrism apps were effective in decreasing mild depressive and anxiety symptoms [ ]. Headspace is a mindfulness app that was used in 2 studies with medical workers and tested against a control group that received face-to-face mindfulness classes. Users engaging in Headspace reported higher self-awareness in their actions, less burnout, and more compassion satisfaction compared with the group receiving face-to-face mindfulness training [ ]. In another study of the Headspace app, a single group design was chosen. This study reported heightened mindfulness skills and more positive emotions as a result of using Headspace post-EMI versus pre-EMI in medical workers [ ].
From a practical clinical perspective, EMIs may postpone the necessity of professional mental health treatment by a psychiatrist or psychologist, eg, for individuals who struggle financially, have limited access to such services in their place of living, or due to health reasons cannot attend psychotherapy [, ]. On the basis of this review and previous studies, EMIs may be specifically helpful for individuals who are facing temporary problems, for instance, in the context of personal or professional change or other transitioning periods. Moreover, symptoms of mental health are still affected by considerable stigmatization [ ], and EMIs may help reduce the fear of stigma as psychological support can be obtained without seeking treatment in a hospital setting. EMIs are also often a first step for many individuals to test personal preferences and affinity to psychological treatment and may lead to face-to-face or other professional therapeutic services later [ ]. The worldwide long-lasting crisis of mental health care, manifested by poor accessibility of cost-free, state-funded psychotherapy, suggests another use of EMIs. Patients could use EMIs as a source of support while awaiting face-to-face treatment [ - ].
EMIs in Context of Pandemics
In the context of COVID-19, the need to implement population-based behavioral measures to counter the rapid spread of the virus has led to home confinement; loss of individual behavioral routines, including exercise; and lack of social contact. Psychological needs are often underaddressed in such times of crises [, ]. EMIs could be an ideally suited approach to reach a large number of individuals affected by mental health problems and to provide scalable global mental health solutions. It is also estimated, based on experiences with previous pandemics, that after the COVID-19 pandemic is over, such solutions will be needed to help populations with continued psychological symptoms, such as depression and anxiety [ ]. Some populations might be at a specific risk of stress in this challenging time, especially doctors, nurses, or emergency personnel [ - ], and 2 of the trials reviewed here investigated effects on the medical staff. The high efficacy of the Headspace app tested in these populations of first responders and frontline workers is encouraging and indicates that mindfulness training delivered via a mobile app can significantly decrease work-related stress. EMIs could thus help to lower the risk of developing unfavorable mental health outcomes in such populations, which have recently been documented [ ].
There are several methodological caveats in the studies reviewed here. The choice of control condition varied. Suggestions for the control groups to call hotlines or visit recommended websites imply heterogeneous and less tangible control groups than control groups that could use comparison apps. At the same time, comparing results from 2 different apps can be difficult if they target different symptoms or use different techniques. Little attention has been paid to the psychological mechanisms (eg, cognitive mediators) that allowed for this change in the outcome to develop. To increase the conclusiveness and reliability of studies assessing efficacy, effectiveness, and feasibility of EMIs, more RCTs with adequate control groups are needed . Ideally, RCTs should be preceded by optimization studies, based, for instance, on MOST (Multiphase Optimization Strategy) [ ] and SMART (The Sequential Multiple Assignment Randomized Trial) [ ], which allow for choosing the right dosage and delivery of intervention based on predefined criteria, for example, individual characteristics of the user, characteristics of psychiatric population, and effectiveness of the intervention. Another valid concern is the reporting of EMI studies. For example, the majority (73%) of the studies did not report a formal power analysis. Consequently, studies might be underpowered, and their quantitative results have not been reported. However, some studies investigated the feasibility rather than efficacy and focused on issues of implementation, launch, and users’ experience with the app; thus, they might have been less concerned about power and sample size. Moreover, it must be noted that there might also be a bias in overestimating the effects of EMI. First, control conditions, if used, cannot simply be blinded, similar to psychotherapy studies. Second, the fact that almost all studies showed a positive effect on the primary outcome makes it likely that there is a publication bias toward positive results.
Despite these limitations, we indicated key strengths and found a number of standalone smartphone-based EMIs that were shown to be effective. It is also in line with findings from previous reviews in this field, although their focus differed, for instance, by focusing on blended therapies [, ] or specific mental health problems [ , - ].
Recommendations for Future Development
Exciting future developments lie ahead in the field of EMIs. This is documented by >40 (pre)registered protocols, which we excluded () and which are likely to produce further evidence about the effectiveness of CBT-based standalone EMIs. Future developments should exploit the potential to scale these interventions and test their effects in larger and global populations in times of crises, such as the COVID-19 pandemic. Moreover, it is possible to combine EMIs with wearable technologies [ ] to gather physiological data and other information for researchers and clinicians alike. EMIs can be used along passive sensing tracking patterns of smartphone usage, and by using context-aware assessment strategies that link the assessment of experiences to specific sensing events (like, for instance, physical distance to the others), would allow for the design of specific interventions in response to COVID-19 pandemic–induced mental health problems, especially induced by isolation whereas, at the same time, complying with the demanded pandemic containment policies. Here, mobile crowdsensing in combination with EMI would constitute the key technology to address this major mental health challenge. Ideally, these tools can be combined with machine learning to derive algorithms for accurate and precise prediction and selection of ecologically valid treatment options [ ]. Machine learning, or more generally artificial intelligence techniques, could be useful in at least three ways. First, such approaches could identify individuals and/or time points where the intervention might be particularly important to avoid substantial degradation of mental health.
Second, such approaches could identify factors that critically affect effectiveness, which could subsequently help in personalizing EMIs both delivery mode and contentwise, and in enhancing decision making at precise time points while using personally tailored JITAIs.
Third, they could significantly decrease the time invested by health professionals and scientists who administer EMIs, such as in the study where clinicians manually adapted the number of EMIs delivered based on app usage .
Taken together, our review showed that standalone CBT-based EMIs can be efficacious in reducing symptoms of psychological disorders in various situations. EMIs may support individuals outside the therapists’ room and could also be helpful when time and resource investment from mental health professionals are limited. At the same time, however, professional clinician-scientists’ input is invaluable in designing and implementing such EMIs. All EMIs reviewed here were based on a rational app design that translates key concepts and findings from CBT to these new interventions. They can provide scalable and evidence-based mental health support for large populations and be readily distributed.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777084. This study reflects only the authors' view and the European Commission is not responsible for any use that may be made of the information it contains.
MM, LS, and BK designed the study. Screening, full text review, and data extraction were conducted by MM (100%) and checked (100%) by JR. Quality assessment of the studies was conducted by MM and CW. MM wrote the first draft of the manuscript. All authors contributed to and have approved the final manuscript.
Conflicts of Interest
Complete search strings.DOC File , 25 KB
- Mental Disorders Affect One in Four People. World Health Organization. URL: https://www.who.int/whr/2001/media_centre/press_release/en/ [accessed 2020-10-01]
- Copeland W, Shanahan L, Costello EJ, Angold A. Cumulative prevalence of psychiatric disorders by young adulthood: a prospective cohort analysis from the great smoky mountains study. J Am Acad Child Adolesc Psychiatry 2011 Mar;50(3):252-261 [FREE Full text] [CrossRef] [Medline]
- Moffitt TE, Caspi A, Taylor A, Kokaua J, Milne BJ, Polanczyk G, et al. How common are common mental disorders? Evidence that lifetime prevalence rates are doubled by prospective versus retrospective ascertainment. Psychol Med 2010 Jun;40(6):899-909 [FREE Full text] [CrossRef] [Medline]
- Ritchie H, Roser M. Mental Health. Our World Data Internet. 2018. URL: https://ourworldindata.org/mental-health [accessed 2020-10-01]
- Pfefferbaum B, Schonfeld D, Flynn B, Norwood AE, Dodgen D, Kaul RE, et al. The H1N1 crisis: a case study of the integration of mental and behavioral health in public health crises. Disaster Med Public Health Prep 2012 Mar;6(1):67-71. [CrossRef] [Medline]
- Reissman D, Watson P, Klomp R, Tanielian T, Prior S. Pandemic influenza preparedness: adaptive responses to an evolving challenge. J Homel Secur Emerg Manag 2006;3(2):- [FREE Full text] [CrossRef]
- Brooks S, Webster R, Smith L, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet 2020 Mar;395(10227):912-920 [FREE Full text] [CrossRef]
- Duan L, Zhu G. Psychological interventions for people affected by the COVID-19 epidemic. Lancet Psychiatry 2020 Apr;7(4):300-302 [FREE Full text] [CrossRef] [Medline]
- Mental Disorders. World Health Organization. URL: https://www.who.int/news-room/fact-sheets/detail/mental-disorders [accessed 2020-10-01]
- Wang PS, Angermeyer M, Borges G, Bruffaerts R, Tat Chiu W, de Girolamo G, et al. Delay and failure in treatment seeking after first onset of mental disorders in the World Health Organization's World Mental Health Survey Initiative. World Psychiatry 2007 Oct;6(3):177-185. [Medline]
- Donker T, Cornelisz I, van Klaveren C, van Straten A, Carlbring P, Cuijpers P, et al. Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: a randomized clinical trial. JAMA Psychiatry 2019 Jul 1;76(7):682-690 [FREE Full text] [CrossRef] [Medline]
- Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Mar;15(Pt 1):1-39 [FREE Full text] [CrossRef] [Medline]
- Wilhelm FH, Grossman P. Emotions beyond the laboratory: theoretical fundaments, study design, and analytic strategies for advanced ambulatory assessment. Biol Psychol 2010 Jul;84(3):552-569. [CrossRef] [Medline]
- Myin-Germeys I, Klippel A, Steinhart H, Reininghaus U. Ecological momentary interventions in psychiatry. Curr Opin Psychiatry 2016 Jul;29(4):258-263. [CrossRef] [Medline]
- Beck AT. Cognitive therapy: nature and relation to behavior therapy. Behav Ther 1970 May;1(2):184-200. [CrossRef]
- Ellis A. Reason and Emotion in Psychotherapy. Oxford, England: Lyle Stuart; 1962.
- Butler AC, Chapman JE, Forman EM, Beck AT. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin Psychol Rev 2006 Jan;26(1):17-31. [CrossRef] [Medline]
- David D, Cristea I, Hofmann SG. Why cognitive behavioral therapy is the current gold standard of psychotherapy. Front Psychiatry 2018;9:4 [FREE Full text] [CrossRef] [Medline]
- Loo Gee B, Griffiths KM, Gulliver A. Effectiveness of mobile technologies delivering ecological momentary interventions for stress and anxiety: a systematic review. J Am Med Inform Assoc 2016 Jan;23(1):221-229. [CrossRef] [Medline]
- Versluis A, Verkuil B, Spinhoven P, van der Ploeg MM, Brosschot JF. Changing mental health and positive psychological well-being using ecological momentary interventions: a systematic review and meta-analysis. J Med Internet Res 2016 Jun 27;18(6):e152 [FREE Full text] [CrossRef] [Medline]
- Schueller SM, Aguilera A, Mohr DC. Ecological momentary interventions for depression and anxiety. Depress Anxiety 2017 Jun;34(6):540-545. [CrossRef] [Medline]
- Anastasiadou D, Folkvord F, Lupiañez-Villanueva F. A systematic review of mHealth interventions for the support of eating disorders. Eur Eat Disord Rev 2018 Sep;26(5):394-416. [CrossRef] [Medline]
- Lan A, Lee A, Munroe K, McRae C, Kaleis L, Keshavjee K, et al. Review of cognitive behavioural therapy mobile apps using a reference architecture embedded in the patient-provider relationship. Biomed Eng Online 2018 Dec 17;17(1):183 [FREE Full text] [CrossRef] [Medline]
- Erbe D, Eichert H, Riper H, Ebert DD. Blending face-to-face and internet-based interventions for the treatment of mental disorders in adults: systematic review. J Med Internet Res 2017 Sep 15;19(9):e306 [FREE Full text] [CrossRef] [Medline]
- Rathbone AL, Clarry L, Prescott J. Assessing the efficacy of mobile health apps using the basic principles of cognitive behavioral therapy: systematic review. J Med Internet Res 2017 Nov 28;19(11):e399 [FREE Full text] [CrossRef] [Medline]
- Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. New York, USA: The Cochrane Collaboration; 2011.
- Ryan R, Hill S, Prictor M. Cochrane Consumers and Communication Review Group. Cochrane Consumers and Communication. URL: http://cccrg.cochrane.org/authorresources [accessed 2020-09-30]
- Anguera JA, Jordan JT, Castaneda D, Gazzaley A, Areán PA. Conducting a fully mobile and randomised clinical trial for depression: access, engagement and expense. BMJ Innov 2016 Jan;2(1):14-21 [FREE Full text] [CrossRef] [Medline]
- Arean PA, Hallgren KA, Jordan JT, Gazzaley A, Atkins DC, Heagerty PJ, et al. The use and effectiveness of mobile apps for depression: results from a fully remote clinical trial. J Med Internet Res 2016 Dec 20;18(12):e330 [FREE Full text] [CrossRef] [Medline]
- Pratap A, Renn BN, Volponi J, Mooney SD, Gazzaley A, Arean PA, et al. Using mobile apps to assess and treat depression in Hispanic and Latino populations: fully remote randomized clinical trial. J Med Internet Res 2018 Aug 9;20(8):e10130 [FREE Full text] [CrossRef] [Medline]
- Birney AJ, Gunn R, Russell JK, Ary DV. MoodHacker mobile web app with email for adults to self-manage mild-to-moderate depression: randomized controlled trial. JMIR Mhealth Uhealth 2016 Jan 26;4(1):e8 [FREE Full text] [CrossRef] [Medline]
- Bakker D, Kazantzis N, Rickwood D, Rickard N. A randomized controlled trial of three smartphone apps for enhancing public mental health. Behav Res Ther 2018 Oct;109:75-83. [CrossRef] [Medline]
- Christoforou M, Sáez Fonseca JA, Tsakanikos E. Two novel cognitive behavioral therapy-based mobile apps for agoraphobia: randomized controlled trial. J Med Internet Res 2017 Nov 24;19(11):e398 [FREE Full text] [CrossRef] [Medline]
- Dahne J, Collado A, Lejuez CW, Risco CM, Diaz VA, Coles L, et al. Pilot randomized controlled trial of a Spanish-language behavioral activation mobile app (¡Aptívate!) for the treatment of depressive symptoms among united states Latinx adults with limited English proficiency. J Affect Disord 2019 May 1;250:210-217 [FREE Full text] [CrossRef] [Medline]
- Donker T, Cornelisz I, van Klaveren C, van Straten A, Carlbring P, Cuijpers P, et al. Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: a randomized clinical trial. JAMA Psychiatry 2019 Jul 1;76(7):682-690 [FREE Full text] [CrossRef] [Medline]
- Horsch CH, Lancee J, Griffioen-Both F, Spruit S, Fitrianie S, Neerincx MA, et al. Mobile phone-delivered cognitive behavioral therapy for insomnia: a randomized waitlist controlled trial. J Med Internet Res 2017 Apr 11;19(4):e70 [FREE Full text] [CrossRef] [Medline]
- Hur J, Kim B, Park D, Choi S. A scenario-based cognitive behavioral therapy mobile app to reduce dysfunctional beliefs in individuals with depression: a randomized controlled trial. Telemed J E Health 2018 Sep;24(9):710-716. [CrossRef] [Medline]
- Levin ME, Haeger J, An W, Twohig MP. Comparing cognitive defusion and cognitive restructuring delivered through a mobile app for individuals high in self-criticism. Cogn Ther Res 2018 Jul 13;42(6):844-855. [CrossRef]
- Roy MJ, Costanzo ME, Highland KB, Olsen C, Clayborne D, Law W. An app a day keeps the doctor away: guided education and training via smartphones in subthreshold post traumatic stress disorder. Cyberpsychol Behav Soc Netw 2017 Aug;20(8):470-478. [CrossRef] [Medline]
- Versluis A, Verkuil B, Spinhoven P, Brosschot J. Effectiveness of a smartphone-based worry-reduction training for stress reduction: a randomized-controlled trial. Psychol Health 2018 Sep;33(9):1079-1099. [CrossRef] [Medline]
- Watts S, Mackenzie A, Thomas C, Griskaitis A, Mewton L, Williams A, et al. CBT for depression: a pilot RCT comparing mobile phone vs. computer. BMC Psychiatry 2013 Mar 7;13:49 [FREE Full text] [CrossRef] [Medline]
- Schlosser DA, Campellone TR, Truong B, Anguera JA, Vergani S, Vinogradov S, et al. The feasibility, acceptability, and outcomes of PRIME-D: a novel mobile intervention treatment for depression. Depress Anxiety 2017 Jun;34(6):546-554 [FREE Full text] [CrossRef] [Medline]
- Arevian AC, O'Hora J, Jones F, Mango J, Jones L, Williams PG, et al. Participatory technology development to enhance community resilience. Ethn Dis 2018;28(Suppl 2):493-502 [FREE Full text] [CrossRef] [Medline]
- Donovan E, Rodgers RF, Cousineau TM, McGowan KM, Luk S, Yates K, et al. Brief report: feasibility of a mindfulness and self-compassion based mobile intervention for adolescents. J Adolesc 2016 Dec;53:217-221. [CrossRef] [Medline]
- Dulin PL, Gonzalez VM, Campbell K. Results of a pilot test of a self-administered smartphone-based treatment system for alcohol use disorders: usability and early outcomes. Subst Abus 2014;35(2):168-175 [FREE Full text] [CrossRef] [Medline]
- Hidalgo-Mazzei D, Reinares M, Mateu A, Nikolova VL, Bonnín CD, Samalin L, et al. OpenSIMPLe: a real-world implementation feasibility study of a smartphone-based psychoeducation programme for bipolar disorder. J Affect Disord 2018 Dec 1;241:436-445. [CrossRef] [Medline]
- Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, et al. Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. J Med Internet Res 2010 Apr 30;12(2):e10 [FREE Full text] [CrossRef] [Medline]
- Wylde CM, Mahrer NE, Meyer RM, Gold JI. Mindfulness for novice pediatric nurses: smartphone application versus traditional intervention. J Pediatr Nurs 2017;36:205-212. [CrossRef] [Medline]
- Prada P, Zamberg I, Bouillault G, Jimenez N, Zimmermann J, Hasler R, et al. Emoteo: a smartphone application for monitoring and reducing aversive tension in borderline personality disorder patients, a pilot study. Perspect Psychiatr Care 2017 Oct;53(4):289-298. [CrossRef] [Medline]
- Roncero M, Belloch A, Doron G. A novel approach to challenging OCD related beliefs using a mobile-app: an exploratory study. J Behav Ther Exp Psychiatry 2018 Jun;59:157-160. [CrossRef] [Medline]
- Stoll RD, Pina AA, Gary K, Amresh A. Usability of a smartphone application to support the prevention and early intervention of anxiety in youth. Cogn Behav Pract 2017 Nov;24(4):393-404 [FREE Full text] [CrossRef] [Medline]
- Shrier LA, Spalding A. 'Just take a moment and breathe and think': young women with depression talk about the development of an ecological momentary intervention to reduce their sexual risk. J Pediatr Adolesc Gynecol 2017 Mar;30(1):116-122. [CrossRef] [Medline]
- Wen L, Sweeney TE, Welton L, Trockel M, Katznelson L. Encouraging mindfulness in medical house staff via smartphone app: a pilot study. Acad Psychiatry 2017 Oct;41(5):646-650. [CrossRef] [Medline]
- Cully J, Teten AL. A Therapist's Guide to Brief Cognitive Behavioral Therapy. MIRECC / CoE Home - Veterans Affairs. 2008. URL: https://www.mirecc.va.gov/visn16/docs/therapists_guide_to_brief_cbtmanual.pdf [accessed 2020-10-22]
- Snyder M. Self-monitoring process. In: Advances in Experimental Social Psychology. New York, USA: Academic Press; 1979.
- Snyder M, Gangestad S. On the nature of self-monitoring: matters of assessment, matters of validity. J Pers Soc Psychol 1986 Jul;51(1):125-139. [CrossRef] [Medline]
- Grant AM. Rethinking psychological mindedness: metacognition, self-reflection, and insight. Behav Change 2012 Feb 22;18(1):8-17. [CrossRef]
- Bennett-Levy J, Thwaites R, Chaddock A, Davis M. Reflective Practice in Cognitive Behavioural Therapy: the Engine of Lifelong Learning. New York, USA: Open University Press; 2001.
- Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-time adaptive interventions (JITAIS) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2018 May 18;52(6):446-462 [FREE Full text] [CrossRef] [Medline]
- Corrigan PW, Morris SB, Michaels PJ, Rafacz JD, Rüsch N. Challenging the public stigma of mental illness: a meta-analysis of outcome studies. Psychiatr Serv 2012 Oct;63(10):963-973. [CrossRef] [Medline]
- Neary M, Schueller SM. State of the field of mental health apps. Cogn Behav Pract 2018 Nov;25(4):531-537. [CrossRef]
- Bójko M, Maślankiewicz R. [ The situation in Polish child psychiatry at the turn of 2018 and 2019 based on data collected through access to public information]. [Child Abused. Theory, research, practice] 2020;19(1):116-154 [FREE Full text]
- Baker C. Mental Health Statistics for England: Prevalence, Services and Funding. Research Briefings. 2020. URL: https://researchbriefings.parliament.uk/ResearchBriefing/Summary/SN06988 [accessed 2020-10-22]
- Grünzig SD, Bengel J, Göhner W, Violetta Krämer L. [low-intensity interventions to reduce depressive symptoms before outpatient psychotherapy - a systematic literature review]. Psychother Psychosom Med Psychol 2019 Jun;69(6):212-223. [CrossRef] [Medline]
- Xiang Y, Yang Y, Li W, Zhang L, Zhang Q, Cheung T, et al. Timely mental health care for the 2019 novel coronavirus outbreak is urgently needed. Lancet Psychiatry 2020 Mar;7(3):228-229 [FREE Full text] [CrossRef] [Medline]
- Douglas PK, Douglas DB, Harrigan DC, Douglas KM. Preparing for pandemic influenza and its aftermath: mental health issues considered. Int J Emerg Ment Health 2009;11(3):137-144. [Medline]
- Goulia P, Mantas C, Dimitroula D, Mantis D, Hyphantis T. General hospital staff worries, perceived sufficiency of information and associated psychological distress during the A/H1N1 influenza pandemic. BMC Infect Dis 2010 Nov 9;10:322 [FREE Full text] [CrossRef] [Medline]
- Liu Z, Han B, Jiang R, Huang Y, Ma C, Wen J. Mental health status of doctors and nurses during covid-19 epidemic in China. Soc Sci Netw 2020:-. [CrossRef]
- Matsuishi K, Kawazoe A, Imai H, Ito A, Mouri K, Kitamura N, et al. Psychological impact of the pandemic (H1N1) 2009 on general hospital workers in Kobe. Psychiatry Clin Neurosci 2012 Jun;66(4):353-360 [FREE Full text] [CrossRef] [Medline]
- Lai J, Ma S, Wang Y, Cai Z, Hu J, Wei N, et al. Factors associated with mental health outcomes among health care workers exposed to coronavirus disease 2019. JAMA Netw Open 2020 Mar 2;3(3):e203976 [FREE Full text] [CrossRef] [Medline]
- Hariton E, Locascio JJ. Randomised controlled trials - the gold standard for effectiveness research: study design: randomised controlled trials. BJOG 2018 Dec;125(13):1716 [FREE Full text] [CrossRef] [Medline]
- Collins L. Optimization of Behavioral, Biobehavioral, and Biomedical Interventions: The Multiphase Optimization Strategy (MOST). New York, USA: Springer; 2018.
- Lei H, Nahum-Shani I, Lynch K, Oslin D, Murphy S. A 'SMART' design for building individualized treatment sequences. Annu Rev Clin Psychol 2012;8:21-48 [FREE Full text] [CrossRef] [Medline]
- Leonard NR, Casarjian B, Fletcher RR, Praia C, Sherpa D, Kelemen A, et al. Theoretically-based emotion regulation strategies using a mobile app and wearable sensor among homeless adolescent mothers: acceptability and feasibility study. JMIR Pediatr Parent 2018;1(1):e1 [FREE Full text] [CrossRef] [Medline]
- Istepanian RS, Al-Anzi T. m-Health 2.0: new perspectives on mobile health, machine learning and big data analytics. Methods 2018 Dec 1;151:34-40. [CrossRef] [Medline]
|CBT: cognitive behavioral therapy|
|EMA: ecological momentary assessment|
|EMI: ecological momentary intervention|
|JITAI: just-in-time adaptive intervention|
|mHealth: mobile health|
|RCT: randomized controlled trial|
|VR: virtual reality|
|WHO: World Health Organization|
Edited by G Eysenbach; submitted 04.05.20; peer-reviewed by N Daniëls, H Alhodaib; comments to author 25.07.20; revised version received 25.08.20; accepted 14.09.20; published 12.11.20Copyright
©Marta Anna Marciniak, Lilly Shanahan, Judith Rohde, Ava Schulz, Carolin Wackerhagen, Dorota Kobylińska, Oliver Tuescher, Harald Binder, Henrik Walter, Raffael Kalisch, Birgit Kleim. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 12.11.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.