Mobile technologies have become popular approaches to promote behavior change and improve health outcomes, offering the ability to reach large populations in an easy, rapid, and low-cost manner [1,2]. Until recently, mobile behavior change interventions were limited to providing automated and predefined generic or minimally tailored messages, mainly based on estimates of baseline or usual behaviors and their determinants [3]. As people’s behaviors are driven by individual and contextual factors that vary across time [4,5], there is a need to make behavior change interventions that are more adaptive to the users’ evolving needs and context. Such an adaptive and dynamic intervention approach might help maintain participant engagement, sustain and support continued behavior change for longer durations, and thereby achieve greater health benefits [4-6].

Ecological momentary interventions (EMIs) are behavior change interventions that deliver support in real time, when most needed [7], for example, when the user is most likely to engage in unhealthy behaviors. To provide the information or treatment in real time and in real settings, EMIs are often based on repeated user reports collected via questionnaires, that is, ecological momentary assessments (EMAs) [8]. These EMA self-reports are usually real time or near real time and can focus on behaviors, contexts, emotional states, beliefs, attitudes, perceptions, exposures, events, or experiences in naturalistic settings (eg, “How are you feeling right now?”, “What are you doing right now?”, and “Are you near anyone smoking?”) [9]. EMAs originated in psychology a few decades ago, when these self-reports were primarily paper-based [8,9].

It has been suggested that tailoring EMIs based on EMAs may lead to higher user engagement and intervention effectiveness [7,10,11]. Given the ubiquity of smartphones [12,13], researchers are starting to explore the use of these mobile technologies to collect EMAs and deliver EMIs [14-17]. Previous systematic reviews of EMAs have focused on sedentary behavior, physical activity, and diet, mixing different EMA media for data collection, such as smartphones, PDAs (precursors of smartphones, now discontinued), and paper-and-pencil diaries [18-22]. The few existing systematic reviews on EMIs have focused on mental health and have also included studies with mixed media for EMIs, such as telephone, SMS text messaging, in-person counseling, computers, PDAs, and smartphones (a minority of included studies) [23-25]. To date, no studies have synthesized the current evidence on the use of smartphone-delivered EMIs using EMAs and their impact on health behaviors, user perspectives, or engagement.

The overall objective of this study is to systematically review the evidence and characteristics of smartphone-delivered EMIs to promote behavior change, using self-reported EMAs, specifically (1) their effects on health behaviors in any health domain, (2) user engagement, and (3) user perspectives. Although not the original aim of this systematic review, another objective arose upon data extraction and analysis—developing a reporting checklist (adapted from an existing checklist [22]) to facilitate interpretation and comparison of findings and enhance transparency and replicability of future studies using EMAs and EMIs.

A literature search was conducted in June 2019 (and updated in March 2020) using MEDLINE (via PubMed interface), Embase, PsycINFO, and CINAHL. Search strings included a combination of free terms and controlled vocabulary when supported (complete search strategy available in Multimedia Appendix 1). The reference lists of relevant articles were also screened to ensure that all eligible studies were included. The authors were contacted if there was a need for any additional information about the included studies.

The eligibility criteria were developed using the PICO (Participants, Intervention, Comparator, Outcomes; Multimedia Appendix 2). Participants included healthy individuals or patients with chronic conditions. We included all experimental studies that incorporated EMIs to improve health behaviors in any health domain. For the purposes of this review, an EMI must have been delivered in real time through smartphone apps and must have been based on data collected from users’ repeated reports in their natural context (ie, EMAs) and also via smartphone apps. Outcomes included any measures that illustrated the effects on health behavior changes (eg, changes in step counts and diet changes). Secondary outcomes included perspectives on EMIs and user engagement behaviors with different types of EMIs, including retention rate. No limiting criteria were used regarding comparison groups. Peer-reviewed studies published in English were included, and no restrictions were set regarding publication dates.

We excluded protocols, reviews, opinion pieces, and design and development papers without user evaluation of EMIs. Studies that used EMAs only for the purpose of data collection or outcome measurement were also excluded. Other exclusion criteria included interventions that relied solely on the automated data collected (eg, only through sensors and no user-reported EMAs) and interventions that were not based on data submitted by the participants (ie, EMAs) via smartphone apps or wearable devices.

A pilot screening of the studies was completed before the actual screening process began. The title and abstract screening and full-text screening were conducted by 2 independent investigators. A third researcher resolved disagreements. Cohen κ was applied to measure the intercoder agreement in each screening phase.

An investigator extracted the information from the included studies into a standardized form, and another researcher reviewed the form for consistency. The data collected from each study included the first author, year of publication, location, health domain, intervention aim, study design and duration, participants’ settings and characteristics, EMA data collection characteristics (eg, type of information collected from participants, prompting design and frequency—following the CREMAS [Checklist for Reporting EMA Studies] reporting checklist [22]), intervention components (eg, app, website, and therapy sessions), smartphone-based EMI characteristics (eg, frequency), health-related outcomes, user’s perspectives regarding EMIs and EMAs, and user engagement. Behavior change techniques (BCTs) were coded by 2 researchers using the BCT taxonomy [26]. Included randomized controlled trials (RCTs) were appraised by 2 researchers using the Cochrane risk of bias tool [27]. Nonrandomized studies were appraised by 2 researchers using the Risk Of Bias In Non-randomized Studies of Interventions tool [28]. A narrative synthesis was conducted for all included studies.

The search returned 2824 results (Figure 1). Of the 2824 studies, after removing duplicates, 2162 (76.56%) studies underwent title and abstract screening. Of the 2162 studies, there were 81 (3.75%) studies for full-text screening; of the 81 studies, 66 (81%) were excluded for not meeting the inclusion criteria (reasons for exclusion are presented in Multimedia Appendix 3). Cohen κ scores were 0.3 and 0.5 for abstract and full-text screening, respectively. We included 15 papers from the original search and 4 additional papers from other sources (reference lists of included studies and database search updates), corresponding to 19 articles, describing 17 unique studies (Table 1).

The 17 included studies (19 papers) involved a total of 652 participants [29-47] (Table 1). Most studies (13/17, 76%; 15/19, 79% papers) were conducted in the United States [29-32,34,37-45,47]. Publication years ranged from 2011 to 2020 (13/17, 76% studies were published from 2016 onward). Study duration ranged from 2-15 weeks, and the average duration was 4 weeks. Sample size varied from 7-121 participants (mean 35.2, SD 33.3; 67% women). The health domains covered were: mental health [29-36], smoking cessation [37-39], and substance abuse control [40-42], as well as diet, weight loss, and physical activity [43,44,46,47]. Studies in the mental health domain mostly recruited patients from outpatient clinics diagnosed with a mental health problem (major depressive disorder [29], schizophrenia [34,36], bipolar disorder [31], and other conditions [30,32,35]), and only 1 study focusing on mood and anxiety management recruited participants without a diagnosis [33]. Studies focusing on smoking recruited participants from smoking cessation clinics [37-39]. Studies on substance abuse control recruited individuals currently in treatment for an alcohol disorder from the community [40], college students with problematic drinking [41], and marijuana users from primary care clinics [42]. Finally, studies on diet, weight loss, and physical activity recruited obese individuals undergoing assessment for bariatric surgery [43], overweight or obese participants from the community [44,45], university students interested in well-being [46], and African American women after breast cancer treatment, recruited from the community [47].

Of the 17 studies, there were 4 (24%; 5/19, 26% papers) RCTs [39,42,44,45,47] and 12 (71%; 13/19, 68% papers) quasi-experimental studies (all with a single-arm design [29-35,37,38,40,41,43,46] except for one with 2 arms [36]). Of the 17 studies, participant retention was reported in 14 (82%; 16/19, 84% papers) studies, ranging from 62.1%-100% in the intervention arm [29-31,34,36-47], with 11 (65%; 13/19, 68% papers) studies having retention rates >75% [29-31,34,36-38,40,41,44-47]. The risk of bias of the 4 RCTs was assessed as unclear for most of the risk of bias tool categories (Multimedia Appendix 4). Overall risk of bias in nonrandomized studies was assessed as serious for most studies (Multimedia Appendix 5).

The commonly collected types of EMA data were affect-related (eg, emotions, feelings, and mood; 12/17, 71% studies; 14/19, 74% papers) [29-33,35-39,41,42,44,45], related to behaviors (eg, self-reported diet, physical activity, alcohol use, and medication adherence; 12/17, 71% studies; 14/19, 74% papers) [31-39,42-47], related to cognitions (eg, reasons for nonadherence and reasons to quit alcohol; 10/17, 59% studies; 12/19, 63% papers) [29,31,33-39,42-45], and related to social and environmental context (eg, distractions while eating and interaction with someone smoking; 9/17, 53% studies; 11/19, 58% papers; Table 2) [29,36-45]. In addition to user-reported EMAs, 18% (3/17) of studies also had sensor-collected data (eg, GPS and accelerometer) [40,48,49]. User-reported data collection was initiated either by the app (user would be prompted by the app to provide certain data) [29,31,32,34-39,41-45] or by the user (eg, as users saw fit; after a certain event, such as a meal) [29,30,33,37-41,44-46], sometimes with more than one modality in the same study [37-39,41,44,45,47]. The daily frequency of EMA prompts in app-initiated data collection was reported in 11 (65%; 13/19, 68% papers) studies [29,31,32,35-39,42-45,47], ranging from 2-8 times, with the most common being 4 to 5 times daily [29,32,37-39,42,43]. In 12% (2/17) of studies, the daily frequency was variable, depending on the number of times the participant needed to take medication daily [34] and depending on a trigger from a sensor [41]. The time window allowed for responding to EMA prompts was reported in 29% (5/17) of studies and varied between 1 and 130 minutes (Multimedia Appendix 6) [31,36-38,43,44].

EMIs consisted mostly of suggesting coping strategies (eg, use of cognitive-behavioral skills) [29,31-33,35-45,47], followed by motivational feedback (eg, positive reinforcement and supportive messages) [29,32,34,37-43,47] and informational feedback (eg, user-tailored graphs; Table 2) [30,31,44-47]. EMI characteristics were poorly reported and were not described in a standardized manner across studies, rarely detailing the decision mechanism (eg, algorithm). The EMI mechanism was not reported in 7 (41%, 8/19, 42% papers) studies [31-33,35-39], predetermined in 8 (47%) studies [30,34,40-43,46,47], and adaptive in 2 (12%; 3/19, 16% papers) studies [29,44,45]. The delivery format was in the form of text in most studies [29,31-34,36-45,47]; approximately 12% (2/17) of studies used tailored graphs [30,46], and 6% (1/17) of studies used texts and images [35]. Most interventions used other components in addition to the app, the most common one being counseling sessions with a therapist, either face-to-face or by telephone [29,31,34,35,37-39,41,42].

There were 35 BCTs identified across the studies (Multimedia Appendices 7 and 8). The most popular BCTs were social support (unspecified; 13/17, 76% studies; 15/19, 79% papers) [29,31,32,34,35,37-45,47], followed by prompt or cue (10/17, 59% studies; 11/19, 58% papers) [29,30,32,34-36,40,43-45,47], problem solving (9/17, 53% studies; 11/19, 58% papers) [29,31,33,36-41,44,45], feedback on behavior (6/17, 35% studies) [31,34,36,41,43,46], self-monitoring of behavior (7/17, 41% studies) [29,31,34,40,42,46,47], and social support (emotional; 6/17, 35% studies) [31-33,40-42]. The most commonly mentioned theories, frameworks, or models were cognitive behavioral therapy [31,33,41,42] and motivational interviewing [40-42].

Participants in 64% (11/17) studies (13/19, 68% papers) received material (eg, movie tickets) or monetary compensations for participating in the study [31,32,36-42,44-47]. Of those 11 studies, 6 (55%) studies (8/13, 62% papers) had incentives associated with EMA completion [31,37-39,42,44,45,47]. Adherence to EMA prompts (ie, to self-reporting data) was mentioned in 59% (10/17) studies (12/19, 63% papers) [31,34,36-39,41-45,47], most often in the form of response rate (Multimedia Appendix 6). The response rate varied from 30.7%-87% (9/17, 53% studies; 11/19, 58% papers; average 64.7%) [31,34,36-39,42-45,47]. Studies with a time limit to respond to EMA (4/17, 24% studies; 5/19, 26% papers) had lower response rates (30.7%, 58%, 62.9%) [31,43-45], except for 6% (1/17) of studies (2/19, 11% papers), with a response rate of 87% and with a high financial incentive for participants (those who completed 50%-74% of assessments received a US $40 gift card; 75-89% completion, US $80 gift card; and >90%, US $120 gift card) [37,38]. Of the 17 studies, 2 (12%) studies (3/19, 16% papers) reported the average time spent on each EMA prompt (ie, time spent self-reporting data), which varied from 2-6 minutes [31,37,38]. Adherence to EMI was reported in 24% (4/17) of studies (6/19, 32% papers) [36-38,44-46], with different measurements in each study (Multimedia Appendix 6).

Of the 17 studies, health-related outcomes were reported in 12 (71%) studies (14/19, 74% papers) [29,31,34,36-40,42-47]. Of the 17 studies, the 4 (24%) included RCTs reported nonstatistically significant improvements in substance abstinence, diet, weight loss, and sedentary time compared with the control group [39,42,44,47], and only 4 (24%) quasi-experimental studies reported statistically significant pre-post improvements in self-reported primary outcomes, namely depressive (P<.001) [29] and psychotic symptoms (P=.03) [36], drinking frequency (P<.001) [40], and eating patterns (P=.01) [46].

Regarding user perspectives (Multimedia Appendix 9), all apps were perceived as useful in supporting behavior change, although to varying degrees. In half of the studies, apps’ ease of use was assessed, with users rating the apps favorably [30-34,36,40-42]. The most helpful aspect of the apps, according to participants, was increasing awareness of their own behavior patterns [29,31-33,36-42,44-46]. The preferred and desirable features of the apps included personalization (eg, tailored prompts, tailored content, and feedback based on user responses) [29,31,32,35-40,44,45], communication with clinicians or coaches [29,30,47], multimedia and interactive capabilities, including voice recording [29,30,32,47], and an appealing design of the graphical user interface [30,32,33,36,40]. Common negative perspectives included EMA prompts being too frequent (more than five times daily), inopportune or tedious to complete [33,35,36,41-45], technical issues (eg, battery drainage and connectivity problems) [29-32,34,40-42], and repetitive content and feedback [31,32,42,44,45]. Of the 17 studies, 2 (12%) studies mentioned the potential negative impacts of momentary prompts on users’ mental well-being, including increased anxiety and stress because of prompts being too frequent or too sudden [41], or prompts giving users an unpleasant degree of self-awareness [35].

EMI and EMA components were rarely reported and were not described in a standardized manner across studies. We found that half of the studies failed to report EMA adherence rates, and this was even lower for EMIs. In addition, the mechanism details for EMAs and EMIs and incentives to complete EMAs and adhere to EMIs have been infrequently reported. On the basis of our findings and on an existing CREMAS [22], we developed a set of reporting items to include in the methods and results sections of EMA and EMI experiments (CREMAIs [Checklist for Reporting EMA- and EMI-specific aspects]; Table 3).

Although the potential for EMIs that build on EMA data for behavior change in the smartphone era seems promising, research on this approach is lacking. We identified 17 studies (only 4 RCTs), all with small sample sizes, short follow-up, and limited evaluation of efficacy. EMIs described were predominantly in mental health management, with a few addressing smoking cessation, substance abuse, diet, weight loss, and physical activity. The most common type of EMA data collected were related to subjective experiences, namely affective states and cognitions, indicating the usefulness of EMAs for this purpose. Behaviors were also often collected via EMAs, with sensors rarely being used. Adherence to collection of EMA data was a common barrier to implementation, with participants disliking the high frequency and tedious nature of EMA data collection. This suggests that EMAs could be gathered via other methods preferred by users (eg, voice). In addition, EMAs could be coupled with passively collected sensor data whenever possible to decrease user burden while still enabling the collection of subjective experiences relevant to user-desired personalization.

Description of interventions and reporting of evaluation measures were heterogeneous in each health domain, and there were few studies per health domain, limiting any conclusion being made on their efficacy on health behaviors, engagement, and outcomes.

To our knowledge, this is the first systematic review of smartphone-delivered EMIs based on self-reported EMAs to support behavioral changes. Existing reviews of EMIs in the treatment of psychotic disorders [24], major depressive disorder [50], alcohol use [51], and eating disorders [14] found that most interventions were in the early stages of development, which aligns with the findings of this review. Notably, the present findings show that most uses of EMIs based on EMAs to date seem to be in the field of mental health, where emotional and cognitive states can vary considerably throughout the day and influence behaviors. Previous systematic reviews on EMIs have all focused on mental health, used mostly older technologies, and did not tailor EMIs based on EMAs, having found mixed results (2 meta-analyses [23,25] showing small but positive effect sizes and another systematic review demonstrating acceptability and feasibility [24]).

Our review found that EMI and EMA components were rarely reported and were not described in a standardized manner across studies, hampering progress in this field. EMA- and EMI-specific aspects, such as the triggering mechanism and incentives, are important determinants of intervention uptake, retention, and efficacy. Hence, this poor reporting makes it difficult to synthesize and replicate existing evidence. Thus, we developed a set of reporting items—a checklist for reporting EMA- and EMI-specific aspects in behavior change experiments (CREMAIs)—based on an existing reporting checklist for EMA studies (CREMAS) [22]. Given that our adapted checklist focuses exclusively on EMA and EMI aspects, it should be used in conjunction with other reporting guidelines, depending on the type of experimental study design [52-55]. Our findings extend on previous systematic reviews in the field and add to the CREMAS checklist [22] by providing a detailed description of both EMI and EMA components (not just EMA) and specifically with respect to interventions that use smartphones.

EMI users had negative feedback regarding technical issues, inopportune and repetitive alerts, and prompts not being tailored enough, which may decrease participant engagement. The most common recommendations for intervention design were to make the intervention more personalized and engaging (eg, personalized coping strategies) and to tailor data collection and reduce reporting burden and invasiveness. These perspectives expand on existing literature by showing that for sustained efficacy of behavior change interventions, user engagement is paramount [4,6,56]. Personalization has been commonly suggested as a way to make interventions more engaging, effective, and better received by users [57-60]. One example includes just-in-time adaptive interventions, which are system-triggered interventions that aim to provide the right type/amount of support, at the right time, by adapting to an individual’s changing internal and contextual state (usually based on sensor-collected data) [61].

This review has several strengths. We developed and followed a protocol that was registered in the PROSPERO database at the start of the study. Intervention components were characterized in detail, including the coding of BCTs. However, the results of this review need to be interpreted in the context of certain limitations. Owing to the small number of RCTs, a meta-analysis was not conducted, and thus it was not possible to provide an estimation of preliminary efficacy. There was low to moderate agreement in screening, which reflects the difficulty in establishing whether a study met the inclusion criteria. Screening was complicated by incomplete intervention descriptions, particularly with regard to EMI and EMA reporting. Finally, the definitions of EMI and EMA are not consensual in the literature. Thus, the studies included in this review reflect the predefined definitions we adopted.

The use of smartphone-delivered EMIs based on EMAs in behavior change interventions is a novel area of research, where more RCTs are needed to determine efficacy. Given the ubiquity of smartphones, these interventions have the potential to support behavioral changes at scale. Nevertheless, it is still uncertain which populations may find the use of EMIs based on EMAs most acceptable and which populations and settings may benefit the most. So far, studies have focused on mental health, smoking, substance abuse, diet, weight loss, and physical activity, with mixed results. Appropriately powered clinical trials are needed to examine the use of EMIs tailored by EMAs in a range of populations and settings and to examine the impact on health outcomes and the longevity of these benefits.

Future studies should explore the combination of EMAs and sensor data to deliver more personalized and minimally burdensome EMIs. EMA involves manual data collection at several points in time, which can be burdensome for users, but remains important to gather individual data that sensors are currently unable to capture, such as subjectively perceived cognitive and affective states [62]. Capturing subjective experiences (eg, cravings, pain, and loneliness) enables a richer and deeper insight into a person’s behavior and can foster the tailoring of an intervention to a person’s needs, which in turn may increase the perceived relevance of EMIs. By combining self-reported EMAs of subjective experiences with additional objective data passively collected via sensors (eg, physical activity patterns and heart rate) [63,64], there is potential to promote a more engaging personalized intervention, as minimally burdensome as possible. Novel machine learning algorithms can further explore these different types of data to increase the precision of personalized interventions [65].

A more seamless EMA and EMI experience is crucial for engagement. User burden associated with data entry is the most reported reason why people stop using mobile health apps [66]. In addition to using sensors whenever possible, another possibility to reduce user burden is to optimize the design of data collection modes. For instance, faster methods, such as speech-based data entry, may be used instead of requiring users to type in response [67]. Another option would be the use of a chatbot to enable data collection in a conversational and more engaging way. Other feasible options include data entry templates, such as dropdown menus, and the use of personalization to autopopulate some data fields [68] based on previous entries or other data sources [33]. Co-designing interventions with users may offer insights into the best options for data collection in each particular case, regarding the types and amount of data, and the mode, frequency, and timing of data collection [69].

Future research in this area should adhere to existing reporting standards, namely, what concerns the detailing of EMA- and EMI-specific characteristics. Reporting guidelines are essential in facilitating the evaluation of study validity and allowing for comparisons across interventions. Consistency and detail in reporting intervention characteristics enable replication efforts and allow for meta-analyses and meta-regression to explore the features associated with the highest user engagement and intervention efficacy. Advancements in the field of EMAs and EMIs and the higher scientific impact of published studies in this area are dependent on the consistent use of reporting guidelines.

This is the first systematic review of smartphone-delivered EMIs based on self-reported EMAs promoting health behaviors. The use of this approach in behavior change is an emerging area of research, with few studies evaluating efficacy and most interventions focusing on mental health management. EMAs were commonly used to capture subjective experiences, as well as behaviors, whereas sensors were rarely used. Future research should explore combining self-reported EMAs of subjective experiences with objective data passively collected via sensors to promote personalization. Studies should also explore the effects of different EMA data collection methods (eg, chatbots) on user burden, engagement, and efficacy. A reporting checklist was developed with the goal of facilitating interpretation and comparison of findings and enhancing transparency and replicability in future studies using EMAs and EMIs.