Smartphone-Delivered Ecological Momentary Interventions Based on Ecological Momentary Assessments to Promote Health Behaviors: Systematic Review and Adapted Checklist for Reporting Ecological Momentary Assessment and Intervention Studies

Background: Healthy behaviors are crucial for maintaining a person’s health and well-being. The effects of health behavior interventions are mediated by individual and contextual factors that vary over time. Recently emerging smartphone-based ecological momentary interventions (EMIs) can use real-time user reports (ecological momentary assessments [EMAs]) to trigger appropriate support when needed in daily life. Objective: This systematic review aims to assess the characteristics of smartphone-delivered EMIs using self-reported EMAs in relation to their effects on health behaviors, user engagement, and user perspectives. Methods: We searched MEDLINE, Embase, PsycINFO, and CINAHL in June 2019 and updated the search in March 2020. We included experimental studies that incorporated EMIs based on EMAs delivered through smartphone apps to promote health behaviors in any health domain. Studies were independently screened. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were followed. We performed a narrative synthesis of intervention effects, user perspectives and engagement, and intervention design and characteristics. Quality appraisal was conducted for all included studies. Results: We included 19 papers describing 17 unique studies and comprising 652 participants. Most studies were quasi-experimental (13/17, 76%), had small sample sizes, and great heterogeneity in intervention designs and measurements. EMIs were most popular in the mental health domain (8/17, 47%), followed by substance abuse (3/17, 18%), diet, weight loss, physical activity (4/17, 24%), and smoking (2/17, 12%). Of the 17 studies, the 4 (24%) included randomized controlled trials reported nonstatistically significant effects on health behaviors, and 4 (24%) quasi-experimental studies reported statistically significant pre-post improvements in self-reported primary outcomes, namely depressive (P<.001) and psychotic symptoms (P=.03), drinking frequency (P<.001), and eating patterns (P=.01). EMA was commonly used to capture subjective experiences as well as behaviors, whereas sensors were rarely used. Generally, users perceived EMIs to be helpful. Common suggestions for improvement included enhancing personalization, multimedia and interactive capabilities (eg, voice recording), and lowering the JMIR Mhealth Uhealth 2021 | vol. 9 | iss. 11 | e22890 | p. 1 https://mhealth.jmir.org/2021/11/e22890/ (page number not for citation purposes) Dao et al JMIR MHEALTH AND UHEALTH


Background
Mobile technologies have become popular approaches to promote behavior change and improve health outcomes, offering the ability to reach large populations in an easy, rapid, and low-cost manner [1,2].Until recently, mobile behavior change interventions were limited to providing automated and predefined generic or minimally tailored messages, mainly based on estimates of baseline or usual behaviors and their determinants [3].As people's behaviors are driven by individual and contextual factors that vary across time [4,5], there is a need to make behavior change interventions that are more adaptive to the users' evolving needs and context.Such an adaptive and dynamic intervention approach might help maintain participant engagement, sustain and support continued behavior change for longer durations, and thereby achieve greater health benefits [4][5][6].
Ecological momentary interventions (EMIs) are behavior change interventions that deliver support in real time, when most needed [7], for example, when the user is most likely to engage in unhealthy behaviors.To provide the information or treatment in real time and in real settings, EMIs are often based on repeated user reports collected via questionnaires, that is, ecological momentary assessments (EMAs) [8].These EMA self-reports are usually real time or near real time and can focus on behaviors, contexts, emotional states, beliefs, attitudes, perceptions, exposures, events, or experiences in naturalistic settings (eg, "How are you feeling right now?", "What are you doing right now?", and "Are you near anyone smoking?") [9].EMAs originated in psychology a few decades ago, when these self-reports were primarily paper-based [8,9].
It has been suggested that tailoring EMIs based on EMAs may lead to higher user engagement and intervention effectiveness [7,10,11].Given the ubiquity of smartphones [12,13], researchers are starting to explore the use of these mobile technologies to collect EMAs and deliver EMIs [14][15][16][17].Previous systematic reviews of EMAs have focused on sedentary behavior, physical activity, and diet, mixing different EMA media for data collection, such as smartphones, PDAs (precursors of smartphones, now discontinued), and paper-and-pencil diaries [18][19][20][21][22].The few existing systematic reviews on EMIs have focused on mental health and have also included studies with mixed media for EMIs, such as telephone, SMS text messaging, in-person counseling, computers, PDAs, and smartphones (a minority of included studies) [23][24][25].To date, no studies have synthesized the current evidence on the use of smartphone-delivered EMIs using EMAs and their impact on health behaviors, user perspectives, or engagement.

Objective
The overall objective of this study is to systematically review the evidence and characteristics of smartphone-delivered EMIs to promote behavior change, using self-reported EMAs, specifically (1) their effects on health behaviors in any health domain, (2) user engagement, and (3) user perspectives.
Although not the original aim of this systematic review, another objective arose upon data extraction and analysis-developing a reporting checklist (adapted from an existing checklist [22]) to facilitate interpretation and comparison of findings and enhance transparency and replicability of future studies using EMAs and EMIs.

Methods
The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines were used when conducting and reporting this systematic review.The protocol was registered in PROSPERO (International Prospective Register of Systematic Reviews; CRD42019138739).

Search Strategy for Identification of Studies
A literature search was conducted in June 2019 (and updated in March 2020) using MEDLINE (via PubMed interface), Embase, PsycINFO, and CINAHL.Search strings included a combination of free terms and controlled vocabulary when supported (complete search strategy available in Multimedia Appendix 1).The reference lists of relevant articles were also screened to ensure that all eligible studies were included.The authors were contacted if there was a need for any additional information about the included studies.

Study Selection Criteria
The eligibility criteria were developed using the PICO (Participants, Intervention, Comparator, Outcomes; Multimedia Appendix 2).Participants included healthy individuals or patients with chronic conditions.We included all experimental studies that incorporated EMIs to improve health behaviors in any health domain.For the purposes of this review, an EMI must have been delivered in real time through smartphone apps and must have been based on data collected from users' repeated reports in their natural context (ie, EMAs) and also via smartphone apps.Outcomes included any measures that illustrated the effects on health behavior changes (eg, changes in step counts and diet changes).Secondary outcomes included perspectives on EMIs and user engagement behaviors with different types of EMIs, including retention rate.No limiting criteria were used regarding comparison groups.Peer-reviewed studies published in English were included, and no restrictions were set regarding publication dates.
We excluded protocols, reviews, opinion pieces, and design and development papers without user evaluation of EMIs.Studies that used EMAs only for the purpose of data collection or outcome measurement were also excluded.Other exclusion criteria included interventions that relied solely on the automated data collected (eg, only through sensors and no user-reported EMAs) and interventions that were not based on data submitted by the participants (ie, EMAs) via smartphone apps or wearable devices.

Screening, Data Extraction, and Synthesis
A pilot screening of the studies was completed before the actual screening process began.The title and abstract screening and full-text screening were conducted by 2 independent investigators.A third researcher resolved disagreements.Cohen κ was applied to measure the intercoder agreement in each screening phase.
An investigator extracted the information from the included studies into a standardized form, and another researcher reviewed the form for consistency.The data collected from each study included the first author, year of publication, location, health domain, intervention aim, study design and duration, participants' settings and characteristics, EMA data collection characteristics (eg, type of information collected from participants, prompting design and frequency-following the CREMAS [Checklist for Reporting EMA Studies] reporting checklist [22]), intervention components (eg, app, website, and therapy sessions), smartphone-based EMI characteristics (eg, frequency), health-related outcomes, user's perspectives regarding EMIs and EMAs, and user engagement.Behavior change techniques (BCTs) were coded by 2 researchers using the BCT taxonomy [26].Included randomized controlled trials (RCTs) were appraised by 2 researchers using the Cochrane risk of bias tool [27].Nonrandomized studies were appraised by 2 researchers using the Risk Of Bias In Non-randomized Studies of Interventions tool [28].A narrative synthesis was conducted for all included studies.

Description of Included Studies
The search returned 2824 results (Figure 1).Of the 2824 studies, after removing duplicates, 2162 (76.56%) studies underwent title and abstract screening.Of the 2162 studies, there were 81 (3.75%) studies for full-text screening; of the 81 studies, 66 (81%) were excluded for not meeting the inclusion criteria (reasons for exclusion are presented in Multimedia Appendix 3).Cohen κ scores were 0.3 and 0.5 for abstract and full-text screening, respectively.We included 15 papers from the original search and 4 additional papers from other sources (reference lists of included studies and database search updates), corresponding to 19 articles, describing 17 unique studies (Table 1).

Checklist for Reporting EMA-and EMI-Specific Aspects in Behavior Change Experiments
EMI and EMA components were rarely reported and were not described in a standardized manner across studies.We found that half of the studies failed to report EMA adherence rates, and this was even lower for EMIs.In addition, the mechanism details for EMAs and EMIs and incentives to complete EMAs and adhere to EMIs have been infrequently reported.On the basis of our findings and on an existing CREMAS [22], we developed a set of reporting items to include in the methods and results sections of EMA and EMI experiments (CREMAIs [Checklist for Reporting EMA-and EMI-specific aspects]; Table 3).c CREMAIs: checklist for reporting EMA and EMI-specific aspects.
d Adapted from Liao et al [22].

Principal Findings
Although the potential for EMIs that build on EMA data for behavior change in the smartphone era seems promising, research on this approach is lacking.We identified 17 studies (only 4 RCTs), all with small sample sizes, short follow-up, and limited evaluation of efficacy.EMIs described were predominantly in mental health management, with a few addressing smoking cessation, substance abuse, diet, weight loss, and physical activity.The most common type of EMA data collected were related to subjective experiences, namely affective states and cognitions, indicating the usefulness of EMAs for this purpose.Behaviors were also often collected via EMAs, with sensors rarely being used.Adherence to collection of EMA data was a common barrier to implementation, with participants disliking the high frequency and tedious nature of EMA data collection.This suggests that EMAs could be gathered via other methods preferred by users (eg, voice).In addition, EMAs could be coupled with passively collected sensor XSL • FO RenderX data whenever possible to decrease user burden while still enabling the collection of subjective experiences relevant to user-desired personalization.
Description of interventions and reporting of evaluation measures were heterogeneous in each health domain, and there were few studies per health domain, limiting any conclusion being made on their efficacy on health behaviors, engagement, and outcomes.

Comparison With Existing Literature
To our knowledge, this is the first systematic review of smartphone-delivered EMIs based on self-reported EMAs to support behavioral changes.Existing reviews of EMIs in the treatment of psychotic disorders [24], major depressive disorder [50], alcohol use [51], and eating disorders [14] found that most interventions were in the early stages of development, which aligns with the findings of this review.Notably, the present findings show that most uses of EMIs based on EMAs to date seem to be in the field of mental health, where emotional and cognitive states can vary considerably throughout the day and influence behaviors.Previous systematic reviews on EMIs have all focused on mental health, used mostly older technologies, and did not tailor EMIs based on EMAs, having found mixed results (2 meta-analyses [23,25] showing small but positive effect sizes and another systematic review demonstrating acceptability and feasibility [24]).
Our review found that EMI and EMA components were rarely reported and were not described in a standardized manner across studies, hampering progress in this field.EMA-and EMI-specific aspects, such as the triggering mechanism and incentives, are important determinants of intervention uptake, retention, and efficacy.Hence, this poor reporting makes it difficult to synthesize and replicate existing evidence.Thus, we developed a set of reporting items-a checklist for reporting EMA-and EMI-specific aspects in behavior change experiments (CREMAIs)-based on an existing reporting checklist for EMA studies (CREMAS) [22].Given that our adapted checklist focuses exclusively on EMA and EMI aspects, it should be used in conjunction with other reporting guidelines, depending on the type of experimental study design [52][53][54][55].Our findings extend on previous systematic reviews in the field and add to the CREMAS checklist [22] by providing a detailed description of both EMI and EMA components (not just EMA) and specifically with respect to interventions that use smartphones.EMI users had negative feedback regarding technical issues, inopportune and repetitive alerts, and prompts not being tailored enough, which may decrease participant engagement.The most common recommendations for intervention design were to make the intervention more personalized and engaging (eg, personalized coping strategies) and to tailor data collection and reduce reporting burden and invasiveness.These perspectives expand on existing literature by showing that for sustained efficacy of behavior change interventions, user engagement is paramount [4,6,56].Personalization has been commonly suggested as a way to make interventions more engaging, effective, and better received by users [57][58][59][60].One example includes just-in-time adaptive interventions, which are system-triggered interventions that aim to provide the right type/amount of support, at the right time, by adapting to an individual's changing internal and contextual state (usually based on sensor-collected data) [61].

Strengths and Limitations
This review has several strengths.We developed and followed a protocol that was registered in the PROSPERO database at the start of the study.Intervention components were characterized in detail, including the coding of BCTs.However, the results of this review need to be interpreted in the context of certain limitations.Owing to the small number of RCTs, a meta-analysis was not conducted, and thus it was not possible to provide an estimation of preliminary efficacy.There was low to moderate agreement in screening, which reflects the difficulty in establishing whether a study met the inclusion criteria.Screening was complicated by incomplete intervention descriptions, particularly with regard to EMI and EMA reporting.Finally, the definitions of EMI and EMA are not consensual in the literature.Thus, the studies included in this review reflect the predefined definitions we adopted.

Implications for Future Studies
The use of smartphone-delivered EMIs based on EMAs in behavior change interventions is a novel area of research, where more RCTs are needed to determine efficacy.Given the ubiquity of smartphones, these interventions have the potential to support behavioral changes at scale.Nevertheless, it is still uncertain which populations may find the use of EMIs based on EMAs most acceptable and which populations and settings may benefit the most.So far, studies have focused on mental health, smoking, substance abuse, diet, weight loss, and physical activity, with mixed results.Appropriately powered clinical trials are needed to examine the use of EMIs tailored by EMAs in a range of populations and settings and to examine the impact on health outcomes and the longevity of these benefits.
Future studies should explore the combination of EMAs and sensor data to deliver more personalized and minimally burdensome EMIs.EMA involves manual data collection at several points in time, which can be burdensome for users, but remains important to gather individual data that sensors are currently unable to capture, such as subjectively perceived cognitive and affective states [62].Capturing subjective experiences (eg, cravings, pain, and loneliness) enables a richer and deeper insight into a person's behavior and can foster the tailoring of an intervention to a person's needs, which in turn may increase the perceived relevance of EMIs.By combining self-reported EMAs of subjective experiences with additional objective data passively collected via sensors (eg, physical activity patterns and heart rate) [63,64], there is potential to promote a more engaging personalized intervention, as minimally burdensome as possible.Novel machine learning algorithms can further explore these different types of data to increase the precision of personalized interventions [65].
A more seamless EMA and EMI experience is crucial for engagement.User burden associated with data entry is the most reported reason why people stop using mobile health apps [66].In addition to using sensors whenever possible, another possibility to reduce user burden is to optimize the design of data collection modes.For instance, faster methods, such as speech-based data entry, may be used instead of requiring users to type in response [67].Another option would be the use of a chatbot to enable data collection in a conversational and more engaging way.Other feasible options include data entry templates, such as dropdown menus, and the use of personalization to autopopulate some data fields [68] based on previous entries or other data sources [33].Co-designing interventions with users may offer insights into the best options for data collection in each particular case, regarding the types and amount of data, and the mode, frequency, and timing of data collection [69].
Future research in this area should adhere to existing reporting standards, namely, what concerns the detailing of EMA-and EMI-specific characteristics.Reporting guidelines are essential in facilitating the evaluation of study validity and allowing for comparisons across interventions.Consistency and detail in reporting intervention characteristics enable replication efforts and allow for meta-analyses and meta-regression to explore the features associated with the highest user engagement and intervention efficacy.Advancements in the field of EMAs and EMIs and the higher scientific impact of published studies in this area are dependent on the consistent use of reporting guidelines.

Conclusions
This is the first systematic review of smartphone-delivered EMIs based on self-reported EMAs promoting health behaviors.The use of this approach in behavior change is an emerging area of research, with few studies evaluating efficacy and most interventions focusing on mental health management.EMAs were commonly used to capture subjective experiences, as well as behaviors, whereas sensors were rarely used.Future research should explore combining self-reported EMAs of subjective experiences with objective data passively collected via sensors to promote personalization.Studies should also explore the effects of different EMA data collection methods (eg, chatbots) on user burden, engagement, and efficacy.A reporting checklist was developed with the goal of facilitating interpretation and comparison of findings and enhancing transparency and replicability in future studies using EMAs and EMIs.

Table 1 .
Characteristics of included studies.
a I: intervention.b C: control c N/A: not applicable.d NR: not reported.e Not available.f NS: not supported.g EMA: ecological momentary assessment.

Table 2 .
Characteristics of EMA a data collection and EMI b in included studies c .EMA and EMI characteristics reported according to items specified in Table3based on information reported in the included studies.

Table 3 .
Adapted checklist for reporting smartphone-delivered EMA a -and EMI b -specific aspects in behavior change experiments (CREMAIs c ) d .
b EMI: ecological momentary intervention.