The search included studies that used mobile apps to measure momentary mood or stress in adults; the studies were published in peer-reviewed journals in English between January 2008 and November 2017. We performed database searches on six online biomedical databases—PubMed, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Embase, the Cochrane Library, PsycINFO, and Web of Science. We also performed hand-searches of the Journal of Medical Internet Research (JMIR) and the website of the Society for Ambulatory Assessment. We used the following search terms: (“ecological momentary assessment” [MeSH] OR “experience sampling” OR “ecological momentary” OR “event sampling” OR “ambulatory assessment” OR “structured diary method” OR “real-time data capture studies” OR “real-time data capture study” OR “beeper studies” OR “beeper study” OR “intensive longitudinal assessment”) AND (“stress, psychological” [MeSH] OR “affect” [MeSH] OR “mood” OR “emotion” OR “affection” OR “stress”) AND (“mobile applications” [MeSH] OR “smartphone” [MeSH] OR “cell phones” [MeSH] OR “smartphone*” OR “cell phone” OR “cellular phone” OR “mobile app*”). The articles identified were inspected, including their reference lists and in-text citations of relevant articles (see Multimedia Appendix 1).

Studies were included that used basic- and advanced-feature mobile phones to measure momentary mood or stress in adult patients. We included those studies that were published in peer-reviewed journals in English. Specifically, included studies used basic- or advanced-feature mobile phones to deliver EMAs. Included studies also involved adult patients in community settings who were diagnosed with a certain disease by their physicians and cared for in outpatient settings. We also included studies that involved people who had mood or stress problems without diagnosis by their physicians of psychiatric disorders, such as affective, anxiety, and mood disorders or of substance addictions. The year 2008 was chosen as the earliest year of publication because the first app downloaded on a mobile device was in 2008 [19]. Studies were excluded if they were studies of smoking, diet, addictions, major psychological problems, or child populations.

A total of 764 articles were retrieved from the six databases, in which 257 records were duplicated. For 507 articles, two reviewers (YSY and GWR) independently screened titles and abstracts. After that, the same two reviewers independently reviewed full-text articles to decide whether each article was relevant to the review. In case of disagreement, a third person (MC) was consulted to reach consensus. Ultimately, 12 full-text articles were selected according to the criteria and relevant data were extracted. Figure 1 shows the process of study selection based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [20].

The following information was extracted: study purpose, sample characteristics, main momentary measurement, data analysis method, and methodological details of EMA, such as operating system, mode of response, contingency, duration of data collection, frequency, and alarm interval for each study.

Various clinical populations were included in this review: patients with chronic fatigue syndrome [21,22], HIV [5,23-25], tinnitus [6], migraine [26], minor traumatic brain injury [27], breast cancer [7], end-stage kidney disease [28], and temporomandibular disorder [29]. Out of 12 studies, 11 (92%) [5,7,21-29] measured mood or stress as major variables by the EMA method; 1 study (8%) assessed stress or stressors [6]; and 5 studies (42%) [5,6,24-26] measured mood or affect along with stress or stressors. A total of 2 studies out of 12 (17%) measured pain along with mood or stress [26,29]. The majority of articles (11/12, 92%) captured changes in mood or stress over time without applying interventions [5-7,21-28]; 1 article out of 12 (8%) reported changes of mood from a pre-post approach [29] (see Table 1).

Out of 12 studies, 2 studies of patients with chronic fatigue syndrome (CFS) (17%) assessed patients’ affect [21,22]. Out of these 2 studies, 1 (50%) focused on activity management strategies, affect, and symptoms to investigate whether activity patterns occurred according to patients’ symptom experience and affect [22]. The other study (1/2, 50%) examined the relationship between significant others’ responses and patient outcomes such as affect, symptom severity, disability, and activity management strategies [21].

Out of 12 studies, 4 EMA studies of HIV patients (33%) measured several variables. Out of these 4 studies, 1 (25%) evaluated momentary mood and cognitive symptoms of HIV patients [23], and another (1/4, 25%) assessed control beliefs, mood, stress, coping, and social support to examine whether momentary motivation is a mechanism by which everyday experiences affect adherence to medication therapy [24]. Cook et al’s study (1/4, 25%) measured thoughts, mood, stress, coping, social support, and treatment motivation to test predictors of electronically monitored adherence at both state and trait levels [25]. Out 4 studies, 1 EMA study (1/4, 25%) also investigated stress, frequency of stressors, stressful life events, and behaviors of HIV patients [5].

Out of 12 studies, 1 (8%) assessed experience of migraine attacks and prodromal features, such as fatigue, cognitive functioning, affect, effort spent (eg, working hard and feeling strained), and stressors, to test and identify individual prodromal features related to the interictal state in moderate-to-severe migraine patients [26].

In a study of minor traumatic brain injury patients (1/12, 8%), mood and affect were assessed to evaluate feasibility and validity of a mobile health system app [27]. A study of patients with breast cancer (1/12, 8%) measured sleep satisfaction, mood, and anxiety to evaluate the potential of a mobile, mental health tracker app using daily mental health ratings as indicators of depression [7].

Out of 12 studies, 1 (8%) evaluated day-to-day and diurnal variability of fatigue, sleepiness, exhaustion, and related symptoms in end-stage kidney disease patients [28]. Out of 12 studies, a pre-post EMA design in 1 study (8%) was applied to measure pain, coping, and affect in order to evaluate the effect of cognitive-behavioral treatment for temporomandibular disorder patients in the context of painful episodes [29].

In terms of main momentary measurement, half of the included studies (6/12, 50%) measured momentary mood, affect, or stress with standardized scales for validation [7,23-25,27,29], while others (6/12, 50%) did not administer or specify them [5,6,21,22,26,28]. Band et al’s study (1/12, 8%) captured mood changes by using two subscales of positive affect and negative affect of CFS patients [22]. In this study, positive affect was assessed using five items: excited, happy, satisfied, relaxed, and cheerful (Cronbach alpha=.87); negative affect was assessed using five items: sad, annoyed, irritated, anxious, lonely, and guilty (Cronbach alpha=.87). In the other study of CFS patients (1/12, 8%), affect was measured by a single item, feeling distressed, which was included with standard items examining patients’ affect at a momentary level [21].

The standardized measures of the Beck Depression Inventory-II and the Profile of Mood States were administered to measure state mood and stress in comparison to the momentary item for assessing mood of older adults with HIV; correlates with state mood (ie, sadness, happiness, and tiredness) and stress were evaluated by item questions developed in the study [23].

In a study of HIV patients (1/12, 8%) [24], three items for mood (Cronbach alpha=.93) and six items for stress (Cronbach alpha=.67) from the Diary of Ambulatory Behavioral States were used after piloting [30]. Another study of HIV patients (1/12, 8%) used the mood scale from the Diary of Ambulatory Behavioral States and the stress scale from the Daily Hassles Scale; they were validated by the trait measurement tools from the Center for Epidemiological Studies-Depression scale and the HIV/AIDS-Targeted Quality of Life instrument [25]. Both trait-level mood and stress predicted their respective state-level measures.

In 1 study out of 12 (8%), the Daily Mood and Affect scale for momentary assessment was developed; the Positive and Negative Affect Schedule and the 9-item Patient Health Questionnaire as standardized measures were applied [27]. In a study of breast cancer patients (1/12, 8%), the author used 3-item short scales for anxiety, mood, and sleep satisfaction, rated by facial emoticon scales, and evaluated the concurrent validity with the standardized mood scale of the 9-item Patient Health Questionnaire [7].

In 1 study out of 12 (8%), evaluating the effects of cognitive-behavioral therapy of patients with temporomandibular pain, a standardized tool—the Center for Epidemiological Studies-Depression scale—was used to compare pre- to posttreatment change of affect using a mood item borrowed from the Coping Strategies Questionnaire [29].

Out of 12 studies, 4 (33%) reported on feasibility or validity of an EMA app [6,23,26,27]. A study with EMA design for patients suffering from tinnitus (1/4, 25%) indicated that they would suggest an EMA method to a friend [6]. Participants expressed their experience with the EMA method positively [23,27]; they reported that they accepted it as usable and were satisfied with the EMA method [26,27].

An evaluation of the usefulness or perceptions by participants of the EMA methods was conducted in another study (1/12, 8%); the results indicated that the EMA using mobile phones was useful and reliable for self-monitoring of functioning ability in daily routines [5]. EMA showed promising results in the field of screening depressive moods in a clinical population by evaluating accuracy of depression screening via the EMA method (1/12, 8%) [7].

Table 2 shows information on methodological details of EMA used in the studies, such as the operating system of mobile phones, mode of response, contingency, duration of data collection, frequency per day, and alarm interval. Different operating systems were used to install the mobile apps, but more than half of the studies (7/12, 58%) used Android operating systems [5,7,21,22,24,25,27].

Out of the 12 studies, 7 (58%) used specific apps directly installed onto mobile phones [5,7,21-23,26,27]; 3 studies (25%) used a Web-based online survey program hyperlinked from the mobile phones [6,24,25]; and the remaining 2 studies (17%) applied an EMA method using an interactive voice-response system [28,29]. A daily repeated voice-recorded EMA design could be a good system for patients with motor dysfunction, instead of a mobile phone app or online survey in which patients have to operate the phones to respond.

Out of 12 studies, 11 (92%) [5,6,21-29] included in the review used signal contingency to prompt momentary measurement; there was 1 study (8%) where the contingency method was not specified [7]. In a study of patients with HIV (1/12, 8%), both signal-based contingency and event-based self-report were applied [5]. Frequency of the contingency varied from once per day [7] to 10 times per day [21,22], and the study durations ranged from a minimum of 6 days [21,22] to a maximum of 48 weeks, which equals 336 days [7].

The studies with the shortest period (2/12, 17%) had the highest frequency per day of assessment [21,22], and studies with lengthy periods of more than 6 weeks (4/12, 33%) had the lowest frequency [7,24,25,27]. Out of 12 studies, 1 (8%) tried various frequencies of momentary assessment by constructs: once a day for measuring medication adherence, four times a day for emotional and physical states, and seven times per day for stressors [5].

The interval of the reminder signal varied according to the study design from random, stratified semirandom, and semirandom to fixed time per participant. A total of 4 studies out of 12 (33%) had set the alarm time as fixed according to the preference or convenience of each participant to improve compliance to the EMA [5,23,27,28].

Study completion rates ranged from 64.6% [22] to 89.5% [26], excluding studies with no reported completion rates (see Table 3). A study of temporomandibular disorder patients (1/12, 8%) paid participants US $5 for every day that they completed at least 50% of scheduled daily assessments [29], while 2 studies of HIV patients (17%) provided incentives of US $25 and the mobile phone used in the study when they finished the EMA measurement [24]. In another study of patients with HIV (1/12, 8%), in which both signal-based and event-based EMA methods were applied, event-based self-reports were encouraged by applying incentives up to US $70 to reach the survey goal of seven times per day [5]. However, the study did not calculate completion rate, since the measure was reported in an event-based way [5]. Other studies included in this review (6/12, 50%) did not mention incentives [21-23,26-28]. No articles evaluated related factors affecting the completion rate.

Because EMA datasets include diverse sources of variance, various analysis methods have been employed to address the complexity and hierarchy of the data. Out of the 12 studies, 7 (58%) reviewed here undertook multilevel or mixed-modeling analysis [21,22,24-26,28,29]. A total of 2 studies out of 12 (17%) used the MTMIXED command in Stata (StataCorp LLC) for continuous outcome variables in multilevel modeling [21,22]. Linear mixed-model multilevel analysis with maximum likelihood estimation was employed [24-26] (see Table 3).

Of the 12 studies, 2 (17%) used descriptive analysis and correlation analysis [23,27], and 2 others (17%) applied the receiver operating characteristic and ordinary least squares according to the characteristics of the variables analyzed [6,7]. Kim et al’s study [7] (1/12, 8%) estimated random-effects logistic regression parameters and thereafter used receiver operating characteristic plots to evaluate the screening accuracy of the model.

Of the 12 studies, 1 study of HIV patients (8%) applied EMA using both quantitative and qualitative measurement with various frequencies according to the target variables. The data analysis method for quantitative data was not specified, while a grounded thematic coding method in Dedoose (SocioCultural Research Consultants LLC), a Web-based mixed-method data analysis program, was applied for qualitative data of the user experience of the usefulness or perceptions regarding the EMA app [5].

While there is no standard for appropriate response rate to assess validity, 1 study out of 12 (8%) clarified that they used all available daily observations [24], and another (1/12, 8%) excluded participants who completed fewer than 20 assessments out of the total of 60 for preliminary analysis but retained all participants in the final analysis [22]; other studies did not specify inclusion criteria for response rate or number of observations for statistical analysis. A study of temporomandibular pain patients (1/12, 8%) used the observations selectively, in accordance with the study purpose, in which pain was nonzero and coping was recorded at the same time [29].

Of the 12 studies, 7 (58%) had a briefing or intake session to ensure that participants understood the EMA app before starting the survey. Participants could practice and ask questions regarding the app during the session. Informed consent and non-EMA measures, such as baseline or laboratory measurement, were also obtained during the session. After finishing the EMA phase, patients were debriefed to evaluate their experiences during the study.

This review identified mobile phone-based systems for monitoring mood or stress of patients seeking health care in outpatient departments. Studies focused on EMA methods using mobile phones, which are feasible for measuring stress and mood in adult patients and elucidating relevant methodological details. The EMA methods used in the included studies were evaluated as feasible for recognizing changes with significant variation in assessment variables [27-29] and for measuring mood and stress of patients [6,7,23,27]. This review presented strategic information on EMA methods, such as mode of response, ways of sending alarm contingencies, time intervals, frequencies, and study durations, along with information about the participants in the survey and the momentary measurements.

The studies in our review used three different modes of EMA response on mobile phones: via mobile app [5,7,21-23,26,27]; via hyperlink to online survey [6,24,25]; and via interactive voice response system [28,29]. Mode of response can be selected in accordance with participants’ clinical conditions.

EMA methods are time-consuming and demanding [4]. Not all patients are willing to participate or comply strictly with the protocol. The studies included in this review showed completion rates that ranged from 64.6% to 89.5%, which was contingent on the nature of the participants. Although there is no agreed-upon gold standard for an acceptable compliance rate in EMA studies, Stone and Shiffman [31] noted that EMA data would not be representative of participants’ daily lives if compliance was lower than 80%, while another study considered that analysis using observations of participants who responded over 75% of the time would be reasonable [6].

One challenge is the complexity of EMA data [32]. An EMA protocol usually must consider item selection, period, intensity, signaling algorithm, event recording, application type, and data storage. Our review showed that the frequency of data collection varied from 1 to 10 times per day over a time period of 6 days to 48 weeks. Repeatedly answering the same questions in an EMA method requires substantial involvement, which increases the respondent’s burden, and this aspect can be frustrating for participants [33]. Related to this complexity of data collection, missing data also presents a limitation [31].

Regarding data analysis, EMA studies tend to produce multilevel datasets from multiple participants who answer a set of questions at multiple times. Therefore, standard linear and logistic regression analysis techniques are insufficient for analysis of EMA datasets. The complexity of EMA data analysis could hinder researchers and clinicians in using this method [5]. This should be taken into account when considering this technology-driven approach.

A limitation of this review is that we did not include studies that utilized other mobile devices, such as wearable sensors or personal digital assistants, since the purpose of this review was to provide insight into methodological strategies for EMA using mobile phones to assess mood or stress.

Future studies would include objective measures of related variables, such as heart rate, physical activity, and walking, which may be affected by mood and stress, to confirm dynamic relationships between symptoms and mood and stress. Additionally, multidisciplinary research involving areas such as medical diagnosis, consultation, nursing care, and ecological momentary interventions (EMIs) with EMA data collection could be an interesting focus. Through these multiple approaches, we expect to perform more accurate and valid mental and physical health monitoring and to provide optimized medical service for patients by applying patient-specific health care interventions.

Prevalence of basic- and advanced-feature mobile phones is high, and mobile technology is readily used as a ubiquitous resource. Mobile phones can be utilized easily in health research to assess patients’ experiences in their daily lives, as they are convenient for patients to carry and are user friendly. In addition, patients may feel comfortable using their own familiar mobile phones with EMA methods installed.

This review provides researchers with information regarding methodological details, such as length of administration period, mode of response, contingency of sending alarms, frequencies and durations, incentives for improving compliance, and statistical methods for data analysis when utilizing EMA to measure mood and stress in adult patients.

Despite the limitations of this study, we believe this review shows that EMA is an effective and reasonable way of measuring momentary mood and stress in an era in which mobile phones are ubiquitous in the general population, including patients. In particular, individuals who have experienced mood changes or stress can benefit from EMA methods by using mobile phones to monitor or track their mood and stress vulnerabilities. This review supports the use of EMA methods to evaluate mood and stress and recommends that researchers utilize EMA methods to measure psychological health outcomes of mood and stress in various patient populations.