Although new mHealth technologies are assumed to be able to improve the quality and quantity of sleep, limited examinations of behavioral sleep interventions using mobile phones exist [13,22]. For example, previous studies [27-31] have focused on examining the technological aspects of mobile phones only (ie, calibration and validation studies, device feasibility). Moreover, previous studies examining the use of the mobile phone’s effectiveness for addressing sleep quality or quantity have only focused on one specific sleep disorder, such as sleep disturbance [32] or obstructive sleep apnea [33,34]. Therefore, the primary purpose of this review was to determine the effectiveness for sleep disorder improvement. We hypothesized that interventions with mobile technology have a positive impact on improving sleep quality and various sleep disorders (eg, sleep apnea, snoring, or insomnia).

This study included articles if they met the following criteria:

Articles published between January 1983 (the year of the first handheld and commercial cellular phone from Motorola [35]) and December 2016 were searched from four electronic databases, including EBSCOhost: Academic Search Complete (ASC) & Cumulative Index to Nursing and Allied Health Literature (CINAHL), PubMed/Medline, Scopus, and Web of Science. The journal search process occurred from February 18 to 19, 2017. To identify components of three research topics, namely, “sleep,” “mHealth,” and “study design,” the specified search keywords were used as a search algorithm (see Multimedia Appendix 1).

After searching the electronic databases, one author (JCS) selected articles that were published in peer-reviewed journals and excluded books, case reports, conference proceedings, product reviews, newspapers, patents, serials, and theses. Secondly, the same author removed duplicate articles from the combined search results and screened articles based on the title and abstract. In addition, JCS hand-searched for relevant articles in the JMIR search engine on February 28, 2017. Once all full-text articles were found, two researchers (JCS and JK) reviewed each article using the eligibility criteria and eliminated unrelated articles. The entire procedure followed the guidelines for Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) [36].

The following information was extracted from each article included in the systematic review: first author’s name, publication year, the country where the study was conducted, sample size, sample age, sample characteristics, study period, retention rate, study design, the technology used in the intervention, and measurement of sleep outcome. The retention rate was calculated using the number of participants who completed all assessments for the intervention study divided by the number of individuals who were originally recruited. Intervention technology was identified using two approaches: (1) intervention using mobile phone app and (2) supplementary mobile phone usage with traditional intervention. Additional information was extracted by using the empirical approach of statistical analysis such as effect size, standard error, and standard deviation for the difference before and after interventions.

The effect size and standard deviation for each study’s sleep outcome measurement were obtained from data extraction. With each effect size, there was a difference of calculation for the total effect size based on the type of study. For example, the effect size of pre-post test was only considered the difference between pretest scores and posttest scores. On the other hand, the effect size of RCTs was calculated using the difference between the intervention and the control groups’ effect sizes. Based on the total effect size, sample size, and standard error, two-sample t tests were performed to determine the mobile phone intervention effectiveness.

The study quality assessment tool was derived from Zhu and An [37], and then revised after discussion and peer review by the researchers for the purpose of this study. The quality of research was assessed using the following criteria: (1) the research question and objectives were stated clearly, (2) mobile health and/or mHealth was defined, (3) a control group was included, (4) participants were randomly recruited from a well-defined population, (5) sample size was more than 30, (6) attrition was analyzed and determined not to significantly differ by respondents’ baseline characteristics between control and experiment groups (<20%), (7) baseline characteristics between control and intervention groups were similar, (8) the intervention period was at least 4 weeks, (9) the sleep disorder measurement tools were shown to be reliable and valid in previous studies, and (10) demographic information was available to control for potential confounders in future analysis. A sample size of 30 was chosen due to a guarantee of a normal sample distribution based on the central limit theorem. The total study quality score was determined by summing items 1 to 10, and categories were created to define a general quality of research as poor (score 0-3); moderate (score 4-6), and high (score 7-10).

The results of the literature search are summarized in Figure 1. A total of 2674 articles were initially selected from four electronic databases (EBSCOhost: CINAHL&ASC, PubMed/ Medline, Scopus, and Web of Science). After excluding 440 duplicate articles, 896 articles were excluded from the first screening of peer-reviewed journal articles and 1339 articles were left. After the first screening, five articles were included from hand searching, and 1304 were excluded based on the title (n=1180) and abstract (n=124). All 39 remaining articles were then downloaded as PDF files for full-text peer review and discussion [13-16,21,22,38-70]. We excluded 23 articles after the full-text screening for the following reasons: (1) a device or program feasibility study (n=2) [66,69], (2) duplicated article (n=1) [21], (3) not an intervention study (n=5) [39,43,47,49,53], (4) not a mobile phone intervention (n=7) [14,42,48,56,58,63,68], (5) no sleep outcome (n=3) [16,44,65], and (6) a protocol or trial (n=5) [40,45,50-52]. The entire selection procedure was independently performed by two researchers (JCS and JK), and then the research team resolved differences by discussion. From individual selections, the interrater reliability analysis using the kappa statistic was performed with peer reviewers to determine consistency among raters. The interrater reliability was found to be Cohen kappa=.80, representing substantial agreement [20]. Therefore, a total of 16 articles that satisfied the inclusion criteria were used for the final analysis.

Table 1 provides summary information of individual studies. A total of 16 studies were included in the analysis: one case study [46], three pre-post studies [13,15,41], and 11 RCTs [21,22,38,54,55,57,59,61,62,64,67,70]. Of the 12 RCTs, three studies [62,64,67] used mobile phone apps as an intervention tool; one study [21] used text messages as part of the intervention. RCTs ranged in total sample size from 30 to 502 and had a larger sample size than pre-post studies, which ranged from three [13] to 12 [41]. The study period for RCTs was from 4 weeks to 6 months, whereas for pre-post studies it was between 2 and 5 weeks. For the study region, eight studies were performed in the Unites States [13,15,21,41,55,61,62,70], two studies in Canada [54,67], two studies in Asia (Hong Kong [57] and Taiwan [46]), and four studies in Europe (Finland [38], France [64], Netherlands [21], and Sweden [59]). Most of the studies were published after 2012, except one study that was published in 2006 [67]. Three studies [15,46,64] focused on an elderly population, whereas three studies [21,67,70] focused on a young adult population.

Table 2 shows the number of studies based on study design and sleep measurement tools. Several questionnaires were used to evaluate various aspects of sleep quality and quantity.

Specifically, the Pittsburgh Sleep Quality Index (PSQI) [13,21,22,55,57,61,70] had a list of items to evaluate sleep quality and it was the most frequently used measure in our review. In addition, three studies used a sleep quality [15,57,59] measure that consisted of a simple question about the participant’s perception about sleep quality using a five-point scale. Moreover, five studies [15,21,57,59,61] used measures of sleep quantity: (1) sleep-onset latency (n=4; the length of time to transition from full wakefulness to sleep completely) [15,57,59,61], (2) number of awakenings (n=1, the frequency of awakening during sleep) [15], (3) wake after sleep onset (n=4; the amount of time [eg, minutes] one wakes up during sleep) [15,57,59,61], and (4) total sleep time (n=4; total length of sleep time) [21,57,59,61]. Total sleep time is also used to calculate sleep efficiency [57,59,61]; that is, the percentage of total sleep time per total bed time. For the measurement of behavioral aspects of sleep, some researchers used the Epworth Sleepiness Scale (ESS) (n=4) [38,41,54,64] to capture participant sleepiness. Also two additional studies measured other specific sleep-related behaviors, such as the Sleep-Related Behavior Questionnaire (SRBQ) [59], the Dysfunctional Beliefs and Attitudes about Sleep Scale (DBAS) [57,59], and Sleep Satisfaction Rate (SSR) [46]. Finally, some sleep measurement tools focused on specific sleep disorders such as the severity of insomnia. The Apnea-Hypopnea Index (AHI) [38,54] is used to evaluate sleep apnea, and the General Sleep Disturbance Scale (GSDS) [67] is meant to examine factors that interrupt sleep. Two studies [15,59] used tools to examine the emotional aspects of sleep, such as bed time stress (BTS) levels [59], and the Hamilton Rating Scale for Depression with sleep (HRSD) [15]. All sleep outcome measurements were conducted by subjective measurement (ie, self-report questionnaire).

Table 3 provides a summary of the effectiveness of each sleep intervention. It is categorized by sleep outcome measurement, and the statistics can be comparable if there are more than two sleep measures. Specifically, half of studies [13,22,41,46,62,64,67,70] used a single sleep outcome measurement, while others had at least two or more measurements. Of nine studies [13,15,38,46,54,57,61,62,64] of standard treatment, five studies compared a mobile phone intervention and standard treatment intervention: (1) continuous positive airway pressure (CPAP) [38,54,64] and (2) cognitive behavioral therapy for insomnia (CBT-I) [57,61]. Five studies [13,22,41,46,61] focused on a mobile phone app. Two of these studies [13,61] used a “CBT-I coach” app that was developed based on CBT-I, which is considered one of the traditional standard treatments for insomnia (Figure 2).

Table 4 summarizes the results of the quality assessment of the studies included in this review. The quality score of the final articles ranged from 3 to 10 out of a possible score of 10, with an overall mean score of 7.41 (SD 0.44). The distribution of quality scores differed substantially across each criterion. Of 16 articles, 12 articles were considered high quality [21,22,38,54,55,57,59,61,62,64,67,70], two articles moderate quality [15,41], and two articles poor quality [13,46]. Most of the studies had clear research questions and objectives [13,15,21,22,38,41,46,54,55,57,59,62,64,67,70], and used valid sleep measurement tools [13,15,21,22,38,41,54,55,57,59,61, 62,64,70] for outcome measures after 4 weeks [15,21,22,38, 41,46,54,55,57,59,61,62,64,67,70]. All RCTs [21,22,38,54,55, 57,59,61,62,64,67,70] satisfied items 3 (ie, existence of control group) and 5 (ie, sample size greater than 30). The mean intraclass correlation (ICC) between peer reviewers for quality assessment was ICC 0.987 (95% CI 0.964-0.996, P<.001).

The mobile phone can be used as a tool to effectively deliver and enhance traditional behavioral interventions due to its portability. We found eight articles that described the advantages of using mobile phones to improve traditional intervention methods for sleep disorders [38,54,55,57,59,61,67,70].

The purpose of our systematic review was to investigate whether mobile phones are a feasible and usable tool to improve sleep disorders and sleep quality in intervention studies. We also assumed that auxiliary mobile phone use helps to enhance the performance of existing intervention methods.

This review presented the articles based on two intervention methods. The first method was utilizing mobile phone apps and the second was conventional mobile phone support through telephone calls or text messages. Although many mobile phone apps were used independently, the conventional mobile phone supporting methods were (1) to use the mobile phone itself as only an intervention tool (eg teleconferencing or telecounseling), or (2) to combine with another treatment (eg, CBT-I, CBT-B, and CPAP) to enhance the effectiveness of sleep interventions. In our review, six studies were related to CBT-I [13,15,46,57,61,62] and three studies to CPAP [38,54,64]. This indicated that sleep intervention studies using mobile phones have been developed based on the standard treatment, which has already shown reliability [33,53,71-73].

Among the sleep outcome measurements, PSQI was the most frequently reported, and PSQI scores of all seven studies [13,21,22,55,57,61,70] were significantly decreased in the intervention group using mobile phones compared to standard treatment and a waitlist group. Participants in the intervention groups with mobile phones had a mean decrease in PSQI scores of 1.73 points compared to the waitlist group, and a mean decrease of 0.97 points compared to other treatment groups. Although the difference of PSQI was statistically significant, clinicians must consider the clinical significance of a one-point change in PSQI scores. According to the Buysse et al [74], a score of 5 of 21 is the cut-off point for poor sleep. Thus, the clinical significance of a one-point difference depends on the baseline score.

The ISI was the second most frequently used sleep outcome measurement tool, and all five articles using the measure [15,57,59,61,62] found a statistically significant decrease in ISI score in the mobile phone use intervention group compared to non-mobile phone use group. Participants in the intervention group with mobile phones had a mean decrease in ISI scores of 5.09 points compared to the waitlist groups [57,59,62], and a mean decrease of 3.2 points in ISI scores compared to other treatments [57,59,61]. Because a change of 7 points in the ISI is interpreted as a different level of insomnia, a 3- or 5-point decrease in ISI scores might be considered a critical improvement. Although ESS, WASO, SOL, and TST were included in at least four studies, there was at least more than one study that reported a lack of statistical significance of intervention effect [15,38,41,54,57,59,61]. Therefore, ISI and PSQI seem to be the most applicable measurement tools to investigate the effect of mobile phone intervention on sleep disorders.

In general, 87.5% (14/16) of the studies [15,21,22,38,41,54,55,57,59,61,62,64,67,70] reviewed supported the evidence of capability and efficacy of mobile phone usage interventions. When mobile phones were used as auxiliary equipment, the intervention that applied telephone calls or mobile phone apps clearly demonstrated equal or enhanced efficacy for sleep quality and index score compared to the traditional sleep intervention [15,38,54,55,57,59,61,64,67,70]. This difference was seen not only for the waitlist control group [54,55,57,59], but also for the telephone-supported intervention and traditional treatment [15,38,61,64,67,70]. The advantage of mobile phone intervention was also illustrated in apps using intervention [13,22,41,46,62] or text message interventions [21].

There are several limitations of the studies included in this review. First, there is no standardized study design, especially for the test period, procedure, and sleep intervention tools. For example, there is a limitation to use all mobile phone apps to compare each study because researchers used their own personally developed or nonpublicly available apps. As such, we were unable to make comparisons between interventions due to differences in app functions and interfaces. Due to the heterogeneity of the study design and sleep measurement tools, it was difficult to compare the effectiveness of the sleep app interventions. This will be an issue for reproducibility in further research. Second, although it is a common limitation for RCT study designs, the uniqueness of each study’s target population limits further analysis and replication. For instance, results from studies with a cannabis disorder use group [13] and a post-deployment soldier with mild traumatic brain injury [70] were difficult to extrapolate to the general population. Third, there are few interventions using mobile technology that have been published in peer-reviewed journals, so we were limited with respect to the number of articles we could include in the study. Although many t test for each sleep measurement showed significant effectiveness of sleep interventions, the small number of studies for subgrouping limits performing a meta-analysis. Fourth, there is the possibility of missed articles. Because the mHealth market is growing rapidly, our search possibly missed some articles that used new mobile phone intervention tools. On the other hand, it also might be possible to miss some articles during the search stage due to the usage of different language or jargon unique to the evolving mHealth industry.

This study has several strengths. To the best of our knowledge, this study was the first to review the effectiveness of mobile phone interventions on sleep quality, quantity, and sleep disorders. By focusing exclusively on the mobile phone itself, it allows us to tailor future mHealth interventions for sleep. Also, our study examined various aspects of sleep measurement tools that account for sleep quality, sleep quantity, and many sleep disorders such as insomnia and sleep latency. Additionally, our study provided evidence of the potential of mobile-based interventions for improving sleep disorders. Along with current research that support the benefits of cost-efficiency of mHealth interventions, these findings provide an impetus for further research examining the empirical evidence of sleep interventions using mobile phones.

In conclusion, our systematic review supports the evidence that mobile technology-based interventions are an effective tool to improve symptoms of sleep disorders and quality of sleep than traditional intervention without mobile phone. Also, we suggest the following research design for future sleep intervention studies: (1) PSQI and ISI as sleep outcome measurements, (2) RCTs, (3) compare with standard treatment (ie, CPAP, CBT-I), and (4) compare to a waitlist control group. In addition to intervention methods, because mobile phone apps vary and many of these apps are not being studied, it is important to perform a content analysis on commercially available apps to determine common functionalities prior to undertaking interventions [75]. Our finding was not only applicable to those with sleep disorders who need clinical care, but also to medical professionals who are interested in ways to determine effective sleep interventions. Moving forward, app developers and sleep experts need to develop evidence-based guidelines with behavioral change components for sleep apps to maximize their efficacy and to take advantage of mobile phones to apply to existing standard treatments for sleep interventions.