Obesity and overweight are considered major public health concerns because of their high prevalence and association with various health complications including cardiovascular disease, type 2 diabetes, and cancer [1,2]. As the aspects that influence overweight and obesity are diverse—comprising individual, genetic, and environmental factors—their prevention and treatment are also complex. For a successful treatment, multifactorial approaches are required, with diet and exercise plans reinforced with psychological therapy and behavioral change strategies [3].

In recent years, we have witnessed a revolution in the use of apps within personal health care, as they are fast, flexible, handy, versatile, manageable, and illustrative tools that can empower patients. Hence, mobile health (mHealth) can play an important adjuvant role in the prevention and treatment of overweight and obesity by tracking physical activity (PA), enabling self-reporting of dietary patterns, providing recommendations to achieve healthier habits, guidance, advice, tips, and motivational strategies to achieve personalized goals; all are relevant aspects for the prevention and treatment of obesity, as recognized in numerous guidelines [3,4].

The Global Observatory for eHealth of the World Health Organization (WHO) defines mHealth as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants, and other wireless devices” [5]. The management—and in some cases the prevention—of chronic diseases has been one focus in recent developments in both electronic health and mobile health (eHealth and mHealth) [6]. There are over 325,000 health apps on the market, with the most downloaded being those relating to PA and weight control: that is, those that support a healthy lifestyle [7]. However, information on how the effectiveness, efficacy, and safety of mHealth apps in overweight and obesity management are assessed is severely lacking. It is important to note that according to mHealth publishers, over 53% of their health apps portfolio available in 2015 were downloaded less than 5000 times [7]. Evidence of the efficacy of mHealth app strategies in improving chronic health and well-being is mixed; therefore, while some mHealth interventions show promise in supporting weight management [8,9], others do not [10,11]. Numerous efforts to address this challenging issue are underway and some aspects that may be linked to a lack of efficacy have been identified. These are, among others, the poor quality of many apps, a lack of guidance on the usefulness of an app, and a low level of support from health professionals [12,13]. Should mHealth apps be rigorously evaluated to ensure they provide evidence-based effectiveness, safety, and efficacy? Up to now, mHealth evaluation methodology has not deviated from customary methods (mainly randomized controlled trials [RCTs]), despite claims that alternative, shorter, and more inexpensive design methods are required [14].

There are several initiatives attempting to define how apps should be evaluated. However, all of these consider only partial aspects of evaluation [15]. Although medical regulatory bodies have not validated the safety and quality of these technologies, individuals have adopted mHealth devices as self-management aids. However, medical professionals are often at a loss as to how to relate to them [16]. Owing to this rapid consumer-based introduction to the world of patient health aids, mHealth solutions present unique and stakeholder-specific challenges to the medical environment. Patients, health care providers, administrators, authorities, and mHealth developers alike are operating without a clear direction, which may lead to problems, including the improper use of mHealth interventions by individuals and the inability of medical systems to react due to a lack of technological and organizational support. Users and health care professionals should be aware of the quality of health apps they use or prescribe. The use of classic methodologies such as RCTs may not be the optimal procedure for evaluating all the dimensions of mHealth. Ideally, clinicians, health administrations, and users need instruments that enable the evaluation of e-interventions as a whole. From a global perspective, these instruments should facilitate the process of verification, validation, impact assessment, and certification that ought to be a requirement for all mHealth implementation.

This lack of rigorous evaluation is an increasing concern for health authorities. A number of recommendations to ensure a minimum quality of mHealth interventions have already been defined by the WHO Technical Evidence Review Group [17]. In addition, both the Food and Drug Administration [18] and the European Commission [19] have made several attempts to establish mHealth assessment and, where appropriate, certification criteria. However, in such a continuously evolving field, it has been difficult to reach a consensus.

The aim of this paper was to undertake a systematic review of efficacy, safety, and effectiveness assessment criteria in use, including both outputs and outcomes, to assess weight control, overweight, and obesity management in mHealth interventions. These criteria will later be included in a tool for assessing mHealth interventions intended to manage overweight and obesity.

Any trial that assessed the efficacy and/or safety and/or effectiveness of mHealth-based interventions for overweight or obesity management was considered. No restrictions in terms of target population were foreseen. We define efficacy as changes in lifestyles on the basis of diet and PA in a controlled population; effectiveness in the general population; and safety as a lack of adverse effects resulting from mHealth interventions. Studies carried out with less than 10 individuals were excluded. We assessed the quality of trials according to the Scottish Intercollegiate Guidelines Network (SIGN) criteria [22]. Taking the objective of this review into consideration, all studies were included regardless of quality.

A systematic search was conducted in the following databases: MEDLINE, EMBASE, PsycINFO, The Cochrane Library (Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials [CENTRAL]), UK Trial Database, and Scopus. This survey was supplemented through the snowballing technique to identify relevant articles in the references of those returned by the search. A manual search was also conducted on the indices of the following publications: Journal of Medical Internet Research and JMIR mHealth and uHealth. The survey period included all articles published up to May 2018. All types of clinical studies published in English, French, or Spanish were considered.

The search strategy included both controlled vocabulary and free-text terms. The terms used were apps, mHealth, eHealth, overweight, obesity, efficacy, security, safety, effectiveness, and evaluation (see Multimedia Appendix 1).

All identified references were imported into Mendeley v1.18 (Elsevier) and duplicates eliminated. A total of 6 researchers undertook the review process, which was conducted in 2 stages. First, each article identified was randomly assigned to 2 reviewers to independently review the title and abstract. Articles that met the inclusion criteria were full-text reviewed and quality-assessed by 2 independent reviewers. In cases of disagreement, a third reviewer made the final decision. Study features and outcomes were entered into a database specifically designed for this review. Risk of bias was assessed according to SIGN codes for study assessment [22]. Those trials that were clearly of an adequate quality were graded as high or ++ (very low risk of bias) or moderate or + (low risk of bias), while those of insufficient quality were graded as low or–(high risk of bias).

A total of 233 potentially relevant publications (17 from a manual search) were identified as eligible. From these, 19.7% (46/233) were identified as duplicates. From the remaining (187), only 49.2% (92/187) were accepted for abstract review. Out of these, 47% (44/92) were excluded for not following inclusion criteria. A full-text review was conducted on 48 studies. After peer review, 30 articles corresponding to 28 different studies (62.5% from the total included for full-text review) were finally included in this nonquantitative review. The exclusion criteria were as follows: published study protocols (n=8), out of scope studies that were not using an mHealth intervention (n=4), or those studies in which final outcomes were other than efficacy or safety (n=6; Figure 1; see Multimedia Appendix 2).

The main characteristics of the 28 studies included are detailed in Multimedia Appendix 3. Studies appear in alphabetical order of the first author within chronological years. All selected studies focus on efficacy; only 1 of them assesses effectiveness and 1 also focused on safety, although this was not the main outcome of the study.

In total, 46% of the studies (13/28) are RCTs, 1 is a cluster-randomized trial [23], 3 are nonrandomized controlled trials, and the remainder (11) single-arm trials; 2 of the RCTs include more than 1 intervention. Carter et al [24] studied the efficacy of a smartphone app or website in self-monitored weight management, and Hurkmans et al [25] compared 1 stand-alone app intervention with face-to-face and blended interventions. All studies compare pre and post outcomes to analyze the intervention’s efficacy. According to SIGN criteria, the majority of studies are of low (15) or moderate (6) quality, with only 7 studies reaching high quality. A low quality rating most often resulted from small sample size, inadequate length of study, or possible selection and information bias.

The number of participants ranged from 10 to 1012, but most studies (17) covered less than 100 people. One trial [26] had 15,310 participants, but the majority (83%) remained nonactive during the intervention. Most studies had a majority of adult women; in 6, all participants were women. There are also 4 studies targeted at children and teens. Most trials were targeted at people with overweight or obesity but no other health condition: exceptions targeted people with a severe mental illness [27], heart disease [28], type 2 diabetes [29] or prediabetes [30,28], cancer survivors [31,32], and people with metabolic syndrome [33].

Apart from one 24-month trial [34], the studies were conducted over short periods of time, ranging from 3 weeks to 6 months. The countries where the studies were carried out were the United States (17), Australia (3), Korea (2), the United Kingdom, Belgium, Spain, The Netherlands, China, and Israel.

In regard to the specificities of mHealth interventions (Multimedia Appendix 4), only 39% (10 out of 28) focused on a specific stand-alone app, with the majority addressing multicomponent interventions—including armband sensors, pedometers, wireless scales, and other monitoring devices, or websites—for weight management, intended to increase PA, reduce sedentary habits, and/or improve dietary patterns. The most common elements included in the trials were the receiving of feedback messages (24 studies out of 28, 85%), setting of goals (20), and self-monitoring (19). These feedback messages could be personalized reminders, recommendations based on the self-monitoring, standard counseling or health coach counseling through the app, and/or a more synchronic intervention. New elements have been introduced to mHealth interventions in recent years, such as gamification [35,30,23,29,26], entertainment aspects [30,36,37,23,38], and peer contact through community blogs [39] or virtual teams [30] on social networks [37,27,40] such as Facebook [27,40,25] and We Chat [26]. It is worth mentioning that only specific frameworks were mentioned when defining strategies for behavioral change, such as the transtheoretical model of behavior change for TXT2BFiT [39] and CITY [34], intervention and Control Systems Theory for eBalance App [41], self-regulation theory for Balance It intervention [23], and increasing adherence, such as Mechanics-Dynamics-Aesthetics for With U App [40]. Social Cognitive Theory is the one most often referred to by the Vegethon app [42], Loose It app [31], and Alive-PD [30] for which several other frameworks were also considered: behavioral economics, positive psychology, and the theory of planned behavior. One study was based on the Diabetes Prevention Program [43] and LookAHEAD (Action for Health in Diabetes) trials [44]. One study was based on an addiction treatment approach [37].

Although their main aim was to measure the efficacy of mHealth interventions, most of the selected studies also measured other outputs that might be relevant to determine primary outcome measures (23 out of 28, 82%). Multimedia Appendix 5 [45,46-50] shows the outputs and the main tools used to measure them.

The end point outcomes of the selected studies were as follows: reduction of weight and body mass index (BMI) as well as fat mass and waist and hip circumferences; changes in dietary habits, PA, and screen time patterns; biochemical measurements; and blood pressure (Multimedia Appendix 6 [58-73]).

In this systematic review, we have identified the range of dimensions and tools used to assess the efficacy of mHealth interventions intended to manage overweight and obesity. We have provided a descriptive analysis of 28 clinical trials along with an account of the components and elements included in each intervention. Outputs and outcomes used for the evaluation of trials have also been identified. No specific criteria for assessing safety or effectiveness have been identified due to the small number of studies focused on these aspects.

Our results show that researchers use the following primary end points to measure a study’s success: (1) reduction in weight and/or BMI; (2) reduction in fat mass; (3) reduction in waist and hip circumference; (4) improvement in dietary habits/patterns; (5) increase in PA; (6) increase in emotional well-being; (7) decrease in screen time patterns; (8) improvement in biochemical measures; and (9) decrease in blood pressure. All these factors are closely linked to obesity and overweight and are risk factors for future chronic disease. Although the main aim of most of the studies was to measure the efficacy of mHealth interventions, they also measured other outputs that might be relevant for determining the success of the intervention, such as (1) acceptability, (2) adherence, usability, and engagement, (3) satisfaction, (4) motivation, (5) intention to continue, and (6) perceived support. All these aspects appear to affect whether an intervention will be successful. Tests and questionnaires are the most prevalent tools used for assessment, whether existing and previously validated or devised for the situation. Objective data tracking of PA performance through the mHealth-based intervention, when possible, was a common strategy for avoiding self-reported data. It appears to be highly important to gather objective data and use standardized protocols when assessing the usability and efficacy of mHealth interventions. The mHealth strategies considered to be more sophisticated usually include a higher number of elements. Although the recent strategies of peer support and gamification appear to improve efficacy by increasing engagement and motivation, there is as yet not enough evidence to state this definitively.

The acknowledgement and evaluation of comprehensive sociodemographic differences, such as race/ethnicity, socioeconomic status, and sex, are severely lacking. Future analyses of mHealth interventions should consider and, whenever possible, include eHealth literacy aspects in an effort to reduce communication inequalities across groups [80]. Unless designers and developers of health care information technologies address security challenges, benefits from health care information technology will be scarce [81]. Another aspect we have found to be lacking from mHealth evaluation studies is assessment of clinical data confidentiality.

Previously published reviews have concluded that despite a lack of evidence concerning the best use of technology in weight loss interventions, when the optimal combination of technological components is determined, technology-based interventions will be a valid tool for weight loss [82]. Others have been less optimistic and feel that future studies must use larger study samples, longer interventions, and follow-up periods [83]. One meta-systematic review concluded that despite the increasing popularity of mHealth, evidence for efficacy is still limited due to the low methodological quality of research [84]. We believe the issue may be how mHealth strategies are assessed and validated: this cannot be carried out in the same manner as research into drugs, and more adapted and/or flexible approaches are needed to explore new evaluation tools. An instrument intended to evaluate mHealth should include verification of its scientific content and mechanisms that ensure data privacy as well as safe usage. Verification of these aspects would ideally be mandatory before release and use in clinical practice. In the second phase, evaluations of effectiveness, efficacy, and usability should include user feedback, and adaptability and cost-effectiveness should also be addressed. This second phase of evaluation could be quantitative, enabling assessment of an mHealth intervention’s quality and comparison with others.

Currently, most apps used or prescribed in daily clinical practices have only received technical verification or partial clinical validation on the basis of a small group of patients.

Future research is necessary to better assess mHealth interventions in development and before clinical application. It is important to find a balance between the necessary development of mHealth, which should be characterized as disruptive, innovative, and rapid, and the imperative need to validate mHealth interventions. From a Public Health point of view, it is necessary to avoid or minimize the potential problems a new mHealth intervention might create without accurate evaluation. It has been argued that the app market regulates itself: the good persist; the bad disappear. However, in such a potentially harmful field as mHealth, there is a need for new approaches and tools, and a multidisciplinary assessment process [14,85-89].

One of the main limitations of this review is publication bias. References from other sources such as conferences and meetings have not been included. Although the number of scientific journals that publish mHealth-related articles has increased in recent years, there is a lot of gray literature surrounding this field that we may have missed. Moreover, only studies published in English, French, or Spanish have been included. The heterogeneity of interventions and populations and the low number of participants in many studies have made it difficult to synthesize results. Most of the studies included were deemed to be of moderate-low quality, and consequently findings need to be considered with caution. In total, 11 studies lacked a control group and therefore results cannot be attributable to the technology-based intervention alone. One must also take into account the established fact that individuals who agree to participate in intervention studies have greater motivation to change their lifestyles than the general population.

Finally, only 1 study was identified with the primary aim of assessing the safety and effectiveness of an mHealth intervention. Given awareness of safety-related issues such as a possible increase in anxiety and stress due to the use of mHealth intervention and the possible promotion of eating disorders, this is rather surprising. Furthermore, the studies reviewed largely assessed dietary habits and PA, ignoring other possible outcomes relating to body weight such as sleeping behavior. This also needs to be addressed in future research.

The potential for apps to positively help users manage their obesity or overweight has yet to be attained. Studies assessing the success of mHealth interventions are remarkably heterogeneous and most have methodological limitations that leave significant room for improvement regarding quality. Further research is needed to identify all relevant criteria for assessing the efficacy of mHealth interventions in the prevention and management of overweight and obesity.