Outputs from this modeling include incremental quality-adjusted life-years (QALYs) gained and costs or cost savings in New Zealand dollars (NZ $). Outputs were discounted at 3%, with 0% and 6% used in scenario analyses. A health system perspective was used, and benefits and costs were modeled using a lifetime horizon. The intervention was modeled as a one-off intervention implemented in 1 year in overweight and obese New Zealand adults.

To estimate the difference in QALYs and health system costs between the model’s intervention and business-as-usual (BAU) comparator, the entire New Zealand population, alive in 2011, was simulated out until death using a dietary multistate life table (MSLT) model built in Excel. The structure and BAU inputs for this generic model are described in detail in the model’s technical report (see Multimedia Appendix 1 [5]). The remainder of this Methods section provides a summary of general structure and BAU inputs and more details on specific intervention parameters for this mHealth intervention.

The BAU comparator was assumed to include the existing level of mHealth promotion—which is negligible in New Zealand (ie, we are aware of no health agencies in the country that promote specific smartphone apps for weight loss). Therefore, we did not strip the baseline component of the model back to a hypothetical “no mHealth” comparator. Briefly, the BAU model uses projected all-cause mortality and morbidity rates by sex and age and separately for Māori (indigenous population) and non-Māori ethnic groups. Running alongside this main life table were 14 body mass index (BMI)–related disease life tables, where proportions of the population simultaneously resided: coronary heart disease (CHD), stroke, type 2 diabetes, osteoarthritis, and multiple cancers (ie, endometrial, kidney, liver, esophageal, pancreatic, thyroid, colorectal, breast, ovarian, and gallbladder). The proportion of the New Zealand population in each disease life table was a function of the disease incidence, case fatality, and remission (the latter in cancers only).

The intervention was modeled as a one-off smartphone app promotion that occurred in year one, 2011. The intervention effect was captured through changes in BMI resulting from the mHealth intervention. The change in BMI was then combined with relative risks for the associations between BMI and diseases through population impact fractions (PIFs; percentage reductions in future BMI-related disease incidence) that alter the inflow to the BMI-related disease life tables. Time lags from change in BMI to change in disease incidence were allowed for by using the average BMI change over a previous window of time of 0 to 5 years for CVD, diabetes, and osteoarthritis and 10 to 30 years for cancers. Probabilistic uncertainty about the boundaries (5, 10, and 30 years) was also specified (see Multimedia Appendix 1 [5]).

In this study, mHealth programs are those that use smartphone apps to deliver health information and behavior change support to participants for weight loss. It was assumed that smartphone apps would be promoted nationally through 1 main medium. First, weblinks to the best 5 iOS and 5 Android weight loss apps (all under NZ $4 to download), as recently identified through a New Zealand study [12], would be displayed on the Ministry of Health (MoH) and other health promotion websites. These apps largely work through their calorie counting and exercise tracking features with extra tips and support features. Promotion of these apps would be through a government-funded mass media campaign.

The proportion of the population that would receive this intervention and how this is calculated is presented in Figure 1. The target population was overweight or obese adults (the target population was those older than 18 years, but relative risks for the association between BMI and disease apply from age 25 years onward) living in New Zealand, who have access to smartphones and who want to try and lose weight. The proportion of New Zealand adults (aged >18 years) who were overweight or obese was taken from the National Nutrition Survey (2008-2009) and was estimated by sex, ethnic, and age groups. The proportion of this population who take up this intervention was calculated as follows.

First, it was estimated that 74.42% (with an uncertainty interval [UI] of 57.49%-88.19%; see Table 2) of the population have access to smartphones apps. A total of 2 estimates of smartphone usage were used: a survey carried out by Research New Zealand [13] reporting 59% smartphone ownership or access by New Zealand adults in 2013 and a forecast of 90% smartphone access by New Zealanders by 2018 [14]. (We use 2011 baseline data but have used more current smartphone usage in New Zealand to give more relevant outputs).

Second, the number of people that would be reached through a mass media campaign was estimated based on the reach of a number of previous national-level Health Promotion Agency (HPA) campaigns listed below. The HPA is a state-funded organization that leads programs to promote health in New Zealand.

Our estimate of the likely reach of the mass media campaign promoting the smartphone apps for weight loss was based on an average of these mass media campaign figures with uncertainty spanning its range: a central estimate of 77.94% with a UI of 70.00% to 89.00%. Nearly half (46.8%) of all New Zealand adults (78% of the 60% of New Zealanders that have access to smartphones) were assumed to be exposed to the promotion of the intervention and have a smartphone. This is probably a relatively conservative estimate as there may be additional reach through mechanisms we did not model, that is, via Web-based activity as suggested in the HPA healthy eating program (the average total reach for the HPA healthy eating program for all Web-based activity for 2013-14 was 2,272,525 hits per month [15]), through referrals by health professionals, and through word-of-mouth.

We used 2 US surveys which reported the percentage of smartphone owners who have downloaded a weight loss app to estimate the proportion of the above population that would download a weight loss app in this study. Fronstin [17] reports 25% of smartphone owners with private health insurance had used a weight management or diet app. In an internet report published by Pew Research Center, 12% of the 19% of smartphone owners that had reported downloading a health app downloaded a weight loss app [18]. This equates to 2.3% of all smartphone owners. This is a wide range of estimates and reflects baseline use of weight loss apps in the US population (not the impact of mass media campaigns) but is the best evidence we could identify as currently available. Wide uncertainty has been incorporated into the estimate to reflect this. The average (13.46% with a UI of 2.50%-25.00%; see Table 2) of these 2 figures was used as an estimate of the overweight and obese smartphone owners who have been reached by the mass media campaign, who are likely to download a weight loss app. Under this standard rate of app downloads, the mass media promotion does not increase the proportion of the target population downloading weight loss apps, but may still change the type of app downloaded to those promoted.

As this figure represents the usual download rate without a mass media campaign, we modeled a range of different but still plausible download rates: standard download rate (the base-case), 50% increase in downloading, and doubling of this download rate, with corresponding proportional UIs.

A Web survey in 2011 by the Consumer Health Information Corporation (N=395) in the United States found that 26% of downloaded smartphone apps are used repeatedly (ie, 10 times or more) [19]. Assuming this applies to overweight or obese people in New Zealand, this gave 1.36% of the target population who are likely to use the app more than 10 times (see Figure 1; note that in this figure, the final percentage of the population that is exposed to the intervention is 1.5% based on the average proportion of New Zealand adults that are overweight or obese, and when this number is calculated in the model using population weightings, it is 1.36%). Uncertainty around previously outlined parameters contributed to uncertainty around this final figure (1.36% [0.39%-2.99%]). See Figure 1 for a pictorial summary of this population selection process. All parameters used to target this intervention were the same for all age, sex, and ethnic groups except for the proportion of the population that is overweight or obese, which differs by age, sex, and ethnicity.

The effect size for reduction in BMI among successful app users (ie, 10 or more uses) was taken from a systematic review of RCTs for mobile devices and weight loss in adults [4]. It included 17 RCTs, 12 of which were primary studies and 5 were secondary analyses of primary studies. Of the 12 studies, 8 used a mobile phone as the intervention medium, specifically smartphones. Of these 12 studies, 9 targeted both diet and physical activity to induce weight loss. The remaining 3 studies concentrated primarily on physical activity to induce weight loss, whereas none of the studies targeted just dietary change. Intervention duration for the 12 studies (ie, not including follow-up) ranged from 4 weeks to 2 years, and studies were all carried out in high-income countries: the United Kingdom, United States, Finland, and Australia. All included studies used an intention-to-treat analysis in accordance with the original assignment. Interventions included a variety of approaches including weight, energy intake and energy expenditure goal setting and self-monitoring, text reminders on various topics, meal and physical activity planning, buddy components, trophy rooms for rewards, and even some group sessions and calls from counselors. The meta-analysis of these 12 studies with weight as the outcome estimated an additional 0.43 kg weight loss (95% CI 0.25-0.61) for the intervention groups compared with controls. This weight change was converted to a change in BMI using average height for the 4 demographic groups in the MSLT model (Māori men, Māori women, non-Māori men, non-Māori women).

The effect size of −0.43 kg (−0.61 to −0.25) equated to a reduction in 0.14 to 0.17 BMI units across sex by ethnic groups in those successfully completing the mHealth intervention. Regarding decay of effect, the trials included in the systematic review [4] did not measure maintenance of the weight loss over time. A meta-analysis of face-to-face dietary advice by Dansinger et al [20] found that BMI increased by 0.03 BMI units per month post dietary counseling from an initial BMI decrease of 1.9 units. Evidence on how weight regain differs by type of weight loss intervention and magnitude of initial weight loss is currently limited, so we used this 0.03 BMI units per month as an estimate of how the effect of the mHealth intervention would decay post intervention. With such a small initial effect size, the BMI decrease returned to 0 approximately 5 months post the year of the intervention.

Microsoft Excel using an Ersatz add-in (Epigear International, created by Jan J Barendregt) was used to run each of the scenarios presented with uncertainty through the model 2000 times. Each of these simulations involved a random draw from the probability density function about those parameters specified with uncertainty in Tables 1 and 2. The main results produced were incremental QALYs gained and net health system costs accrued. The net health system cost was the sum of the intervention cost and any difference in projected future health system expenditure resulting from changes in disease incidence because of the mHealth intervention (including extra health costs from any increased life span).

The modeled intervention is based on a meta-analysis of mobile device interventions, which reported that those following the mHealth weight loss interventions lost an average of 0.43 kg (95% CI −0.61 to −0.25) more weight than the controls [4]. When modeled through to changes in incidences of BMI-associated diseases and then health gain in QALYs and health system costs, this intervention was found to be of borderline cost-effectiveness for the total population when download rates increased by 50% but cost-effective for Māori, based on threshold of NZ $45,000 per QALY gained. However, it would be cost-effective for the total population if download rates doubled (NZ $29,900 or US $20,100 per QALY) as a result of the mass media campaign. Further research to improve the estimate of download rates resulting from this type of campaign would be useful. The total impact on health was small, with total QALYs gained ranging from 29 to 59 over the remaining life span of 4.4 million people entering the simulation. Costs to the health system ranged from NZ $1.8 to NZ $2.3 million.

Even with higher download rates (and therefore higher health gain), the total health gain in these scenarios was still small from a population per capita perspective. For example, health gains seen with annual tobacco tax increases gave a total of 60,400 QALYs in the New Zealand population [23]. Reducing dietary salt intake by 35% (through mandatory maximum salt levels in packaged foods and reductions in salt through fast foods and restaurant food and discretionary intake) generated 235,000 QALYs [24] over the lifetime of the New Zealand population.

Assuming that the intervention itself is equally effective for Māori and non-Māori, age-standardized population per capita QALYs for Māori were double of those for non-Māori, reflecting higher rates of overweight and obesity in Māori compared with non-Māori. Even greater benefit for Māori could potentially be achieved from an app and promotional campaign designed specifically for this population group.

The effect size used for modeling is an important parameter that drives the health and cost outputs presented. This was taken from a meta-analysis, the best quality evidence available, but the effect seen in the New Zealand context may vary from the meta-analysis effect size. The mHealth apps in the meta-analysis are different from the apps that would be promoted through this intervention if it was implemented in New Zealand. The meta-analysis effect size also included other mobile devices other than smartphones and other elements of mHealth. These differences between the modeled effect and the likely real effect are important to consider along with the fact that the uncertainty around the effect size was the second largest contributor to the overall uncertainty of the results. The lack of evidence on the effectiveness of a mass media campaign on download rates is another limitation of this work. This parameter contributes the most uncertainty to the health outcomes, and variation in this parameter changes the intervention from being cost-ineffective to being cost-effective. Therefore, future research regarding this parameter would better inform understanding of the cost-effectiveness of promoting smartphone apps for weight loss.

The modeled health benefits may actually be underestimated for a number of reasons. This study only models the effect of the intervention in those that actually take up the intervention, but there may be additional spill-over benefits to other members of the household through health-promoting changes in household meals or additional family physical activity. Furthermore, the impact of the intervention on physical activity itself was not modeled through to disease incidence. The intervention was also modeled as a one-off intervention when in reality the intervention could be ongoing, having the potential to both recruit more people over a number of years and/or sustain behavior change among initial participants. It is also likely that smartphone app usage and the quality of the apps available will increase over time, with effectiveness potentially increasing the effect size relative to that from the meta-analysis published in 2014. For example, higher-quality apps can integrate the collection of data on dietary energy intake with automatic estimates of energy expenditure based on the pedometer built into the smartphone. Additionally, the scope for this type of intervention may expand if smartphone weight loss apps were integrated with dietary counseling in primary care. These types of apps might become more integrated into daily routines and so weight loss achievements might be sustained for longer periods into the future.

The evidence for the rate at which the weight loss attenuates back to baseline, the BMI decay, is sourced from a meta-analysis [20] and is based on weight loss dietary counseling interventions. However, weight loss decay may differ for mHealth interventions where individuals can continue to access the app in the future, unlike with face-to-face dietary counseling, which is time limited. The scenario analysis where weight loss is maintained over the life course shows much greater health gains (2420 QALYs) and produces substantial cost savings (NZ $44.3 million). It is likely that the truth lies somewhere between the base-case and this scenario. Furthermore, as discussed above, app design and also changes to the obesogenic environment may impact the intervention decay rate. These factors may warrant additional research to improve estimation of health gains.

Finally, this study takes a health system perspective, but this intervention might result in wider societal benefits, for example, modifications made to people’s diets for weight loss could result in lower consumption of energy-dense dairy products and meat products, therefore reducing greenhouse gas emissions [25] and other livestock-related environmental damage (water use, water pollution, erosion, and reduced biodiversity). Improved health from a lower BMI and increased physical activity could also result in higher productivity in the workplace (eg, from reduced illness-related absenteeism, early retirement because of illness, and premature death before retirement age).

As the field of mHealth develops, further research into the proportion of overweight and obese people who would regularly use an mHealth weight loss intervention, how this would be influenced by mass media campaigns, their subsequent weight loss, and how long their weight loss is maintained would all be useful. Consideration could also be given to determining app usage in the context of smartphone-based digital assistants, which can access apps (eg, Google’s “Google Assistant” and Apple’s “Siri”), or the provision of weight loss support from home-based digital assistants (eg, Amazon’s “Alexa”).

The results of modeling this mHealth intervention suggest it is likely to have relatively small absolute health gains at a population level (given the current levels of app use and current app design). As such, smartphone weight loss apps should not be a priority for inclusion in current obesity reduction strategies. Resources should instead be prioritized toward cost-effective or cost-saving interventions likely to have greater health impacts, such as food or beverage taxes and subsidies [26], restrictions on marketing of unhealthy foods [27], and improved nutrition labeling [28]. The scope for smartphone weight loss apps may expand with increasing smartphone use, more easy-to-use and effective apps becoming available, and integration of app promotion with dietary counseling by health workers. mHealth for weight loss may therefore become a more viable component of obesity prevention strategies in the future.