This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Large-scale initiatives to improve diet quality through increased vegetable consumption have had small to moderate success. Digital technologies have features that are appealing for health-related behavior change interventions.
This study aimed to describe the implementation and evaluation of a mobile phone app called VegEze, which aims to increase vegetable intake among Australian adults.
To capture the impact of this app in a real-world setting, the Reach, Effectiveness, Adoption, Implementation, and Maintenance framework was utilized. An uncontrolled, quantitative cohort study was conducted, with evaluations after 21 and 90 days. The app was available in the Apple App Store and was accompanied by television, radio, and social media promotion. Evaluation surveys were embedded into the app using ResearchKit. The primary outcomes were vegetable intake (servings per day) and vegetable variety (types per day). Psychological variables (attitudes, intentions, self-efficacy, and action planning) and app usage were also assessed. Descriptive statistics and multiple linear regression were used to describe the impact of the app on vegetable intake and to determine the characteristics associated with the increased intake.
Data were available from 5062 participants who completed the baseline survey; 1224 participants completed the 21-day survey, and 273 completed the 90-day survey. The participants resided across Australia and were mostly women (4265/5062, 84.3%) with a mean age of 48.2 years (SD 14.1). The mean increase in intake was 0.48 servings, from 3.06 servings at baseline to 3.54 servings at the end of the 21-day challenge (t1223=8.71;
The VegEze app was able to increase intake by half a serving in a large sample of Australian adults. Testing the app in a real-world setting and embedding the consent process allowed for greater reach and an efficient, robust evaluation. Further work to improve engagement is warranted.
Poor diet quality is a risk factor for the development of chronic disease [
Most large-scale population campaigns [
Digital technologies are appealing for health-related behavior change interventions as they may overcome some of the limitations of traditional delivery approaches. For example, the ubiquitous nature of mobile phone ownership and its broad application and usage mean that mobile health (mHealth) interventions have the potential to reach large audiences at nearly any time or place. Mobile technology can also be highly interactive and can be used to deliver health-related information in a way that is engaging and rewarding. The ability to tailor content over time based on user inputs or objective measures (eg, wearable devices) can create personalization, which may increase engagement—the equivalent of exposure to or a dose of a traditional intervention—increasing the likelihood of success [
Preliminary evidence from mHealth interventions shows promise as they appear to be feasible and acceptable to users [
Research apps are typically evaluated using approaches such as randomized controlled trials, which emphasize internal validity as opposed to external validity. Such approaches provide robust evidence for efficacy in the study sample but have limited reach and provide little insight into their generalizability to the target population [
This study aimed to implement and evaluate the impact of a mobile phone app called VegEze, which aims to improve vegetable intake among Australian adults. The Australian Dietary Guidelines recommend “plenty of vegetables, including different types and colours” [
To better capture the impact of this app in a real-world setting, the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework was utilized. The framework has 5 components—reach, effectiveness, adoption, implementation, and maintenance—which are commonly used to translate research and understand impact and generalizability across populations [
How many Australian adults are willing to participate in the VegEze intervention? (Reach)
What is the impact of VegEze on increasing vegetable variety and intake? (Effectiveness)
Who used the VegEze app most? (Adoption)
How do participants use the features of the VegEze app and is app usage associated with success? (Implementation)
Does the VegEze app support participants to maintain their consumption for a longer term? (Maintenance)
An uncontrolled, quantitative cohort study was used to assess the impact of the VegEze app on the daily intake and variety of vegetables, with an evaluation conducted after 21 and 90 days of the program. The VegEze research study launched in the Apple App Store (free of charge) on November 8, 2017. To facilitate timely dissemination of the results, data for inclusion in this evaluation were extracted after 6 months (app download and baseline survey were completed between November 2017 and May 2018).
There was associated media coverage, including free-to-air television and radio interviews as well as social media promotions from November 13, 2017. Emails were also sent to an existing database of people who had opted in to receive nutrition-related newsletters.
Participants were eligible for the study if they were aged 18 years and above, were living in Australia, owned a compatible iPhone Operating System (iOS )10 or 11 device (including iPhone or iPad) with an internet connection, and were willing and able to download the app and participate in the trial. Those who had any condition or self-prescribed diet that prevented them from consuming vegetables were excluded. This study was approved by the Commonwealth Scientific & Industrial Research Organisation (CSIRO) Health and Medical Human Research Ethics Committee Low Risk Review Panel (proposal number 13/2017) and was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12618000481279).
All data were collected via mobile phones. We leveraged the Apple ResearchKit framework with the app because of its innovative features in transforming the way a large-scale cohort study can be delivered via a mobile phone. The onboarding process, including participant information and electronic consent, and the surveys were embedded into the app. Evaluation surveys were administered at 3 time points: baseline, end of the 21-day challenge, and at the 90-day follow-up. Although the surveys were designed to be as short as possible, they were still able to capture critical outcomes and predictors of behavior. Standard demographic questions about age, gender, and self-reported height and weight were also asked in the baseline survey only.
The primary outcomes were vegetable intake (reported in servings per day) and vegetable variety (reported in types per day) and were assessed at each time point. Vegetable intake was assessed using a series of short questions from the previously validated CSIRO Healthy Diet Score survey [
Vegetable variety was assessed in a single question asking about the number of different types of vegetables consumed in the past 2 days. Changes in vegetable intake and variety were calculated as end of challenge minus baseline consumption, where a positive value indicated an increase in consumption. One question asked about the frequency (always, usually, sometimes, and never) of achieving the target behavior, ie, having 3 different types of vegetables at dinner.
Attitudes, intentions, and self-efficacy are 3 well-established personal factors that have been shown to predict changes in health behavior [
Firebase by Google was used to collect extensive app-related data, including screen views, engagement with notifications, and app events (ie, interactions such as tapping navigation buttons), which helped to explain the behavior of participants when using the app.
A detailed description of the development process has been published previously [
Vegetable log feature as a core component of the VegEze app.
Two-way user feedback was also central to the app. On the home screen, users could review their daily progress for the current day (
Other features of the app included a challenge and awards section that provided awards to support goal setting and further feedback on the level reached in the challenge. Content sections were divided into
Other features of the VegEze app.
The app usage and interaction logs were extracted from Big Query by Google into Excel files and processed in R. Any logs related to app updates, iOS updates, and errors were removed, and time stamps were converted from microseconds to the date and time format. Participants’ app log data for 21 days from the date of starting the challenge were used for the analysis.
Google Analytics was used to interpret high-level app usage for the entire cohort. The app usage data were linked via a unique ID to a custom database that recorded participation data such as vegetable logs, demographic information, and survey submissions. Structured Query Language queries were used to extract summarized participation data into Comma-Separated Values files for analysis.
Membership duration was calculated for individuals as the length of time (in days) between the start date of the challenge and the last date of app usage. The attrition rate of the app over 21 days of the program was calculated using the membership duration. The number of active days for which individuals visited the app (ie, app usage) and the feature usage (Home, Veg Lookup, Challenges, Notifications, Meal Ideas, and Learn) were calculated for users who had at least one log instance for that day.
Survey data were extracted from a service called SurveyGizmo into IBM SPSS Statistics files. Extreme outliers were removed based on vegetable intake that was greater than 12 servings of vegetables per day, reported at any time point (equivalent to the mean±3SD).
Descriptive statistics (means, standard deviations, and percentages) were used to describe the characteristics of the sample. We applied a per-protocol analysis to calculate the significance of the change in vegetable consumption between baseline and the end of the 21-day challenge. This change was tested using paired samples
A previous review of digital interventions to increase vegetable intake, albeit in an adolescent population, reported increases of 0.1 to 0.4 servings of vegetables per day [
The media coverage resulted in over 86,000 impressions within the App Store, over 16,000 product views, and 12,777 people downloading the VegEze app (
A flow chart of participants in the VegEze study.
Following data cleaning, survey data, including vegetable intake measures, were available for 5062 participants at baseline, 1224 participants at the end of the 21-day challenge, and 273 participants at the 90-day follow-up. At baseline, 4683 users had both app log and survey data, with 1219 having app and survey data at the end of the 21-day challenge.
Participants who installed the app and completed the baseline survey were from across Australia, were mostly women (4265/5062, 84.3%), and were aged between 31 and 70 years (4183/5062, 82.7%; the mean age of the sample was 48.2 years, SD 14.1). The sample had a greater proportion of obese adults than the Australian population (590/5062, 31.4% in the sample vs 27.5% in the Australian population) and consequently a lower proportion in the normal weight category (
Demographic characteristics of the baseline sample of participants and their comparison with the Australian population.
Demographic characteristics | Sample (N=5062), n (%) | Australian populationa, % | |||
|
|||||
|
Male | 774 (15.29) | 49.4 | ||
|
Female | 4265 (84.25) | 50.6 | ||
|
Unisex | 23 (0.45) | —b | ||
|
|||||
|
18-30 | 675 (13.33) | 18.6 | ||
|
31-50 | 1997 (39.45) | 37.7 | ||
|
51-70 | 2186 (43.18) | 30.5 | ||
|
≥71 | 204 (4.03) | 13.1 | ||
|
|||||
|
Underweight | 52 (1.02) | 1.7 | ||
|
Normal weight | 1587 (31.35) | 35.5 | ||
|
Overweight | 1833 (36.21) | 35.3 | ||
|
Obese | 1590 (31.41) | 27.5 | ||
|
|||||
|
New South Wales | 1529 (30.21) | 32.2 | ||
|
Victoria | 1484 (29.32) | 24.9 | ||
|
Queensland | 916 (18.10) | 20.1 | ||
|
Western Australia | 446 (8.81) | 10.4 | ||
|
South Australia | 435 (8.60) | 7.4 | ||
|
Tasmania | 76 (1.50) | 2.3 | ||
|
Northern Territory | 12 (0.24) | 1.0 | ||
|
Australian Capital Territory | 93 (1.84) | 1.7 |
aAustralian population estimates were taken from the 2016 census, available from the Australian Bureau of Statistics [
bData unavailable.
Average vegetable intake per day was 3.1 servings (SD 1.9) and 2.5 types (SD 1.3) at baseline. This translates to 14.4% (729/5062) meeting the Australian Dietary Guidelines recommendation for vegetable intake. Furthermore, 22.1% (1119/5062) of the sample reported of
In the sample that completed the 21-day survey (n=1224), the distribution of vegetable intake, in servings and types, at baseline and at the end of the 21-day challenge is shown in
Distribution of (a) vegetable intake and (b) vegetable variety at baseline and at the end of the 21-day challenge.
The changes in vegetable intake varied for different demographic groups. Women significantly increased their number of servings (0.51 servings; t1077=8.74;
Change in vegetable consumption (servings and types) at baseline and at the end of the 21-day challenge by demographic characteristics.
Demographic characteristics | Sample (N=1224), |
Amount in servings | Number of varieties | ||||||||
Baseline |
End of challenge (day 21), mean (SD) | Change, mean (SD) | Baseline |
End of |
Change, mean (SD) | ||||||
|
|||||||||||
|
Male | 139 (11.36) | 3.06 (1.98) | 3.30 (2.00) | 0.25 (1.92) | .134 | 2.55 (1.26) | 2.78 (1.13) | 0.23 (1.17) | .02 | |
|
Female | 1078 (88.07) | 3.07 (1.72) | 3.58 (1.92) | 0.51 (1.92) | <.001 | 2.78 (1.24) | 3.15 (1.19) | 0.36 (1.29) | <.001 | |
|
Total | 1224 (100.00) | 3.06 (1.76) | 3.54 (1.93) | 0.48 (1.92) | <.001 | 2.75 (1.25) | 3.10 (1.19) | 0.35 (1.27) | <.001 | |
|
|||||||||||
|
18-30 | 95 (7.76) | 3.08 (1.68) | 3.30 (1.87) | 0.22 (1.74) | .225 | 2.73 (1.33) | 3.10 (1.17) | 0.38 (1.30) | .01 | |
|
31-50 | 411 (33.58) | 2.75 (1.63) | 3.28 (1.76) | 0.53 (1.83) | <.001 | 2.75 (1.26) | 3.09 (1.20) | 0.34 (1.32) | <.001 | |
|
51-70 | 654 (53.43) | 3.24 (1.81) | 3.74 (2.01) | 0.50 (2.01) | <.001 | 2.77 (1.22) | 3.12 (1.19) | 0.35 (1.26) | <.001 | |
|
≥71 | 63 (5.15) | 3.23 (1.88) | 3.54 (2.00) | 0.31 (1.79) | .172 | 2.57 (1.32) | 2.94 (1.08) | 0.37 (1.03) | .01 | |
|
|||||||||||
|
Normal weight | 392 (32.03) | 3.01 (1.68) | 3.40 (1.89) | 0.40 (1.84) | <.001 | 2.90 (1.22) | 3.27 (1.17) | 0.37 (1.19) | <.001 | |
|
Overweight | 436 (35.62) | 3.08 (1.78) | 3.61 (1.98) | 0.53 (1.99) | <.001 | 2.69 (1.26) | 3.05 (1.19) | 0.36 (1.29) | <.001 | |
|
Obese | 386 (31.53) | 3.10 (1.80) | 3.61 (1.91) | 0.51 (1.94) | <.001 | 2.67 (1.24) | 2.99 (1.19) | 0.32 (1.33) | <.001 |
aAn inadequate sample size to report on unisex (n=6) or underweight categories (n=9).
In this sample, the proportion of the sample meeting the dietary guidelines recommendation for vegetable consumption increased from 15.9% (195/1224) at baseline to 22.6% (277/1224) at the end of 21 days. The proportion of the sample reporting of
There was a small but significant increase in positive attitudes toward eating a greater variety of vegetables during the challenge period, albeit there was a highly positive attitude at baseline (3.84 to 3.87 points out of 4; t1223=2.85;
The attrition curve for the usage of the app is shown in
Participant attrition over the 21-day challenge (n=4683).
On average, participants actively used the app for 6.3 days out of the 21 days of the challenge. Furthermore, 49.2% (2304/4683) actively used the app for 2 to 7 days, 19.1% (894/4683) used the app for 8 to 14 days, 11.6% (543/4683) for 15 to 20 days, and 1.2% (56/4683) used the app every day during the 21-day challenge period (
Percentage of the sample actively using the VegEze app, for the baseline sample and for those that completed the program.
App usage | Baseline (n=4683) | Completers (n=1219) | |
|
|||
|
1 | 19.0 | 2.1 |
|
2-7 | 49.2 | 18.4 |
|
8-14 | 19.1 | 38.6 |
|
15-20 | 11.6 | 36.8 |
|
21 | 1.2 | 4.1 |
|
|||
|
Total usage | 6.3 | 12.5 |
|
Home screen | 6.1 | 12.3 |
|
Veg Lookup | 3.7 | 8.3 |
|
Challenges | 1.3 | 2.4 |
|
Meal Ideas | 0.9 | 1.6 |
|
Notifications | 1.0 | 2.0 |
|
Learn | 0.6 | 1.1 |
The average active usage was higher among the participants who had app data and completed the 21-day survey (n=1219/1224), ie, 12.5 days out of the 21-day challenge. Furthermore, 36.8% (449/1219) of this sample used the app for 15 to 20 days, and 4.1% (50/1219) used the app every day during the 21-day challenge (
Percentage of active users using app features throughout the 21-day challenge period (n=4683).
When app usage was divided into tertile groups, it was found that participants with the highest usage had actively used the app almost every day during the 21-day challenge. This group increased their vegetable intake by 0.63 servings (SD 2.02) per day over the 21-day challenge compared with 0.32 servings (SD 1.69) per day for those with the lowest app usage (difference=0.31 servings;
Change in vegetable consumption by level of app usage* (n=1219). *Lowest tertile of app usage: active days ranged from 1 to -10 days, average 6.4 days; medium app usage: active days ranged from 11 to -15 days, average 13.1 days; and high app usage: active days ranged from 16 to -21 days, average 18.3 days.
On the basis of multiple linear regression, gender; age; BMI; psychological variables of self-efficacy, attitudes, intentions, and action planning specific to vegetables; baseline vegetable intake; and active days of app usage accounted for 23.3% of the variance associated with the change in intake (
Of those who completed the 90-day survey (n=273), 93% (254/273) were women, 25.6% (70/273) were aged between 31 and 50 years, 65.2% (178/273) were aged between 51 and 70 years, 34.1% (93/273) were overweight, and 36.6% (100/273) were obese. In this sample, vegetable intake increased significantly from 3.1 servings at baseline to 3.8 servings (average increase 0.68 servings;
In this sample, the percentage of participants meeting the Australian Dietary Guidelines for vegetables increased from 16% (44/273) at baseline to 25% (68/273) and 26% (72/273) at the end of 21 and 90 days, respectively. Participants reporting of
The VegEze app was designed to be an engaging 21-day challenge to increase the amount and variety of vegetables consumed by Australian adults. Central to this was a clear and specific behavioral target of having 3 different types of vegetables at dinner each day. At baseline, 22% of the sample reported
This research app was made available through the Apple App Store, and the use of ResearchKit negated the need for any face-to-face contact and streamlined the consent process. This novel approach combined with a structured promotional campaign allowed us to reach a large national sample of over 5000 participants within a relatively short period. That said, only 40% of those who downloaded the app completed the baseline survey. Therefore, including the evaluation surveys as a compulsory part of the onboarding process appeared to be a barrier to the overall uptake. This may be because of the time burden and delayed gratification associated with completing the surveys for people who simply wanted to download and explore the app.
Compared with the Australian population, the sample of people reached by the recruitment process was largely women (84% in the sample vs 51% in the Australian population) and slightly younger (4% aged over 70 years in the sample vs 13% of the Australian population). Other characteristics were fairly similar to the broader population. Furthermore, the 1200 participants who completed surveys at the beginning and at the end of the challenge may represent a biased sample of those more motivated to participate in research, and the results may overstate the likely impact on intake. Therefore, whether the reported changes in consumption are generalizable to the population more broadly is unknown.
In terms of the impact of the intervention on critical outcomes, the app showed great promise. Evaluation data indicated that the 1224 users who completed the survey at the end of the challenge reported an average increase of 0.5 servings in daily vegetable intake and an increase in variety of 0.35 types over 21 days. This change is consistent with, or greater than, the changes reported by large-scale population campaigns [
We found positive changes in attitudes, self-efficacy, and action planning during the challenge period, which can help change behavioral intention into action [
It was a purposeful decision to test this app in the real world; therefore, participants of this study used the app in representative settings. With regard to the representativeness of those who engaged with the program, women and those aged 51 years and above used the app more than men and younger adults. Despite recruiting participants through multiple channels—such as free-to-air television and radio coverage, social media, and an existing database—similar to the findings of previous health-related interventions, the majority of this study sample was women. Traditionally, mobile phone usage has been higher in younger adults, but more recently, the largest growth has been among older Australians [
Baseline vegetable consumption and app usage were the 2 strongest predictors of increased vegetable intake and variety. Participants with the highest usage, ie, using the app almost every day (16-21 days; average 18 days), increased their consumption by twice as much as those with the lowest app usage. The cause and effect cannot be assumed; however, it is encouraging that higher engagement with the app was associated with a more positive outcome in the behavior of interest. The fact that lower vegetable consumers reported greater increases in consumption was also promising. Although all Australians need to increase their vegetable intake to meet recommendations [
Long-term maintenance of behavior is important for realizing the health benefits associated with higher vegetable intake. Initial increases in vegetable intake were sustained up till a 3-month follow-up for which 87% of the sample reported of
Other digital interventions have not been able to sustain changes in vegetable intake over 3 months [
There are limitations to this study that warrant discussion. The study design was chosen to maximize reach and replicate a real-world setting, but a cohort study evaluation is not as strong to show efficacy as other methods such as a randomized controlled trial, and therefore caution is required in the interpretation of results. The subsample of participants who completed the follow-up survey was less than half of those doing the baseline survey, and data imputation was not used to account for missing data. Therefore, the change in vegetable consumption reported here is likely to overstate the potential of this app on intake more generally. Australia is largely a mobile phone market dominated by Apple and Samsung, with 42% and 35% of the market share, respectively [
Maintaining interest in an app is difficult. Only 15% of the sample actively used the app for the entire 21 days of the challenge. Nonetheless, the attrition rate observed here was similar to what has been reported elsewhere [
Inadequate vegetable intake is a global problem, and the health impact of increased vegetable intake is well known, meaning, there is a real need for novel strategies and interventions that achieve successful increase in vegetable consumption. The VegEze app was designed as a 21-day challenge to increase vegetable consumption, and results indicate that focusing on a specific behavioral target around increasing variety was successful in increasing the amount of vegetables consumed. We were able to shift the distribution of intake in a large sample and increase the average consumption by half a serving. Utilizing Apple’s ResearchKit allowed us to embed the evaluation process in the app and still place it in the App Store, contributing to the overall reach, generalizability, and rigorous evaluation of outcomes. Future research apps may also choose this approach to allow for more frequent evaluation of mHealth nutrition interventions in an uncontrolled real-world setting. Given that app usage was associated with successful behavior change, further work to improve engagement is warranted.
Commonwealth Scientific & Industrial Research Organisation
iPhone Operating System
Horticulture Innovation Australia Ltd
mobile health
Reach, Effectiveness, Adoption, Implementation, and Maintenance
The authors would like to thank the SP Health technical team for their role in supporting the app development company in building the VegEze app databases.
This project was funded by Hort Innovation through the research and development levy. Hort Innovation had no involvement in the evaluation or interpretation of results or preparation of the manuscript.
GH and AC led the development of the app. GM and GW contributed to the design and content of the app. GH, MH, and EB designed and conducted the analysis and wrote the results. GH, MH, and AC contributed to the presentation of results. GH wrote the initial draft of the manuscript. All authors have critically reviewed the iterations of the manuscript and approved it for publication. The team of authors listed were involved in the development of the app and responsible for planning and conducting the evaluation.
None declared.