This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
The Carrot Rewards app was developed as part of an innovative public-private partnership to reward Canadians with loyalty points, exchangeable for retail goods, travel rewards, and groceries for engaging in healthy behaviors such as walking.
This study examined whether a multicomponent intervention including goal setting, graded tasks, biofeedback, and very small incentives tied to daily step goal achievement (assessed by built-in smartphone accelerometers) could increase physical activity in two Canadian provinces, British Columbia (BC) and Newfoundland and Labrador (NL).
This 12-week, quasi-experimental (single group pre-post) study included 78,882 participants; 44.39% (35,014/78,882) enrolled in the Carrot Rewards “Steps” walking program during the recruitment period (June 13–July 10, 2016). During the 2-week baseline (or “run-in”) period, we calculated participants’ mean steps per day. Thereafter, participants earned incentives in the form of loyalty points (worth Can $0.04 ) every day they reached their personalized daily step goal (ie, baseline mean+1000 steps=first daily step goal level). Participants earned additional points (Can $0.40) for meeting their step goal 10+ nonconsecutive times in a 14-day period (called a “Step Up Challenge”). Participants could earn up to Can $5.00 during the 12-week evaluation period. Upon meeting the 10-day contingency, participants could increase their daily goal by 500 steps, aiming to gradually increase the daily step number by 3000. Only participants with ≥5 valid days (days with step counts: 1000-40,000) during the baseline period were included in the analysis (n=32,229).The primary study outcome was mean steps per day (by week), analyzed using linear mixed-effects models.
The mean age of 32,229 participants with valid baseline data was 33.7 (SD 11.6) years; 66.11% (21,306/32,229) were female. The mean daily step count at baseline was 6511.22. Over half of users (16,336/32,229, 50.69%) were categorized as “physically inactive,” accumulating <5000 daily steps at baseline. Results from mixed-effects models revealed statistically significant increases in mean daily step counts when comparing baseline with each study week (
Providing very small but immediate rewards for personalized daily step goal achievement as part of a multicomponent intervention increased daily step counts on a population scale, especially for physically inactive individuals and individuals who engaged more with the walking program. Positive effects in both BC and NL provide evidence of replicability.
The health benefits of regular physical activity are unquestionable. Regular moderate-intensity physical activity, brisk walking, for example, reduces the risk of several noncommunicable diseases, such as type 2 diabetes [
Behavioral economics, a branch of economics complimented by insights from psychology [
Evidence supporting the use of financial health incentives is growing, with 2 systematic reviews [
In some cases, however, offering incentives for longer periods may be suitable, as Finkelsetin et al (2016) suggest—until a time when physical activity motives are internalized (“I walk because it makes me feel good”) or until clinically meaningful health outcomes are achieved [
To reduce the cost of incentives and realistically operate within fixed government or insurer budgets, several incentive program features or reinforcement properties can be manipulated (eg, size, immediacy, probability, timing, type of incentive) [
In Canada, such features are now available via the Carrot Rewards app, a new mHealth initiative that rewards Canadians with loyalty points (eg, retail goods, travel, groceries) to engage in healthy behaviors (eg, visiting flu shot clinic, walking) [
Carrot Insights Inc. is a private company that developed the free Carrot Rewards app with support from the Public Health Agency of Canada. The British Columbia (BC) Ministry of Health was the company’s founding provincial Ministry partner. Newfoundland and Labrador (NL) was the second Canadian province to offer the app to its residents. Carrot Rewards was made available for BC and NL residents on the Apple iTunes and Google Play app stores on March 3 and June 13, 2016, respectively, in both English and French (Canada’s official languages). Upon downloading the app, the users were asked to enter their age, gender, postal code, and loyalty program card number to complete registration (users without loyalty cards were directed to an easy sign-up page). To register successfully, users must have entered a valid BC or NL postal code and have been ≥13 years (age cutoff of participating loyalty programs). The walking program was not initially available in BC, but was introduced the day the app launched in NL. Carrot Insights Inc. partnered with 4 major Canadian loyalty programs to offer a variety of popular incentives (ie, points could be redeemed for groceries, travel, movies, or gas). While BC users could earn points via any of the 4 participating loyalty programs, NL users could earn points only for the 2 loyalty programs with a regional presence (ie, movies and travel). In addition to the 4 participating loyalty programs, Carrot Insights Inc. also partnered with 4 Canadian health charities (ie, Heart and Stroke Foundation of Canada, Diabetes Canada, Young Men’s Christian Association Canada, and the BC Healthy Living Alliance), primarily for the purpose of reviewing and approving health education content offered in the app. The Behavioural Research Ethics Board of the University of British Columbia approved this study (UBC BREB Number H17-02814).
The marketing assets of the 4 loyalty programs and 1 charity partner were leveraged so that in the first few weeks, partners could heavily promote the app in both provinces (ie, in BC, partners sent 1.64 million emails to their loyalty members; in NL, the number of emails is unknown). The users were not automatically enrolled in the walking program, but were rather asked to opt-in. Study recruitment was open for approximately 1 month from June 13 to July 10, 2016. To participate, users had to agree to allow the app to access step data tracked and stored in their smartphones and were rewarded Can $0.60 in loyalty points for doing so.
Registered users from BC (n=65,414) and NL (n=13,468) were eligible to participate in the walking program. However, only those with iPhone version 5S or higher could participate (ie, the Health Kit app, step data aggregator, is supported and preinstalled on these devices). Android smartphone users could also participate, but they were required to download the Health Kit equivalent (ie, Google Fit app) first. Only those who enabled the walking program on their smartphones (ie, allowed the app to access their data) received the intervention. From June 13 to July 10, 2016, 78,882 users from two Canadian provinces (BC and NL) were eligible to participate in the walking program, and 44.39% (35,014/78,882) ultimately activated it on their smartphones during the recruitment period. To examine the effect of this multicomponent intervention on objectively measured daily step counts, a 12-week quasi-experimental (single group pre-post) study design was employed. Testing the walking program simultaneously in 2 provinces provided a direct replication condition.
This intervention was theoretically based on principles from behavioral economics and self-determination theory. While behavioral economics describes how incentives exploit “present bias” to
For a personalized walking goal to be generated (ie, steps per day), users must have accumulated at least 5 valid days during the initial 14-day baseline or “run-in” period. A valid day was defined as any day with step counts from 1000 to 40,000, as these numbers were considered reasonable, not outliers [
After the 14-day baseline period, users could begin to earn incentives for reaching or exceeding their individualized daily step goals; a progress wheel illustrated progress for the day (see
The primary outcome variable was mean daily step counts as measured by either built-in smartphone accelerometers, for example, iPhone 5S or higher for 53.63% (42,304/78,882) of users, Android devices for 37.48% (29,565/78,882) of users, or any Fitbit device for 7.18% (5664/78,882) of users. Recent validation studies found that the iPhone step counting feature (version 6 or newer), as well as those for Android smartphones (eg, HTC, Motorola) and Fitbit trackers (eg, hip-worn Zip, wrist-worn Flex) were accurate in laboratory and field conditions [
Carrot Rewards app’s “Steps” walking program screenshots.
The majority of demographic variables used to describe the study sample were self-reported (eg, age, gender, province). Median personal income was inferred by linking user postal codes with census data (ie, 2011 National Household Survey) at the local health area level (89) in BC and regional health authority level (4) in NL.
Three different analytical approaches were used to account for missing data and to test the sensitivity of our assumptions with the analytical sample: (1) The “any” data approach included participants with valid baseline data (≥5 days in acceptable range during the 14-day baseline period) and at least 1 other valid week (ie, at least 4 valid days in a 7-day week) from study week 1 to 12 (32,229/35,019, 92.03% of those enabling the walking program met these criteria); (2) the “completer” approach included just participants with valid data at baseline and study week 12 (19,964/32,229, 61.94%); and (3) the “imputed” approach included participants with valid baseline data, but no valid data at study week 12 (29,261/32,229, 90.79%). Then, we imputed participants’ “Pseudo study week 12” by carrying forward their baseline values. Therefore, among those included in the analysis (n=32,229), 61.94% (19,964/32,229) had complete datasets (completers). No differences were observed in demographic characteristics between completers and noncompleters (see
Statistical analysis was performed using R 3.3.0.68 Mavericks build (7202) Rstudio Version 1.0.136 (RStudio, Boston, MA, USA). Study week was treated as a categorical variable (baseline=0, study week 1=1, ..., study week 12=12) to allow for the nonlinear trajectory of daily step counts. Also, the estimate for each study week helped refine the program to maintain user engagement. Mixed-effects models were performed to examine whether there were significant changes in mean daily step counts between baseline and study week 12. We fitted a simple linear mixed-effects model that included study week as the independent variable (baseline data were used as the reference), followed by an adjusted model with random intercepts to account for measurements nesting within individuals and by controlling for age, gender, median personal income, and province as covariates. Analyses were performed on the entire sample, and participants were stratified by physical activity status as defined by Tudor-Locke et al [
As suggested by previous studies [
Baseline characteristics of Carrot Rewards users, by completion status, and for the general Canadian population.
Characteristics | Completersa |
Noncompletersb |
Canadian population |
Age in years, mean (SD) | 33.8 (11.4) | 33.5 (11.9) | 40.6 (median) |
Gender (% female) | 66.1 | 66.1 | 50.4 |
Province (% British Columbia) | 72.1 | 70.3 | 13.2 |
Median personal income (Can $1000/year), mean (SD) | 29.7 (4.1) | 29.6 (4.0) | 33.9 |
Steps per day, baseline mean (SD) | 6665.6 (4220.7) | 6157.5 (4388.9) | N/Ac |
Engagementd (% high) | 59.4 | 19.8 | N/A |
aParticipants with valid data at baseline and study week 12.
bParticipants with valid data at baseline, but not at study week 12.
cN/A: not applicable.
dA variable dichotomizing participants into 2 categories, “high” or “low” engagers, based on the median percentage of days when a “Step Up Challenge” was accepted.
The mean age of the 32,229 participants with valid baseline data was 33.7 (SD 11.6) years; 66.11% (21,306/32,229) were female (
The trends of daily step counts for the total group and the physically inactive subgroup over the 12-week intervention period are illustrated in
Least-square means for daily steps at baseline and for each study week during the 12-week evaluation period for the total sample and physically inactive participants.
Changes in mean daily step counts between baseline and study week 12.
Analysis | Baseline least-square meansa (95% CIs) | Week 12 least-square meansa (95% CIs) | Differences (Week 12 – baseline) least-square meansa (95% CIs) | Cohen f2b | ||
Total sample analysis | 6511.22 (6242.24 to 6780.19) | 6626.92 (6357.34 to 6896.50) | 115.70 (74.59 to 156.81) | 0.0059 | ||
Physically inactive | 3760.64 (3543.31 to 3977.96) | 4634.83 (4416.56 to 4853.09) | 874.19 (827.98 to 920.40) | 0.0234 | ||
Physically active | 8778.01 (8392.20 to 9163.81) | 8297.19 (7910.38 to 8684.00) | −480.82 (−545.17 to −416.46) | 0.0073 | ||
British Columbia | 7064.82 (6796.12 to 7333.52) | 7282.83 (7013.40 to 7552.26) | 218.01 (169.56 to 266.46) | 0.0061 | ||
Newfoundland and Labrador | 6071.87 (5790.32 to 6353.43) | 5938.22 (5654.07 to 6222.37) | −133.66c (−155.98 to −3.37) | 0.0087 | ||
Low engager | 6229.27 (5958.75 to 6497.80) | 5738.52 (5466.67 to 6010.37) | −490.75 (−551.21 to −428.29 | 0.0073 | ||
High engager | 6780.37 (6509.93 to 7050.81) | 7411.27 (7140.55 to 7681.99) | 630.90 (575.43 to 686.36) | N/Ad | ||
Physically inactive, low engager | 3650.63 (3432.54 to 3868.72) | 4132.55 (3 911.07 to 4354.04) | 481.92 (414.62 to 549.22) | 0.0055 | ||
Physically inactive, high engager | 3838.79 (3628.93 to 4068.65) | 5073.45 (4853.31 to 5293.59) | 1224.66 (1160.69 to 1288.63) | N/A | ||
Physically active, low engager | 8588.00 (8200.39 to 8975.61) | 7258.26 (6866.58 to 7649.93) | −1329.74 (−1427.93 to −1231.56) | 0.0096 | ||
Physically active, high engager | 8928.63 (8540.95 to 9316.31) | 9131.09 (8742.81 to 9519.36) | 202.26 (117.20 to 287.72) | N/A | ||
British Columbia, low engager | 6771.26 (6500.00 to 7042.52) | 6353.11 (6078.918 to 6627.31) | −418.15 (−491.12 to −345.17) | 0.0071 | ||
British Columbia, high engager | 7316.68 (7044.75 to 7588.61) | 8055.38 (7783.15 to 8327.61) | 738.70 (673.81 to 803.54) | N/A | ||
Newfoundland and Labrador, low engager | 5769.62 (5480.24 to 6059.00) | 5120.22 (4822.23 to 5418.21) | −649.40 (−763.50 to −535.30) | 0.0074 | ||
Newfoundland and Labrador, high engager | 6369.92 (6072.63 to 6667.21) | 6715.92 (6417.20 to 7014.64) | 346.00 (239.26 to 452.74) | N/A |
aLeast-square means adjusted for age, median personal income, gender, and province.
bCohen f2≥0.02, ≥0.15, and ≥0.35 representing small, medium, and large effect sizes, respectively. For the engagement subgroup analysis only, Cohen f2 was calculated for the pre-post difference in steps between the low and high engagement groups (high engagement as the referent group).
cThe difference between baseline and week 12 were statistically significant at
dN/A: not applicable.
The results from mixed-effects models revealed statistically significant increases in mean daily step counts when comparing baseline with each study week (
The intervention effect was more pronounced in physically inactive users than in physically active users. As with the total sample analysis, the mean daily steps were significantly higher for physically inactive users at each study week than at baseline (
Participant engagement showed a significant moderating effect on the intervention outcome in all models (
In this large quasi-experimental study examining the impact of a multicomponent intervention on objectively measured daily step count, a small but significant effect overall was observed (5% average daily step count increase over 12 weeks vs baseline) with a more pronounced effect (21% increase) among physically inactive users (representing over half of the total sample). Notably, this effect was evident irrespective of age, gender, or median personal income. While the overall effect was small (ie, 116 steps per day), these results underscore the potential public health impact of using modest incentives (Can $ 0.04 per day) to stimulate physical activity, particularly among higher risk, physically inactive populations. When considering the clinical significance of this study’s results, it is likely that health benefits (eg, better glucose control) [
Other reinforcement-based methods of increasing health behaviors have included using deposit contracts (ie, participants wager their own money) [
Behavioral decay (ie, steps per day decline) was noted as time passed, with weekly steps per day averages dropping below the intervention mean in later weeks. While this was observed in the total sample (driven by the 480.82 daily step count
Regarding provincial differences, NL users did not respond as well as BC users (−133.66 steps per day vs +218.01 steps per day at study week 12, respectively). This could be due to a number of factors. The most important factor may have to do with the walking program’s availability to
The results of this population-level study should be interpreted with caution because there are a number of limitations to consider. First, neither the randomization of participants into intervention and control groups was logistically feasible within this quasi-experimental design nor was the identification of a nonequivalent control group (ie, a group not randomly assigned to receive or not receive the intervention) [
To increase internal validity in this quasi-experimental environment, future studies might incorporate interrupted time series, stepped-wedge, intervention removal, or designs with a nonequivalent control group [
Until recently, financial health incentive programs have shown promise, but little potential for scalability given rewards’ cost. This study adds to the understanding of how incentives can be delivered in ways that are not prohibitively costly. Providing immediate rewards for personalized daily step goal achievement as part of a multicomponent intervention appears to have increased daily step counts on a population scale, especially for higher risk, physically inactive individuals. Positive effects in both BC and NL provide evidence of replicability.
British Columbia
Newfoundland and Labrador
randomized controlled trials
The Carrot Rewards initiative has been made possible in part through funding from the Public Health Agency of Canada. The views expressed herein do not necessarily represent the views of the Public Health Agency of Canada, the BC Ministry of Health, or the Ministry of Children, Seniors, and Social Development in NL. The authors also thank staff from the BC Ministry of Heath, the Ministry of Children, Seniors, and Social Development in NL, the BC Healthy Living Alliance, the Heart and Stroke Foundation of Canada, Diabetes Canada, and Young Men’s Christian Association Canada for providing expert health content advice over the course of this project.
MM reports grant support from the Canadian Institutes of Health Research, the University Health Network, Green Shield Canada Inc, as well as in-kind research support from Cookson James Loyalty Inc Furthermore, he reports consulting income from Carrot Insights Inc and stock options in Carrot Insights Inc. LW is a Carrot Insights Inc employee and also reports stock options in Carrot Insights Inc. GF is supported by a Canadian Institutes of Health Research-Public Health Agency of Canada Chair in Applied Public Health. The authors with no financial relation to Carrot Insights Inc conducted the analyses (GF, EL). The other coauthors report no conflicts of interest.