Published on 28.03.19 in Vol 7, No 3 (2019): March
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/12181, first published Sep 12, 2018.
Efficacy of a Mobile Social Networking Intervention in Promoting Physical Activity: Quasi-Experimental Study
Background: Technological interventions such as mobile apps, Web-based social networks, and wearable trackers have the potential to influence physical activity; yet, only a few studies have examined the efficacy of an intervention bundle combining these different technologies.
Objective: This study aimed to pilot test an intervention composed of a social networking mobile app, connected with a wearable tracker, and investigate its efficacy in improving physical activity, as well as explore participant engagement and the usability of the app.
Methods: This was a pre-post quasi-experimental study with 1 arm, where participants were subjected to the intervention for a 6-month period. The primary outcome measure was the difference in daily step count between baseline and 6 months. Secondary outcome measures included engagement with the intervention and system usability. Descriptive and inferential statistical tests were conducted; posthoc subgroup analyses were carried out for participants with different levels of steps at baseline, app usage, and social features usage.
Results: A total of 55 participants were enrolled in the study; the mean age was 23.6 years and 28 (51%) were female. There was a nonstatistically significant increase in the average daily step count between baseline and 6 months (mean change=14.5 steps/day, P=.98, 95% CI –1136.5 to 1107.5). Subgroup analysis comparing the higher and lower physical activity groups at baseline showed that the latter had a statistically significantly higher increase in their daily step count (group difference in mean change from baseline to 6 months=3025 steps per day, P=.008, 95% CI 837.9-5211.8). At 6 months, the retention rate was 82% (45/55); app usage decreased over time. The mean system usability score was 60.1 (SD 19.2).
Conclusions: This study showed the preliminary efficacy of a mobile social networking intervention, integrated with a wearable tracker to promote physical activity, particularly for less physically active subgroups of the population. Future research should explore how to address challenges faced by physically inactive people to provide tailored advices. In addition, users’ perspectives should be explored to shed light on factors that might influence their engagement with the intervention.
JMIR Mhealth Uhealth 2019;7(3):e12181
There is strong evidence of the effectiveness of regular physical activity in the prevention of several chronic diseases and associated premature death [, ]. Furthermore, there appears to be a dose-response relationship between physical activity and health status [ , ]. Yet, despite the importance of physical activity, 27.5% of adults worldwide are insufficiently active [ ], highlighting the need for interventions to promote physical activity.
Behavioral informatics interventions (ie, using health information technology to facilitate behavior change) have become increasingly popular in recent years . A key element to behavior change success is the use of behavior change theories, models and techniques to better understand the causal mechanisms and influencing factors of the behavior, and the context of the intervention [ ]. In addition, in recent years, researchers have encouraged intervention developers to describe their interventions in terms of the specific behavior change techniques [ ]. A behavior change technique is an observable, replicable, and irreducible component of an intervention, intended to alter causal processes that regulate behavior [ ]. Behavior change techniques can be linked to existing theories and models, and they provide a more transparent, replicable approach to the design and evaluation of behavior change interventions [ , ].
To date, several behavior change theories and models have indicated the importance of the link between social factors and health-related behaviors [- ]. In particular, researchers have demonstrated that existing networks of friends and family exert great influence on individual health behavior [ , ], suggesting the potential of leveraging social networks to deliver physical activity interventions [ ]. Social networks refer to the webs of an individual’s relationships, which give rise to various functions such as social influence, social companionship, social support, and social comparison [ ]. To date, several studies have found strong evidence that behavior change techniques such as social support and social comparison increase physical activity levels [ - ]. Though these interventions seem promising, their potential can be missed when they are not easily disseminated or accessible to a large audience [ ]. A potentially useful way to disseminate social network interventions for physical activity is through the use of Web-based social networks. Web-based social networks, which are now ubiquitous in our lives, allow users to create a personal profile and connect with other users [ ]. Several meta-analyses have found that online social networks can have positive, significant effects on behavior change [ , ].
In addition to social aspects, many studies have also highlighted the importance of other behavior change techniques, such as self-monitoring or goal setting, in physical activity [, ]. Mobile health (mHealth) technologies such as mobile apps and wearable trackers offer new opportunities to deliver these behavior change techniques. In particular, recent mHealth technologies can reach individuals continuously, allowing users to self-monitor their physical activity [ ] and providing real-time feedback [ ]. mHealth interventions have increasingly been used in physical activity interventions, reporting significant, moderate improvements in step counts [ - ]. Given their potential, interventions combining mHealth technologies and online social networks might be particularly effective in promoting physical activity.
To date, researchers have largely examined the effects of mHealth and Web-based social networks on physical activity in isolation [- ]. There are a few studies that evaluated the feasibility and effectiveness of interventions with both mHealth and Web-based social network components, showing user acceptability and moderate increases in physical activity levels [ - ]. However, these studies often examine online social networks as an additional feature (eg, a Facebook group), not integrated within a mobile app. In addition, it is also essential to examine usage metrics and usability determinants of mHealth interventions, as these factors reflect true user engagement and can largely influence the effects of the intervention [ ].
The aim of this study was to pilot test a social networking mobile app, connected with a wearable tracker to promote physical activity. In particular, we investigated (1) the intervention efficacy on physical activity and (2) participant engagement and usability of the intervention. The secondary aims were to explore the effects of social features on physical activity levels and the association between engagement with the mobile app and physical activity levels.
This study is part of a larger mixed-methods feasibility study on the use of a social networking mobile app to promote physical activity and weight management . In particular, this paper reports on the quantitative results related to the physical activity outcomes of a pre-post, 1-arm quasi-experiment where participants were subjected to the intervention for a 6-month period. Results related to weight outcomes of the study will be reported in a forthcoming publication. The design and conduct adhered to the Consolidated Standards of Reporting Trials 2010 statement—extension to randomized pilot and feasibility trials, where applicable [ ].
Ethics approval was granted by Macquarie University’s Human Research Ethics Committee for Medical Sciences (ethics reference number 5201600716).
Study Settings and Participants
A total of 55 participants (mean age 23.6 years, 51% [28/55] female), mostly Macquarie University students and staff (Sydney, Australia), were recruited using purposive sampling techniques. Given the nature of this study, the sample size was pragmatically chosen to enable a comprehensive assessment of the feasibility of the intervention before conducting a randomized controlled trial (RCT) . Recruitment channels included posters around university campus, website information, and Facebook. Eligible participants were healthy adults with sufficient knowledge of English to understand and participate in the study, they had planned to be living in Sydney for the duration of the study and owned a mobile phone (iOS or Android) with internet access. Exclusion criteria were pregnancy, body mass index (BMI) below 17, earlier history of eating disorders, or having diabetes or other comorbid conditions that could impact study participation (eg, severe mental illness, end-stage disease). Participants were screened for eligibility via an online questionnaire.
Eligible participants were invited to attend the initial study session at the research center, where they received information about the purpose of the study and signed the consent form. Subsequently, participants filled in a questionnaire about their demographic characteristics and smartphone usage (eg, type of smartphone used, hours per day using the smartphone), and their baseline measurements (ie, weight, height) were assessed. At the end of the study, participants were invited to attend a postintervention session in which they completed the System Usability Scale (SUS) survey , and their weight was measured again.
The intervention bundle involved 3 components, including a mobile app (named fit.healthy.me), a wearable tracker, and short message service (SMS) text messages and emails. In particular, the fit.healthy.me app was developed on the basis of several behavior change techniques, such as self-monitoring of physical activity, social support, and social comparison. In the app, the social features were composed of My team, Social forum, and Private messages. My team allowed participants to visualize and compare their step counts with others and follow other people, whereas Social forum and Private messages allowed participants to interact and provide social support to each other.
To enable the automation of self-monitoring, the fit.healthy.me app was integrated with the Fitbit Flex 2 wearable tracker . In particular, the Fitbit Flex 2 was wirelessly synced with fit.healthy.me (via the Fitbit app programming interface). Fitbit Flex 2 uses accelerometer technology to measure acceleration signals, which are then converted to step count—a common indicator of physical activity. Research has demonstrated good reliability and validity in using Fitbit Flex 2 for measuring step count in free-living conditions [ , ].
In addition, prompts and cues (ie, SMS text messages and emails) were sent every 2 weeks to remind users to wear the fitness tracker during waking hours and check fit.healthy.me at least once every day. A detailed description of the modes of delivery and features of the intervention is presented in. Screenshots of the mobile app are provided in .
Before the study commencement, the fit.healthy.me app underwent development testing  within the research center. Participants were provided access to the intervention by downloading the app from the Apple app Store or Google Play. During the study, participants could email or call the study team if they required any technical assistance. A research team member with clinical expertise also regularly monitored the study and responded to any concerns raised by participants. As an incentive for participation in the study, individuals were offered to keep the tracker at the end of the 6-month period.
This paper specifically reports on 3 aspects of the study results: (1) the efficacy of the intervention on physical activity measures, (2) participant engagement with the intervention, and (3) the usability of the fit.healthy.me app.
Efficacy in Promoting Physical Activity
The primary outcome measure for this study was the difference in the daily step count between baseline and 6 months, which was measured using the Fitbit Flex 2 and retrieved via the Fitbit app programming interface. To enable the collection of baseline daily step count, participants underwent a 7-day period after the initial study session where they were not able to log in to fit.healthy.me but were asked to use the Fitbit Flex 2 every day; the baseline measure was obtained by averaging the number of steps per day the first 7 days. The final step count was determined by computing the average number of steps per day on the last week where participants had at least four valid days . A valid day of step count was defined as at least 10 hours of wear time during that day ( ) [ ]. The wear time was calculated by subtracting nonwear time from 24 hours; nonwear time was defined if no step counts were detected over a period of at least 60 continuous min, allowing for 2 min of counts between 0 and 100 [ , ].
Posthoc subgroup analysis was carried out for participants with different physical activity levels at baseline (≥10,000 steps per day vs <10,000 steps per day). A total of 10,000 steps per day were used as a threshold, as this goal is acknowledged as a reasonable target for healthy adults [- ].
Participant engagement with the intervention was assessed using multiple measures (). In particular, retention was defined as attendance at the 6-month final session. Participants who came to the final sessions were considered completers and participants who did not come were considered to have dropped out of the study. For the Fitbit Flex 2, engagement was measured by the mean number of days a valid step count was logged (participants were considered to have a valid step count if they wore the Fitbit for at least ten hours on any given day). For the fit.healthy.me app, engagement was measured by both the length of usage (ie, the mean number of days of usage) and frequency of usage (ie, the number of times participants used the app and each feature). A participant was considered to have used the app in a day if he or she used any features of the app at any time of that day. Similarly, a participant was considered to have used a social feature if he or she clicked on any of My team, Social forum, and Private messages features at any time. Every time a participant used an app feature, the timestamp and the name of that feature were automatically saved into our local database. These data were summarized to show participant engagement with the fit.healthy.me app at the end of the study.
|Modes of delivery||Features||Behavior change techniquesa|
|fit.healthy.me app||My measures||Self-monitoring of behavior (ie, number of steps per day)|
|My team||Social comparison|
|Social forum||Social support (emotional); Social comparison|
|Private messages||Social support (emotional); Social comparison|
|My journey||Instruction on how to perform the behavior|
|Fitbit Flex 2||Fitness wearable tracker||Self-monitoring of behavior (ie, physical activity)|
|SMSb text messages and emails||Reminders||Prompts/cues|
aClassified according to the behavior change techniques taxonomy developed by Michie et al .
bSMS: short message service.
|Completers||Participants who came to the final sessions|
|Noncompleters||Participants who did not come to the final sessions (dropout attrition)|
|Retention rate||Percentage of completers out of all 55 participants|
|Fit.healthy.me app usage|
|Length of usage||The mean number of days of usage|
|Frequency of usage||The mean number of times participants used the app and each feature|
|Nonusage attrition||Participants who did not use the app at all in the last month of the study|
|Fitbit Flex 2 tracker usage|
|Length of usage||The mean number of days a valid step count was logged|
|A valid day of step count||Having at least ten hours of wear time|
|Wear time||Calculated by subtracting nonwear time from 24 hours|
|Nonwear time||Defined if no step counts were detected over a period of at least sixty continuous minutes, allowing for 2 min of counts between 0 and 100 [, ]|
aAdapted from Eysenbach (2005) .
Participants completed the SUS  to assess the usability of the fit.healthy.me app. The SUS is a validated questionnaire comprising a standard set of 10 statements that seek users’ opinions on the usability of a system [ ]. SUS has been widely used to evaluate usability within commercial and research studies (including mobile apps) for over 30 years [ - ]. Participants were asked to rank the statements on a 5-point Likert scale from strongly disagree (scored as 1) to strongly agree (scored as 5). Final scores of the SUS can range from 0 to 100, with higher scores indicating better usability [ ]. A study collecting 10-years’ worth of SUS data from over 200 studies found that the average score is around 70, suggesting that a SUS score of 70 might be considered acceptable [ ]. A list of the statements and explanation for calculation of the SUS scores is provided in .
Participants’ demographic characteristics, intervention usage data, and engagement metrics were analyzed descriptively using means, SD, and frequency counts. Wilcoxon signed-rank test was used to determine whether the number of days participants used the fit.healthy.me app differed between the first and last (sixth) month of the study. SUS score was calculated to determine the usability of the fit.healthy.me app .
To investigate the efficacy of the intervention, the difference between average step count at baseline and final weeks was assessed using a paired, 2-tailed t test. A total of 3 participants did not have valid data for at least four days at the end of the study, and thus they were excluded from the analysis. Kendall tau-b test was used to measure the correlation between total engagement with the fit.healthy.me app and changes in daily step count.
Posthoc subgroup analyses were carried out for participants with different levels of steps at baseline, app usage, and social features usage. As mentioned above, in terms of physical activity, 10,000 steps per day were used as a cut-off point to define high- versus low-level physical activity [- ]. In terms of app usage and social features usage, the median was used as a cut-off point to determine frequent versus nonfrequent usage. Independent 2-sample t tests were used for normally distributed numerical data; for nonnormal data, the Wilcoxon rank-sum test was used. Chi-square tests were used for categorical data. For statistically significant results, effect sizes (ie, Cohen d) were calculated [ ].
Data were analyzed using R version 3.5.0 (R Foundation for Statistical Computing) [- ]. The significance level for all statistical tests was set at P<.05, 2-tailed, and 95% CIs were calculated where applicable.
Participant Flow and Recruitment
Recruitment occurred from April to May 2017. A total of 423 people completed an online questionnaire to assess their eligibility; 55 of them met the eligibility criteria, consented to participate, and attended the preintervention session. The most common reasons for ineligibility were pregnancy and chronic diseases. After each participant completed the 6-month period, they were sent an automatic email, inviting them back for the final sessions. Out of 55 initial participants, 45 participants returned for the final session (ie, completers). Step data were collected for all 55 participants during the 6-month intervention period. Given our definition of valid days and the condition that at least four valid days were needed to compute the weekly average, not all participants had the final step count in week 26 (median final week number: 21; interquartile range: 10-25).
A summary of the differences in baseline characteristics between enrolled participants and completers is presented in. At baseline, participants had a mean age of 23.6 years (SD 4.6). Furthermore, 28 (51%) were female, and 42 (76%) were university students. The average BMI was 26.5 kg/m2 (SD 6.8), with nearly half of the participants (24/55, 44%) in the normal weight range. Participants reported using a smartphone for 5.6 hours (SD 3.4) per day, on average; most users (36/55, 66%) had an iPhone. The majority of participants (49/55, 89%) said that the most used apps in their phones were social media apps, whereas 10% (6/55) said fitness apps. There were no statistically significant differences between enrolled participants and completers.
Physical Activity Measures
On average, daily step count did not change between baseline and 6 months (mean difference=14.5, P=.98, 95% CI –1136.5 to 1107.5). A subgroup analysis comparing the higher physical activity group with the lower physical activity group (at baseline) showed that the lower physical activity group experienced a statistically significant increase of 3025 steps in daily step count between baseline and post intervention (P=.008, 95% CI 837.9-5211.8], Cohen d=0.80;and ). shows box plots for participants’ daily step count at each week of the study. There were no statistically significant changes in average daily step count between different levels of app usage (P=.42; ) or different levels of social feature usage (P=.25; ). Total engagement with the fit.healthy.me was not directly associated with change in daily step counts (Kendall tau-b=–0.11, P=.25).
|Measures||Enrolled participants (n=55)||Study completers (n=45)||P value|
|Age, mean (SD)||23.6 (4.6)||24.2 (4.7)||.51a|
|Female, n (%)||28 (51)||22 (50)||.52b|
|Weight (kg), mean (SD)||78.1 (22.3)||77.8 (21.2)||.99a|
|Body mass index (kg/m2), mean (SD)||26.6 (6.8)||26.7 (6.5)||.94a|
|Body mass index categoriesc, n (%)|
|18-18.49||3 (6)||1 (2)||.14b|
|18.5-24.99||24 (44)||22 (49)||.19b|
|25-29.99||15 (27)||10 (22)||.16b|
|≥30||13 (24)||12 (27)||.48b|
|Steps/day, mean (SD)||10,967.2 (3907.4)||10,896.3 (4206.2)||.93a|
aAssessed using 2-sample t tests.
bAssessed using chi-square tests.
cAccording to the World Health Organization, a body mass index of less than 18.5 is classified as underweight, 18.5-24.9 is normal, 25-29.9 is preobese, and ≥30 is obese .
|Measures||<10,000 steps/day (n>=20), mean (SD)||≥10,000 steps/day (n=35), mean (SD)||P value (95% CI)|
|Baseline weight (kg)||77.0 (26.3)||78.6 (20.1)||.80a (–14.3 to 11.0)|
|Baseline body mass index (kg/m2)||26.4 (7.8)||26.6 (6.2)||.91a (–4.1 to 3.6)|
|Duration of app usage (days)||16.1 (15.3)||15.4 (17.0)||.51b (–4.0 to 7.0)|
|Intensity of app usage (times)||1487.0 (1244.7)||1719.1 (1561.6)||.79b (–559 to 860)|
|Pre-post intervention step difference||1992.3 (3598.3)||–1032.6 (3894.7)||.008c (837.9-5211.8)|
aAssessed using 2-sample t test.
bAssessed using Wilcoxon rank-sum test.
cdenotes statistical significance.
Participant Retention and Engagement
The retention rate was 82%. Overall, the length of usage of the Fitbit Flex 2 tracker was higher than app and social features. My Team and My Measures had a higher level of engagement compared with Social Forum and Private Messages (). In general, app usage decreased over time ( ). In particular, the number of days participants used the app in the last month of the study significantly decreased from the first month of the study (P<.001, 95% CI –5.5 to –4). In total, 4 participants did not use the app at all throughout the study. Subgroup analyses showed that there were no statistically significant differences in any characteristics between frequent and nonfrequent app users ( ).
System Usability Scale
Out of 55 participants, only 45 returned to the postintervention sessions and completed the SUS. The mean SUS score was 60.1 (SD 19.2). Two-third of the participants (N=30) gave a SUS score lower than 70, indicating low usability . Furthermore, 7 participants rated the app’s usability as moderate and 8 participants rated it as having high usability. presents responses to individual SUS statements. Posthoc subgroup analysis indicated that frequent app users gave a higher SUS score than nonfrequent users (P=.04, 95% CI 0.6-25.3; ).
|Engagement measures and usage data||Mean (SD)||Range|
|Fitbit Flex 2 usage|
|Days valid step count were logged via Fitbit (days)||66 (48.7)||5-183|
|Length (days)||15.7 (16.2)||0-63|
|Frequency (times)||1634.7 (1446.8)||0-6317|
|App features usage|
|My measures||44.2 (47.8)||0-228|
|My teama||59.0 (51.6)||0-203|
|Social foruma||21.8 (37.5)||0-213|
|Private messagesa||9.2 (20.8)||0-88|
|My journey||17.0 (13.0)||0-63|
aSocial features included My team, Social forum, and Private messages.
There was a nonstatistically significant increase in the average daily step count between baseline and 6 months. Subgroup analysis comparing the higher and lower physical activity groups at baseline showed that the latter experienced a statistically significant increase in average daily step count between baseline and postintervention, suggesting the app might be more beneficial for specific subgroups of the population (eg, less physically inactive individuals). At 6 months, the retention rate was 82%; 42% participants used the fit.healthy.me app at least once during the last month of the study.
To the best of our knowledge, our study is the first to evaluate a mobile social networking intervention integrated with a wearable tracker. Other studies have examined interventions composed of either mobile technologies [- ] or online social networks [ - ] in isolation, and thus evidence on the efficacy and feasibility of an intervention combining both was limited until now. Even though several studies have incorporated social features in mHealth interventions, these features were often included as an additional component (eg, Facebook group) rather than being fully integrated with the mobile app [ , , , , , ].
Efficacy in Promoting Physical Activity
Our study found that compared with the higher physical activity group, the lower physical activity group at baseline experienced a significant increase of 3025 steps in daily step count, suggesting that specific populations (eg, less physically active people) might benefit more from the use of a mobile social networking app. Earlier research has outlined the importance of considering particular challenges and barriers that inactive people might face when designing fitness technology. For example, several studies have suggested that although self-regulation techniques (ie, goal setting, self-monitoring, and feedback on behavior) and social support are often present in fitness technology, other behavior change techniques such as action planning or environment restructuring are present less often and might be particularly useful for inactive people [, ]. It is worth noting that even increases of 2000 steps per day are associated with reduced risk of cardiovascular disease, given the dose-response relationship between physical activity levels and health benefits [ ]. Altogether, the use of behavioral informatics such as ours seem promising, and it should be confirmed by fully powered RCTs.
User Retention, Engagement, and Usability
The retention rate of our study was 82%, which is consistent with the reported retention rates of around 70% to 90% in other mHealth and online social networks interventions [, , , - ]. Our study also revealed that app usage declined over time—a phenomenon frequently observed in other apps for physical activity [ , , ]. It is known that initially, users tend to be attracted to new technologies; over time, disengagement can be triggered by either internal factors such as lack of time, or it can be triggered by external factors such as usability issues and technological problems [ ]. A possible explanation for the decline in usage of our app could be usability issues. In fact, two-third of our users gave a SUS score of lower than 70 to the fit.healthy.me app, indicating low usability [ ]; nonfrequent users were more likely to give a lower SUS score. Indeed, when a user experiences a usability flaw, the negative experience might outweigh other positive features of the technology (a phenomenon known as negativity bias) [ ], and it can subsequently lead to lower engagement. The link between usability and engagement has been frequently demonstrated in previous research [ ]. Of note, the technology acceptance model highlights the importance of perceived usefulness and perceived ease of use (concepts overlapping with many aspects of usability) [ , - ] in users’ acceptance and adoption of technology [ , ]. Hence, it is important to address usability to maximize user engagement.
We also found that usage levels varied among different features. In particular, My team attracted a significantly higher level of engagement compared with Social forum and Private messages. This difference could possibly be because of the format and content presented in each feature: My team supports social comparison via displaying summary statistics and graphs, whereas the Social forum and Private messages features support discussion among users. It can be hypothesized that users found more utility in the numerical and graphical social comparison aspects of My team to the discussion-based nature of other social features, suggesting the need to explore how to effectively deliver social behavior change techniques to maximize engagement.
Strengths and Limitations
This study has several strengths. First, we assessed a range of features supporting different behavior change techniques to examine the individual aspects of this multicomponent intervention. Second, we reported different measures of engagement, including retention rate, nonusage attrition, and engagement metrics with different intervention components to shed light on the attrition problems in behavioral informatics interventions [, ]. Finally, the intervention was fully integrated with wireless tracking devices, and thus the wireless tracking devices eliminated the reliance on self-reported data.
The findings of this study must be interpreted in light of some limitations. Given that this was a quasi-experimental study with a single-arm pre-post design, we cannot infer causation from our results. Possible confounders might have been at play, and thus the results should be interpreted with caution. Moreover, we had a purposely small and homogenous sample, which affects generalizability of the study. Another limitation is related to the handling of missing data in daily step count. Due to our definition of valid days of step count and the condition that participants needed to have at least four valid days of daily step count within a week to compute the weekly average, not all participants had the final step count in the last month of the study; therefore, we calculated the final step count on the basis of the last week where participants had at least four valid days. Although this method allowed us to include more participants in the analysis (and thus avoid selection bias resulting from excluding participants from the analysis), it can potentially bias the results in other ways (eg, overestimation of the final step count in case the daily step count decreases over the study duration). In addition, as the fit.healthy.me app was developed for research purposes, it lacked the advanced features and design aspects that would be available in commercially available fitness apps. Usability testing was assessed using the SUS and not done extensively. All posthoc subgroup analyses were exploratory and might be subject to type I error. In particular, in our analysis comparing different physical activity subgroups, our focus was on the difference between baseline and final weeks, and the analysis did not take into account all 26 weeks (). Future work exploring the time series nature of physical activity data and analyzing and modeling weekly trends might reveal more in-depth information about users’ behavioral patterns and provide more robust results. Finally, in this study, we only used step count as a measure of physical activity. Future research might consider other measures, such as intensity of physical activity (light, moderate, and vigorous) or sedentary time [ , ].
This study highlights several important implications regarding the design and implementation of behavioral informatics interventions for physical activity. First, our findings suggest that wearable devices and mobile social networking apps can work in synergy to facilitate behavior change, particularly in physically inactive groups. In particular, wearable trackers can automate self-monitoring—an important task in behavior change [, ], whereas mobile apps can provide a platform to support other relevant behavior change techniques, such as providing feedback on behavior, goal setting, or social comparison [ ]. Several studies have also suggested that social interaction can enhance engagement [ , ], highlighting the potential of integrating social features in technological interventions.
Furthermore, it is important to note that physically inactive groups might face additional challenges, and thus future research should also consider the potential of other behavior change techniques in these interventions. Perhaps fitness technology could prompt individuals to identify the particular barriers they face regarding physical activity , and it could facilitate the tailoring of specific recommendations accordingly. Tailored advices can be more helpful and relevant to users [ , ], potentially leading to more effective interventions in this subgroup of the population. In addition, future research should also explore users’ preferences and perspectives on factors that might influence their engagement, to maximize the effectiveness of mHealth interventions in promoting physical activity.
Our study showed preliminary evidence that mobile social networking interventions, integrated with wearable trackers, can help to promote physical activity. Future research needs to explore how to best support barriers faced by physically inactive people and accordingly provide tailored recommendations to maximize intervention effectiveness.
HLT was supported by the International Macquarie University Research Training Pathway Master of Research (iMQRTPMRES) Scholarship (2016302). This research was supported by a grant received from the National Health and Medical Research Council Centre of Research Excellence in Informatics and E-Health (1032664). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Study conceptualization: HLT, EC, LL. Data collection: HLT, PM. Data analysis: HLT, EC, WT, YW, JCQ, LL. First draft: HLT, LL. All authors critically revised the manuscript and approved the final version.
Data availability statement: Summary data supporting the findings are available within the paper and its supplementary information files. The raw datasets are not publicly available because of ethical restrictions.
Conflicts of Interest
EC could benefit from the commercialization of fit.healthy.me.
Multimedia Appendix 1
Screenshots of fit.healthy.me app.PDF File (Adobe PDF File), 241KB
Multimedia Appendix 2
Responses to individual System Usability Scale statements.PDF File (Adobe PDF File), 113KB
Multimedia Appendix 3
Boxplots of the 55 participants' daily step count over 26 study weeks.PDF File (Adobe PDF File), 88KB
Multimedia Appendix 4
Differences in characteristics between frequent app users and nonfrequent app users.DOCX File, 15KB
Multimedia Appendix 5
Differences in characteristics between frequent users and nonfrequent users of the social features in the fit.healthy.me app.DOCX File, 14KB
- Ding D, Lawson KD, Kolbe-Alexander TL, Finkelstein EA, Katzmarzyk PT, van Mechelen W, Lancet Physical Activity Series 2 Executive Committee. The economic burden of physical inactivity: a global analysis of major non-communicable diseases. Lancet 2016 Sep 24;388(10051):1311-1324. [CrossRef] [Medline]
- Lear SA, Hu W, Rangarajan S, Gasevic D, Leong D, Iqbal R, et al. The effect of physical activity on mortality and cardiovascular disease in 130 000 people from 17 high-income, middle-income, and low-income countries: the PURE study. Lancet 2017 Dec 16;390(10113):2643-2654. [CrossRef] [Medline]
- Kyu HH, Bachman VF, Alexander LT, Mumford JE, Afshin A, Estep K, et al. Physical activity and risk of breast cancer, colon cancer, diabetes, ischemic heart disease, and ischemic stroke events: systematic review and dose-response meta-analysis for the Global Burden of Disease Study 2013. Br Med J 2016 Aug 9;354:i3857 [FREE Full text] [CrossRef] [Medline]
- Arem H, Moore SC, Patel A, Hartge P, Berrington DGA, Visvanathan K, et al. Leisure time physical activity and mortality: a detailed pooled analysis of the dose-response relationship. JAMA Intern Med 2015 Jun;175(6):959-967 [FREE Full text] [CrossRef] [Medline]
- Guthold R, Stevens GA, Riley LM, Bull FC. Worldwide trends in insufficient physical activity from 2001 to 2016: a pooled analysis of 358 population-based surveys with 1·9 million participants. Lancet Glob Health 2018 Oct;6(10):e1077-e1086 [FREE Full text] [CrossRef] [Medline]
- Laranjo L, Lau A, Coiera E. Design and implementation of behavioral informatics interventions. In: Patel VL, Arocha JF, Ancker JS, editors. Cognitive Informatics in Health and Biomedicine: Understanding and Modeling Health Behaviors. Cham, Switzerland: Springer Publishing; 2017:13-42.
- Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95. [CrossRef] [Medline]
- Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008 May;27(3):379-387. [CrossRef] [Medline]
- Bandura A. Social Foundations Of Thought And Action: A Social Cognitive Theory. New Jersey: Prentice Hall; 1986.
- Glanz K, Rimer B, Viswanath K. Health Behavior and Health Education: Theory, Research, and Practice. New Jersey: John Wiley & Sons; 2008.
- Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011;6:42 [FREE Full text] [CrossRef] [Medline]
- Christakis NA, Fowler JH. The collective dynamics of smoking in a large social network. N Engl J Med 2008 May 22;358(21):2249-2258 [FREE Full text] [CrossRef] [Medline]
- Christakis NA, Fowler JH. The spread of obesity in a large social network over 32 years. N Engl J Med 2007 Jul 26;357(4):370-379. [CrossRef] [Medline]
- Coiera E. Social networks, social media, and social diseases. Br Med J 2013;346:f3007. [CrossRef] [Medline]
- Heaney C, Israel B. Social networks and social support. In: Glanz K, Rimer BK, Viswanath K, editors. Health Behavior and Health Education: Theory, Research, and Practice. New Jersey: John Wiley & Sons; 2008:189-207.
- Heath GW, Parra DC, Sarmiento OL, Andersen LB, Owen N, Goenka S, Lancet Physical Activity Series Working Group. Evidence-based intervention in physical activity: lessons from around the world. Lancet 2012 Jul 21;380(9838):272-281. [CrossRef] [Medline]
- McNeill LH, Kreuter MW, Subramanian SV. Social environment and physical activity: a review of concepts and evidence. Soc Sci Med 2006 Aug;63(4):1011-1022. [CrossRef] [Medline]
- Patel MS, Volpp KG, Rosin R, Bellamy SL, Small DS, Fletcher MA, et al. A randomized trial of social comparison feedback and financial incentives to increase physical activity. Am J Health Promot 2016 Jul;30(6):416-424. [CrossRef] [Medline]
- Laranjo L, Lau AY, Martin P, Tong HL, Coiera E. Use of a mobile social networking intervention for weight management: a mixed-methods study protocol. BMJ Open 2017 Jul 12;7(7):e016665 [FREE Full text] [CrossRef] [Medline]
- Boyd D, Ellison N. Social network sites: definition, history, and scholarship. J Comput Mediat Commun 2007;13(1):210-230. [CrossRef]
- Laranjo L, Arguel A, Neves AL, Gallagher AM, Kaplan R, Mortimer N, et al. The influence of social networking sites on health behavior change: a systematic review and meta-analysis. J Am Med Inform Assoc 2015 Jan;22(1):243-256. [CrossRef] [Medline]
- Maher CA, Lewis LK, Ferrar K, Marshall S, de Bourdeaudhuij I, Vandelanotte C. Are health behavior change interventions that use online social networks effective? A systematic review. J Med Internet Res 2014;16(2):e40 [FREE Full text] [CrossRef] [Medline]
- Michie S, Abraham C, Whittington C, McAteer J, Gupta S. Effective techniques in healthy eating and physical activity interventions: a meta-regression. Health Psychol 2009 Nov;28(6):690-701. [CrossRef] [Medline]
- Greaves CJ, Sheppard KE, Abraham C, Hardeman W, Roden M, Evans PH, et al. Systematic review of reviews of intervention components associated with increased effectiveness in dietary and physical activity interventions. BMC Public Health 2011;11:119 [FREE Full text] [CrossRef] [Medline]
- Evenson KR, Goto MM, Furberg RD. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int J Behav Nutr Phys Act 2015;12(1):159 [FREE Full text] [CrossRef] [Medline]
- Riley WT, Rivera DE, Atienza AA, Nilsen W, Allison SM, Mermelstein R. Health behavior models in the age of mobile interventions: are our theories up to the task? Transl Behav Med 2011 Mar;1(1):53-71 [FREE Full text] [CrossRef] [Medline]
- Fanning J, Mullen SP, McAuley E. Increasing physical activity with mobile devices: a meta-analysis. J Med Internet Res 2012;14(6):e161 [FREE Full text] [CrossRef] [Medline]
- Bort-Roig J, Gilson ND, Puig-Ribera A, Contreras RS, Trost SG. Measuring and influencing physical activity with smartphone technology: a systematic review. Sports Med 2014 May;44(5):671-686. [CrossRef] [Medline]
- Flores-Mateo G, Granado-Font E, Ferré-Grau C, Montaña-Carreras X. Mobile phone apps to promote weight loss and increase physical activity: a systematic review and meta-analysis. J Med Internet Res 2015;17(11):e253 [FREE Full text] [CrossRef] [Medline]
- Hartman SJ, Nelson SH, Weiner LS. Patterns of Fitbit use and activity levels throughout a physical activity intervention: exploratory analysis from a randomized controlled trial. JMIR Mhealth Uhealth 2018 Feb 5;6(2):e29 [FREE Full text] [CrossRef] [Medline]
- Hurling R, Catt M, Boni MD, Fairley BW, Hurst T, Murray P, et al. Using internet and mobile phone technology to deliver an automated physical activity program: randomized controlled trial. J Med Internet Res 2007;9(2):e7 [FREE Full text] [CrossRef] [Medline]
- Kirwan M, Duncan MJ, Vandelanotte C, Mummery WK. Using smartphone technology to monitor physical activity in the 10,000 Steps program: a matched case-control trial. J Med Internet Res 2012;14(2):e55 [FREE Full text] [CrossRef] [Medline]
- Fukuoka Y, Vittinghoff E, Jong SS, Haskell W. Innovation to motivation--pilot study of a mobile phone intervention to increase physical activity among sedentary women. Prev Med 2010;51(3-4):287-289 [FREE Full text] [CrossRef] [Medline]
- Edney S, Looyestyn J, Ryan J, Kernot J, Maher C. Posts, pics, or polls? Which post type generates the greatest engagement in a Facebook physical activity intervention? Transl Behav Med 2018 Apr 5. [CrossRef] [Medline]
- Rovniak LS, Kong L, Hovell MF, Ding D, Sallis JF, Ray CA, et al. Engineering online and in-person social networks for physical activity: a randomized trial. Ann Behav Med 2016 Dec;50(6):885-897 [FREE Full text] [CrossRef] [Medline]
- Althoff T, Jindal P, Leskovec J. Online actions with offline impact: how online social networks influence online and offline user behavior. Proc Int Conf Web Search Data Min 2017 Feb;2017:537-546 [FREE Full text] [CrossRef] [Medline]
- Cavallo DN, Tate DF, Ward DS, DeVellis RF, Thayer LM, Ammerman AS. Social support for physical activity-role of Facebook with and without structured intervention. Transl Behav Med 2014 Dec;4(4):346-354 [FREE Full text] [CrossRef] [Medline]
- Ashton L, Morgan P, Hutchesson M, Rollo M, Collins C. Feasibility and preliminary efficacy of the 'HEYMAN' healthy lifestyle program for young men: a pilot randomised controlled trial. Nutr J 2017 Dec 13;16(1):2 [FREE Full text] [CrossRef] [Medline]
- Mendoza J, Baker K, Moreno M, Whitlock K, Abbey-Lambertz M, Waite A, et al. A Fitbit and Facebook mHealth intervention for promoting physical activity among adolescent and young adult childhood cancer survivors: a pilot study. Pediatr Blood Cancer 2017 Dec;64(12). [CrossRef] [Medline]
- Schoenfelder E, Moreno M, Wilner M, Whitlock K, Mendoza J. Piloting a mobile health intervention to increase physical activity for adolescents with ADHD. Prev Med Rep 2017 Jun;6:210-213 [FREE Full text] [CrossRef] [Medline]
- Kernot J, Olds T, Lewis LK, Maher C. Usability testing and piloting of the Mums Step It Up program--a team-based social networking physical activity intervention for women with young children. PLoS One 2014;9(10):e108842 [FREE Full text] [CrossRef] [Medline]
- Pumper M, Mendoza J, Arseniev-Koehler A, Holm M, Waite A, Moreno M. Using a Facebook group as an adjunct to a pilot mHealth physical activity intervention: a mixed methods approach. Stud Health Technol Inform 2015;219:97-101. [CrossRef] [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Eldridge SM, Chan CL, Campbell MJ, Bond CM, Hopewell S, Thabane L, et al. CONSORT 2010 statement: extension to randomised pilot and feasibility trials. Br Med J 2016 Oct 24;355:i5239 [FREE Full text] [CrossRef] [Medline]
- Brooke J. SUS: A 'quick and dirty' usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability Evaluation in Industry. London: Taylor & Francis; 1996:189-194.
- Kooiman TJ, Dontje ML, Sprenger SR, Krijnen WP, van der Schans CP, de Groot M. Reliability and validity of ten consumer activity trackers. BMC Sports Sci Med Rehabil 2015;7:24 [FREE Full text] [CrossRef] [Medline]
- Sushames A, Edwards A, Thompson F, McDermott R, Gebel K. Validity and reliability of Fitbit Flex for step count, moderate to vigorous physical activity and activity energy expenditure. PLoS One 2016;11(9):e0161224 [FREE Full text] [CrossRef] [Medline]
- Hamburg M, McKay J. International Software Testing Qualifications Board. 2010. Standard glossary of terms used in Software Testing URL: https://www.istqb.org/downloads/send/20-istqb-glossary/210-istqb-glossary-3-2-ga-release-notes-final.html [accessed 2019-02-18] [WebCite Cache]
- Colley R, Connor GS, Tremblay MS. Quality control and data reduction procedures for accelerometry-derived measures of physical activity. Health Rep 2010 Mar;21(1):63-69 [FREE Full text] [Medline]
- Dominick GM, Winfree KN, Pohlig RT, Papas MA. Physical activity assessment between consumer- and research-grade accelerometers: a comparative study in free-living conditions. JMIR Mhealth Uhealth 2016 Sep 19;4(3):e110 [FREE Full text] [CrossRef] [Medline]
- Queensland Government. 10,000 Steps URL: https://www.healthier.qld.gov.au/about/support/10000-steps/ [accessed 2019-02-18] [WebCite Cache]
- 10,000 Steps. Heart Foundation Walking URL: https://www.10000steps.org.au/articles/heart-foundation-walking/ [accessed 2019-02-18] [WebCite Cache]
- Tudor-Locke C, Craig CL, Brown WJ, Clemes SA, de Cocker K, Giles-Corti B, et al. How many steps/day are enough? For adults. Int J Behav Nutr Phys Act 2011;8:79 [FREE Full text] [CrossRef] [Medline]
- Sauro J. A practical guide to the system usability scale: background, benchmarks & best practices. In: A Practical Guide to the System Usability Scale. California: CreateSpace Independent Publishing Platform; 2011.
- Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst 2015 Feb;39(2):1. [CrossRef] [Medline]
- Torbjørnsen A, Jenum AK, Småstuen MC, Arsand E, Holmen H, Wahl AK, et al. A low-intensity mobile health intervention with and without health counseling for persons with type 2 diabetes, part 1: baseline and short-term results from a randomized controlled trial in the Norwegian part of RENEWING HEALTH. JMIR Mhealth Uhealth 2014;2(4):e52 [FREE Full text] [CrossRef] [Medline]
- Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale. Int J Hum Comput Interact 2008 Jul 30;24(6):574-594. [CrossRef]
- Cohen J. Statistical Power Analysis For The Behavioral Sciences Second Edition. Abingdon: Routledge; 1988.
- R Core Team. The R Project for Statistical Computing. Vienna: R Foundation for Statistical Computing; 2013. URL: http://www.R-project.org/ [accessed 2019-02-19] [WebCite Cache]
- Wickham H, François R, Henry L, Müller K. The Comprehensive R Archive Network. dplyr: A Grammar of Data Manipulation URL: https://CRAN.R-project.org/package=dplyr [accessed 2019-02-19] [WebCite Cache]
- Wickham H. ggplot2: Elegant Graphics for Data Analysis. New York: Springer; 2009.
- Torchiano M. The Comprehensive R Archive Network. 2017. effsize: Efficient Effect Size Computation URL: https://CRAN.R-project.org/package=effsize [accessed 2019-02-19] [WebCite Cache]
- Komsta L, Novomestky F. The Comprehensive R Archive Network. 2015. moments: Moments, cumulants, skewness, kurtosis and related tests URL: https://cran.r-project.org/web/packages/moments/index.html [accessed 2019-02-18] [WebCite Cache]
- World Health Organisation: Regional Office for Europe. Body mass index - BMI URL: http://www.euro.who.int/en/health-topics/disease-prevention/nutrition/a-healthy-lifestyle/body-mass-index-bmi [accessed 2019-02-18] [WebCite Cache]
- Greene J, Sacks R, Piniewski B, Kil D, Hahn JS. The impact of an online social network with wireless monitoring devices on physical activity and weight loss. J Prim Care Community Health 2013 Jul 1;4(3):189-194. [CrossRef] [Medline]
- Chung AE, Skinner AC, Hasty SE, Perrin EM. Tweeting to health: a novel mHealth intervention using Fitbits and Twitter to foster healthy lifestyles. Clin Pediatr (Phila) 2016 Jun 16;56(1):26-32. [CrossRef] [Medline]
- Sullivan AN, Lachman ME. Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Front Public Health 2016;4:289 [FREE Full text] [CrossRef] [Medline]
- Gardner B, Smith L, Lorencatto F, Hamer M, Biddle SJ. How to reduce sitting time? A review of behaviour change strategies used in sedentary behaviour reduction interventions among adults. Health Psychol Rev 2015 Sep 16;10(1):89-112. [CrossRef] [Medline]
- Yates T, Haffner SM, Schulte PJ, Thomas L, Huffman KM, Bales CW, et al. Association between change in daily ambulatory activity and cardiovascular events in people with impaired glucose tolerance (NAVIGATOR trial): a cohort analysis. Lancet 2014 Mar 22;383(9922):1059-1066. [CrossRef] [Medline]
- Kernot J, Olds T, Lewis LK, Maher C. Effectiveness of a facebook-delivered physical activity intervention for post-partum women: a randomized controlled trial protocol. BMC Public Health 2013;13:518 [FREE Full text] [CrossRef] [Medline]
- King AC, Hekler EB, Grieco LA, Winter SJ, Sheats JL, Buman MP, et al. Effects of three motivationally targeted mobile device applications on initial physical activity and sedentary behavior change in midlife and older adults: a randomized trial. PLoS One 2016;11(6):e0156370 [FREE Full text] [CrossRef] [Medline]
- al Ayubi SU, Parmanto B, Branch R, Ding D. A persuasive and social mHealth application for physical activity: a usability and feasibility study. JMIR Mhealth Uhealth 2014;2(2):e25 [FREE Full text] [CrossRef] [Medline]
- Duncan M, Vandelanotte C, Kolt GS, Rosenkranz RR, Caperchione CM, George ES, et al. Effectiveness of a web- and mobile phone-based intervention to promote physical activity and healthy eating in middle-aged males: randomized controlled trial of the ManUp study. J Med Internet Res 2014;16(6):e136 [FREE Full text] [CrossRef] [Medline]
- Direito A, Jiang Y, Whittaker R, Maddison R. Apps for IMproving FITness and increasing physical activity among young people: the AIMFIT pragmatic randomized controlled trial. J Med Internet Res 2015;17(8):e210 [FREE Full text] [CrossRef] [Medline]
- O'Brien HL, Toms EG. What is user engagement? A conceptual framework for defining user engagement with technology. J Am Soc Inf Sci 2008 Apr;59(6):938-955. [CrossRef]
- Rozin P, Royzman EB. Negativity bias, negativity dominance, and contagion. Pers Soc Psychol Rev 2016 Dec 21;5(4):296-320 [FREE Full text] [CrossRef]
- Nielsen J. Usability Engineering. San Francisco: Elsevier; 1994.
- International Organization for Standardization. 2018. Ergonomics of human-system interaction - Part 11: Usability: Definitions and concepts URL: https://www.iso.org/standard/63500.html [accessed 2019-02-18] [WebCite Cache]
- Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 2013;1(1):1 [FREE Full text] [CrossRef]
- Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q 1989 Sep;13(3):319. [CrossRef]
- Davis FD, Bagozzi RP, Warshaw PR. User acceptance of computer technology: a comparison of two theoretical models. Manage Sci 1989;35(8):982-1003. [CrossRef]
- Taki S, Lymer S, Russell CG, Campbell K, Laws R, Ong K, et al. Assessing user engagement of an mHealth intervention: development and implementation of the growing healthy app engagement index. JMIR Mhealth Uhealth 2017 Jun 29;5(6):e89 [FREE Full text] [CrossRef] [Medline]
- Dumuid D, Stanford T, Martin-Fernández JA, Pedišić Z, Maher C, Lewis L, et al. Compositional data analysis for physical activity, sedentary time and sleep research. Stat Methods Med Res 2018 Dec;27(12):3726-3738. [CrossRef] [Medline]
- Chastin S, Egerton T, Leask C, Stamatakis E. Meta-analysis of the relationship between breaks in sedentary behavior and cardiometabolic health. Obesity (Silver Spring) 2015;23(9):1800-1810 [FREE Full text] [CrossRef] [Medline]
- Bandura A. Social cognitive theory of self-regulation. Organ Behav Hum Decis Process 1991;50(2):248-287. [CrossRef]
- Wang JB, Cataldo JK, Ayala GX, Natarajan L, Cadmus-Bertram LA, White MM, et al. Mobile and wearable device features that matter in promoting physical activity. J Mob Technol Med 2016 Jul;5(2):2-11. [CrossRef] [Medline]
- Yardley L, Spring BJ, Riper H, Morrison LG, Crane DH, Curtis K, et al. Understanding and promoting effective engagement with digital behavior change interventions. Am J Prev Med 2016 Nov;51(5):833-842. [CrossRef] [Medline]
- Fadhil A. arXiv. 2018. Towards Automatic & Personalised Mobile Health Interventions: An Interactive Machine Learning Perspective URL: https://arxiv.org/abs/1803.01842 [accessed 2019-02-18] [WebCite Cache]
- Nahum-Shani I, Smith SN, Spring BJ, Collins LM, Witkiewitz K, Tewari A, et al. Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Ann Behav Med 2016 Sep 23. [CrossRef] [Medline]
|BMI: body mass index|
|mHealth: mobile health|
|RCT: randomized controlled trial|
|SUS: System Usability Scale|
Edited by G Eysenbach; submitted 12.09.18; peer-reviewed by L Weiner, C Becker, L Becker, J Bort-Roig; comments to author 05.11.18; revised version received 13.12.18; accepted 30.01.19; published 28.03.19
©Huong Ly Tong, Enrico Coiera, William Tong, Ying Wang, Juan C Quiroz, Paige Martin, Liliana Laranjo. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 28.03.2019.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.