Published on 27.04.16 in Vol 4, No 2 (2016): Apr-Jun
Behavior Change Techniques Present in Wearable Activity Trackers: A Critical Analysis
Background: Wearable activity trackers are promising as interventions that offer guidance and support for increasing physical activity and health-focused tracking. Most adults do not meet their recommended daily activity guidelines, and wearable fitness trackers are increasingly cited as having great potential to improve the physical activity levels of adults.
Objective: The objective of this study was to use the Coventry, Aberdeen, and London-Refined (CALO-RE) taxonomy to examine if the design of wearable activity trackers incorporates behavior change techniques (BCTs). A secondary objective was to critically analyze whether the BCTs present relate to known drivers of behavior change, such as self-efficacy, with the intention of extending applicability to older adults in addition to the overall population.
Methods: Wearing each device for a period of 1 week, two independent raters used CALO-RE taxonomy to code the BCTs of the seven wearable activity trackers available in Canada as of March 2014. These included Fitbit Flex, Misfit Shine, Withings Pulse, Jawbone UP24, Spark Activity Tracker by SparkPeople, Nike+ FuelBand SE, and Polar Loop. We calculated interrater reliability using Cohen's kappa.
Results: The average number of BCTs identified was 16.3/40. Withings Pulse had the highest number of BCTs and Misfit Shine had the lowest. Most techniques centered around self-monitoring and self-regulation, all of which have been associated with improved physical activity in older adults. Techniques related to planning and providing instructions were scarce.
Conclusions: Overall, wearable activity trackers contain several BCTs that have been shown to increase physical activity in older adults. Although more research and development must be done to fully understand the potential of wearables as health interventions, the current wearable trackers offer significant potential with regard to BCTs relevant to uptake by all populations, including older adults.
JMIR mHealth uHealth 2016;4(2):e40
Chronic Illness, Physical Activity, and Sedentary Behavior
Physical inactivity contributes to an estimated 3.2 million deaths each year . As of 2010, almost one in four adults was receiving less than the recommended 150 minutes of moderate-intensity physical activity per week [ ]. Physical inactivity is the fourth leading cause of mortality, behind only hypertension, tobacco use, and high blood glucose [ ]. The prevalence of physical inactivity is increasing, and it has been identified as a major risk factor for breast and colon cancers, diabetes, and heart disease [ ]. Increased exercise can reduce frailty, lower blood pressure, and lead to a longer independent life [ , ].
Sedentary behavior is an independent risk factor for chronic disease that is separate from physical inactivity. Sedentary behavior is defined as “any waking behavior characterized by an energy expenditure of < 1.5 metabolic equivalents while in a sitting or reclining posture” . For example, children who watch more than 2 hours of television a day have poorer body composition, physical fitness, self-esteem, prosocial behavior, and academic achievement [ ]. Sedentary behavior is also associated with metabolic syndrome and cardiovascular disease in adults independently of physical activity [ , ].
Wearable Activity Trackers
Wearable activity trackers are an emerging solution for motivating people to improve their physical activity levels and reduce sedentary behavior. Wearable trackers are activity monitors that track daily movement through sensors and companion smartphone or computer applications. As of 2015, at least half of consumers have heard of wearable activity trackers such as Fitbit or Jawbone and one in three have plans to purchase one . And although the predicted buyers for most products are young people who are already living a healthy lifestyle [ ], participants aged more than 60 years also appear to be receptive to using wearable activity trackers and learn to use them quite easily [ ]. The newer-generation wearable activity trackers are also fairly accurate when compared with research-grade devices, [ ] but only when used by people who do not have the atypical gaits often seen in those who experienced stroke or traumatic brain injury or have Parkinson's disease [ - ].
Behavior Change Techniques
One question that emerges is how well wearable activity trackers align with the evidence-based techniques that have been shown to increase physical activity levels. One approach to identifying the behavior change techniques (BCTs) present in new and emerging technologies is to use a taxonomy such as the Coventry, Aberdeen, and London-Refined (CALO-RE) taxonomy. First published in 2011, the CALO-RE taxonomy contains 40 techniques derived from behavior change theories. It was based on an earlier 26-item taxonomy developed in 2008 by Abraham and Michie [, ] and was refined using systematic reviews of physical activity and healthy eating interventions. The CALO-RE taxonomy was designed to help developers of new interventions identify and apply evidence-based techniques [ ]. In 2013, Michie et al expanded the CALO-RE taxonomy to Behavior Change Technique Taxonomy (BCTT), which contains 93 items sorted into a hierarchy and is intended for multiple behaviors and disciplines (eg, health, environment) [ , , ]. However, unlike BCTT, CALO-RE was specifically designed for physical activity and healthy eating behaviors and continues to be widely used [ , ]. Furthermore, the CALO-RE taxonomy has been widely used to characterize physical activity interventions such as smartphone apps [ ], health coaching [ ], and interventions for preventing and managing obesity in children [ ].
Wearable activity trackers are being targeted at users of mobile apps as a way to promote physical activity . However, there are key differences between mobile apps and wearable activity trackers. Direito et al [ ] found that the 40 top-rated physical activity and diet apps most commonly provided instruction, set graded tasks, and prompted self-monitoring. In a similar study of 167 physical activity apps, Conroy et al [ ] also identified that it was common for apps to model or demonstrate target behaviors and provide feedback on performance. More recently, Lyons et al [ ] used the BCTT to systematically analyze 13 wearable activity trackers and found that all trackers helped users to self-monitor behavior, obtain feedback on behavior, and add objects to the environment while also generally supporting users in goal setting and comparing their behavior with their goal. Lyons et al [ ] also found that wearable activity monitors contain a wide range of techniques that are typically used in clinical behavioral interventions and that the trackers are a medium by which these interventions may be translated into widespread use. However, one aspect not addressed by the above studies is self-efficacy. In recent years, self-efficacy has emerged as a priority for improving physical activity in older adults. Self-efficacy is “the belief in one's capabilities to organize and execute the courses of action required to produce given attainments” [ ]. Theoretically, people who have a high self-efficacy for physical activity should be more likely to initiate, increase, and maintain physical activity, even in the face of obstacles and setbacks [ ]. A 2011 systematic review identified self-efficacy as one of the most consistent predictors of physical activity in adults of all ages [ ]. A 2009 literature review of interventions for older adults also found that self-efficacy was one of the most intensely studied and constant predictors of physical activity maintenance [ ]. Furthermore, a 2014 systematic review by French et al [ ] concluded that self-regulation techniques such as goal setting, feedback, and social support are effective for younger adults, whereas older adults may benefit more from problem-solving, rewards for successful behavior, and modeling or demonstrating behavior. Therefore, the objective of this study was to use the CALO-RE taxonomy to examine whether the design of wearable activity trackers incorporates BCTs. A secondary objective was to critically analyze if the BCTs present relate to known drivers of behavior change in older adults, such as self-efficacy.
We ran this study concurrently with another study examining the real and perceived acceptance of wearable activity trackers by older adults (aged more than 50 years) living with chronic illness . We wanted to analyze BCTs as they relate to the overall population, an added nod to the applicability to the older adult population. The team of researchers included a pharmacist (KG), a pharmacy student (ML), two systems design engineers (CB and LR), a kinesiologist (LG), and an information specialist (KM). We used the same trackers for both studies, as they reflected the available wearable fitness trackers in Canada. Our inclusion criteria for the wearable activity trackers included (1) continuous monitoring of some kind of physical activity outcome (steps, minutes of activity, points) and (2) provision of feedback via a separate mobile device or personal computer. We considered a device to be a wearable activity tracker if it contained an accelerometer and connected with a mobile platform. The device also had to be able to be wirelessly paired with handheld or desktop computers with at least Bluetooth 2.0 and be compatible with either Android 1.6+ or Apple's operating system iOS 6.0+. The following wearable activity trackers were evaluated for BCT content: Misfit Shine, Fitbit Flex, Jawbone UP24, Withings Pulse, Nike+ FuelBand SE, Polar Loop, and SparkPeople ( ).
We coded the wearable activity trackers using the CALO-RE taxonomy, which is a standardized tool for describing and comparing the BCTs used in lifestyle interventions . As a test for calibration, the mobile apps Runkeeper and MyFitnessPal were coded using the CALO-RE taxonomy. To resolve differences in interpretation between raters, we discussed any ambiguous descriptors or definitions from the CALO-RE taxonomy until we reached agreement during the calibration process. This is correct For each wearable activity tracker, we downloaded the associated mobile app onto an Apple device (iPad mini, iPhone 5) and/or an Android device (Samsung Galaxy S4 smartphone or Google Nexus tablet). Two independent raters (ML and KG) wore each activity tracker for 1 week between May and August 2014 and rated the seven activity trackers and their associated mobile/Web-based platforms using the CALO-RE taxonomy. Images and website links that indicated the presence of a BCT were compiled during ratings and used later in discussion to resolve any conflicts between raters. If consensus was not reached after consulting the referenced item, a third user, who had used the activity tracker for at least 1 month, was brought in to resolve the disagreements. With 1 month of use, the third user was more likely to have been exposed to all possible taxonomy elements, if present, in the device.
We coded each technique using a dichotomous score of either 0 (not present) or 1 (present). For mobile apps and websites that had archives of information articles, we only coded a technique if the user was prompted to read a specific article or community post through an email alert or daily message. Upon installation of certain mobile apps, animations would prompt users to look at specific tabs and features or everything listed in the main menu. If any of these elements fit in with a BCT, it was also coded. We did not consider any information that was not immediately accessible or that was not presented through prompting.
We used descriptive statistics to summarize the CALO-RE ratings. We calculated interrater reliability using Cohen's kappa on SPSS Statistics (version 22, IBM). Cohen's kappa is used to describe the degree of interrater reliability between two raters and can be applied to dichotomous data . We also calculated the total number of techniques present in each activity tracker and the frequency of each technique.
Seven wearable activity trackers were rated by two independent raters using the 40-item CALO-RE taxonomy . The number of BCTs ranged from 10 to 23, the most common shown in . The 40 BCTs rated in each tracker were based on the CALO-RE taxonomy and included consequences of behavior, goal setting, problem-solving, outcome review, prompting, practicing, teaching, and social encouragement. As shown in , a total of 9 techniques were present in every tracker. All trackers prompt users to self-monitor their activity levels, review goals, and review past successes while also providing feedback on performance, either by comparing step counts with user goals or by sending summary emails on activity levels over a certain period of time (daily, weekly, monthly). All trackers also included an optional social component to help users see others' approval or to plan for social support and change. Six trackers encouraged users to set a physical activity goal. Seven BCTs that were used less commonly included focusing on past and future performances, teaching prompts and cues, and instructing on how to perform a behavior.
There were 15 techniques absent in all wearable activity trackers, as listed in. The missing techniques were related to self-efficacy, planning, negative feelings (eg, fear), consequences, and comfort zones.
The kappa scores for interrater reliability of the taxonomy for the seven wearable activity trackers ranged from .746 to 1 () [ ]. According to the benchmarks assigned by Landis and Koch [ ], the strength of agreement is “substantial” to “almost perfect.”
|Wearable activity tracker||Cohen\'s kappa (95% CI)|
|Fitbit Flex||1 (0.00)|
|Jawbone UP24||.899 (0.764-1.034)|
|Misfit Shine||1 (0.00)|
|Nike+ FuelBand SE||.944 (0.836-1.052)|
|Polar Loop||.881 (0.722-1.040)|
|SparkPeopl Spark Activity Tracker||.746 (0.544-0.948)|
|Withings Pulse||.899 (0.764-1.034)|
The mean number of BCTs incorporated in wearable activity trackers was 16.3/40 (SD 4.6), with Withings Pulse having the highest number at 23/40 and Misfit Shine having the lowest at 10/40 (). In case of Withings Pulse, most of the techniques, particularly those that focused on the provision of information, were addressed in a comprehensive “Frequently Asked Questions” section of the mobile app. All of the BCTs coded this way were related to the provision of information, such as “provide instruction on how to perform the behavior.” In comparison, for the SparkPeople device, which had 21/40 BCTs, most techniques were found in articles and videos available on the Web-based platform, which were brought to the attention of the user via email notifications. Misfit Shine and Polar Loop bands incorporated the lowest number of techniques, with only 10/40 and 13/40 techniques, respectively. This in part can be explained by the absence of outcome goals (eg, weight), minimal information in the accompanying mobile apps and websites, and a lack of a virtual reward system.
|Wearable activity tracker||Number of behavior change techniques (N=40)||Possible behavior change techniques present (%)|
|SparkPeople Spark Activity Tracker||21||52.5|
|Nike+ FuelBand SE||14||35.0|
Many of the newest generation wearable activity trackers include BCTs in the user interface. Results from the CALO-RE taxonomy ratings demonstrated that the devices tend to focus on techniques for goal setting, self-regulation, and social support. Techniques related to self-efficacy (such as planning, consequences, and knowledge) were present in less than half of the trackers or absent in all seven devices.
Our findings were similar to the ratings of activity trackers by Lyons et al . However, there are some notable differences. For example, in our study, none of the reviewers identified problem-solving as a feature in the Jawbone tracker, whereas the coders in the study by Lyons et al did. Although Lyons et al [ ] used the 93-item taxonomy, they recalculated their results with the CALO-RE taxonomy and found an average of 9 techniques across 13 activity trackers. In comparison, we found an average of 16 techniques, which suggests that identification of BCTs depends on how users experience the devices. While rating these devices, we were also actively testing the devices with individuals aged more than 50 years who were living with chronic illness. By concurrently using the devices alongside active users, we may have had the opportunity to identify more features than if we had limited our use of the trackers to the research team.
In previous evaluations of physical activity interventions among adults aged 50 years and older, social support from peer mentors, families, and friends significantly increased initiation and maintenance of physical activity . Online communities have also been associated with increasing step counts in walking programs in study participants with an average age of 50 years and greater [ ]. Behavior change techniques relating to social support were present in every wearable activity tracker and associated platforms through online communities and connectivity to popular social media networks.
A 2014 systematic review by French et al  identified that the three BCTs with the greatest effect on physical activity in older adults are problem-solving, rewards for successful behavior, and modeling or demonstrating the behavior. We found that rewards were present in all seven trackers, while behavior modeling was only present in one tracker, and problem-solving was not present in any of the trackers. French et al [ ] also identified that interventions had the greatest effect on older adults when they provided a combination of normative information about others' behavior and information on where and when to perform behavior and helped participants plan social support or social change. We also found that two techniques (information about others' behavior and planning) were present in all seven trackers but that only two trackers helped users identify where and when to perform the behavior.
Self-efficacy is an important predictor for starting and maintaining physical activity in adults aged more than 50 years . However, the effect of wearable activity trackers on self-efficacy is not clear, as O’Brien et al found no increase in physical activity self-efficacy in a 12-week study of older adults using Nike FuelBand [ ]. The systematic review by French et al [ ] found that the techniques associated with greater self-efficacy include prompt use of imagery, motivational interviewing, and prompt generalization of target behaviors, none of which were identified in the trackers in our study. This suggests that there is potential if self-efficacy techniques are increased in wearable activity trackers for increase in physical activity.
The focus of wearable activity trackers on self-monitoring and self-regulation is to be expected. The main design and purpose of these devices is to monitor past, present, and future activity. A qualitative analysis of an 18-month physical activity intervention in older adults found that targeting self-regulation behaviors may support long-term increases in physical activity . All but one tracker required users to set an activity goal, often in the form of steps per day. Goal setting may also significantly increase physical activity among older adults, particularly if goals are specific and related to the desired behavior [ ].
Although we had a high interrater reliability, the ratings may not represent the most current version of the wearable activity tracker and associated platforms because of frequent updates. We minimized these elements by ensuring that all notifications were enabled, saving screenshots and website links representing each technique, consulting an experienced tracker user when necessary, and downloading all updates as of August 8, 2014. This method of evaluating the physical tracker and downloaded platforms allowed us to identify more BCTs than if Web-based descriptions alone were used, which was the approach used by Conroy et al  to rate mobile apps for physical activity using the CALO-RE taxonomy.
One challenge for studies of this nature is that there is no guarantee that the user will encounter a technique even if it is present. For example, if a user does not wish to use or does not have access to social media, then the user will not encounter any of the tracker's social support techniques. A further challenge for rating trackers is that they are complex tools with multiple features and different users are likely to have different experiences. For example, a user who wants to perform more physical activity may not use the device in the same way as a user who wants to be less sedentary. Similarly, a user who has a lower health or technology literacy may not explore the features as deeply as a health professional or expert technology user, regardless of age. As a result, this study should be considered a snapshot of the BCTs of wearable activity trackers and of the potential of how these trackers can relate to self-efficacy in the older user. By determining the similarities and differences between the overall population and the older adult population, there is great potential to develop wearable activity trackers and their affiliated apps to be the most broadly reaching.
Wearable activity trackers are a promising innovation for promoting physical activity behaviors in a wide age range of users, including the older adult population. They can easily be distributed across a wide population and integrated as a part of physical activity interventions through pharmacies and prescribers. Behavior change techniques most commonly found in the evaluated wearable activity trackers, such as self-monitoring and self-regulation techniques, are likely to appeal more to younger and middle-aged adults. To make wearable activity trackers more appealing to older adults, additional BCTs that are specific to older adult needs may be necessary, such as helping users find ways to overcome barriers to physical activity by problem-solving and modeling or demonstrating ways to increase physical activity. Future collaborations between tracker developers and health behavior change experts may enhance the potential to elicit behavior change.
The authors wish to thank Joslin Goh, Statistical Consultant, University of Waterloo.
Conflicts of Interest
Multimedia Appendix 1
CALO-RE Taxonomy Rating Data.PDF File (Adobe PDF File), 30KB
- Lim SS, Vos T, Flaxman AD, Danaei G, Shibuya K, Adair-Rohani H, et al. A comparative risk assessment of burden of disease and injury attributable to 67 risk factors and risk factor clusters in 21 regions, 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012 Dec 15;380(9859):2224-2260. [CrossRef] [Medline]
- Mendis S. World Health Organization. Geneva, Switzerland; 2014. Global status report on noncommunicable diseases 2014 URL: http://apps.who.int/iris/bitstream/10665/148114/1/9789241564854_eng.pdf?ua=1 [accessed 2014-07-27] [WebCite Cache]
- World Health Organization. World Health Organization Report. Geneva, Switzerland: World Health Organization; 2009. Global health risks: mortality and burden of disease attributable to selected major risks URL: http://www.who.int/healthinfo/global_burden_disease/GlobalHealthRisks_report_full.pdf [accessed 2016-03-24] [WebCite Cache]
- Christmas C, Andersen RA. Exercise and older patients: guidelines for the clinician. J Am Geriatr Soc 2000 Mar;48(3):318-324. [Medline]
- Tanaka H, Dinenno FA, Monahan KD, Clevenger CM, DeSouza CA, Seals DR. Aging, Habitual Exercise, and Dynamic Arterial Compliance. Circulation 2000 Sep 12;102(11):1270-1275. [CrossRef]
- Arnold B. NPD Group. 2014. Wearable tech device awareness surpasses 50 percent among US consumers, according to NDP URL: https://www.npd.com/wps/portal/npd/us/news/press-releases/wearable-tech-device-awareness-surpasses-50-percent-among-us-consumers-according-to-npd/ [accessed 2016-03-25] [WebCite Cache]
- Harvey JA, Chastin SFM, Skelton DA. Prevalence of sedentary behavior in older adults: a systematic review. Int J Environ Res Public Health 2013 Dec;10(12):6645-6661. [CrossRef] [Medline]
- Fox S, Duggan M. Pew Research Center.: Pew Research Center, Internet, Science & Tech; 2013. Tracking for Health URL: http://www.pewinternet.org/2013/01/28/tracking-for-health/ [accessed 2016-03-25] [WebCite Cache]
- Khalaf S. Flurry Insights. 2014 Jun 19. Health and Fitness Apps Finally Take Off, Fueled by Fitness Fanatics URL: http://www.flurry.com/blog/flurry-insights/health-and-fitness-apps-finally-take-fueled-fitness-fanatics [accessed 2016-03-25] [WebCite Cache]
- O'Brien T, Troutman-Jordan M, Hathaway D, Armstrong S, Moore M. Acceptability of wristband activity trackers among community dwelling older adults. Geriatr Nurs 2015;36(2 Suppl):S21-S25. [CrossRef] [Medline]
- Takacs J, Pollock CL, Guenther JR, Bahar M, Napier C, Hunt MA. Validation of the Fitbit One activity monitor device during treadmill walking. J Sci Med Sport 2014 Sep;17(5):496-500. [CrossRef] [Medline]
- Adam NJ, Spierer DK, Gu J, Bronner S. Comparison of steps and energy expenditure assessment in adults of Fitbit Tracker and Ultra to the Actical and indirect calorimetry. J Med Eng Technol 2013 Oct;37(7):456-462. [CrossRef] [Medline]
- Sasaki JE, Hickey A, Mavilia M, Tedesco J, John D, Kozey KS, et al. Validation of the Fitbit wireless activity tracker for prediction of energy expenditure. J Phys Act Health 2015 Feb;12(2):149-154. [CrossRef] [Medline]
- Lauritzen J, Muñoz A, Luis SJ, Civit A. The usefulness of activity trackers in elderly with reduced mobility: a case study. Stud Health Technol Inform 2013;192:759-762. [Medline]
- Fulk GD, Combs SA, Danks KA, Nirider CD, Raja B, Reisman DS. Accuracy of 2 activity monitors in detecting steps in people with stroke and traumatic brain injury. Phys Ther 2014 Feb;94(2):222-229. [CrossRef] [Medline]
- Fortune E, Lugade V, Morrow M, Kaufman K. Validity of using tri-axial accelerometers to measure human movement - Part II: Step counts at a wide range of gait velocities. Med Eng Phys 2014 Jun;36(6):659-669. [CrossRef] [Medline]
- Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008 May;27(3):379-387. [CrossRef] [Medline]
- Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychol Health 2011 Nov;26(11):1479-1498. [CrossRef] [Medline]
- Hill B, Richardson B, Skouteris H. Do we know how to design effective health coaching interventions: a systematic review of the state of the literature. Am J Health Promot 2015;29(5):e158-e168. [CrossRef] [Medline]
- French DP, Olander E, Chisholm DP, McSharry J. Which behaviour change techniques are most effective at increasing older adults' self-efficacy and physical activity behaviour? A systematic review. Ann Behav Med 2014 Oct;48(2):225-234. [CrossRef] [Medline]
- Conroy DE, Yang C, Maher J. Behavior change techniques in top-ranked mobile apps for physical activity. Am J Prev Med 2014 Jun;46(6):649-652. [CrossRef] [Medline]
- Olander EK, Fletcher H, Williams S, Atkinson L, Turner A, French DP. What are the most effective techniques in changing obese individuals' physical activity self-efficacy and behaviour: a systematic review and meta-analysis. Int J Behav Nutr Phys Act 2013;10:29. [CrossRef] [Medline]
- Martin J, Chater A, Lorencatto F. Effective behaviour change techniques in the prevention and management of childhood obesity. Int J Obes (Lond) 2013 Oct;37(10):1287-1294. [CrossRef] [Medline]
- Direito A, Dale LP, Shields E, Dobson R, Whittaker R, Maddison R. Do physical activity and dietary smartphone applications incorporate evidence-based behaviour change techniques? BMC Public Health 2014;14:646. [CrossRef] [Medline]
- Lyons EJ, Lewis ZH, Mayrsohn BG, Rowland JL. Behavior change techniques implemented in electronic lifestyle activity monitors: a systematic content analysis. J Med Internet Res 2014 Aug;16(8):e192. [CrossRef] [Medline]
- Bandura A. Self-efficacy: The exercise of control. New York, NY: Freeman; 1997.
- Koeneman MA, Verheijden M, Chinapaw MJ, Hopman-Rock M. Determinants of physical activity and exercise in healthy older adults: a systematic review. Int J Behav Nutr Phys Act 2011;8:142. [CrossRef] [Medline]
- van Stralen MM, De Vries H, Mudde A, Bolman C, Lechner L. Determinants of initiation and maintenance of physical activity among older adults: a literature review. Health Psychology Review 2009 Sep;3(2):147-207. [CrossRef]
- Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of Commercially Available Wearable Activity Trackers Among Adults Aged Over 50 and With Chronic Illness: A Mixed-Methods Evaluation. JMIR Mhealth Uhealth 2016;4(1):e7. [CrossRef] [Medline]
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977 Mar;33(1):159-174. [Medline]
- Multon K. Interrater Reliability. In: Salkind NL, editor. Encyclopedia of Research Design. Thousand Oaks, California: Sage Publications, Inc; 2010:626-628.
- Floegel TA, Giacobbi PR, Dzierzewski JM, Aiken-Morgan AT, Roberts B, McCrae CS, et al. Intervention markers of physical activity maintenance in older adults. Am J Health Behav 2015 Jul;39(4):487-499. [CrossRef] [Medline]
- Richardson CR, Buis LR, Janney AW, Goodrich DE, Sen A, Hess ML, et al. An online community improves adherence in an internet-mediated walking program. Part 1: results of a randomized controlled trial. J Med Internet Res 2010;12(4):e71. [CrossRef] [Medline]
|BCT: behavior change technique|
|BCTT: Behavior Change Technique Taxonomy|
|CALO-RE: Coventry, Aberdeen, and London-Refined|
Edited by G Eysenbach; submitted 25.03.15; peer-reviewed by E Lyons, A Müller; comments to author 29.04.15; revised version received 21.07.15; accepted 19.01.16; published 27.04.16
©Kathryn Mercer, Melissa Li, Lora Giangregorio, Catherine Burns, Kelly Grindrod. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 27.04.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.