This paper is in the following e-collection/theme issue:

Fitness Trackers and Smart Pedometers/Accelerometers (561) Mobile Health (mhealth) (2939) Human Factors and Usability Case Studies (787) Usability of Apps and User Perceptions of mHealth (974)

Published on in Vol 6, No 11 (2018): November

Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation

Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation

Usability Study of Mainstream Wearable Fitness Devices: Feature Analysis and System Usability Scale Evaluation

Authors of this article:

Jun Liang1, 2 Author Orcid Image ;   Deqiang Xian3 Author Orcid Image ;   Xingyu Liu4 Author Orcid Image ;   Jing Fu5 Author Orcid Image ;   Xingting Zhang6 Author Orcid Image ;   Buzhou Tang7 Author Orcid Image ;   Jianbo Lei8, 9 Author Orcid Image
Article Authors Cited by (86) Tweetations (9) Metrics

    Original Paper

    1IT Center, Second Affiliated Hospital, School of Medicine, Zhejiang University, Hangzhou, China

    2College of Information Engineering, China Jiliang University, Hangzhou, China

    3Sichuan College of Traditional Chinese Medicine, Mianyang, China

    4Affiliated Hospital of Stomatology, Southwest Medical University, Luzhou, China

    5International School, Southwest Medical University, Luzhou, China

    6Peking University Third Hospital, Beijing, China

    7Shenzhen HIT Campus, Harbin Institute of Technology, Shenzhen, China

    8Center for Medical Informatics, Peking University, Beijing, China

    9School of Medical Informatics and Engineering, Southwest Medical University, Luzhou, China

    Corresponding Author:

    Jianbo Lei, MD, PhD

    Center for Medical Informatics

    Peking University

    38 Xueyuan Road

    Haidian District

    Beijing, 100191

    China

    Phone: 86 8280 5901

    Email: jblei@hsc.pku.edu.cn


    Background: Wearable devices have the potential to promote a healthy lifestyle because of their real-time data monitoring capabilities. However, device usability is a critical factor that determines whether they will be adopted on a large scale. Usability studies on wearable devices are still scarce.

    Objective: This study aims to compare the functions and attributes of seven mainstream wearable devices and to evaluate their usability.

    Methods: The wearable devices selected were the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor B2, and Misfit Shine. A mixed method of feature comparison and a System Usability Scale (SUS) evaluation based on 388 participants was applied; the higher the SUS score, the better the usability of the product.

    Results: For features, all devices had step counting, an activity timer, and distance recording functions. The Samsung Gear S had a unique sports track recording feature and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Gear S, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices were able to monitor sleep, except the Apple Watch. For product characteristics, including attributes such as weight, battery life, price, and 22 functions such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, we found a very weak negative correlation between the SUS scores and price (r=−.10, P=.03) and devices that support expandable new features (r=−.11, P=.02), and a very weak positive correlation between the SUS scores and devices that support the activity type identification function (r=.11, P=.02). The Huawei Honor B2 received the highest score of mean 67.6 (SD 16.1); the lowest Apple Watch score was only 61.4 (SD 14.7). No significant difference was observed among brands. The SUS score had a moderate positive correlation with the user’s experience (length of time the device was used) (r=.32, P<.001); participants in the medical and health care industries gave a significantly higher score (mean 61.1, SD 17.9 vs mean 68.7, SD 14.5, P=.03).

    Conclusions: The functions of wearable devices tend to be homogeneous and usability is similar across various brands. Overall, Mi Band had the lowest price and the lightest weight. Misfit Shine had the longest battery life and most functions, and participants in the medical and health care industries had the best evaluation of wearable devices. The perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. A consumer’s SUS rating for a wearable device is related to their personal situation instead of the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products by integrating more cognitive behavior change techniques.

    JMIR Mhealth Uhealth 2018;6(11):e11066

    doi:10.2196/11066

    Keywords

    wearable devices; usability; System Usability Scale; function comparison; fitness 


    Background

    In recent years, wearable devices have gained tremendous momentum and have become part of people’s daily lives. Wearable devices are portable mobile intelligent devices that can be worn directly or integrated into clothing [1]. The wearable devices in this study are wristbands and watches with step counting as their core function [2]. After combining IMS market research consulting with 2012 market research data from ABI Research, the US market research firm BI Intelligence believed that less than 40% of US consumers were familiar with wearable devices in 2012, whereas at least 54% of consumers developed a strong interest in them for health monitoring and fitness tracking in 2013. BI Intelligence expects the global wearable market to grow at a compound annual rate of 35% between 2015 and 2019 [3]. According to data from the International Data Corporation’s Worldwide Quarterly Wearable Device Tracker, worldwide shipments of wearable devices grew by 1.2% in the first quarter of 2018 [4]. In addition, global smartwatch shipments in the second quarter of 2018 grew by 56% year on year [5]. Meanwhile, based on their original wearable market forecast from 2013 and interviews with industry experts, BI Intelligence concluded that, based on current trends, at least 33% of the US population will own wearable devices by the second quarter of 2017 [6]. The Tencent ISUX User Research Center released a report (smart wearable equipment market white paper) in 2015 [7] that contained a large-scale questionnaire survey with a total of 8083 domestic Chinese Tencent Instant Messenger user respondents. This stated that approximately 60% of internet users were familiar with wearable devices and that the number of Chinese internet users who owned a wearable device increased from 2.9% to 8.4% between November 2014 and May 2015. Among these users, wristband ownership was 4.6%, which was significantly higher than the 3.1% ownership of intelligent watches. Awareness of intelligent watches increased from 48% to 52%, and awareness of intelligent wristbands increased from 35% to 40% over the same period.

    Wearable devices have significant potential for health monitoring and they are being widely investigated [8-13]. As a very popular tool, the use of wearable devices in medical and fitness applications is rapidly increasing. These devices can monitor vital signs such as motion [14], nutrition status [15], and heart rate [16], as well as a diabetic patient’s blood sugar [17], cardiac disease status, apnea during sleep, and numerous other health parameters and statuses [18]. Luo and Fan [19] believed that the development of wearable devices may trigger innovation in medical treatment models. Kirk [20] believed that wearable medical technology may significantly improve the curative effect for patients, reduce physician time, and lower medical costs, representing future trends of precise personalized medical treatment. Appropriate technology, such as smart wearable devices, may create unprecedented opportunities for medical and health care service providers to obtain the accurate real-time data required to prevent, diagnose, and treat various chronic diseases in a more economical manner [21]. Recent studies have expressed concerns about the long-term use of wearable devices, emphasizing the need to combine behavioral change techniques such as target setting, feedback, and rewards with other evidence-based techniques in order to use them effectively to prompt behavior changes [22].

    However, consumer acceptance of wearable devices has not been as positive as expected. Many important issues that affect the ultimate adoption of wearable devices by consumers have been investigated, including the reliability [23,24] and validity of the measures that they monitor [25-27]. In addition, studies have confirmed that the key to a product’s success is the acceptance of wearable products by ordinary consumers and the comfort level of these products; product usability is an important aspect of this [28,29]. According to a definition by the International Organization for Standardization [30], usability refers to the effectiveness, efficiency, and user satisfaction rating of a product in a specific environment by a specific user for a specific purpose. It includes three aspects: effectiveness (ie, the accuracy and completeness of a goal that is achieved by a product); efficiency (ie, the effort required for a user to complete a task); and satisfaction (ie, the comfort and acceptability of a product). Usability tests and evaluations aim to make medical equipment easier, safer, and more effective and pleasant for users. A usability evaluation helps wearable devices to satisfy the requirements of the market and consumers [31].

    In traditional usability studies, a thematic analysis based on heuristic evaluations, cognitive walkthroughs, think-alouds, and interviews is used; the results are then integrated together [32-34]. Although these methods are very popular, they have some limitations because results obtained in a laboratory environment can be difficult to interpret and translate into practice [35]. Questionnaires are another commonly employed method. They have been widely accepted by clinical medical informatics and consumer health informatics researchers and practitioners, and they are also widely applied in electronic health records, computerized physician order entry, and health applications [36-39]. At the same time, as a reliable, easy-to-operate, and low-cost method of usability evaluation, they have been recommended by the Agency for Healthcare Research and Quality [40]. Common questionnaire scales include the System Usability Scale (SUS) [41]; Ages and Stages Questionnaire; Computer System Usability Questionnaire; Post-Study System Usability Questionnaire [42]; Use, Satisfaction, and Ease [43]; and Web Analysis and Measurement Inventory [44].

    Research has continuously established that usability evaluation and the resulting improvements in wearable device usability are critical to the wide and rapid adoption of wearable devices [45]. Research on the usability of wearable devices for health monitoring is still in the early stages. Schenkenfelder et al [46] investigated the user experience during running for three types of wearable devices (sports wristband, intelligent watch, and intelligent glasses). By analyzing feedback from 18 participants they found that the wristband and watch received higher SUS scores than the glasses. However, no significant differences between the wristband and watch were observed. Kaewkannate and Kim [47] compared the functions and features of four wearable devices (Withings Pulse, Misfit Shine, Jawbone Up24, and Fitbit Flex) and recruited seven users for objective and subjective evaluations. The Withings Pulse received the highest user acceptance, followed by the Fitbit Flex, Jawbone Up24, and Misfit Shine.

    Significance of This Study

    This study has some advantages over other similar studies: it used a larger number of devices (seven different intelligent watches and wristbands), including mainstream international and domestic devices, and involved a significantly larger number of participants (388 persons, the largest sample size among similar studies). We compared the functions and features of various devices and evaluated the usability of wearable health tracking devices currently on the market using a mature and stable SUS scale. These results will help to assess the acceptance level of wearable devices and to identify the influencing factors. As there is a growing consensus that wearable devices should be developed for health applications, this study will help to examine the issues affecting their large-scale use and it will contribute to the research and development of better health applications.


    Device Selection

    The device selection was based on open market performance data from NPD (a leading global consumer and retail data supplier) and Canalys (a leading global technology market analyst with a distinct channel focus) [48-50]. Our inclusion criteria for the wearable activity trackers included (1) continuous monitoring of some kind of physical activity (eg, steps), and (2) provision of feedback via a separate mobile device or personal computer. We considered a device to be a wearable activity tracker if it contained an accelerometer and connected to a mobile platform. The device also had to be able to connect wirelessly with handheld or desktop computers, and be compatible with either Android 1.6+ or Apple’s operating system iOS 6.0+. Therefore, current mainstream wearable devices with distinct market performance were selected as the research objects. These included the Apple Watch, Samsung Gear S, Fitbit Surge, Jawbone Up3, Mi Band, Huawei Honor Band B2, and Misfit Shine. Among these devices, the Apple Watch and Samsung Gear S are flagship products from leading mobile digital gadget vendors. The Fitbit Surge, Jawbone Up3, and Misfit Shine are well-known international sports wristband brands representative of different price ranges. Finally, the Mi Band and Huawei Honor Band B2 are leading brands of sports wristband in the Chinese market.

    Questionnaire Selection

    Usability refers to the effectiveness, efficiency, and user satisfaction rating of a product in a specific environment by a specific user for a specific purpose [51]. We chose to use the SUS, proposed by John Brooke in 1986, as the usability test tool. It is a simple scale based on a questionnaire and has been widely adopted in product usability evaluations. The SUS has the following advantages: (1) it is versatile and can be used to evaluate websites [52], software [53], mobile devices [54], and medical systems [55]; (2) it is a short questionnaire that is quick to answer; (3) a final score is provided with interpretation based on a well-established reference standard [56]; (4) it is free; (5) it is suitable even when applied to small samples (N<14); and (6) it has excellent reliability (0.85). Overall, the SUS is a quick and simple method for usability evaluation.

    The SUS contains 10 questions based on the Likert five-point scale; questions 1, 3, 5, 7, and 9 are positive and questions 2, 4, 6, 8, and 10 are negative. The 10 questions are closely related and are employed for the comprehensive evaluation of a product. A higher SUS score indicates better product usability.

    Details about the questionnaire’s design (the 10 questions) and scoring method are given in Multimedia Appendix 1.

    Subject Recruitment and Test Design

    Recruitment was based on a convenience sampling method. It leveraged WeChat, the social media platform with the largest user population in China. Over a period from July 1, 2016 to October 31, 2016, the research team used their WeChat accounts to send the questionnaire link to 2180 friends (between 8 and 314 each) and various WeChat groups (approximately 20 groups) and invited their friends to forward the link. A total of 388 volunteers who had experience with the seven wearable devices were recruited.

    Once the volunteers agreed to take part in the study, consent was obtained and they were invited to complete an evaluation questionnaire about their experiences with these products.

    This research passed an audit of the Peking University Biomedical Ethics Committee #IRB00001052 - 16008 - Exampt, and consent was obtained from all participants.

    Data Collection

    The questionnaires were distributed and the results collected via “sojump.com,” the largest free questionnaire survey platform in the world, which satisfied all the requirements for the questionnaires in this research [57]. The questionnaires were sent and completed via personal computers and mobile terminals. The final questionnaire “Comprehensive Evaluation Questionnaire for Intelligent Wearable Devices” is accessible online [58].

    Statistical Analysis

    For a descriptive statistical analysis, basic information about the respondents was collected, including gender, age, education, profession, and monthly income. Then, the product attributes (eg, price, weight, and battery life) and specific functions (whether 22 functions, such as step counting, activity time, activity type identification, sleep monitoring, and expandable new features, were supported) were summarized for the seven wearable devices. Finally, the SUS scores for the seven wearable devices used by the respondents were calculated.

    For the inferential statistical analysis, the relationships between the SUS scores and product attributes (product price, weight, and battery life) were explored using Pearson product moment correlation (since price data are continuous and have a normal distribution) and Spearman rank correlation coefficient (since the weight and battery life do not have a normal distribution). The correlations between the SUS score and the functions of the 22 products (eg, step counting, activity time, activity type identification, sleep monitoring, and expandable new features) were analyzed using point-biserial correlation. The strength of the correlation was assessed based on Cohen criteria: correlations less than .30 are considered small, correlations between .30 and .50 are considered medium, and correlations greater than .50 are considered strong [59,60]. Finally, to analyze the participants’ attitudes to the usability of different products, the analysis of variance (ANOVA) method was used with the brand and user demographic information. These analyses were all completed using IBM SPSS version 20 software (IBM Corp, Armonk, NY, USA).


    Comparison of Functions and Attributes of Devices

    Based on the product specifications and official websites, the features and general attributes of the seven devices were summarized and compared (see Tables 1 and 2). Table 1 summarizes the functions of each device. All seven devices had very powerful functions in three major categories (activity, health, and miscellaneous) and 20 features. The Fitbit Surge had the most features, followed by the Apple Watch, Huawei Honor B2, Samsung Gear S, Jawbone UP3, Mi Band, and Misfit Shine. The Samsung Gear S had the most sports features, the Jawbone UP3 had the most health features, and the Apple Watch had the most other additional features. All the devices supported three basic features: step counting, activity timer, and distance record. The Fitbit Surge was the only device to record the number of floors climbed, the Samsung Gear S had a unique sports track recording feature, and the Huawei Honor B2 had a unique wireless earphone. The Apple Watch, Samsung Watch, Jawbone Up3, and Fitbit Surge could measure heart rate. All the devices could monitor sleep, with the exception of the Apple Watch. Table 2 lists some of the major attributes of each device. Among the seven devices, the Misfit Shine had the longest battery life of 3 months, followed by the Mi Band, Fitbit Surge, Jawbone Up3, Huawei Honor wristband, Samsung Gear S, and Apple Watch, which had a battery life of only 18 hours. The Fitbit Surge, which weighed 354 grams, was the heaviest device, followed by the Jawbone UP3, Misfit Shine, Apple Watch, Huawei Honor Band B2, Samsung Gear S, and Mi Band, which weighed only 5 grams.

    System Usability Scale Questionnaire Respondents

    A total of 388 volunteers completed the questionnaire for this research. The volunteers included 83 Apple Watch users, 36 Samsung Gear S users, 37 Fitbit Surge users, 32 Jawbone Up3 users, 122 Mi Band users, 47 Huawei Honor Band B2 users, and 31 Misfit Shine users. Of the volunteers, 257 (66.2%) were male and 131 (33.8%) were female. The demographic information of the participants is shown in Table 3.

    System Usability Scale Scores for Each Device and Each Question

    Based on the SUS calculation formula, the SUS scores for each brand are shown in Table 4. The mean SUS score for each question was calculated using SPSS, as shown in Table 5 (for full results, see Multimedia Appendix 2). For further visualization, please see corresponding boxplot diagrams in Multimedia Appendixes 3 and 4.

    Correlation Between System Usability Scale Score and Product Characteristics

    We first explored the relationship between the SUS score and product characteristics (see Table 2). Pearson product moment correlation was used to analyze the relationship between the SUS value and the price, and Spearman rank correlation coefficient was used to analyze the relationship between the SUS values and the weight and battery life. We found a very weak correlation between the SUS score and the price of the product (r=–.10, P=.03), and no significant correlation between the SUS scores and the weight (r=–.04, P=.39) or battery life (r=.09, P=.10).

    We then explored the relationship between the SUS scores and the functions supported by the products (see Table 1) using point-biserial correlation. We found that the SUS score had a very weak positive correlation (r=.11, P=.02) with the devices that supported activity type identification, and a very weak negative correlation (r=–.11, P=.02) with devices that supported expandable new features, as shown in Table 6.

    Correlation Between System Usability Scale Score, Device Brand, and Participants’ Demographic Information

    To investigate a participant’s perception of the usability of each of the seven brands of wearable devices, seven ANOVA tests were performed. These were used to determine the correlation between the SUS scores, device brand, and participants’ demographic information (length of time the device was used, gender, age, education, profession, and monthly income) (see Table 7).

    Table 1. Function summary of the seven wearable devices.
    ItemsApple WatchSamsung Gear SJawbone Up3Fitbit SurgeMisfit ShineHuawei Honor B2Mi Band
    Activity







    		Step countingYesYesYesYesYesYesYes

    		Activity timeYesYesYesYesYesYesYes

    		Activity type identification—a—YesYesYesYesYes

    		Floor climbing———Yes———

    		Energy consumptionYesYesYesYesYesYesYes

    		Running track—Yes—————

    		Activity goalYesYesYesYesYesYesYes

    		Professional fitness recordYesYes—Yes——Yes
    Health







    		Heart rateYesYesYesYes———

    		Sleep monitoring—YesYesYesYesYesYes

    		Diet record——YesYesYes——

    		Inactivity notificationYes—Yes—YesYesYes
    Miscellaneous







    		Incoming call alertYesYes—YesYesYesYes

    		Vibrating alarm clockYesYesYesYesYesYesYes

    		Text message displayYesYes—YesYes——

    		Time displayYesYes—YesYes——

    		Remote control photographYes———Yes——

    		Sports achievement sharingYes—YesYesYesYesYes

    		Bluetooth synchronizationYesYesYesYes—YesYes

    		Expandable new featuresYesYes—————
    Total15141216141112

    aDevice does not support this feature.

    Table 2. Major attributes of the seven wearable devices.
    DeviceCountryDate to marketPrice (US$)Weight (g)Battery life (days)
    Apple WatchUSASeptember 10, 2014$40240.000.75
    Samsung Gear SKoreaSeptember 25, 2013$404122.502.00
    Fitbit SurgeUSAOctober 27, 2014$279354.007.00
    Jawbone Up3USANovember 5, 2014$18329.007.00
    Mi BandChinaJuly 22, 2014$125.0030.00
    Huawei Honor B2ChinaMarch 1, 2015$15540.005.00
    Misfit ShineUSASeptember 14, 2013$779.40180.00
    Table 3. Demographic information of the respondents (N=388).
    Itemsn (%)
    Device to be evaluated

    		Apple Watch83 (21.4)

    		Samsung Gear S36 (9.3)

    		Fitbit Surge37 (9.5)

    		Jawbone Up332 (8.2)

    		Mi Band122 (31.4)

    		Huawei Honor B247 (12.1)

    		Misfit Shine31 (8.0)
    Length of time the device was used

    		<1 week49 (12.6)

    		1 week-1 month96 (24.7)

    		1-3 months66 (17.0)

    		3-6 months53 (13.7)

    		6 months-1 year69 (17.8)

    		1-2 year49 (12.6)

    		>2 years6 (1.5)
    Gender

    		Male257 (66.2)

    		Female131 (33.8)
    Age

    		Under 1810 (2.6)

    		18~25107 (27.6)

    		26~3088 (22.7)

    		31~40113 (29.1)

    		41~5056 (14.4)

    		51~6013 (3.4)

    		>601 (0.3)
    Education

    		Under primary school3 (0.8)

    		Primary school-high school19 (4.9)

    		Junior college46 (11.9)

    		Bachelor185 (47.7)

    		Master of Science or Master of Arts98 (25.3)

    		PhD37 (9.5)
    Profession

    		Internet114 (29.4)

    		Finance28 (7.2)

    		Health care92 (23.7)

    		Manufacturing19 (4.9)

    		Fast consumer goods4 (1.0)

    		Education29 (7.5)

    		Law6 (1.5)

    		Trade6 (1.5)

    		Service20 (5.2)

    		Advertising and media6 (1.5)

    		Agriculture, forestry, fishery2 (0.5)

    		Miscellaneous62 (16.0)
    Monthly income (US$)

    		<14527 (7.0)

    		145~29034 (8.8)

    		291~58045 (11.6)

    		581~116093 (24.0)

    		>1160189 (48.7)
    Table 4. Total System Usability Scale (SUS) score of each device (N=388).
    DevicenMean (SD)High scoreLow score% of total sum
    Apple Watch8361.36 (14.69)95.0027.5020.40
    Samsung Gear S3662.08 (19.29)100.005.009.00
    Fitbit Surge3763.85 (21.97)100.000.009.50
    Jawbone Up33265.94 (17.53)100.0027.508.50
    Mi Band12265.12 (14.73)100.0035.5031.80
    Huawei Honor B24767.61 (16.12)100.0022.5012.70
    Misfit Shine3165.97 (20.21)100.0010.008.20
    Table 5. Mean System Usability Scale (SUS) score for each question and device.
    DeviceSUS score, mean (SD)

    		12345678910
    Apple Watch (n=83)2.86 (1.00)1.71 (1.16)2.90 (0.89)2.30 (1.26)2.60 (0.95)2.11 (1.14)2.84 (0.89)2.20 (1.23)2.78 (0.96)2.23 (1.26)
    Samsung Gear S (n=36)2.72 (1.26)1.61 (1.29)3.08 (1.02)2.31 (1.37) 2.78 (0.96)1.92 (1.27)2.86 (1.02)2.14 (1.44)3.06 (0.95)2.36 (1.31)
    Fitbit Surge (n=37)2.64 (1.38)2.00 (1.41)3.00 (1.07)2.56 (1.52)2.53 (1.06)2.06 (1.26)2.81 (1.17)2.44 (1.38)2.92 (1.08)2.67 (1.22)
    Jawbone Up3 (n=32)2.90 (1.14)1.90 (1.40)2.97 (0.91)2.58 (1.41) 2.71 (1.01)2.16 (1.21)2.94 (1.06)2.55 (1.29)3.10 (0.91)2.42 (1.52)
    Mi Band (n=122)2.65 (1.02)1.87 (1.30)2.94 (1.04)2.80 (1.33)2.43 (1.01)2.27 (1.13)2.93 (1.04)2.82 (1.11)2.50 (1.13)2.84 (1.19)
    Huawei Honor B2 (n=47)2.70 (1.07)1.72 (1.19)2.96 (1.07)2.85 (1.28)2.78 (0.94)2.46 (1.21)2.83 (0.97)2.76 (1.21)3.13 (0.88)2.80 (1.13)
    Misfit Shine (n=31)2.87 (1.01)2.33 (1.45)3.10 (1.09)2.87 (1.36)2.40 (1.13)2.27 (1.28)2.83 (1.21)2.60 (1.25)2.77 (1.14)2.50 (1.36)
    Total (N=388)2.74 (1.09)1.84 (1.09)2.97 (1.00)2.61 (1.35)2.57 (1.00)2.20 (1.19)2.87 (1.02)2.54 (1.25)2.80 (1.05)2.58 (1.27)
    Table 6. Point-biserial correlation analysis between the System Usability Scale (SUS) scores and product functions (N=388).
    Product functionPoint-biserial correlation

    		rP value
    Activity type identification.11.02
    Expandable new features−.11.02
    Floor climbing.03.58
    Running track−.04.40
    Professional fitness record−.09.08
    Heart rate−.09.07
    Sleep monitoring.09.07
    Diet record.04.45
    Inactivity notification.01.81
    Incoming call alert.01.85
    Text message display−.06.22
    Time display−.06.22
    Remote control photograph−.05.35
    Sports achievement sharing.04.40
    Bluetooth synchronization.08.13
    Table 7. ANOVA tests of the System Usability Scale (SUS) scores of the seven devices and of the volunteer’s demographic information (profession, length of time the device was used).
    MeasuresSum of squaresDegrees of freedomMean squareF (df1,df2)P value
    SUS scores of devices





    		Between groups1671.526278.590.98 (6,381).44

    		Within groups108,057.91381283.62—a—

    		Total109,729.43387———
    Profession





    		Between groups5618.2311510.751.84 (11,376).04

    		Within groups104,111.19376276.89——

    		Total109,729.43387———
    Length of time device used





    		Between groups12,581.1962096.868.22 (6,381).001

    		Within groups97,148.24381254.98——

    		Total109,729.43387———

    aNot applicable.

    First, we found that there was no statistically significant difference between the SUS scores of the different brands (see Table 7), but there were significant differences in the SUS scores according to occupation and user experience (length of time the device was used). No significant difference was found for other features such as gender, age, education, and monthly income, as detailed in Multimedia Appendix 5. Second, we further used independent sample t tests to compare the mean SUS scores of respondents in the health care and internet industries; this indicated that the former scores were significantly higher than the latter. The SUS scores for the health care industry (n=92) were mean 68.7 (SD 14.5, range 35-100); for the internet industry (n=114) they were mean 61.1 (SD 17.9, range 5-100; t204=–3.24, P=.001). Third, we used Pearson product moment correlation method to explore the relationship between the SUS scores and the length of time the device was used. The SUS score had a moderate positive correlation (r=.32, P<.001) with user experience. In other words, the SUS score was related to a volunteer’s user experience (how long they had used the product). Other factors had no significant influence on the SUS score.


    Low System Usability Scale Scores for Seven Wearable Devices

    The mean SUS score for the seven devices was 64.3 (SD 16.8). Although the Huawei Honor Band B2 received the highest score (mean 67.6, SD 16.1), based on the ANOVA analysis, there was no statistically significant difference in the mean SUS score of the seven products. Based on the SUS score, we believe that these products have little difference in terms of the ease of use. Aaron et al from AT&T Labs [61] added one question to the end of the SUS questionnaire, “Overall, I would rate the user friendliness of this product as…” This allowed users to rate the user interface using adjectives such as poor, OK, and good. The purpose of this was to associate the SUS with these adjectives. According to the data in this research, the user interfaces rated as “good” by users received a mean SUS score of 71.4. Therefore, the usability of all seven wearable devices was “good” or “OK.” The Huawei Honor Band B2, which received the highest SUS score of 67.6 (SD 16.1), was also in the “good” range, whereas the Apple Watch received the lowest score of 61.3 (SD 14.6). Although the seven brands received different SUS scores, the ANOVA test of the SUS scores versus device brands revealed no significant correlation between them. We believe that the usability of a wearable health tracking device is moderate with no significant differences between brands. These products are immature and additional breakthroughs in the core technology are required. In this sector, there is no leading brand with an absolute competitive edge, and there is no significant difference between domestic and foreign brands.

    Correlation Between System Usability Scale Score and Demographic Characteristics

    The SPSS analysis indicated that the SUS scores for wearable devices are related to demographic characteristics. Factors such as user experience and profession may affect the scores. The results also revealed that user experience (the time length the device was used) could be an important factor affecting the SUS score. We found that there was a moderate positive correlation between the SUS scores and user experience (r=.32, P<.001); this matches the conclusions of MacDorman et al [62] and Kortum et al [63]. The participants in medical and health care industries gave a significantly higher score (mean 68.7, SD 14.5) than the participants in other industries, particularly those in the internet industry (mean 61.1, SD 17.9), with the former giving scores that approached the “good” level (71.4) in the acceptance diagram [61]. We conducted independent sample t tests to compare the mean SUS scores of participants in the health care and internet industries, which indicated that the former score was significantly better than the latter (P<.001). This suggests that in our sample set medical and health care employees evaluated the devices more highly and that their acceptance level of wearable device usability was higher than internet employees, indirectly showing that the demand for wearable devices is more urgent in health-related industries, a potential reason or phenomenon worthy of further research and analysis in the future. This may be due to the serious shortage of medical resources in China, especially in remote areas. The gap between supply and demand provides opportunities for mobile medical-based wearable equipment, whereas the rapid development of mobile internet and big data technologies provide the support required for the development of mobile medical treatments. In the future, patients with chronic diseases such as coronary heart disease, hypertension, and diabetes will not only receive drug treatment, but integrated disease management programs including remote monitoring, tele-treatment plan adjustments, and lifestyle management through wearable technology. Previous SUS research [61] indicated that no control was needed for gender or that gender was not a key influencing factor for consumer-grade products. Our result has validated this conclusion.

    Correlation Between System Usability Scale Score and Product Characteristics

    Some characteristics of wearable devices are related to the SUS scores. Our study found that the intelligent recognition of activity type had a very weak positive correlation with the SUS score, and that expandable new functions and price have a very weak negative correlation with the SUS score. We believe that these findings, to some extent, show a “user portrait” of Chinese wearable device consumers who are likely to prefer devices that are functional and easy to operate (a weak positive correlation between the SUS score and the intelligent recognition function of the device). The price was a favorable factor, although it was not strong (the SUS score had a weak negative correlation with the price of device). We found that this is in line with the current market strategy of the Mi Band bracelet which applies the lowest price (only US $12) and the most core functions (such as steps, exercise time, sleep time, and resting time) to maximize the market share. Using this strategy, Mi Technologies Inc sold more than 3.6 million Mi Band bracelets in China in the first quarter of 2017, surpassing the giant manufacturers Fitbit, Apple, and Samsung to become the world’s largest wearable device maker [64]. In addition, we found that Q3 and Q7 (see Multimedia Appendix 1) had the highest scores: 2.97 and 2.87, respectively (see Table 5 and Multimedia Appendix 2). These scores were related to the ease of use (accessibility) of the product and the fact that people are generally more inclined to use products that do not require experience or focus and do not produce, or produce less, cognitive stress [65]. We think that this may be one reason for the high short-term usage rate of wearable devices. Although the use of wearable devices can enhance the ability to monitor the user’s behavior, if they perform similar activities and movements every day then, over time, it becomes easier to stop using the device as the user’s perception of the link between the results of the behavior intervention and the device monitoring is lost [66]. In order to prolong the use-life cycle and long-term wear rate of wearable devices, we need to integrate more cognitive behavior change strategies and functions to promote and consolidate the changes in users’ habits. This includes the identification of behavioral disorders and adjustment of cognitive attitudes [67]. At the same time, we believe that the low SUS scores of the current fitness wearable devices are due to the users’ dissatisfaction with the function and effect of the equipment, or their lack of cognitive motivation to change their behavior for a healthy lifestyle [22]. Therefore, we suggest that developers focus on the core requirements of the user, combined with social support, environmental support factors, customizable self-monitoring schemes, and personalized feedback, which should be integrated into the wearable device’s regular monitoring functions. In this way, we can help to improve the user’s overall usability evaluation of such products, prolong their use time, increase product adhesion, and cultivate user loyalty.

    Implications of Feature Analysis: Function Homogeneity, Awaiting Killer Function

    Wearable devices have great potential in personal health management and clinical care. In future, the cost of wearable devices will decrease, such as the Mi Band which is sold for only US $12. Wearable devices can record sports data, help set exercise goals, and avoid excessive movement, making it easy to obtain sports performance data and maintaining and promoting the passion for sport. In addition, wearable devices can play a role in clinical practice. An obvious characteristic of wearable devices is the continuous monitoring of health data, which can provide clues for disease diagnosis, identify disease symptoms in a timely manner, and help doctors to make more comprehensive and accurate judgments.

    Current mainstream wearable devices, including the seven brands in this study, are powerful; they not only track and record health data, such as sports data, sleep, and heart rate, but also provide numerous additional functions (including call notification, social media sharing, and Bluetooth). However, the function and feature analyses show that their functions are homogeneous and most concentrated on three functions: activity, reminders, and health monitoring. From this point of view, the homogenization of the product functions is very serious, and the functions of each product are lacking. There are no killer functions that solve major health management or clinical questions. Accordingly, the perceived usability of mainstream wearable devices is unsatisfactory and customer loyalty is not high. In fact, some recent studies have shown that many users use activity trackers for only a short time [68].

    Characteristics and Limitation of This Research

    In this research, the features of different devices were compared and their usability evaluated using the SUS. Compared to similar studies, we investigated the most extensive range of devices and used the largest group of participants. However, this research has its limitations. First, we adopted the snowball convenient randomization method; the participants were recruited based on the research teams’ circle of WeChat friends. Although the final respondents recruited were distributed across the country, this sample is not sufficient to represent consumers of wearable devices nationwide. Nonetheless, since our sample size was large and the responses were autonomous, we believe that our group still has a certain degree of representativeness. This is reinforced by the fact that the proportion of each brand in our study (Mi 31.4%, Huawei 12.1%, and Samsung 9.3%) is similar to the market share of intelligent wristbands in the 2014 report by Tencent ISUX [69] (Mi 34.3%, Huawei 12.5%, and Samsung 10.5%). Second, the survey was conducted using online questionnaires. Although this method is more convenient, faster, and more efficient than a conventional paper questionnaire in terms of distribution, collection, and analysis, many factors are uncontrollable and some factors, such as behavioral cognitive factors [70], social network factors [71], and environmental support factors [72], may affect the results.

    Conclusions

    The homogenization of wearable device functions is obvious, the usability of popular wearable devices on the market is unsatisfactory, and these devices have not been completely accepted by consumers. Usability is similar among the different brands with no absolute leader. No significant differences between domestic and international brands were observed. A consumer’s SUS score for wearable devices is related to their personal situation rather than the device brand. Device manufacturers should put more effort into developing innovative functions and improving the usability of their products.

    Acknowledgments

    We thank all volunteers who participated in this research. This study was sponsored by the National Natural Science Foundation of China (grants #81471756, #81771937, and #81871455) and the Medical and Health Planning Project of Zhejiang Province of China (grant #2017KY386).

    Authors' Contributions

    XZ and DX were responsible for the questionnaire preparation and data collection; XL was responsible for literature review; JL was responsible for the data analysis and manuscript draft; JF and BT supervised the data analysis and interpretation; and JL was responsible for study design and revision of the manuscript. All authors reviewed this final manuscript and expressed agreement regarding its content.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    System Usability Scale (SUS) questionnaire design and scoring method.

    PDF File (Adobe PDF File), 33KB
    ‎

    Multimedia Appendix 2

    System Usability Scale (SUS) score of each question for each device.

    PDF File (Adobe PDF File), 30KB
    ‎

    Multimedia Appendix 3

    System Usability Scale (SUS) scores for all devices.

    PDF File (Adobe PDF File), 511KB
    ‎

    Multimedia Appendix 4

    System Usability Scale (SUS) scores for all questions.

    PDF File (Adobe PDF File), 436KB
    ‎

    Multimedia Appendix 5

    ANOVA tests of the System Usability Scale (SUS) scores and the volunteer’s demographic information.

    PDF File (Adobe PDF File), 32KB
    1. Sungmee P, Jayaraman S. A transdisciplinary approach to wearables, big data and quality of life. Conf Proc IEEE Eng Med Biol Soc 2014;2014:4155-4158. [CrossRef] [Medline]
    2. Bateman DR, Srinivas B, Emmett TW, Schleyer TK, Holden RJ, Hendrie HC, et al. Categorizing health outcomes and efficacy of mHealth apps for persons with cognitive impairment: a systematic review. J Med Internet Res 2017 Aug 30;19(8):e301. [CrossRef] [Medline]
    3. Danova T. Business Insider. 2015 Jan 07. Apple Watch will control nearly half of the entire smartwatch market by 2017   URL: http://www.businessinsider.com/wearable-computing-market-statistics-2015-1 [accessed 2018-03-05] [WebCite Cache]
    4. Shirer M, Liamas R, Ubrani J. IDC. 2018 Jun 04. Wearable device shipments slow in Q1 2018 as consumers shift from basic wearables to smarter devices, according to IDC   URL: http://www.idc.com/getdoc.jsp?containerId=prUS43900918 [accessed 2018-08-21] [WebCite Cache]
    5. Waltzer S. Strategy Analytics. 2018 Aug 02. Global smartwatch vendor market share by region: Q2 2018   URL: https://tinyurl.com/y72zuydk [accessed 2018-08-21] [WebCite Cache]
    6. Danova T. Yahoo! Finance. 2014 Oct 29. The Wearable Computing Market Report: growth trends, consumer attitudes, and why smartwatches will dominate   URL: https://tinyurl.com/y86hey85 [accessed 2018-03-05] [WebCite Cache]
    7. 36kr. 2015 Jul 03. Tencent ISUX released domestic wearable devices report, the user retention rate is low, professional is the main demand   URL: http://36kr.com/p/5035222.html [accessed 2018-03-08] [WebCite Cache]
    8. Brakenridge CL, Healy GN, Winkler EA, Fjeldsoe BS. Usage, acceptability, and effectiveness of an activity tracker in a randomized trial of a workplace sitting intervention: mixed-methods evaluation. Interact J Med Res 2018 Mar 02;7(1):e5 [FREE Full text] [CrossRef] [Medline]
    9. Koydemir HC, Ozcan A. Wearable and implantable sensors for biomedical applications. Annual Rev Anal Chem 2018 Jun 12;11(1):127-146. [CrossRef]
    10. Niwano S, Sekiguchi Y, Ishii Y, Iwasaki Y, Kato R, Okamura H, et al. Clinical usefulness of wearable cardioverter defibrillator (WCD) and current understanding of its clinical indication in Japan. Circ J 2018;82(6):1481-1486. [CrossRef]
    11. Shen J, Naeim A. Telehealth in older adults with cancer in the United States: The emerging use of wearable sensors. J Geriatr Oncol 2017 Nov;8(6):437-442. [CrossRef] [Medline]
    12. Liu Y, Wang H, Zhao W, Zhang M, Qin H, Xie Y. Flexible, stretchable sensors for wearable health monitoring: sensing mechanisms, materials, fabrication strategies and features. Sensors (Basel) 2018 Feb 22;18(2) [FREE Full text] [CrossRef] [Medline]
    13. Nenggan Z, Zhaohui W, Man L, Yang L, Gang P. Infrastructure and reliability analysis of electric networks for e-Textiles. IEEE Trans Syst Man Cybern C 2010 Jan;40(1):36-51. [CrossRef]
    14. Chen J, Lum PS. Pilot testing of the spring operated wearable enhancer for arm rehabilitation (SpringWear). J Neuroeng Rehabil 2018 Mar 02;15(1):13 [FREE Full text] [CrossRef] [Medline]
    15. Bell W, Colaiezzi BA, Prata CS, Coates JC. Scaling up dietary data for decision-making in low-income countries: new technological frontiers. Adv Nutr 2017 Nov;8(6):916-932 [FREE Full text] [CrossRef] [Medline]
    16. Lee H, Lee H, Whang M. An enhanced method to estimate heart rate from seismocardiography via ensemble averaging of body movements at six degrees of freedom. Sensors (Basel) 2018 Jan 15;18(1) [FREE Full text] [CrossRef] [Medline]
    17. Lee H, Hong YJ, Baik S, Hyeon T, Kim D. Enzyme-based glucose sensor: from invasive to wearable device. Adv Healthc Mater 2018 Apr;7(8):e1701150. [CrossRef] [Medline]
    18. Lee J, Lee DH, Oh TJ, Kim KM, Choi SH, Lim S, et al. Clinical feasibility of continuously monitored data for heart rate, physical activity, and sleeping by wearable activity trackers in patients with thyrotoxicosis: protocol for a prospective longitudinal observational study. JMIR Res Protoc 2018 Feb 21;7(2):e49 [FREE Full text] [CrossRef] [Medline]
    19. Luo X, Fan W. Medical wearable devices and the health care model innovation. Chinese J Health Manage 2016;2016(1):70-75. [CrossRef]
    20. Kirk SR. Wearable medical technology-the future of personalized medicine is here. Chinese J Med Manage Sci 2016;6(1):34-37. [CrossRef]
    21. Phillips SM, Cadmus-Bertram L, Rosenberg D, Buman MP, Lynch BM. Wearable technology and physical activity in chronic disease: opportunities and challenges. Am J Prev Med 2018 Jan;54(1):144-150. [CrossRef] [Medline]
    22. Sullivan AN, Lachman ME. Behavior change with fitness technology in sedentary adults: a review of the evidence for increasing physical activity. Front Public Health 2016;4:289 [FREE Full text] [CrossRef] [Medline]
    23. Wen D, Zhang X, Liu X, Lei J. Evaluating the consistency of current mainstream wearable devices in health monitoring: a comparison under free-living conditions. J Med Internet Res 2017 Dec 07;19(3):e68 [FREE Full text] [CrossRef] [Medline]
    24. Gruwez A, Libert W, Ameye L, Bruyneel M. Reliability of commercially available sleep and activity trackers with manual switch-to-sleep mode activation in free-living healthy individuals. Int J Med Inform 2017 Dec;102:87-92. [CrossRef] [Medline]
    25. Cadmus-Bertram L, Gangnon R, Wirkus EJ, Thraen-Borowski KM, Gorzelitz-Liebhauser J. The accuracy of heart rate monitoring by some wrist-worn activity trackers. Ann Intern Med 2017 Apr 18;166(8):610-612 [FREE Full text] [CrossRef] [Medline]
    26. Salisbury JP, Keshav NU, Sossong AD, Sahin NT. Concussion assessment with smartglasses: validation study of balance measurement toward a lightweight, multimodal, field-ready platform. JMIR Mhealth Uhealth 2018 Jan 23;6(1):e15 [FREE Full text] [CrossRef] [Medline]
    27. Xie J, Wen D, Liang L, Jia Y, Gao L, Lei J. Evaluating the validity of current mainstream wearable devices in fitness tracking under various physical activities: comparative study. JMIR Mhealth Uhealth 2018 Apr 12;6(4):e94 [FREE Full text] [CrossRef] [Medline]
    28. Wen D, Zhang X, Lei J. Consumers' perceived attitudes to wearable devices in health monitoring in China: a survey study. Comput Methods Programs Biomed 2017 Mar;140:131-137. [CrossRef] [Medline]
    29. Mercer K, Giangregorio L, Schneider E, Chilana P, Li M, Grindrod K. Acceptance of commercially available wearable activity trackers among adults aged over 50 and with chronic illness: a mixed-methods evaluation. JMIR Mhealth Uhealth 2016 Jan 27;4(1):e7 [FREE Full text] [CrossRef] [Medline]
    30. Usability Partners. 1998. ISO Standards in usability and user-centred design—ISO 9241-11 (1998) guidance in usability   URL: http://www.usabilitypartners.se/about-usability/iso-standards [accessed 2018-03-08] [WebCite Cache]
    31. Yan Y, Wang G, Liu S, Wu H, Zhang Q. Usability evaluation of infusion pump based on system usability scale. China Medical Devices 2012;10(1):25-27. [CrossRef]
    32. Khajouei R, Hajesmaeel Gohari S, Mirzaee M. Comparison of two heuristic evaluation methods for evaluating the usability of health information systems. J Biomed Inform 2018 Apr;80:37-42. [CrossRef] [Medline]
    33. Khajouei R, Zahiri Esfahani M, Jahani Y. Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. J Am Med Inform Assoc 2017 Apr 01;24(e1):e55-e60. [CrossRef] [Medline]
    34. Hudson D, Kushniruk A, Borycki E, Zuege DJ. Physician satisfaction with a critical care clinical information system using a multimethod evaluation of usability. Int J Med Inform 2018 Apr;112:131-136. [CrossRef] [Medline]
    35. Ho GW. Examining perceptions and attitudes. West J Nurs Res 2017 Dec;39(5):674-689. [CrossRef] [Medline]
    36. Hoonakker PL, Carayon P, Walker JM. Measurement of CPOE end-user satisfaction among ICU physicians and nurses. Appl Clin Inform 2010 Jan 01;1(3):268-285 [FREE Full text] [CrossRef] [Medline]
    37. Alfonsson S, Maathz P, Hursti T. Interformat reliability of digital psychiatric self-report questionnaires: a systematic review. J Med Internet Res 2014 Dec 03;16(12):e268 [FREE Full text] [CrossRef] [Medline]
    38. Li J, Talaei-Khoei A, Seale H, Ray P, Macintyre CR. Health care provider adoption of eHealth: systematic literature review. Interact J Med Res 2013 Apr 16;2(1):e7 [FREE Full text] [CrossRef] [Medline]
    39. Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res 2004 Sep 15;6(3):e29 [FREE Full text] [CrossRef] [Medline]
    40. Johnson C, Johnston D, Crowley P, Culbertson H, Rippen H, Damico D. EHR Usability Toolkit: A Background Report on Usability and Electronic Health Records. Rockville, MD: Agency for Healthcare Research and Quality; 2011.   URL: https:/​/healthit.​ahrq.gov/​sites/​default/​files/​docs/​citation/​EHR_Usability_Toolkit_Background_Report.​pdf [accessed 2018-08-01] [WebCite Cache]
    41. Jordan P, Jordan JW, McClelland I. Usability Evaluation in Industry. London: Taylor & Francis; 1996.
    42. Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum-Comput Int 1995 Jan;7(1):57-78. [CrossRef]
    43. Lund A. ResearchGate. 2001. Measuring usability with the USE questionnaire   URL: https:/​/www.​researchgate.net/​profile/​Arnold_Lund/​publication/​230786746_Measuring_Usability_with_the_USE_Questionnaire/​links/​56e5a90e08ae98445c21561c/​Measuring-Usability-with-the-USE-Questionnaire.​pdf [accessed 2018-08-01] [WebCite Cache]
    44. Kirakowski J, Cierlik B. Measuring the usability of web sites. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 1998 Oct 02 Presented at: 42nd Human Factors and Ergonomics Society Annual Meeting; Oct 5-9, 1998; Chicago, IL p. 424-428. [CrossRef]
    45. McCallum C, Rooksby J, Gray CM. Evaluating the impact of physical activity apps and wearables: interdisciplinary review. JMIR Mhealth Uhealth 2018 Mar 23;6(3):e58 [FREE Full text] [CrossRef] [Medline]
    46. Schenkenfelder R, Selinger S. A Comparison of Multiple Wearable Devices Regarding their User Experience During Running.   URL: http://ffhoarep.fh-ooe.at/bitstream/123456789/686/1/125_218_Schenkenfelder_FullPaper_en_Final.pdf [accessed 2018-03-08] [WebCite Cache]
    47. Kaewkannate K, Kim S. A comparison of wearable fitness devices. BMC Public Health 2016 Dec 24;16:433 [FREE Full text] [CrossRef] [Medline]
    48. Canalys. 2015 May 20. Media alert: Fitbit maintains leadership share of wearable band market before Apple Watch entrance   URL: https://tinyurl.com/y9bus445 [accessed 2018-03-08] [WebCite Cache]
    49. Canalys. 2014 May 21. Fitbit accounted for nearly half of global wearable band shipments in Q1 2014   URL: http://www.canalys.com/newsroom/fitbit-accounted-nearly-half-global-wearable-band-shipments-q1-2014 [accessed 2018-03-08] [WebCite Cache]
    50. Dolan B. MobiHealthNews. 2014 Jan 15. Fitbit, Jawbone, Nike had 97 percent of fitness tracker retail sales in 2013   URL: http:/​/www.​mobihealthnews.com/​28825/​fitbit-jawbone-nike-had-97-percent-of-fitness-tracker-retail-sales-in-2013 [accessed 2018-03-08] [WebCite Cache]
    51. Bevan N, Carter J, Harker S. ISO 9241-11 Revised: what have we learnt about usability since 1998? Lect Notes Comput Sci 2015;9169(1):143-151. [CrossRef]
    52. Flavián C, Guinalíu M, Gurrea R. The role played by perceived usability, satisfaction and consumer trust on website loyalty. Inform Manage 2006 Jan;43(1):1-14. [CrossRef]
    53. Greene K, Byrne M, Everett S. A comparison of usability between voting methods. In: EVT. 2006 Presented at: USENIX/Accurate Electronic Voting Technology Workshop 2006 on Electronic Voting Technology Workshop; July 31-Aug 4, 2006; Vancouver, BC.
    54. Campbell BA, Tossell CC, Byrne MD, Kortum P. Voting on a smartphone: evaluating the usability of an optimized voting system for handheld mobile devices. In: Proc Hum Factors Ergon Soc. 2011 Sep 01 Presented at: Human Factors and Ergonomics Society Annual Meeting; Sep 19-23, 2011; Las Vegas p. 1100-1104. [CrossRef]
    55. Trafton J, Martins S, Michel M, Lewis E, Wang D, Combs A, et al. Evaluation of the acceptability and usability of a decision support system to encourage safe and effective use of opioid therapy for chronic, noncancer pain by primary care providers. Pain Med 2010 Apr;11(4):575-585. [CrossRef] [Medline]
    56. Lewis J, Sauro J. The factor structure of the system usability scale. Lect Notes Comput Sc 2009;5619(1):94-103. [CrossRef]
    57. WJX. Introduction of Sojump   URL: http://www.wjx.cn/html/aboutus.aspx [accessed 2018-03-08] [WebCite Cache]
    58. Liang J, Xian D, Liu X, Fu J, Zhang X, Tang B, et al. Comprehensive evaluation questionnaire for intelligent wearable devices. 2018.   URL: https://www.wjx.cn/jq/5733344.aspx [accessed 2018-11-02] [WebCite Cache]
    59. Wong WS, Chen PP, Chow YF, Wong S, Fielding R. A study of the reliability and concurrent validity of the Chinese version of the Pain Medication Attitude Questionnaire (ChPMAQ) in a sample of Chinese patients with chronic pain. Pain Med 2016 Jun;17(6):1137-1144. [CrossRef] [Medline]
    60. Cohen J. Statistical Power Analysis for the Behavioral Sciences. London: Taylor & Francis; 1988.
    61. Aaron B, Philip K, James M. Journal of Usability Studies. 2009. Determining what individual SUS scores mean: adding an adjective rating scale   URL: https://tinyurl.com/yax6fx5n [accessed 2018-08-01] [WebCite Cache]
    62. MacDorman KF, Whalen TJ, Ho C, Patel H. An improved usability measure based on novice and expert performance. Int J Hum-Comput Int 2011 Feb 08;27(3):280-302. [CrossRef]
    63. Kortum PT, Bangor A. Usability ratings for everyday products measured with the system usability scale. Int J Hum-Comput Int 2013 Jan;29(2):67-76. [CrossRef]
    64. IDC. 2017 Oct. Worldwide wearables market shares, 1H17: Xiaomi leads the market by volume, Apple leads by value   URL: http://www.idc.com/getdoc.jsp?containerId=US43141317 [accessed 2018-06-01] [WebCite Cache]
    65. Lidwell W, Holden K, Butler J. Universal Principles of Design, Revised and Updated 125 Ways to Enhance Usability, Influence Perception, Increase Appeal, Make Better Design Decisions. Beverly, MA: Rockport Publisher; 2010.
    66. Karapanos E, Gouveia R, Hassenzahl M, Forlizzi J. Wellbeing in the making: peoples' experiences with wearable activity trackers. Psychol Well Being 2016;6:4 [FREE Full text] [CrossRef] [Medline]
    67. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95. [CrossRef] [Medline]
    68. Patel MS, Asch DA, Volpp KG. Wearable devices as facilitators, not drivers, of health behavior change. JAMA 2015 Feb 3;313(5):459-460. [CrossRef] [Medline]
    69. Tencent ISUX. 2014. 2014 smart wearable market white paper   URL: http://digi.tech.qq.com/a/20141231/032756.html [accessed 2018-03-08] [WebCite Cache]
    70. Herning MM, Cook JH, Schneider JK. Cognitive behavioral therapy to promote exercise behavior in older adults: implications for physical therapists. J Geriatr Phys Ther 2005;28(2):34-38. [Medline]
    71. Rovniak LS, Kong L, Hovell MF, Ding D, Sallis JF, Ray CA, et al. Engineering online and in-person social networks for physical activity: a randomized trial. Ann Behav Med 2016 Dec;50(6):885-897 [FREE Full text] [CrossRef] [Medline]
    72. Sallis JF, Cerin E, Conway TL, Adams MA, Frank LD, Pratt M, et al. Physical activity in relation to urban environments in 14 cities worldwide: a cross-sectional study. Lancet 2016 May 28;387(10034):2207-2217. [CrossRef] [Medline]

    ‎
    ANOVA: analysis of variance
    SUS: System Usability Scale

    Edited by G Eysenbach; submitted 17.05.18; peer-reviewed by A Sullivan, G Signorelli, J Quiroz; comments to author 13.06.18; revised version received 03.08.18; accepted 20.10.18; published 08.11.18

    Copyright

    ©Jun Liang, Deqiang Xian, Xingyu Liu, Jing Fu, Xingting Zhang, Buzhou Tang, Jianbo Lei. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 08.11.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.