This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Accurate, continuous heart rate measurements are important for health assessment, physical activity, and sporting performance, and the integration of heart rate measurements into wearable devices has extended its accessibility. Although the use of photoplethysmography technology is not new, the available data relating to the validity of measurement are limited, and the range of activities being performed is often restricted to one exercise domain and/or limited intensities.
The primary objective of this study was to assess the validity of the Polar OH1 and Fitbit Charge 3 devices for measuring heart rate during rest, light, moderate, vigorous, and sprint-type exercise.
A total of 20 healthy adults (9 female; height: mean 1.73 [SD 0.1] m; body mass: mean 71.6 [SD 11.0] kg; and age: mean 40 [SD 10] years) volunteered and provided written informed consent to participate in the study consisting of 2 trials. Trial 1 was split into 3 components: 15-minute sedentary activities, 10-minute cycling on a bicycle ergometer, and incremental exercise test to exhaustion on a motorized treadmill (18-42 minutes). Trial 2 was split into 2 components: 4 × 15-second maximal sprints on a cycle ergometer and 4 × 30- to 50-m sprints on a nonmotorized resistance treadmill. Data from the 3 devices were time-aligned, and the validity of Polar OH1 and Fitbit Charge 3 was assessed against Polar H10 (criterion device). Validity was evaluated using the Bland and Altman analysis, Pearson moment correlation coefficient, and mean absolute percentage error.
Overall, there was a very good correlation between the Polar OH1 and Polar H10 devices (
Compared with our criterion device, Polar OH1 was accurate at assessing heart rate, but the accuracy of Fitbit Charge 3 was generally poor. Polar OH1 performed worse during trial 2 compared with the activities in trial 1, and the validity of the Fitbit Charge 3 device was particularly poor during our cycling exercises.
Consumer wearables constitute an ever-evolving industry with applications across multiple sectors of society. One key demand for wearable technology is to monitor and use parameters associated with physical activity for sport performance, health, and well-being. For example, a recent systematic review and meta-analysis concluded that the utilization of a consumer-based wearable activity tracker, used either as the primary component of an intervention or as part of a broader physical activity intervention, has the potential to increase participation in physical activities [
The autonomic nervous system (ANS) is interlinked with many physiological systems, and heart rate (HR) measures are considered surrogate markers of ANS status [
Polar OH1 (Polar Electro Oy) and Fitbit Charge 3 (Fitbit Inc) are two of the latest available devices that use PPG technology. They differ in that the Fitbit device constitutes a watch worn on the wrist, whereas Polar OH1 may be worn on either the forearm or the temple and is a stand-alone optical HR sensor. It has previously been shown that Polar OH1 is accurate at measuring HR during moderate-intensity yoga compared with a Polar H7 chest strap (mean bias: −0.76 beats·min-1; 95% limits of agreement [LoAs]: −3.83 to 5.35 beats·min-1) [
Therefore, the objective of this study was to assess the validity of Polar OH1 and Fitbit Charge 3 for measuring HR during rest, cycling, walking, and running activities. Validity during cycling, walking, and running activities was assessed across a range of intensities, including light, moderate, vigorous, and uniquely sprint-type exercise.
A total of 20 healthy adults (9 female; height: mean 1.73 [SD 0.1] m; body mass: mean 71.6 [SD 11.0] kg; fat mass: mean 17.0% [SD 7.8%]; age: mean 40 [SD 10] years; and International Physical Activity Questionnaire-Short Form [IPAQ-SF] [
The study consisted of 2 visits to the Active Health Exercise Laboratory at the University of the Highlands and Islands, Inverness, which were conducted a minimum of 3 days apart. Participants were asked to refrain from intense physical activity (24 h), alcohol (12 h), caffeine (6 h), and food (3 h) before arrival at the laboratory for each visit. During visit 1, participants completed 15 minutes of sedentary activities, 10 minutes of cycling, and a treadmill protocol. For visit 2, each participant completed cycling and treadmill-based HIIE protocols. During each of the trials, participants’ HRs were continuously monitored by a Polar H10 heart rate monitor (Polar Electro Oy; criterion measure), Polar OH1, and Fitbit Charge 3.
Polar H10 was used as the criterion device. The HR sensor was attached to a Polar Pro heart rate strap placed over the sternum. Polar H10 live data were transmitted to a spiroergometry system (METALYZER 3B, CORTEX Biophysik GmbH), which recorded HR data at 1-second intervals. Polar H10 has previously been found to be valid when compared with ECG, with a correlation of
Polar OH1 was attached to an arm band and strapped securely to the nondominant forearm, according to the manufacturer’s instructions. HR data were recorded at 1-second intervals using 6 light-emitting diode sensors and live transmitted via Bluetooth to a smartphone with the Polar Beat app (Polar Electro Oy). After completion of each visit, data were uploaded to the Polar Flow web service (Polar Electro Oy).
Fitbit Charge 3 was attached to the nondominant wrist, 2-finger widths above the ulnar styloid process, following the manufacturer’s instructions. According to the manufacturer, Fitbit Charge 3 uses
Upon arrival at the laboratory, participants were briefed on the protocol before anthropometric variables were measured. Height, body mass, and body composition were measured with participants wearing light exercise clothing. Height was measured to the nearest 0.1 cm using a portable stadiometer (Model 213, Seca), body mass was measured to the nearest 0.1 kg using a floor scale (Model 875, Seca), and body composition was assessed using bioelectrical impedance (MC780MA, Tanita Corporation). The HR measuring devices were subsequently attached as described above.
The trial was split into 3 components. Component 1 consisted of 15 minutes of sedentary activities. The participants remained seated in a chair for the duration of component 1 and were instructed to keep their movement to a minimum. During the first 5 minutes, participants sat quietly before watching 5 minutes of a nature documentary. All participants watched the same 5-minute section of the documentary. Finally, the participants completed a cognitive task where they were provided with a choice of either a word search, crossword, or sudoku puzzle and were instructed to complete as much as possible within the time frame. Before component 2, a target HR range was calculated to determine 60%-85% of HR reserve target intensity using the equation:
Resting HR was calculated as an average of the final minute of sitting quietly, whereas the age-predicted maximum HR was calculated using the following formula by Gellish et al [
Component 2 consisted of 10-minute cycling on a bicycle ergometer (Lode Corival). During the cycling tasks, each participant completed 5 minutes of light work at 50 W. After 5 minutes, the intensity was increased to substantially elevate HR to between 60% and 85% of the HR reserve. Between components 2 and 3, participants rested for a minimum of 10 minutes.
Component 3 consisted of an incremental exercise test on a motorized treadmill (Skillrun, Technogym). The treadmill test consisted of a range of intensities from light to high intensity, increasing at 3-minute intervals until volitional exhaustion. The initial phase of each test was continuous, beginning with walking at speeds of 4, 5, and 6 km·h-1 and then running at speeds of 8 and 10 km·h-1 at 1% gradient for 3 minutes. For those who were able to continue, a subsequent discontinuous phase of the test immediately followed the continuous phase. The discontinuous phase consisted of running for 3 minutes at 12, 14, 16, and 18 km·h-1 (or until volitional exhaustion), with each stage separated by 3 minutes of active recovery (walking at 4 km·h-1). The discontinuous phase was used to allow the participants to complete as many stages as possible. Participants were not required to complete all stages to be included in the analysis.
Upon arrival at visit 2, the devices were attached to the participants, as described in visit 1. Visit 2 was split into 2 components. Component 1 consisted of a 3-minute warm-up followed by 4 maximal sprints, each lasting 15 seconds and interspersed with 3-minute active recovery on a cycle ergometer (Wattbike Pro, Wattbike). The airbrake resistance was set at 1 during the warm-up and active recovery phases and was increased based on body mass, as per the manufacturer’s guidelines (
Component 2 was performed using the Technogym Skillrun treadmill’s
Data from the 3 devices were time-aligned and split into the following parts:
Visit 1: rest, light cycling, vigorous cycling, and treadmill
Visit 2: sprint cycling and sprint running
The validity of Polar OH1 and Fitbit Charge 3 was compared with the validity of Polar H10 (criterion device) for all data points and as an average HR for each segment. Data alignment and filtering were performed in R Studio using the packages dplyr and tidr. Before analysis, the normality of data was assessed using histograms and quantile-quantile plots. Validity was subsequently evaluated using the Bland and Altman [
Combined data across all activity types showed that Polar OH1 underestimated HR by 1 beat·min-1 (LoA: −20 to 19 beats·min-1) versus the Polar H10 device, and there was a very good correlation between the devices (
Validity of measuring heart rate with the Polar OH1 and Fitbit Charge 3 devices. Data are unaveraged across all data points (overall) and within each of the activity domains.
Device | Overall (n=35,639a) | Rest (n=5448a) | Cycling light (n=1903a) | Cycling hard (n=1928a) | Treadmill (n=14,489a) | Sprint cycling (n=5516a) | Sprint running (n=6355a) | |
|
||||||||
|
Heart rate (beats·min-1), mean (SD) | 114 (33) | 63 (10) | 89 (12) | 119 (16) | 124 (31) | 131 (25) | 123 (19) |
|
||||||||
|
Heart rate (beats·min-1), mean (SD) | 113 (33) | 63 (10) | 89 (12) | 118 (17) | 124 (30) | 125 (26) | 125 (20) |
|
Mean bias (beats·min-1) | −1 | 0 | 1 | −1 | 0 | −6 | 2 |
|
Limits of agreement (beats·min-1) | −20 to 19 | −4 to 4 | −5 to 7 | −7 to 5 | −9 to 8 | −38 to 27 | −27 to 31 |
|
MAPEb (%) | 0.4 | 0.2 | −0.8 | 0.6 | 0.2 | 3.9 | −1.9 |
|
Correlation coefficient | 0.957 | 0.974 | 0.983 | 0.985 | 0.990 | 0.794 | 0.722 |
|
95% CI of correlation coefficient | 0.956 to 0.958 | 0.973 to 0.975 | 0.981 to 0.984 | 0.983 to 0.987 | 0.990 to 0.991 | 0.784 to 0.803 | 0.710 to 0.734 |
|
||||||||
|
Heart rate (beats·min-1), mean (SD) | 107 (31) | 62 (9) | 81 (12) | 98 (24) | 123 (27) | 111 (30) | 114 (16) |
|
Mean bias (beats·min-1) | −7 | −1 | −7 | −21 | −1 | −20 | −10 |
|
Limits of agreement (beats·min-1) | −46 to 33 | −7 to 5 | −38 to 23 | −70 to 29 | −30 to 28 | −79 to 40 | −49 to 30 |
|
MAPE (%) | −4.4 | −1.4 | −7.1 | −16.4 | 0.7 | −13.5 | −6.3 |
|
Correlation coefficient | 0.807 | 0.946 | 0.272 | 0.183 | 0.879 | 0.390 | 0.348 |
95% CI of correlation coefficient | 0.804 to 0.811 | 0.943 to 0.949 | 0.230 to 0.328 | 0.127 to 0.238 | 0.875 to 0.882 | 0.368 to 0.412 | 0.326 to 0.369 |
an: number of data points analyzed for each domain.
bMAPE: mean absolute percentage error.
Validity of measuring heart rate with Polar OH1 and Fitbit Charge 3 devices. Data are aggregated to a single data point for each of the activity domains. Data are analyzed for all data points (column “Overall”) and for each of the activity domains.
Device | Overall (n=111a) | Rest (n=19a) | Cycling light (n=19a) | Cycling hard (n=19a) | Treadmill (n=19a) | Sprint cycling (n=18a) | Sprint running (n=17a) | ||||||||
|
|||||||||||||||
|
Heart rate (beats·min-1), mean (SD) | 106 (27) | 62 (9) | 89 (11) | 119 (12) | 124 (14) | 128 (17) | 120 (12) | |||||||
|
|||||||||||||||
|
Heart rate (beats·min-1), mean (SD) | 105 (27) | 62 (9) | 89 (10) | 118 (12) | 124 (14) | 122 (17) | 121 (15) | |||||||
|
Mean bias (beats·min-1) | 1 | 0 | 0 | 1 | 0 | 5 | 1 | |||||||
|
Limits of agreement (beats·min-1) | −8 to 10 | 0 to 1 | −3 to 2 | −2 to 4 | −1 to 2 | −8 to 18 | −16 to 15 | |||||||
|
MAPEb (%) | 0.6 | 0.2 | 0.7 | −0.8 | 0.2 | −4.1 | 0.5 | |||||||
|
Correlation coefficient | 0.954 | 0.983 | 0.974 | 0.985 | 0.992 | 0.807 | 0.67 | |||||||
|
95% CI of correlation coefficient | 0.953 to 0.955 | 0.982 to 0.984 | 0.971 to 0.977 | 0.083 to 0.987 | 0.992 to 0.993 | 0.795 to 0.819 | 0.651 to 0.687 | |||||||
|
|||||||||||||||
|
Heart rate (beats·min-1), mean (SD) | 97 (26) | 61 (9) | 81 (10) | 100 (21) | 123 (10) | 109 (23) | 112 (11) | |||||||
|
Mean bias (beats·min-1) | −9 | −1 | −7 | −19 | −2 | −18 | −8 | |||||||
|
Limits of agreement (beats·min-1) | −41 to 23 | −2 to 0 | −36 to 22 | −63 to 26 | −13 to 10 | −63 to 26 | −28 to 12 | |||||||
|
MAPE (%) | 7.37 | −1.5 | −7 | −15 | −1 | −13.9 | −6 | |||||||
|
Correlation coefficient | 0.888 | 0.884 | −0.056 | 0.183 | 0.924 | 0.771 | 0.496 | |||||||
|
95% CI of correlation coefficient | 0.885 to 0.890 | 0.876 to 0.891 | −0.113 to 0.002 | 0.127 to 0.238 | 0.921 to 0.927 | 0.756 to 0.784 | 0.471 to 0.520 |
an: number of data points analyzed for each domain.
bMAPE: mean absolute percentage error.
The mean bias and LoA for Polar OH1 and the criterion device were consistent for visit 1 activities; however, the LoA was much wider during HIIE exercise (
Bland and Altman plots for unaveraged data across each activity domain. Subpart A shows data from Polar OH1, and subpart B shows data from Fitbit Charge 3. Solid blue line represents the mean bias, and blue dashed lines represent the limits of agreement.
The mean absolute percentage error for each participant for Polar OH1 and Fitbit Charge 3 across all available data points.
Participant | Polar OH1 (%) | Fitbit Charge 3 (%) |
1 | 0.22 | 6.34a |
2 | −3.92 | 5.30a |
3 | 1.37 | 2.80 |
4 | 0.06 | 2.06 |
5 | 0.31 | 4.01 |
6 | 1.70 | 1.53 |
7 | 0.22 | 2.35 |
8 | −0.44 | 2.59 |
9 | 0.71 | 3.14 |
10 | −0.09 | 2.83 |
11 | 0.59 | -3.25 |
12 | 1.81 | 4.00 |
13 | −0.10 | 3.91 |
14 | −0.09 | −5.43a |
15 | 0.69 | 14.90a |
16 | 0.87 | 5.97a |
17 | 0.19 | 2.75 |
18 | 1.18 | 11.40a |
19 | 2.58 | 15.80a |
Overall | 0.41 | 4.37 |
aExceeds 5% mean absolute percentage error threshold.
Individual traces during sprint cycling. Green line represents Polar H10 (criterion device), the blue line represents Polar OH1, and the red line represents Fitbit Charge 3. Traces show 4 peaks in heart rate for each of the sprints followed by a recovery period of 3 minutes.
To our knowledge, this is the first study to assess the validity of Fitbit Charge 3 and Polar OH1 across a range of activity types, including HIIE or SIE. The main findings were that Polar OH1 showed good agreement in assessing HR versus the criterion measure (Polar H10) across activity domains or types in trial 1, whereas the validity of Fitbit Charge 3 was only acceptable during rest and treadmill activities. Fitbit Charge 3 performed particularly poorly during cycling exercise, where the mean bias ranged from −7 to −21 beats·min-1, and LoA were very wide compared with other activity types. Finally, our data suggest that both Polar OH1 and Fitbit Charge 3 devices performed poorly during the visit 2 sprint cycling and sprint running protocols compared with the visit 1 activities.
The findings in this study suggest that Polar OH1 performs within acceptable tolerance limits for measuring HR during a range of activity types and intensities (ie, MAPE range 0%-4%). These findings are consistent with those of previous work [
This is the first published study to examine the validity of Fitbit Charge 3, although several studies have investigated the validity of its predecessors, Fitbit Charge HR (released 2015) and Fitbit Charge 2 (released 2016). We observed HR measurements that were within an acceptable percentage error range (0%-5%) during rest and the incremental treadmill test. However, Fitbit Charge 3 exhibited MAPE >5% during sprint running and during light, hard, and sprint cycling, with particularly large MAPE and mean bias observed during hard and sprint cycling. Previous studies have reported that Fitbit Charge HR and Fitbit Charge 2 underestimate HR in comparison with criterion devices during cycle-based activities [
We can only speculate as to why Fitbit Charge 3 performed worse than Polar OH1 during cycling and high-intensity exercise in this study. Olstad and Zinner [
This study included a variety of exercise intensities and both cycling and treadmill activities. In addition, few studies have examined the validity of consumer wearables in measuring HR during SIE, an increasingly prevalent exercise modality. However, despite providing novel insights into the accuracy of Fitbit Charge 3 and Polar OH1 in the detection of HR, this study has limitations. The sample size of this study, which is consistent with other similar studies [
In conclusion, our data suggest that Polar OH1 is a suitable method for measuring HR during cycling, walking, and running activities within a healthy population. In contrast, data pertaining to Fitbit Charge 3 should be interpreted with caution, particularly during cycling activities. This may have significant implications for exercise training or rehabilitation purposes, where attainment of exercise intensity is a key aspect for cardiorespiratory fitness progression or where safety considerations exist. Furthermore, both PPG sensors evaluated in this study performed worse during the SIE activities. Given the rise in popularity of HIIE or SIE, we recommend that more traditional ECG/HR monitors are used when performing these activities. For the general population and scientific community to appropriately interpret PPG data, researchers should continue to assess the validity of new and existing devices among various populations and settings.
Wattbike resistance settings.
Sprint running distance settings.
Sprint running treadmill belt resistance (parachute size) settings.
autonomic nervous system
electrocardiogram
high-intensity interval exercise
heart rate
International Physical Activity Questionnaire-Short Form
limits of agreement
mean absolute percentage error
photoplethysmography
sprint interval exercise
The authors would like to thank all participants who volunteered to participate in this study. DM, OG, DC and IM are supported by the European Union’s INTERREG VA Programme, managed by the Special European Union Programmes Body. DC is also supported by a grant from Highlands and Islands Enterprise (HMS 9353763). KH is partly funded by a grant from Highlands and Islands Enterprise (HMS 9353763) and partly funded by Inverness and Highland City-Region Deal.
DM, TG, IM, and DC conceptualized the study. DM, KH, AD, and DC collected the data. DM and DC analyzed the data and prepared the first draft of the paper. All authors have reviewed and approved the final manuscript.
None declared.