Abnormalities in body temperature are the key indicators for the prognosis of various illnesses [1-4] and the most common symptoms of defensive immune responses [4]. Abnormal body temperature patterns are informative and specific to poor progress during an illness, regardless of whether people meet the criteria for fever or not [5-10]. Large body temperature variations before the development of a fever indicate the evolution of sepsis syndrome [9]. A retrospective study found that afebrile critically ill patients with abnormal temperature patterns in any 24-hour period had a 4.43-fold risk for subsequent diagnosis of sepsis as compared to patients without such patterns, and the sensitivity and specificity of the abnormal temperature curves to predict sepsis were 0.69 and 0.76, respectively [8]. At the individual level, an unexplained variation in 1 SD increase in body temperature among outpatients is a significant predictor linked to 8.4% higher rate of 1-year mortality [11]. Therefore, body temperature is one of the most important vital signs for evaluating human health, and continuous monitoring should be done for the early identification of patients who are at a risk for poor prognosis [12,13].

Thermometers such as glass mercury thermometers, electronic thermometers, infrared ear thermometers, and infrared forehead thermometers are used to measure the human body temperature. They are usually used at predetermined intervals or when a patient’s condition changes [7,14]. Considering the equivalents of intravascular or direct brain thermometry, the measurement tools for continuous body temperature monitoring in clinical practice are invasive and include urinary catheters or esophageal temperature probes [15]. However, such invasive continuous body temperature measurement tools might not be applicable to the general population.

To date, wearable devices with the advantage of minimizing discomfort and interference with normal human activities have garnered increased attention in the field of continuous monitoring of body temperatures [13,16,17]. Different wearable devices, including skin-like arrays of precision temperature sensors or wearable adhesive devices such as watches, vest, patches, and earphones, have been developed to continuously examine the body temperature [18-21]. Among the human body parts, the wrist is the most responsive body part for wearable devices that measure thermal sensation [19,22]. However, the applications of wearable devices in the continuous monitoring of body temperature are still limited in terms of pilot studies [18,20,23]. In addition, the validity of wrist skin temperature monitoring using novel wearable devices in both laboratory settings and on human subjects has not been sufficiently investigated.

An early study conducted in 2006 was the first study to use iButton, which has a mean accuracy of 0.05°C through a validation study by using a water bath with different temperatures and a reference thermometer. This study further underscored the generalizability of laboratory findings to clinical settings [24]. Most studies were conducted to assess the feasibility of the wrist skin temperature measured by iButtons in terms of the human circadian system [25-27]. Two studies conducted on approximately 100 university students in Spain proved that the wrist temperature rhythm is a valuable index for assessing the circadian rhythm [25,26]. Another study conducted on 121 shift workers in Korea provided evidence that lower wrist temperature amplitudes appeared more in the nightshift workers than in the dayshift workers [27]. In addition, several studies were conducted to assess the effects of continuously monitoring body temperature by using wearable devices for the detection of worsening conditions in patients [6,10,23,28,29]. A prospective observational study consisting of 56 patients was carried out to explore the effects of a Holter device with both central and peripheral infrared temperatures being recorded and stored every minute for 24 hours. It was found that 0.7 peaks of fever per patient could be detected by continuous monitoring but this was unobserved during conventional care, and 16% of the patients considered afebrile by conventional care had at least one fever peak detected during continuous monitoring [29]. Another pilot study of pediatric patients indicated that 2 fever events could not be detected by routine temperature monitoring but they could be detected 12 hours earlier by a continuous temperature monitoring patch [23]. However, the limitations of these devices included a long response time due to their relatively large sizes, and the maximum sampling rate was 1 per minute [24]. Moreover, these studies did not show the reliability and validity of the wearable devices in terms of monitoring body surface temperature [25-27]. Therefore, the objective of this study was to validate the performance of HEARThermo, an innovative wearable device, which was developed to continuously monitor the body temperature in humans.

We used a multi-method research design in this study, which included 2 validation studies—one in the laboratory and one with human subjects. We conducted these studies to determine the reliability of HEARThermo in the laboratory setting and the agreement and validity of HEARThermo for application on human skin. The Institutional Review Board at the National Cheng Kung University Hospital (no. B-BR-106-044) approved this study’s protocol.

Snowball sampling was conducted to recruit people who were able to communicate in Taiwanese or Mandarin, who lived in the Tainan community, and who were willing to participate in this study. The exclusion criteria included people (1) with fever or if they experienced physical discomfort 3 days before participation in this study; (2) with severe brain injury, neurological disease, severe cardiovascular disease, ear structure problem, peripheral artery disease, or musculoskeletal disease in the limbs; (3) with mental disorders or cognitive disability; and (4) under medication with effects on the vital markers such as corticosteroids, nonsteroidal anti-inflammatory drugs, or anti-fever medications 4 hours before the commencement of this study.

We recruited 66 participants in this study and all of them completed the experimental study. The power analysis was calculated using “ICC.Sample.Size” packages by R Version 3.4.1 (R Statistics Software). The post hoc power analysis indicated a sample size of 66 participants to detect a high-power level (power=0.99).

We examined the differences between the groups by using analysis of variance and chi-square test for the categorical variables. Then, we calculated the measurement bias of each HEARThermo as the mean (SD) of the differences between the temperatures of HEARThermo and that of the water bath. Moreover, we calculated the linear regressions between the temperatures of the water bath and each HEARThermo to determine the relationship between the values of the wearable devices and those of the water bath. We applied the linear regression model for each HEARThermo to an independent data set to correct the values of each wearable device and tested the improvements in the values with a two-tailed paired t test by using the biases of the absolute values. To determine the test-retest reliability of the individual wearable device, we analyzed the intraclass correlation coefficient (ICC) of each HEARThermo. We analyzed the agreement between HEARThermo and the infrared skin thermometer by using Bland-Altman plots. Additionally, we calculated the means, namely bias, and SD, namely precision, of the differences, and 95% limits of agreement. Furthermore, we analyzed the criterion validity of the HEARThermo by using ICC two-way random effect model with 95% CIs. We conducted all the statistical analyses by using R Version 3.4.1.

We included 66 participants in Tainan City, Taiwan for this experimental study (Multimedia Appendix 1). A total of 325,022 data were observed, and 152,881 completed data were analyzed. The age of the participants ranged from 10 years to 77 years, with a mean (SD) age of 39.47 (19.02) years, and 39 out of the 66 participants (59%) were female. The mean (SD) height of these participants was 161.4 (10.85) cm, the mean (SD) weight was 60.82 (14.83) kg, the mean (SD) BMI was 23 (3.41), the mean (SD) body fat percentage was 26.61 (6.98)%, and the mean (SD) hand circumference was 15.38 (1.61) cm. The mean (SD) stabilized time after walking was 357.06 (192.24) seconds, and the mean (SD) stabilized time after running was 790.67 (259.69) seconds.

We tested 39 wearable devices thrice in a water bath at 16°C and 40°C. The mean (SD) measurement bias at 16°C amounted to −1.16 (1.84)°C (range –4.91°C to 2.18°C). The mean (SD) measurement bias at 40°C amounted to 1.21 (2.01)°C (range –2.39°C to 5.63°C). The ICC for the water bath at 16°C was 0.96, whereas the ICC for the water bath at 40°C was 0.98 (Table 1).

We conducted linear regressions to calibrate each wearable device. The intercept calibration coefficients ranged from –3.20 to 7.07°C and the slope calibration coefficients ranged from 0.77 to 1.11°C. The corrected measurement bias averaged –0.02 (SD 0.28)°C (range –1.79°C to 0.81°C). The values of each HEARThermo improved by the regression models were not significantly different from the temperatures of the water bath (P=.19).

Figure 3 shows the level of agreement of body surface temperatures between HEARThermo and the infrared skin thermometer. The mean (SD) temperature of HEARThermo was 31.94 (2.04)°C as compared with 30.43 (2.02)°C for the infrared skin thermometer. Bland-Altman analyses indicated that the bias of HEARThermo was 1.51°C (95% CI 1.50-1.51) with a precision of 1.45. The limit of agreement was –1.34°C (95% CI –1.35 to –1.32) to 4.35°C (95% CI 4.34-4.36). Table 2 shows the ICC values between the wearable devices and the infrared skin thermometer at different room temperatures, which included ≤19.9°C, 20°C-27.9°C, and ≥28°C. The ICC was high with an estimate of 0.72 for the room temperatures of 20°C-27.9°C. At room temperatures of 20°C-27.9°C, the mean (SD) temperature of HEARThermo was 32.44 (1.67)°C (range 25.45°C-37.53°C) compared with 30.86 (1.85)°C (range 25.7°C-36.6°C) for the infrared skin thermometer. The ICC values declined rapidly with variations in room temperature with low ICC values of 0.45 and 0.49. At room temperature ≤19.9°C, the mean (SD) temperature of HEARThermo was 28.41 (0.92)°C (range 24.07°C-29.81°C) compared with 26.72 (2.05)°C (range 23.0°C-29.4°C) for the infrared skin thermometer. At room temperatures ≥28°C, the temperature variation of HEARThermo was 34.17 (1.22)°C (range 26.93°C-36.49°C) compared with 33.42 (1.78)°C (range 30.2°C-36.6°C) for the infrared skin thermometer.

Table 2

Intraclass correlation coefficients between HEARThermo and infrared skin thermometer.

This is the first study to investigate the reliability and validity of novel wearable wrist devices with thermopiles for continuous monitoring of body surface temperature in a laboratory setting and on human subjects. Our study findings indicated that HEARThermo shows good reliability and adequate validity with infrared skin thermometers. The validated HEARThermo could be expected to provide more useful information by continuously monitoring variations in the body surface temperatures to support medical decisions more effectively. HEARThermo is a reliable device with a high ICC of 0.96-0.98 for the repeated measurement of body surface temperatures, which is consistent with that reported in a previous comparative study conducted on humans [37]. A recent systematic review of the measurement of body surface temperatures using contact thermometry showed that 94% of the studies lack detailed information about sensor calibration, thereby resulting in reduced validity of the data [38]. In this study, HEARThermo devices were calibrated using a water bath, and the corrected measurement bias averaged –0.02°C, which was smaller than that of the previously developed devices with ranges from 0.05°C to 0.2°C [23,24,29,33]. Measurement bias less than 0.5°C is considered minor [38]. However, the correction models should be adjusted occasionally because of the complex process of validating results due to the high demands on the specifications of the calibration methods and the reference thermometer [24]. Therefore, we suggest that the validation and calibration process for an innovative device should be based on the purposes of application in monitoring body surface temperatures under different scenarios.

HEARThermo was sufficiently validated with the infrared skin thermometer at room temperatures ranging from 20°C to 27.9°C, which concurred with the comparative study between iButton and reference mercury thermometer in a temperature-controlled water bath [39]. Studies have shown that the most used sensors in wearable wrist devices for monitoring body surface temperatures are thermistors [21,34,38], but only limited studies have tried to validate these wearable devices by using a temperature-controlled water bath from 19°C to 41°C [19,24,40]. Our study is the first to provide supporting evidence for the validity of infrared wearable devices in real time for measuring body temperatures in humans. Furthermore, HEARThermo showed no visualized systematic bias in terms of the criterion for infrared skin thermometers. However, the limits of agreement between the 2 measurement tools were wider than was the case in previous studies, which suggested that the range for the adequate agreement should be within 1°C in in vivo studies [38] and 0.5°C in clinical practice [41]. The effects of external factors on changing emissivity of the body surface may be more significant for infrared skin thermometers than for wearable devices due to the intermittent measurements by infrared skin thermometers [19,42], which may account for the wider range of limits of agreement found in this study. Our results reinforce that measuring body surface temperature using wearable devices should consider the tight contact with the skin surface. To date, wearable devices in the form of patches to monitor body temperature with features of tight contact have been developed and approved by the US Food and Drug Administration [23]. However, the ability to compare the agreement among different measurement tools for body surface temperatures might be limited due to the lack of a universal gold standard method for measuring body surface temperatures [38]. Further studies are needed to calibrate and validate the agreement between HEARThermo and different forms of medical thermometers in clinical settings.

This study has the following limitations. First, the sensors of HEARThermo could not be calibrated using a high-precision blackbody while controlling the source of thermal radiation in this study. Second, a gold standard for measuring body surface temperatures is lacking, which might cause validation results to vary from one criterion measurement tool to another. Therefore, our study only highlights the relationship between HEARThermo and the infrared skin thermometer used in this study. Third, the generalizability of the study findings was limited to people aged ≥10 years since there was no appropriate size of bracelet for children aged 0-9 years. In addition, relatively healthy people with greater willingness might be recruited because of the snowball sampling method. However, as compared to the previous studies with small sample sizes and only males, this study tried to recruit both females and males and involved a larger sample size and wider age range to improve the generalizability of the findings. Fourth, the results of this study would be more applicable in Asian countries as only Asian people were recruited in this study. We recommend that further studies be conducted in different countries and races to validate wearable devices with infrared sensors in consideration of the emissivity of different human skin tones.

HEARThermo showed good test-retest reliability in the laboratory setting, with the highest correlation with the infrared skin thermometer at room temperatures of 20°C-27.9°C. Moreover, HEARThermo showed no visualized systematic bias and had a bias of 1.51°C with the criterion for infrared skin thermometers. This study validated an innovative wearable device for continuous monitoring of body surface temperatures. Further studies are needed to calibrate and validate the agreement between HEARThermo and different forms of medical thermometers in clinical practice.