Intraoperatively, continuous hemodynamic monitoring is the standard of care for patients undergoing surgery [1]. Continuous monitoring of patients’ vital signs in the operating room (ie, blood pressure [BP], heart rate, respiratory rate, blood oxygen saturation [SpO₂], core body temperature, and electrocardiogram [ECG]) facilitates immediate recognition of hemodynamic instability and patient deterioration [1]. In contrast, once patients are transferred to surgical wards, their vital signs are assessed only periodically [2]. Hospital policies typically dictate that nursing staff assess patients’ vital signs every 4 hours to 12 hours on surgical wards [2-6]. Patients are then discharged home routinely without surveillance [2]. Such infrequent in-hospital monitoring, followed by no monitoring at home, presents a danger to surgical patients. Cumulative published data support that deteriorations in patients’ physiologic status in hospital, for example, often go undetected [7,8], conferring risk for hemodynamic compromise and serious postoperative adverse events (eg, hypotension leading to myocardial ischemia, stroke, and death).

New remote automated monitoring (RAM) technologies that enable continuous acquisition of physiologic data from biosensors; transmission, integration, and syntheses of multiple data sources to indicate patient status; as well as real-time alerts to clinicians have the potential to revolutionize the science of RAM [2]. Major developments in the field over the last decade include (1) the evolution of RAM systems capacity for semi-automatic (ie, clinician-promoted) discrete measurement of vital signs to fully automatic continuous measurement of vital signs; (2) the development of ultra-lightweight, unobtrusive sensors that facilitate unencumbered patient ambulation; and (3) the incorporation of more powerful microprocessors that enable higher sampling frequencies and, ultimately, higher fidelity signal inputs for increased precision of early adverse event detection [2,9,10]. These advancements are now seeing the commercial availability of a few noninvasive systems [11,12] that are capable of incorporating combinations of a number of vital signs parameters and related metrics, including heart rate, respiratory rate, skin temperature, SpO₂, BP, and movement.

Although significant progress has been made, continuous RAM systems are not yet in routine use in clinical care. A number of tactical and feasibility-related barriers remain, related to signal transmission, range, and speed; duration of power supply; as well as cybersecurity concerns [2,10]. At the clinical care level, a key barrier to advancing RAM has been the need to rely on systems that employ traditional methods for measuring BP noninvasively [2]. Such methods include the use of a sphygmomanometer with manual measurements by auscultation of Korotkoff sounds [13] or palpatory methods [14] and the derivation of automatic measurements through oscillometry [13]. These methods provide discrete or interval-based measurements with a pneumatic cuff typically situated on the brachial or radial arteries.

A challenge with systems that feature intermittent, pneumatic cuffs for the measurement of noninvasive BP is that they can be uncomfortable for patients and infeasible for longer-term patient monitoring [2]. Moreover, reliance on pneumatic cuffs does not help to overcome the problem of episodic vital sign measurement on surgical wards [2]. It is crucial that reliable, continuous, noninvasive blood pressure (cNIBP) measurement be achieved—while clinically important hypotension has been shown to have significant population-attributable risk for postoperative death and stroke, prolonged episodes of hypotension (and hypertension) are often missed in the context of intermittent BP monitoring [6-8].

Recent technologies for cNIBP measurement have emerged that utilize volume-clamp and arterial applanation tonometry methods [15]. Although these cNIBP methods have resulted in clinically accurate medical devices, they are limited in terms of portability and ambulatory use, shorter durations of application due to patient discomfort, and high cost [15]. The pursuit of cNIBP methods that provide seamless integration into a patient’s daily activities and that offer a low-cost alternative while delivering clinical-grade BP metrics is a current focus for the biomedical engineering and RAM communities [10]. Cloud DX has developed one such device called the Vitaliti continuous vital signs monitor (CVSM), which supports the derivation of cNIBP through fundamental principles of biomechanics and pulse wave velocity [2].

In accordance with standards set forth by the International Organization for Standardization (ISO 81060-2:2018) [16], we sought to establish the accuracy of Vitaliti CVSM cNIBP measurements versus gold standard invasive continuous arterial BP measurements in postsurgical patients. A secondary objective was to examine the usability of the Vitaliti CVSM with respect to perceived patient acceptance.

The Vitaliti CVSM [2,17] (Figure 1) is a wearable CVSM that can continuously and noninvasively measure 5-lead ECG, heart rate and heart rate variability, respiration rate, temperature (infrared sensor applied to the ear), SpO₂, and cNIBP [2]. See Multimedia Appendix 1 for details on Vitaliti CVSM donning, device configuration and features, and clinical workflow including calibration procedure.

The verification testing portion of this study received an investigational testing authorization (STP-VIT-002) for Class II medical devices from Health Canada. Study setting, inclusion criteria, and methods were in compliance with ISO 81060-2:2018 requirements [16], as described in the following sections.

This study required comparison of the Vitaliti CVSM to a gold-standard comparator for continuous BP measurement. We therefore required access to patients with an invasive arterial catheter for hemodynamic monitoring. Recruited participants provided written, informed consent and included patients 18 years of age or older who underwent cardiac surgery and were admitted for immediate postoperative recovery in the Hamilton Health Sciences (Hamilton General Hospital site) Cardiac Surgical Intensive Care Unit (ICU) with an arterial line in situ. The ICU setting was chosen given that arterial lines for the continuous measurement of SBP, DBP, and mean arterial pressure are the standard of care in this setting. Moreover, based on operating room schedules, the cardiac surgical unit had predictable patient flows, allowing for planning and efficient execution of study procedures. The study coordinating center was the Population Health Research Institute (PHRI) in Hamilton, Ontario.

Given minimal ISO requirements [16] for participant numbers and prespecified baseline BP ranges, we targeted a maximum recruitment of 80 participants for verification and acceptance testing. We intentionally oversampled given the high likelihood of labile hemodynamic status in postoperative cardiac surgery patients. Based on clinical experience, we anticipated that abrupt changes in baseline BP, or other aspects of physiologic status, would preclude moving forward with testing procedures for some participants. Given the complexity of the clinical setting, we also oversampled in anticipation of interruptions to study procedures (eg, immediate patient care needs, emergencies) and technical challenges with respect to data downloads and intersystem comparisons in the context of real-time cNIBP monitoring.

Study flow is depicted in Figure 2. Patients expected to fulfill eligibility criteria were first approached and invited to participate by the ICU nurse educator. Those interested in hearing more were then approached by study personnel to obtain written, informed consent and collect baseline demographic information.

At the conclusion of each test session, captured cNIBP measurements were extracted from the arterial catheter ICU monitor device using MediCollector BEDSIDE [19] data extraction software. Vitaliti CVSM measurements were extracted by connecting the device USB Type C port to a secure study laptop with a cable. Custom Python scripts were provided by Cloud DX to extract measurements from the Vitaliti CVSM at 1-second time-stamped intervals, for comparison against the arterial catheter data. Both devices were synchronized to ensure time alignment in post-signal processing.

The 10-minute intervals of cNIBP recordings with patients in seated (static) and supine position alignments were verified with Vitaliti CVSM accelerometer and gyroscope data collected during the test period (Figure 3). Per ISO requirements [16], for each patient, we isolated 3 separate 30-second intervals of cNIBP measurements for seated and supine positions; each of these determinations was at least 1 minute apart. Each 30-second interval also had to feature uninterrupted cNIBP measurements without any measurement loss from either the arterial catheter or the Vitaliti CVSM. The 30-second intervals selected for analysis were extracted to allow for 2-minute transition periods between patient positions in order to ensure stable measurements.

Human factors testing is utilized to evaluate if a medical device can support users in the intended environment for all critical tasks [20]. To provide an assessment of elements of human factors and usability of the Vitaliti CVSM from the patient perspective, an exit interview was conducted with patients by the clinical team, and an online survey was completed to capture their responses. A customized questionnaire was developed in keeping with regulatory guidance provided by the Food and Drug Administration (FDA) [20] for applying human factors and usability engineering to medical devices. This questionnaire consisted of 13 questions to establish user acceptance of the Vitaliti CVSM with respect to comfort, ease of application, sustainability of positioning, and aesthetics; possible responses to each item ranged from “strongly disagree” to “strongly agree.” See Multimedia Appendix 2 for the questionnaire.

Derivation of the sample is presented in Figure 4. In total, 202 patients were screened for inclusion in the cardiac ICU between June 2018 and October 2019. Of these, 118 were ineligible due to baseline BPs outside of ISO requirements [16], current arrhythmia (ie, atrial fibrillation), or pregnancy; 7 patients declined participation; and 77 eligible patients consented to participate. Of the 77 eligible patients who consented to participate, an additional 22 were excluded due to technical challenges that precluded completion of the test sessions, shift in BP outside of study requirements, and development of a new arrhythmia (ie, atrial fibrillation) prior to the start of testing procedures. Technical challenges included wireless communication problems, data extraction software failures, reference device data transfer problems, and sensor disconnections. A total of 55 patients were included for validation testing procedures. Of these, 35 patients (64%) were male, and 20 patients (36%) were female; the mean age was 64 (SD 11.5) years (Table 1). Most patients had undergone cardiac surgery (33/55, 60%) including coronary artery bypass grafting or aortic valve repair.

Based on signal analysis, the data for an additional 30 patients were excluded due to arterial catheter reference SBP (≥20 mm Hg) or DBP (≥12 mm Hg) ranges falling outside of ISO requirements during each 30-second measurement interval in both static and supine testing positions. The data for 2 additional patients were excluded due to missing data segments (for either the arterial catheter reference or Vitaliti CVSM) that precluded analysis. Missing data were caused by unforeseen interruptions in data transmission related to excessive movement of the arterial catheter transducer or accidental disconnection or displacement of the Vitaliti CVSM earpiece or ECG electrodes. Finally, the data for the last patient recruited with baseline BP within the normal range were excluded, as this group would have been over-represented for the required ISO BP range distributions. Following signal analysis, the cNIBP data of 20 patients were included for validation analyses. Table 1 presents the demographic characteristics of all 55 patients enrolled, as well as the demographic characteristics of the 20-patient subgroup (of the 55 enrolled patients) who was included in validation analyses.

In accordance with the AAMI-ISO guidelines [16], of the 20 patients whose data were included for validation analyses, a minimum of 30% were male, and a minimum of 30% were female. Baseline arterial catheter cNIBP measurements also spanned high and low systolic and diastolic ranges, with at least 10% of readings falling into each AAMI-ISO prespecified category (See Table 2).

For each of the 20 patients included in the final analysis, 3 determinations were calculated for both the reference arterial catheter and Vitaliti CVSM within each position (static and supine), resulting in a total of 60 average SBP and DBP measurements. The average time elapsed from calibration to first measurement in the static position was 133.85 seconds (2 minutes 14 seconds). The average time elapsed from calibration to first measurement in the supine position was 535.15 seconds (8 minutes 55 seconds). With respect to delimitation of the total validated time frame across patient positions, (1) the minimum and maximum times elapsed from calibration to first measurement in the static position were 14.0 seconds and 1392.0 seconds, respectively, and (2) the minimum and maximum times elapsed from calibration to last measurement in the supine position were 575.0 seconds and 2274.0 seconds, respectively.

The errors of determination between the 2 devices were calculated, as described in the Methods section. Bland-Altman plots [18], illustrating agreement between the arterial catheter reference and the Vitaliti CVSM for each of these errors of determination by patient position (static and supine), are presented in Figures 5-8. The mean (horizontal axis) and errors (vertical axis) of each determination are presented, along with the mean error and limits of agreement (±1.96 SD). In the static position, Bland-Altman plots illustrated a mean error of determinations of –0.62 mm Hg and 95% limits of agreement of –9.64 mm Hg to 8.40 mm Hg for SBP measurements and a mean error of determinations of 0.46 mm Hg and 95% limits of agreement of –2.80 mm Hg to 3.71 mm Hg for DBP measurements. In the supine position, Bland-Altman plots revealed a greater mean error of determinations (2.72 mm Hg) and 95% limits of agreement of –7.40 mm Hg to 12.84 mm Hg for SBP measurements and mean error of determinations of 2.65 mm Hg and 95% limits of agreement of –3.61 mm Hg to 8.91 mm Hg for DBP measurements.

Per ISO requirements, Table 3 summarizes the errors of determination for SBP and DBP measurements in the static and supine positions. The overall means of the errors of determination for the static position were –0.621 (SD 4.640) mm Hg for SBP and 0.457 (SD 1.675) mm Hg for DBP. Errors of determination were slightly higher for the supine position, at 2.722 (SD 5.207) mm Hg for SBP and 2.650 (SD 3.221) mm Hg for DBP. These results indicate compliance with the ISO standard [16], which stipulates that errors of determination should not exceed 5 mmHg and that the SD of the error not exceed 8 mm Hg.

The responses from the 58 patients who responded to the human factors and usability feedback questionnaire are summarized in Figure 9. Questions related to ease of donning Vitaliti and device accessories yielded high acceptance ratings, with 54 (54/58, 93%) agreeing or strongly agreeing. Responses regarding device comfort in multiple positions were also positive, ranging from 52 (52/58, 90%) participants responding with strongly agree/agree to 40 (40/58, 75%) participants responding with strongly agree/agree. Questions regarding aesthetics of the device provided more neutral responses (range: 16/58, 28% to 20/58, 35%), indicating that these aspects were of lesser importance for most. The item regarding the sustainability of the positioning of the earpiece yielded the greatest amount of negative responses (disagree/strongly disagree: 15/58, 26%), indicating that dislodgment of this sensor during testing was an issue for just over 25% of respondents.

This validation study addressed the accuracy and usability of the Vitaliti CVSM for the measurement of cNIBP in an ICU setting. Device performance was evaluated based on the ISO 81060-2:2018 standard [16] for clinical investigation with a reference invasive arterial catheter BP monitor. This standard is accepted by the FDA and Health Canada. Human factors validation testing was also performed to evaluate if the Vitaliti CVSM was acceptable to users. A voluntary exit interview was conducted to capture participants’ usability feedback. In total, 55 participants completed the testing procedures and the exit interview. Data for 20 of these participants were included for validation analyses.

When comparing the accuracy of the Vitaliti CVSM to a gold-standard reference, we found errors of determination of –0.621 (SD 4.640) mm Hg for SBP and 0.457 (SD 1.675) mm Hg for DBP in the static (seated) position. Errors of determination were slightly higher when patients were supine, at 2.722 (SD 5.207) mm Hg for SBP and 2.650 (SD 3.221) mm Hg for DBP. These results indicate compliance with the ISO standard [16], which stipulates that errors of determination should not exceed 5 mm Hg and that the SD of those errors should not exceed 8 mm Hg.

This validation study also demonstrated a high degree of usability in terms of perceived patient acceptance of the Vitaliti CVSM throughout the test procedures and positions (ie, static and supine). Displacement of the earpiece was the most negative aspect of patient experience. Earpiece displacement was caused by our use of an off-the-shelf disposable ear sheath to cover the light emitting diode (LED) sensor in order to comply with hospital infection requirements. In future work, the Vitaliti CVSM will require custom earpiece sheathing with improved fit in order to provide more secure mounting of the LED sensor and improved patient comfort.

A few comparable studies have assessed the accuracy of wearable technologies for continuous vital sign measurement in hospital. A pilot study by Weenk et al [11] (n= 20) investigated the use of 2 wearable technologies, ViSi Mobile and HealthPatch, to continuously measure vital signs (ie, heart rate, respiration rate, SpO₂, BP) of patients admitted to an internal medicine and surgical ward; patients’ vital signs were measured continuously for 2 days to 3 days. A comparison was performed between continuous ViSi Mobile and HealthPatch measurements and nurses’ manual vital sign observations entered into an electronic medical record. Results demonstrated that, in general, nurses’ manual vital sign observations correlated well with paired instances of continuous vital sign measurements. Data artifacts and data outages were noted concerns and were attributed to Wifi connectivity challenges and signal noise in the context of patient ambulation [11].

More recently, Downey et al [12] investigated the reliability of a wireless wearable patch, SensiumVitals, to monitor vital signs (ie, temperature, heart rate, respiration rate) continuously from patients (n=51) following major elective general surgery. Nurses’ manual vital sign observations were compared against paired instances of SensiumVitals biometric measurements. A median of 19 sets of manual measurements was captured for each patient, for a total of 1135 observation sets of paired comparisons for analysis [12]. In contrast to the results of this study, the error between manual and continuous vital sign measurements in the study by Downey et al [12] did not fall within prespecified limits of agreement, as defined through clinical expert consensus. Wide error distributions were again attributed to patient ambulation and related signal artifact, as well as possible human error during various manual vital sign measurements.

Data from these pilot studies support that wearable technologies capture continuous vital sign measurements from hospitalized ambulatory patients with varying degrees of accuracy. Whether prespecified levels of agreement with a reference standard are met, signal artifact and other sources of error, such as human error, pose challenges to validation of wearable sensor technologies in real-world clinical settings. Although our comparison of the Vitaliti CVSM to a continuous invasive reference (in an ICU) met strict ISO prespecified limits of agreement [16], our study patients were necessarily restricted to their hospital bed while undergoing cNIBP monitoring with an arterial catheter. Hence, in this environment, excessive patient motion and human measurement did not pose major challenges. The compliance of the Vitaliti CVSM with rigorous ISO standards [16] in a complex ICU environment is nonetheless promising; further validation testing in ambulatory patients is required.

A few studies have also examined wearable biosensor user acceptance from the patient perspective. In the pilot study of the ViSi Mobile and HealthPatch systems by Weenk et al [11], user experience was captured through semistructured interviews after patients had worn these devices for 2 to 3 consecutive days. Thematic content analysis revealed largely positive experiences, with most patients reporting that the monitoring devices were reassuring for them because nurses could monitor them from a distance [11]. Most also felt that these sensors did not encumber their personal care activities (eg, dressing, bathing). Good device adhesive and small sensor size were also noted as factors that were important to patients with respect to wearability of several device components [11]. Wearability of the Visi Mobile system was reported by several patients to be impacted negatively by the size and weight of the wristwatch component, numerous cables, as well as short battery life [11].

In a reactive post hoc analysis report, Harsha et al [24] examined challenges with implementing continuous oximetry monitoring in the VItal siGns monItoring with continuous puLse oximetry And wireless cliNiCian notification aftEr surgery (VIGILANCE) study (n= 2049), a randomized controlled trial testing the effectiveness of the Nellcor Oxinet III system (Covidien, Mansfield, MA) for continuous pulse oximetry (CPOX) monitoring on the incidence of postoperative respiratory complications among noncardiac surgery patients. VIGILANCE investigators found that 10.68% of patients withdrew from CPOX monitoring before intervention completion. Analysis of trial case report forms found a number of reasons for patient nonadherence, including obtrusiveness of the CPOX cables, an uncomfortable SpO₂ probe, restrictions to ambulation, and device-related agitation of carpal tunnel syndrome [24].

In contrast to the usability assessments by Weenk et al [11] and Harsha et al [24], our examination of patient acceptance of the Vitaliti CVSM was in the context of a controlled measurement study, rather than in the context of live clinical system deployment where patients were ambulating on hospital wards and engaged in personal care activities. Although context differed, our results corroborate that patients value unobtrusive and comfortable sensor components. Our results also corroborate that patients are impacted negatively by technology features that are experienced as cumbersome or that require continual repositioning or reapplication.

Strengths of this study include the validation of the Vitaliti CVSM in the context of ISO standards for cNIBP measurement [16], as well as rigorous methods and planned approaches to post hoc signal analyses in order to ensure high data quality. Conduct of this study in a complex ICU setting also required the interconnection of numerous pieces of technology with operational independence to achieve time-matched cNIBP data sets for comparison of the Vitaliti CVSM with a gold-standard arterial catheter reference. An additional strength, therefore, was our plan to oversample to compensate for anticipated data losses due to technical problems and inevitable changes to patient hemodynamic status in an ICU setting.

Although the ISO standard [16] is rigorous from a measurement perspective, a limitation it imposed was the restriction of study participants to static (seated) and supine positions. Patients were also confined to bed while undergoing invasive cNIBP monitoring, thereby limiting the generalizability of our results to nonambulatory environments. It should also be noted that, in a complex ICU setting, this investigation could not span the full duration of the calibration period for the Vitaliti CVSM, which is 24 hours (Multimedia Appendix 1). Rather, this study was limited to evaluation of the device during a very short validation period after calibration (ie, that commenced within 2 minutes after calibration and that lasted for a short duration of time). For practical reasons within an acute ICU setting, we had to set up our equipment and take our BP measurements on each participant as expediently as possible to minimize disruption to nursing and medical care by virtue of the presence of our study team and related equipment. This timing precluded possible BP variations from calibration values that may otherwise be observed during a full 24-hour calibration period. Hence, results of this study cannot be applied to the BP accuracy of the Vitaliti CVSM over a 24-hour period with BP variations that may be normal, including individual readings that may vary considerably from calibration values. Future research should incorporate evaluation over the full calibration period of the device.

Our results for patients in the static and supine positions are within the clinically allowable tolerances for accuracy according to ISO. As such, claims regarding the accuracy of Vitaliti cNIBP can only be made at this point in the context of accuracy requirements set forth by Health Canada and the FDA. More research will be required to examine the accuracy of Vitaliti cNIBP measurements according to international standards, such as those set forth by the European Society of Hypertension [23] and the British Hypertension Society [25]. Examination of the accuracy of the Vitaliti cNIBP measurement according to these standards was not within the scope of this study.

Finally, patients enrolled in this study were postsurgical cardiac ICU patients given the requirement to compare the Vitalilti CVSM to an invasive gold-standard arterial catheter. The hemodynamic profile of patients in the postsurgical cardiac ICU typically features a greater degree of variability than seen in other populations in the early postoperative period. Moreover, some patients may experience atrial fibrillation following cardiac surgery. It should therefore be recognized that the included sample is not representative of all postsurgical patient populations—particularly those patients who undergo noncardiac and same day surgeries.

Wearable RAM technologies that enable continuous acquisition of physiologic data from biosensors have the potential to transform postoperative care. This study found that one such wearable technology, Cloud DX’s Vitaliti CVSM, demonstrated cNIBP measurement in compliance with ISO 81060-2:2018 standards [16] in the context of evaluation that commenced within 2 minutes of device calibration; this device was also well-received by patients in a postsurgical ICU setting. Future studies will examine the accuracy of the Vitaliti CVSM in ambulatory contexts for both cardiac and noncardiac surgery patients, with attention to assessment of the impact of excessive patient motion on data artifacts and signal quality. The Vitaliti CVSM will also be evaluated longitudinally as part of a postoperative remote patient monitoring solution both in hospital and while patients are recovering at home for up to 30 days following surgery. This work will feature intensive focus on the use of derived vital metrics and high-fidelity physiological data collected with the Vitaliti CVSM in order to develop predictive models with machine learning. The aim of these predictive models will be to identify early signs of postoperative complications in order to facilitate timely clinical interventions.