With the recent movement toward people (patient)-centered care and the widespread routine use of devices/technologies, person-generated health data (PGHD) have emerged as a promising data source for biomedical research [1]. A survey conducted in 2019 reported that 38% of Americans currently use technologies such as mobile apps or wearables to track their health data, and 28% have used them in the past [2]. Examples of PGHD include data collected passively through sensors, such as step count, heart rate, and sleep quality; data entered directly by people, such as diet, stress levels, and quality of life; and social or financial information that is not specifically health related but could potentially provide health-related insights [3]. Among the different PGHD, data generated through wearable devices are unique in that they are passively, continuously, and objectively collected in free-living conditions; such data are different from those generated through other technologies that require the manual input of data (eg, dietary tracking mobile apps) [4-7]. Therefore, person-generated wearable device data are becoming a valuable resource for biomedical researchers to provide a more comprehensive picture of the health of individuals and populations.

There are two ways to use wearable device data for research purposes. Typically, researchers collect wearable device data for a specific research by recruiting eligible participants and asking them to use the device for a certain period. For example, Lim et al [8] issued Fitbit devices to 233 participants and asked them to use the device for 5 days. Collecting data with this traditional method can be beneficial in that people can collect data that fits their needs, but it can be costly to recruit and follow a large number of participants for an extended period.

Researchers can also reuse existing data, which is a timely and cost-effective way to conduct research. Previous studies have used existing wearable device data collected for other research studies for their own research [8,9]. For example, McDonald et al [9] used a data set collected as part of the SingHEART/Biobank study to investigate the association between sleep and body mass index. In addition, Cheung et al [10] used data collected from a study by Burg et al [11] to develop a novel methodology to reduce the dimension of data while maintaining core information.

More recently, real-world wearable device data collected through routine use of devices have been reused for research purposes [7,12,13]. For example, the All of Us research program, which is the precision medicine initiative launched by the National Institutes of Health (NIH), initiated a Fitbit Bring-Your-Own-Device project, which allows participants to connect their Fitbit account to share data, such as physical activity, sleep, and heart rate [14]. In addition, multiple studies have shown the potential of routinely collected wearable device data for use in large-scale longitudinal multinational studies. Menai et al [15] used Withings Pulse activity tracker data of 9238 adults from 37 countries collected from 2009 to 2013 to examine the association between step counts and blood pressure. Kim et al [16] used data of more than 50,000 individuals from 185 countries collected over a month, with nearly 17 million measurements generated by Nokia Health Wireless blood pressure monitors to characterize blood pressure variability. These studies underscore the potential secondary uses of person-generated wearable device data for generating health insights from large real-world population that might not have been possible using traditional methods of data collection. Furthermore, the studies demonstrate how wearable device data add value by expanding the scope of biomedical research that can be conducted, which would not have been feasible if relying on electronic health record (EHR) data alone.

Data used in research studies, even data originally collected to support research, may not meet the ideal level of quality [13,17,18]. For instance, data collected daily through consumer wearables are meant to be used for routine use of devices rather than for research. Therefore, although the quality of collected data may be sufficient for an individual’s health management, it may be insufficient for research purposes. Hicks et al [19] presented the best practices for reusing large-scale consumer wearable device data that were collected through routine use. The study describes challenges with data quality, such as missing data or inaccuracy of sensor data, as these data are collected from individuals through their daily use of wearables (not through a research study). Thus, as recommended for the use of any data set, the study recommends assessing the quality of wearable device data set before conducting research. Once the research question and data set to be analyzed are identified, it is important to assess its fitness-for-use to ensure that it would produce valid analytical results that answer the research question [19].

There have been previous efforts to understand the data quality challenges for wearable device data. For example, Codella et al [7] identified the data quality dimensions that influence the analysis of PGHD. The concerns and expectations of PGHD stakeholders were identified through a literature review and mapped to the relevant data quality dimensions of an established framework [7]. However, the review does not systematically provide the steps of how they screened and selected the literature and what information they extracted within the studies. Another systematic review by Abdolkhani et al [20] identified factors influencing the quality of medical wearable device data and their corresponding dimensions from the literature. However, this review did not include literature on data from nonmedically approved wearables (eg, consumer wearable devices). As such, there is a research gap in understanding data quality challenges that arise from consumer wearables, specifically those from passively collected data, as there might be unique quality challenges associated with these types of data.

While assessing data quality, having a full understanding of the types of data quality challenges and the factors associated with them can be useful in implementing additional analytic procedures to ameliorate potential negative impacts or false conclusions. However, one of the barriers is that there is a lack of studies investigating the data quality challenges of wearable device data specifically for research purposes. Therefore, this study aims to (1) identify factors influencing the quality of person-generated wearable device data and potential intrinsic data quality challenges (data quality in its own right or, in other words, data quality challenges inherent to the data itself) for research, and (2) discuss implications for the appropriate use of person-generated wearable device data for research purposes based on the findings [21].

We performed a rapid review following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines. The literature search was conducted in four scholarly databases (PubMed, Association for Computing Machinery [ACM] Digital Library, Institute of Electrical and Electronics Engineers [IEEE], and Google Scholar) in June 2019. In PubMed, we used a combination of MeSH terms and keywords related to wearable devices and data quality. Terms related to mobile health were not searched because they include mobile apps or telemedicine, although the scope of this review focused specifically on passively collected data through wearable devices. The search results were limited to studies published within the past 5 years, studies conducted with human species and studies written in English language. The search was limited to 2014 onward because the characteristics of devices may change with advances in technologies, and this may result in changes in data quality challenges. Thus, the search was focused on recent publications using the year with the largest increase in the emergence of new consumer fitness trackers as a heuristic cutoff for determining recent studies [12]. The publications were sorted by best match, which is appropriate for searching studies that meet the informational needs on a topic [22].

In the ACM Digital Library and IEEE Xplore Digital Library, we used a query that combined search terms related to data quality and wearable devices. The search results were limited to studies published since 2014. To complement the search results from the 3 scholarly databases, we performed an additional literature search on Google Scholar. In total, 4 searches were conducted using different queries. The search excluded patents and citations, examined studies published since 2014, and sorted the results by relevance. Although all of the search results were reviewed for other scholarly databases, only the first 100 results for each of the 4 queries in Google Scholar were reviewed. To prevent the filter bubble effect, which customizes search results based on the search history of users, Google accounts were logged out when conducting the literature search [23]. The full query used in each database can be found in Table S1 in Multimedia Appendix 1.

Inclusion criteria were as follows: (1) papers that contained content on the data quality of wearable devices or sensor data; (2) papers that demonstrated the scope of wearable devices, including devices such as fitness trackers, sleep monitors, continuous glucose monitors, and remote blood pressure trackers; (3) papers on research-grade and consumer-grade devices; and (4) not only peer-reviewed studies, but also conference proceedings and book chapters to expand the search space.

Although smartphones can passively collect health data, studies that exclusively focused on smartphones were excluded, as they are not worn on the body. In addition, as we were interested in passively collected person-generated wearable device data being used for research, studies were excluded if (1) the study was on wearable device data that were generated by providers in a clinical setting (eg, device being used for clinician or surgical training), (2) the study was on wearable device data being used for clinical care of patients, and (3) the study was on data that were manually recorded (eg, food logging by user). Device validation studies such as testing the accuracy, reliability, or validity of the device were also excluded, as those studies were about testing the accuracy of the device rather than conducting analyses on data.

One reviewer (SC) screened the retrieved literature based on the title and abstract. After filtering based on titles and abstracts, the full text of the remaining studies was reviewed based on the same selection criteria by two reviewers (SC and KN). The reviewers discussed any discrepancies to reach a consensus on the final set of studies. The literature selection process was conducted using Covidence (Veritas Health Innovation), which is a web-based systematic review production tool.

Overall, two reviewers (SC and KN) examined the papers to extract sentences about the factors affecting data quality. Although our focus was on wearable device data, sentences that apply to both mobile app and wearable device data were extracted as long as the content did not exclusively apply to mobile app data. The reviewers extracted the sentences and annotated the relevant factors. In addition, intrinsic data quality challenges associated with those factors were extracted if any were mentioned. Microsoft Excel was used to manage qualitative data. Codes were assigned to phrases that indicated factors influencing data quality by 1 reviewer (SC). Coded concepts were reviewed, and semantically similar concepts were consolidated into the same category. The categories were iteratively refined to derive core categories. The categories were then iteratively reviewed by domain experts (one data quality expert [KN] and one wearable device expert [IE]) to refine and validate the results. Domain experts commented on whether they agreed with the categorization and names used for each category. The discussion continued until a consensus among the reviewers and domain experts was reached.

A total of 1290 publications were retrieved for screening. Among the retrieved publications, 139 duplicates were removed, leaving 1151 unique publications to be screened by title and abstract. The screening of titles and abstracts resulted in 131 studies after removing 1020 publications that did not meet the eligibility criteria. The full texts of the remaining 131 publications were reviewed. After removing 112 irrelevant publications, 19 studies remained. The literature selection process is depicted in Figure 1, and a summary of the included studies can be found in Table S2 in Multimedia Appendix 1.

Some extracted sentences were specifically related to wearable device data. For instance, sentences within a study by Wright et al [24] describe the challenges associated with using consumer fitness trackers in biomedical research:

On the other hand, there were sentences that could apply to both wearable devices and mobile apps. For example, Bietz et al [25] examined data quality challenges of routine use of devices data and explicitly stated the challenges that researchers face:

Not all concerns regarding wearable device data were extracted from these studies. For example, Bietz et al [25] mentioned selection bias, which was not extracted, as we believe that bias is not an intrinsic data quality challenge but is a byproduct of data quality and a universal challenge to research design:

After 5 iterations of categorizing the factors influencing data quality with domain experts, 3 broad categories emerged, which are summarized in Textbox 1. The mappings between the factors and the intrinsic data quality challenges are presented in Table 1.

One of the goals of this study was to identify potential data quality challenges when reusing data from the routine use of devices for research purposes. However, because of the lack of literature on the reuse of wearable device data, data quality challenges for research in general have been investigated. As a result of the review, three intrinsic data quality challenges were identified—completeness, correctness, and heterogeneity. Missing data were indicated as challenges occurring because of device malfunction, lost satellite connection or synchronization error, users not wearing the device, and devices unstably contacting the skin [7,26,30,34,36]. Incorrect data, which were more frequently stated as inaccurate data in studies, was another potential data quality challenge [26,27,33,35]. Poor sensor quality, the unknown limitations of proprietary algorithms, or user errors such as incorrect device placement can all contribute to incorrect data [26,33]. Another problem is the potential heterogeneity across data sources, which can lead to difficulty in intra- and intersubject comparisons [25,35,38]. This is because (1) companies do not always reveal whether or when they update their device algorithms or whether or when the users install the provided software updates, and (2) different devices may use different algorithms or data definitions for the same variable [25,35,38]. The focus of this study was on intrinsic data quality challenges, which are challenges on the data in its own right [21]. Thus, challenges extrinsic to data such as data accessibility, security, and privacy were not included.

Device- and technical-related, user-related, and data governance-related factors were identified as factors that influence the quality of wearable device data. These factors can potentially affect 3 intrinsic data quality challenges: completeness, correctness, and heterogeneity of data. Of note, the factors identified in this review are inherent to the characteristics of wearable device data as opposed to factors that could occur while processing the data, such as factors in extract, transform, and load (ETL) processes [45]. Researchers conducting multicenter studies should keep in mind that converting their wearable device data by using a common data model may induce additional errors during ETL processes [46].

Factors associated with data quality problems were classified into 3 main categories; however, the authors realized that the identified factors were highly connected to each other, and thus, the categorization could be subjective. For example, limited battery life is a device-related feature, but as a low battery level could make the user take off the device to charge the device, it was classified as a user-related factor. In addition, the proprietary algorithm of devices can be a data governance-related factor as proprietary algorithms lead to heterogeneity in multidevice data due to lack of data standards. However, the proprietary algorithm of devices was classified as a device-related factor because algorithms are part of the device and can produce data heterogeneity in single-device data as well. Despite the subjective nature of this work, three researchers iteratively refined the categories until a consensus was reached. As this is an early attempt to investigate data quality challenges for wearable device data, the authors expect this categorization to be refined in the future as researchers start to apply this framework while assessing data quality.

One limitation of this study is that only one researcher went through the process of screening the title and abstract of studies. Therefore, the selection of literature could have been subjective in the initial phase of screening, and there is the possibility that some factors or challenges were not extracted because of potential biases in selecting the literature. However, the reviewer followed the systematic, a priori–defined selection criteria and data extraction rules to ensure consistency and reproducibility [53]. Although the initial screening of the literature was performed by a single author, other activities such as full-text screening, determining search queries, and categorizing extracted data were conducted by multiple authors. Another limitation is that although we excluded device validation studies in our review, these studies may mention factors affecting data quality for research. However, our full-text screening contained a few device validation studies, and we did not find unique information that was not captured from the final list of 19 studies.

The goals of this review were to (1) summarize the factors associated with data quality reported in the literature with respect to passive data collection methods using wearable devices, (2) identify data quality challenges of wearable device data, and (3) deduce implications on data quality challenges for using data for research purposes. With this goal in mind, we identified three categories—namely device- and technical-related, user-related, and data governance-related factors—along with the associated data quality problems mentioned in the literature—namely completeness, correctness, and heterogeneity. In the case of the secondary use of data, knowing the factors may not directly help researchers, as most of the problems cannot be retrospectively amended. However, the value of this study is that it facilitates the understanding of the potential causes of data quality challenges, which is a complex and time-consuming process that requires thorough discussions among domain experts, analysts, and researchers [45,54]. Moreover, it could guide the application of appropriate analytical procedures to mitigate the negative impact on analytic results. Our review provides some insight into potential data quality problems, such as the incorrectness, incompleteness, and heterogeneity of data. However, further work is required to gain a deeper understanding of each challenge, to investigate if there are any other existing challenges that have not been discovered in the literature, and to provide guidance on data quality assessments for person-generated wearable device data.