This study was performed between May 2024 and August 2025 using the methodological framework for scoping reviews proposed by Arksey and O’Malley [16]. The registration and protocol for this review are available on the Open Science Framework (OSF) [17]. This review is reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) extension for scoping reviews (PRISMA-ScR; Checklist 1) guidelines [18].

Papers eligible for inclusion in this scoping review were those assessing the criterion validity, the agreement of a new measurement method with a criterion measure, of one or more commercial wearable sensors for measuring pulse rate, HRV, blood pressure, respiratory rate, and/or oxygen saturation in humans. The following exclusion criteria were applied:

A literature search was conducted on June 11, 2024, in the databases Scopus, PubMed, and IEEE Xplore. To update the results before publication, the same search was repeated on July 16, 2025. The main concepts included in the search strategy were “PPG-based measurements,” “validity,” and “consumer wearable.” The keywords for these concepts were selected through pilot searches. The search strategy for all 3 databases can be found in Multimedia Appendix 1. Additionally, the reference lists of reviews returned by the database search were reviewed to identify additional papers.

The database search results were uploaded to EndNote 20 (Clarivate) for duplicate removal, after which the remaining papers were uploaded to Rayyan for screening. Prior to initiating the official screening process, the screening guide was tested by 2 reviewers (RMS and FOD) during a pilot screening round with 10 papers. There were no conflicts between the 2 reviewers, and no adjustments were made to the screening guide. The reviewers, RMS and FOD, then proceeded to independently screen the papers for eligibility based on title and abstract, after which a full-text screening took place. Conflicts were resolved through discussion between the 2 reviewers.

Extracted data included information on the type of publication, wearable, study context, recruitment and eligibility, and study population. A complete overview of the data items extracted is presented in Table 1. Most studies do not report how they assessed sex; therefore, it is often unclear whether they report sex or gender. For consistency within this review, we will refer to this characteristic as sex. For skin tone, it was decided to extract the Fitzpatrick scale as this is how most studies report skin tone [13], and as it is the measure recommended by several guidelines, including the US Food and Drug Administration (FDA) guideline on pulse oximeter evaluation, the guidelines for the evaluation of consumer wearable pulse rate validity published by the INTERLIVE network, and the ANSI/CTA (American National Standards Institute/Consumer Technology Association) standard [19-21].

The data charting form was created by RMS in Microsoft Excel. This form was used independently by 2 data extractors in a data charting pilot (RMS and FOD) with 10% of the included papers. For the main data items of interest, for example, study population characteristics, the percentage agreement between the 2 extractors was 95.5%. However, the agreement on the level of physical activity was low (40%). Therefore, additional guidelines for extracting the level of physical activity were added to the data extraction form, after which RMS proceeded with charting the data of the remaining papers independently.

With the extracted data, the number of studies per PPG-based measurement, the number of wearable brands and models, studies per year, and per study location were determined. Additionally, the recruitment locations and eligibility criteria were summarized.

To provide an overview of the representation of population groups, 2 different approaches were applied. First, for the characteristic sex, the percentage of women in a study was extracted or calculated based on the number of women and the sample size. Based on this percentage, studies were categorized as equal distribution (at least 40% men or women), men-dominant or women-dominant (more than 60% men or women, respectively), men only, or women only. As none of the studies reported the participation of intersex individuals, this review can therefore only report the distribution of men and women.

Second, for age, weight, height, BMI, and Fitzpatrick scale, the weighted mean (SD) was calculated to capture the distribution of these characteristics in the included studies. Additionally, for age, BMI, and Fitzpatrick scale, the percentage of participants in commonly used groups of these characteristics was estimated to better visualize the representation of these different groups (Textbox 1) [22-24]. The percentages were estimated based on probability, using the mean and SD reported by the included studies [25]. First, the z-scores for the boundaries of each group were calculated using the provided mean and SD. Second, to estimate the probability of a value equal to or less than the boundary z-scores occurring, known as the cumulative distribution function, the pnorm function was used in RStudio (version 2023.6.1.524) with R (version 4.4.1). The percentage of participants in a group was then estimated by subtracting the cumulative distribution function of the lower boundaries from the cumulative distribution function of the upper boundaries. A threshold was applied to ensure that percentages lower than the contribution of 1 participant were rounded to 0. For age, the distribution was often truncated at 18 years of age, as studies only included adults. In these cases, the ptruncnorm function with a lower bound of 18 was used instead of the pnorm function.

To visualize the distribution of the representation across studies, the median (IQR) of the percentage of participants within each category was reported. To quantify the representation in all studies combined, the total number of participants within each category was calculated based on the estimated percentages. All plots and quantitative analyses were performed in RStudio (version 2023.6.1.524) with R (version 4.4.1).

To evaluate whether a population group was sufficiently represented, the median and total percentage of participants in this population group were compared to a predetermined benchmark [26]. For the purpose of this review, the selected benchmark is equal representation of all population groups. For example, for BMI with 4 groups, the percentage of participants in each BMI group should approach 25%. This benchmark was selected as equal representation of all population groups is desirable when statistically comparing group results.

A total of 1403 papers were identified and extracted from PubMed, Scopus, and IEEE Xplore (Figure 1), of which 695 were identified as duplicates and removed, leaving 708 papers for title and abstract screening. Based on title and abstract screening, 500 papers were excluded, and an additional 5 papers were excluded because the full text could not be retrieved. Screening the reference lists of relevant reviews identified 29 additional papers for full-text screening. In the end, 233 papers were included in full-text screening. Based on full-text screening, 187 papers remained for inclusion, providing results from 186 individual studies.

Of the 186 included studies (n=8786 participants), 160 (n=6461 participants) examined pulse rate, 25 (n=2053) oxygen saturation, 19 (n=887) HRV, 7 (n=326) blood pressure [27-33], and 5 (n=329) respiration rate [34-38]. The median (IQR) sample size was 30 (19-49) participants [19,39]. The publication year ranged from 2015 to 2025 (Figure 2), and studies were performed in 32 countries (Table 2). Across all studies, 33 wearable brands and 129 different models were used. Most of the examined wearables were wrist-worn (119 models), and some were worn on the upper arm (3 models), the finger (2 models) [40-44], or in the ear (5 models) [42,45-47]. Most studies (106 studies) validated 1 wearable device; the remaining 80 studies validated 2 to 7 wearable devices. Additional information about the study characteristics of studies for each examined PPG-based measurement can be found in Multimedia Appendix 2. The extracted data for each paper can be found in Multimedia Appendix 3.

In the 186 studies reviewed, the most reported study population characteristics were age and sex (n=178, 96.0%, and n=179, 96.0%, respectively), followed by BMI (n=101, 54.0%), weight (n=102, 55.0%), height (n=98, 53.0%), and finally the Fitzpatrick scale (n=35, 19.0%; Table 3). Over time, the percentage of studies reporting weight and height remained stable at ~50% (Figure 3). The percentage of studies reporting BMI fluctuated over time, with a minimum of 0% (0/2) in 2015 and a maximum of 77.0% (10/13) in 2025. An upward trend in the reporting of the Fitzpatrick scale was observed from 2019 to 2025 (1/24, 4.0% to 5/13, 38.0%).

Age, BMI, weight, and height were mostly reported as mean or median combined with an SD, SE, or IQR (age: 158/178, 89.0%; BMI: 98/101, 97.0%; weight: 98/102, 96.0%; height: 94/98, 96.0%). A lower percentage of studies additionally reported the range (age: 44/178, 25.0%; BMI: 25/101, 25.0%; weight: 25/102, 25.0%; height: 23/98, 23.0%). Reporting of the Fitzpatrick scale was highly variable, with studies either reporting a combination of mean, median, SD, or IQR (12/35, 34.0%), the percentage of participants in each level or in a combination of at least 2 levels (6/35, 17.0%, for both), the range (5/35, 14.0%), or a combination of these options (6/35, 17.0%).

This scoping review aimed to provide an overview of the representation of different sex, age, BMI, and skin tone groups in studies assessing the validity of PPG-based consumer wearable measurements. Based on the inclusion of 186 studies, our results indicate that the representation of the different population groups is not equal. In particular, older adults, people who were underweight, people living with obesity, and people with darker skin tones (Fitzpatrick type V or VI) are underrepresented compared to other groups. When interpreting our findings, it is important to consider the following points of discussion: (1) the reporting of user characteristics by the included studies; (2) the benchmark used for evaluating representation; and (3) the relevance of evaluating both study-level and total representation.

First, the results of this study revealed that a large percentage of studies assessing the validity of consumer wearables’ PPG-based measurements did not report BMI and skin tone. Not only does this limit our ability to fully evaluate the representation of these characteristics in this review, but it also indicates a critical gap in the available evidence. The nonreporting of these characteristics provides a barrier to investigating the influence of these characteristics on measurement validity. To increase the reporting of study population characteristics like BMI and skin tone, reporting guidelines for consumer wearable validation studies, such as those provided by the INTERLIVE network and the ANSI/CTA, could help. These reporting guidelines recommend the reporting of study populations’ sex, age, weight, height, BMI, and skin tone distribution for studies validating pulse rate measurements and could be extended toward other PPG-based validation studies [19,20]. To truly achieve improved reporting, guidelines like those from the INTERLIVE network should additionally be endorsed by journals that publish validation studies [214,215].

Second, the benchmark used to evaluate representation impacts what is considered sufficient representation and underrepresentation. For example, using equal representation as a benchmark, we consider people with underweight, with a median representation of 3%, to be underrepresented. However, using the prevalence of underweight in the general population, namely 1.6%, as a benchmark, the representation of people who are underweight would be considered sufficient [216,217]. The most suitable benchmark depends on the study’s aim. When the aim is to investigate the influence of sex, age, BMI, and skin tone on wearable measurement validity, equal representation of these groups is a suitable benchmark. However, if a validation study aims to reflect performance in a specific target population or the general population, then equal representation of all groups would not be a suitable benchmark. For this purpose, the prevalence of sex, age, BMI, and skin tone groups in the target or general population should be used as a benchmark. The Clinical Trial Diversity Rating framework, developed by the Institute for Clinical and Economic Review, can then be used to evaluate if this benchmark is sufficiently met [26]. Selecting a suitable benchmark is an important step toward achieving study population representativeness.

Third, in this review, we evaluated representation at the level of individual studies as well as total representation. There are population groups that had low representation at both levels, such as children and adolescents, people who are underweight, and people with darker skin tones (Fitzpatrick types V and VI). However, for older adults and people with obesity, although the representation at the study level was very low, the overall representation was higher. This discrepancy can occur if there is a particular subset of studies in which a group is highly represented, while in other studies, the representation is low. For example, in this scoping review, there was a subset of studies with large sample sizes conducted in patient populations in which older adults were highly represented, while in the remaining studies with smaller sample sizes, almost no older adults participated. In these validation studies, the older adults represented are thus mostly older adults living with disease, who might not be an accurate representation of healthy older adults. This pattern was only observed because we evaluated representation both at the level of individual studies and at the overall level, indicating the relevance of evaluating both levels. In addition to evaluating representation at these 2 levels, it would have been valuable to examine the representation of demographic characteristics intersectionally. Based on the information provided by the included studies, this was not feasible. In reality, the 4 characteristics investigated in this review coexist in one individual, and their cumulative effect is what influences signal quality. An older individual with obesity and dark skin, for example, could present an “intersectional stress test” that current algorithms frequently fail. This necessitates stratified sampling and reporting in future validation studies.

There are 2 explanations for the underrepresentation observed in this review. The first cause is the active exclusion of some population groups through exclusion criteria. For example, many of the validation studies investigating pulse rate measurement validity included in this review had a vigorous exercise component, which can be harmful to some population groups, including older adults. Likely due to this ethical consideration, 31 studies excluded older adults. The second cause is barriers to participation related to other elements of the study. These elements included the research question, the composition of the research team, the recruitment methods, the complexity of research processes, and the required investment of time and resources from the participants [218]. For example, none of the studies had skin tone-based eligibility criteria, yet the representation of people with darker skin tones was very low, which suggests that some other study elements could have prevented their participation.

Irrespective of the cause, the underrepresentation of older adults, people who are underweight, people with obesity, and people with darker skin tones currently provides a barrier for us to gain a better understanding of the differences in measurement validity between population groups. If there are relevant differences in consumer wearable measurement validity between these groups, as suspected, we need to take them into account when analyzing consumer wearable measurement data and interpreting the results. If not, differences in the accuracy of consumer wearable-based risk prediction, disease screening, and monitoring tools between populations may be created, exacerbating already existing health disparities as the implementation of these tools progresses. Additionally, a reduced accuracy of these tools in older adults and people with obesity poses a public health risk, as these tools can be used to screen for conditions that are more prevalent in these population groups (eg, atrial fibrillation and sleep apnea) [219]. Increasing the representation of the “missing” population groups in consumer wearable validation studies is therefore vital. To increase the representation of these groups, a suitable benchmark for representation should be selected during the study design process, and the representativeness of the study population should be evaluated based on that benchmark throughout the entire study. Ideally, this evaluation should not only be conducted by the research team but also by funding agencies, journals, peer reviewers, and other stakeholders in the research process. Additionally, researchers should involve their study populations in the study design process to reduce barriers to participation.

This scoping review is the first to provide a systematic overview of the representation of sex, age, BMI, and skin tone groups in consumer wearable validation studies. Through an extensive search in 3 electronic bibliographic databases and the reference lists of 58 reviews, we were able to include 186 relevant papers. To ensure the transparency of our methods, the entire review process was guided by a protocol that is openly available on OSF. Nevertheless, this review has some limitations that need to be considered.

The representation of age, BMI, and skin tone groups was estimated based on the reported mean (SD), assuming a normal distribution. Though this approach allowed us to visualize the representation of these population groups, there is likely some overestimation or underestimation in these percentages. We hypothesize, based on our results, that the distribution of age, BMI, and skin tone is more likely to be skewed to the right in the included validation studies. An overestimation of the percentage of older adults, people with obesity, and people with darker skin tones in this review is therefore more likely. This suggests that the true representation of these population groups could be even lower than the percentages presented in this review. This is important to take into account when interpreting our findings, as is the fact that there is likely still variation in the representation within the categories applied in this review. Especially within the category of adults, which includes both young adults (18‐40 y) and middle-aged adults (41‐64 y), the high representation of young adults might mask the underrepresentation of middle-aged adults. Similarly, as no breakdown of the representation of the demographic groups per brand or model is reported in this review, it is important to note that there may be differences in the adequacy of representation between different devices, which are not reported in this review.

Another limitation of this review is the use of the Fitzpatrick scale to assess the representation of skin tone. The Fitzpatrick scale was included in this review because alternative measures are currently limited, and this scale is recommended by current reporting guidelines. However, it does not accurately depict the true representation of different skin tones with different melanin content, which truly impacts the PPG signal by absorbing and scattering the light [5]. Therefore, we recommend that future validation studies consider the use of melanometry, a method using reflected light that has been suggested as an objective quantitative measurement of skin melanin [220]. Future studies should also consider using arterial blood gas as a reference standard for measuring SpO₂ instead of finger pulse oximeters. It is a critical limitation that many included studies used finger pulse oximeters, as these devices also exhibit pigment-related bias. Comparing wearables against them, rather than arterial blood gas, may obscure the true extent of skin tone inaccuracies. Additionally, it should be noted that BMI, like the Fitzpatrick scale, is a crude proxy for what it aims to measure, namely, adiposity. Although BMI is convenient for reporting and provides a general overview of the representation of different body sizes, future validation studies focusing on the mechanistic determinants of the validity of PPG-based measurements should consider measuring body composition.

Additionally, it is important to consider that this review only included studies that were published in English. This could have resulted in an underrepresentation of studies conducted in Asia, Africa, and South America, thereby reducing the representation of darker skin tones observed in this review. During the screening of the search results, only 3 studies were excluded based on the language criterion. This suggests that our search strategy, in general, was not very effective in retrieving non-English validation studies. If a similar review is performed in the future, we therefore recommend investigating methods to improve the retrieval of non-English studies, such as including local scientific literature databases.

With this scoping review, we aimed to identify underrepresented population groups by mapping the representation of sex, age, BMI, and skin tone groups in 186 consumer wearable PPG-based measurement validation studies. Based on the results, we conclude that older adults, people with underweight or obesity, and people with darker skin tones are generally strongly underrepresented in consumer wearable validation studies. Researchers making use of PPG-based consumer wearable measurements in their studies should, therefore, be aware of the limited knowledge of the measurement validity in these population groups and take this uncertainty into account when interpreting their findings. To avoid the creation of further health disparities, there should be more focus on the representativeness of consumer wearable validation study populations. More transparent reporting of study population characteristics and the use of benchmarks for representation will allow researchers and other stakeholders in the research process to evaluate the representativeness of a study population. To achieve study populations that properly represent the target population, representatives from the target population should be included in the study design process.