Motion tracking data are pivotal for understanding physical activity in hospitalization and rehabilitation [1,2]. However, the procedures for investigating motion tracking data have not yet been systematically reported, especially in the context of user activity recognition in health care application studies. Several findings indicate that living patterns, such as daily activity trajectories, physical activity, and dietary behaviors, are associated with the initial stages of disease development [3-5]. Inadequate daily physical activity exacerbates symptoms of mental distress and contributes to medical illnesses, including cardiovascular and kidney diseases [6,7]. Conversely, regular physical activity can enhance wellness by activating the immune system and reducing inflammation [8,9]. Monitoring activities of daily living (ADLs) [10] is vital for self-care and long-term health [11,12], making the analysis of daily living and physical activity data crucial for disease prevention and promoting a healthy lifestyle.

A comprehensive analysis of motion tracking data in daily living and physical activities scenarios is essential for delving into human activity patterns and behavior intentions. Research shows that activity patterns are linked to behavior intentions [13,14], and motion data help uncover these patterns, improving diagnostic efficiency [15]. The motion data of ADLs represent activity patterns [16,17], while the formation of these patterns is multifaceted [18], with external influences playing a role [19]. Accumulated movement behaviors, when stimulated by the environment, become habitual actions [20,21]. By analyzing the motion data of ADLs, researchers have found that the formation of activity patterns is attributed to individuals’ cognitive representations [22] and environmental factors, such as sedentary behaviors in working and learning spaces [10,23], as well as daily routines at home [24]. Thus, motion tracking data, enriched by environmental context, effectively represent and interpret human behaviors across different settings.

When applied as representations of ADLs, motion data can facilitate behavioral change through interventions. The analysis and visualization of motion data concerning ADLs can potentially enhance mutual understanding between physicians and patients in medical education and remote clinical consultations [25,26]. Patients’ trajectories of past daily activity and medical histories are essential indicators for physicians to understand symptoms [27]. Human physical activity patterns can reflect the formation of disease and aid in the prevention and management of illness conditions [28,29]. In addition, many researchers have devised human activity interventions and suggested that early care physical activity interventions are feasible for promoting healthy lifestyles [30,31].

The evolution of motion tracking technologies in health care has transitioned from basic motion detection systems aimed at rehabilitation to artificial intelligence (AI)–enhanced wearable sensors for personalized care. Early systems focused on simple motion detection, primarily for rehabilitation [1]. Advancements led to the inclusion of wearable sensors and AI for personalized health care [32]. Recent developments have embraced Internet of Things (IoT) frameworks for more comprehensive health monitoring [33]. In addition, the shift toward patient-centered care has been facilitated by technologies that assess and support daily activities and physical functions [11,34]. The integration of machine learning and sensor data fusion has further enhanced the ability to monitor and analyze patient movement in real time, contributing to preventive health care and improved treatment outcomes [12,35]. Researchers in health care and computer science have discussed the possibility of using motion tracking. Data classification, precision, validity, and diagnostic prediction in activity recognition are popular areas of investigation [36,37]. Several studies have also discussed how collecting voluminous data might contribute to personalized health surveillance systems [32,33]. Furthermore, scholars in computing have claimed that high-quality data would enhance data-driven AI processing [38,39]. IoT, with motion tracking techniques (eg, inertial measurement unit, smartphone, smartwatch, Mocap system, etc) [40,41] and AI processing (eg, human activity recognition, positioning system, etc) [35,42], have been widely used. It is anticipated that AI processing using high-quality motion tracking data would provide accurate and timely health information. For instance, smartphones with built-in motion-tracking sensors can record activity trajectories and routine data to identify activity types and patterns [43]. Studies have also found that motion data can be used to learn about humans’ typical behaviors and identify any outlier activities [44,45]. Moreover, it can boost individuals’ likelihood of engaging in physical activity over the long term, which is associated with habit formation [46,47].

However, the data interpretation of human activity trajectories, environment, and process has not been discussed as extensively as technology. Conversely, several studies have reported that learning from health information could promote health literacy [48,49]. Yet, the public has a long-standing misunderstanding of technology [50,51]. The technology-based health information needs to be in readable language [52,53]. In the meantime, the foundational criteria (ie, ease of use, usefulness, and enjoyment) of the technology acceptance model (TAM) provide insights to evaluate human-technology interaction [54,55]. TAM exemplifies the 3 needs of humans in interpreting technology-based health information.

Designers play a key role in making data legible, efficiently transmitted, and actively engaging for users [56-58]. To optimize the utility and interpretability of motion tracking for both professionals and users, designers should gather health information needs from the user’s cognitive standpoint, thereby refining the functionality and analytical clarity of motion-tracking systems [59,60]. The adoption of data-enabled design (DED) [61-63] in health care emphasizes continuous data refinement and context-specific design. On the basis of these, health care–related applications, as intelligent solutions [62], should integrate data, users, and environments to optimize the transmission of health information and the intelligent ecosystems. Scholars asserted that design studies could illuminate the nature of nonsocial entities, such as data, to create durable systems [64,65]. This approach has been implemented in designing health interventions [66,67]. It overcomes the barrier between people and technology, offering guidance for enhancing the sustainable hybridization of the social and the technical entities.

Motion tracking serves as a bridge, delivering activity data to health care professionals and users. Understanding data collection and analysis is vital, involving motion data (eg, trajectories and durations) and environmental elements in physical settings. Analysis entails identifying and classifying activities. Yet, interactions among researchers, participants, devices, and environments are seldom explored. Therefore, it is critical to methodically study these interactions within health care applications and their environments.

In health care application research on ADLs, interactions occur between participants, sensors, cameras, and scenario objects. The actor network theory (ANT) has been instrumental in analyzing these interactions within a network [68,69], especially in human-computer interaction and design studies [70]. ANT outlines three critical aspects for network analysis: (1) actor-led activities, where initiators, such as individuals or devices, lead activities and interact with others; (2) purpose-oriented interactions, which serve as the guiding principle for activities; and (3) the dynamic interplay between devices, environmental factors, and individuals, shaping the network. Integrating ANT into motion tracking data analysis illuminates the movement trajectories and environmental influences, clarifying human-device-environment relationships.

Prior research has thoroughly examined motion tracking in health care, mainly focusing on algorithm validation and technology deployment for rehabilitation and hospitalization [42,71]. Meanwhile, several studies have reviewed the benefits of physical activity for medical purposes [10,26]. However, systematic reviews exploring the use of motion tracking techniques in ADLs and their connection with humans for health care and physical wellness research are scarce. Furthermore, studies investigating the process of analyzing ADLs for health care applications are even rarer. This study shifts focus to motion data, participant engagement, and situational contexts as key components in health care technology. It delves into how participants interact with technology in physical environments, technology adoption, and participant behavior. This systematic review aimed to map and collate literature on the motion tracking data of daily living and physical activities for health care and physical wellness application research. It aimed to evaluate the research landscape, identify literature gaps, and endorse a designer-engaged approach to studying ADLs. The overarching research questions are as follows:

Our research focuses on investigating the motion tracking data, interaction, and process of ADLs in health care and physical wellness applications from the designers’ perspective. Owing to the exploratory and descriptive nature of our research question, we opted for a systematic review [72,73], with the aim of compiling and comprehensively summarizing the relevant research. To guarantee rigor and coherence with our research objectives, 2 researchers meticulously screened the studies for inclusion. These 2 researchers independently conducted screening work on the same data set and met weekly for discussions. Following 5 detailed discussions about the screening methodology, centered on eligibility criteria and research focus, we conducted 2 pilot searches to refine our search strategy and ensure the accuracy of our results. In addition, we discussed the framing of questions being addressed with reference to participants, interventions, comparisons, and outcomes (PICO) [74], ensuring a comprehensive and structured approach to our review (Table 1). The overarching goal of the systematic review is to identify and map the available evidence investigating the technology use in everyday health care and physical wellness apps and its relationship with humans and environments. We then applied ANT [68,69] to analyze the interaction in activity scenarios and used design narrative through the lens of DED [62] to analyze the process of motion tracking data in health care and physical wellness application research scenarios.

Given the independent nature of technology, user behaviors, and environments in our research, we developed bespoke criteria for bias assessment. This approach enabled 2 independent reviewers to identify potential biases in study design, execution, and reporting accurately. Discrepancies between reviewers were resolved through consensus, ensuring a balanced evaluation. To further validate our review, we engaged relevant scholars to critique the search strategies of both pilot and final searches stages, substantially enhancing the review’s validity and comprehensiveness. Our thorough assessments of potential biases, both at the study and outcome levels, were integrated into our data synthesis, paving the way for a credible interpretation of the evidence (detailed in Multimedia Appendix 1 [75-128]).

Drawing from previous research in health care and motion tracking, our study builds upon the insights provided by Straczkiewicz et al [42], who illustrated the relationships among different human activity recognition processing phases through data visualizations. Their approach to data visualization reveals the connections among co-occurring factors identified across the reviewed articles over time. In addition, we also considered the work of van Kollenburg et al [62], who explored design narratives for situated design exploration, thereby enriching our understanding of the data analysis process and its application in health care scenarios [62].

Through the data extraction process from the selected articles, we organized the data into 11 categories: (1) study outcome (subcategories encompassing providing feedback, visualization, real-time monitoring, and data analysis); (2) sensor types (subcategories include multiple sensors, other sensors, motion-tracking sensors, and smartphone or smartwatch); (3) motion capture using cameras; (4) position of sensors (eg, physical space, on object, full body, upper limbs, or lower limbs); (5) participant types (individuals on whom sensors were placed or who were being monitored, including athletes, people with disabilities, older adults, children, pregnant women, people in scenarios, general public, or nonhuman subjects); (6) number of participants (categorized as >101, 51-100, 10-50, or <10); (7) activity types (covering interactions in physical spaces, sports, activities related to upper or lower limbs, and general daily activities); (8) monitoring types (differentiated by event or time-based monitoring); (9) environment types (distinguished as in the wild or controlled); (10) data types (differentiating between data collected independently or from external data sets); and (11) Al involvement. Then, using the timeline and the categories of the included studies, we provided a publication year–based summary that illustrates the Network of Daily Living and Physical Activity (NDLPA) among included studies, as depicted in Figure 2.

The electronic search yielded 1501 studies. After removing internal duplicates, 956 studies remained. On the basis of the eligibility criteria, 258 articles were chosen through title screening (Figure 1). A total of 54 articles were included for full-text appraisal after the abstract screening process [75-128]. The results from database search indicated that the relevant research from 2013 to 2018 grew steadily, while research articles published from 2019 to 2022, aligning with our interests, experienced a significant uptick and consistently maintained a high volume (Figure 3; Table 2). The findings (Figure 2) revealed that 54 articles used motion tracking data in health care and physical wellness applications, of which 42 (78%) articles used motion-tracking sensors [75-81,84-86,88-97,101, 103-116,119,121-123,125,126]. Overall, 70% (38/54) of the articles used AI technology combined with motion tracking data [75,77,79,82-85,87,89-91,93,94,97,99-101,103,105,106,108,109,111-113,117-128]. The role of AI mainly functions for data classification, monitoring, and visualization among the included studies. Furthermore, 80% (43/54) of the studies recruited participants for user study [75,76,78-81,83-88,90,92,93,95-105,107-110, 113-116,118-124,127,128]. Twelve articles used public data set [78,82,89,91,94,106,111,112,117,125-127]. Most of the public data set that were used are University of California, Irvine human activity recognition, center of advanced studies in adaptive system, mobile health, University of Milano Bicocca smartphone-based human activity recognition, mobile sensor data anonymization, a large multipurpose human motion and video data set, and wireless sensor data mining data sets.

In conducting a thorough risk of bias and quality assessment of the selected studies, our scrutiny was directed toward several critical dimensions: participant selection transparency, the accuracy and completeness of data acquisition methodologies, the analytical rigor, and the integrity in reporting outcomes. Notably, prevalent biases included selection bias, rooted in participant recruitment strategies; measurement bias, owing to sensor placement inconsistencies or activity categorization; and reporting bias. Studies using public data sets or conducted within laboratory confines typically showed reduced bias risks. In contrast, studies with constrained participant demographics or undertaken in less regulated environments faced heightened bias risks, primarily affecting data representativeness and the influence of external variables on activity assessments.

The included articles variably reported on participants’ gender and age specifics—57% (31/54) of the articles provided detailed demographic data [77-79,82,86,89-91,93,94,97-99,101,103, 105-107,109-113,115,116,119,122,124-127], while others (22/54, 41%) omitted such specifics [75,76,80,81,83-85,87,88,92,95,96,100,102,104,108, 114,117,118,120,121,123,128]. The sensor models used across studies are meticulously documented within the articles and are detailed in the Multimedia Appendix 1. It was observed that the spectrum of daily living activities discussed was somewhat narrow, seldom addressing activities that entail object interaction. The theoretical underpinning, such as that provided by ANT [68,69] and activity classification [130], lends robust support to the motion data analysis, facilitating a nuanced understanding of the data. The selected articles did not address the incorporation of verbal language or cultural customs in the context of data collection or the execution of physical activities. Thus, we consider that the concentration on physical movement to the exclusion of linguistic or cultural contexts likely accounts for the omission of cultural background in the analysis. Data collection in laboratory settings is controlled, thereby eliminating interference from factors such as socioeconomic elements. The physical tasks assigned to participants encompass routine daily movements or activities well-acquainted with athletes, effectively eliminating potential interferences from policy-related factors.

The features of the selected studies were systematically categorized into study outcome, sensor configuration, participant types, activity types, monitoring, environment types, and AI involvement, as illustrated in NDLPA among included studies (Figure 2). This organization was informed by the visualization of Straczkiewicz et al [42], who elucidated the complex interrelations among diverse human activity recognition process phases. Drawing upon the theoretical foundation provided by ANT [64,68,69], our analysis delved into the NDLPA by investigating the dynamics of actor-led activity; purpose-oriented interaction; and interaction among devices, individuals, and environmental factors.

Activity trajectory is demonstrated through the interaction between humans, sensors, and environments. Motion-tracking devices, sensor positions, and environments factors are the main features to be evaluated.

Activity identification and recognition have been the primary focus of research on ADLs and physical activities in health and wellness applications over the past decade. The fundamental objectives are to enhance the accuracy of motion tracking data recognition and its practical use for health professionals. From the perspective of NDLPA, researchers acted as initiators of activity networks, leading the content and environments of activities in most activity recognition studies. Participants provided standard movement data within a restricted time and environment, lacking personal purpose and awareness of real-life activity. Participants did not fully become the initiators of the activity networks, but real-life activity data could be more complex. The variety of activity scenarios, user-friendly sensor design, data collection at specific time points, and data variances produced by complex activities and various real-life purposes could present research opportunities for studying daily living and physical activities for health and physical wellness applications. Meanwhile, designers have the opportunity to engage in designing user-initiated activities and collect technology-based motion data for health and physical wellness application research.

According to the descriptions in the included articles, the investigation of ADLs and physical activities in health care applications covers 4 phases: data acquisition, data classification, data analysis, and data interpretation, as shown in Figure 4A. In the process of motion tracking data acquisition, researchers recruited participants and conducted data collection. Participants performed assigned activities under full or partial surveillance by researchers. The data were supervised and categorized into standardized movements corresponding to specific activities. At this stage, the activities varied by different engaged areas on participants’ bodies but not by the content of daily living activities. Researchers trained various algorithm models for data classification to achieve high activity recognition accuracy and used different data sets to test the system’s validity. In the data analysis stage, researchers identified standardized activities and examined the difference values through motion tracking parameters. Using these parameters, researchers analyzed activity norms and behaviors. This stage is also associated with data monitoring. Data classification and analysis can be combined or conducted independently for different research evaluation purposes. Data interpretation, such as data visualization and feedback provision, was created by researchers from the health, engineering, and computing domains. It interacted with health information beneficiaries (eg, professionals, users, and caregivers). The health information was presented in either video or image forms for visualization or in text forms for feedback to users.

On the basis of the investigation process of ADLs and physical activities in health care applications, participants’ involvement occurs at an early stage. However, participants are situated in a controlled environment for experimental purposes, with devices attached to their bodies without active interaction. Although the data are clear for classification and analysis, they rarely explain the diversity of ADLs. In the data interpretation stage, there is a weak connection between the data and the general public owing to the dominant purpose of activity identification research aimed at professional use. Designers can aid in bridging communication between information and the public. As described in the literature in previous sections, human motion data are essential components of activity recognition and analysis [1,2], with the public being the beneficiary of physical activity intervention research [11,12]. We propose that designers, coupled with researchers’ guidelines, participate in the investigation and application process, which promises to facilitate data collection and interpretation.

The research of health care and physical wellness applications is directed toward professional deployment and public health enhancement. However, it has been observed that there exists a gap between the motion data collected from participants and their applicability to real-life situations. In addition, research in health care applications often fails to encompass the breadth of data diversity; there is a noticeable lack of meaningful engagement between devices and participants, and the process of making data understandable for the public is fraught with difficulties. To address these issues, we advocate for a research methodology that integrates DED framework [61-63], a designer-involved research process of motion data on ADLs and physical activity for health care application research, as illustrated in Figure 4B.

In this proposed methodology, designers collaborate with health care researchers to understand the specific information required for participant recruitment in a given project. They assist health care researchers in designing data collection strategies while considering the daily behavior patterns of the target population, thereby refining the data collection plan. Researchers from computing and engineering fields undertake the collection and categorization of data. These data span various demographics, purposes, devices, and scenarios from designers and are tailored for a multitude of applications. Such research process encompasses a cycle of prototype iterations. Researchers from the health field analyze classified data, identify abnormal motion data, and then provide guidelines for designers. Designers, positioned as intermediaries between users and researchers, leverage these guidelines and their nuanced understanding of human daily experiences including participant involvement and activity motivation [131] to reinterpret professional data into user-friendly formats. Furthermore, designers play a pivotal role in collecting motion data that reflect a diverse array of demographics, intentions, devices, and scenarios. They are able to simplify the data and assist with data interpretation for the recipients of health information.

The integration of design principles into the development of health care applications is key to enhancing user engagement, offering more intuitive interfaces that potentially increase adherence to health interventions through user-friendly designs. Our findings indicate that in intricate settings, standardizing data to mitigate external variables might lead to binary misjudgments. Given that daily activities are influenced by individual attributes, environments, and habits, the role of designers becomes critical in tailoring solutions to meet the varied needs of distinct user groups. They fine-tune the implementation methods and locations for motion-tracking devices, such as using cameras or accelerometer and gyroscope sensors and determining specific areas for deployment, such as full-body tracking or environmental interaction. They also specify the nature of the data to be collected, differentiating between event-driven data, which capture specific actions such as walking across varied material surfaces, and time-sequential data, which records information over continuous periods such as walking at night. This process enhances the accuracy and applicability of collected information, facilitating the development of data-driven products that reflect real user experiences. During the experiment, the dialogue between users and designers’ aids in identifying actual and hidden needs, thus refining information precision and application. Upon analyzing the data, health care researchers collaborate with designers to reformat health information into accessible, engaging formats on digital interfaces, using techniques such as digital twins, animations, visualizations, and gamification. This method enhances user engagement, elicits valuable feedback, and fosters ongoing improvement in data accuracy, thus promoting a cycle of continuous information enhancement.

In addition, designers can assist engineers in designing user-friendly devices and facilitate actuator developments to provide users with timely feedback. The iterative refinement of technology that monitors ADLs and physical activity motion data in health application research is driven by the diversity of the population, necessitating continuous optimization. Design’s contribution is pivotal in promoting effective communication, nurturing relationships, and pursuing sustainable innovation. By involving designers in the process, the development of activities initiated by users and the creation of user-friendly motion-tracking sensors are significantly improved. This, in turn, supports the cyclical development of health and physical wellness applications [64,65], ensuring that they are more aligned with real-world needs and contexts. Thus, in the sphere of health care and physical wellness application research, the involvement of designers is crucial for the effective acquisition and interpretation of data, ensuring that the data are relevant and beneficial in everyday scenarios.

Designers’ involvement is essential in shaping design and implementation strategies, ensuring that technology solutions are tailored to meet the inherent human needs for interpreting health-related information. This approach is instrumental in improving the monitoring and management of health conditions, thereby improving patient outcomes. This not only improves usability but also ensures that technologies are perceived as useful and enjoyable, mirroring the foundational criteria of the TAM [54,55]. However, potential challenges include ensuring effective communication and collaboration across multidisciplinary teams, aligning different objectives and methodologies, and the complexity of translating complex health data into accessible and actionable information for users.

This review process identified key limitations and future research directions in the use of motion tracking technologies for health care. It pinpointed risks of selection and measurement biases due to participant recruitment strategies and inconsistencies in sensor placement or activity categorization. Highlighting the inadequate exploration of feedback mechanisms, this review emphasized the necessity for future studies to optimize user communication. The use of sensors for motion data collection offers a method to preserve privacy, contrasting with cameras or recorded videos, which might inadvertently compromise it. One included study introduced a Kinect and smartwatch system, focusing on health care professionals’ privacy and showcasing a strong commitment to privacy and ethical considerations in system design [79]. The minimal focus on privacy protection in other included studies indicates a potential research gap. Furthermore, this review underscored the unexplored potential of integrating geolocation data into physical activity interventions, proposing future research to delve into scenario-based health care applications. The investigation did not fully address the influences of cultural, socioeconomic, and policy-related factors on physical activity, or the budget constraints that may have curtailed the exploration of these dimensions. Moreover, it acknowledged the potential publication bias and the influence of study quality on the findings, especially given researchers’ inclination to report positive outcomes of health care application development. Although the search included major databases, there is a possibility that relevant studies in other databases were overlooked. In addition, database search confined to English language might have excluded pertinent non-English studies. These areas present fertile ground for future research to ensure a more holistic understanding and application of motion tracking technologies in diverse settings.

This study significantly contributes to future research by demonstrating the importance of a multidisciplinary approach in health care application development, particularly in the integration of design principles with engineering and health sciences. It lays a foundation for further exploration into how designers can enhance the usability and effectiveness of health applications through user-centric design and data interpretation. Future studies can build on this framework to investigate specific design strategies and their impact on user engagement, adherence, and overall health outcomes, potentially leading to more personalized and effective health interventions. For real-world applications, this approach could fundamentally transform the development of health care apps, making them more accessible, intuitive, and tailored to individual needs, ultimately leading to improved health outcomes and patient engagement.

The study systematically reviewed the implementation of motion tracking in health care application research, revealing a discrepancy between the data collection and its effective application in real-world scenarios. Informed by ANT, we explore the dynamics of actor-led activities and purpose-oriented interactions, focusing on participants, sensors, cameras, and environmental factors in health care research concerning ADLs. The guiding roles of activity initiators significantly influence the phases of data acquisition, classification, analysis, and interpretation, underscoring the potential to enhance the accuracy of motion tracking and activity recognition when considering contextual factors. By advocating a designer-involved research process of motion data on ADLs and physical activity for health care application research, this study emphasizes the essential role of incorporating design principles via a DED framework in developing health care applications. This integration is crucial for aligning motion data collection with practical, real-life applications. By adopting a multidisciplinary approach and combining insights from design, engineering, and health sciences, the research demonstrates potential pathways for making health applications more user-friendly and effective. It underscores the necessity of engaging designers in the research process to ensure that health technologies are accessible, intuitive, and tailored to meet the diverse needs of users. Designers play a key role in customizing the deployment and data collection methods of motion-tracking devices, which improves the relevance and accuracy of collected data, leading to better-informed health applications. The iterative dialogue between users and designers during development refines the precision of information and its application, fostering a cycle of continuous improvement. By transforming complex health data into engaging, understandable formats, designers help translate user needs into actionable health solutions, promoting better monitoring and management of health conditions. The study advocates for further research to explore and refine these integrations, offering a direction that promises to transform health care monitoring and interventions, ultimately enhancing patient outcomes and engagement.