Mild cognitive impairment (MCI) is an intermediate stage between age-related cognitive decline and dementia, which affects the individual’s cognitive, social, and mental aspects and can lead to emotional problems that affect daily living [1]. A recent study has revealed that the incidence of MCI is 6.7% for individuals aged 60 to 64 years, 8.4% for those aged 65 to 69 years, 10.1% for those aged 70 to 74 years, 14.8% for those aged 75 to 79 years, and 25.2% for those aged 80 to 84 years [2]. Previous evidence has shown that patients with MCI develop dementia or Alzheimer dementia (AD) at a rate of approximately 10% to 15% per year [3,4], significantly higher than the incidence of dementia in the general population, which is 1% to 2% per year. The increasing prevalence of dementia, coupled with a high conversion rate, presents challenges not only for those directly impacted by the condition, including individuals, caregivers, and families, but also for society.

Despite the high risk of dementia for people with MCI and the need for early intervention [5], there are currently no drugs or other treatments to modify the clinical course or delay the onset of dementia [6]. In clinical settings, the diagnostic process for MCI could be costly, often involving expensive and sometimes invasive or time-consuming examinations. In addition, the early symptoms of MCI are subtle, making it easy for patients to attribute them to the normal aging process rather than to cognitive impairment and leading to an early stage where patients may not undergo clinical examination. Therefore, achieving an effective MCI diagnosis remains one of the most challenging tasks in geriatric psychiatry [7].

As a comprehensive activity, drawing requires various cognitive skills including orientation, selective and sustained attention, visual memory and reconstruction, visuospatial organization, and motor performance [8]. These requirements suggest that older adults may be susceptible to cognitive dysfunction, and an accurate assessment of their involvement in relevant activities might aid in identifying such disabilities [9]. With the development of portable devices and mobile computing, some researchers have attempted to improve drawing tests using digital methods to capture more information and influencing factors related to screening for MCI, such as working memory, attention allocation, cognitive flexibility, as well as visual and spatial processing. These factors play an essential role in the screening and assessment processes for MCI and help provide more comprehensive information about cognitive functioning, leading to a better understanding of cognitive health and possible problems in participants.

Recently, Müller et al [10] used a Windows Surface Pro 4 digitizer and a handheld stylus pen to evaluate the digital Clock Drawing Test (dCDT) and found a significant difference in time in air between the healthy older adults and the patient group. Afterward, Dion et al [11] discovered a difference in the “thinking time” percentage between the groups. The MCI group took 10 seconds longer to draw their clocks from memory, suggesting that the difference between those with and without MCI may lie in grouping and coordinating the necessary cognitive resources rather than severe drawing errors. Another study demonstrated differences in the time to completion, total pen stroke count, and higher-order decision-making latencies of dCDT between age groups, involving participants recruited as free of dementia and stroke [12]. As they transition from one part of the drawing to the next (eg, postclock delay), participants may use more diverse neurocognitive resources than simply processing speed. In addition to assessing the Clock Drawing Test (CDT), other drawing tasks were also studied. Kim et al [13] created a simplified Rey-Osterrieth Complex Figure Test (sRCFT) to assess digital pen strokes, spatial arrangement, and similarity of drawings in the drawing process, finding that these measures could serve as valuable digital biomarkers for studying visual structural dysfunction in AD. In addition, Müller et al [14] investigated the movement kinematics (ie, time in air, time on surface, and total time) of older adults with MCI when copying a 3D house to reflect their manual dexterity, visual space construction, and other cognitive abilities. Finally, in the digital tree drawing test, patients with MCI and early-stage AD used fewer colors and line widths, and their images displayed reduced contrast and heterogeneity [15].

Digital drawing tasks provide standardized test administration, more detailed feedback metrics, and automated scoring than traditional paper-based tests. However, these assessment tasks typically focus on the patient’s overall performance, neglecting to capture detailed features (ie, clock face, numbers, and pointers in dCDT). In modern digital devices, tracking subtle cognitive changes in patients’ selection of drawing areas, organization of shapes, and determining image sequences may offer a more practical approach to MCI detection.

The dual-task paradigm is an experimental design approach for exploring the effects of multitasking on cognitive performance and brain function. In the dual-task paradigm, participants must perform ≥2 tasks simultaneously, involving different cognitive processes, such as attention, stimulus encoding, decision-making, working memory, response selection, and execution. This paradigm could detect changes in dual-task performance early in the progression of the most common neurodegenerative diseases [16]. Compared to other cognitive assessment tools, the dual-task paradigm is less influenced by the education level and is more applicable to participants with different educational backgrounds. In addition, the dual-task paradigm is quick, practical, and easy to apply in clinical practice, making it a promising method for cognitive assessment [17].

In studies related to the diagnosis of MCI, the analysis of dual-task gait tests has proven to be a valid method. The dual-task gait test is a commonly used clinical assessment that requires participants to perform a cognitive task while simultaneously walking. In other words, participants must complete additional cognitive tasks such as counting, recalling words, or performing attention shifts while walking. For example, Montero-Odasso et al [18] conducted a dual-task gait test on 112 older adults with MCI and followed them for 6 years. Interestingly, high dual-task gait costs were associated with a 3.8-fold and a 2.4-fold increased risk of progression to dementia when counting backward and naming animals, respectively. Whitson et al [16] investigated gait-cognitive dual-task performance in 29 older adults and found that APOE ɛ4 carriers exhibited more pronounced dual-task interference than low-risk participants. Compared to electronic sidewalks, Aoki et al [19] used a Kinect sensor to capture the whole-body movements of participants and extracted more substantial gait feature information by improving the motion performance capture method. Their findings revealed that a classifier based on dual-task gait features could detect older adults who scored low on the Mini-Mental State Examination (MMSE) test [20]. In addition, Ali et al [21] used the Vicon Nexus 2.8 motion capture system to obtain kinematic gait parameters such as velocity, peak knee extension angle, and dual-task cost for participants. Their study showed that the kinematic gait parameters of the dual-task peak knee extension angle for story recall gait could effectively differentiate the MCI group from the healthy control (HC) group.

Currently, methods for detecting MCI using the dual-task paradigm focus on gait-motor assessment or developing new screening tools. Commonly used screening methods for cognitive impairment in older adults (ie, CDT and Rey-Osterrieth Complex Figure Test [RCFT]) by neurologists, psychiatrists, and general practitioners still have good clinical outcomes due to their rapid administration, patient acceptability, and simple scoring rules [22,23]. Thus, integrating a dual-task paradigm into clinical screening tools may provide more alternatives to MCI detection.

We designed tablet-based “drawing and dragging” tasks, prototyped with clinically validated drawing tests, and explored the feasibility and efficiency of assessing MCI through movement kinetics under these tasks. In this study, the drawing task improves the traditional test clinicians use to fulfill the requirements for detecting richer cognitive domains. However, the dragging task is designed to meet the cognitive screening of older adults alone at home by providing a broader and more convenient means of self-examination and early warning. Finally, we hypothesized that movement kinetics features extracted from participants during drawing or dragging could effectively distinguish patients with MCI from healthy older adults.

This research was reviewed and approved by the Biomedical Ethics Review Committee of Taiyuan University of Technology (20230118). The patients and participants provided their written informed consent to participate in this study. We provided US $10 to eligible older adults as compensation for participation.

Before initiating the study, we conducted a power analysis to estimate the sample size required to detect significant differences between the MCI and HC groups. This analysis considered an expected effect size of 0.3, a significance level of .05, and a statistical power of 0.8, requiring at least 82 participants per group [37]. Similar research in the field also informed our sample size, which typically used participant numbers within this range for comparable outcomes [38-41].

We rerecruited 207 participants from the geriatrics and neurology department research clinics at our collaborating hospitals who had not previously been involved in the design or development of the experiment. Participants were identified using a purposive sampling method [42], with the process being meticulously overseen by experienced neurologists. The inclusion criteria were that participants must (1) have normal hearing and vision or corrected to normal, (2) be aged >65 years, (3) have completed the MMSE test, (4) have completed the MoCA test, (5) have completed the CDR test, (6) be capable of moderate exercise without physical disabilities, (7) have no severe depressive symptoms or other mental illnesses, (8) be capable of using smart devices (eg, smartphones and tablets). Neurologists contacted potential participants during their clinic visits and explained the study’s purpose, related procedures, and the possible impact of the research findings. Once potential participants expressed interest, neurologists conducted comprehensive medical evaluations, including detailed medical history collection; physical examinations; brain imaging (magnetic resonance imaging or computed tomography scans); and cognitive function assessments (using the MMSE, MoCA, and CDR scales). Of 207 participants, the MCI group comprised 108 (52.2%) participants who scored <26 on the MoCA and had a CDR score of 0.5, while the HC group included 99 (47.8%) healthy older adults without symptoms of cognitive decline. Brain imaging scans revealed no structural abnormalities causing cognitive impairment. Furthermore, all patients with MCI met the criteria proposed by the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association [43]. We also administered the habitual hand questionnaire [44], which consisted of 13 items, to all participants. The MCI and healthy groups were matched for age, gender, hand preference, education, average sleep duration (in general), exercise habit (regularly engaging in exercise or infrequently), and years of smart device use. Table 2 summarizes the clinical and demographic information for all participants.

We implemented digital “drawing and dragging” tasks on the Lenovo Qitian M530-A154 (AMD Ryzen 7 PRO 2700/16 GB). Our system introduction video can be found in Multimedia Appendix 1. Before the experiment began, participants could draw freely with an Apple Pencil until they were prepared to collect data. After the practice mode, given the sRCFT (delayed recall) task rules, we clarified the execution sequence of digital drawing tasks, namely (1) sRCFT raw copy, (2) dCDT single task, (3) dCDT dual task, (4) sRCFT delayed recall, (5) clock “drag and drop,” and (6) sRCFT “point and line.” All tests were run on an iPad 2019 tablet (seventh generation, 3 GB/128 GB, with 10.2 inches and 2160×1620 touch screen), and Apple Pencil was configured for drawing and lining. The experimenter is engaged in the entire data collection process, clarifying the experiment’s rules and procedures for the participants and documenting their feedback. At the beginning of the experiment, the experimenter explained the whole process to the participants and encouraged them to draw freely to familiarize themselves with the use of Apple Pencil. Once participants felt prepared, the test commenced.

A structured questionnaire was used to investigate users’ experiences of the overall cognitive screening process. The questionnaire is rated on a scale of 1 to 5, with higher scores indicating better performance in this area. The questionnaire consisted of four questions.

The unstructured interview was designed to gather user feedback on the digital cognitive tests, aiming to enhance the system further. The discussion centered on two primary themes:

(1) what are your thoughts on the “drawing and dragging” tasks? (2) What aspects do you believe require optimization?

Tablet-based “drawing and dragging” tasks can assess the hand motor function and cognitive abilities of older adults. The data obtained from these tests were used for feature extraction, serving as a digital biomarker to distinguish individuals with MCI from those who were healthy. Therefore, we conducted data cleaning procedures before extracting the features, which involved removing outliers and ensuring data consistency. Next, we extracted 5 features related to hand motor function, including time, stroke, frequency, score, and sequence. All these features were extracted from the drawing or dragging tasks participated by older adults. A comprehensive overview of the details of these 5 categories of features can be found in Multimedia Appendix 2.

Statistical analysis was conducted using SPSS software (IBM Corp) to analyze tablet device demographic characteristics and cognitive data. Age, years of education, sleep duration, years of smart device use, MoCA scores, and MMSE scores were described using means (SDs), while gender, hand preference, and exercise habits were presented as percentages. Furthermore, we conducted the Kolmogorov-Smirnov test to assess the normal distribution of all variables. For those variables conforming to a normal distribution, we applied 2-tailed t tests (for continuous variables) and chi-square tests (for categorical variables) to determine the significance of intergroup differences. For nonnormally distributed variables, we used the nonparametric Mann-Whitney U test to assess intergroup differences and estimated CIs using the Hodges-Lehmann estimator. In addition, logistic regression analysis was performed to evaluate the diagnostic value of the chosen variables in discriminating between healthy older adults and patients with MCI. The statistical significance level for all tests was set at P<.05.

Table 3 exhibits the statistical comparison results between the MCI and HC groups concerning hand motor function features. In terms of MCI detection, the findings reveal discrepancies in the features selected across different tasks, indicating variations in the number of valid features among them. For instance, in the dCDT tasks (ie, dCDT single task, dCDT series 1, dCDT animal names, and dCDT series 3), the number of features with significant (P<.05) was 7, 7, 2, and 8, respectively. Furthermore, our analysis based on the number of selected features indicates that time- and score-based features outperformed stroke- and frequency-based features. Notably, features with significant (P<.001) were predominantly observed in time- and score-based features, while they were rare in stroke- and frequency-based features (only observed in sRCFT raw copy and clock drag and drop). Particularly, significant differences between the 2 groups were observed regarding time in the air, time on the surface, and time being dragged. Compared to the MCI group, the HC group exhibited significantly shorter times for switching between strokes and quicker drawing or dragging speeds. Similarly, the P values of the score features were mostly <.05, except for the sRCFT task (raw copy), which yielded the preferred features for subsequent logistic regression model construction. However, the dCDT animal names outcome was deemed unsatisfactory. During data collection, it was observed that most participants (151/207, 72.9%) were more accustomed to using the names of the 12 Chinese zodiac animals (Chinese folk culture), which may coincide with the clock numbers (1-12).

We converted the drawing of the dCDT and sRCFT into pseudocolor images encoded by a series of colors in the order of strokes (Multimedia Appendix 3). In the dCDT single task, patients with MCI tended to rotate the tablet when writing numbers (making some numbers look upside down) and drawing lines to determine the 12 o’clock direction (MCI group: numbers 2 and 5 in Multimedia Appendix 3). Furthermore, upon analyzing the dCDT dual task, we noted that healthy older adults may exhibit errors in number positioning and slight gaps between numbers in series 1s due to increased cognitive load (HC group: numbers 2, 3, and 4 in Multimedia Appendix 3). As anticipated, both groups encountered challenges with the relatively complex series 3s. Notably, even some healthy older adults displayed significant gaps between numbers. However, in the spatial organization of the sRCFT, there was no discernible difference between the 2 groups (ie, global-first or local-first approach or top-first and bottom-second strategies). Particularly, healthy older adults exhibited good recall of figures, albeit with minor omissions in some details. Conversely, patients with MCI showed poor recall of figures, indicating deficits in visual memory and spatial construction abilities.

To further explore the diagnostic value of the designed tasks in distinguishing patients with MCI from HC participants, we used a forward stepwise inclusion method in which the features extracted from the task were entered into a logistic regression model, where the diagnostic group (HC vs MCI) was considered as the dependent variable, and the task-extracted features were used as the independent variables, as shown in Table 4.

Furthermore, we used 4 metrics to measure the classification performance of the models, including accuracy, sensitivity, specificity, and area under the curve, as shown in Table 5. Receiver operating characteristic curves for logistic regression models were plotted, as depicted in Figure 5. Notably, the sRCFT delayed recall reached 88.4% (183/207) of the highest detection accuracy, while its specificity results were also the best (90/99, 91%). Meanwhile, the area under the curve indicating the authenticity of the detection method is the highest in sRCFT delayed recall, signifying superior performance. In the dCDT dual task, 88% (95/108) of all predicted patients with MCI were actual patients with MCI. In contrast, the sRCFT raw copy yielded lower classification results, with an accuracy of 82.1% (170/207). Our interpretation of these findings is that relatively complex tasks (eg, dCDT dual task and sRCFT delayed recall) are associated with integrated cognitive abilities, including executive functioning, language comprehension and expression, information extraction, and spatial visualization, making them more effective in detecting cognitive deficits.

We compared 4 aspects of sensitivity, specificity, administration time, and self-administration with digital cognitive tests identified in recent years to identify MCI and dementia, as shown in Table 6. For specificity and sensitivity, we averaged the results of the 4 drawing tasks and the 2 dragging tasks. The findings suggest that our cognitive system is equivalent to the best-performing Vigilance and Memory Test [45] in accurately discriminating patients with MCI. Similarly, in identifying healthy older adults, the performance on both types of tasks surpassed that observed in most recent studies, except for slightly lower performance than the screening system based on ROCF [46]. Although the total time for the digital drawing tasks was approximately 15 minutes, each task ranged from 3 to 5 minutes, aligning with the administration time (typically between <5 and 35 minutes) observed in most studies. Moreover, in the home dragging task, the time was generally shorter, and participants could achieve self-administration, a feature rarely observed in previous studies.

Figure 6 illustrates the results of the questionnaire experiment, where the error bars represent SD. All 207 participants rated the screening process highly across all aspects. The mean score for question 1 (4.60) was the highest among all questions, suggesting that our system was user-friendly, enabling most older adults to complete each test. However, the score for question 3 (3.93) was the lowest of all 4 questions, possibly due to the system’s limited interaction, resulting in tasks being perceived as less engaging. Furthermore, most older adults (179/207, 86.5%) reported that our system engaged their cognition and helped identify cognitive deficits during self-screening (question 2: 4.33), expressing an interest in continued use in the future (question 4: 4.20).

Similarly, we gleaned valuable insights from unstructured interviews. All participants concurred that the screening tool is user-friendly and practical. Furthermore, older adults offered valuable suggestions for enhancing the screening tool: (1) integrating social features to bolster user engagement and (2) providing instructional videos for each test to aid participants in comprehending the task requirements and procedures more effectively.

First, we digitized the traditional drawing test and assessed cognitive performance using a digital pen and tablet. The screening system recorded task-related data (eg, time, score, stroke, frequency, and sequence) and automatically uploaded these data to a remote cloud server where they could be analyzed and visualized through statistical analysis for easy access by neurologists and patients. Second, we captured the hand motor function features of each module (ie, clock face, number, and pointer) in dCDT to further explore the specific cognitive deficits of patients with MCI, which are rarely mentioned in the existing literature. In addition, the sRCFT was well received by older adults in practical testing settings and was easier to administer. Third, we introduced 3 tasks related to language function in the dCDT, expanding the assessment capabilities of the drawing system and increasing its potential application in detecting various cognitive impairments. Fourth, each task was time efficient, taking only 3 to 5 minutes. Older adults were willing to engage with and accept this, effectively avoiding the negative emotional impact of prolonged testing. Fifth, the screening system provides drawing and dragging interaction, which maintains the traditional drawing test requirements while meeting patients’ needs for self-perception screening. In other words, the system can be applied to hospitals and clinics to facilitate physicians to understand patients’ conditions in a timely and effective manner and can also meet the self-assessment of the cognitive level of older adults at home and provide an early warning means.

As for the cognitive assessment results, the 4 drawing and 2 dragging tasks performed well in discriminating abilities (average accuracy of 85.2%), comparable to the results of alternative state-of-the-art methods [47,50,51]. In addition, statistical hypothesis testing was conducted on the 5 categories of critical features acquired during the cognitive assessment. The analysis results provide three insights: (1) time features are the most prominent in cognitive assessment; (2) key features vary in different tasks; (3) most older adults strive for perfection in drawing or dragging tasks and ignore the features of movement kinematics, such as long-term stagnation in the air, multiple drawings to make the figure symmetrical, and slow drawing or dragging speed.

Regarding the digital pen stroke data analysis, we compared differences between the 2 groups in the dCDT. Typically, healthy older adults prioritized drawing the numbers 12, 6, 3, and 9 to divide the clock face area accurately and make the clock more perfect, partly reflecting their ability to generate adequate decision-making strategies [52].

In contrast, patients with MCI were accustomed to rotating the tablet to write numbers in sequence, counterclockwise to write numbers from 12, unable to indicate “10 minutes past 11 o’clock” and refine the scale between numbers 11 and 12. These behavioral patterns reveal deficiencies in patients with MCI’s abilities related to time perception, numerical comprehension, and the organization and planning required for executing cognitive tasks [30,53]. On the other hand, when comparing drawing sequences between healthy older adults and patients with MCI in the sRCFT, most participants followed similar patterns: (1) from top to bottom, (2) from global to local, and (3) checking and filling in missing information. Some patients used unconventional methods, such as mirror drawing or completing the global structure last, possibly due to personal painting habits. Notably, patients with MCI scored lower and omitted many local features in delayed recall, suggesting a link between the loss of detailed features and visual-spatial dysfunction or deficits in visual-spatial working memory [54,55]. Furthermore, our study primarily focused on extracting local features, which may offer better identification of patients with MCI.

In terms of user experience, 207 participants rated the ease of use positively, with an average score of 4.60 (SD 0.398). In contrast, the average score for task enjoyment was 3.93 (SD 0.765), which is less desirable. The “drawing and dragging” tasks are designed as a serious game to maintain rigor while assessing participants’ perceptions through teaching and training methods. However, this approach may be less conducive to generating enjoyment and may challenge user interest, leading to a perception of dullness and tedium. Moving forward, our focus will be on enhancing plot design and incentives and making it resemble a real game by incorporating interactivity and feedback mechanisms.

For our study, it is important to acknowledge several potential limitations and proposed solutions. First, our sample size was relatively small, comprising only 207 participants. To address this, we intend to conduct a longitudinal study to validate the efficacy of the developed MCI screening tool, aiming to recruit a larger and more representative sample. Second, considering the visual condition of older adults, the “drawing and dragging” tasks were performed on a 10.2-inch touchscreen tablet. Therefore, whether this study’s experimental results can be extrapolated to smaller-screen smartphones remains to be determined. We will validate these smartphone tasks to fully evaluate their applicability, considering the user experience on different devices.

We present an MCI detection system based on the digital “drawing and dragging” by assessing the movement kinetics of older adults during various tasks. The interactive system comprises 4 digital drawing tasks and 2 home dragging tasks. We report how these 6 cognitive tasks are designed, optimized, and evaluated through research on traditional clinical screening tools, discussions with related neurologists, and user experience feedback. Then, specific parameters were combined for different tasks, and different mixed models were constructed. The experimental outcomes demonstrate the efficacy of the proposed model in distinguishing patients with MCI from healthy older adults, garnering positive reception from multiple users. In a broader context, this study advances novel approaches for nuanced feature extraction of movement kinetics in individuals with MCI, offering tangible support for developing cognitive impairment detection systems.