Development of an Android-Based Self-Report Assessment for Elderly Driving Risk (SAFE-DR) App: Mixed Methods Study

Introduction

Driving increases the autonomy of elderly individuals, which helps them to participate in activities in the local community and at home [1]. Moreover, elderly drivers who have higher independence and community participation show higher life satisfaction [2]; however, aging is accompanied by factors that impede safe driving, such as perceptual, motor, and cognitive impairments [3,4]. Due to health decline, elderly drivers have much higher accident rates than all other age groups and are more likely to suffer severe injury and death in the event of an accident [5,6]. Therefore, it is becoming increasingly important to develop ways of testing their health status and assessing their suitability for driving [7].

Elderly drivers have the self-control to reflect on their driving ability and prevent driving risk themselves [8]. They generally tend to avoid dangerous driving situations, such as long-duration driving, high-speed driving, and uncomfortable driving situations such as driving at night or in the rain [9]. Using these characteristics of elderly drivers, continuing research is being conducted to develop various self-report assessments and test their correlation with driving risks [10].

Self-report assessment plays an important role in elderly drivers’ decision-making on the continuation of driving and also presents the following advantages: individuals can test their status using a relatively simple method, the assessment can be distributed to a broad population online, and it provides rapid feedback [11]. Existing self-report assessments for elderly drivers were previously only available as paper-based tests; they are now available on CD-ROM [12] and via websites [13]. As such, self-assessment services for elderly drivers are shifting away from the conventional method of paper-based testing to a more efficient way that fits social reality, and the new service modalities should be able to provide older drivers with easier access and faster feedback.

Recently, numerous mobile-based health care apps have been developed to aid health care across a wide population [14]. Although the elderly population may have restricted access to mobile-based services, this is outweighed by various benefits, such as diverse access to information, convenience, and reduced social isolation [15]. Mobile-based services for self-report assessments for elderly drivers have not been attempted so far because elderly persons are not familiar with the mobile environment. Nevertheless, according to the mobile technology acceptance model for the elderly aged 65 years and older, they are strongly influenced by surrounding reference social groups (children and relatives) and do not show major differences from other age groups in motivation to access mobile devices [16]. In other words, even though elderly drivers may experience some difficulties performing mobile-based self-report assessments, it is important to investigate the usability of new methods that provide easier access and feedback.

In the United States, Austria, and most European countries, screening policies based on driving self-assessments are widely used for driver’s license renewals for the elderly population; assessments include the Driving Decisions Workbook, Older Driver’s Self-Assessment Questionnaire, Driving Safely While Aging Gracefully, Safe Driving Behavior Measure, and Driver 65 Plus, among others [17-20]. South Korea also has a traffic safety policy that shortens the length of license renewal and provides professional evaluations for elderly drivers [21]. Within these policies, a means of rapidly screening the driving risk for a wide range of elderly people is needed.

Patterns of driving behavior vary depending on sociocultural characteristics, and elderly drivers’ driving abilities are strongly influenced by perceptual ability and culture [22]. Therefore, it is necessary to thoroughly examine the reliability and validity of self-report assessments in which elderly drivers examine their own status within the corresponding sociocultural environment. To this end, self-report assessments have been developed to suit the characteristics of various countries, such as the United States, Canada, and Australia [11,23]. Likewise, the Self-Report Assessment For Elderly Driving Risk (SAFE-DR) was developed as a self-report assessment for elderly drivers suited to the cultural characteristics of South Korea (Multimedia Appendix 1) [24].

With the above evidence, the purpose of this study was to develop and test a mobile version of a self-report assessment app for elderly persons contextualized to the South Korean culture. We then tested its validity and reliability by comparing its results to that of an established, validated, paper-based test.

Methods

Study Design

This was a mixed methods study that included a qualitative exploratory method of a focus group to develop a mobile-based version of the SAFE-DR and a cross-sectional survey conducted among elderly South Korean drivers to assess the validity and reliability of a self-administered Android-based mobile app in assessing their driving risk. Before starting the study, the entire protocol was approved by the Kwangju Women’s University Institutional Review Board (No. 1041485-201709-HR-001-29). All participants received and understood an oral and written explanation of the entire study and provided written informed consent to participate.

Procedure

This study constituted three phases: (1) development and implementation of the SAFE-DR app, (2) testing of the reliability and validity of the SAFE-DR app, and (3) analysis of the correlation between the SAFE-DR app and a paper-based self-report assessment for elderly drivers.

Phase One: Implementing the SAFE-DR App

App Configuration

To implement the SAFE-DR as an Android app, we used the qualitative exploratory method of focus group interactions [25]. The focus group consisted of 2 app developers and 3 professors specializing in elderly driving rehabilitation. The investigators in this study directed communications within the focus group; the professors specializing in elderly driving rehabilitation composed app scenarios, and the 2 app developers used the Android Studio to produce a pilot file based on these scenarios. App scenarios consisted of using functional and timing point of views to describe or visualize interactions between users and mobile systems, and app developers anticipated use episodes before coding into computer languages. Elderly driving experts discussed scenario visualization, content application on a page, score expression as an evaluative result, and explanations to users about items with low scores. The scenario, including the contents discussed and the message sequence chart, was written using the Excel 2016 program (Microsoft Corp) for each app page. Based on this scenario, the app development specialists used a programming language (Java) to construct the app. Next, the whole group participated in a process of qualitative communication to finalize the function and design of the app from the users’ perspective. The app was designed in 4 sections: explanation of evaluation, guidance, and consent to use; data entry of personal and driving characteristics; performance of test through items of each subdomain; and presentation of test results. The test can be completed in about 20 minutes; the app analyzes the subject’s response to each item (based on a 3-point scale) to provide information on the user’s driving ability. It further cautions the driver to be careful or see an expert for a driving ability test. Finally, an app administration website was created for data management.

SAFE-DR

The SAFE-DR was developed by SYC to reflect the characteristics of elderly drivers in South Korea [24]. Via a Delphi survey, a 28-person panel selected 44 assessment items and, after analysis of offline reliability and validity, 38 items were included in the final instrument, with subdomains of on-road, coping, and health. Each item was measured on a 3-point Likert scale of agree, disagree, and strongly disagree. The reliability coefficients for all items (on-road, coping, and health subdomains) were 0.906, 0.921, and 0.913, respectively. The instrument demonstrated content, construct, and predictive validity. In the implementation of the app version of the test in this study, we used the 44 items that showed content validity in the paper-based SAFE-DR, collected data from elderly drivers, and assessed the test items. The app provides information to elderly drivers on any reduction in driving abilities and whether they should discuss their abilities with a driving expert. Based on this information, elderly drivers will be able to make decisions as to whether they should seek expert evaluation or exercise caution in situations to practice safe driving.

Phase Two: Testing the Reliability and Validity of the SAFE-DR App

Testing the Reliability and Validity

Construct validity was analyzed to ensure that the hypotheses for the subdomain configuration of SAFE-DR developed offline were consistent with the actual app-based assessment. The reliability of the configuration items for each subarea was analyzed to select items that were inconsistent. To analyze test-retest reliability, the SAFE-DR app was readministered 2 weeks later to the 81 participants who consented to the retest.

Participants and Data Collection

Data were collected between August 2018 and May 2019 through the SAFE-DR app made by the focus group and through one-to-one interviews with elderly drivers. Table 1 shows the general characteristics of the focus group. The focus group developed the app by discussing the topics of organizing app scenarios (function, point of view, interaction, and visualization) and organizing elderly driver assessment content (expression of evaluation content, items, and results).

Since it is difficult to specify the population of elderly drivers in South Korea, we used convenience sampling and snowball sampling. After obtaining consent from 3 public health centers and 3 general hospitals in Seoul, Gyeonggi-do, and Gwangju, which have large elderly dynamic populations, we installed testing booths and advertised the study to recruit elderly drivers. In addition, we recruited further participants by advertising by word of mouth through the elderly drivers who had taken the test. All individuals who participated in the study received written and verbal guidance for the entire research process and provided written informed consent to participate in the study.

Participants were enrolled between August 2018 and May 2019, and a total of 567 elderly drivers participated in the study. All participants were interviewed to check general and driving-related characteristics; they completed the SAFE-DR assessment themselves using the developed Android app. In addition, participants who consented to further testing completed an offline self-report assessment using the Driver 65 Plus. Individuals met the inclusion criteria if they were individuals aged 65 years and over who possessed a driving license and had at least 1 year of driving experience and whose cognitive and verbal abilities imposed no restrictions on completing the app-based self-report assessment. Based on the inclusion criteria, of the 567 participants, 51 persons were excluded because they were under 65 years of age or provided incomplete responses to the self-report assessment. The remaining 516 participants were included in the final analysis.

Phase Three: Analyzing the Correlation Between the SAFE-DR App and a Paper-Based Self-Report Assessment for Elderly Drivers (Driver 65 Plus Test)

Correlation Between App and Paper-Based Assessment

The third phase was to test the validity of the assessment results of the elderly drivers, who may not be familiar with mobile apps, by analyzing the correlation between the results of the SAFE-DR app and a paper-based offline test. Among the elderly drivers who used the SAFE-DR app, those who consented to further testing were asked to complete the Korean version of the Driver 65 Plus test, which is one of the most widely used offline self-report assessments for elderly drivers. The correlation between the SAFE-DR app and paper-based Driver 65 Plus results was analyzed to verify consistency between the results of the offline and online tests. In the Driver 65 Plus test, lower scores indicated a more positive result, whereas in the SAFE-DR test, higher scores indicated a more positive result. Therefore, we expected to observe a negative value for the correlation coefficient between the two tests.

Driver 65 Plus

This test was developed by the American Automobile Association based on the Safety Research and Education Project of the Teacher’s College of Columbia University [26]. A feature of this test is that the scoring scale for each item is presented differently depending on the content such that elderly drivers cannot predict the question’s intentions. The Korean Driver 65 Plus has shown test-retest reliability of .95 and construct and concurrent validity [27].

Statistical Analysis

For data analysis, we used PASW Statistics (version 18.0, IBM Corp). Participant characteristics were analyzed using the descriptive statistics of mean, frequency, and percentage. Cronbach α and the correlation among all items were analyzed to examine the internal consistency of provisional items in each subdomain of the SAFE-DR and the stability of a retest after 2 weeks.

Exploratory factor analysis was performed to classify the subdomains of SAFE-DR. For exploratory factor analysis, we used principal component analysis, selecting factors with an initial eigenvalue ≥1.0, and used a varimax rotation. To interpret the results, we checked whether the Kaiser-Meyer-Olkin value in Bartlett test of sphericity was ≥0.8, that the factor loading of each item was ≥0.4, and that the eigenvalue of each factor was ≥1.0. Although a factor loading ≥0.5 for each item is ideal, given that this was a sociological study, we interpreted the results using a criterion of 0.4 [28,29]. Pearson correlation analysis was used to estimate the correlation coefficients for each item in test-retest reliability and to estimate the correlation between the paper-based self-report assessment results and the app-based self-report assessment results.

Results

General Characteristics of Participants

Table 2 shows the general characteristics of the elderly drivers who participated in this study. Of the elderly drivers, 79.7% (411/516) were male and 20.3% (105/516) were female, while the mean age was 72.03 years. A total of 45% (232/516) were taking regular medication for hypertension (187/516, 80.6%), diabetes (25/516, 10.8%), hyperlipidemia (11/516, 4.7%), and prostate disorders (6/516, 2.6%); however, 3 drivers (3/516, 1.3%) were also taking antiepileptic medication, which could affect driving. The automobile types most commonly used by the elderly drivers were sport utility vehicles (168/516, 32.6%), midsize sedans (144/516, 27.9%), and compact cars (114/516, 22.1%). The mean driving time of the 516 participants was 22.75 years, the average driving time per week for the 479 currently driving drivers was 4.49 hours, and the average driving cessation period for the 37 participants who were not currently driving was 1.78 years. A total of 4.4% (21/516) had experienced driving accidents during the last 3 months, of which the majority (13/516, 2.5%) experienced a collision due to their own fault.

Internal Consistency

To analyze the internal consistency of the SAFE-DR app, we analyzed Cronbach α and the correlation of each item with the overall score. For all items, Cronbach α=.975, and the correlation of each item with the overall score ranged from r=.520 to r=.823, meaning that all items showed a correlation of at least .50 (Table 3).

For each of the subdomains defined when the SAFE-DR app was constructed, any item that increased Cronbach α when removed was excluded from the assessment; 5 items showed either increased or equal reliability when removed. For items that showed the same reliability, we made decisions regarding deletion based on subdomain composition in the factor analysis. As a result of this process, we decided to remove items 1, 12, 13, and 18.

Construct Validity

In the exploratory factor analysis, the Kaiser-Meyer-Olkin value was 0.963, which was ≥0.8 and close to 1, and the eigenvalues of each factor were ≥1.0, which was independently sufficient to form subdomains. The result of Bartlett test of sphericity, which indicates the suitability of the total data in the factor model, was statistically significant (P<.001, chi-square: 17,265.9; Table 4).

Factor Exploration

Although a factor loading ≥0.5 for each item is ideal, given that this was a sociological study, we derived a solution that satisfies factor loading ≥0.4. Considering the semantic units of individual items, we decided on a factor solution that did not disturb the semantic units of items included in the theoretically chosen subdomains of on-road, coping, and health. Therefore, during factor analysis, the input arrangement of items was used instead of ordering the items by factor component size. As a result, in the final factor matrix, the initial 3 putative subdomains were divided into 5 subdomains. The on-road and coping subdomains maintained their theoretical arrangements, whereas the items included in the theoretical health subdomain were divided into 5 subdomains. Reviewing the semantic content of the grouped items, we defined items 29-33 as the cognitive functions subdomain, items 34-41 as the general conditions subdomain, and items 42-44 as the medical health subdomain (Table 3).

Stability

To analyze measurement stability using the SAFE-DR app, we retested the app using the same test after 2 weeks. We analyzed the correlation between initial test results and retest results, and the correlation coefficient ranged from r=.898 to r=.982. The mean correlation coefficient across all items was r=.951 (Table 2).

Correlation Between App-based SAFE-DR and Driver 65 Plus

To investigate whether elderly drivers, who may not be accustomed to a mobile environment, showed similar trends in the results of the SAFE-DR app and that of the paper-based test, we analyzed the correlation between Driver 65 Plus scores and the total SAFE-DR score with the subdomains. The total SAFE-DR score showed a negative correlation of –0.864 with the Driver 65 Plus score; among subdomains, coping showed the strongest correlation (–0.812). A Pearson correlation coefficient ≥0.8 indicates a very strong correlation. Meanwhile, the on-road, cognitive functions, and general conditions subdomains also showed strong correlations of –0.768, –0.758, and –0.767, respectively. The medical health subdomain showed a significant but moderate correlation of –0.456 (Table 5). Therefore, elderly drivers did not show especially different responses in the app-based method from the paper-based method.

Final App for SAFE-DR

The final app for SAFE-DR is easily accessible through the Google Play app store. Examples of app screens are shown in Figures 1 and 2.

‎View this figure

‎View this figure

Discussion

Principal Findings

In this study, we developed the SAFE-DR as an app-based self-report assessment for elderly drivers in South Korea. We tested the reliability and validity of the SAFE-DR in a mobile environment and demonstrated that the app-based testing method can be effectively used for elderly drivers.

For the development of the conventional paper-based SAFE-DR, 44 items were tested for content validity by an expert panel in a Delphi survey, and 38 items were selected in the subdomains of on-road, coping, and health. In the app-based SAFE-DR test in this study, 40 out of the 44 items were selected; items 9, 19, and 43 had been excluded from the paper-based test but were not excluded from the app-based test. In the factor analysis in our study, these items showed the lowest communality scores in their respective factors. In exploratory factor analysis, communality is a measure of how well each factor represents a given item [28]. Therefore, these 3 items can be interpreted as showing a relatively low correlation with other items in the same factor in the app-based test as well. Likewise, in the analysis of item reliability, these items tended to show lower reliability. When the paper-based test was developed, a sample of 339 participants was used, whereas, in our study, the sample size was 516 persons; reliability and validity analysis results are more likely to show higher values with a larger sample size [30]. Therefore, although the items showing low reliability and validity were similar between the paper-based and app-based tests, we can cautiously surmise that, due to the larger sample size, the results in our test had higher acceptance. The fact that the results of both the offline and online tests showed similar trends can be considered an important indicator in the development of the app-based test.

This study provides new findings as we implemented the SAFE-DR in an app, collected data, and verified its objectivity. Elderly drivers showed similar trends in the app-based test that were similar to using a paper-based, offline method. The results of the analysis showed a very strong negative correlation between the app-based SAFE-DR and the paper-based Driver 65 Plus; among subdomains of the SAFE-DR, medical health showed a moderate negative correlation and other subdomains showed strong negative correlations. These results suggest that the results of the app-based test have a linear relationship with the results of the paper-based test. Therefore, elderly drivers who may not be familiar with a mobile environment can use the app-based SAFE-DR without any major differences from the paper-based assessment. A study on the use of mobile-based services by elderly populations in various countries indicated that most elderly individuals are already prepared to use mobile-based services; their use reportedly improves independent living among the elderly [31]. Accordingly, it will be necessary to provide various services using a mobile environment such as apps for elderly drivers in Korea as well, including self-report assessments that can provide immediate and faster access to more information.

Some differences were observed between the paper-based test and our study’s app-based test in subdomain composition. We first hypothesized theoretical factors based on the 3 subdomains of the basic paper-based SAFE-DR, and from the performed exploratory factor analysis, 5 factors emerged. This is because, among the putative subdomains, the health subdomain was split into 3 subdomains; these subdomains’ items are related to fatigue during driving, driving-related cognition, visual perception, physical ability, and medical health status, including medication. The fact that health factors were further divided indicates that for the participants in our study the response to driving-related health status differed depending on specific health factors. Items 29-33, which were combined into one factor, were defined as the cognitive functions subdomain, as they shared content relating to memory, cognition, and geographical orientation. Items 34-41 were defined under the broad heading of general conditions, as they shared content on fatigue during driving, visual perception, auditory perception, physical fatigue, and flexibility. Finally, items 42-44 were defined as the medical health subdomain, as they showed content relating to medication, doctor consultations, epilepsy, and seizures.

The heterogeneity of digital literacy in the elderly population’s use of mobile phones prevents older people from accessing smartphones. In a study on attitudes regarding mobile phone use in the daily lives of the elderly, there are individual differences in accessibility to mobile phones. If they refused to use a mobile phone, they felt uncomfortable reading the text on the touch screen. However, there were individuals who learned through personal networks and user manuals and used mobile phones [32]. In this respect, we tried to improve the visibility for older adults in app-based SAFE-DR configurations. Specifically, we increased the font size, reduced the number of characters displayed per screen, and encouraged the use of scroll functions. In a society where single-person mobile phone use has become routine, the elderly, like the populations of other age groups, can no longer avoid the use of smartphones. Therefore, the elderly should continue to try to use mobile devices. In this study, there was a difference between the paper-based evaluation and the app-based evaluation, but it was not a critical issue and it can be modified through feedback from elderly users. The app is currently being distributed through Google Play and is available free of charge. The researcher will continue to maintain this service for elderly drivers and plans to supplement it through feedback from elderly users.

Comparison With Prior Work

Regarding the subdomain compositions of previous self-report assessments for elderly drivers, the Driving Decisions Workbook developed in the United Kingdom contains on-road, seeing, thinking, getting around, and health subdomains [33]; the Self-Awareness and Feedback for Responsible Driving developed by the US Department of Transportation contains seeing, thinking, and getting around subdomains [13], and the Royal Automobile Club of Queensland Older Drivers’ Self-Assessment Questionnaire developed by the Australian Automobile Association contains driving and health subdomains [34]. Among the subdomains in our study, on-road was included using the same name as previous assessments, and coping refers to the ability to cope with specific situations that can occur while driving on the road, which is consistent with the previous getting around subdomain. For health-related items, the cognitive functions subdomain in our study has similar content to the items in the thinking subdomains in previous assessments. Conversely, the general conditions and medical health subdomains showed differences from those of previous assessments; items in the previous seeing subdomain were partially included in general conditions. Content in the medical health subdomain relating to medication or doctor consultations was previously included in the health subdomain in other assessments; however, since the division of roles between doctors and pharmacists in South Korea, the rate of patients seeking preventive medication guidance or consultations with doctors before disease or an accident has declined [35]. It is thought that these items were classified as a single factor, because unlike in other countries, elderly drivers showed different health status and different patterns in questions related to their medical state. This supports the idea that the app-based SAFE-DR test developed in this study properly reflects the cultural circumstances in South Korea.

The Australasian Model License Assessment Procedure states that the ideal procedure for driver’s license renewal for elderly drivers follows a process of applying for testing through the local community, medical testing, multistage driving ability assessment, and provision of license options [36]. In this testing process, self-report assessments are used to screen the elderly person’s driving ability in the phase of applying for testing through the local community to verify a potential need for specialized tests [37]. Based on this need, various countries use self-assessment tools to monitor the population of elderly drivers [12,13]. Likewise, the SAFE-DR was developed in South Korea as a self-report assessment for elderly drivers suited to the local culture [14], and our study has demonstrated the reliability and validity of a SAFE-DR test provided in an app. Compared to assessments that require meeting with a professional, self-assessment tools allow individuals to test their own abilities, which has the advantage of reducing stress and readily identifying potential problems in a broad population of elderly drivers [23]. In addition, mobile-based services can provide the elderly with various benefits such as access to diverse information, convenience, and reduced social isolation [16]. Therefore, this study’s app-based SAFE-DR test can be used for South Korean elderly drivers both to conveniently test their own driving ability and determine the need for further testing.

Although services provided in a mobile environment have the advantage of easy, repeated access, if the results are not consistent each time the service is used, the user may become confused. Therefore, we analyzed the test-retest reliability when testing was repeated after 2 weeks; most items showed a very high reliability of ≥0.9, demonstrating the test’s ease of access and repeated use.

Limitations

Our study has some limitations. We were unable to control certain factors during participant recruitment, resulting in a high ratio of males, a low proportion of persons aged ≥80 years, and a low proportion of elderly drivers who were not currently driving. Since we were developing an assessment to be used by South Korean elderly drivers, it was necessary to recruit a better-matched sample of participants and compare the test results according to participants’ characteristics. In addition, because we only provided an app-based service for Android, it will be necessary to expand the test to other platforms to allow use by a larger population.

Key Findings

Despite these limitations, this study developed, for the first time, the SAFE-DR into an app-based self-report assessment that reflects the cultural characteristics of Korean elderly drivers. While many mobile-based services have recently been offered due to the advantages of easy access and fast information delivery, elderly persons who prefer conventional ways may have difficulty using such services [38]. This study confirmed that the results of both the app-based SAFE-DR evaluations and the paper-based assessments, which are familiar to the elderly, were consistent. Further, this study identified the factors for screening the driving abilities of Korean elderly drivers and tested the reliability for repeated use of self-assessment apps.

This assessment has implications on policies and traffic safety for Korean elderly drivers. Restrictions on the license of elderly drivers are beneficial for their traffic safety and that of the general population; however, these policies should include methods for education and preventive inspections of equipment to help maintain the licenses of elderly drivers as long as possible [39]. The app-based evaluation in this study can be used as means for preventive monitoring and education of the elderly driver population. Therefore, the assessment can help with the education and screening sections of the recent policy for renewing elderly drivers’ licenses in South Korea [21]. Within these policies, this assessment can contribute to the safe continuation of driving by facilitating testing of driving ability and providing relevant information to elderly drivers in South Korea. Finally, future research should focus on expert driving ability assessment and licensing restriction systems for the policy on renewal of elderly Korean’s driver’s licenses.

Conclusions

In this study, we developed the SAFE-DR into an app-based self-report assessment for elderly drivers and tested its reliability and validity. In South Korea, where the aging population is rapidly increasing, the app can help elderly drivers to easily diagnose their driving skills and protect themselves from accidents while driving. It was designed to be easily used by elderly drivers and to provide essential information related to driving. Therefore, we anticipate that this assessment can contribute to safe continuation of driving by facilitating testing of driving ability and providing relevant information to elderly drivers in South Korea.