This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
To decrease the burden of diabetes in society, early screening of undiagnosed diabetes and prediabetes is needed. Integrating a diabetes risk score into a mobile app would provide a useful platform to enable people to self-assess their risk of diabetes with ease.
The objectives of this study were to (1) assess the profile of Diabetes Risk Score mobile app users, (2) determine the optimal cutoff value of the Finnish Diabetes Risk Score to identify undiagnosed diabetes and prediabetes in the Chinese population, (3) estimate users’ chance of developing diabetes within 2 years of using the app, and (4) investigate high-risk app users’ lifestyle behavior changes after ascertaining their risk level from the app.
We conducted this 2-phase study among adults via mobile app and online survey from August 2014 to December 2016. Phase 1 adopted a cross-sectional design, with a descriptive analysis of the app users’ profile. We used a Cohen kappa score to show the agreement between the risk level (as shown in the app) and glycated hemoglobin test results. We used sensitivity, specificity, and area under the curve to determine the optimal cutoff value of the diabetes risk score in this population. Phase 2 was a prospective cohort study. We used a logistic regression model to estimate the chance of developing diabetes after using the app. Paired
A total of 13,289 people used the app in phase 1a. After data cleaning, we considered 4549 of these as valid data. Most users were male, and 1811 (39.81%) had tertiary education or above. Among them, 188 (10.4%) users agreed to attend the health assessment in phase 1b. We recommend the optimal value of the diabetes risk score for identifying persons with undiagnosed diabetes and prediabetes to be 9, with an area under the receiver operating characteristic curve of 0.67 (95% CI 0.60-0.74), sensitivity of 0.70 (95% CI 0.58-0.80), and specificity of 0.57 (95% CI 0.47-0.66). At the 2-year follow-up, people in the high-risk group had a higher chance of developing diabetes (odds ratio 4.59,
The Diabetes Risk Score app has been shown to be a feasible and reliable tool to identify persons with undiagnosed diabetes and prediabetes and to predict diabetes incidence in 2 years. The app can also encourage high-risk people to modify dietary habits and reduce sedentary lifestyle.
Prevention of diabetes is at the top of the agenda for health promotion worldwide and in Hong Kong [
mHealth has been used for different purposes in health promotion and maintenance. The proliferation of mobile phones and software apps has provided a new channel for health promotion, including symptom recording [
Integration of a diabetes risk score into a mobile app is an innovative and potentially powerful tool for promoting diabetes self-assessment. Evidence has shown that the Finnish Diabetes Risk Score (FINDRISC) has supported diabetes screening and prevention because of its cheap, easy-to-administer, convenient, and noninvasive features [
The DRS app was developed by a university project team comprising a nursing faculty with rich experience in health literacy interventions, 2 family medicine experts, a professor in endocrinology, and a statistician. User tests were conducted with 22 Chinese adults in April 2013. Half of the participants of the user test had secondary education or above. Comments were collected from these app users on the clarity of the instructions given in the app, the logic of the sequence of the questions in the app, and the ease of data input by the participants. The app was then revised according to their comments. The DRS app (version 2) was officially launched on August 28, 2014 in a media interview with 11 newspapers and 1 electronic media, who reported its launch. The Quick Response code of the app was posted on the university website to promote the app. This free-of-charge app could be downloaded from both the Google Store (for Android devices) and the App Store (for iOS devices; eg, iPad, iPhone) by searching the term “HKUDRS.”
The DRS app included the Chinese FINDRISC and questions related to lifestyle (such as smoking, drinking, dietary pattern, and physical activity engagement). In the DRS app, the exact diabetes risk score was not shown, but the risk level was shown in a figure in which a pointer fell in one of the two color zones, with red indicating high risk and green indicating low risk.
The objectives of this study were to (1) assess the profile of DRS mobile app users, (2) determine the optimal cutoff value of the diabetes risk score to identify UDPD in the Chinese population, (3) estimate users’ chance of developing diabetes within 2 years of using the app, and (4) investigate high-risk app users’ lifestyle behavior changes after ascertaining their risk level from the app.
We divided the whole study into 2 phases. We conducted phase 1 from August 2014 to October 2016, using a cross-sectional design. Phase 1a assessed the users’ profile, while phase 1b assessed the appropriate cutoff value of the diabetes risk score to identify UDPD in the Chinese population. We conducted phase 2 from October 3, 2016 to November 6, 2016 with a prospective cohort design. Phase 2a followed up the app users to estimate their chance of developing diabetes within 2 years. Phase 2b assessed the app users’ lifestyle changes after knowing their risk of diabetes from the app.
Since this is a free app in the app stores, anyone who was capable of accessing the internet and app stores and of reading and understanding Chinese could download the app to their mobile phones. On the first screen of the app, we indicated that it was developed for research purposes, and its use implied that users agreed to join the research study and consented to having the project team use their data in aggregate for research purposes and future analysis. We included those who met the following criteria in the analysis in phase 1a: (1) aged 18 years or over, and (2) having a phone number indicating the country code 852 (for Hong Kong). We selected participants for phase 1b from phase 1a who met the following criteria: (1) provided phone or email addresses in the app and agreed to allow the project team to approach them, (2) had never had a diagnosis of diabetes (of any kind), and (3) were willing to attend a comprehensive health assessment (including blood taking) in a university campus. Participants in phase 2 were those who (1) used the DRS app in 2014 and 2015, (2) provided email addresses in the app, and (3) were willing to complete an online survey. We excluded those with known diabetes in 2014 and 2015 or those who used the app less than 1 year from the time we conducted the online survey.
We calculated the sample size for phase 1b for the receiver operating characteristic (ROC) curve analysis using MedCalc software version 15.8 (MedCalc Software bvba). We assumed that an area under the curve (AUC) of 0.70 for a particular test was significant from the null hypothesis value of 0.5. The prevalence of people with UDPD in Hong Kong was almost 14% [
We calculated the sample size for phase 2a using G*power version 3.1.9.2. With reference to the previous studies, the OR of developing diabetes among persons with UDPD was 4.5 [
We calculated diabetes risk score by using the FINDRISC formula and the score for the following 8 items in the app: age (<45 years=0, 45-54 years=2, 55-64 years=3, >64 years=4), body mass index (BMI; <25 kg/m2=0, 25-30 kg/m2=1, >30 kg/m2=3), waist circumference (for men: <94 cm=0, 94-102 cm=3, >102 cm=4; for women: <80 cm=0, 80-88 cm=3, >88 cm=4), history of using drugs for high blood pressure (no=0, yes=2), history of being told by health professionals about the possibility of having high blood glucose (no=0, yes=5), participation in physical activity every day (yes=0, no=2), habit of consuming fruit and vegetables every day (yes=0, no=2), and family history of diabetes (no=0, yes=5). We calculated the total risk score by adding the scores of all items, with a possible range from 0 to 26 [
In phase 1a, users input data into the app on their own, and the data included the 8 items (such as daily intake of fruit and vegetables and daily physical activity) that we used to calculate the users’ diabetes risk. Following the launch of the app, we periodically monitored the number of downloads and ensured that the app was downloadable from the app stores. For the sake of protecting privacy, only the principal investigator (AYML) had right of access to the server. Before passing the data to the trained research assistant for data cleaning and to develop the database for this study, the principal investigator removed all app users’ personal data to protect privacy.
In phase 1b, we sent emails to the app users and invited them to receive a 1-hour comprehensive health assessment in the university campus between June and August 2015. The inclusion criteria were stated clearly in the invitation emails, and the app users replied to the emails and indicated their willingness to join the health assessment. In the assessment, a research nurse took 5 mL of venous blood from each participant. The blood samples were sent to the laboratory of a regional public hospital in which glycated hemoglobin (HbA1c) was measured by high-performance liquid chromatography (Variant II Turbo Hemoglobin Testing System, Bio-Rad Laboratories, Inc, Hercules, CA, USA). According to the American Diabetes Association, HbA1c of 48 mmol/mol (6.5%) or greater is considered to indicate diabetes, while HbA1c of 39 mmol/mol (5.7%) or greater but less than 48 mmol/mol (6.5%) is considered to indicate prediabetes [
We contacted app users with HbA1c levels higher than 48 mmol/mol (6.5%) by phone and encouraged them to consult their family doctors. We did this for ethical reasons, so that the app users with abnormal readings were not placed at a disadvantage by not receiving necessary treatments.
For phases 2a and 2b, we sent invitation emails to the app users and encouraged them to complete a follow-up questionnaire via Zoho Survey (Zoho Corporation) from October 3, 2016 to November 6, 2016. We identified duplicate inputs by checking the respondents’ email addresses. We deleted a few duplicate inputs before doing the analysis. Diabetes incidence was recorded when the app users self-reported having diabetes during this period. If the app users had a diagnosis of diabetes, we asked them to provide the dates of receiving the diagnosis (month and year). We also asked app users to input their daily intake of fruit and vegetables, daily physical activity, and the relevant diabetes-related items in the questionnaire.
We obtained approval from the Institutional Review Board of the University of Hong Kong/Hospital Authority Hong Kong West Cluster. The consent form was embedded on the first screen of the app. App users clicked a button on the app to indicate their consent to join the study.
We considered duplicate inputs, incomplete inputs, or inputs that were exactly the same as the default figures in blood pressure, body height, and body weight as invalid inputs and excluded these from the analysis. We computed descriptive statistics, including means, standard deviations, frequencies, and percentages, to present the participants’ sociodemographic information and the distribution of diabetes risk scores. In phase 1b, we used ROC curve analysis to determine the optimal cutoff level of diabetes risk scores with reference to the usual clinical practice for suspecting the risk of diabetes (or the stage of prediabetes) in Hong Kong. We also determined the optimal cutoff point of the diabetes risk score by the sensitivity, specificity, maximum value of the Youden index (sensitivity + specificity – 1), and Cohen kappa of the diabetes risk score with reference to the agreement with HbA1c. Cohen kappa values of zero or less indicated no agreement, 0.01 to 0.20 none to slight, 0.21 to 0.40 fair, 0.41 to 0.60 moderate, 0.61 to 0.80 substantial, and 0.81 to 1.00 almost perfect agreement [
In phase 2a, we used multiple logistic regression models to examine the OR of diabetes incidence between the high-risk group and the low-risk group (classified by the recommended FINDRISC cutoff value found in phase 1b). Educational level is a confounder in diabetes. Many studies have proved that people with a higher educational level have a better lifestyle in their daily lives, and people with a healthy lifestyle will have less risk of developing diabetes in the future [
In phase 2b, we applied paired
We collected data in the period of August 28, 2014 to December 31, 2016. A total of 13,289 Chinese residents downloaded the DRS app and self-assessed their risk. After cleaning the data, we considered 4549 as valid data in phase 1a. A total of 3171 (69.71%) were Android users and the rest were iPhone users. The mean (SD) diabetes risk score was 9.10 (SD 4.85). A total of 1042 (22.91%) were shown to be at risk of developing diabetes.
A total of 972 people left their contact information for the project team and were included in the invitation in this phase. Only 210 participants (21.6%) agreed to attend the health assessment. Of these, we excluded 22 people who had type 1 or type 2 diabetes from the study, so eventually we included only 188 app users in the analysis.
Demographics of the users of the Diabetes Risk Score mobile app.
Variables | Phase 1a (n=4549) | Phase 1b | Phases 2a and 2b (n=127) | |||||
HbA1ca <39 mmol/mol (5.7%), (n=109), n (%) | HbA1c ≥39 mmol/mol (5.7%), (n=79), n (%) | DRSb<9, n (%) | DRS≥9, n (%) | |||||
Male | 2738 (60.19) | 66 (60.6) | 50 (63) | .62 | 82 (64.6) | 122 (61.3) | .56 | |
Female | 1811 (39.81) | 43 (39.5) | 29 (37) | 45 (35.4) | 77 (38.7) | |||
≤44 | 1005 (22.09) | 14 (12.8) | 4 (5) | .43 | 23 (18.1) | 4 (2.0) | <.001 | |
45-54 | 610 (13.4) | 45 (41.3) | 31 (39) | 15 (11.8) | 26 (13.1) | |||
55-64 | 1328 (29.19) | 41 (37.6) | 36 (46) | 35 (27.6) | 79 (39.7) | |||
≥65 | 1606 (35.30) | 9 (8.3) | 8 (10) | 54 (42.5) | 90 (45.2) | |||
Primary or below | 914 (20.1) | 2 (1.8) | 4 (5) | .76 | 17 (13.4) | 29 (14.6) | .31 | |
Secondary | 1724 (37.90) | 63 (57.8) | 49 (62) | 60 (47.2) | 76 (38.2) | |||
Tertiary or higher | 1911 (42.01) | 44 (40.4) | 26 (33) | 46 (36.2) | 85 (42.7) |
aHbA1c: glycated hemoglobin.
bDRS: diabetes risk score, based on the Finnish Diabetes Risk Score.
Most of these participants were aged between 45 and 64 years (153/188, 81.4%) and 116 (61.7%) were male. These app users had a relatively higher educational level than the general public, with 182 (96.8%) reporting secondary school or higher qualifications [
Among the 188 users who participated in blood tests, 79 (42.0%) had HbA1c of 39 mmol/mol or greater (ie, 5.7%). Thus, we considered them to have UDPD. Of these 79 participants, 14 (17.7%) had an HbA1c even higher than 48 mmol/mol (ie, 6.5%), and we considered them to have undiagnosed diabetes. There were no significant differences in age, sex, and educational level between app users with UDPD and those with normal HbA1c level (
In phases 2a and 2b, 326 app users replied to the invitation emails and completed the online survey. Nearly half of the app users were aged 65 or over (144/326, 44.2%), and only 27 (8.3%) were aged 44 years old or younger. Most of these app users were male (204/326, 62.6%). Regarding their educational level, 136 (41.7%) of these app users had secondary qualifications and 131 (40.2%) had tertiary qualifications or higher. Among these, we considered 199 app users (61.0%) to be in the high-risk group when a diabetes risk score of 9 or higher was applied. People in the high-risk group were older than those in the low-risk group (
The mean follow-up time of the app users was 22.66 (SD 5.83) months. After a nearly 2-year follow-up, 15 participants had developed diabetes. The mean time of diagnosis of diabetes during follow-up was 12.93 (SD 8.28) months, ranging from 1 to 25 months. The incidence rate of diabetes was 25.56 per 1000 person-years. For the high-risk group, the incidence rate of diabetes was 36.50 per 1000 person-years. For the low-risk group, the incidence rate of diabetes was 8.59 per 1000 person-years. Fisher exact test showed that the association between the risk groups and diabetic incidence was marginally insignificant (
App users who were informed that they had a higher risk of developing diabetes improved their daily intake of vegetables (baseline: mean 0.76, SD 0.43; follow-up: mean 0.93, SD 0.26;
Characteristics of the Finnish Diabetes Risk Score (FINDRISC) using different cutoff values to predict undiagnosed diabetes and prediabetes (glycated hemoglobin ≥39 mmol/mol, or 5.7%).
FINDRISC cutoff values | Sensitivity (95% CI) | Specificity (95% CI) | Positive predictive value (95% CI) | Negative predictive value (95% CI) |
>6 | 0.78 (0.68-0.87) | 0.39 (0.30-0.49) | 0.48 (0.40-0.57) | 0.72 (0.59-0.83) |
>7 | 0.75 (0.64-0.84) | 0.48 (0.38-0.58) | 0.51 (0.41-0.60) | 0.72 (0.60-0.82) |
>8a | 0.70 (0.58-0.80) | 0.57 (0.47-0.66) | 0.54 (0.44-0.64) | 0.72 (0.61-0.81) |
>9 | 0.61 (0.49-0.72) | 0.62 (0.53-0.72) | 0.54 (0.43-0.65) | 0.69 (0.59-0.78) |
>10 | 0.52 (0.40-0.63) | 0.75 (0.66-0.83) | 0.60 (0.48-0.72) | 0.68 (0.59-0.77) |
>11 | 0.41 (0.30-0.52) | 0.82 (0.73-0.88) | 0.62 (0.47-0.75) | 0.65 (0.57-0.73) |
aFINDRISC >8 was the optimal value.
Receiver operating characteristic (ROC) curve analysis of the performance of the Finnish Diabetes Risk Score (FINDRISC) in identifying undiagnosed diabetes and prediabetes. Diagonal segments are produced by ties. The area under the ROC was 0.67 (95% CI 0.60-0.74). When the FINDRISC cutoff value was >8, its sensitivity was 0.70 (95% CI 0.58-0.80) and specificity was 0.57 (95% CI 0.47-0.66).
Logistic regression model of diabetes incidence between the high-risk app users and low-risk app users.
Covariates | Model 1 (unadjusted) | Model 2 (adjusted) | |||||
Odds ratio | 95% CI | Odds ratio | 95% CI | ||||
High | 4.37 | 0.97-19.69 | .06 | 4.59 | 1.01-20.81 | .05 | |
Low (reference) | 1 | – | – | 1 | – | – | |
Male | – | – | – | 1.21 | 0.38-3.88 | .74 | |
Female (reference) | – | – | – | 1 | – | – | |
Primary or below | – | – | – | 1.33 | 0.42-4.31 | .63 | |
Secondary | – | – | – | 0.96 | 0.19-5.00 | .96 | |
Tertiary or higher (reference) | – | – | – | 1 | – | – |
This study examined the development of a mobile app for identifying people with UDPD and its use over a period of 2 years. Both Android and iPhone users found this app accessible, and more than 13,000 people had downloaded and used this app. Although we included only app users in Hong Kong in the analysis, we noted that many downloads were from countries in Asia, the United States, and Europe. This showed the potential for expanding the use of this mobile app to people who can read and understand Chinese around the world. To the best of our knowledge, this app was the first DRS app targeted at a Chinese population for diabetes risk self-assessment in 2014. The American Diabetes Association recommends screening for diabetes at 3-year intervals after the age of 45 years, particularly for those who are overweight (whose BMI is ≥25 kg/m2) [
This DRS app, adopting FINDRISC as the key measure for estimating diabetes risk, has been shown to be a reliable tool, although it cannot replace diagnostic investigations such as the oral glucose tolerance test, HbA1c level, and clinical judgment. Validated with HbA1c measures, this DRS app performed well in detecting people with UDPD. The ROC curve analysis suggested that a FINDRISC greater than 8 had sufficient properties for identifying persons with UDPD. We considered other values such as FINDRISC greater than 9; however, the sensitivity of this score decreased dramatically to 0.61. Therefore, we would recommend a FINDRISC of 9 as the optimal cutoff point for identifying UDPD in a Chinese population. The sensitivity and specificity of the recommended cutoff points are reasonably good. The new recommended cutoff points in this DRS app could identify nearly 70% of persons at high risk of diabetes. Approximately 1 million of the Hong Kong population were not aware of their UDPD status [
The new recommended optimal cutoff value of FINDRISC in the DRS app was comparable with the cutoff values in other populations (
In this prospective study, two important pieces of evidence were worth noting. First, this DRS app had not only a concurrent but also a predictive nature (indicating the chance of developing diabetes in the next 2 years). Evidence showed that app users in the high-risk group had a significantly higher chance (4.59 times) of developing diabetes than those in the low-risk group. This finding echoed the findings of previous studies in other populations. In Colombia, the risk of incidence of type 2 diabetes in 1 year among participants with high risk scores was 4.8 times that among the low-risk group [
Second, the app encouraged people to change their lifestyle. App users who were informed of having a high risk of developing diabetes by the DRS app significantly improved their daily intake of vegetables and did more physical activities in the follow-up period. This showed that the DRS app was a practical tool in health promotion for the general public. App users seemed to be more cautious about their lifestyles and started to develop healthier habits that could protect them from serious health problems such as diabetes [
Some researchers have developed different sets of diabetes risk score models for the Chinese population in recent years. Tian et al developed the Dagang dysglycemia risk score model to identify UDPD for the oil field working-age population [
This study has some limitations that need to be addressed. First, we used only the HbA1c test as the diagnostic standard to identify people with UDPD; however, the 75-g oral glucose tolerance test is the reference standard for diagnosing diabetes. We considered HbA1c because it was more convenient and time saving than the oral glucose tolerance test. Second, the BMI and waist circumference cutoff values used in the calculation of FINDRISC are for white populations [
First, future studies could explore the existence of other predictors such as dietary sodium, beverage, and fat intake to improve the predictive validity of FINDRISC in the Chinese population. Predictors of diabetes have been used to modify FINDRISC. Studies in Germany and the Philippines modified and simplified FINDRISC for their populations [
This DRS app was a reliable tool for identifying persons who had UDPD. The odds of developing diabetes were much higher among the high-risk app users than among the low-risk users, evidencing the predictive power of the app in diabetes incidence. The app can also encourage high-risk app users to modify their lifestyle for the sake of reducing their progression from prediabetes to diabetes. This is an illustration of the use of a mobile app in health promotion and disease prevention.
Finnish Diabetes Risk Score (FINDRISC) cutoff values for the detection of undiagnosed diabetes, prediabetes, and hyperglycemia in various populations.
area under the curve
body mass index
Diabetes Risk Score app
Finnish Diabetes Risk Score
glycated hemoglobin
receiver operating characteristic
undiagnosed diabetes and prediabetes
The authors wish to acknowledge the anonymous reviewers for their thoughtful review and guidance. This study was funded by the Knowledge Exchange (KE) Funding Exercise for Impact Project from the University of Hong Kong (project no: KE-IP-2013/14-41), University of Hong Kong/China Medical Board Grants and University of Hong Kong Seed Funding Programme for Basic Research, 2015-16 (Project no: 4003430).
AYML contributed to the study design, data analysis, interpretation of the findings, intellectual input, and revision of the manuscript. XYX and MKTC contributed to data collection, data analysis, interpretation of the findings, and drafting of the manuscript, and read and approved the final manuscript. PHC, YTEY, and JYHW contributed to study design, data analysis, and interpretation of the findings. CKHW and DYTF contributed to data analysis and interpretation of the findings. CLKL contributed to study design, intellectual input, and revision of the manuscript.
None declared.