A 4-round modified Delphi study was conducted to identify mobile app features and engagement strategies for T2DM self-management. There were three reasons for choosing the Delphi technique: (1) the Delphi technique has been useful for measuring group consensus concerning medical information technology when used in health care; (2) there was no established evidence regarding self-management of T2DM or engagement strategies in the context of developing mobile apps, and (3) the outcomes of Delphi decisions focus on decision making in fields that are strongly susceptible to change. Individual decision makers have more influence than those with underlying rules [20,25]. We used supplemental techniques to ensure the validity of study outcomes by creating a steering committee that includes subject experts to oversee the design, execution, and analysis of all study phases. An agreement between the steering committee members was reached by discussion to approve the participant selection, consensus threshold, survey format and questions, and analysis process; in addition, they wrote down comments and discussed any open points during all 4 rounds. Thus, a modified Delphi technique was used to identify mobile app features and engagement strategies for T2DM self-management.

In total, 11 multidisciplinary experts were purposively selected from a range of scientific networks. Their professions included family physician, clinical informatician, diabetologist, clinical pharmacist, endocrinologist, exercise physiologist, nurse diabetes educator, health educator, clinical dietitian, consultant in medical education, and psychologist. Previous literature reports have indicated that the Delphi technique does not require a particular sample size. The minimum number of participants should be at least 7, although 10 to 20 are advisable [26,27]. A purposive sample is necessary to ensure the variability of the invited experts’ background and experience. The research project steering committee used their expertise to judge the suitability of the invited experts. After identifying the experts, formal contact was established through email, phone, or an in-person meeting. The experts were also asked whether they could further recommend experts who could add value to the project, as a snowball sampling technique. All of the experts who were approached agreed to participate in the project. The experts were all working in Saudi Arabia. Detailed profiles of the respondents are provided in Table 1. The selection of experts participating in the study was based on their diverse professional backgrounds and experience related to the care of people with T2DM. Apart from the clinical informatician, knowledge related to diabetes apps was not one of the selection criteria. Having a similar professional background and experience of a preexisting participant was the only exclusion criteria for the study.

Experts were selected from hospitals, academic institutes, and associations in Saudi Arabia. Moreover, we also invited those who had participated in recent regional diabetes conferences. The inclusion criteria were comprised of 3 parameters: (1) clinicians or researchers with expertise in diabetes care, (2) who possessed at least 3 years of experience in the diabetes care field, and (3) who were willing to participate.

This study included steering committee that was comprised of a diabetes educator; a health informatics specialist and consultant in family medicine and medical informatics; and a consultant in family medicine and diabetologist. The committee oversaw the design, execution, and analysis of all phases of this study. An agreement was reached by discussion to approve the participant selection, consensus threshold, survey format and questions, and analysis process.

A minimum of 3 survey rounds were prospectively planned; rounds would continue until a consensus agreement was reached (Figure 1). The Delphi process started in November 2017 and ended in June 2018. Each round was conducted over 5 weeks, starting with the distribution of the materials. A reminder email was sent 1 week and 24 hours before the closure of the round, and repeated until the experts responded to the survey. The results generated from the previous round would be used in each subsequent round. For data collection in the first round, a questionnaire was sent via email. From the second round onward, an electronic survey instrument (SurveyMonkey) was used to enable efficient and timely data collection from the participants.

The primary draft survey was pilot tested by two experts with expertise in diabetes research who not included in the original study. Pilot testing determined the time it took to complete the survey and whether the survey wording was clear. After minor amendments arising from the pilot testing, the first round survey format was considered complete.

By using a survey with open-ended questions, the first round aimed to generate a list of items. Participants were asked to consider which items should be included when creating a mobile app designed to help people with diabetes self-manage their illness. The initial survey consisted of three sections: (1) a brief overview of the Delphi process; (2) demographic information; (3) two open-ended questions asking participants which items should be included when designing the mobile app. The experts were invited to provide at least three responses to each question:

Each question was preceded with the following statement: “Please answer the questions below based on your clinical experience, knowledge of the guidelines, and relevant theories. Please provide at least 3 responses for each question.”

In the second round, a checklist was generated from the first round results submitted by the experts, and they were asked their opinion. Furthermore, they were also asked to accept, delete, or modify each item. Qualitative responses offered insight into the differences in experts’ perspectives. The items had been modified via qualitative feedback.

In the third round of the Delphi process, participants were invited to rate the importance of each item of the alphabetically ordered checklist generated from the first and second rounds. They were asked to rate their agreement with each of these techniques for the 3 different questions on a 5-point Likert-scale. The scale consisted of 5 responses: 1=strongly disagree, 2=disagree, 3=neither agree nor disagree, 4=agree, and 5=strongly agree, as well as the option, “I do not know.”

Experts were provided with the items listed alphabetically, alongside the mean of agreement ratings. Experts were asked to rank their responses from 1 (most likely to be the best) to n=the total number of items (least likely to be the best) for each question. At this stage, participants were only asked to rank responses for which there had been broad agreement in previous rounds. At this point, there was the option to make final comments, if any.

Ethical approval to conduct this study was obtained from the King Saud University Institutional Review Board (number E-17-2608). All participants were asked to sign a consent form prior to starting the rounds. Experts were asked to create a list of the best features and engagement strategies, which were subsequently rated and ranked anonymously to avoid biases.

A total of 11 experts consented to participate in the study. They had a mean age of 47.09 years (SD 11.70; range 28-62). In total, 4 rounds of Delphi exercises were completed over 6 months, and all of the expert participants (n=11) completed all 4 rounds.

A total of 77 features were generated in response to the first question, “what are the diabetes management components that diabetes experts agree are most likely to be effective when delivered via an app,” with an average of 7 features mentioned per participant in the first and second rounds (Multimedia Appendix 1). Duplicate features were removed, and some were merged (n=36), and the resulting 36 features were categorized into 5 domains (Table 2). No further items were added after rounds 3 and 4 for the whole list of items. Of the 36 features, 11 had a high mean agreement rating (above 4.50). These features were “a reminder for the health care providers' appointments, screening, and routine lab tests” (4.73), “carbohydrate counter” (4.72), and “diet planning” (4.72). All of the next 6 features scored the same (4.64): “nutrition education,” “contacts for guidance if needed,” “offer patient-specific education tailored to the user’s goals, needs, and blood glucose readings,” “find the nearest urgent health care services/centers,” “database for local restaurants and stores providing diabetes-friendly foods,” and “providing Arabic and English versions”. Both “medication reminders by notifications” and “the synchronization with the system, syncing with electronic medical records/personal health records, syncing with glucometers, and continuous glucose monitors or insulin pumps” scored the same (4.54). Overall, the original group of experts displayed weak agreement (Kendall W=0.275) in their ranking of intervention components (χ²₃₅=106.017, P<.001).

There were 5 top features: (1) adjusting insulin doses according to carbohydrate counting and blood glucose readings (5.36); (2) alert caregiver (such as by text message) for abnormal or critical readings (6.09); (3) nutrition education (12.45); (4) contacts for guidance if required (12.64); and (5) offering patient-specific education tailored to the user’s goals, needs, and blood glucose readings (12.90).

A total of 53 engagement strategies were generated in response to the second question, “what are the engagement strategies that diabetes experts believe are the most likely to be effective in diabetes management when using a mobile app.” There were an average of 3.5 strategies suggested per participant in the first and second rounds (Multimedia Appendix 2). After the merging of similar strategies, the number of engagement strategies was reduced to 21 (Table 3). The resultant 21 strategies were categorized into 5 domains. No further strategies were added after Rounds 3 and 4 for the whole list of strategies. Out of the 21 engagement strategies, 4 had a high mean of agreement rating (above 4.40): (1) the app should be a free app (4.64); (2) allow the user to communicate or send information/data to a health care provider (4.54); (3) a user-friendly design (such as educational and age-appropriate design; 4.45); and (4) flexibility and customization (4.45). Overall, the experts showed a moderate degree of consensus in their ranking of the strategies (Kendall W=0.454, χ²₂₀=99.924, P<.001).

The top 5 engagement strategies included several parameters: (1) a user-friendly design (such as an educational and age-appropriate design; 2.82); (2) the app should be free (3.73): (3) allowing the user to communicate or send information/data to a health care provider (5.36); (4) health care providers prescribing the mobile app in the clinic and asking about the patients’ app use compliance during clinical visits (6.91), and (5) flexibility and customization (7.91).

In this study, participating diabetes experts expected many features to be included in a health app for people with diabetes. However, their level of agreement was slightly low, likely due to the variety of the experts’ backgrounds and the differing amounts of experience [29].

There is sufficient evidence from the literature supporting the effectiveness of the top features generated in this study, which are listed in descending order according to the level of rating agreement: (1) a reminder for the health care providers’ appointments, screening, and routine lab tests, which has been mentioned in several studies as a common use of health technology [30] that significantly affects outcomes [31]; and (2) carbohydrate counter and health diet planning, similar to tracking diary, carbohydrate, and meal intake [32,33] were recommended in several studies by diabetes educators who expressed their enthusiasm for viewing detailed dietary macronutrients [29]. The policy statement by the International Diabetes Federation (IDF) Europe uses some categories to differentiate between mobile health apps directed at people with diabetes. The first differentiating category was tracking, logging, and making dietary recommendations [34]. A qualitative study for weight and health management design indicates that the core components of the app should include tailored meal recommendations and assistance with meal planning [35]. For the nutrition education feature, a review found that the most useful mobile health app would help the patient with lifestyle education, self-management, and designing a suitable diet [36]. The IDF Europe categories also mention that such apps should help people with diabetes make food choices, undertake carbohydrate and calorie counting, and calculate medication dosages (similar to an insulin bolus calculator) [34]. A nutrition education app effectively raises awareness in people with diabetes [37].

For the contacts for guidance feature, the current National Health Service (NHS) services offer telephone contact points for youth. In addition, services with transition nurses, coordinators, or support workers offer contact via SMS text messaging [38]. The Saudi Arabian Ministry of Health (MOH) has a Service Center (that can be reached by dialing 937), which offers 24/7 medical doctor consultations [39] and can be added to mobile apps in Saudi Arabia. Several health apps suggested contacting HCP without providing specific contact details, access to experts, or just-in-time resources that could provide this type of guidance [40]. Other apps provided contact details for the diabetes health care team [41,42] or emergency contact lists [38].

For the “offering patient-specific education tailored to the user’s goals, their needs, and blood glucose readings” feature, end users appreciate that it saves time and provides instructions tailored to their specific condition [43]. Additionally, app designers should take into consideration context-sensitive details and condition information [33]. A study conducted to evaluate the effectiveness of diabetes apps discovered that 73% (11/15) of the apps on the market allowed the user to set goals and to visualize times they did not meet their goals, generally using a specific color to indicate hypoglycemia or hyperglycemia [32]. The app should facilitate goal setting (such as weight loss targets, fitness goals) [44] because diabetes is very individual; therefore, the app should be customizable [34]. Matching the participants' perspective, a study showed that participants wanted the medical app to provide information regarding health screening and functions that can assess their health; these must be personalized to them and trustworthy [45]. Targeted exploration of the literature found that action treatment plans and personalized health goals provided by HCP resulted in statistically significant outcomes in several studies [31].

Regarding the “finding the nearest urgent health care services/centers” feature, the app could detect the location of the user with the help of GPS and the global system for mobile communications (GSM) network, and thereby display information about the nearest medical centers. By clicking on a particular hospital, all information regarding that hospital could be provided; Furthermore, this is particularly advantageous for travelers as they could be connected with nearby health care centers if needed, which is a common concern for travelers with chronic diseases [46]. For the “database listing local restaurants and stores providing diabetes-friendly food” feature, a study modified a food database to provide symbolic food names and calorie information, including recipes, ingredients, and local food names [41], which can be added to the diabetes self-management health apps. Another study showed that 11 of 15 available diabetes management health apps on the market focused on tracking carbohydrate intake, while only 3 of 15 had a built-in food database [32]. One study showed that women with gestational diabetes liked that the illustrations of diet-related information could be customized to their culture [41].

For the “provision of Arabic and English versions” feature, a recent Chinese study showed that English- and Chinese-language diabetes self-management apps constitute more than 80% of the 2000 available diabetes apps. Furthermore, the Chinese and English apps have more downloads and are more comprehensive with regard to clinically relevant functions compared to diabetes apps in other languages [42]. A study showed that women with gestational diabetes had previously experienced challenges with care provision that involved the help of an interpreter; furthermore, translating medical terminology into other languages can be challenging [41]. To our knowledge, no study has evaluated the quantity and quality of Arabic diabetes self-management mobile apps.

For the “medication reminders by notifications” feature, this factor had a statistically significant relationship with the outcomes [31]. The IDF Europe has recommended that health apps should adopt SMS text messaging or push notifications for insulin injections. These reminders are essential as they could prevent hyperglycemia and long-term complications associated with prolonged uncontrolled diabetes [34].

For the “syncing with electronic medical records/personal health record” feature, a study showed that educators favored the integration of mobile phone–collected data into the health information system [29]. From the users’ perspective, a study showed that people with diabetes also wanted the app to be integrated into existing systems since it could help increase the dissemination of the app and improve app uptake [39]. For the “syncing with meters, continuous glucose monitors, or insulin pumps” feature, a study that surveyed youths with T1DM or their parents found that a glucometer-connected mobile app could increase an individual's engagement with other aspects of care (such as self-monitoring of blood glucose frequency) [47]. IDF Europe’s position on mobile apps for diabetes mentioned that interoperability is an essential feature of such apps [34].

Overall, there was a moderate degree of consensus among experts on their rankings of the engagement strategies that they believe are most likely to be effective in diabetes management when using a diabetic self-management health app. Strategies with the highest mean of agreement (ratings above 4.40) have already been addressed in the literature. Regarding the “app being free” feature, 36/48 (75%) of the included apps were free, and the average cost of the paid apps was Aus $4.37 (US $3.13) [48]. User ratings and prices are important factors determining app attractiveness, with variations across countries [49]. A study analyzing both free and paid apps found that more expensive and popular paid apps tended to have more drawbacks. This relationship between popularity and drawbacks emerges from the fact that more expensive and popular paid apps tend to have more functionality [50].

The importance of “allowing the user to communicate or send information/data to a health care provider” has been demonstrated at the patient and health care levels [34]. A thematic analysis

study focused on app-based interventions in managing chronic respiratory diseases, diabetes, and hypertension demonstrated the perceived ability of HCP-motivated patients and empowered them to properly self-manage their condition. The use of health technology in two-way communication between HCPs and patients proved to impact the patients’ health outcomes [31].

Studies found that user-friendly design, simplicity, and intuition were vital aspects for engaging younger adults to use health apps [34,51-53]. IDF Europe’s position on mobile apps for diabetes mentioned that determining the target audience is crucial for the uptake of an app [34].

Several studies have mentioned “flexibility and customization” as a requirement [29,54,55] and a key strategy to facilitate engagement with therapies. The ability to evaluate the app as a guest user, as well as the ability to modify the welcome message and color palate, are examples of customization and flexibility [56].

One of the strengths of our study was having two people with diabetes on the study panel of experts. Therefore, besides their professional experience, they provided input into how the app could contribute to their life as patients with diabetes. The number of experts included was within the recommended range. Moreover, all the rounds were summarized and discussed by the steering committee, who are experts in the field.

There were several potential limitations identified in this study. First, we recruited experts with various backgrounds, which could have caused the significant levels of disagreement in the ranking. At the same time, if we had selected other experts, that could have led to a different result. Furthermore, there was considerable overlap among many of the information items volunteered by the Delphi participants. The use of a Delphi approach for selecting intervention components is not guaranteed to result in the best choices, which might affect the results’ reliability; this is a known limitation of the Delphi technique.

Another limitation is that our panel consisted only of experts. We did not include patients with diabetes, although two of the participants coincidentally had diabetes. To determine whether these same features and engagement strategies are desired by the end users, another validation study should be conducted among a population of people with diabetes to bridge the gap between the perspectives of experts and end users.

To our knowledge, this is the first study in the Middle East and North Africa that gathered a local panel of experts from the diabetes field and used an iterative process to combine the experts’ opinions into a group consensus. Consensus agreement does not mean that all of the right answers have been found, but rather indicates that a level of participant agreement has been reached. The information items resulting from this modified Delphi survey represent the opinions of an expert panel.

This study allowed us to reach a consensus on several important questions related to diabetes management components that experts agreed were the most likely to be effective when delivered via a mobile app. Furthermore, it shed light on the engagement strategies that diabetes experts believed would be the most likely to be effective in diabetes management when using a mobile app. The results of this study could thus be useful for health authorities and HCPs for future decision making on this topic.

The 36 features and 21 engagement strategies identified in this study should guide developers considering mobile app development targeting people with diabetes and other similar chronic diseases. For further research, we recommend that the results of this study should be verified and validated by building a prototype app that includes the features and strategies described in this study and recruiting people with diabetes to test it. Moreover, it would be more interesting to know if any of the top 10 app downloads globally or regionally contain these features and which combinations are most commonly found. How easily are these features implemented in an app? Are there cultural contexts that make certain feature more appropriate? Are any of the features not currently found in any app? These and other questions remain to be answered.