A detailed description of the Mobile Diabetes Intervention Study was published previously [33]. The study was a cluster-randomized clinical trial including 26 primary care physician groups across 4 geographic areas of Maryland. Randomization took place at the practice level to avoid contamination among physicians regarding care of their patients.

Eligible patients followed physician randomization assignment. Inclusion criteria for patients included diagnosis of type 2 diabetes at least 6 months prior to enrollment in the study, HbA_1c ≥7.5%, and age 18 to 64 years. Patients who were uninsured or Medicare or Medicaid beneficiaries were not included. Baseline data was collected from all participants, including demographic information, health history, current health status (including HbA_1c) and medications, risk factors for complications associated with poor diabetes management, and lifestyle and self-management behaviors.

The MDIS enrolled 163 patients across 3 intervention groups and 1 control group. The control group received care from their physicians as usual. Participants in each intervention group had access to a patient portal where they could record self-care behavior while also assigned to varying levels of physician access to patient data. In the most complex intervention group, physicians could review raw patient data, see analyzed patient data reports every 3 months, and make treatment recommendations based on these summaries. All patients received their choice of 1 of 2 smartphones with an unlimited 1-year data plan as well as a OneTouch Ultra 2 (LifeScan Inc) glucose meter and enough testing supplies for the duration of the 1-year study.

For this secondary analysis of the MDIS, group 1 data was not evaluated because control patients were not able to message their providers.

In addition to tracking information related to self-management of their diabetes, the secure patient portal allowed participants to communicate with CDEs throughout the study. When patients input data into the system, the computer would automatically generate feedback messages with encouragement or advice based on recently recorded data. For example, if a participant input a low blood glucose value, the system would provide a feedback message such as “This blood sugar is low! Eat 15 grams of carbs and recheck in 15 minutes.” Additionally, the data would be reviewed by the patient’s assigned CDE who could provide feedback intermittently. Most patients used the portal messaging system to communicate with educators over the course of the study, seeking advice, feedback, and answers to questions; however, using the messaging feature was not required for patients, and there was no set schedule of communication as part of the study intervention. The portal contained a variety of diabetes education materials including information on healthy eating, counting carbohydrates, being active, self-monitoring blood glucose, medications, and coping with and adjusting to living with diabetes.

Patients covered a wide variety of content in their messages to the CDEs from asking questions about healthy eating to changing medications to optimizing their medication schedule. To investigate the association of patient engagement with improved patient outcomes observed in previous studies [23,28-32], we evaluated patient engagement in our study through a qualitative analysis of messages sent through the secure patient portal.

For this analysis, we used the grounded theory approach [34] to analyze patient messages. As its name suggests, the theory is grounded in the observation of qualitative data and is used (for the purpose of this study) to categorize the data into core concepts. Based on review of a few sample patients, we created a coding scheme based on the 7 self-care behaviors for healthy living recommended by the American Association of Diabetes Educators (AADE) and the American Diabetes Association (ADA) [35,36]. After a pilot coding of 2 complete patient files, additional codes to account for patient-reported motivation and learning as well as general discussion about diet, medication, or self-monitoring of blood glucose were added (27 codes). Patient messages were then coded by EB and CQ (team members) based on the 27 codes developed for this project, with the appropriate codes assigned to each patient message, allowing for multiple codes assigned to a single message depending on content. Each team member independently coded the same message narrative line by line in Atlas.ti (ATLAS.ti Scientific Software Development GmbH), a qualitative data management program. Messages were coded individually without accounting for message threads on a single subject.

The Institutional Review Board of the University of Maryland, Baltimore approved this study. A data and safety monitoring board was designated to review the study procedures and adverse events. After enrollment was closed, errors in consent were found and all participants, both physicians and patients, were asked to sign consent forms again as recommended by the Institutional Review Board. All patients in the final analysis were reconsented.

The frequency of message themes was computed based on coding to categorize messages using Atlas.ti. Baseline characteristics are expressed as mean and standard deviation for continuous variables comparing users versus nonusers with 2-sample t tests or frequencies and proportions for categorical variables comparing users with nonusers with chi-square tests. A mixed methods approach was used to determine the effect of patient engagement on HbA_1c. Using qualitative analysis data, regression models were developed to determine the predicted change in HbA_1c for a patient based on the number and theme of messages sent over the 1-year study period. SAS 9.2 (SAS Institute Inc) was used to perform all statistical analyses. A P<.05 was considered statistically significant.

Among adults with type 2 diabetes, engagement in the portal messaging system of the MDIS was associated with an absolute decrease in HbA_1c of 0.75 percentage points. A 0.5 to 1.0 percentage point change in HbA_1c is considered clinically significant to reduce risk of comorbid conditions [37,38]; the US Food and Drug Administration requires a 0.4 percentage point change in HbA_1c for drug evaluations [39]. Although any sending of messages was related to a reduction in HbA_1c, glucose monitoring and medication use themes were also associated with decreases in HbA_1c. Patients sent the most messages on glucose monitoring, medication use, and reducing complication risks themes. The average number of messages sent per patient was highest for glucose monitoring, medication use, and healthy eating themes.

The AADE and the ADA provide patients, researchers, and clinicians with current self-care and lifestyle behavior guidelines for the management of diabetes and the prevention of its complications [35,36]. These guidelines, developed from the findings of the UK Prospective Diabetes Study [40], supply individuals with type 2 diabetes the knowledge needed to better understand their disease. Physicians in this study were given current ADA patient care guidelines but were not explicitly told to use them to care for study patients. Our findings support other studies that have shown the benefits of lifestyle interventions on diabetes outcomes [41-45]. In particular, digital health interventions targeting behavior change have shown lower HbA_1c levels, lower random [43] and postprandial [44] plasma glucose levels, and lower body weight [43] as well as improved self-efficacy [45].

Among the behavior themes measured, most messages contained either monitoring, healthy eating, or medication themes. Since messages sent regarding the medication and glucose monitoring themes also significantly decreased HbA_1c, patients may need more education surrounding medication use and monitoring blood glucose to ensure HbA_1c goals can be achieved effectively on their own.

Previous studies that assessed patient engagement in telemedicine and digital health interventions showed that race, age, and health literacy all play significant roles in patient participation [46-48]. Racial minorities, older patients, and patients with low health literacy showed the least engagement in telemedicine and digital health interventions [46,48]. In a 3-month mobile health intervention involving adults with type 2 diabetes, Nelson and colleagues [47] found that those who were younger or were diagnosed with type 2 diabetes closer to the start of the intervention displayed higher engagement activities and had more favorable experiences than older individuals or those with a longer diabetes duration. Our results are consistent with others, showing that nonwhite patients were less likely to send messages to assigned CDEs. Likewise, among participants who did not use the messaging portal, most had a high school education or less, perhaps also indicating a lower health literacy rate. However, unlike previous studies, we observed that users of the messaging portal tended to be older than nonusers. This suggests that older age does not imply disengagement from mobile health technology [49-51]. In fact, in a study evaluating the self-efficacy and use of a mobile health diabetes intervention among older adults, we previously concluded that participants experienced high self-efficacy in making changes to manage their diabetes and demonstrated their ability to use the intervention and communicate with educators [52]. We recommend including older adults and nonwhite individuals in mobile technology development with specific aims to evaluate improving patient engagement.

Other studies concluded that patients’ high engagement in digital health interventions was related to feedback received from physicians or assigned caregivers. From this feedback, patients felt more motivated and were able to attain higher self-efficacy [53,54]. Patients in this study who elected to send messages regarding any self-care behavior reported significant decreases in HbA_1c. Although the influence of CDE messages on patients’ outcomes was not examined, knowing a diabetes educator was available may have improved patient confidence and encouraged them to participate.

We believe that patient engagement in an intervention cannot be determined simply by a quantitative value but must also include qualitative data that demonstrates the effectiveness of the intervention from the participants’ perspectives. To accurately interpret the extensive data collected from digital health studies, it is important to include a qualitative component [55], as information on individual experience influences the effectiveness of the intervention. We used a mixed-methods approach to evaluate patient engagement data for participants in MDIS. We identified coding themes reflecting patient messages sent to CDEs and analyzed these themes against changes in patient HbA_1c values. Results of this study reinforce findings from previous mobile health investigations that use a mixed-methods approach to examine data, collecting self-care behavior and self-efficacy data to measure outcomes [47,56,57]. These studies add valuable knowledge about the usability of digital health applications for the management of diabetes and reveal areas lacking in development that, if revised, could enhance patient user experience and improve diabetes outcomes. This secondary analysis of the MDIS affirms that it is not enough to simply give patients information about diabetes; patients must also be given actionable items that drive behavior change.

The secondary data analysis is, to our knowledge, the first of its kind. Few previous studies have used a mixed-methods approach to evaluate patient engagement. While prior interventions included a patient messaging component [48] or analysis of self-management behaviors [58], none performed a qualitative evaluation of patient messages that was then used to create models predicting the impact on patient clinical outcomes. Furthermore, previous studies show that although participants preferred to use mobile health applications for diabetes management, currently available apps do not offer functions that would allow proper disease monitoring and management [59,60]. Results of this analysis may help pinpoint behavioral features that could improve existing mobile health technologies and satisfy the lack in functionality.

There are a few limitations of this secondary analysis. One is that although the models give a statistically significant prediction of change in HbA_1c based on certain message themes, it cannot be definitively stated that this is a direct result of solely the message content. It is important to consider the other aspects of the intervention, such as tracking data, accessing the learning library, or receiving directed care from their primary care physicians as also potentially influencing the patient’s outcome. Also, engagement was not randomized, so there is potential for confounding.

It is also important to note that based on the structure of the program, some patients engaged in external email and phone messages with the CDEs that are not in the portal message records; without knowing the content of these messages, it is impossible to get a complete picture of patient engagement over the year of the study. Furthermore, the role that messages from the diabetes educators play in patient outcome is unknown. While a future analysis may explore the impact of CDEs on patient outcomes, this analysis cannot account for the influence of the content of those messages on patient engagement or overall patient outcomes.

Since each message was analyzed and coded individually, we did not account for message threads. A conversation spanning several messages could have been counted each time the patient mentions the subject when really it is all part of the same conversation on the subject. Our analysis of dichotomies may be based on more tenable assumptions than the analysis per message.

In this study, messages sent in the combined healthy eating, monitoring, and medication themes or monitoring and medication themes separately significantly improved HbA_1c over the study period. Our results provide insight into the importance of health provider feedback and essential self-care behaviors that require greater emphasis when developing mobile health technologies for diabetes populations.