This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Mobile health apps are commonly used to support diabetes self-management (DSM). However, there is limited research assessing whether such apps are able to meet the basic requirements of retaining and engaging users.
This study aimed to evaluate participants’ retention and engagement with My Care Hub, a mobile app for DSM.
The study employed an explanatory mixed methods design. Participants were people with type 1 or type 2 diabetes who used the health app intervention for 3 weeks. Retention was measured by completion of the postintervention survey. Engagement was measured using system log indices and interviews. Retention and system log indices were presented using descriptive statistics. Transcripts were analyzed using content analysis to develop themes interpreted according to the behavioral intervention technology theory.
Of the 50 individuals enrolled, 42 (84%) adhered to the study protocol. System usage data showed multiple and frequent interactions with the app by most of the enrolled participants (42/50, 84%). Two-thirds of participants who inputted data during the first week returned to use the app after week 1 (36/42, 85%) and week 2 (30/42, 71%) of installation. Most daily used features were tracking of blood glucose (BG; 28/42, 68%) and accessing educational information (6/42, 13%). The interview results revealed the app’s potential as a behavior change intervention tool, particularly because it eased participants’ self-care efforts and improved their engagement with DSM activities such as BG monitoring, physical exercise, and healthy eating. Participants suggested additional functionalities such as extended access to historical analytic data, automated data transmission from the BG meter, and periodic update of meals and corresponding nutrients to further enhance engagement with the app.
The findings of this short-term intervention study suggested acceptable levels of participant retention and engagement with My Care Hub, indicating that it may be a promising tool for extending DSM support and education beyond the confines of a physical clinic.
Mobile health (mHealth) apps offer a unique opportunity to deliver health promotion interventions to reach any population due to their ubiquitous nature [
Inadequate participant retention is a major methodological challenge experienced by many mHealth app interventions [
An effective mHealth intervention requires not only retention but also continuous and active engagement by users, as lack of engagement leads to study dropout and dampening of the treatment effect [
System usage is measured through the collection of noninvasive data on the frequency of access to the app, push notifications opened, and average time spent per usage [
Rate of use alone is not a sufficient indicator of engagement with an mHealth intervention [
Within the field of mHealth engagement, models and frameworks provide a richer understanding of the core components that influence user engagement to achieve the behavioral goal of the intervention [
The BIT framework [
Owing to poor retention and engagement with previous diabetes apps, we performed an initial study to explore user needs and preferences to foster engagement with a diabetes app [
Therefore, this study aimed to examine levels of user retention and engagement with My Care Hub in a short-term single-arm pilot trial. Retention was measured through completion of follow-up surveys, and engagement with the app was assessed in 2 areas: (1) system usage data and (2) qualitative feedback from users on behavioral interactions with the intervention. We expect that the app’s contents and features, which were developed based on results from our previous study on users’ needs [
This study received ethics approval from the James Cook University Human Research Ethics Committee (reference #H7716). Participants were informed about the study aims, and consent was implied by survey submission. Verbal consent was obtained for telephone interviews.
This study utilized a sequential explanatory mixed methods design with quantitative surveys and qualitative interviews. This design captures both the engagement with technology and the process of behavioral change by triangulating the results of multiple measures [
The study used a maximum variation purposive sampling tailored to recruit participants who showed interest in the study within the time available. This sampling method is appropriate for an implementation feasibility assessment as related to this study [
Participants were recruited through a single invitation email sent to patients registered with the Australian National Diabetes Service Scheme. Email invitations were limited to patients who have type 1 or type 2 diabetes and live in North Queensland, Australia. North Queensland has a relatively high prevalence of diabetes [
Participants enrolled through the web by completing an eligibility screening form, providing consent, and completing the baseline survey, which entailed questions regarding socio-health demographics, email address, and residential postcode. Participants were emailed a unique code to enable them to download My Care Hub from Google Play store of any android-powered phone, an app manual, and a 5-min video explaining how to install the app, features, and functionalities. Participants could contact the first-named author (MA) for assistance with technical difficulties or for study clarification. It was emphasized that there was no limit to the frequency of use of My Care Hub as participants could engage with it at a level they considered useful and desired. My Care Hub is intended to be a stand-alone intervention; therefore, push notifications (aimed at improving patients’ awareness about diabetes distress and potential ways to reduce its impact on their self-management) were sent from the app during the first 2 weeks of the intervention and withheld in the third week to see the achievable level of engagement with the app with or without push notifications. Throughout the study period, no log-in reminders or calls were made from the study researchers to participants.
A detailed description of the development of My Care Hub and the methods of usability studies have been previously published [
In
At the end of the study, participants were sent an email (with 1 reminder email sent to noncompleters), which directed them to the poststudy survey on the acceptability of the app and its preliminary efficacy (
Retention was assessed using the following indicators of study completion per protocol: number of participants enrolled, number of participants who used the app during the intervention period, and completion rate of the poststudy survey.
Engagement with My Care Hub was measured using participants’ app usage log and verbal feedback. App usage data were extracted from the app’s activity database. The following time frames were considered: (1) date of log-in into the app to 2 weeks of use when the daily push notification was administered (referred to as week 1 and week 2) and (2) data during the third week (referred to as week 3) after the termination of push notifications. Key metrics collected from the database included app use (number of active users, frequency of daily access to app), data logs/time spent (for BGL, exercise, food activity, and weight), and number of opened notifications. Metrics were presented using an adapted version of the Frequency, Intensity, Time, and Type (FITT) principle index [
Frequency index (
Intensity index (
Type index (
Time index (
Interviews were conducted using a semistructured interview guide that explored behavioral engagement with the app through questions on patterns of use, perceived ease of use, perceived usefulness of app features enabling motivation for continued engagement with DSM, and recommendations on how the app could be improved. The interview guide has been provided in
Interviews were conducted by 1 author (AD), who is well experienced in qualitative research. The interview guide was pilot tested between MA and AD before actual use. The interviewer was located in a private office at James Cook University, Australia, while participants were asked if they were in a comfortable location before commencement of the interview. The first 3 interviews were used to reflect on the guide, although there were no resultant changes. Data saturation was achieved as judged by no emerging new information [
Descriptive statistics were calculated for all quantitative variables. Baseline characteristics comparison between those who completed the study and those who did not were done using a Pearson chi-square test. All statistical analyses were performed using SPSS version 23 [
Interviews were completed in an average of 15 min (range 9-30 min). Participant responses were transcribed verbatim by 1 researcher (AD). In this analysis, a combination of data and a concept-driven strategy was applied. Initially, inspired by the work of Schreier [
Participant demographics and health characteristics are shown in
Of the 22 participants who indicated an interest in participating in the interview, only 17 were contactable within 3 call attempts. Most were males (12/17, 71%), had type 2 diabetes (13/17, 77%), and had been diagnosed for an average of 6 years (range 1-17 years). Overall, participants were between the ages of 36 to 64 years (mean 51.58, SD 11.31), except for one who was aged 20 years.
Of the 4984 patients who were emailed an invitation to participate in the study, 79 (1.59%) completed the eligibility form. However, only 84% (67/79) of those who responded met the inclusion criteria and were provided access to download the app. Some participants (17/67, 25% of those eligible) failed to log in to the app, resulting in 50 enrolled participants (75% of eligible participants). Most enrollees (43/50, 86%) activated the app within the same day (range 0-5 days) of having access to it. One participant logged out of the app on the second day of installation stating that it did not meet her requirement. At the end of the study period, 41 of the enrolled participants completed the study per protocol by providing feedback about the app using the poststudy survey (retention rate: 41/50, 82%). Reasons for noncompletion of the study protocol were not recorded. In assessing baseline characteristics associated with retention, only employment status emerged as a significant predictor, with those unemployed being less likely to complete the study than those who were employed (50.0% versus 14.7%, respectively;
Participant characteristics.
Characteristics | Baseline (N=50) | Completers (n=41), n (%) | Lost to follow-up (n=9), n (%) | |||
|
.75 | |||||
|
Male | 31 | 25 (81) | 6 (19) |
|
|
|
Female | 19 | 16 (84) | 3 (16) |
|
|
Age (years), mean (SD) | N/Aa | 49.29 (12.74) | 48.67 (11.25) | .82 | ||
|
.82 | |||||
|
18-29 | 5 | 4 (80) | 1 (20) |
|
|
|
30-39 | 6 | 5 (83) | 1 (17) |
|
|
|
40-49 | 12 | 10 (83) | 2 (17) |
|
|
|
50-59 | 15 | 11 (73) | 4 (27) |
|
|
|
60-65 | 12 | 11 (92) | 1 (8) |
|
|
|
.81 | |||||
|
Type 1 | 15 | 12 (80) | 3 (20) |
|
|
|
Type 2 | 35 | 29 (83) | 6 (17) |
|
|
|
.32 | |||||
|
None | 2 | 1 (50) | 1 (50) |
|
|
|
Oral drugs alone | 33 | 28 (85) | 5 (15) |
|
|
|
Oral and insulin | 1 | 1 (100) | 0 (0) |
|
|
|
.92 | |||||
|
<5 | 27 | 23 (85) | 4 (15) |
|
|
|
6-10 | 10 | 8 (80) | 2 (20) |
|
|
|
11-15 | 9 | 6 (67) | 3 (33) |
|
|
|
>16 | 4 | 4 (100) | 0 (0) |
|
|
|
.59 | |||||
|
High school equivalent | 17 | 12 (71) | 5 (29) |
|
|
|
Technical college | 10 | 9 (90) | 1 (10) |
|
|
|
First degree | 11 | 10 (91) | 1 (9) |
|
|
|
Postgraduate | 8 | 7 (88) | 1 (12) |
|
|
|
Missing | 4 | 3 (75) | 1 (25) |
|
|
|
.87 | |||||
|
Caucasian/white | 47 | 38 (81) | 9 (19) |
|
|
|
Missing | 3 | 3 (100) | 0 (0) |
|
|
|
.02c | |||||
|
Unemployed | 8 | 4 (50) | 4 (50) |
|
|
|
Partly/fully employed | 34 | 29 (85) | 5 (15) |
|
|
|
Retired | 8 | 8 (100) | 0 (0.00) |
|
|
|
.26 | |||||
|
Remote | 13 | 12 (92) | 1 (8) |
|
|
|
Rural | 37 | 29 (78) | 8 (22) |
|
|
|
.42 | |||||
|
1-5 | 13 | 11 (85) | 2 (15) |
|
|
|
6-10 | 28 | 24 (86) | 4 (14) |
|
|
|
>10 | 9 | 6 (67) | 3 (33) |
|
|
|
.93 | |||||
|
Yes | 16 | 13 (81) | 3 (19) |
|
|
|
Never | 34 | 28 (82) | 6 (18) |
|
|
|
.38 | |||||
|
Poor | 1 | 1 (100) | 0 (0) |
|
|
|
Fair | 19 | 14 (74) | 5 (26) |
|
|
|
Good | 21 | 17 (81) | 4 (19) |
|
|
|
Very good | 9 | 9 (100) | 0 (0) |
|
aN/A: not applicable.
bN=35.
c
Most (42/50, 84%) enrolled participants logged data into the app at least once (during week 1 of installation) with the frequency index showing that they actively used the app on an average of 11 of the 14 days in the first 2 weeks when push notifications were sent (range 2-14 days; week 1 average: 5.2 days, week 2 average: 4.8 days). This reduced to an average of 4 of 7 days (range 2-5) in week 3: average 3.8 days. Furthermore, all participants who logged in to the app used it during week 1, and most returned to use the app after week 1 (36/42, 85%) and week 2 (30/42, 71%) of installation. With regard to the intensity index related to daily use of each app feature (
My Care Hub sections and engagement indices (N=42).
Functions and features | Elements | Purpose | User engagement | ||||||
|
|
|
Percentage of daily users ( |
Average time spent per user per day ( |
|||||
|
|||||||||
|
BGc activity ( |
•BG log |
•Monitoring and tracking of BG values over time |
29 (69) | 2 min 2 seconds | ||||
|
Physical activity ( |
•Log of time spent on physical activity |
•Monitoring of physical activity behavior over time | 4 (10) | 0 min 7 seconds | ||||
|
Food activity ( |
•Record of food intake |
•Monitoring and tracking of food intake and their carbohydrate content over time | 1 (2) | 0 min 17 seconds | ||||
|
Weight log ( |
•Body weight log | •Body weight assessment over time | 2 (5) | 0 min 22 seconds | ||||
|
Analytics ( |
•Graphical display of data log into each documentation feature | •Keeping track of trends in lifestyle activities and observe impact on BGLf over time | 3 (6) | 0 min 20 seconds | ||||
|
|||||||||
|
Textual screens for management tips and food choices ( |
•Information on behaviors in DMg management |
•Assess current knowledge on DSMh |
6 (13) | 1 min 35 seconds | ||||
|
Push notifications ( |
•Messages on diabetes distress | •Create awareness about diabetes distress and ways to reduce its impact on self-management | 24 (57) | —j |
a
b
cBG: blood glucose.
d
e
fBGL: blood glucose level.
gDM: diabetes management
hDSM: diabetes self-management.
i
jNot captured due to the tracking limitations of the system usage database.
Different themes emerged from the data with interconnection among the themes over the course of My Care Hub usage. Overall, the results suggest that the use of the app has the potential to ease the effort in aiming for improved self-management and for better awareness of BGLs. In addition, participants provided their recommendations for extra functionalities that may further enhance engagement with self-management behaviors. We present our findings in relation to themes related to components of the behavioral intervention model [
Summary of behavioral intervention technology model as adapted to My Care Hub intervention.
BITa components | BITa components | Details in MCHb | |||
|
|||||
|
Why | Broader goal: self-management support | Aims: Improved BGc—long-term impact Increased physical activity Healthy eating Decreased diabetes stress |
||
|
How | Behavioral change strategies |
Elements or strategies |
||
|
|
|
Documentation and Analytics: Feedback response: Carbohydrates in foods: Educational tips: |
||
|
|||||
|
What | Elements (app features) | Documentation (logs)and analytics: Feedback response Carbohydrates in foods Educational tips screen Push notifications |
||
|
How (technic) | Characteristics | Aesthetic: Beautiful Simple and straight forward Few difficulties |
||
|
When | Pattern of use | User defined Type of diabetes Established self-management routines Frequency: Daily Partly, with reasons |
aBIT: behavioral intervention technology.
bMCH: My Care Hub.
cBG: blood glucose.
dHP: health provider.
Patterns of app use depended on users’ type of diabetes and self-management routines, with most participants using the app multiple times per day, where those with type 1 diabetes input their BGL any time it was measured:
I use it multiple times per day, basically any BGL I took I enter it at any time I took it.
In contrast, participants with type 2 diabetes described that the frequency of usage depends on the self-management activity carried out on that day.
I used it at least once a day. if I had done exercise, then I was putting in an exercise and blood test virtually every day. On every second day I was using it to stick in weight but the exercise was done at a different time.
Conversely, some participants were only able to use the app infrequently because of issues such as limited internet access or multiple competing interests:
I didn’t use it fully, because at the moment I am having a problem with my internet, so I didn’t get a chance to watch the video that comes with it.
I used it a few times to start with, but then I stopped pretty much because I was juggling between doing a lot of writing, doing a course, and was having other things to do.
Participants described the design as:
Very well crafted and well put together, really easy to use [P007, T2D],
and could be used even by the elderly who may not be too proficient in using mobile technology:
I would even say that like an older person in their 60s or so, once they get an idea of how to use it properly, would have no worries using it if they were in that way inclined.
Some participants found a few aspects of the app difficult:
There was one for the activities you had to put in what calories you might have burned off and I didn’t have a clue how I was going to find out that information.
I had a problem figuring out how to put dates in it, but I think it does it itself, so yeah.
The goal of developing My Care Hub was to enhance engagement with self-management activities such as improved BG, increased participation in physical exercise, and healthy eating. Participants identified multiple elements (features) that support this overall goal. They also described the perceived benefits (mechanism of action) of each of the elements that encouraged their interaction with it, and toward achieving an improved DSM. The commonly mentioned features are noted below, as well as reasons why participants found the features engaging.
Participants mentioned that the documentation element strengthened the sense of responsibility to keep up with routines in DSM:
I liked the activity log, because it gives you accountability, when did you go to the gym, how long were you there, what did you do.
Participants explained that visualization of logged data using
Just the tracking of my fitness, exercise and my blood sugars, it is much better for me seeing it in a graph, makes it really clear how you are going.
The feature also hinted at some participants to consult their physician for medication review or consultation if their BGLs were not in the recommended range:
I liked the graphs…, that was what gave me the red flag…maybe I have to see the doctor to have my medications changed.
Participants noted that although they have a BG meter that provides BG measurement history, having the graphical output of their BGL in My Care Hub further improved awareness of any fluctuations in BGLs:
It was quite good to see longitudinal things, obviously on my blood monitor I can see by just hitting the back key what the previous readings are..., But to see it in a graphical linear form was really good. It showed me where my blood sugar was, if I went up and down.
The analytic feature enabled participants to pay attention to daily calorie intake or carbohydrates consumed:
I liked that idea of putting it all in and seeing how your graphs went up and down, and it sort of kept you a bit more mindful of how many calories or carbs you are eating during the day.
Feedback received in response to logged BGL is an element that reinforced the doctor’s recommendation about participants’ BGLs. A participant with hypoglycemia unawareness noted that his doctor suggested continuing using the app to serve as an alert in the event of low BGL:
It is one thing that made me maintain my BGLs. I tend to be what my doctor calls hypoglycaemia insensitive. So, he suggested that I stick with the app because it reminds me to do regular BGL tests to make sure that I am not dropping too low.
Feedback feature serves as an alert about a potential problem in users’ BGLs:
I got confirmation that was somewhat reassuring. I mean if it was out and higher, it just alerts you to a potential problem that you may or may not be aware of.
It aided decision making for improved self-management:
If my levels were over the target range, it gave me very helpful ways to reduce the blood glucose level back into the range.
Participants valued the
I try to stay between 20 and 50 grams a day, so the carb counting feature was very useful because then you can make an informed decision on what you are going to put on your plate, and you can plan out your week.
Participants who had difficulties knowing the carbohydrate content of foods found this feature useful through outlining the best foods for consumption to ensure proper health management:
I have a lot of trouble with how much carbohydrate is in one food but it (app) sort of gets you to realise okay then I have got to check on that.
Furthermore, engaging with the
There is so much to take in, like reading labels, it is so much to take in. So I found it (app) quite interesting that it is a bit more set out with carbs and how much is in it, and some of them are low and you thought it would be high. Just reinforcing the information because I just can’t remember everything.
Educational tips were also acknowledged as a tool for knowledge reinforcement and fostered the use of the app. Participants found information on 7 essential ways to manage diabetes quite useful and reflective:
It is useful, I have got a couple of books, and there is a lot of information, and whilst I may have read it, I am not sure I can regurgitate it.
It was just interesting to read it and think about it.
In addition, participants felt that the element provided more comprehensive information in comparison with the feedback element:
That (educational tips) was more useful than the little hint things (feedback messages) yeah… I think it probably covered it (all information) fairly thoroughly.
Participants’ recommendations were primarily based on extended functionality in the app, including the following:
Automation of data input: Some participants found the manual recording of BGL, physical activity, and carbohydrate content of foods consumed as burdensome and expressed that the addition of Bluetooth, which could automatically extract data from the BGL meter, would not only encourage users’ engagement with My Care Hub but also improve BGL monitoring. Furthermore, the desire for the app to automate the tracking of time spent on physical activities and equivalent calorie expended was expressed. In addition, it was recommended that the app should have features to calculate the calorie content of composite dishes.
More analytic histories: Participants suggested extended historical data access and believed this would provide further opportunity to study patterns in self-management activities and have long-term data that could be reviewed by their health care providers.
Information update: It was suggested that the Carbs in Foods feature needed more food lists and varieties of composite dishes. Participants suggested that this information could be provided in monthly updates because users’ awareness of finding new information in the app on a regular basis could foster fresh interest in using the app.
Feedback on physical activities: The idea of providing motivational feedback in the app, especially when users achieve certain levels of physical exercise, was raised. This behavioral change strategy in My Care Hub is presently limited to the BGL documentation; presumably, participants want an extension of it to the physical activity documentation.
The My Care Hub mobile app intervention was intended to encourage ongoing participation in DSM activities. This paper reports the levels of participant retention and engagement (usage and behavioral aspects) with the technology over a 3-week pilot study. The findings of the study revealed an acceptable level of participant retention with the intervention, where the majority completed the study per protocol. Furthermore, participants reported that the intervention eased and improved their effort in participating in self-management activities. Thus, suggesting the app’s potential as a tool for DSM support and education. Nevertheless, a larger sample and longer-term studies are required to establish these claims.
The retention rate was relatively high, with more than three-quarters (82%) of participants completing the study per protocol, which is similar to previous short-term pilot studies of diabetes app interventions [
Retention was not influenced by participant characteristics measured, with the exception that unemployed participants were less likely to complete the study, which was contrary to the results of a previous mHealth study [
Users in our study actively used the app for 11 of 14 days (11/14, 79%) in the first 2 weeks, where they all used the app at least once during the first week and 85% returned to use the app during week 2 and 71% during week 3. To put these rates into perspectives, we refer to studies of Faridi et al [
The intensity of usage showed that participants interacted more with features for monitoring of BGL and physical activities, which are in congruence with previous studies [
The active time spent on the documentation features demonstrated that the duration of app usage necessary to generate consistency is a parameter that depends on individual users [
Although the My Care Hub system log recorded participants’ passive usage of the education textual screens, there are no standard measures to compare these data with similar diabetes-focused interventions. However, the interviews indicated that participants appreciated this feature as an important element that provided knowledge reinforcement as a behavioral strategy for DSM. Nonetheless, the app system was unable to capture whether participants were actually reading and comprehending the embedded information or simply clicking them. An approach to address this limitation is to incorporate eye-tracking technology [
Generally, engagement indices were initially high but decreased in subsequent weeks. Previous studies using mHealth interventions over short- and long-term periods have identified similar trends [
Participants’ perceptions related to behavioral change strategies in My Care Hub derived from the documentation, feedback response, calories in foods, and education tips features are consistent with the needs analysis study conducted as part of the predevelopment phase of the app [
Perceived ease of use of mHealth positively affects continuance in intention to use [
The educational component of the app was informed by our previous study, which shows that information on basic guidelines for the management of diabetes and approaches to problem solving in diabetes were highly desired by both type 1 and type 2 diabetes patients [
A mixed methods study design was used to evaluate patient engagement with My Care Hub, which is a strength of the study compared with previous studies that have arbitrarily classified engagement as high or low based on frequency of use [
This study has some limitations that should be taken into account when interpreting the findings. The short intervention period is acknowledged. However, 3 weeks is the minimum time required for anyone to form a behavioral habit [
This study provided a comprehensive understanding of participant retention, technology usage, behavioral change process, and engagement with My Care Hub app during a short trial period. Retention was high, although further strategies may be required to further sustain retention when the app is used in long-term trials. The system log indices of FITT of engagement reveal a reasonable level of technology usage during the intervention period. The BIT model employed to measure behavioral change and engagement suggests that My Care Hub could be a behavior change intervention tool to support self-management behaviors in people with type 1 or type 2 diabetes. Information obtained through the use of multicomponent measures of engagement in this study provides rich and useful data regarding the strengths and weaknesses of My Care Hub and areas requiring improvement to foster increased engagement, sustainable long-term use, and effective health behavioral intervention.
Sample screenshots of the My Care Hub app.
Semistructured interview guide.
Consolidated criteria for reporting qualitative research checklist.
blood glucose
blood glucose level
behavioral intervention technology
diabetes self-management
electronic gift
frequency, intensity, time, and type
mobile health
This research was supported by an Australian International Research Training Scholarship and a James Cook University top up scholarship given to the first author. The authors would like to thank Ian Aitkinson and Saira Viqar of the eResearch team at James Cook University for their assistance with the app development. They sincerely appreciate all the study participants and the National Diabetes Service Scheme, Australia, for their invaluable support with the advertisement of the study.
BM, UM, AM, and MA conceived and designed the study. MA and AD collected and analyzed the data. MA prepared the original draft, and BM, UM, AM, and AD reviewed and edited the paper. BM is the project lead. All authors have read and approved the final paper.
None declared.