This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
While optimal blood glucose control is known to reduce the long-term complications associated with type 1 diabetes mellitus, adolescents often struggle to achieve their blood glucose targets. However, their strong propensity toward technology presents a unique opportunity for the delivery of novel self-management interventions. To support type 1 diabetes self-management in this population, we developed the diabetes self-management app
The primary objective was to evaluate
We enrolled 92 adolescents into a 12-month RCT, with 46 receiving usual care and 46 receiving usual care plus
Linear mixed models showed no changes in primary and secondary clinical outcomes. However, exploratory regression analysis demonstrated a statistically significant association between increased SMBG and improved HbA1c in the intervention group. For a subgroup of
Although primary analysis of clinical outcomes did not demonstrate differences between the
ClinicalTrials.gov NCT01899274; https://clinicaltrials.gov/ct2/show/NCT01899274 (Archived by WebCite at http://www.webcitation.org/6qWrqF1yw)
Type 1 diabetes mellitus is among the most common chronic diseases affecting children, adolescents, and adults, with an increasing worldwide incidence of approximately 3% to 4% a year [
Overall, advancements in the mechanism of insulin delivery (ie, insulin pump or multiple daily injections) has had a limited impact on glycemic control among youth [
Recently, the use of mHealth apps as a tool for improved diabetes self-management has proliferated, as illustrated by the number of diabetes apps available for download on the iOS App Store and Google Play [
Furthermore, many of the existing apps require manual entry of blood glucose values and focus primarily on the display of diabetes-related data, such as blood glucose readings, carbohydrate intake, and insulin doses [
Therefore, the objective of this research was to design, develop, and evaluate
Adolescents with a diagnosis of type 1 diabetes, between the ages of 11 and 16 years, were randomly assigned to 1 of 2 groups: (1) the
Before initiating the study, protocol approval was obtained from all site-specific ethical review boards and/or committees (The Hospital for Sick Children: #1000036524; University Health Network: #13-6237-BE; Trillium Health Partners: #619).
We recruited participants from August 2013 to December 2014 from 2 pediatric endocrinology centers in Toronto, Ontario, Canada. The final study visit was completed in January 2016. Patients were eligible to participate if they (1) had a diagnosis of type 1 diabetes mellitus (as defined by Canadian Diabetes Association guidelines [
Sample size was determined based on a nominal 2-sided type I error rate of 5% and 80% power. Estimates of standard deviation in HbA1c ranging from 0.50% to 0.75% were used to determine the minimum number of participants required to detect a clinically relevant (≥0.5%) change in HbA1c levels [
At enrollment, participants were assigned equally to an intervention or control arm using randomly allocated block sequences of 4 to 6 participants. To ensure equal distribution between arms, we stratified random allocation for treatment modality (insulin pump vs insulin injection), as well as study center (The Hospital for Sick Children vs Trillium Health Partners). The RCT was an unblinded, open-label study, as both the participants and those delivering the intervention were aware of allocation based on whether or not the
The initial design of
Key features of
Feature | Description |
Automatic Data Transfer | Blood glucose readings are wirelessly transferred from a Bluetooth-enabled blood glucose meter, using an adaptor (BluGlu), to |
Electronic Logbook | Current and past blood glucose readings categorized by context (eg, lunch) are displayed over multiple time frames (eg, 1 week, 1 month). |
Trends | Percentages of readings in or out of target, per context, are displayed over various time frames (eg, over 30 days, 10% of breakfast readings were high). |
Trend Wizard | Algorithm that detects and informs the user of consecutive out-of-range readings for the same context (eg, 3 consecutive high dinner readings) and prompts the user to identify the likely cause of the trend and potential fixes. |
Reward System | Reward mechanism that awards points to encourage the following behaviors: (1) taking up to 5 readings per day, (2) getting readings in target range, (3) avoiding out-of-range trends, and (4) resolving any identified 3-day trends. |
A private social media community that allowed trial participants to communicate with each other. | |
Personal Health Record | Integration with TELUS health space, a secure personal health record that stored blood glucose data and enabled sharing with members of the care team. |
The intervention includes an iPhone 4S loaded with
Adolescents who met the inclusion criteria and provided informed consent were randomly allocated to receive either usual clinical care (control group) or usual clinical care plus
Baseline visits were followed by 3-, 6-, 9-, and 12-month research visits for all participants. All research visits coincided with the participant’s standard quarterly clinic visit; however, these visits were conducted separate from the clinic visit by trained research staff. Qualitative and quantitative data were collected at all follow-up visits via semistructured interviews, validated instruments, downloads of blood glucose meters, and electronic chart review. Halfway between each follow-up visit, we contacted participants in the
The primary outcome of the study was change in HbA1c (measured in percentage) from baseline to 12 months, between the intervention and control group. HbA1c was measured during routine clinical blood work and accessed by research staff through electronic chart review. The primary research site (The Hospital for Sick Children) used a high-performance liquid chromatography assay (Bio-Rad Laboratories, Inc, Waterloo, ON, Canada) or an enzymatic assay (Abbott Laboratories, Ltd, North York, ON, Canada) to measure HbA1c, with internal quality control demonstrating excellent agreement among samples assayed by both methods (
The frequency of mild and severe hypoglycemic events was assessed as secondary measures of glycemic control. A severe hypoglycemic event was defined as any episode that required the assistance of another individual and a blood glucose reading below 2.8 mmol/L and/or a subsequent reversal of clinical symptoms with intake of oral carbohydrate, glucagon injection, or intravenous glucose [
The frequency of severe hypoglycemic events was self-reported by participants and/or their guardians during semistructured interviews conducted at baseline and all follow-up research visits. To capture the frequency of mild hypoglycemic events, the previous 50 days of blood glucose readings were downloaded from all available (study and/or personal) blood glucose meters and/or insulin pumps during the participant’s clinic appointment. All downloads were completed by trained staff using the manufacturer-provided electronic downloading programs, specific to each blood glucose meter or pump brand. In cases where not all hardware was available, participants estimated what percentage of their total blood glucose readings were on the devices they brought to clinic that day.
All individual readings below 3.4 mmol/L were recorded as an individual mild hypoglycemic event, except for low blood glucose readings taken within the same or consecutive-hour timeslots. Grouping contemporaneous readings together and counting them as a single episode ensured that a singular hypoglycemic event was not recorded multiple times.
We measured the average number of daily SMBG using all data collected from the 50-day blood glucose meter and/or insulin pump printout(s). Each blood glucose reading was counted individually, except when taken within the same hour, in which case readings were grouped. Readings taken over a 2-hour period in apparent response to an initial low (<4.1 mmol/L) or high (>17.9 mmol/L) were also grouped together. Using the total counted readings and number of days collected, we calculated the average number of daily SMBG at baseline as well as each follow-up visit, and when warranted, corrected for the percentage of readings available as estimated by participants.
We assessed the number of self-initiated adjustments made to a participant’s type 1 diabetes insulin regimen during qualitative interviews conducted at baseline and all follow-up visits to determine whether use of
Validated instruments were used to capture quality of life, self-care, and management data. The Diabetes Quality of Life for Youth (DQOLY) questionnaire [
We assessed overall satisfaction with
We collected mobile usage data through a third-party service, Flurry (Yahoo, Sunnyvale, CA, USA), which tracked (1) the number of times users accessed
Preliminary
Subsequently, we used linear mixed models to determine whether there were any statistically significant differences between the treatment and control groups for the above-mentioned outcomes. As all outcomes of interest were continuous, a linear mixed-model approach provided a simple method to assess treatment efficacy while adjusting for the correlation of each participant over time (using a random effect). Moreover, this approach is more powerful than a repeated-measures analysis of variance (ANOVA), as it allows participants with missing values at 1 or more time points to contribute some information to the analysis, while a repeated-measures ANOVA requires the availability of data at all time points for each participant [
Secondary analyses relied on comparison between groups at the primary end point of 12 months using 2-sample
Using the study inclusion criteria, we identified eligible participants from clinical databases and enrolled them sequentially until recruitment targets were met. Through this process, 199 eligible patients were identified; 42 patients declined to participate, 31 patients no longer met eligibility criteria, and 34 patients were excluded for other reasons, including planning to change clinics within the study time frame, having recently switched insulin regimens, and participating in another study with similar outcome measures. As
Baseline characteristics of intervention and control groups.
Characteristics | Treatment group (n=46) | Control group (n=46) | |
Sex (male/female), n | 21/25 | 20/26 | >.99 |
Age at baseline in years, mean (SD) | 14.1 (1.7) | 13.9 (1.5) | .54 |
Age at diagnosis in years, mean (SD) | 7.1 (3.6) | 7.4 (3.3) | .71 |
Duration of type 1 diabetes mellitus in years, mean (SD) | 7.1 (3.2) | 6.6 (3.2) | .48 |
Insulin regimen (pump/injection), n | 23/23 | 22/24 | .84 |
Hemoglobin A1c in %, mean (SD) | 8.96 (0.7) | 8.92 (0.6) | .77 |
Participant enrollment.
There were no significant differences in HbA1c between the intervention and control groups over the duration of the 12-month trial (
Between group analyses also showed no significant improvements in any of the predefined secondary outcomes between the intervention and control groups (
Secondary outcome measures.
Outcome measures | Intervention | Control | ||||
Baseline | 12 months | Baseline | 12 months | |||
Mild hypoglycemic eventsa, mean (SD) | 10 (8.2) | 11.52 (10.7) | 8.49 (9.6) | 7.54 (7.7) | .047 | |
Severe hypoglycemic eventsb, mean (SD) | 0.23 (0.6) | 0.16 (0.4) | 0.41 (1.3) | 0.48 (1.2) | .13 | |
Self-monitoring blood glucosea, mean (SD) | 3.98 (1.6) | 3.49 (1.8) | 3.55 (1.6) | 3.39 (1.5) | .42 | |
Number of adjustments to regimenb, mean (SD) | 1.85 (2.3) | 1.77 (2.7) | 2.08 (3.4) | 1.10 (1.3) | .25 | |
SCI scorec, mean (SD) | 35.73 (4.6) | 35.42 (5.0) | 36.07 (5.4) | 35.57 (6.4) | .81 | |
Impact of Symptoms | 3.58 (1.7) | 3.33 (1.7) | 3.55 (1.8) | 3.16 (1.6) | .15 | |
Impact of Treatment | 2.76 (2.3) | 2.53 (2.1) | 2.73 (2.0) | 2.28 (2.2) | .51 | |
Impact on Activities | 3.00 (2.2) | 2.96 (3.0) | 3.04 (2.8) | 3.42 (3.0) | .72 | |
Parental Issues | 5.13 (3.3) | 5.20 (3.6) | 5.12 (3.1) | 4.67 (3.6) | .71 | |
Worries About Diabetes | 6.83 (5.5) | 6.84 (5.8) | 6.51 (5.8) | 4.81 (5.0) | .17 | |
Health Perception | 2.00 (0.7) | 1.96 (0.7) | 1.90 (0.6) | 2.10 (0.6) | .50 | |
General Health Domain | 12.76 (2.2) | 13.70 (2.4) | 12.53 (2.1) | 13.31 (2.8) | .60 | |
Social Presentation Domain | 8.62 (1.6) | 8.86 (1.5) | 8.81 (1.5) | 9.08 (1.4) | .38 | |
Regimen Domain | 13.90 (2.4) | 14.60 (2.1) | 13.61 (2.5) | 14.40 (2.7) | .64 | |
Total DFRQ score | 35.29 (4.9) | 37.16 (4.3) | 34.94 (4.6) | 36.79 (5.7) | .78 |
aAverage number over 50 days prior to study clinic appointment.
bAverage number between study clinic appointments (typically 90 days).
cSCI: Self-Care Inventory, a 14-item questionnaire using 6-point scale (1 to 5, and “not applicable” option) to measure adherence to treatment recommendations. Overall score ranges from 10 to 50.
dDQOLY: Diabetes Quality of Life for Youth questionnaire, a 22-item questionnaire measuring quality of life, split across 6 subscales. Subscales use an inverted 5-point Likert scale (0 to 4), with the exception of the Health Perception subscale, which uses an inverted 4-point scale (1 to 4). Higher scores are associated with poorer quality of life; possible subscale scores range from 1 to 4 (Health Perception), 0 to 12 (Impact of Symptoms, Impact of Treatment, Parental Issues), 0 to 20 (Impact on Activities), and 0 to 28 (Worries About Diabetes).
eDFRQ: Diabetes Family Responsibility Questionnaire, a 17-item questionnaire measuring adolescent-guardian interaction around care, split across 3 subscales. All subscales use a 3-point scale (1 to 3). Higher scores are associated with increased adolescent involvement in care. Overall score ranges from 17 to 51; subscales range from 7 to 21 (General Health Domain), 4 to 12 (Social Presentation Domain), and 6 to 18 (Regimen Domain).
Mean hemoglobin A1c values for the intervention and control groups from baseline to 12 months.
Using all available data at each time point, we performed additional analyses to identify potential relationships between measured clinical outcomes, both within and between the intervention and control groups.
In further exploratory analyses, we identified a subgroup of patients with a frequency of SMBG of 5 or more per day at 12 months within both the intervention (n=8) and control (n=5) groups. This threshold was chosen because it is a commonly recommended daily SMBG target in The Hospital for Sick Children diabetes clinic, and this group represented a population of users who were actively engaged with daily SMBG at the end of the trial. No significant difference in daily SMBG was noted between the control subgroup (mean 7.02, SE 0.57) and the intervention subgroup (mean 6.32, SE 0.45) at baseline
HbA1c did not significantly differ between the 2 subgroups at baseline (control mean 8.84%, SE 0.27% vs intervention mean 8.40%, SE 0.21%;
In addition to the subset with SMBG of 5 or more per day, we also conducted subgroup analyses for insulin regimen (insulin pump vs insulin injections), as well as baseline HbA1c levels (participants with baseline HbA1c ≥9.0% vs <9.0%); however, no statistically significant differences were noted.
Regression analysis for self-monitoring of blood glucose (SMBG) and hemoglobin A1c.
Longitudinal mean hemoglobin A1c for intervention and control participants with 12-month self-monitoring of blood glucose of 5 or more per day.
To assess use of
Engagement thresholds, determined by the frequency of reading uploads, during the 12-month trial (n=46).
Engagement levels | Definitions | Injections (n) | Insulin pump (n) | Total (n) | % of all participants within each threshold |
Very low | Less than 1 of 14 days | 9 | 8 | 17 | 37 |
Low | Less than 1 of 7 days | 6 | 7 | 13 | 28 |
Moderate | Less than 3 of 7 days | 5 | 7 | 12 | 26 |
High | 3 of 7 days or more | 3 | 1 | 4 | 9 |
Total | 23 | 23 | 46 | 100 |
Overall, usage of
Participants reported high levels of satisfaction with
We also asked users to rank the features of
Overall satisfaction with
The aim of this 12-month RCT was to evaluate the effectiveness of
While primary clinical outcomes remained unchanged, a post hoc exploratory analysis provided additional insights. A significant and strengthening relationship between increased SMBG and improved HbA1c was observed exclusively in the intervention group (
To identify any factors that may have influenced the overall trial results, we conducted several secondary analyses, including the characteristics of the study population and potential trial design artifacts. This RCT purposefully targeted adolescents who were experiencing difficulty in managing their diabetes, as defined by sustained HbA1c values between 8.0% and 10.5%, who might benefit greatly from enhanced self-management skills and motivation. However, it is possible that, by extending the HbA1c inclusion range to 10.5%, patients whose poor glycemic control was caused by multiple complex factors, requiring support beyond the scope of the
In addition, with equal numbers of participants on an insulin pump versus insulin injections, it was also possible that the insulin regimen may have affected clinical outcomes. However, secondary subgroup analysis was conducted, which showed no significant impact of
We also hypothesized that a poorly motivated participant population could have resulted in the lack of improvement in clinical outcomes. However, the Readiness to Change Survey data showed that, on average, the intervention and control groups were classified in similar stages of change at baseline—including the “preparation” stage of change (for increased SMBG), associated with individuals who are ready to implement a plan of action to improve their health outcomes [
The
First, participants in the intervention arm were given
Interestingly, in the 2011 study (n=20),
Second, we developed the RCT version of
Another aspect that should be considered is the role of caregivers in the self-management activities adolescents perform using mobile tools. One of the key themes that emerged during the initial user-centered design of
The study results illustrate the importance of rigorously evaluating mHealth apps, not only for understanding the impact on clinical outcomes and user engagement, but also for assessing the methods used to evaluate these tools. While traditional RCTs have been considered as the “gold standard” for evaluation of interventions, a recent review by Pham et al emphasized that RCTs may not be best suited for the evaluation of rapidly evolving software interventions [
Robust and scalable research methods, coupled with adaptive RCT study designs, have the potential to reshape mHealth research. These approaches can enable the rigorous evaluation of apps in a more timely manner, while facilitating the rapid and iterative development of an intervention, keeping pace with the rapidly and continuously evolving mHealth landscape.
While adolescents are increasingly accessing technologies to support the self-management of type 1 diabetes, the impact of these tools on clinical outcomes remains unclear. Although this RCT found no changes in primary and secondary outcomes, exploratory analysis demonstrated improved HbA1c among
Number of times (measured as days per month) users uploaded blood glucose data to
CONSORT-EHEALTH checklist V1.6.1.
Overview of the main pages of bant.
Participant Management Questionnaire.
analysis of variance
Diabetes Family Responsibility Questionnaire
Diabetes Quality of Life for Youth
hemoglobin A1c
randomized controlled trial
Sequential Multiple Assignment Randomized Trial
self-monitoring of blood glucose
The authors wish to thank the patients and staff of the diabetes clinics at The Hospital for Sick Children (HSC) and Trillium Health Partners for their participation in this study. We are also very grateful to the team (project managers, designers, and software developers) at the Centre for Global eHealth Innovation, University Health Network, for their dedication to the ongoing enhancement and maintenance of
SG and CAN co-led the authorship of the manuscript; CAN led the Introduction and the synthesis of the data collected, SG led the Discussion, and both SG and CAN contributed to the Methods and Results sections. MR and ABC provided the statistical analysis content. DKK, SR, and AS provided guidance with trial development and execution, and edited the manuscript. MRP and JAC were cosupervisors.
The Hospital for Sick Children and University Health Network jointly own intellectual property rights to the