For many patients with chronic conditions, including type 2 diabetes (T2D), consistent, daily self-care (eg, taking medications, eating healthy, and exercising) is critical to avoid adverse health outcomes. However, patients with T2D face many barriers to self-care, including a lack of diabetes-related education and information, social barriers (eg, lack of motivation and caregiver stress), and difficulties with finances and transportation [1]. Racial/ethnic minorities and patients with low socioeconomic status (SES) tend to experience more barriers, which leads to worse self-care and health outcomes compared with non-Hispanic white patients and those with a high SES [2-5]. Text message–delivered interventions may offer an opportune pathway to extend self-care support to the hardest to reach and most vulnerable populations. More than 96% of adults in the United States use cell phones and find them to be essential for daily functioning [6,7]. Although smartphone ownership is growing and the digital divide has narrowed, individuals who have lower incomes, have less education, or live in rural areas are still less likely to have internet access or to own a smartphone [6,8,9]. Text messaging does not require internet access and is the most popular cell phone activity among all mobile phone users [10]. Evidence is growing for the efficacy of text message–delivered interventions to improve self-care for T2D and other chronic conditions [11-13]; however, there is a paucity of research examining how users engage with these interventions, particularly in the long term [14,15].

User engagement is essential to the ultimate success of any intervention. If target users are not attending to the intervention or lose interest quickly, potential effects may be attenuated or nullified. Approaches to measuring user engagement with technology-delivered interventions vary considerably across studies, making it a challenge to identify patterns and generate predictions. To help address this issue, Perski et al [16] synthesized past work on engagement with digital behavior change interventions to develop an integrative definition of engagement. They proposed engagement to be a multidimensional construct defined by both the extent of usage (often measured objectively) and a subjective experience characterized by attention, interest, and affect (often captured through qualitative data such as interviews) [16]. In many text message–delivered intervention studies, engagement is either not reported at all or reported using only a single metric (eg, average text message response rate) [15], which limits understanding of why participants were engaged, how engagement may change throughout an intervention, and factors contributing to engagement. Furthermore, among the studies reporting on engagement, intervention duration is typically 6 months or less [15,17] and, therefore, not representative of how users might engage in a longer experience. In these studies, text message response rates vary widely, but are often low, with some studies reporting rates as low as 17% [15,18,19]. Variability is likely due to variation in user attributes (eg, health literacy status and diabetes self-efficacy) and/or intervention characteristics across studies, but associations (or lack thereof) are often not reported [15]. In the few studies reporting associations, which span texting, interactive voice response, and internet-based interventions, patient characteristics such as older age [20], being nonwhite [20,21], and lower health literacy [20,22,23] have been linked to lower engagement.

Closely examining the features of the intervention may inform the conditions under which participants are more likely to engage. In text message interventions, specifically, the ideal frequency (ie, dose) of sending text messages to sustain engagement is unclear [12]. Suffoletto [24] referred to this as the Goldilocks problem: sending too few texts may not produce a strong enough effect, whereas sending too many texts may lead to participant fatigue and burnout. Furthermore, text messaging interventions often include additional modalities for delivering content, such as a human support component [25]. However, few studies have compared engagement between a condition receiving only automated text messages and a condition receiving texts in conjunction with human support.

Exploring engagement in long-term text messaging interventions is a critical step in understanding whether text messages are a viable option for improving and sustaining health outcomes. Identifying the determinants of intervention engagement can inform the design of interventions to optimize engagement [12,26]. Studies reporting on engagement with text messaging interventions tend to be short term, based on small sample sizes, and involve predominantly non-Hispanic white patients, limiting the understanding of long-term, generalizable results [17]. Important gaps in knowledge must be addressed to enhance the expanding use of text messaging technology to support chronic disease self-care in vulnerable groups at the greatest risk for worse outcomes.

We used mixed methods to examine user engagement in a 12-month text message–delivered intervention designed to support diabetes self-care, called REACH (Rapid Education/Encouragement And Communications for Health) [27,28]. We sought to explore how engagement changed over the course of the intervention, patient characteristics associated with engagement, whether the addition of a human component or letting participants choose their text frequency affected engagement, and patients’ reasons for their preferred text message frequency.

This research was conducted as part of a larger randomized controlled trial (RCT), evaluating the effects of mobile phone–based support on diabetes self-care and hemoglobin A_1c (HbA_1c) [27]. The trial and intervention details have been previously described in our development and protocol papers [27-29]. For this study, we analyzed data for participants who were randomly assigned to the intervention.

We recruited patients from federally qualified health centers (FQHCs) and Vanderbilt University Medical Center (VUMC) primary care clinics in and around Nashville, Tennessee. Eligible patients were aged ≥18 years, diagnosed with T2D, prescribed a daily diabetes medication, and responsible for administering their diabetes medications. In addition, patients were required to own a cell phone with text messaging capability and speak and read in English. We excluded participants whose most recent HbA_1c value within 12 months was <6.8% and who had auditory limitations or an inability to orally communicate, as determined by trained research assistants (RAs). We also excluded patients who failed a brief cognitive screener [30] to help ensure the accuracy of the measures and data integrity. Finally, due to the requirements of the intervention, we excluded patients who were unable to receive, read, or send text messages after a demonstration by an RA.

The Vanderbilt University institutional review board approved all study procedures. Interested and eligible patients completed informed consent, a baseline survey, and an HbA_1c test. RAs collected additional information from the participants’ electronic health record. RAs entered participants’ data into Research Electronic Data Capture (REDCap) [31]. Participants’ relevant survey responses were transferred from REDCap to a digital health platform called MEMOTEXT, via an automated application programming interface. MEMOTEXT used participant information to tailor, schedule, and send text messages to participants. When participants completed follow-up assessments at 3 and 6 months, updated patient information was used to retailor the text message content. When participants completed their 6-month assessment, they were given the option to continue receiving daily text messages or to receive fewer text messages, for the remaining 6 months of the intervention (described in The Intervention below). Participants could earn a total of US $210 for completing all study measures (ie, through 15 months for the larger RCT), with the payment schedule increasing for longer-term follow-ups.

We used strategic purposeful sampling to invite a subset of intervention participants to complete a follow-up interview after they finished their participation in the trial. Interviews were designed to assess perceptions of the intervention and understand dose choice and potential for implementation. Factors that informed our sampling approach included assigned condition, age, gender, race, education, income, clinic site, whether they owned a basic phone or smartphone, and their chosen text frequency. Interviews were conducted either in person or by phone and took approximately 20 min to complete (range 11-40 min). All interviews were audio recorded and transcribed verbatim. Participants were paid an additional US $40 for completing the interview. Herein, we report specifically on the participants’ comments about responding to text messages and reasons for their text message frequency choice after 6 months of receiving daily texts.

All participants received self-care promotion text messages. Each message was either tailored to address user-specific barriers to medication adherence based on the Information-Motivation-Behavioral Skills model [32,33] or addressed other self-care behaviors (ie, diet, exercise, and self-monitoring of blood glucose). Participants chose a window of time to receive this text (eg, between 3 PM and 6 PM). Participants also received interactive text messages asking about medication adherence. We scheduled these to be sent after participants took their last dose of diabetes medication, but before their bedtime, to allow time to respond. At the end of each week, participants received an adherence feedback message based on their responses to the interactive text; feedback included an encouraging statement tailored to whether the participant’s adherence improved, stayed the same, or declined relative to the week prior.

For the first 6 months, all participants received daily self-care texts and daily interactive texts. After 6 months, participants had the option to receive fewer text messages for the remaining 6 months of the intervention (ie, low-dose) or continue to receive daily text messages (ie, high-dose; Figure 1). If we could not reach participants for their 6-month assessment and therefore were not able to present this option, they continued to receive daily text messages (high-dose).

Participants assigned to receive REACH text messages were randomly assigned to also receive monthly phone coaching to set diabetes self-care goals for the first 6 months at a ratio of 1:1 [27,29]. After 6 months, phone coaching ended for those so assigned, all participants were offered the low-dose option described earlier, and all continued to receive REACH text messages for the next 6 months.

When sent daily, the interactive text message asked participants if they took all their diabetes medication that day, requesting a yes or no response (Figure 2). On the basis of usability testing results [28], we allowed a range of acceptable response options representing yes and no (eg, yeah, Y, yep, N, and nope). If participants provided a no response, a follow-up text was sent asking why with various response options. Participants received their adherence feedback text based on their daily responses at the end of the week. When sent weekly for participants who chose low-dose, the interactive text asked participants how many days that week they had taken all their diabetes medication, requesting a 0 to 7 response. After participants provided a response, they received their adherence feedback text message (Figure 2).

If the participant responded to the interactive text message with something other than a solicited response, error messages were triggered (Figure 2). The system accepted any response until the next text message was sent the subsequent day.

We monitored participants’ responses to the interactive texts weekly throughout the trial. If a participant did not respond to the interactive message for 2 consecutive weeks, an RA called the participant to identify any technical problems. To avoid pressuring the participants to respond, the RA only asked whether they were experiencing technical problems with receiving or responding to text messages and would troubleshoot as needed. If a participant remained nonresponsive after we confirmed that they were not having problems, we did not make repeat calls.

All statistical analyses were performed using R version 3.5.1. We described patient characteristics using means and SDs or frequencies and percentages as appropriate. We computed the following descriptive statistics regarding text message response rates: means, medians, and the first and third quartiles (interquartiles).

In the trial, 256 participants were assigned to receive text messages. Of those enrolled, 8 participants were not included in these analyses due to early withdrawal (n=5), a technical error resulting in text messages not being sent (n=2), or opting out of text messages during the first 30 days (n=1). The remaining 248 participants were included in the quantitative analyses. Participants were, on average, aged 55.8 (SD 9.8) years, 55% (137/248) were female, 52% (129/248) were nonwhite, 39% (97/248) were black, 41% (101/248) had a high school degree or less education, and 41% (101/248) had an annual household income of <US $25,000 (Table 1).

Of the 46 participants invited to complete a follow-up interview, 36 (78%) did so. The characteristics of the interviewed participants were similar to those of the larger sample: aged 51.5 (SD 11.0) years, 56% (20/36) female, 67% (24/36) nonwhite, 53% (19/36) black, 44% (16/36) with a high school degree or less education, and 53% (19/36) with an annual household income of <US $25,000.

Participants responded to a mean of 81% (SD 23%; median 91%; IQR 75%-97%) of the interactive text messages over the 12-month intervention. Less than 1% of participants’ responses to these texts were unexpected responses (ie, a response other than yes or no to the daily text message or 0 through 7 to the weekly text message). Response rates were negatively skewed such that over two-thirds of participants’ response rates were greater than 75%. Participants’ average response rate decreased over time, although the predicted rate never dropped below 70% (Figure 3).

We also examined engagement separately during the first and last 6 months of the intervention. During the first 6 months, participants responded to a mean of 85% (SD 20%; median 94%; IQR 80%-98%) of the interactive text messages, and during the last 6 months, participants responded to a mean of 81% (SD 22%; median 92%; IQR 75%-97%). For both time periods, we found statistically significant time trends such that participants had decreased odds of responding over time, although there was little substantive change (Table 2). We estimated the odds of responding to be about 0.7% lower on a given day as compared with the previous day during the first 6 months and to be about 0.4% lower on a day-by-day basis during the latter 6 months (Table 2).

Patients’ clinic site, age, gender, education, income, diabetes duration, insulin status, health literacy, numeracy, and diabetes self-efficacy were not associated with 6-month engagement (Table 3). However, black race (compared with white), worse baseline medication adherence and worse HbA_1c were associated with lower engagement, although the effects were small (2%-7% lower; Table 3). We did not find evidence of an association between any measured patient characteristics and change in 6-month engagement, and we did not find evidence of an association between any measured patient characteristics (including prechoice response rate) and dose choice (Table 3).

Participants’ assigned condition (ie, whether they received text messages only or text messages plus phone coaching) was not associated with their 6-month response rate (Table 2). In addition, the odds of responding over time did not differ by condition (Table 2).

Among participants given the option to receive fewer text messages at 6 months (n=213), 55.9% (119/213) of participants chose to receive fewer texts and 44% (94/213) chose to continue receiving daily texts. The remaining 35 participants were not asked this question because of not completing the 6-month assessment, being unreachable by phone, or an RA error. Dose choice was not associated with the odds of responding postchoice (Table 2). When we examined whether dose choice was associated with a change in the odds of response, from before to after making the choice, there was no association. Odds of responding were lower over time in both groups but did not differ by choice (Table 2).

We examined user engagement in a 12-month text messaging intervention designed to support diabetes self-care in a sample overrepresenting adults with low SES and racial/ethnic minorities. We wanted to understand how intervention characteristics and patient characteristics affected engagement with the text messages, with particular attention to the addition of a human component and participants’ opportunity to choose their text message frequency after 6 months of daily texts. Engagement gradually decreased throughout the intervention but remained relatively high over 12 months. We did not find any differences in engagement by participants’ clinic site, age, gender, education, income, diabetes duration, insulin status, health literacy, or numeracy. Engagement was lower among participants who were black and who had worse medication adherence and HbA_1c at baseline; however, the magnitude of these differences was relatively small. When given the option at 6 months to receive fewer text messages, nearly half of the participants chose to continue receiving daily text messages for the remaining 6 months. We did not find differences in engagement based on participants’ receipt of phone coaching or their selected text message frequency.

The literature reporting on engagement with mobile health (mHealth) interventions is growing, although few studies report on interventions lasting more than 6 months [17]. Our findings support the use of text messages to engage participants long term. Before starting the RCT, we conducted iterative usability testing with participants from the target population; we addressed participant concerns regarding content and functionality, which may have helped sustain engagement in the trial [26,28]. In addition, the personalization used in REACH (eg, tailored content and message timing) may have promoted engagement, which is consistent with other studies employing individualized content [12]. Black participants and those with worse baseline medication adherence and HbA_1c had lower engagement. Notably, although these differences were statistically significant, average engagement was still relatively high across all participants (eg, the average response rate among black participants was 80%, whereas for white participants, it was 87%). On the basis of the qualitative data, participants’ engagement appeared to be influenced by the perceived helpfulness of the text messages with reminding them to take their medication and feelings of accountability engendered by the texts.

Studies exploring the role of human support in engagement with automated technologies have shown inconsistent findings [26,41]. Variation in results are likely due to variation in the type of human involvement (eg, remote vs in person, familiar vs new health care professional, and providing structured support vs following up as issues arise), the type of technology, the engagement measure, and users’ needs and preferences [41]. In this study, there was no difference in text message response rates between users who received text messages only and those who received both text messages and monthly phone coaching. Mohr et al [42] proposed a model for how human support increases adherence to electronic health interventions, such that patients experience accountability to a person they view as trustworthy, benevolent, and having expertise. Although the text messages in this study were automated, the content contained encouragement, information, and bidirectional communication, which may have been perceived as human support, thereby increasing engagement, regardless of assignment to phone coaching. In a study that explored perceptions of a similar automated text messaging program for diabetes self-care, black patients with T2D shared how they felt high levels of social support and cared for by a person or friend despite knowing that the program was automated [43]. Furthermore, in this study, all participants experienced a human component from interacting with the study staff for other elements of the study (eg, completing surveys), which may have dampened our ability to detect additive effects of the monthly coaching.

The ideal dose or frequency of text messages to sustain engagement in an intervention is unclear and likely varies by individual user. We did not find any patient characteristics associated with frequency choice, which suggests that preference was idiosyncratic. We included the low-dose option to sustain engagement among participants who may prefer receiving text messages less often than daily. The variability in choice and the specific reasons for making the choice suggest it was valuable to participants. The choice may also have kept engagement high for some participants who otherwise would have stopped responding. Reconciling dose preference with the dose needed to impact patient outcomes will be integral for the design of future text messaging interventions [13].

This study recruited patients with T2D from a specific region in middle Tennessee. Therefore, we acknowledge that the findings may not be generalizable to patients with T2D in other locations or to other patient populations. In addition, because patients who did not want to respond to text messages were likely those who chose not to participate in this study, this may have led to a self-selection bias among our sample. As mentioned earlier, interactions with study staff may have reduced our ability to detect the effects of human coaching on engagement. We did not find evidence that patient characteristics were associated with dose choice; however, it is possible that characteristics not assessed in this study may have impacted the decision. For example, in the case of employment status, a person who is retired may be more receptive to receiving a higher frequency of messages than someone who is working full time. Finally, we were unable to compare how engagement would have been impacted if we did not give a choice for frequency preference.

There is growing evidence for the efficacy of text message–delivered interventions to improve adherence and clinical outcomes [11], and this study addresses important gaps by examining long-term engagement and associations between engagement and participant and intervention characteristics. As the literature on engagement is growing, there have been calls for more standardized engagement reporting to compare results across studies and draw more concrete conclusions about the role of engagement [44,45]. Kelders and Kip [46] recently developed a self-report scale to capture multiple components of engagement with health technologies (ie, behavior, cognition, and affect). Similarly, we encourage reporting on usage that goes beyond the average engagement level, but rather examines how engagement changes over the course of an intervention, and factors that influence engagement. In any intervention, users will become fatigued over time, so testing engagement promotion strategies (including new content to sustain interest over time and/or gamification) will also provide insights on how to improve and sustain engagement [47,48]. Finally, robust measurement of engagement will also help us explore the effect of engagement on outcomes to enhance our understanding of how and for whom mHealth interventions improve health. As a next step to this study, we plan to evaluate how engagement with REACH impacts outcomes, including HbA_1c and medication adherence. Specifically, we will explore whether there is a minimum engagement threshold (ie, response rate) needed for patients to experience improvements in outcomes.

As patients with low SES, who are less likely to have smartphones and internet access, also tend to have worse health outcomes, the spread of apps and other internet-dependent technologies may unintentionally widen health disparities [49]. Compared with other forms of mHealth technology, text messages offer key advantages given their ubiquity and potential for scalability [12,50]. This study contributes to evidence for the case that these interventions may help create equity if implemented in clinical care.