Worldwide, cardiovascular disease (CVD) is a major cause of premature morbidity and mortality [1]. Observational studies have shown that a few potentially modifiable risk factors account for most of the risk of CVD outcomes [2,3]. These risk factors, which include abnormal blood lipids, smoking, diabetes, and high blood pressure, are common and often occur together in the same individual. For example, more than two-thirds of people with diabetes also have high blood pressure [4]. There is good evidence now that low-cost medications and changes in lifestyle can improve risk profiles for individuals [5]. However, suboptimal adherence to prescribed medication and lifestyle modification is a major barrier to the health benefits of these evidence-based treatments [6].

Mobile communications technology has the potential to support behavior change and treatment adherence by facilitating remote, interactive, and timely access to relevant information and providing context-specific support and prompts to action [7]. The simplest and most widely available mobile communications standard is the short message service (SMS) text message. Results from some but not all trials suggest that SMS text message interventions may be a useful adjunct to usual care for the prevention and management of CVD [8-10]. The quality and success of these interventions partially depend on how they have been developed. Guidelines for the development of digital [11] and mobile Health (mHealth) behavior change interventions [12,13] now exist. They propose a number of steps to follow to maximize intervention benefit, including formative research for insights into the target audience and health behavior, designing the text messaging program, pretesting the text messaging program concept and messages, and revising the text messaging program.

However, the development process of mHealth interventions is frequently underreported and not well understood, having been described as “hidden within a black box” [14]. Understanding how mHealth interventions are developed is crucial to allow replication and adaptation of successful interventions across settings—a necessary step to accumulating evidence around the effectiveness of these interventions. In 2016, the International Consortium of Text2PreventCVD Trial Collaborators Group was established as part of an initiative to gather and analyze individual participant data from randomized clinical trials identified as part of a systematic literature review of text messaging interventions to prevent CVDs [15]. This international consortium offered a unique opportunity for an in-depth examination of the development processes of these interventions, based on information directly supplied by the developers of the interventions (rather than merely relying on data reported in available publications, as commonly done in conventional systematic literature reviews).

The aim of this study was to systematically describe and compare the processes followed to develop the text messaging interventions for the prevention and management of CVD that were included in the Text2PreventCVD meta-analysis.

We undertook a systematic description and comparison of the development processes of the interventions included in the Text2PreventCVD systematic review (the results from the individual participant data meta-analysis examining the effectiveness of the interventions in the prevention of CVD will be published elsewhere).

The protocol including the methods for the identification of trials included in the Text2PreventCVD systematic review is available elsewhere [15]. In short, 4 electronic databases of published studies (MEDLINE, EMBASE, PsycINFO, and Cochrane Library) and international trial registries were searched to identify potentially relevant studies. The eligibility of the retrieved references was assessed by 2 independent reviewers based on the following inclusion criteria: (1) randomized clinical trials of mobile phone SMS or text message intervention with a control group receiving standard care (no messages or some form of control message), (2) follow-up period of at least 6 months, (3) with a minimum 70% of completed follow-up of patients, (4) focusing on CVD primary and secondary prevention in men and women aged 18 years and older, (5) evaluating interventions applying at least 2 behavioral techniques to support behavior change (as an indicator of a comprehensive CVD prevention program rather than a single focus, eg, medication adherence reminders), and (6) with a minimum total sample size of 30 participants (as sample size was perceived to be a surrogate marker of study quality).

The principal investigators of the 9 selected trials [8,9,16-34] meeting the inclusion criteria described above were invited by email to join the International Consortium of Text2PreventCVD Trial Collaborators Group. Reminders were sent after a week to nonresponders, followed by approaching other investigators by email, phone calls, and fax. Investigators were requested to share their data after obtaining a signed agreement. Authors of 5 of the trials (Tobacco, Exercise and Diet Messages [TEXT ME] [9,19,32], Islam et al [22,33], Heart Exercise And Remote Technologies [Heart] [24-26,30], Mobile Phone Text Messages to Support Treatment Adherence in Adults With High Blood Pressure [StAR[ [8,18], and Text message and Internet-based comprehensive cardiac rehabilitation intervention [Text4Heart] [21,29]) agreed to participate by providing data on the development of their interventions.

Before data extraction, 3 researchers (IRC, AF, and KB) designed a data extraction form in an iterative process. The Medical Research Council framework for the development of complex interventions [35] and other available frameworks [11,12,36,37] were used to ensure the data extraction form covered all relevant areas in the development of mHealth interventions. The resulting data extraction form covered the following 4 domains: intervention planning, intervention design, intervention development, and intervention pretesting. The final form was reviewed and approved by the senior authors of the 5 trials included in this study. One researcher (IRC) then used the final form to extract all the relevant data available in the full texts and appendices of the manuscripts previously identified. Subsequently, the extracted information was sent by email to the authors of the 5 trials that agreed to take part in this study. They were requested to cross-check the accuracy of the extracted information and to provide additional details not reported in the available publications. When needed, we recontacted the authors for additional details or clarifications about the methods used to develop their intervention.

We carried out a narrative synthesis of the results, describing the different methods identified for developing the interventions. We classified the methods according to the 4 development stages considered (intervention planning, design, development, and pretesting). We calculated, in terms of frequency, the number of interventions that applied each of the methods identified and tabulated this information (Table 1). To allow head-to-head comparisons across interventions, we present in tables (Tables 2-4), and describe narratively, more detailed information about the methods used for the development of each intervention.

We extracted information about the main characteristics of the selected mHealth interventions, including the level of customization of the messages, timing and frequency of delivering the messages, content (degree of personalization, readability, length, and presence of sender´s signature), directionality (unidirectional or bidirectional), and type of platform used to deliver the messages (see Multimedia Appendices 1 and 2). All the interventions were based on text messages containing less than 160 characters, which included the sender’s signature, and which were sent free of charge to participants. Most interventions included some degree of customization either in the content of the message (TEXT ME [9,19,32] and Heart [24-26,30]), preferred timing (Text4Heart [21,29] and StAR [8,18]), or language (StAR [8,18]). The number of messages sent per week ranged from 1 (StAR [8,18]) to 7 (Text4Heart [21,29]). Furthermore, 3 interventions (TEXT ME [9,19,32], Islam [22,33], and Heart [24-26,30]) used unidirectional messages, whereas 2 interventions (Text4Heart [21,29] and StAR [8,18]) used bidirectional messages (ie, allowing the participants to not only receive but also send messages).

Although the interventions generally aimed at preventing CVD, they varied in terms of their specific goal and population targeted. The TEXT ME intervention aimed at decreasing cardiovascular risk by encouraging lifestyle change in people with coronary heart disease in Australia [9,19,32]. The Text4Heart intervention aimed to support adherence to healthy lifestyle behaviors in adults with coronary heart disease in New Zealand [21,29]. The intervention developed by Islam et al [22,33] aimed to improve glycemic control in patients with type 2 diabetes in Bangladesh. The Heart intervention aimed to promote exercise capacity and physical activity behavior in people with ischemic heart disease in New Zealand [24-26,30]. The StAR intervention aimed to support treatment adherence and blood pressure control in adults with hypertension in South Africa [8,18].

All the interventions followed a comprehensive development process (or framework) that involved 1 or more studies to inform each development stage, including the planning, design, development, and pretesting of the interventions (Table 1).

In this study, we conducted an in-depth examination of the development process for 5 of the 9 mHealth interventions identified as a part of the Text2PreventCVD meta-analysis. We observed that the development process of these interventions was complex and comprehensive, involving formative work with multiple studies using both quantitative and qualitative designs to gather evidence to inform decisions about the scope, content, and structure of the intended interventions. Although formative studies varied in terms of the way the interventions were developed, the following processes were identified in all the interventions: use of theory and behavior change techniques for intervention design, use of end-user feedback for intervention development, and pilot studies for intervention pretesting.

It has been argued that, as in any health promotion practice, quality, rigor, and careful development of mHealth interventions remain essential [49]. However, a number of recent systematic reviews highlight that a significant proportion of health behavior interventions are not based on theory [50-52], do not follow a user-centered approach [11,53], and lack adequate pretesting procedures [54,55]. Indeed, it has been argued that the SMS text message development is commonly hidden within a black box, lacking deconstruction of the steps involved in creating messages as well as rigorous evaluations of the quality of the health communication messages [14]. None of this is supported by findings from our study, which showed that the development process of the interventions contributing to the Text2PreventCVD meta-analysis included those elements. At least 3 factors could contribute to explaining this apparent discrepancy. First, as opposed to previous reviews, our findings are mostly based on data supplied by the authors of the studies themselves rather than merely by information in published manuscripts. Therefore, it is plausible that some interventions were not underdeveloped but rather underreported. Previous studies have already noted that the documentation and critical analysis of the formative research process required in the development and refinement of messages is a neglected area in the published literature [56]. For quality assessment purposes and to improve future research and program implementation, this research should be accessible for scientific scrutiny. This should include a description of the development of the SMS content (and quality of the evidence on which the messages are based), the theoretical basis of the messages (and the rationale for choosing it), cultural adaptation to the target population, and context along with an analysis of how the content is received by users [54,57,58]. This would enable translation and assessment of generalizability of findings of the research. Second, although all the examined interventions included as part of their development the use of theory, user feedback, and pretesting activities, it could be argued that these elements were not rigorously applied, a common criticism in the current literature [11,37,52]. This may be particularly relevant with the use of theory; superficial use of theory in the development of health behavior interventions has been identified as one of the main barriers hindering progress in quantifying its impact on behavior change [52]. Third, almost half of the interventions initially identified were not able to be included, and this could have resulted in a selection bias (leading to an overrepresentation of more comprehensively developed interventions), as discussed as part of the limitations below.

Future studies are needed to explore the extent to which the development process is associated with intervention success and to identify active ingredients in the development process. To do so, valid and reliable measures of intervention development complexity are very much needed as well as data from a larger number of more heterogeneously developed interventions. Publication bias (a common source of potential bias in systematic reviews [59]) should also be adequately accounted for in such studies, as less successful interventions may be less likely to be published and may be less likely to have followed a thorough development process. Intervention success should not only be evaluated in terms of clinical effectiveness observed within the context of a clinical trial but also in terms of research translation capacity, that is, the extent to which the interventions are successfully implemented at the population level and are acceptable and perceived as useful by the target populations.

This study provides for the first time a detailed and systematic description of the development process of mHealth interventions to support behavior change in people with CVD. The data described in our study were initially extracted from available publications and subsequently reviewed by the authors of the relevant trials to maximize their accuracy and completeness. Our study has some limitations. First, although 9 trials were identified in the Text2PreventCVD meta-analysis, there was no contribution beyond published data for 4 of them (ie, the information we initially extracted from the publications was not reviewed and added by the developers). This could bias our results if the reason for not providing further data was related to the methods used to develop those interventions. Second, in our study, we could not examine the important question of whether more comprehensive development processes lead to more successful interventions. The main reason for not being able to explore this was the lack of sufficient variability across intervention development methods. As shown in Table 1, all the interventions included in this study were developed using complex and comprehensive methods: all of them carried out studies to identify the key behavioral issues, needs, and challenges that the planned intervention was intended to address; all of them used 1 or more theory and behavior change techniques to design the intervention; all developed the messages based on a number of different information sources; all gathered end-user feedback; and all conducted feasibility studies to pretest the interventions.

A detailed examination of the development of SMS interventions to improve CVD showed the processes used were complex and comprehensive. Our study identified a wide range of different methodologies to inform intervention planning, intervention design, intervention development, and intervention pretesting. Literature review and stakeholder consultations were the most frequently used methods to guide intervention planning and design. Regarding the development of the interventions, clinical guidelines and expert consultations were the main information sources for message content, whereas literature reviews and end-user feedback were the main source to inform decisions about other components of the interventions such as timing, directionality, or level of tailoring. Pilot studies with relatively small sample sizes were frequently used for intervention pretesting. Further work is needed to establish whether effective messaging systems can be adapted from work already done or whether this level of development is always needed for application in other conditions and settings. Additional research is also needed to examine the potential association between development methods and intervention success as well as to identify active ingredients in the development methods.