Effectiveness and Implementation of a Text Messaging mHealth Intervention to Prevent Childhood Obesity in Mexico in the COVID-19 Context: Mixed Methods Study

Introduction

Because of the short and long-term adverse effects of overweight/obesity on health and human capital, the World Health Organization (WHO) emphasizes its prevention early in life as a critical priority [1]. There is mounting evidence indicating that children from low socioeconomic backgrounds exhibit higher rates of overweight/obesity [2,3]. Mexico is particularly alarmed by the escalating prevalence of overweight/obesity among children under 5 years old, with the most recent national estimates projecting a combined rate of approximately 8% in 2022 [4]. Nevertheless, socioeconomically disadvantaged families are often more challenging for the health sector to engage with, and they may be less inclined to participate in programs promoting healthy behaviors [5]. Mobile health (mHealth) technologies and telecommunications present themselves as appealing low-cost interventions capable of reaching vast and remote populations [6].

In low- and middle-income countries [7], resources have been directed toward the development of mHealth interventions, including behavior change communication (BCC) strategies [8]. A 2019 systematic review [9] revealed that the most commonly used mHealth technology was SMS text messaging (60%). Additionally, there is evidence suggesting that eHealth and mHealth interventions are effective in promoting physical activity (PA) and healthy feeding (HF) in developing countries [10].

Hence, leveraging mobile phones for BCC interventions presents a promising opportunity to advocate for HF and PA practices among primary caregivers (PCs) of the Mexican child population. This is especially significant considering the rising and widespread use and accessibility of this technology, which has increased from 71.5% of the total population in 2015 to 79.2% in 2022 [11]. In particular, in emergency contexts where health services are disrupted and face-to-face information dissemination to the population is hindered, as was evident during the COVID-19 pandemic, leveraging innovative mHealth interventions becomes crucial. In this regard, testing and evaluating such interventions in emergency settings can significantly contribute to the existing evidence on mHealth. Moreover, it can serve to stimulate further research endeavors to complement and expand upon the findings of this study.

Hence, this research aimed to assess the effectiveness and implementation of an mHealth intervention, referred to as NUTRES, in promoting PA and HF practices among Mexican PCs of children under 5 years. A secondary objective was to document the lessons learned from implementing and evaluating an mHealth intervention during the COVID-19 pandemic.

Methods

Study Design

This study constitutes an effectiveness-implementation hybrid trial [12], incorporating elements from both effectiveness and implementation research. The trial is registered under the Trial Registration ID NCT04250896. We used a mixed methods approach [13], utilizing a “convergent advanced design” [14]. This design involved the simultaneous use of qualitative and quantitative methods, allowing for the triangulation of results from both methodologies. By combining statistical findings with insights gleaned from individuals’ real-life experiences, we aimed to gain a comprehensive understanding of the effects and implementation process of NUTRES.

Eligibility Criteria and Recruitment of Participants

NUTRES participants were recruited from urban and rural health units situated in 2 states of Mexico: Morelos in the central region and Yucatán in the southern region. Randomization was conducted at the health unit level. Inclusion criteria comprised being a primary health care unit, being located in an area with access to a mobile phone network, having more than 50 registered users under 5 years, and having over 80% of the population with Spanish as their first language. A total of 308 eligible urban and rural health units from the 2 states were included in the study, assigned either to the intervention group (IG) or the comparison group (CG) (Figure 1).

‎

The eligibility criteria for participants in both the IG and the CG were being the PC of an infant under 5 years, having ownership/access to any type of mobile phone, being able to speak and read Spanish, being aged 18 years or older, and being a resident in the coverage area of the participating health unit. Participants were recruited through 3 methods: (1) invitation from primary health providers, (2) snowball sampling, and (3) face-to-face encounters by field workers at health units. The CG had the same eligibility criteria for recruitment, except for the requirement of mobile phone ownership/access.

Sample Size

Quantitative

Sample size calculations indicated that 100 participants per study group would yield 80% power at the .05 significance level to detect a 20%-point difference in the proportion of change for at least one outcome variable (eg, knowledge, attitudes, or practices related to infant PA or HF) between study groups, using a 2-sided test. These calculations accounted for a design effect ranging between 1.5 and 1.9 [15] and factored in a 20% dropout rate [16]. Notably, this sample size estimation did not consider states or areas as strata, thus precluding intergroup comparisons based on geographical regions.

Qualitative

The sample selection was purposive, with the aim of capturing a diverse range of experiences associated with NUTRES. Informants were chosen from both rural and urban areas, encompassing caregivers with children aged 0-23 months and 2-5 years, representing varying levels of interactivity. The use of 2-way SMS text messaging served as an indicator of their engagement with NUTRES (details on interactivity provided below).

NUTRES Intervention

NUTRES is an mHealth BCC strategy designed to prevent childhood overweight/obesity. It achieves this by disseminating SMS text messages to PCs of children under 5 years, as well as to health personnel operating within primary unit services in 2 Mexican states. In this paper, we will primarily concentrate on the outcomes of the SMS text messaging interventions aimed at promoting infant PA [17] and HF [18]. For a comprehensive understanding of the development of NUTRES, readers are referred to previous publications [19,20].

In essence, the NUTRES intervention comprised 1- or 2-way SMS text messages, each containing fewer than 150 characters, disseminated to PCs over 9 months (equivalent to 36 weeks). Participants in the IG received approximately 108 SMS text messages, adjusted based on the child’s age. These SMS text messages addressed various aspects, including attitudes and practices concerning infant PA and HF. Additionally, SMS text messages addressing nutrition within the context of the COVID-19 pandemic were incorporated due to the emergency situation. The design of the SMS text message content was informed by extensive formative research [21], which aimed to identify both barriers and facilitators to the adoption of healthy practices within the target population. Every factor that could either facilitate or hinder behavior change was systematically addressed in line with the Theory of Planned Behavior [22,23]. This was accomplished through the dissemination of truthful information, practical tips, healthy recipes, challenges, and socioemotional support messages to PCs. All SMS text messages were meticulously aligned with the latest national [24] and international guidelines concerning infant PA and HF [17]. See Multimedia Appendix 1 for examples of SMS text messages delivered.

PCs who consented to participate in NUTRES were asked to provide details about the child’s name, age, and gender during registration. This information was used to personalize and tailor SMS text messages for each participant accordingly. NUTRES incorporated 2 types of SMS text messages: 1- and 2-way (the latter being sent following a response from PCs to the initial SMS text messages). On average, participants received approximately 3 SMS text messages per week. In addition to informational messages, a weekly positive socioemotional support SMS text message was dispatched to PCs, as research has demonstrated its effectiveness in fostering participation and maintaining interest [25]. These SMS text messages were programmed to be automatically delivered to PCs via the Rapid Pro platform [26], and interactivity, including responses from PCs, was recorded and tracked.

Data Collection and Analysis

Overview

Data collection took place from September 2020 to September 2021. The quantitative component encompassed both the IG and the CG, involving both baseline and final assessments. Meanwhile, the qualitative component solely focused on the IG, conducted after 36 weeks of exposure to NUTRES. Both the qualitative and quantitative teams, consisting of approximately 4 members each, possessed extensive fieldwork experience and proficiency in communication technology. They underwent a week-long virtual training session via Zoom (Zoom Video Communications, Inc.).

Quantitative Data

Because of the COVID-19 context, both baseline and follow-up surveys were conducted via mobile or fixed telephone. The surveys covered a range of topics, including sociodemographic information, health status, and knowledge pertaining to infant PA and HF. Knowledge of recommended practices was assessed by querying respondents about the advantages, disadvantages, or known recommendations regarding PA and HF. Each answer was assigned a score based on its correctness, as outlined in Multimedia Appendix 2.

Furthermore, exposure to NUTRES was gauged by querying PCs regarding their receipt of the SMS text messages, as well as their perception of the usefulness and practicality of the messages. For instance, PCs were asked if they recall a specific message from NUTRES that they had implemented in practice, and their responses were recorded without any prompts or suggestions from the interviewer.

Data collection was conducted using the REDCap (Research Electronic Data Capture; Vanderbilt University) app, with daily verification and backup procedures in place. Following data collection, thorough exploration, cleaning, and recategorization processes were undertaken to prepare for descriptive statistics and basic comparative analyses between study groups. Topics of interest were examined and translated into knowledge and practice indices, with summary variables generated for each question. Practices were categorized into 3 equally sized tertiles for evaluation: the first tertile was termed “limited,” the intermediate tertile was termed “moderate,” and the tertile with higher values was termed “adequate.” Lastly, we used a double-difference approach [27] to evaluate the variance in the change of outcomes between the presence (IG) and absence (CG) of the intervention. A significance level of P<.05 was deemed significant. All statistical analyses adhered to an “intention-to-treat” principle [28] and were conducted using the Stata version 14.2 statistical package (StataCorp).

Qualitative Data

Interviews were conducted after the quantitative survey and delved into various topics, including opinions regarding NUTRES; technical issues encountered with receiving, reading, and responding to SMS text messages; overall impressions of the SMS text message content; behaviors encouraged by the messages; and the perceived impact of the intervention on the intention or ability to carry out recommended practices. Additionally, we explored 3 implementation indicators with PCs from the IG:

• “Acceptance” of NUTRES and behaviors promoted by SMS text messages.
• “Pertinency,” which examines the relevance of information and behaviors promoted by the SMS text messages in NUTRES.
• “Coverage,” which examines the extent to which PCs received the SMS text messages.

The first 2 implementation indicators were proposed by Proctor and colleagues [29], while the last one was by Peters and colleagues [30]. According to these authors, these indicators are crucial determinants of the success, in terms of both implementation and expected outcomes, of an intervention.

The interview guide underwent a pilot phase involving role-play to ensure its smooth flow and enable interviewers to practice its implementation. Telephone interviews typically lasted between 40 and 80 minutes, and they were recorded and transcribed verbatim. Following the principles of grounded theory [31], data analysis was conducted using a coding tree [31] to streamline the coding process. This analysis was performed utilizing NVivo 2020 (QSR International). See Multimedia Appendix 3 for the category tree used for coding.

Ethical Approval and Consent

This study received approval from the Ethics, Research, and Biosafety Committees of the National Institute of Public Health (INSP) of Mexico, with the reference number CI 1547. All participants provided verbal informed consent to take part in the study.

Results

General Data About Design, Participants, and Sample Size

Quantitative

The total number of participants at baseline was 494 PCs, with 230 in the IG and 264 in the CG. Out of these, a total of 334 participants completed the study, comprising 118 from Morelos and 216 from Yucatán (Figure 1) [32]. The primary reasons for the loss to follow-up were the inability to locate the PC for final measurement, accounting for 9.1% (20/220) from the IG and 18.1% (41/227) from the CG, and the unwillingness of some participants to continue with the research, constituting 4.3% (8/187) from the IG.

The baseline characteristics of PCs revealed that the majority were young, with an average age of approximately 28 years. Most PCs were married or cohabiting and had a low level of education, with less than one-quarter having completed primary education. Additionally, at baseline, it was observed that the children of PCs from the CG were on average 3.7 months older than those from the IG (P<.001). Furthermore, PCs from the CG were less likely to report having a paid job compared with PCs from the IG (P<.004; Table 1). Baseline characteristics were found to be similar between PCs who continued in the study and those who were lost to follow-up. This is detailed in Multimedia Appendix 4, which presents the descriptive characteristics of individuals lost to follow-up versus those who completed the study.

Qualitative

Twenty-four PCs from the IG were interviewed, with 12 participants from each state. These PCs had an average age of 30.2 years, with 14 participants having completed high school education or higher. The majority of interviewed PCs were homemakers, totaling 16 individuals. Multimedia Appendix 5 provides further details on the main characteristics of PCs interviewed for the NUTRES study conducted between 2020 and 2021. Interestingly, only a few differences were observed in the experiences reported by PCs of children aged 0-23 months and those aged 24-59 months, as well as between rural and urban areas. Consequently, qualitative findings and PC experiences are presented in a general manner.

NUTRES Intervention Implementation

PCs expressed appreciation for the SMS text messages from NUTRES, noting that they valued their brevity and clarity. Informants unanimously agreed that the NUTRES strategy was “good,” “useful,” and “helpful.” PCs highlighted that NUTRES served as a “reminder” of previously acquired knowledge, while also introducing “new” information, particularly regarding the promotion of PA. PCs also remarked that the changing topics in the SMS text messages as their children grew made them feel supported and motivated to implement the recommendations, highlighting the acceptability and pertinence of the intervention (Textbox 1; Multimedia Appendix 6). Additionally, an overwhelming majority of PCs (96/98, 98%), expressed their desire to continue receiving NUTRES SMS text messages. However, some PCs raised concerns about sharing sensitive information, such as personal data, during the registration phase. Furthermore, a portion of PCs, specifically 32/145 (22%), reported encountering barriers to receiving SMS text messages from NUTRES. These barriers were damaged equipment (11/32, 34%) and issues related to connectivity or lack of mobile phone credit (7/32, 22%), indicating challenges in coverage (Table 2). Additionally, PCs indicated that they consistently use (43/141, 30.5%) and interact (27/140, 19.3%) with the 2-way SMS text message feature. Testimonies revealed that low interaction by PCs did not necessarily correspond to a lack of interest or rejection of NUTRES but rather to external circumstances such as a lack of credit or signal to receive SMS text messages, or a lack of awareness about the reply option (Textbox 1).

The majority of PCs (92/141, 65.2%) reported that the SMS text messages were practical. Additionally, 57/145 (39.3%) PCs expressed a preference for the informative format over other formats, such as tips or challenges. PCs particularly appreciated the challenges related to PA, with 61/133 (45.9%) expressing appreciation because it involved other family members (Table 2). This aspect was particularly valued, especially in the context of COVID-19.

What I liked most about the SMS was that they sent advice, physical activity, not only for Mateo, but for the whole family and everything. And the same advice for the whole family to follow, not just for the baby, for my son
[#3]

PCs recalled messages regarding the importance of avoiding certain unhealthy food items for their children, such as sugar-sweetened beverages or sausages. These messages emphasized that these items were not suitable for children and were associated with the risk of early chronic diseases.

I remember that they recommended not to give him boxed juices, but to give him plain water or natural juice, that is, natural fruit juice
[#2]

The same, [NUTRES] would send us, for example (...) the sausages and all that, because it was not the right thing to do, because they have fat, if not, that it was better for them to eat chicken, pork, beef
[#2]

NUTRES Intervention Effectiveness

At baseline, PCs from the IG exhibited greater knowledge and awareness regarding the risk of developing anxiety or depression due to lack of PA compared with the CG (4/179, 2.2% vs 13/149, 8.7%, respectively; P=.02). Following the intervention, significant differences between the 2 groups were observed, with the IG showing higher rates of favoring social interaction and integration in relation to knowledge of PA (CG: 8/135, 5.9% vs IG: 20/137, 14.6%; P=.048). Regarding HF practices, significant differences were observed between the IG and the CG in recognizing that diabetes and chronic diseases can develop as a consequence of sugar-sweetened beverage consumption (CG: 92/157, 58.6% vs IG: 110/145, 75.9%; P=.003; Table 3). However, in the difference-in-differences model, there was no significant improvement observed in the practice of infant PA and HF (P>.05; Table 4). The only significant increase between the study groups was observed in terms of knowledge about the benefits of PA (CG: mean 0.13, SD 0.10 vs IG: mean 0.16, SD 0.11; P>.02; Table 4). Nevertheless, several PCs reported improvements in their perceived control and intention related to some of the behaviors promoted by NUTRES, although detailed data on this aspect are not shown.

Well, as far as I know, I used to give my older children juice...soda, and [now] I don’t want to give her, I mean, any sweets
[#16]

Discussion

Principal Findings

To our knowledge, this study represents the first report on the feasibility of implementation and effectiveness of an mHealth intervention aimed at preventing childhood overweight/obesity in Mexico within the context of COVID-19. This research offers valuable insights into the implementation process of NUTRES and its impact on expected outcomes. Although NUTRES demonstrated limited impact on target knowledge and behaviors, the SMS text messages served as reminders or sources of new knowledge among PCs of Mexican children under 5 years.

The absence of significant effects observed in this study may be partially attributed to methodological adaptations necessitated by the COVID-19 context, such as recruitment and data collection conducted via telephone. However, it is conceivable that NUTRES could have served to mitigate the impact of the pandemic on the participating population. Given that the COVID-19 pandemic exacerbated food insecurity and contributed to increased consumption of ultra-processed products, along with physical inactivity during lockdowns, the intervention may have provided some degree of protection to the population involved [33]. Furthermore, the COVID-19 pandemic severely disrupted health services, significantly limiting the population’s access to essential nutritional counseling [34].

Additionally, the lack of significant effects could be attributed to the high rate of loss to follow-up, which, although not markedly different between the study groups (71/230, 30.9%, loss in the IG and 89/264, 33.7%, loss in the CG), represented a loss to follow-up exceeding 20% (160/494, 32.4%) [35,36]. However, considering a sample size of 150 participants per study group and assuming a correlation of 0.6 between repeated measures within participants, our analysis sample achieved an approximate power of 83% to detect a difference of 0.15 in the change of proportions [37]. Nevertheless, it is plausible that the registration and follow-up process, which involved requesting contact information such as the child’s name or age, may have discouraged the participation of PCs due to concerns about privacy and security. To address this issue, other studies have implemented strategies such as providing payments or reminders, resulting in a high retention rate of over 80% [38]. Such approaches could potentially enhance participant engagement and retention in future interventions.

Another important consideration to mention is that, due to the nature of the intervention, access to and use of a mobile phone was a prerequisite for inclusion in the IG. This aspect could introduce potential selection bias [39], especially considering that mobile phone ownership was not a requirement for the CG and mobile phone ownership might be correlated with income level. However, this potential bias was addressed in the analysis by adjusting for paid maternal employment (40/230, 17.4%, in the IG vs 22/264, 8.3%, in the CG), among other baseline characteristics. Adjusting for these baseline characteristics helps mitigate the potential influence of confounding factors on the study outcomes.

Previous studies have not consistently shown significant effects of exposure to BCC mHealth interventions on the intended audience [8]. In this trial, although 96/144 (66.7%) PCs always read NUTRES SMS text messages, only 43/141 (30.5%) always used the information provided. However, based on PC testimonies, NUTRES served as a reliable source of recommendations or reminders for child care, which they incorporated into their practices. Simultaneously, these recommendations empowered them within their immediate family and social environment. Furthermore, while PC narratives expressed some enthusiasm, the limitations of conducting telephone interviews prevented the observation of informants’ physical responses, hindering further exploration and discussion of their enthusiasm. Therefore, future mHealth interventions should incorporate features to enhance engagement over time, using various methods such as incentives, push notifications, and other interactive elements. These strategies could help sustain interest and participation in the intervention among the target audience.

The results from both the qualitative and quantitative components of NUTRES are consistent with the lack of effectiveness observed in other studies regarding the impact of mHealth interventions on PC practices related to PA and HF. For example, a study conducted in China found that a weekly SMS text messaging intervention over 12 months had no significant effect on infant feeding practices [40]. It could therefore be speculated that the 9-month duration of exposure in NUTRES may have been insufficient to bring about changes in infant HF practices. However, NUTRES did demonstrate an improvement in PC knowledge regarding some benefits of PA. Indeed, PCs emphasized in interviews that family challenges played a crucial role in guiding and supporting them in implementing the recommendations provided by NUTRES. While the effectiveness of NUTRES could not be conclusively demonstrated, the mixed methods methodology used in this study identified that the obstacles to implementation were primarily external to NUTRES and largely technological in nature. Similarly, other studies conducted in Mexico have documented persistent barriers to mHealth implementation, such as the cost of mobile phones, lack of network infrastructure, and low digital literacy [41]. These barriers to implementing mHealth interventions may diminish the potential impact of smaller-scale studies and hinder their scale-up [41].

Our findings suggest that multicomponent and multichannel interventions have greater potential to promote behavior change compared with single-channel multitheme interventions. While the original design of NUTRES aimed for health providers to reinforce the intervention topics among PCs, this approach was not feasible due to the disruption of health services during the COVID-19 pandemic [42,43]. Nevertheless, an mHealth intervention could serve as a promising channel to maintain contact when face-to-face interaction is not feasible, complementing other remote interventions such as telemedicine [44,45].

One strength of this study is its effectiveness-implementation hybrid trial with an advanced convergent design, which enabled the comparison and interpretation of quantitative and qualitative results. Using a mixed methods approach, we were able to comprehensively observe the effects of the intervention and implementation process [13]. For example, our study revealed that the low interaction with SMS text messages by PCs did not necessarily indicate a lack of interest in NUTRES. Instead, it was often attributed to external factors such as poor signal reception, economic constraints, and other challenges. However, despite these insights, we were generally unable to demonstrate a significant effect of the NUTRES intervention on the target behaviors. Nevertheless, previous studies [46] investigating the usefulness of SMS text messages, as reported by PCs, suggest that valuable behavioral changes could potentially be achieved in the long term. Additionally, our difference-in-differences models were adjusted for child age and maternal employment, which were variables that differed between the groups at baseline. This adjustment allowed us to estimate the specific effect of the intervention. Furthermore, the double-difference approach accounts for time-invariant observed factors (eg, education) and unobserved variables (eg, motivation to comply) [27], providing a more robust analysis of the intervention’s impact.

Lessons Learned

The positive feedback received through both quantitative and qualitative data aligns with findings from another mHealth intervention [47,48]. A systematic review conducted in 2019 [9] identified the top 3 barriers among participants, which included signal coverage, lack of equipment (unaffordable phones), and technology gap (limited knowledge of how to use a phone). In this study, despite the participants’ positive perceptions, similar challenges were encountered in terms of interaction with SMS text messages. For instance, some participants reported that SMS text messages were blocked, requiring them to contact their mobile operator to resolve the issue, which they were often unaware of. Additionally, other reasons for not receiving the SMS text messages were dropped lines, being out of coverage area, or changes in phone numbers (15/32, 47%). When participants did not respond to the SMS text messages within 12 hours, they were removed from the screen (7/42, 17%), and a significant portion of participants dropped out of the study (160/494, 32.4%). Overcoming such barriers is crucial to increase exposure to the intervention, thereby enhancing its impact and sustainability [49].

Based on the results presented here, this study provides valuable insights that can serve as a foundation for further research using mHealth interventions. The advantages of mHealth interventions, including broad coverage, cost-effectiveness, and feasibility of implementation in the population, underscore the potential for future studies in this area. It is suggested that future research should focus on addressing the identified challenges to improve the effectiveness, efficiency, and scalability of interventions. Some potential areas for improvement are (1) using WhatsApp (Facebook, Inc.) instead of SMS text messages (or in addition to SMS text messages), as it is a useful and accepted medium among the target population; (2) using informative material to recruit PCs; (3) improving SMS text message reception; (4) clarifying that SMS text message interaction has no cost; and (5) including resubscription for loss/change of mobile phone (Table 2).

Another valuable lesson learned is the importance of engaging key health care providers [50] and identifying informal strategies to recruit participants during emergency situations [33]. Furthermore, we have observed that mHealth interventions during lockdowns may represent the sole means of supporting socioeconomically vulnerable populations, who were often the hardest hit [51].

Conclusions

The findings contribute to the existing evidence on the opportunities and limitations associated with mHealth interventions for BCC aimed at enhancing knowledge and practices related to PA and HF among caregivers of infants. Integrating mobile phone technology into BCC interventions facilitates the promotion of healthy lifestyles, particularly during emergency situations and among vulnerable populations. Moreover, the integration of effectiveness and implementation findings is valuable for optimizing mHealth interventions in particular settings, enhancing their sustainability, and fostering their application in diverse contexts [52].