Six-Month Outcomes from the NEXit Junior Trial of a Text Messaging Smoking Cessation Intervention for High School Students: Randomized Controlled Trial With Bayesian Analysis

Introduction

A steady decline in smoking prevalence has been observed in Sweden over the past decade, with the most recent data indicating that 7% of the general population were daily smokers in 2018 [1]. Although this decline is promising, the prevalence of daily smoking among individuals aged 16 to 29 years was 5% in 2018, and this rate increased to 16% when including occasional smokers. Among Swedish high school students specifically, approximately 5% self-reported being everyday smokers in 2019, and this rate increased to 20% when including occasional smokers [2]. Thus, young individuals in Sweden still start smoking. This is problematic since unhealthy lifestyle behaviors, including smoking, are established in adolescence and track into adulthood [3-5]. Effective smoking cessation interventions that target adolescents are therefore important for the declining trend in smoking prevalence to continue in Sweden.

Mobile phone–based interventions could potentially have far reach among adolescents, as mobile phone ownership in this group is almost universal in Sweden. Of particular interest are text message interventions, as they rely on standard technology that is ubiquitous in all mobile phones, and they have shown promise in other populations. Three meta-analyses have concluded that text messaging interventions have a positive effect on smoking cessation: one reported a summary effect size of 0.25 (95% CI 0.13-0.38) [6], the second meta-analysis reported an overall summary odds ratio (OR) of 1.37 (95% CI 1.25-1.51) of smoking cessation in favor of text messaging interventions [7], and the third analysis similarly found that quit rates were higher among those who had access to text messaging interventions (OR 1.36, 95% CI 1.23-1.51) [8]. Thus, there exists a relatively strong body of evidence for the effects of text messaging interventions. In addition, text messaging interventions may increase access to education and support services that promote smoking cessation [7].

However, only three trials have investigated the effects of smoking cessation text messaging interventions in adolescent populations exclusively [9-11], and only two of these have been conducted in high school student populations [10,11]. Two out of the three were small-scale trials including 179 [11] and 72 [9] participants, respectively, whereas the third was a large-scale cluster randomized trial including 2638 participants [10]. Findings from the large-scale trial suggested that although the number of cigarettes smoked per day was lower among those with access to the intervention, there were no marked differences with respect to smoking abstinence.

In 2018, we conducted the Nicotine Exit Junior (NEXit Junior) trial in response to the lack of evidence for the effects of text message smoking cessation interventions among high school students. We found that a 12-week text message smoking cessation intervention had a positive effect on the 4-week point prevalence of smoking cessation 3 months after randomization among weekly and daily smoking students (OR 1.87, 95% CI 1.12-3.17, P=.02) [12]. This report presents the results from the 6-month follow-up of the trial. Analyses prespecified in the study protocol are presented [13], as are unplanned Bayesian analyses of primary outcomes for both the 3- and 6-month follow-ups.

Methods

Design

A 2-arm, single-blind, parallel-groups randomized controlled trial was employed to estimate the effects of the NEXit Junior intervention on smoking cessation. The trial was prospectively registered (ISRCTN15396225) and a trial protocol was published in advance of trial recruitment [13]. This report adheres to CONSORT guidelines, and a CONSORT-EHEALTH checklist has been made available with this report. The trial received ethical approval by the Regional Ethical Committee in Linköping, Sweden (Dnr 2017/388-31).

Participants

Participants were simultaneously recruited from 630 high school units in Sweden. Students were recruited through paper advertising (poster and leaflets) and digital advertising (email, school website, app), and by promotion by school staff (teachers, mentors, school health centers). Recruitment began on January 10, 2018, and ended 1 year later (January 10, 2019).

Students interested in participating in the trial sent a text message to a dedicated telephone number. An automatic response was sent back with a hyperlink to a webpage where information about the trial was presented and students were asked to leave informed consent by pressing a button. Students who consented to participate were taken to a web-based questionnaire including items for both eligibility screening and baseline assessment.

High school students who reported being weekly or daily smokers, and were willing to make a quit attempt, were eligible for the trial. We included weekly smokers for two reasons: (1) this was the criterion for our previous trial of a text messaging smoking intervention among university students [14,15], and (2) weekly smoking in adolescents is cause for concern as it may escalate to daily smoking and long-term dependence. In addition, it was expected that participants have access to a mobile phone throughout the trial period. Individuals reporting not smoking, or doing so on a monthly basis only, were excluded from the trial.

Interventions

The text message intervention was a 12-week automatic, and unguided, program consisting of a total of 121 messages. During the first 2 weeks of the program, participants received 2 to 4 messages per day, which reduced to 2 messages per day during week 3, 1 message per day during weeks 4 to 7, and on average 5 messages per week during weeks 8 to 12.

The content of the messages was based on a similar intervention targeting university students in Sweden [14,15], which was based on existing evidence-based practice [10,16-22]. The content was further developed and refined for high school students using formative methods and behavior change technique analysis [23-27]. The content of the messages focused on the health risks of smoking and behavior change advice. In particular, participants were asked to make a public declaration about their intention to quit, encouraged to ask for support, taught distraction techniques, given tips on how to cope with cravings, and given advice on how to avoid smoking triggers.

Individuals allocated to the control group were advised that they would not obtain access to the novel intervention and that they instead could use the website of the national quit line [28] or contact their high school’s health service for more help. This was considered treatment as usual at the time of the trial, as this was in general what high school students who wanted help quitting were advised to do.

Outcomes

At 3 and 6 months after randomization, all participants were sent a text message with a hyperlink to a web-based follow-up questionnaire. Participants were sent two reminders 2 days apart, after which they were called by phone for follow-up (maximum of 10 attempts). The questionnaire contained items for the two primary outcomes: prolonged abstinence and point prevalence of smoking cessation.

Prolonged abstinence, following the Russel standard [29], was defined at the 3-month follow-up as not smoking more than 5 cigarettes in the past 8 weeks. At 6 months, the definition was altered to not smoking more than 5 cigarettes in the past 5 months. This outcome thus measures abstinence from the start of the 12-week smoking cessation program, allowing for a 4-week grace period.

Point prevalence of smoking cessation, a recommended outcome by the Society for Research on Nicotine and Tobacco [30], was defined as not having smoked a single cigarette in the past 4 weeks. This question measures current behavior, and thus was the same at both the 3- and 6-month follow-ups.

Sample Size

Sample size was determined by assuming that differences in cessation rates between the two groups would be similar to what was observed in a previous study of a text messaging smoking cessation intervention targeting Swedish university students [14,15]. We therefore expected a difference of approximately 10 percentage points between the two groups, which would require 195 individuals in each group to be detected with 80% power at the .05 (two-sided) significance threshold. We assumed that we would have 30% attrition at follow-up, resulting in a required total sample size of 558.

It should be noted that the expected 10 percentage point difference is much higher than what has been synthesized in meta-analyses of text messaging smoking cessation support versus minimal smoking cessation advice. The meta-analyses suggested that the differences may be in the range of 3-4 percentage points [31], which may be a more conservative choice for other trials to adopt. We used a greater difference in consideration of our previous research among university students in Sweden that yielded this difference using a similar intervention and trial design [14,15].

Randomization

After baseline assessment, eligible students were randomly allocated to the intervention or control group with equal probability. This was done on the backend computer server using Java’s built-in random number generator; thus, the sequence generation was computerized and fully automatic. Allocation was thereby also fully concealed until interventions were assigned.

Research personnel were blind to allocation at the time of randomization; however, participants were informed of which group they were allocated to. Follow-up assessments were completed using online questionnaires by participants on their phones; thus, group allocation was not revealed at the time of follow-up. However, there was a potential risk of group disclosure among those who did not respond to text message prompts and subsequently were called by research personnel to collect follow-up data (see Limitations).

Statistical Methods

The analyses conformed with the prespecified analyses in the trial protocol [13]. Specifically, logistic regression models were used to compare groups at the different follow-up intervals, adjusting for baseline variables: gender, number of years smoking, average number of cigarettes smoked weekly, severity of dependence (Fagerström Test for Nicotine Dependence), and the use of snuff. Participants were analyzed in the groups to which they were randomized.

Attrition analyses were planned to investigate potential differences between those who did and did not respond to follow-up. A penalized logistic regression model (lasso) was used to identify the baseline characteristics that were potentially predictive of nonresponse. This allowed us to explore departures from the missing completely at random (MCAR) assumption underlying the primary complete case analyses. Sensitivity analyses using multiple imputation with chained equations [32] were performed to assess robustness of the findings (using predictive mean matching, with 500 imputations and 30 iterations).

In addition to the prespecified analyses, unplanned Bayesian analyses were performed. The higher than anticipated attrition rate underpowered the planned null hypothesis tests, and the Bayesian analyses were included to calculate the probability that the intervention had an effect on smoking outcomes. Using a Bayesian approach also removes the emphasis on null hypothesis testing and prespecified significance thresholds, which can be sensitive to single data points and can create counterintuitive results [33-38]. Normal priors with mean 0 and SD 1 were used for all regression coefficients to encode an a priori skepticism of effect. Inference was carried out using the Hamiltonian Monte Carlo procedure (50,000 samples, using the first half as burn-in).

All analyses were performed in R (version 3.6.1); Bayesian inference was performed using the probabilistic programming language Stan (RStan version 2.19.1).

Results

Participant Characteristics

The CONSORT flow diagram in Figure 1 describes the progress of the two groups in the NEXit Junior trial, and Table 1 presents the baseline characteristics of the two groups. A total of 621 high school students were screened for eligibility between January 10, 2018, and January 10, 2019. Among these, 86 students were excluded due to either reporting that they did not smoke on a daily or weekly basis or that they did not want to make a quit attempt, resulting in 535 randomized students. Of those randomized, 276 students were allocated to the intervention group and 259 were allocated to the control group.

‎View this figure

Primary Analysis

Complete case analysis was possible on 57.9% (310/535) of participants at 6 months and on 77.2% (413/535) of participants at 3 months. Sample estimates of prolonged abstinence and smoking cessation (primary outcomes), along with ORs, 95% CIs, and P values, are presented in Table 2. Results are presented for both 3- and 6-month outcomes for completeness.

The OR for 4-week point prevalence of smoking cessation, which was statistically significant at the 3-month follow-up, was no longer distinguishable from 1 at the 6-month follow-up at the conventional P<.05 threshold; thus, a null finding could not be ruled out. None of the other ORs was statistically significant.

Sensitivity Analysis

We explored potential deviations from the MCAR assumption underlying the primary analyses in Table 1. Penalized logistic regression (lasso) did not reveal any baseline characteristics that were predictive of nonresponse to 6-month outcomes, offering no evidence against the MCAR assumption.

The analyses of 6-month primary outcomes with imputed data found similar effect estimates of the intervention as the complete case analysis for 5-month prolonged abstinence (OR 1.34, 95% CI 0.81-2.20, P=.25) and 4-week point prevalence of smoking cessation (OR 1.51, 95% CI 0.93-2.46, P=.09). This suggests that the findings in Table 1 are robust to the missing data. Despite no evidence against the MCAR assumption, interpretation of the findings from the imputed values should be made with caution as the rate of attrition (42.1%) was higher than is generally advised when using multiple imputation [32].

Bayesian Analysis

Figure 2 shows histograms of the samples drawn during the Hamiltonian Monte Carlo simulations. These histograms are representative of the marginal posterior probability of the ORs contrasting the intervention and control groups. The histograms indicate which ORs are more likely than others with respect to the number of samples above or below a specific value. For instance, in the top right histogram, a strong majority of samples lie above 1, indicating a high probability that the intervention had an effect on the 4-week point prevalence of smoking cessation at the 3-month follow-up.

‎View this figure

Table 3 presents the probabilities for three different OR thresholds (1, 1.25, and 1.5) for the primary outcomes. The probability that the text message intervention had an effect on prolonged abstinence (ie, OR>1) was 73.8% at 3 months and was 76.9% at 6 months. The probability that the intervention had an effect on 4-week smoking cessation was 98.4% at 3 months and was 87.5% at 6 months. In addition, the probability that the OR was greater than 1.25 for 4-week smoking cessation at 3 months was 89.6% and was 61.7% at 6 months.

Discussion

Principal Findings

The primary analyses indicated that null findings could not be ruled out when estimating the effect of a 12-week text message intervention on prolonged abstinence and point prevalence of smoking cessation among high school students 6 months after randomization. Sensitivity analyses where missing data were imputed indicated that these findings were robust. Unplanned Bayesian analyses of both 3- and 6-month outcomes indicated that the effects of the intervention were in the anticipated direction.

Among the several studies of smoking cessation text message interventions to a wide range of populations carried out over the past decade, a follow-up period of 6 months is not uncommon. A meta-analysis from 2015 summarized the evidence of nine studies at the 6-month follow-up [8], finding an overall standardized OR of 1.35 (95% CI 1.18-1.49), suggesting that there was a small detectable effect. This is in contrast to our findings, which may be severely affected by attrition and targeted a less explored population; however, this also contrasts with the findings of a large pragmatic randomized trial that did not suffer from severe attrition [39]. The pragmatic trial was performed among both treatment and nontreatment seekers in Australia, with a total of 3550 participants being recruited with high retention (86.5% for the 6-month follow-up); nevertheless, no significant effect of a text message program was found on its own or in combination with a tailored online program. Therefore, although the synthesized evidence does suggest that there are persistent effects, there are certainly some questions remaining regarding the size of these effects and their overall generalizability on health for both subpopulations and the general population.

As mentioned in the Introduction, few trials of smoking cessation text messaging interventions have been conducted targeting adolescent populations exclusively [9-11]. The findings have suggested no marked influence of text messages on smoking abstinence, which is corroborated by our findings in this trial. However, our Bayesian analyses do suggest that effect estimates are in the anticipated direction, even after having taken a skeptical a priori view on the magnitude of the effect. Moreover, although compliance is hard to measure when investigating text messaging interventions, as it is not possible to know with any certainty the degree to which participants read and comply with the messages they receive, during this trial, 21% (n=114) of participants decided that they no longer wished to receive any more messages. This number is difficult to interpret without further data, as some may have been displeased about the support received, some may have just been curious about the trial, while others may have quit smoking and not needed any more support. In any case, we believe that retaining 79% of participants to the end of a 12-week intervention should be considered a success in this population; thus, it is feasible to deliver a smoking cessation intervention via text messages to high school students. The question remains if such an intervention can be designed to also be effective in the long term. Future research should therefore carefully consider if text messaging interventions to this population are worth pursuing. If so, factorial trial designs should be employed to assess multiple components simultaneously [40,41], and Bayesian group sequential designs should be used to inform both successful and futile experiments early on [42-44].

Smoking is considered to be the second leading risk factor for disability adjusted life years globally [45]; thus, effective ways to help individuals quit smoking permanently are needed. Synthesized evidence of smoking cessation interventions targeting adolescents suggest that group counseling may be effective in the long term (risk ratio 1.35, 95% CI 1.03-1.77), whereas individual counseling, pharmacological interventions, and digital interventions did not show long-term effectiveness [4]. However, the quality of the body of evidence is low, as there are several concerns regarding risk of bias and heterogeneity. Although an intervention that encourages smoking cessation and promotes abstinence for a period of time may be considered helpful, we should be careful to offer interventions that may take time away from other smoking cessation activities. It is important to develop interventions that utilize digital platforms as an option for young individuals in Sweden, as they are digital natives and nicotine products can only be bought by people 18 years or older. Moreover, text message interventions targeting university students have been found to be more effective among those without a strong nicotine dependence [46], suggesting that acting early by offering text messaging interventions at the high school level may still be an important public health measure.

Generalizability

Recruitment to the trial was performed to closely mimic how the intervention could be disseminated to high schools in a full-scale implementation (ie, through print and digital advertisement managed by each school). There was no additional contact with participants throughout the trial period; thus, the trial closely resembles how the intervention would be used in routine care. In addition, inclusion criteria were broad, excluding only those who smoked on a monthly basis or less. The trial may therefore be seen as measuring effectiveness rather than efficacy, which strengthens the external validity of the findings. However, the limitations of this trial should be taken into account when interpreting the findings and the generalizability of the results, especially the high attrition rates.

Limitations

The most severe limitation of the 6-month findings from the NEXit Junior trial was the risk of bias induced by the high attrition rate (42.1%). Although we found no evidence of systematic missingness, and the findings were robust under imputation, high attrition increases the probability that unobserved factors may predict nonresponse, and thereby bias both the primary and sensitivity analyses. This issue has previously been raised as a concern when studying young adults [47], and future trials on this age group need to implement strategies to retain participants, potentially through incentives.

A second limitation was the lack of blinding of trial participants, which may induce performance bias, an issue prevalent in trials of health interventions as participants are often aware that they signed up for a trial of a particular intervention. Performance bias may be the source of the measured effects rather than the interventions themselves; thus, future trials should prioritize making study information to participants consonant with allocation to intervention and control groups. Additionally, caution should be taken when assessing the present findings since the outcomes were self-reported. This does not necessarily warrant a concern of bias in and of itself; however, in combination with the nonblinding of participants, the risk of performance bias is exacerbated.

Finally, although all processes were automated and participants were treated equally within trial groups, there was a risk of detection bias due to calling nonresponders to collect follow-up data. Despite the fact that the research personnel responsible for the telephone interviews were experienced and instructed not to prompt participants to disclose allocation, some participants are expected to reveal the group to which they were randomized to, and thereby create a risk of bias. However, we judged this risk to be inferior to the increased risk of attrition bias, which would have been induced by not collecting outcome data from those not responding to initial attempts. Furthermore, the analyses revealed no differences in baseline or outcome data between early and late responders, suggesting that the risk of disclosure bias did not lead to any actual bias [12].

These limitations may all influence effect estimates by introducing bias. One way to address these sources of bias post hoc is to consider accounting for them in analyses using causal models [48]. Toward this end, future research should consider estimating the magnitude of effects induced by these sources of bias so that unbiased effects of interventions may be estimated.

Conclusions

Higher than expected attrition rates to follow-up 6 months after randomization led to null hypothesis tests being underpowered. Although null findings could not be ruled out, unplanned Bayesian analyses suggested that the effects of the intervention were in the anticipated direction both at the 3- and 6-month follow-ups. Future trials of smoking cessation interventions targeting high school students should aim to blind participants to avoid risk of performance bias, and also prepare strategies for increasing retention to mid- and long-term follow-up.

When making policy decisions to implement text messaging interventions for smoking cessation in high schools, consideration should be taken to other alternatives with stronger evidence for long-term effects. However, such decisions should also consider that text messaging interventions are relatively cheap to implement and potentially have great reach among those who may benefit. Moreover, although the effects past the 3-month follow-up period have been difficult to establish due to study limitations, the evidence put forth herein suggests that the intervention was more likely than not to have a positive effect, even in the long run. Considering the health benefits that could come from quitting smoking at such an early age, text messaging interventions could be an important part of a public health campaign in this population.