Stepwise Development of a Text Messaging-Based Bullying Prevention Program for Middle School Students (BullyDown)

Introduction

Background

Bullying is a significant adolescent health issue: depending on the type of bullying, an estimated 9% to 38% of middle school students bully their peers sometimes or more often each semester [1]. Defined as intentional peer aggression that occurs repetitively, over time, between at least two people for whom differential power exists [2], bullying is associated with negative psychosocial correlates, including externalizing problems (eg, alcohol use) for bullies and internalizing behaviors (eg, depressive symptomatology) [3-5], and suicidal ideation and attempts for victims [5,6]. Emerging research suggests that bullying also has a negative physiological impact: victims have greater changes in C-reactive protein as they age into adulthood [7]. Research also suggests dysregulation of the hypothalamic-pituitary-adrenal axis among adults who were victims of bullying, resulting in memory deficits [8]. Recent work further notes shorter telomeres, which implicates a shorter life expectancy, among those exposed to violence (eg, bullying victimization) as children compared with nonexposed youth [9].

The Current State of Bullying Prevention Programs

Existing bullying prevention programs seem to be having a modest impact. [10-16]. Existing approaches, which rely on teachers to deliver proscriptive content, may cause youth to dismiss the messages as a way of rejecting authority and exerting control over their social selves [16]. Programs that are able to remove the teacher from the content delivery may be able to overcome this challenge [17].

The Potential for Technology to Affect Behavior Change

Many are exploring the potential for digital technology to enhance the educational experience and to promote prosocial behaviors [18,19]. The wide adoption of cell phones provides novel opportunities to “go where youth are.” Indeed, 88% of 12- to 17-year-old youth have access to a cell phone. Ninety percent of these youth use text messaging (short message service, SMS) interventions [20], which is the preferred mode of communication between peers [21]. Furthermore, text messaging is cost-effective: compared with the high personnel and infrastructure costs of in-person interventions, text messaging-based programs (ie, mHealth) are scalable and cost less than US$.02 per message to send and receive. Moreover, emerging evidence supports the efficacy of these programs [22-26].

The Theory of Social-Emotional-Learning–Based Bullying Prevention Programs

Research suggests that effective behavior change programs are guided by strong theoretical models[27]. Social-Emotional- Learning (SEL)-based programs involve “the systematic development of a core set of social and emotional skills that help children more effectively handle life challenges and thrive in both their learning and their social environments” [28]. The model has emerged from influences across different movements that focused on resiliency and teaching social and emotional competencies to children and adolescents [29]. SEL is based on many well-established theories, including theories of emotional intelligence, social and emotional competence promotion, social developmental model, social information processing, and self-management [30]. It also integrates important aspects of several other behavior change models, including the health belief model, the theory of reasoned action, problem behavior theory, and social-cognitive theory [27,31].

SEL-based programs use social skills instruction to address behavior, discipline, safety, and academics in order to help youth become more self-aware, manage their emotions, build social skills (eg, empathy, perspective-taking, respect for diversity), build friendship skills, and decrease engagement in delinquent behavior [32-34]. SEL-based bullying prevention programs [34,35] target risk and protective factors that have consistently been associated with bullying and victimization in cross-sectional and longitudinal studies [36-43]. For example, empathy has been reliably found to be negatively associated with aggression and positively associated with prosocial skills [44-46]. The inverse correlation between aggression and empathy was found to be stronger in studies that focused on the emotional component of empathy than in studies in which cognitive empathy was measured [47]. Endresen and Olweus [47] conducted a study that specifically explored the association between empathy and bullying and brought attention to attitudes toward bullying, which have demonstrated positive correlations with bullying behavior [48]. Anger and hostility routinely emerge as important correlates of bullying. In several studies of bullying behavior, anger was the strongest predictor of bullying both cross-sectionally and longitudinally [48,49]. Impulsivity also plays an important role in bullying perpetration and victimization. In a prospective study of factors that predicted bullying over a 4-month period, Espelage et al [50] found a significant association between impulsivity and bullying behavior among a sample of 214 6th grade students. In addition, bullying prevention efforts have increasingly focused on the importance of bystander intervention, with significant results [37,51].

A Description of BullyDown, a mHealth Bullying Prevention Program

SEL-based programs have reported positive results in terms of reducing bullying and other disruptive behaviors in middle school [34,52,53]. As such, we used the SEL model to guide the development of BullyDown, a mHealth bullying prevention program designed for middle school students (Figure 1).

According to the SEL model, interactivity increases participant engagement, and new behaviors need to be practiced to be integrated into one’s behavioral script. BullyDown therefore includes several features aimed at engaging youth, while also encouraging them to practice the skills discussed in the content. Principal among these is the “Text Buddy” feature, which pairs intervention participants as “buddies,” allowing them to discuss what they learn through the program with each other. Text Buddy has been used successfully in other text messaging–based programs and is associated with behavior change [54-56]. As such, we posit that the Text Buddies feature will provide an opportunity for youth to process the program information and to practice their new skills in a safe environment. To further integrate interactivity and also ensure that youth understand the program information, youth are given the opportunity to “level up” (ie, move to the next level) at the end of each week by answering a quiz question correctly.

A third interactive feature tested for acceptability is Happy Genie, which sends intervention participants on-demand messages that provide positive encouragement. This type of feature is specific to the target audience and the topic, but is based upon other behavior change programs that have also used on-demand features [54-56].

‎View this figure

Current Paper

Similar to iterative methods used in other mHealth programming [57,58], Ybarra and colleagues [56,59,60] have described a stepwise approach to developing and testing health behavior change content delivered via text messaging. Here, we describe the procedures and experiences encountered while developing BullyDown. Findings can inform the future design of similar programs for middle school youth, as well as those aimed at reducing bullying and other aggressive behaviors.

Methods

The Chesapeake institutional review board reviewed and approved the research protocol. We sequentially implemented five activities: (1) online focus groups (FGs), (2) ongoing content development, (3) a Content Advisory Team (CAT), and (4) an internal team test, followed by (5) a beta test. We drafted content after conducting the FGs and continued refining content during the subsequent research activities. Research materials described herein are available via the Internet [61].

The development work spanned approximately 1 year: March 2014 to May 2015. For all research activities, study eligibility criteria matched those of the intended users of the intervention. Participants were enrolled in 7th or 8th grade (of exception, 6th graders took part in the FGs), owned a cell phone, were enrolled in an unlimited text messaging plan, intended to keep the same phone number for the next 6 months, had parental permission, and provided informed assent.

Focus Groups Methods

The FGs aimed to: better understand how young people experience bullying and their prior exposure to bullying prevention programs in school, to obtain the “voice” of the target

population, and to query process issues. To gather feedback from a diverse group of youth nationally, our initial recruitment strategy relied on Facebook advertising targeted to US youth aged 13- to 14-years old (Figure 2).

Upon yielding a low response rate to the Facebook advertisements, we made subsequent changes to the recruitment strategy: increasing the Facebook ad budget from US$50 to US$100 per day, reducing the number of items on the Web-based screener to lower the completion time, and adding the company logo and link to the company website in the screener to increase the website legitimacy. We also contacted partner youth organizations who agreed to advertise for the research activity. When these attempts did not noticeably increase the number of eligible screeners, we contracted a research recruitment firm.

The research team developed a script of questions to guide the FG discussions. Example questions included:

When you think of the word “bullying,” what do you think of? How (if at all) is this different from what you think of when you think about “aggression?” What about being angry? Or getting into fights with friends?

Has someone asked you to think about what it might feel like to be bullied? What are things that you do that help you understand how others might be feeling? Like how it might feel to be bullied or to bully someone?

Additionally, the FGs queried process issues, including the preferred number and timing of daily text messages and feedback on possible intervention names.

Two FGs of 20 participants each were conducted. The script was written to span 3 days of questions. We posited that youths’ experiences and ability to articulate them would vary by grade level. As such, this was our stratification variable: FG1 was conducted with 6th and 7th graders, and FG2 with 7th and 8th graders. Participants logged into the bulletin board–style FG discussion on a password-protected site two to three times each day when and where it was convenient for them. Because of its asynchronous format, participants from across a wide geographic area could be included. Participants received a US$50 Amazon gift card as an incentive for complete participation during all 3 days.

‎View this figure

‎View this figure

Content Advisory Team (CAT) Methods

We next convened a CAT to review and gather feedback about the saliency and understandability of the program content. The CAT also explored the acceptability of the three proposed program features: (1) Text Buddy, (2) Happy Genie, and (3) level-up questions.

To gather feedback, participants, or parents of participants when only the parent email address was available, received a Word document of the draft program text messages. The instructions asked youth to review and comment on each message within each week’s module (eg, Communication). Probes were included to help give direction about what they could consider when reading the proposed content (eg, Are you able to read it and understand it easily?). Instructions also emphasized that the messages were still being developed and also necessarily short given the character limit of text messages. However, during this stage, the proposed text messages were longer than 160 characters (e.g., 180 characters), as the aim was to first identify appropriate tone and content.

Next, participants took part in a 2-day online discussion to share feedback as a group. For each weekly topic, moderators asked youth to identify their favorite, least favorite, most helpful, and least helpful messages; any messages that felt too long or short; and what they learned from the module. Moderators also asked participants about the proposed program features and to suggest three of their own Happy Genie messages. Participants earned up to a US$50 amount in an Amazon gift card, US$30 for their individual review, and US$20 for their participation in the online discussion.

Beta Test Methods

Findings were integrated into the program content, including feedback about the level-up questions and Happy Genie features. For example, to combat the judgment that bullies are “bad” (and therefore, implicitly, neither you nor your friends could be bullies) that emerged during the formative work, the level-up question during week 5 was:

Good morning and welcome to Week 5! Let's get you to Level 4. True or false: Bullies are mean to others because they are bad people. Text me back your answer.

Those who responded correctly received the following reinforcing message:

Bullies are mean for lots of reasons but that doesn't mean they are bad people. Maybe they just need help learning how to share their feelings. Hello, Level 4!

Those who responded incorrectly received the following response:

Actually, bullies can be mean for lots of reasons. That doesn't mean they are bad people. They might just need some help learning how to share their feelings.

Youth who answered incorrectly were given additional chances to answer a new question correctly before they were automatically moved on to the next level:

Let's try again. For Level 4: True or false: Young people who act “differently” are asking to be bullied.

Correct responses received the following:

Right! We're all different. Kids who act or look differently are just being themselves. They want to have friends just like everyone else. Hello, Level 4!

Incorrect responses resulted in this text:

We're all different. Teens who act or look differently are just being themselves. They want to have friends just like everyone else. Onward to Level 4!

Following a successful internal team test that confirmed the program software performed as intended and that the content was readable on cell phone screens, we conducted a beta test to confirm the feasibility of the protocol and technology in a school setting. To facilitate resolution of technology problems if needed, the beta test was conducted with participants from a low-income middle school in Illinois. Research staff went to the school in early December 2014 to screen 7th and 8th grade students from four preselected classrooms. Staff issued parental permission forms to eligible students to return in 1 week. During subsequent visits, research staff picked up the signed permission forms and facilitated enrollment (ie, obtaining informed assent from the students, helping the students complete the baseline Web-based survey in the school computer room, and verifying the students’ phones for compatibility with the software program).

Participants were randomized to either the 7.5-week intervention or control group at a 2:1 ratio. This maximized the amount of information gathered about the intervention, while also gathering sufficient data about the control group experience to determine its feasibility and acceptability. To avoid school hours, participants received one program message in the morning (7:15 AM) and the remainder of messages after school hours (between 4:00 and 9:00 PM). Intervention participants received three to six messages per day for a total of 214 messages, with an additional 15 to 29 messages from the level-up feature; were randomly assigned a Text Buddy; and had access to Happy Genie. Control participants received two messages weekly, one message encouraging thoughtful behavior (eg, A message from BullyDown: “Try doing one nice thing for someone today”) and the other thanking them for their participation, for a total of 14 messages. Control group participants did not have access to the Text Buddy and Happy Genie components.

In addition to the Web-based baseline survey, all participants were asked to complete a brief survey about their program experience, conducted via text message, every other week. We also conducted a Web-based follow-up survey 1 month post-intervention: youth received a link to the survey and two subsequent reminders via text message to facilitate self-completion. Those who did not complete the follow-up survey on their own could choose to complete the survey at school with research staff.

Participants received a US$10 cash incentive for returning the parent permission form, irrespective of whether they received permission. Participants received a US$25 Amazon gift card upon completion of the 1 month post-intervention follow-up survey.

Results

Demographic characteristics of participants in the FGs and CAT, which were enrolled nationally; and the beta test, which was fielded in Illinois, are shown in Table 1. Bullying was somewhat common. For example, 23% (5/24) of beta test participants reported bullying someone via text messaging in the past 30 days.

Focus Group Results

Recruitment and Enrollment

FG activities occurred between March and May 2014. Although the Facebook ads generated a high number of impressions (343,148) and unique clicks (2334) after 10 days of advertising, only 13 screeners were received. Of these, four were from eligible youth. Reasons for ineligibility included not owning a cell phone (7/13, 54%), not being enrolled in an unlimited text messaging plan (1/13, 8%), and being too young (1/13, 8%).

The recruitment firm identified 18 participants to take part in FG1 and 19 participants in FG2. Two participants from FG1 and one from FG2 assented and enrolled, but neither completed the FG nor provided a reason for their nonparticipation.

Process Issues

In total, the FG moderators posted 23 threads, each containing between one and five questions. As predicted, participant age was an important process variable: 6th grade youth had difficulty responding to questions. Homeschooled youth did as well.

For example, in response to familiarity with bullying prevention programming at school, a 12-year-old male said: “Im homeshooled so we don't have any of that.” (Note here, and throughout, that youth quotes are presented exactly as they were typed by participants.)

Participants overwhelmingly voted for the program name “BullyDown.” As one participant shared: “I like BullyDown.com the best. It states the purpose of the website and it's easy to remember!” (13-year-old female). Youth also expressed a preference for receiving two to four program text messages per day and felt that more would be too many.

Program Content

While some youth reported that their schools had not discussed bullying, most youth were able to articulate and also agreed with school definitions of bullying: “Our school considers any type of picking on or hurting someone's feelings bullying and I feel the same.” (12-year-old male). Overall, youth related to the definition of bullying as verbal, physical, and relational forms of aggression that can happen among friends and nonfriends:

Bullying comes in many forms. Physical, mental and cyber are all kinds of bullying. To me bullying is picking on or hurting someone you know can't defend themselves. A person bullies others to feel better or more important about themselves. Being aggressive towards another person is not the same as bullying. To me aggression is more about anger and the anger one feels towards another.
[13-year-old female]

Youth perceptions of victims varied, and in some cases, youth felt that some victims behaved in ways that contributed to their victimization. Youth also recognized that some students who are socially popular engage in bullying, making it more difficult to stand up to these bullies. Because emotion regulation is a core component of bullying prevention, we also asked FG participants about coping strategies they used when stressed or upset. Answers included: reading/writing, listening to music, counting, belly breathing, and talking to siblings and parents.

Integrating Focus Group Feedback into BullyDown

Based upon the SEL model, we drafted seven modules: (1) communication, (2) coping with stress and problem solving, (3) empathy, perspective-taking, and respect for diversity, (4) recognizing bullying, attitudes toward bullying, and attitudes supportive of aggression, (5) emotion regulation: anger, hostility, and impulsivity, (6) bystanders and intentions to intervene to help others, and (7) resources and support.

We incorporated quotes, as well as personal experiences shared by youth, from the FGs into program messages. For example, youth responses included:

Bullying is something done to someone weaker then you that you know cant defend themselves. When you go behind the other persons back it usally means your scared to say it to there face.
[14-year-old male]

I saw a boy making fun of a handicapped student right in front of a teacher and they did not say anything, so I did and then they stopped it and the boy who was bullying was sent to the office.
[13-year-old female]

These became adapted into the respective program messages:

Bullying can even be spreading a rumor about someone (true or not). It can happen face-to-face or behind your back. It can be online or through text messaging.

Another teen said: A boy was making fun of a handicapped teen in front of a teacher, but she didn't say anything. So I did. The boy was sent to the office.

In response to a question about how bullying might make a person feel, a participant shared:

it makes you feel badly about yourself and that feeling lasts for a long time. It doesn't go away quickly, even if others try to tell you it's not true. Deep inside you feel like it is true what they are saying. It stays with you and keeps coming back and being in your mind.
[12-year-old female]

This was translated into the following program message:

A girl who was bullied told me: It makes you feel badly about yourself and that feeling lasts for a long time. It doesn't go away quickly……even if others try to tell you it is not true. Deep inside you feel like what they’re saying is true. It stays with you and keeps coming back into your mind.

Content Advisory Team Results

Recruitment and Retention

We conducted the CAT between August and September 2014. Given the success using a recruitment firm during the FGs, we determined this strategy to be the most efficient for subsequent research activities. We excluded 6th graders and homeschooled youth in these subsequent activities because of their difficulty responding to questions during the FGs. The recruitment firm recruited 12 youth for the CAT (Table 1). Fifty percent (6/12) of participants completed both the individual text message review and the online discussion. Twenty-five percent (3/12) of participants completed only the individual review. Of these three nonparticipants, one parent withdrew their child from the study due to illness and lack of time to complete the CAT. Another parent requested a Web-based version of the survey, which was provided although feedback was not received. The reason for the third youth’s nonparticipation is unknown.

Engagement with the CAT was low in part because of participants’ confusion about the protocol. Both the recruitment firm and the research staff received multiple requests from parents for clarification about the process. This confusion seemed to persist even after research staff explained the process to parents over the phone. Following up with participants and/or their parents was also difficult because messages from research staff often went unanswered. In many cases, the recruitment firm needed to contact parents and/or participants on behalf of research staff to resolve the issue. Response rates may also have been low because the CAT field period overlapped with the start of a new school year. After yielding little feedback, we extended the 7-day deadline by a week. To help invigorate completion, research staff provided participants with examples of the type of feedback that would be informative to program development. Interestingly, two participants requested that the messages be made available via the Internet through Google Docs. One did not have access to Word, and the other requested it for anticipated convenience.

Atypical of our past experience with adolescents [62], parents of CAT participants were often the point-of-contact. This is perhaps because the recruitment firm identified youth through their parents. As a result, we often lacked contact information for the youth. To avoid this issue in the future, researchers should consider including a study eligibility criterion that requires youth contact information or partnering with a recruitment firm that allows the project staff to talk directly to both parent and child at the outset of the study. Researchers should also be mindful of major transition periods for the target audience (eg, the beginning of a school year) when identifying desired field dates.

Text Message Review

As shown in Table 3, youth feedback to each of the messages were generally positive (eg, “Great job describing it”) and, in some cases, elicited personalized responses (eg, “I hate being accused of being bully”). Although some program messages received more negative feedback than others (eg, the message for recognizing bullying; emotion regulation: impulsivity; a quote from cartoon character BMO), at least one-half of the participants positively endorsed most messages. A participant also made a comment in the “emotion regulation: anger, hostility” section suggesting that some of the daily transitions between topics needed improvement.

Online Discussion

During the 2-day online group discussion, participants generally rated the program modules positively: between 80% (4/5) and 100% (5/5 or 4/4, depending on the number of youth who responded to the poll) of youth liked/strongly liked them. As an exception, 75% (3/4) of youth disliked the problem-solving module. Only one youth provided details on his negative reaction to the module:

I liked the how to deal with stress messages, the tips on what to do. The examples about starting rumors was nothing I would ever do, so did not like that.
[12-year-old male]

When asked about their overall impression of the program, many participants responded positively:

the program is perfect no changes needed.
[13-year-old male]

Less enthusiastic feedback was also received:

some of the messages were too long and boring and some of them were kinda confusing
[12-year-old female]

it was jumping from topic to topic.
[13-year-old female]

Four youth had negative reactions to the tone of some messages. For example, one youth shared:

[the messages] sounded like an adult trying to be ‘cool’ - the messages were too long and wordy and sounded like things my mom would say.
[12-year-old male]

When asked for further detail, youth specifically disliked the use of the phrase “rock star” in one of the messages. Although many participants liked the inclusion of quotes from other teens, one youth questioned the authenticity of the quotes (ie, emotion regulation: anger, hostility message; Table 3).

The level-up feature was well received, although it too was not immune to criticism. For example, some thought that while the feature was acceptable, the questions needed to be more challenging:

the questions to get to the next level were way too easy. the second try was even more easy. you should probably make it a little harder.
[12-year-old male]

Another participant thought that the questions felt like homework:

… I don't like having to 'earn' levels - it makes it more like a school assignment, not something that should be have to be done. why do you have to pass a level to move on? is there a prize or something to get through the levels. If I was doing this in real life and did not pass a level, I would just stop and not continue anymore.
[12-year-old male]

Regarding program features, participants were enthusiastic about Happy Genie:

This genie feature looks amazing this is just what is needed when their life feels like a bore and nothing is great about it. The feature that it sends them a positive quote is very nice indeed. Overall great feature really perked up my spirits with those quotes i am sure it will do just the same for the participants.
[13-year-old male]

This sentiment was echoed by a 12-year-old female:

I think this feature is great! It could be really helpful for a person whose having a bad day.

Youth also suggested several name alternatives, including Forever Friend, Good Day Genie, and the Happy Doctor.

Integrating Content Advisory Team Feedback Into BullyDown

Overlapping feedback provided by two or more youth were integrated into the program content where possible. For example, we deleted the word “rock star” and messages deemed particularly confusing (eg, the quote from BMO). We also modified the tone of messages to sound less motherly or adult-like. The message:

You can tell your friend how you feel. It might be hard. And it might not go well. But if you’re in a healthy friendship and you use assertive communication, it could go great!

Subsequently became:

Tell your friend how you feel. It might be hard, and it might not go well. But if you’re in a healthy friendship, and you talk assertively, it might go great!

We also reviewed the messages scheduled for the beginning and end of each program day for their flow and transitions, based on youth feedback.

We chose not to make the level-up questions more challenging as this seemed likely to make the feature further simulate homework. The level-up protocol was already designed to automatically move nonresponsive participants to the next module if they did not respond to the questions after a certain period. We established a plan to closely monitor participant reactions to the level-up feature during the beta test for further indication of acceptability.

We deemed participants’ positive review of Happy Genie as supportive of its acceptability. We renamed the feature “Forever Friend” based on the names suggested by participants and added inspirational messages (eg, A new day = a new beginning) and quotes from famous people (eg, “I’ve failed over and over again in my life. And that is why I succeed.” – Michael Jordan).

Beta Test Results

Recruitment and Enrollment

Enrollment spanned 3 months and required research staff to visit the partnered middle school four times. Research staff screened 78 youth, 45% (35/78) of whom were eligible. Reasons for ineligibility included: not having a cell phone (21/78, 27%), not or uncertain if they were enrolled in an unlimited text messaging plan (9/78, 12%), and not planning to or uncertain if they would have the same phone number for at least 6 months (21/78, 27%), including the possibility that parents may take away phone privileges as a form of punishment.

Thirty-three eligible youth returned their permission forms, all of whom had parental permission to participate. Twenty-four eligible students provided written assent to participate, of which 22 completed the registration process and were randomized. Of the nine youth who had parental permission but did not enroll, three changed their mind, two moved, one was absent every time the research staff went to the school, and three had problems with phone access (eg, one shared the phone with a parent). Reasons for not completing the registration process included no longer being interested (n=1) and no longer having a cell phone (n=1). Participants were successfully randomized, resulting in 14 students assigned to BullyDown and eight to the control group (Table 1).

Process Issues

We conducted the beta test between December 2014 and May 2015. No concerns from the school administration or staff were expressed during the field period. Participants encountered some technology difficulties, however. The school’s firewall initially blocked the project enrollment website, but school technology staff quickly resolved the issue. Additionally, many students forgot to bring their phones to verify its compatibility with the program software. These students completed the Web-based baseline survey and were instructed to complete enrollment at home by responding with the word “verify” to the text message verification. Those randomized to the intervention group received a link to the Text Buddy Code of Conduct to read and accept. Some students had trouble completing the tasks, requiring research staff to return to the school to assist them in person.

Thirty-six percent (8/22) of participants completed the Web-based follow-up survey on their own time, and 14 completed it at school. The primary reason for not completing the survey independently was forgetting to do it. One participant thought the survey link did not look legitimate and so was hesitant to click on the link.

Program Acceptability and Feasibility

During the program delivery period, 86% (12/14) of intervention participants sent at least one message to their Text Buddy (range, 2-52), and 29% (4/14) sent at least one message to Happy Genie (range, 1-8). Half of intervention participants (7/14) responded to all seven weekly level-up questions, and 21% (3/14) responded to two or fewer level-up questions.

Feedback from the biweekly brief survey (Table 4) suggested that many intervention participants thought the messages were interesting and fun, although some felt the messages were boring (eg, week 2 messages). Seventy-five percent (9/12) of intervention youth were able to provide an example of a message they found memorable (week 4), and 75% (9/12) of youth provided a specific example of something they liked about the program (week 6).

Moreover, all participants said they would recommend the program to a friend when asked at the end of week 6 (Table 5). Opportunities for improvement were also noted: although 80% (8/10) of intervention participants agreed that reading the messages the same day they were received was easy (Table 5), five youth shared concerns about the message quantity or timing when asked at the end of week 6 about something they may not like about the program (Table 4).

Feedback from intervention participants suggested that they were enthusiastic about having a Text Buddy. This feature also drew at least one negative response each week, however (Table 4). Subsequent follow-up with youth revealed that their discomfort centered on being paired with someone from the same school and having to talk about bullying with a schoolmate: “…A Lot of kids in this area kno each other and don't want to talk about things like this.” (12-year-old male). Some youth also expressed frustration that their buddies were unresponsive to their messages; or felt discomfort about being paired with someone of the other sex or in a different grade.

As shown in Table 5, both the intervention and control group content appeared to be acceptable: more than four in five youth in each arm agreed that they liked the program. With 88% (7/8) of control participants saying that the program helped them not bully others in the future, this arm, although of lesser intensity than the intervention arm, appeared to blind participants to their arm assignment. In addition to high scores for understandability and salience of program content, the experience also seemed to be feasible: most youth disagreed or strongly disagreed that the program sent too many messages, and none of the beta test participants in either arm agreed that they stopped reading the messages by the end of the program.

The enrollment process seemed to require in-person facilitation by research staff at a level that is at least commensurate with an in-person intervention. Both interesting and potentially problematic, 33% (11/33) of youth who were eligible and had parental permission lost interest in participating before enrollment. Given the small sample size, this may be an anomaly. Further investigation of this feasibility issue in a larger trial is needed before conclusions can be drawn. Also, some participants shared that they would have been more likely to respond to the follow-up survey if the link had been sent by email instead of text message. Future efforts should consider collecting multiple forms of contact (eg, Facebook, email address).

Despite CAT participants’ expressed interest in the concept of Text Buddy, most beta test participants only sent one or two messages to their buddy. Based upon their feedback, future trials could ideally match Text Buddies who are attending different schools and, when possible, of the same sex and grade level. An interactive guide, either in-person or via the Internet, instructing how to use the feature could also be incorporated into the content. Additionally, only participants with expressed interest in the Text Buddy feature could be matched to reduce the likelihood of unresponsive buddies.

Several intervention youth expressed concern about the intensity and/or timing of the daily messages during the field period but not when it was assessed at 1-month follow-up. This feedback most likely arose on days that had five or six messages, resulting in participants receiving four or five messages across a 4-hour window after school. To ameliorate this, the total number of messages per day in future iterations of the program will be limited to four where possible.

Consistent with feedback from some CAT participants who thought the level-up questions were too easy, one beta test participant thought that the questions should be more meaningful. Given the lack of negative feedback from other beta test participants about the level-up experience and the desire to avoid simulating homework however, more feedback from randomized controlled trial participants will be solicited before making significant changes.

Discussion

Principle Findings

Stepwise development of BullyDown over a 1-year period helped ensure the program was generally well received by middle school students. Findings reveal important insights about developing mHealth interventions for younger youth, particularly those focused on bullying prevention, including the preferred number of messages per day (ie, 2-4 messages), a desire for shorter messages, and the ideal age range (ie, 7th grade and older). In interventions aimed at older adolescents (e.g. 14- to 17-years old), others have noted a similar preference among participants for a low-intensity experience in interventions [63,64]; however, higher intensity programs (eg, 8-15 messages per day) have also received indications of acceptability [56]. The desired threshold seems likely to be affected not only by the age of the youth but also the topic in question. This variability in participants’ acceptability of various program intensities highlights the importance of iteratively developing new mHealth program using ongoing feedback from the target population.

Also contrasting other mHealth adolescent recruitment efforts [65], Facebook was less efficient in nationally recruiting youth than was a more traditional recruitment method (ie, using a recruitment firm). Initial interest in the ads was commensurate with expectations for a successful campaign, but completion of the Web-based screener seemed to be an issue. Perhaps younger youth are more cautious when providing contact information to unknown people or organizations via the Internet, or perhaps the topic (ie, bullying prevention), when more fully explained, is of little interest to youth. However, the school administrator’s enthusiasm for the beta test suggests that the program is acceptable and feasible for implementation in school-based samples. It also supports a hypothesis that the eventual program will need to be promoted to school administrators rather than youth directly.

Several additional differences between the experiences in the BullyDown beta test with the implementation of other text messaging–based health behavior change programs should be noted. For example, compared with high school-aged youth [56], middle school students' relationship with their phones seems to be less predictable: (1) they do not carry their phones everywhere – at least not to school and not necessarily outside of school, (2) they might not remember their phone number accurately, and (3) their access appears to be more tenuous – parents are more likely to restrict their cell phone use. Also, staff resources needed to enroll youth appear to be commensurate with in-person interventions. This means that cost savings in enrollment will not be realized. Once the intervention program messages begin however, most 7th and 8th graders are able to engage with the program on their own, suggesting that once engaged, a standalone technology intervention has promise. These differences highlight the heterogeneity of mHealth research: simply because a particular protocol works for one program and population does not mean the same protocol will work with other programs and populations.

Although control and intervention participants attended classes side by side, the control arm appears to have been blinded: they were equally likely as the intervention arm to say they liked the program, and a similar percent of youth said they learned valuable things about bullying during the program. This suggests that randomization at the individual-level may not pose a serious threat to contamination; however, results should be replicated in a larger sample size.

Limitations

Specific program recommendations that emerged from this work may not generalize to other health behavior change topics or youth populations. Although the national recruitment strategy is a strength, FG and CAT participants may not be representative of all youth given they were identified through a recruitment firm. Beta test participants were recruited from a disadvantaged public middle school in Illinois and may not be representative of students in other settings. More generally, youth who have a cell phone and are enrolled in an unlimited text messaging plan may be different from youth who do not meet these criteria. It seems unlikely however that youth who lack a cell phone or are not on an unlimited text messaging plan would opt into such a program if it was publicly available. Additionally, confirming that youth actually read the text messages is not possible beyond their self-report; therefore, actual exposure is unknown.

Conclusion

Increasingly, SEL programs are being implemented in schools across the United States to address a wide-range of problematic behaviors (eg, bullying, delinquency) and to promote academic success. However, the majority of these programs include curricula that range from 15 to 20 or more lessons, requiring significant instructional time and resources that some school districts may not be able to provide. BullyDown harnesses the benefits of text messaging, such as being able to go where youth are and communicating important information in consumable chunks, to offer a bullying prevention program that can be delivered outside of school and without the need for facilitator time or resources. While expensive in time and money, iterative intervention refinement increases the likelihood that the resulting intervention is salient to the target population while retaining its adherence to theory.

Given the positive feedback from beta test participants about the program content and the experience as well as the 100% (22/22) retention rate, the BullyDown mHealth bullying prevention program appears to be both feasible and acceptable for implementation in a middle school setting with 7th and 8th grade youth. Our next step will be to test the intervention in a large randomized controlled trial to see if exposure is associated with reduction in bullying behavior.