Published on 13.06.16 in Vol 4, No 2 (2016): Apr-Jun
Stepwise Development of a Text Messaging-Based Bullying Prevention Program for Middle School Students (BullyDown)
Background: Bullying is a significant public health issue among middle school-aged youth. Current prevention programs have only a moderate impact. Cell phone text messaging technology (mHealth) can potentially overcome existing challenges, particularly those that are structural (e.g., limited time that teachers can devote to non-educational topics). To date, the description of the development of empirically-based mHealth-delivered bullying prevention programs are lacking in the literature.
Objective: To describe the development of BullyDown, a text messaging-based bullying prevention program for middle school students, guided by the Social-Emotional Learning model.
Methods: We implemented five activities over a 12-month period: (1) national focus groups (n=37 youth) to gather acceptability of program components; (2) development of content; (3) a national Content Advisory Team (n=9 youth) to confirm content tone; and (4) an internal team test of software functionality followed by a beta test (n=22 youth) to confirm the enrollment protocol and the feasibility and acceptability of the program.
Results: Recruitment experiences suggested that Facebook advertising was less efficient than using a recruitment firm to recruit youth nationally, and recruiting within schools for the pilot test was feasible. Feedback from the Content Advisory Team suggests a preference for 2-4 brief text messages per day. Beta test findings suggest that BullyDown is both feasible and acceptable: 100% of youth completed the follow-up survey, 86% of whom liked the program.
Conclusions: Text messaging appears to be a feasible and acceptable delivery method for bullying prevention programming delivered to middle school students.
JMIR mHealth uHealth 2016;4(2):e60
Bullying is a significant adolescent health issue: depending on the type of bullying, an estimated 9% to 38% of middle school students bully their peers sometimes or more often each semester . Defined as intentional peer aggression that occurs repetitively, over time, between at least two people for whom differential power exists [ ], bullying is associated with negative psychosocial correlates, including externalizing problems (eg, alcohol use) for bullies and internalizing behaviors (eg, depressive symptomatology) [ - ], and suicidal ideation and attempts for victims [ , ]. Emerging research suggests that bullying also has a negative physiological impact: victims have greater changes in C-reactive protein as they age into adulthood [ ]. Research also suggests dysregulation of the hypothalamic-pituitary-adrenal axis among adults who were victims of bullying, resulting in memory deficits [ ]. Recent work further notes shorter telomeres, which implicates a shorter life expectancy, among those exposed to violence (eg, bullying victimization) as children compared with nonexposed youth [ ].
The Current State of Bullying Prevention Programs
Existing bullying prevention programs seem to be having a modest impact. [- ]. Existing approaches, which rely on teachers to deliver proscriptive content, may cause youth to dismiss the messages as a way of rejecting authority and exerting control over their social selves [ ]. Programs that are able to remove the teacher from the content delivery may be able to overcome this challenge [ ].
The Potential for Technology to Affect Behavior Change
Many are exploring the potential for digital technology to enhance the educational experience and to promote prosocial behaviors [, ]. The wide adoption of cell phones provides novel opportunities to “go where youth are.” Indeed, 88% of 12- to 17-year-old youth have access to a cell phone. Ninety percent of these youth use text messaging (short message service, SMS) interventions [ ], which is the preferred mode of communication between peers [ ]. Furthermore, text messaging is cost-effective: compared with the high personnel and infrastructure costs of in-person interventions, text messaging-based programs (ie, mHealth) are scalable and cost less than US$.02 per message to send and receive. Moreover, emerging evidence supports the efficacy of these programs [ - ].
The Theory of Social-Emotional-Learning–Based Bullying Prevention Programs
Research suggests that effective behavior change programs are guided by strong theoretical models. Social-Emotional- Learning (SEL)-based programs involve “the systematic development of a core set of social and emotional skills that help children more effectively handle life challenges and thrive in both their learning and their social environments” [ ]. The model has emerged from influences across different movements that focused on resiliency and teaching social and emotional competencies to children and adolescents [ ]. SEL is based on many well-established theories, including theories of emotional intelligence, social and emotional competence promotion, social developmental model, social information processing, and self-management [ ]. It also integrates important aspects of several other behavior change models, including the health belief model, the theory of reasoned action, problem behavior theory, and social-cognitive theory [ , ].
SEL-based programs use social skills instruction to address behavior, discipline, safety, and academics in order to help youth become more self-aware, manage their emotions, build social skills (eg, empathy, perspective-taking, respect for diversity), build friendship skills, and decrease engagement in delinquent behavior [- ]. SEL-based bullying prevention programs [ , ] target risk and protective factors that have consistently been associated with bullying and victimization in cross-sectional and longitudinal studies [ - ]. For example, empathy has been reliably found to be negatively associated with aggression and positively associated with prosocial skills [ - ]. The inverse correlation between aggression and empathy was found to be stronger in studies that focused on the emotional component of empathy than in studies in which cognitive empathy was measured [ ]. Endresen and Olweus [ ] conducted a study that specifically explored the association between empathy and bullying and brought attention to attitudes toward bullying, which have demonstrated positive correlations with bullying behavior [ ]. Anger and hostility routinely emerge as important correlates of bullying. In several studies of bullying behavior, anger was the strongest predictor of bullying both cross-sectionally and longitudinally [ , ]. Impulsivity also plays an important role in bullying perpetration and victimization. In a prospective study of factors that predicted bullying over a 4-month period, Espelage et al [ ] found a significant association between impulsivity and bullying behavior among a sample of 214 6th grade students. In addition, bullying prevention efforts have increasingly focused on the importance of bystander intervention, with significant results [ , ].
A Description of BullyDown, a mHealth Bullying Prevention Program
SEL-based programs have reported positive results in terms of reducing bullying and other disruptive behaviors in middle school [, , ]. As such, we used the SEL model to guide the development of BullyDown, a mHealth bullying prevention program designed for middle school students ( ).
According to the SEL model, interactivity increases participant engagement, and new behaviors need to be practiced to be integrated into one’s behavioral script. BullyDown therefore includes several features aimed at engaging youth, while also encouraging them to practice the skills discussed in the content. Principal among these is the “Text Buddy” feature, which pairs intervention participants as “buddies,” allowing them to discuss what they learn through the program with each other. Text Buddy has been used successfully in other text messaging–based programs and is associated with behavior change [- ]. As such, we posit that the Text Buddies feature will provide an opportunity for youth to process the program information and to practice their new skills in a safe environment. To further integrate interactivity and also ensure that youth understand the program information, youth are given the opportunity to “level up” (ie, move to the next level) at the end of each week by answering a quiz question correctly.
A third interactive feature tested for acceptability is Happy Genie, which sends intervention participants on-demand messages that provide positive encouragement. This type of feature is specific to the target audience and the topic, but is based upon other behavior change programs that have also used on-demand features [- ].
Similar to iterative methods used in other mHealth programming [, ], Ybarra and colleagues [ , , ] have described a stepwise approach to developing and testing health behavior change content delivered via text messaging. Here, we describe the procedures and experiences encountered while developing BullyDown. Findings can inform the future design of similar programs for middle school youth, as well as those aimed at reducing bullying and other aggressive behaviors.
The Chesapeake institutional review board reviewed and approved the research protocol. We sequentially implemented five activities: (1) online focus groups (FGs), (2) ongoing content development, (3) a Content Advisory Team (CAT), and (4) an internal team test, followed by (5) a beta test. We drafted content after conducting the FGs and continued refining content during the subsequent research activities. Research materials described herein are available via the Internet .
The development work spanned approximately 1 year: March 2014 to May 2015. For all research activities, study eligibility criteria matched those of the intended users of the intervention. Participants were enrolled in 7th or 8th grade (of exception, 6th graders took part in the FGs), owned a cell phone, were enrolled in an unlimited text messaging plan, intended to keep the same phone number for the next 6 months, had parental permission, and provided informed assent.
Focus Groups Methods
The FGs aimed to: better understand how young people experience bullying and their prior exposure to bullying prevention programs in school, to obtain the “voice” of the target
population, and to query process issues. To gather feedback from a diverse group of youth nationally, our initial recruitment strategy relied on Facebook advertising targeted to US youth aged 13- to 14-years old ().
Upon yielding a low response rate to the Facebook advertisements, we made subsequent changes to the recruitment strategy: increasing the Facebook ad budget from US$50 to US$100 per day, reducing the number of items on the Web-based screener to lower the completion time, and adding the company logo and link to the company website in the screener to increase the website legitimacy. We also contacted partner youth organizations who agreed to advertise for the research activity. When these attempts did not noticeably increase the number of eligible screeners, we contracted a research recruitment firm.
The research team developed a script of questions to guide the FG discussions. Example questions included:
When you think of the word “bullying,” what do you think of? How (if at all) is this different from what you think of when you think about “aggression?” What about being angry? Or getting into fights with friends?
Has someone asked you to think about what it might feel like to be bullied? What are things that you do that help you understand how others might be feeling? Like how it might feel to be bullied or to bully someone?
Additionally, the FGs queried process issues, including the preferred number and timing of daily text messages and feedback on possible intervention names.
Two FGs of 20 participants each were conducted. The script was written to span 3 days of questions. We posited that youths’ experiences and ability to articulate them would vary by grade level. As such, this was our stratification variable: FG1 was conducted with 6th and 7th graders, and FG2 with 7th and 8th graders. Participants logged into the bulletin board–style FG discussion on a password-protected site two to three times each day when and where it was convenient for them. Because of its asynchronous format, participants from across a wide geographic area could be included. Participants received a US$50 Amazon gift card as an incentive for complete participation during all 3 days.
Content Advisory Team (CAT) Methods
We next convened a CAT to review and gather feedback about the saliency and understandability of the program content. The CAT also explored the acceptability of the three proposed program features: (1) Text Buddy, (2) Happy Genie, and (3) level-up questions.
To gather feedback, participants, or parents of participants when only the parent email address was available, received a Word document of the draft program text messages. The instructions asked youth to review and comment on each message within each week’s module (eg, Communication). Probes were included to help give direction about what they could consider when reading the proposed content (eg, Are you able to read it and understand it easily?). Instructions also emphasized that the messages were still being developed and also necessarily short given the character limit of text messages. However, during this stage, the proposed text messages were longer than 160 characters (e.g., 180 characters), as the aim was to first identify appropriate tone and content.
Next, participants took part in a 2-day online discussion to share feedback as a group. For each weekly topic, moderators asked youth to identify their favorite, least favorite, most helpful, and least helpful messages; any messages that felt too long or short; and what they learned from the module. Moderators also asked participants about the proposed program features and to suggest three of their own Happy Genie messages. Participants earned up to a US$50 amount in an Amazon gift card, US$30 for their individual review, and US$20 for their participation in the online discussion.
Beta Test Methods
Findings were integrated into the program content, including feedback about the level-up questions and Happy Genie features. For example, to combat the judgment that bullies are “bad” (and therefore, implicitly, neither you nor your friends could be bullies) that emerged during the formative work, the level-up question during week 5 was:
Good morning and welcome to Week 5! Let's get you to Level 4. True or false: Bullies are mean to others because they are bad people. Text me back your answer.
Those who responded correctly received the following reinforcing message:
Bullies are mean for lots of reasons but that doesn't mean they are bad people. Maybe they just need help learning how to share their feelings. Hello, Level 4!
Those who responded incorrectly received the following response:
Actually, bullies can be mean for lots of reasons. That doesn't mean they are bad people. They might just need some help learning how to share their feelings.
Youth who answered incorrectly were given additional chances to answer a new question correctly before they were automatically moved on to the next level:
Let's try again. For Level 4: True or false: Young people who act “differently” are asking to be bullied.
Correct responses received the following:
Right! We're all different. Kids who act or look differently are just being themselves. They want to have friends just like everyone else. Hello, Level 4!
Incorrect responses resulted in this text:
We're all different. Teens who act or look differently are just being themselves. They want to have friends just like everyone else. Onward to Level 4!
Following a successful internal team test that confirmed the program software performed as intended and that the content was readable on cell phone screens, we conducted a beta test to confirm the feasibility of the protocol and technology in a school setting. To facilitate resolution of technology problems if needed, the beta test was conducted with participants from a low-income middle school in Illinois. Research staff went to the school in early December 2014 to screen 7th and 8th grade students from four preselected classrooms. Staff issued parental permission forms to eligible students to return in 1 week. During subsequent visits, research staff picked up the signed permission forms and facilitated enrollment (ie, obtaining informed assent from the students, helping the students complete the baseline Web-based survey in the school computer room, and verifying the students’ phones for compatibility with the software program).
Participants were randomized to either the 7.5-week intervention or control group at a 2:1 ratio. This maximized the amount of information gathered about the intervention, while also gathering sufficient data about the control group experience to determine its feasibility and acceptability. To avoid school hours, participants received one program message in the morning (7:15 AM) and the remainder of messages after school hours (between 4:00 and 9:00 PM). Intervention participants received three to six messages per day for a total of 214 messages, with an additional 15 to 29 messages from the level-up feature; were randomly assigned a Text Buddy; and had access to Happy Genie. Control participants received two messages weekly, one message encouraging thoughtful behavior (eg, A message from BullyDown: “Try doing one nice thing for someone today”) and the other thanking them for their participation, for a total of 14 messages. Control group participants did not have access to the Text Buddy and Happy Genie components.
In addition to the Web-based baseline survey, all participants were asked to complete a brief survey about their program experience, conducted via text message, every other week. We also conducted a Web-based follow-up survey 1 month post-intervention: youth received a link to the survey and two subsequent reminders via text message to facilitate self-completion. Those who did not complete the follow-up survey on their own could choose to complete the survey at school with research staff.
Participants received a US$10 cash incentive for returning the parent permission form, irrespective of whether they received permission. Participants received a US$25 Amazon gift card upon completion of the 1 month post-intervention follow-up survey.
Demographic characteristics of participants in the FGs and CAT, which were enrolled nationally; and the beta test, which was fielded in Illinois, are shown in. Bullying was somewhat common. For example, 23% (5/24) of beta test participants reported bullying someone via text messaging in the past 30 days.
Focus Group Results
Recruitment and Enrollment
FG activities occurred between March and May 2014. Although the Facebook ads generated a high number of impressions (343,148) and unique clicks (2334) after 10 days of advertising, only 13 screeners were received. Of these, four were from eligible youth. Reasons for ineligibility included not owning a cell phone (7/13, 54%), not being enrolled in an unlimited text messaging plan (1/13, 8%), and being too young (1/13, 8%).
The recruitment firm identified 18 participants to take part in FG1 and 19 participants in FG2. Two participants from FG1 and one from FG2 assented and enrolled, but neither completed the FG nor provided a reason for their nonparticipation.
In total, the FG moderators posted 23 threads, each containing between one and five questions. As predicted, participant age was an important process variable: 6th grade youth had difficulty responding to questions. Homeschooled youth did as well.
For example, in response to familiarity with bullying prevention programming at school, a 12-year-old male said: “Im homeshooled so we don't have any of that.” (Note here, and throughout, that youth quotes are presented exactly as they were typed by participants.)
Participants overwhelmingly voted for the program name “BullyDown.” As one participant shared: “I like BullyDown.com the best. It states the purpose of the website and it's easy to remember!” (13-year-old female). Youth also expressed a preference for receiving two to four program text messages per day and felt that more would be too many.
While some youth reported that their schools had not discussed bullying, most youth were able to articulate and also agreed with school definitions of bullying: “Our school considers any type of picking on or hurting someone's feelings bullying and I feel the same.” (12-year-old male). Overall, youth related to the definition of bullying as verbal, physical, and relational forms of aggression that can happen among friends and nonfriends:
Bullying comes in many forms. Physical, mental and cyber are all kinds of bullying. To me bullying is picking on or hurting someone you know can't defend themselves. A person bullies others to feel better or more important about themselves. Being aggressive towards another person is not the same as bullying. To me aggression is more about anger and the anger one feels towards another.
Youth perceptions of victims varied, and in some cases, youth felt that some victims behaved in ways that contributed to their victimization. Youth also recognized that some students who are socially popular engage in bullying, making it more difficult to stand up to these bullies. Because emotion regulation is a core component of bullying prevention, we also asked FG participants about coping strategies they used when stressed or upset. Answers included: reading/writing, listening to music, counting, belly breathing, and talking to siblings and parents.
Integrating Focus Group Feedback into BullyDown
Based upon the SEL model, we drafted seven modules: (1) communication, (2) coping with stress and problem solving, (3) empathy, perspective-taking, and respect for diversity, (4) recognizing bullying, attitudes toward bullying, and attitudes supportive of aggression, (5) emotion regulation: anger, hostility, and impulsivity, (6) bystanders and intentions to intervene to help others, and (7) resources and support.
|Demographic characteristics||Focus groups (n=37) |
|CATa (n=9) |
|Beta testb (n=22) |
|Male||18 (48.7)||6 (66.7)||8 (36.4)|
|Female||19 (51.4)||3 (33.3)||14 (63.6)|
|11 years||1 (2.7)||NA||NA|
|12 years||17 (46.0)||4 (44.4)||10 (45.5)|
|13 years||16 (43.2)||4 (44.4)||8 (36.4)|
|14 years||3 (8.1)||1 (11.1)||4 (18.2)|
|Caucasian||18 (48.7)||6 (66.7)||10 (45.5)|
|Black/African American||10 (27.0)||1 (11.1)||7 (31.8)|
|Asian||1 (2.7)||0 (0.0)||0 (0.0)|
|Mixed racial background||1 (2.7)||0 (0.0)||4 (18.2)|
|Native American or Alaskan Native||0 (0.0)||1 (11.1)||0 (0.0)|
|Hispanic||7 (18.9)||1 (11.1)||4 (18.2)|
|Do not want to answer||0 (0.0)||0 (0.0)||1 (4.56)|
|6th grade||12 (32.4)||NA||NA|
|7th grade||14 (37.8)||4 (44.4)||11 (50.0)|
|8th grade||11 (29.7)||5 (55.6)||11 (50.0)|
|Northeast||9 (24.3)||2 (22.2)||0 (0.0)|
|South||12 (32.4)||3 (33.3)||0 (0.0)|
|Midwest||8 (21.6)||2 (22.2)||22 (100.0)|
|West||8 (21.6)||2 (22.2)||0 (0.0)|
|Type of residence|
|Urban||12 (32.4)||3 (33.3)||Not asked|
|Suburban||15 (40.5)||3 (33.3)||Not asked|
|Rural||10 (27.0)||3 (33.3)||Not asked|
|Been bulliede||10 (27.0)||6 (66.7)||14 (63.6)|
|Bullied someonee||5 (13.5)||2 (22.2)||5 (22.7)|
aContent Advisory Team.
bBeta test participants were enrolled from a partner school in Illinois, as such all participants are from the Midwest region.
cGrade eligibility criterion was modified prior to the CAT to include grades 7 and 8 only.
dHispanic ethnicity was queried as part of race for the focus groups and CAT, and as a separate identity in the beta test. As such, race/ethnicity sums to more than 22 for the latter activity.
eFor the focus groups and CAT, youth were asked if they had ever bullied and if they had ever been bullied. In the beta test, a more complex series of questions were asked, including different ways in which youth could be bullied, the mode through which they could be bullied (eg, in-person, via the Internet), and the place that they could be bullied (eg, at school, on the way to and from school). The timeframe was limited to the past 30 days.
|SEL Component||BullyDown program text message|
|Communication||Passive communication is when you don\'t really share an opinion: “I don\'t care;” “Whatever you want is fine;” and things like that. (Week 1)|
|Communication||It is also passive if you don\'t say anything. If someone teases you and you don\'t say anything, people may think you are ok with it or that it doesn’t upset you. (Week 1)|
|Problem-solving||Step 1 in problem-solving is figuring out what the problem is. While this may sound easy, sometimes it can be harder than we think. (Week 2)|
|Respect for diversity||We\'re all different. Teens who act or look differently are just being themselves. They want to have friends just like everyone else. (Week 5)|
|Recognizing bullying||It can be a fine line between bullying and messing around. So how do you know where the line is? (Week 4)|
|Recognizing bullying||One teen said that it is no longer drama, but becomes bullying when the other person doesn\'t fight back - or they fight back but it doesn\'t work. (Week 4)|
|Attitudes supportive of bullying and aggression||Peer pressure is hard: maybe your friends dare you; or you think your friends will like you better; or your friends are teasing someone and you join in. (Week 4)|
|Bystanders and intentions to intervene||Good news! You don\'t have to do it alone. Let\'s talk about ways you can help kids who are in trouble. (Week 7)|
|Coping with stress||Let’s talk more about what to do when you’re down. Ask for help. Staying quiet doesn\'t make you stronger, often times it just makes you feel alone. (Week 6)|
|Empathy and perspective taking||It can be easy to make bad decisions when we\'re angry. This is really true if we don\'t try to understand what we\'re feeling and why. (Week 4)|
|Emotion regulation: anger, hostility||It\'s okay to get angry - we all get angry sometimes. It\'s what you do with that anger that is good or bad. (Week 4)|
|Emotion regulation: impulsivity||Listening to music, writing about it, or talking to someone can help. Even just sleeping on it can help you see that it is not such a big deal the next day. (Week 4)|
We incorporated quotes, as well as personal experiences shared by youth, from the FGs into program messages. For example, youth responses included:
Bullying is something done to someone weaker then you that you know cant defend themselves. When you go behind the other persons back it usally means your scared to say it to there face.
I saw a boy making fun of a handicapped student right in front of a teacher and they did not say anything, so I did and then they stopped it and the boy who was bullying was sent to the office.
These became adapted into the respective program messages:
Bullying can even be spreading a rumor about someone (true or not). It can happen face-to-face or behind your back. It can be online or through text messaging.
Another teen said: A boy was making fun of a handicapped teen in front of a teacher, but she didn't say anything. So I did. The boy was sent to the office.
In response to a question about how bullying might make a person feel, a participant shared:
it makes you feel badly about yourself and that feeling lasts for a long time. It doesn't go away quickly, even if others try to tell you it's not true. Deep inside you feel like it is true what they are saying. It stays with you and keeps coming back and being in your mind.
This was translated into the following program message:
A girl who was bullied told me: It makes you feel badly about yourself and that feeling lasts for a long time. It doesn't go away quickly……even if others try to tell you it is not true. Deep inside you feel like what they’re saying is true. It stays with you and keeps coming back into your mind.
Content Advisory Team Results
Recruitment and Retention
We conducted the CAT between August and September 2014. Given the success using a recruitment firm during the FGs, we determined this strategy to be the most efficient for subsequent research activities. We excluded 6th graders and homeschooled youth in these subsequent activities because of their difficulty responding to questions during the FGs. The recruitment firm recruited 12 youth for the CAT (). Fifty percent (6/12) of participants completed both the individual text message review and the online discussion. Twenty-five percent (3/12) of participants completed only the individual review. Of these three nonparticipants, one parent withdrew their child from the study due to illness and lack of time to complete the CAT. Another parent requested a Web-based version of the survey, which was provided although feedback was not received. The reason for the third youth’s nonparticipation is unknown.
Engagement with the CAT was low in part because of participants’ confusion about the protocol. Both the recruitment firm and the research staff received multiple requests from parents for clarification about the process. This confusion seemed to persist even after research staff explained the process to parents over the phone. Following up with participants and/or their parents was also difficult because messages from research staff often went unanswered. In many cases, the recruitment firm needed to contact parents and/or participants on behalf of research staff to resolve the issue. Response rates may also have been low because the CAT field period overlapped with the start of a new school year. After yielding little feedback, we extended the 7-day deadline by a week. To help invigorate completion, research staff provided participants with examples of the type of feedback that would be informative to program development. Interestingly, two participants requested that the messages be made available via the Internet through Google Docs. One did not have access to Word, and the other requested it for anticipated convenience.
Atypical of our past experience with adolescents , parents of CAT participants were often the point-of-contact. This is perhaps because the recruitment firm identified youth through their parents. As a result, we often lacked contact information for the youth. To avoid this issue in the future, researchers should consider including a study eligibility criterion that requires youth contact information or partnering with a recruitment firm that allows the project staff to talk directly to both parent and child at the outset of the study. Researchers should also be mindful of major transition periods for the target audience (eg, the beginning of a school year) when identifying desired field dates.
Text Message Review
As shown in, youth feedback to each of the messages were generally positive (eg, “Great job describing it”) and, in some cases, elicited personalized responses (eg, “I hate being accused of being bully”). Although some program messages received more negative feedback than others (eg, the message for recognizing bullying; emotion regulation: impulsivity; a quote from cartoon character BMO), at least one-half of the participants positively endorsed most messages. A participant also made a comment in the “emotion regulation: anger, hostility” section suggesting that some of the daily transitions between topics needed improvement.
During the 2-day online group discussion, participants generally rated the program modules positively: between 80% (4/5) and 100% (5/5 or 4/4, depending on the number of youth who responded to the poll) of youth liked/strongly liked them. As an exception, 75% (3/4) of youth disliked the problem-solving module. Only one youth provided details on his negative reaction to the module:
I liked the how to deal with stress messages, the tips on what to do. The examples about starting rumors was nothing I would ever do, so did not like that.
When asked about their overall impression of the program, many participants responded positively:
the program is perfect no changes needed.
Less enthusiastic feedback was also received:
some of the messages were too long and boring and some of them were kinda confusing
it was jumping from topic to topic.
Four youth had negative reactions to the tone of some messages. For example, one youth shared:
[the messages] sounded like an adult trying to be ‘cool’ - the messages were too long and wordy and sounded like things my mom would say.
When asked for further detail, youth specifically disliked the use of the phrase “rock star” in one of the messages. Although many participants liked the inclusion of quotes from other teens, one youth questioned the authenticity of the quotes (ie, emotion regulation: anger, hostility message;).
The level-up feature was well received, although it too was not immune to criticism. For example, some thought that while the feature was acceptable, the questions needed to be more challenging:
the questions to get to the next level were way too easy. the second try was even more easy. you should probably make it a little harder.
Another participant thought that the questions felt like homework:
… I don't like having to 'earn' levels - it makes it more like a school assignment, not something that should be have to be done. why do you have to pass a level to move on? is there a prize or something to get through the levels. If I was doing this in real life and did not pass a level, I would just stop and not continue anymore.
Regarding program features, participants were enthusiastic about Happy Genie:
This genie feature looks amazing this is just what is needed when their life feels like a bore and nothing is great about it. The feature that it sends them a positive quote is very nice indeed. Overall great feature really perked up my spirits with those quotes i am sure it will do just the same for the participants.
This sentiment was echoed by a 12-year-old female:
I think this feature is great! It could be really helpful for a person whose having a bad day.
Youth also suggested several name alternatives, including Forever Friend, Good Day Genie, and the Happy Doctor.
|Message type||Specific program message||Participant feedbacka|
|12-year-old male||13-year-old female||12-year-old male||12-year-old male|
|Communication||Assertive communication is when you’re clear about how you feel or what you want. Say someone is being mean to your friend. You could say: “I think the way you put him down was mean.”||Ok, good||I like the messages when they describe an example of the type of communication which is what is happening in this one||Great job describing it||Glad itg explains it – but is too wordy|
|Recognizing bullying||A teen told me that it\'s no longer drama, but becomes bullying when the other person doesn\'t fight back - or she fights back but it doesn\'t work.||good||I like this message because as you know, I like hearing from other teens||Way to use examples||Not sure if that is right|
|Attitudes supportive of bullying and aggression||Remember: There are lots of ways to make friends without bullying. If your friends are pressuring you to be mean to others, maybe they\'re not such good friends.||Good message||I like this message because I think that so many people get caught up in who they want to be their friend, when they aren’t such good friends||Put a but before maybe||I stopped being friends with someone because I was tired of them bullying people all the time|
|Coping with stress||There are lots of ways to deal with stress. You could go on a long walk, play basketball, read a book, go for a run or a bike ride, or even write a poem!||Sounds good||I like this message because it has a variety of things to do when your stressed||Add “anything that helps||Good examples – but who would write a poem? Ick!|
|Empathy and perspective taking||Maybe someone has called you a bully but you didn\'t know you hurt the person. This can be frustrating. Take a moment to think about why the other person may have felt bullied.||yes||I like this message because I feel like if someone is accused of bullying, they seem to forget||I hate being accused of being bully||I have never been told I bullied someone|
|Emotion regulation: anger, hostility||Imagine you’re in a fight with your friend. To deal with it, you could: stop talking to her. Shove her to show her you\'re mad. Make up a rumor about her. Or, tell her how you feel.||good||I like this message but it is kind of a weird ending message for the day||They know what the right answer, but that would be awkward to say||Sometimes I would not talk to her but I would never shove her or start a rumor – that’s mean!|
|Emotion regulation: anger, hostility||A teen told me: “I try to think about the situation before acting. If I can remember to do that, it works. But when I\'m angry I usually forget those things.”||ok||I like this message because I like hearing words from other teens||Way to put in a quote||No teen talks like that|
|Emotion regulation: impulsivity||You don’t want to say something you’ll regret. Trying to get even with a bully continues the cycle. And it might put you at risk for something unsafe to happen.||yes||I like this message because it is very clear and is easy to understand||Yea, the bully could bet you up||Not needed – bad message|
|Quote||Great advice: When bad things happen, I know you want to believe they are a joke, but sometimes, life is scary and dark. That is why we must find the light.” - BMO)||agree||I’m kind of confused with this and I’m not quite sure if this is supposed to be a quote or not||What is bmo?||Find the light? Not sure what this means|
|13-year-old male||13-year-old female||12-year-old female||13-year-old male|
|Communication||Assertive communication is when you’re clear about how you feel or what you want. Say someone is being mean to your friend. You could say: “I think the way you put him down was mean.”||Great example on this type of communication.||i dont really use assertive communication||Needs a better example. Definition is clear||Bad|
|Recognizing bullying||A teen told me that it\'s no longer drama, but becomes bullying when the other person doesn\'t fight back - or she fights back but it doesn\'t work.||I don’t think this statement is true because I believe being able to identify it isn’t so simple||i dont agree||Ok|
|Attitudes supportive of bullying and aggression||Remember: There are lots of ways to make friends without bullying. If your friends are pressuring you to be mean to others, maybe they\'re not such good friends.||Another good comment and a way to point out that maybe someone who is pressuring you isn’t good to be around.||i hate it when my friend wants me to be mean. i hate being mean. i get a bad feeling inside||Ok|
|Coping with stress||There are lots of ways to deal with stress. You could go on a long walk, play basketball, read a book, go for a run or a bike ride, or even write a poem!||Great useful advice on how to deal with stress||i go outside||Ok|
|Empathy and perspective taking||Maybe someone has called you a bully but you didn\'t know you hurt the person. This can be frustrating. Take a moment to think about why the other person may have felt bullied.||I good suggestion and way to get someone to stop and think about their actions.||okay||Good|
|Emotion regulation: anger, hostility||Imagine you’re in a fight with your friend. To deal with it, you could: stop talking to her. Shove her to show her you\'re mad. Make up a rumor about her. Or, tell her how you feel.||This a great example for this type of situation.||id just tell her how i feel i guess||Make it a little more clear but good options||Good|
|Emotion regulation: anger, hostility||A teen told me: “I try to think about the situation before acting. If I can remember to do that, it works. But when I\'m angry I usually forget those things.”||I like this comment I think it’s very true||i think sometimes before i act||Ok|
|Emotion regulation: impulsivity||You don’t want to say something you’ll regret. Trying to get even with a bully continues the cycle. And it might put you at risk for something unsafe to happen.||A good comment and way to look at it||getting even makes me feel bad even though they\'ve hurt me||Feel like im being talked down to|
|Quote||Great advice: When bad things happen, I know you want to believe they are a joke, but sometimes, life is scary and dark. That is why we must find the light.” - BMO)||Very helpful advice and positive.||nice quote||Ok|
aTo maintain participant voice, quotes are verbatim. As such, grammar and spelling errors exist.
Integrating Content Advisory Team Feedback Into BullyDown
Overlapping feedback provided by two or more youth were integrated into the program content where possible. For example, we deleted the word “rock star” and messages deemed particularly confusing (eg, the quote from BMO). We also modified the tone of messages to sound less motherly or adult-like. The message:
You can tell your friend how you feel. It might be hard. And it might not go well. But if you’re in a healthy friendship and you use assertive communication, it could go great!
Tell your friend how you feel. It might be hard, and it might not go well. But if you’re in a healthy friendship, and you talk assertively, it might go great!
We also reviewed the messages scheduled for the beginning and end of each program day for their flow and transitions, based on youth feedback.
We chose not to make the level-up questions more challenging as this seemed likely to make the feature further simulate homework. The level-up protocol was already designed to automatically move nonresponsive participants to the next module if they did not respond to the questions after a certain period. We established a plan to closely monitor participant reactions to the level-up feature during the beta test for further indication of acceptability.
We deemed participants’ positive review of Happy Genie as supportive of its acceptability. We renamed the feature “Forever Friend” based on the names suggested by participants and added inspirational messages (eg, A new day = a new beginning) and quotes from famous people (eg, “I’ve failed over and over again in my life. And that is why I succeed.” – Michael Jordan).
Beta Test Results
Recruitment and Enrollment
Enrollment spanned 3 months and required research staff to visit the partnered middle school four times. Research staff screened 78 youth, 45% (35/78) of whom were eligible. Reasons for ineligibility included: not having a cell phone (21/78, 27%), not or uncertain if they were enrolled in an unlimited text messaging plan (9/78, 12%), and not planning to or uncertain if they would have the same phone number for at least 6 months (21/78, 27%), including the possibility that parents may take away phone privileges as a form of punishment.
Thirty-three eligible youth returned their permission forms, all of whom had parental permission to participate. Twenty-four eligible students provided written assent to participate, of which 22 completed the registration process and were randomized. Of the nine youth who had parental permission but did not enroll, three changed their mind, two moved, one was absent every time the research staff went to the school, and three had problems with phone access (eg, one shared the phone with a parent). Reasons for not completing the registration process included no longer being interested (n=1) and no longer having a cell phone (n=1). Participants were successfully randomized, resulting in 14 students assigned to BullyDown and eight to the control group ().
We conducted the beta test between December 2014 and May 2015. No concerns from the school administration or staff were expressed during the field period. Participants encountered some technology difficulties, however. The school’s firewall initially blocked the project enrollment website, but school technology staff quickly resolved the issue. Additionally, many students forgot to bring their phones to verify its compatibility with the program software. These students completed the Web-based baseline survey and were instructed to complete enrollment at home by responding with the word “verify” to the text message verification. Those randomized to the intervention group received a link to the Text Buddy Code of Conduct to read and accept. Some students had trouble completing the tasks, requiring research staff to return to the school to assist them in person.
Thirty-six percent (8/22) of participants completed the Web-based follow-up survey on their own time, and 14 completed it at school. The primary reason for not completing the survey independently was forgetting to do it. One participant thought the survey link did not look legitimate and so was hesitant to click on the link.
Program Acceptability and Feasibility
During the program delivery period, 86% (12/14) of intervention participants sent at least one message to their Text Buddy (range, 2-52), and 29% (4/14) sent at least one message to Happy Genie (range, 1-8). Half of intervention participants (7/14) responded to all seven weekly level-up questions, and 21% (3/14) responded to two or fewer level-up questions.
Feedback from the biweekly brief survey () suggested that many intervention participants thought the messages were interesting and fun, although some felt the messages were boring (eg, week 2 messages). Seventy-five percent (9/12) of intervention youth were able to provide an example of a message they found memorable (week 4), and 75% (9/12) of youth provided a specific example of something they liked about the program (week 6).
|Week||Program Message||Feedback per participanta,b|
|13-year-old male||12-year-old male||13-year-old female||12-year-old female|
|2||What do you think of the messages? Are they confusing? Boring? Interesting? Fun? Text me back what you think and why. The more details, the better.||They are interesting and informative!||They have been helpful||Borin and fun, I already know half of the information you send me but I like the interactive texts||They are fun bc it tells me more information about the program|
|Is there anything about your experience in BullyDown that you\'d like me to know at this point? Text your feedback or text “no.”||I think there are too many texts per day.||Not right now no||No||U guys have told me most of the stuff I didn\'t know|
|4||What one BullyDown text message sticks out in your mind, and why? This could be any message you\'ve received since the start of the program.||Being your loudest supporter because it shows that you should push yourself to do your best.||Believe you can and your halfway there.||All of the things to do when you are stressed..||To close ur eye when ur stressed|
|Is there anything about your experience in the program that you\'d like me to know at this point? Text your feedback or text “no”.||No||No||No.||No|
|6||What is one thing that you really like about BullyDown and why? The more detailed you can be, the better.||I like the daily advice and the inspirational messages.||I like all of the possible solutions. To the problems that kids have to deal with everyday. A Lot of. Kids don\'t kno how to handle situations||The one thing that I like is that it gives advice, it may be things that you already know it\'s something..||That I can learn more stuff about bullying and stuff like that I al so can ask questions about anything I need to know about bullying|
|And what is one thing that you really do not like about BullyDown or think that we need to make better, and why? Again, detail is helpful.||Maybe not text as much through the day. Maybe just once in the morning and once at night.||Sometimes there are a lot of txts and it gets. Confusing||I think that BullyDown needs to be a place that people feel safe to explain how they feel and why they are feeling that way, but I honestly don\'t feel th at way. So to make it better you could let people talk with someone through text, instead of just giving advice.||I like everything about bully down|
|What else about your experience in the program would you like me to know at this point? Text your feedback or text “nothing”.||Nothing||When I txt my bullydown friend. They don\'t reply||Nothing||Nothing|
|8||How would you rate the number of messages that we send you each week? Too few, too many, or just right? Text me what you think.||Too many.||Sometimes too many||Just right.||Just right|
|Is there anything about the program that you\'d like me to know about your experience at this point? Text “no” or text your feedback.||No||Having a bully buddy isn\'t a good idea. They don\'t reply to the text questions||No||No|
|2||What do you think of the messages? Are they confusing? Boring? Interesting? Fun? Text me back what you think and why. The more details, the better.||I am confused about what I should be texting to my text Buddy and when. I respond to your text when you ask a question but up until yesterday I hadnt texted my Buddy.||Interesting because I don\'t know what stuff you are going to text me.||Boring and interesting||Really idk its kinda confusing because when u asked me some of the question I got confused when I went to look for the answer and I got it wrong|
|Is there anything about your experience in BullyDown that you\'d like me to know at this point? Text your feedback or text “no”.||No||No||No||No not really|
|4||What one BullyDown text message sticks out in your mind, and why? This could be any message you\'ve received since the start of the program.||When Im angry I usually go to my room and try to get something else on my mind. I end up later thinking about the situation and thinking it through.||The Forever Friendctexts. They cheer me up||The one about anger and the things I could do to prevent it. I liked that one because sometimes I get carried away and I get anger for no reason..||about calming your anger down why because it really helps when I am angry and when I am depressed about something|
|Is there anything about your experience in the program that you\'d like me to know at this point? Text your feedback or text “no”.||No||No.||No||No not really|
|6||What is one thing that you really like about BullyDown and why? The more detailed you can be, the better.||Ok||Forever friemd because it cheers me up||I like how y\'all go in depth with every situation||ack and view what I have learned on here and what I already knew and if I was being a bully I would remember all the stuff I was taught|
|And what is one thing that you really do not like about BullyDown or think that we need to make better, and why? Again, detail is helpful.||Sometimes the times of the texts are not convenient for a response since we are students that may be involved in other activities.||The texts I get back to back because I can\'t keep up||Maybe that y\'all could slow down on the text messages||I like how they go step by step trying to prevent us from being and getting bullied I like that because if someone was gettin bullied I can either look b|
|What else about your experience in the program would you like me to know at this point? Text your feedback or text “nothing”.||Ok||Nothing||Nothing||Nothing|
|8||How would you rate the number of messages that we send you each week? Too few, too many, or just right? Text me what you think.||Ok||Just right||Just right||Too many|
|Is there anything about the program that you\'d like me to know about your experience at this point? Text “no” or text your feedback.||No||No||No||No|
|12-year-old female||13-year-old female||12-year-old female||13-year-old male|
|2||What do you think of the messages? Are they confusing? Boring? Interesting? Fun? Text me back what you think and why. The more details, the better.||Confusing Also I still havent talk to my buddy||Boring|
|Is there anything about your experience in BullyDown that you\'d like me to know at this point? Text your feedback or text “no”.|
|4||What one BullyDown text message sticks out in your mind, and why? This could be any message you\'ve received since the start of the program.||Ok||The one about if my friend doesn\'t tell me she isn\'t coming to lunch|
|Is there anything about your experience in the program that you\'d like me to know at this point? Text your feedback or text “no”.||Who is my buddy and how do I contact them||No|
|6||What is one thing that you really like about BullyDown and why? The more detailed you can be, the better.||Yes||It. Gives you advice||I like Bullydown because it has really made me see how people can get bullied and how it can effect them and the different types of buling there is|
|And what is one thing that you really do not like about BullyDown or think that we need to make better, and why? Again, detail is helpful.||Nothing||Make questions more deep||I really don\'t think that you men and women missed anything major|
|What else about your experience in the program would you like me to know at this point? Text your feedback or text “nothing”.||Nonthing||Nothing|
|8||How would you rate the number of messages that we send you each week? Too few, too many, or just right? Text me what you think.|
|Is there anything about the program that you\'d like me to know about your experience at this point? Text “no” or text your feedback.|
aTo maintain participant voice, quotes are verbatim. As such, grammar and spelling errors exist.
bBlank spaces indicate a lack of response.
c“Forever Friend” refers to the Happy Genie feature.
Moreover, all participants said they would recommend the program to a friend when asked at the end of week 6 (). Opportunities for improvement were also noted: although 80% (8/10) of intervention participants agreed that reading the messages the same day they were received was easy ( ), five youth shared concerns about the message quantity or timing when asked at the end of week 6 about something they may not like about the program ( ).
Feedback from intervention participants suggested that they were enthusiastic about having a Text Buddy. This feature also drew at least one negative response each week, however (). Subsequent follow-up with youth revealed that their discomfort centered on being paired with someone from the same school and having to talk about bullying with a schoolmate: “…A Lot of kids in this area kno each other and don't want to talk about things like this.” (12-year-old male). Some youth also expressed frustration that their buddies were unresponsive to their messages; or felt discomfort about being paired with someone of the other sex or in a different grade.
As shown in, both the intervention and control group content appeared to be acceptable: more than four in five youth in each arm agreed that they liked the program. With 88% (7/8) of control participants saying that the program helped them not bully others in the future, this arm, although of lesser intensity than the intervention arm, appeared to blind participants to their arm assignment. In addition to high scores for understandability and salience of program content, the experience also seemed to be feasible: most youth disagreed or strongly disagreed that the program sent too many messages, and none of the beta test participants in either arm agreed that they stopped reading the messages by the end of the program.
|Youth responses||Control |
|Biweekly survey in field|
|How much are you liking BullyDown? (Week 2)||NAb,c||8 (80.0)|
|How easy or hard has it been to read your texts the *same* day we send them to you? (Week 4)||5 (83.3)||8 (80.0)|
|How likely are you to recommend BullyDown to your friends? (Week 6)||3 (60.0)||8 (100.0)|
|One-month follow-up survey||(n=8)||(n=14)|
|Acceptability (Agree/Strongly Agree)|
|I like the program||7 (87.5)||12 (85.7)|
|I learned things in BullyDown…|
|That will help me not bully others in the future||7 (87.5)||14 (100.0)|
|That will help me not be bullied by others in the future||6 (75.0)||12 (85.7)|
|That will help me stop bullying when I see it happening to others||6 (75.0)||13 (92.9)|
|I do not think people like me should go through the BullyDown program (Disagree/Strongly Disagree)||7 (87.5)||10 (71.4)|
|BullyDown talked too much about… (disagree/strongly disagree)|
|Feelings||7 (87.5)||8 (57.1)|
|Bullying||7 (87.5)||11 (78.6)|
|The text messages were easy to understand||6 (75.0)||12 (85.7)|
|BullyDown talked about things my friends and I experience in our lives||6 (75.0)||11 (78.6)|
|Feasibility (disagree/strongly disagree)|
|BullyDown sent too many messages||7 (87.5)||10 (71.4)|
|I stopped reading the messages by the end of the BullyDown program||8 (100.0)||10 (71.4)|
|BullyDown got in the way of my daily schedule||8 (100.0)||10 (71.4)|
aPercentages reflect those in the extreme categories: agree or strongly agree (4 or 5 on a 5-point Likert scale; 7-10 on a 10-point ordinal scale); disagree or strongly disagree (1 or 2 on a 5-point Likert scale; 1 or 2 on a 5-point ordinal scale).
bThe control group inadvertently did not receive the Week 2 biweekly survey.
The enrollment process seemed to require in-person facilitation by research staff at a level that is at least commensurate with an in-person intervention. Both interesting and potentially problematic, 33% (11/33) of youth who were eligible and had parental permission lost interest in participating before enrollment. Given the small sample size, this may be an anomaly. Further investigation of this feasibility issue in a larger trial is needed before conclusions can be drawn. Also, some participants shared that they would have been more likely to respond to the follow-up survey if the link had been sent by email instead of text message. Future efforts should consider collecting multiple forms of contact (eg, Facebook, email address).
Despite CAT participants’ expressed interest in the concept of Text Buddy, most beta test participants only sent one or two messages to their buddy. Based upon their feedback, future trials could ideally match Text Buddies who are attending different schools and, when possible, of the same sex and grade level. An interactive guide, either in-person or via the Internet, instructing how to use the feature could also be incorporated into the content. Additionally, only participants with expressed interest in the Text Buddy feature could be matched to reduce the likelihood of unresponsive buddies.
Several intervention youth expressed concern about the intensity and/or timing of the daily messages during the field period but not when it was assessed at 1-month follow-up. This feedback most likely arose on days that had five or six messages, resulting in participants receiving four or five messages across a 4-hour window after school. To ameliorate this, the total number of messages per day in future iterations of the program will be limited to four where possible.
Consistent with feedback from some CAT participants who thought the level-up questions were too easy, one beta test participant thought that the questions should be more meaningful. Given the lack of negative feedback from other beta test participants about the level-up experience and the desire to avoid simulating homework however, more feedback from randomized controlled trial participants will be solicited before making significant changes.
Stepwise development of BullyDown over a 1-year period helped ensure the program was generally well received by middle school students. Findings reveal important insights about developing mHealth interventions for younger youth, particularly those focused on bullying prevention, including the preferred number of messages per day (ie, 2-4 messages), a desire for shorter messages, and the ideal age range (ie, 7th grade and older). In interventions aimed at older adolescents (e.g. 14- to 17-years old), others have noted a similar preference among participants for a low-intensity experience in interventions [, ]; however, higher intensity programs (eg, 8-15 messages per day) have also received indications of acceptability [ ]. The desired threshold seems likely to be affected not only by the age of the youth but also the topic in question. This variability in participants’ acceptability of various program intensities highlights the importance of iteratively developing new mHealth program using ongoing feedback from the target population.
Also contrasting other mHealth adolescent recruitment efforts , Facebook was less efficient in nationally recruiting youth than was a more traditional recruitment method (ie, using a recruitment firm). Initial interest in the ads was commensurate with expectations for a successful campaign, but completion of the Web-based screener seemed to be an issue. Perhaps younger youth are more cautious when providing contact information to unknown people or organizations via the Internet, or perhaps the topic (ie, bullying prevention), when more fully explained, is of little interest to youth. However, the school administrator’s enthusiasm for the beta test suggests that the program is acceptable and feasible for implementation in school-based samples. It also supports a hypothesis that the eventual program will need to be promoted to school administrators rather than youth directly.
Several additional differences between the experiences in the BullyDown beta test with the implementation of other text messaging–based health behavior change programs should be noted. For example, compared with high school-aged youth , middle school students' relationship with their phones seems to be less predictable: (1) they do not carry their phones everywhere – at least not to school and not necessarily outside of school, (2) they might not remember their phone number accurately, and (3) their access appears to be more tenuous – parents are more likely to restrict their cell phone use. Also, staff resources needed to enroll youth appear to be commensurate with in-person interventions. This means that cost savings in enrollment will not be realized. Once the intervention program messages begin however, most 7th and 8th graders are able to engage with the program on their own, suggesting that once engaged, a standalone technology intervention has promise. These differences highlight the heterogeneity of mHealth research: simply because a particular protocol works for one program and population does not mean the same protocol will work with other programs and populations.
Although control and intervention participants attended classes side by side, the control arm appears to have been blinded: they were equally likely as the intervention arm to say they liked the program, and a similar percent of youth said they learned valuable things about bullying during the program. This suggests that randomization at the individual-level may not pose a serious threat to contamination; however, results should be replicated in a larger sample size.
Specific program recommendations that emerged from this work may not generalize to other health behavior change topics or youth populations. Although the national recruitment strategy is a strength, FG and CAT participants may not be representative of all youth given they were identified through a recruitment firm. Beta test participants were recruited from a disadvantaged public middle school in Illinois and may not be representative of students in other settings. More generally, youth who have a cell phone and are enrolled in an unlimited text messaging plan may be different from youth who do not meet these criteria. It seems unlikely however that youth who lack a cell phone or are not on an unlimited text messaging plan would opt into such a program if it was publicly available. Additionally, confirming that youth actually read the text messages is not possible beyond their self-report; therefore, actual exposure is unknown.
Increasingly, SEL programs are being implemented in schools across the United States to address a wide-range of problematic behaviors (eg, bullying, delinquency) and to promote academic success. However, the majority of these programs include curricula that range from 15 to 20 or more lessons, requiring significant instructional time and resources that some school districts may not be able to provide. BullyDown harnesses the benefits of text messaging, such as being able to go where youth are and communicating important information in consumable chunks, to offer a bullying prevention program that can be delivered outside of school and without the need for facilitator time or resources. While expensive in time and money, iterative intervention refinement increases the likelihood that the resulting intervention is salient to the target population while retaining its adherence to theory.
Given the positive feedback from beta test participants about the program content and the experience as well as the 100% (22/22) retention rate, the BullyDown mHealth bullying prevention program appears to be both feasible and acceptable for implementation in a middle school setting with 7th and 8th grade youth. Our next step will be to test the intervention in a large randomized controlled trial to see if exposure is associated with reduction in bullying behavior.
Drs. Ybarra and Espelage conceived the study. Dr. Ybarra identified the iterative development steps. Dr. Espelage mapped the SEL model to content areas and Dr. Ybarra led the writing of the content, including the integration of findings into the messaging. Drs. Ybarra and Espelage co-moderated the focus group discussions. Dr. Espelage and Ms. Prescott co-moderated the CAT. Ms. Prescott managed participant recruitment and retention activities. All three authors helped finalize the program content and contributed to the fielding of the beta test. The manuscript was drafted by Dr. Ybarra and Ms. Prescott, and significantly edited by Dr. Espelage. All authors approved the final version of the manuscript.
The project described was supported in part by a grant from HopeLab. The content is solely the responsibility of the authors and does not necessarily represent the official views of the HopeLab. We would like to thank the entire study team at the Center for Innovative Public Health Research and the University of Illinois at Urbana-Champaign for their contributions to the research and the participants for their time in the study. We also thank Ms. Emilie Chen for her assistance proofreading the article.
Conflicts of Interest
- Wang J, Nansel TR, Iannotti RJ. Cyber and traditional bullying: differential association with depression. J Adolesc Health 2011;48:415-417 [FREE Full text] [CrossRef] [Medline]
- Gladden R, Vivolo-Kantor A, Hamburger M, Lumpkin C. Bullying Surveillance Among Youths: Uniform Definitions for Public Health and Recommended Data Elements, Version 1. Atlanta: National Center for Injury Prevention and Control, Centers for Disease Control and Prevention and U.S. Department of Education; 2013.
- Sourander A, Helstelä L, Helenius H, Piha J. Persistence of bullying from childhood to adolescence--a longitudinal 8-year follow-up study. Child Abuse Negl 2000;24:873-881. [Medline]
- Bond L, Carlin JB, Thomas L, Rubin K, Patton G. Does bullying cause emotional problems? A prospective study of young teenagers. BMJ 2001;323:480-484 [FREE Full text] [Medline]
- Arseneault L, Bowes L, Shakoor S. Bullying victimization in youths and mental health problems: 'much ado about nothing'? Psychol Med 2010;40:717-729. [CrossRef] [Medline]
- Klomek AB, Sourander A, Gould M. The association of suicide and bullying in childhood to young adulthood: a review of cross-sectional and longitudinal research findings. Can J Psychiatry 2010;55:282-288. [Medline]
- Copeland WE, Wolke D, Lereya ST, Shanahan L, Worthman C, Costello EJ. Childhood bullying involvement predicts low-grade systemic inflammation into adulthood. Proc Natl Acad Sci U S A 2014 May 27;111:7570-7575 [FREE Full text] [CrossRef] [Medline]
- Vaillancourt T, Duku E, Becker S, Schmidt LA, Nicol J, Muir C, et al. Peer victimization, depressive symptoms, and high salivary cortisol predict poorer memory in children. Brain Cogn 2011;77:191-199. [CrossRef] [Medline]
- Shalev I, Moffitt TE, Sugden K, Williams B, Houts RM, Danese A, et al. Exposure to violence during childhood is associated with telomere erosion from 5 to 10 years of age: a longitudinal study. Mol Psychiatry 2013;18:576-581 [FREE Full text] [CrossRef] [Medline]
- Merrell KW, Gueldner BA, Ross SW, Isava DM. How effective are school bullying intervention programs? A meta-analysis of intervention research. School Psychology Quarterly 2008;23:26-42. [CrossRef]
- Ttofi MM, Farrington DP. Effectiveness of school-based programs to reduce bullying: a systematic and meta-analytic review. J Exp Criminol 2011 Sep 16;7:27-56. [CrossRef]
- Ferguson CJ, San Miguel C, Kilburn JC, Sanchez P. The effectiveness of school-based anti-bullying programs: a meta-analytic review. Criminal Justice Review 2007;32:401-414. [CrossRef]
- Smith J, Schneider B, Smith P, Ananiadou K. The effectiveness of whole-school antibullying programs: A synthesis of evaluation research. School Psychology Review 2004;33:547-560.
- Vreeman RC, Carroll AE. A systematic review of school-based interventions to prevent bullying. Arch Pediatr Adolesc Med 2007;161:78-88. [CrossRef] [Medline]
- Ttofi MM, Farrington DP. Bullying prevention programs: the importance of peer intervention, disciplinary methods and age variations. J Exp Criminol 2012;8:443-462. [CrossRef]
- Yeager DS, Fong CJ, Lee HY, Espelage DL. Declines in efficacy of anti-bullying programs among older adolescents: Theory and a three-level meta-analysis. Journal of Applied Developmental Psychology 2015;37:36-51. [CrossRef]
- Paluck EL, Shepherd H. The salience of social referents: a field experiment on collective norms and harassment behavior in a school social network. J Pers Soc Psychol 2012;103:899-915. [CrossRef] [Medline]
- O'Bannon BW, Thomas KM. Mobile phones in the classroom: preservice teachers answer the call. Computers & Education 2015;85:110-122. [CrossRef]
- Whittaker R, Merry S, Stasiak K, McDowell H, Doherty I, Shepherd M, et al. MEMO--a mobile phone depression prevention intervention for adolescents: development process and postprogram findings on acceptability from a randomized controlled trial. J Med Internet Res 2012;14:e13 [FREE Full text] [CrossRef] [Medline]
- Lenhart A. Teens, Social Media, & Technology Overview. Washington, DC: Pew Internet & American Life Project; 2015. URL: http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015/ [accessed 2016-04-22] [WebCite Cache]
- Lenhart A, Ling R, Campbell S, Purcell K. Teens and Mobile Phones. Washington, DC: Pew Internet & American Life Project; 2010. URL: http://www.pewinternet.org/2010/04/20/teens-and-mobile-phones/ [accessed 2016-04-22] [WebCite Cache]
- Whittaker R, McRobbie H, Bullen C, Borland R, Rodgers A, Gu Y. Mobile phone-based interventions for smoking cessation. Cochrane Database Syst Rev 2012;11:CD006611. [CrossRef] [Medline]
- Preston KE, Walhart TA, O'Sullivan AL. Prompting healthy behavior via text messaging in adolescents and young adults. American Journal of Lifestyle Medicine 2011;5:247-252. [CrossRef]
- Wei J, Hollin I, Kachnowski S. A review of the use of mobile phone text messaging in clinical and healthy behaviour interventions. J Telemed Telecare 2011;17:41-48. [CrossRef] [Medline]
- Cole-Lewis H, Kershaw T. Text messaging as a tool for behavior change in disease prevention and management. Epidemiol Rev 2010;32:56-69 [FREE Full text] [CrossRef] [Medline]
- Haug S, Schaub MP, Venzin V, Meyer C, John U. Efficacy of a text message-based smoking cessation intervention for young people: a cluster randomized controlled trial. J Med Internet Res 2013;15:e171 [FREE Full text] [CrossRef] [Medline]
- Greenberg MT, Weissberg RP, O'Brien MU, Zins JE, Fredericks L, Resnik H, et al. Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. Am Psychol 2003;58:466-474. [Medline]
- Ragozzino K, Utne O’Brien M. Social and Emotional Learning and Bullying Prevention. Chicago: Collaborative for Academic, Social, and Emotional Learning (CASEL); 2009. URL: https://static1.squarespace.com/static/513f79f9e4b05ce7b70e9673/t/5367958ee4b0dbc1364dbb7b/1399297422536/3_SEL_and_Bullying_Prevention_2009.pdf [accessed 2016-04-22] [WebCite Cache]
- Elias M, Zins JE, Weissberg RP, Frey KS, Greenberg MT, Haynes NM, et al. Promoting social and emotional learning: guidelines for educators. Alexandria, Va: Association for Supervision and Curriculum Development; 1997.
- Guerra NG, Bradshaw CP. Linking the prevention of problem behaviors and positive youth development: core competencies for positive youth development and risk prevention. New Dir Child Adolesc Dev 2008;2008(122):1-17. [CrossRef] [Medline]
- Hawkins J, Smith B, Catalano R. Social development and social and emotional learning. In: Zins JE, Weissberg RP, Wang MC, Walberg HJ, editors. Building Academic Success on Social and Emotional Learning: What Does the Research Say?. New York, NY: Teachers College Press; 2004:135-150.
- Zins JE, Weissberg RP, Wang MC, Walberg HJ. Building Academic Success on Social and Emotional Learning: What Does the Research Say?. New York: Teachers College Press; 2004.
- Brown EC, Low S, Smith BH, Haggerty KP. Outcomes from a school-randomized controlled trial of Steps to Respect: a school bullying prevention program. School Psychology Review 2011;40:423-243.
- Espelage DL, Low S, Polanin JR, Brown EC. The impact of a middle school program to reduce aggression, victimization, and sexual violence. J Adolesc Health 2013;53:180-186. [CrossRef] [Medline]
- Botvin GJ, Griffin KW, Nichols TD. Preventing youth violence and delinquency through a universal school-based prevention approach. Prev Sci 2006;7:403-408. [CrossRef] [Medline]
- Basile KC, Espelage DL, Rivers I, McMahon PM, Simon TR. The theoretical and empirical links between bullying behavior and male sexual violence perpetration. Aggression and Violent Behavior 2009;14:336-347. [CrossRef]
- Polanin J, Espelage D, Pigott T. A meta-analysis of school-based bullying prevention programs’ effects on bystander intervention behavior. School Psychology Review 2012;41:47-65.
- Hong JS, Espelage DL. A review of research on bullying and peer victimization in school: An ecological system analysis. Aggression and Violent Behavior 2012;17:311-322. [CrossRef]
- Espelage DL, Bosworth K, Simon TR. Examining the social context of bullying behaviors in early adolescence. Journal of Counseling and Development 2000;78:326-333. [CrossRef]
- Espelage D, Green H, Polanin J. Willingness to intervene in bullying episodes among middle school students: individual and peer-group influences. The Journal of Early Adolescence 2011;32:776-801. [CrossRef]
- Holt MK, Espelage DL. Perceived social support among bullies, victims, and bully-victims. J Youth Adolescence 2006;36:984-994. [CrossRef]
- Cook CR, Williams KR, Guerra NG, Kim TE, Sadek S. Predictors of bullying and victimization in childhood and adolescence: a meta-analytic investigation. School Psychology Quarterly 2010;25:65-83. [CrossRef]
- Espelage DL, Holt MK, Henkel RR. Examination of peer-group contextual effects on aggression during early adolescence. Child Dev 2003;74:205-220. [Medline]
- Feshbach N, Feshbach S. Empathy training and the regulation of aggression: potentialities and limitations. Academic Psychology Bulletin 1982;4:399-413.
- Kaukiainen A, Björkqvist K, Lagerspetz K, Österman K, Salmivalli C, Rothberg S, et al. The relationships between social intelligence, empathy, and three types of aggression. Aggressive Behavior 1999;25:81-89.
- Miller PA, Eisenberg N. The relation of empathy to aggressive and externalizing/antisocial behavior. Psychol Bull 1988;103:324-344. [Medline]
- Endresen I, Olweus D. Self-reported empathy in Norwegian adolescents: Sex differences, age trends,relationship to bullying. In: Bohart AC, Stipek DJ, editors. Constructive & Destructive Behavior: Implications for Family, School, & Society. Washington, DC: American Psychological Association; 2001:147-165.
- Boulton MJ, Trueman M, Flemington I. Associations between secondary school pupils' definitions of bullying, attitudes towards bullying, and tendencies to engage in bullying: age and sex differences. Educational Studies 2002;28:353-370. [CrossRef]
- Bosworth K, Espelage D, Simon T. Factors associated with bullying behavior in middle school students. Journal of Early Adolescence 1999;19:341-362.
- Espelage DL, Bosworth K, Simon TR. Short-term stability and prospective correlates of bullying in middle-school students: an examination of potential demographic, psychosocial, and environmental influences. Violence Vict 2001;16:411-426. [Medline]
- Salmivalli C. Bullying and the peer group: a review. Aggression and Violent Behavior 2010;15:112-120. [CrossRef]
- Durlak JA, Weissberg RP, Dymnicki AB, Taylor RD, Schellinger KB. The impact of enhancing students' social and emotional learning: a meta-analysis of school-based universal interventions. Child Dev 2011;82:405-432. [CrossRef] [Medline]
- Espelage DL, Rose CA, Polanin JR. Social-emotional learning program to reduce bullying, fighting, and victimization among middle school students with disabilities. Remedial and Special Education 2015. [CrossRef]
- Rodgers A, Corbett T, Bramley D, Riddell T, Wills M, Lin R, et al. Do u smoke after txt? Results of a randomised trial of smoking cessation using mobile phone text messaging. Tob Control 2005;14:255-261 [FREE Full text] [CrossRef] [Medline]
- Ybarra ML, Holtrop JS, Prescott TL, Rahbar MH, Strong D. Pilot RCT results of Stop My Smoking USA: a text messaging-based smoking cessation program for young adults. Nicotine Tob Res 2013;15:1388-1399 [FREE Full text] [CrossRef] [Medline]
- Ybarra ML, Prescott TL, Philips GL, Bull SS, Parsons JT, Mustanski B. Iteratively developing an mHealth HIV prevention program for sexual minority adolescent men. AIDS Behav 2015. [CrossRef] [Medline]
- Waterlander W, Whittaker R, McRobbie H, Dorey E, Ball K, Maddison R, et al. Development of an evidence-based mHealth weight management program using a formative research process. JMIR Mhealth Uhealth 2014;2:e18 [FREE Full text] [CrossRef] [Medline]
- Whittaker R, Merry S, Dorey E, Maddison R. A development and evaluation process for mHealth interventions: examples from New Zealand. J Health Commun 2012;17 Suppl 1:11-21. [CrossRef] [Medline]
- Ybarra ML, Prescott TL, Holtrop JS. Steps in tailoring a text messaging-based smoking cessation program for young adults. J Health Commun 2014;19:1393-1407. [CrossRef] [Medline]
- Ybarra ML, Holtrop JS, Bağci Bosi T, Emri S. Design considerations in developing a text messaging program aimed at smoking cessation. J Med Internet Res 2012;14:e103 [FREE Full text] [CrossRef] [Medline]
- Center for Innovative Public Health Research. San Clemente, CA. 2015. BullyDown URL: http://innovativepublichealth.org/projects/bullydown-b/ [accessed 2016-04-22] [WebCite Cache]
- Mustanski B, DuBois LZ, Prescott TL, Ybarra ML. A mixed-methods study of condom use and decision making among adolescent gay and bisexual males. AIDS Behav 2014;18:1955-1969. [CrossRef] [Medline]
- Thompson D, Cantu D, Bhatt R, Baranowski T, Rodgers W, Jago R, et al. Texting to increase physical activity among teenagers (TXT Me!): rationale, design, and methods proposal. JMIR Res Protoc 2014;3:e14 [FREE Full text] [CrossRef] [Medline]
- Devine S, Bull S, Dreisbach S, Shlay J. Enhancing a teen pregnancy prevention program with text messaging: engaging minority youth to develop TOP ® Plus Text. J Adolesc Health 2014;54(3 Suppl):S78-S83 [FREE Full text] [CrossRef] [Medline]
- Amon KL, Campbell AJ, Hawke C, Steinbeck K. Facebook as a recruitment tool for adolescent health research: a systematic review. Acad Pediatr 2014;14:439-447.e4. [CrossRef] [Medline]
|CAT: content advisory team|
|FG: focus group|
|SMS: short message service|
Edited by G Eysenbach; submitted 13.07.15; peer-reviewed by C Heminger, R Haines-Saah, I Montagni; comments to author 24.09.15; revised version received 25.11.15; accepted 04.02.16; published 13.06.16
©Michele L. Ybarra, Tonya L. Prescott, Dorothy L. Espelage. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 13.06.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.