This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Bullying is a significant public health issue among middle school-aged youth. Current prevention programs have only a moderate impact. Cell phone text messaging technology (mHealth) can potentially overcome existing challenges, particularly those that are structural (e.g., limited time that teachers can devote to non-educational topics). To date, the description of the development of empirically-based mHealth-delivered bullying prevention programs are lacking in the literature.
To describe the development of BullyDown, a text messaging-based bullying prevention program for middle school students, guided by the Social-Emotional Learning model.
We implemented five activities over a 12-month period: (1) national focus groups (n=37 youth) to gather acceptability of program components; (2) development of content; (3) a national Content Advisory Team (n=9 youth) to confirm content tone; and (4) an internal team test of software functionality followed by a beta test (n=22 youth) to confirm the enrollment protocol and the feasibility and acceptability of the program.
Recruitment experiences suggested that Facebook advertising was less efficient than using a recruitment firm to recruit youth nationally, and recruiting within schools for the pilot test was feasible. Feedback from the Content Advisory Team suggests a preference for 2-4 brief text messages per day. Beta test findings suggest that BullyDown is both feasible and acceptable: 100% of youth completed the follow-up survey, 86% of whom liked the program.
Text messaging appears to be a feasible and acceptable delivery method for bullying prevention programming delivered to middle school students.
Bullying is a significant adolescent health issue: depending on the type of bullying, an estimated 9% to 38% of middle school students bully their peers sometimes or more often each semester [
Existing bullying prevention programs seem to be having a modest impact. [
Many are exploring the potential for digital technology to enhance the educational experience and to promote prosocial behaviors [
Research suggests that effective behavior change programs are guided by strong theoretical models[
SEL-based programs use social skills instruction to address behavior, discipline, safety, and academics in order to help youth become more self-aware, manage their emotions, build social skills (eg, empathy, perspective-taking, respect for diversity), build friendship skills, and decrease engagement in delinquent behavior [
SEL-based programs have reported positive results in terms of reducing bullying and other disruptive behaviors in middle school [
According to the SEL model, interactivity increases participant engagement, and new behaviors need to be practiced to be integrated into one’s behavioral script. BullyDown therefore includes several features aimed at engaging youth, while also encouraging them to practice the skills discussed in the content. Principal among these is the “Text Buddy” feature, which pairs intervention participants as “buddies,” allowing them to discuss what they learn through the program with each other. Text Buddy has been used successfully in other text messaging–based programs and is associated with behavior change [
A third interactive feature tested for acceptability is Happy Genie, which sends intervention participants on-demand messages that provide positive encouragement. This type of feature is specific to the target audience and the topic, but is based upon other behavior change programs that have also used on-demand features [
Social-Emotional Learning logic model.
Similar to iterative methods used in other mHealth programming [
The Chesapeake institutional review board reviewed and approved the research protocol. We sequentially implemented five activities: (1) online focus groups (FGs), (2) ongoing content development, (3) a Content Advisory Team (CAT), and (4) an internal team test, followed by (5) a beta test. We drafted content after conducting the FGs and continued refining content during the subsequent research activities. Research materials described herein are available via the Internet [
The development work spanned approximately 1 year: March 2014 to May 2015. For all research activities, study eligibility criteria matched those of the intended users of the intervention. Participants were enrolled in 7th or 8th grade (of exception, 6th graders took part in the FGs), owned a cell phone, were enrolled in an unlimited text messaging plan, intended to keep the same phone number for the next 6 months, had parental permission, and provided informed assent.
The FGs aimed to: better understand how young people experience bullying and their prior exposure to bullying prevention programs in school, to obtain the “voice” of the target
population, and to query process issues. To gather feedback from a diverse group of youth nationally, our initial recruitment strategy relied on Facebook advertising targeted to US youth aged 13- to 14-years old (
Upon yielding a low response rate to the Facebook advertisements, we made subsequent changes to the recruitment strategy: increasing the Facebook ad budget from US$50 to US$100 per day, reducing the number of items on the Web-based screener to lower the completion time, and adding the company logo and link to the company website in the screener to increase the website legitimacy. We also contacted partner youth organizations who agreed to advertise for the research activity. When these attempts did not noticeably increase the number of eligible screeners, we contracted a research recruitment firm.
The research team developed a script of questions to guide the FG discussions. Example questions included:
When you think of the word “bullying,” what do you think of? How (if at all) is this different from what you think of when you think about “aggression?” What about being angry? Or getting into fights with friends?
Has someone asked you to think about what it might feel like to be bullied? What are things that you do that help you understand how others might be feeling? Like how it might feel to be bullied or to bully someone?
Additionally, the FGs queried process issues, including the preferred number and timing of daily text messages and feedback on possible intervention names.
Two FGs of 20 participants each were conducted. The script was written to span 3 days of questions. We posited that youths’ experiences and ability to articulate them would vary by grade level. As such, this was our stratification variable: FG1 was conducted with 6th and 7th graders, and FG2 with 7th and 8th graders. Participants logged into the bulletin board–style FG discussion on a password-protected site two to three times each day when and where it was convenient for them. Because of its asynchronous format, participants from across a wide geographic area could be included. Participants received a US$50 Amazon gift card as an incentive for complete participation during all 3 days.
Focus group Facebook advertisements (2a).
Focus group Facebook advertisements (2b).
We next convened a CAT to review and gather feedback about the saliency and understandability of the program content. The CAT also explored the acceptability of the three proposed program features: (1) Text Buddy, (2) Happy Genie, and (3) level-up questions.
To gather feedback, participants, or parents of participants when only the parent email address was available, received a Word document of the draft program text messages. The instructions asked youth to review and comment on each message within each week’s module (eg, Communication). Probes were included to help give direction about what they could consider when reading the proposed content (eg, Are you able to read it and understand it easily?). Instructions also emphasized that the messages were still being developed and also necessarily short given the character limit of text messages. However, during this stage, the proposed text messages were longer than 160 characters (e.g., 180 characters), as the aim was to first identify appropriate tone and content.
Next, participants took part in a 2-day online discussion to share feedback as a group. For each weekly topic, moderators asked youth to identify their favorite, least favorite, most helpful, and least helpful messages; any messages that felt too long or short; and what they learned from the module. Moderators also asked participants about the proposed program features and to suggest three of their own Happy Genie messages. Participants earned up to a US$50 amount in an Amazon gift card, US$30 for their individual review, and US$20 for their participation in the online discussion.
Findings were integrated into the program content, including feedback about the level-up questions and Happy Genie features. For example, to combat the judgment that bullies are “bad” (and therefore, implicitly, neither you nor your friends could be bullies) that emerged during the formative work, the level-up question during week 5 was:
Good morning and welcome to Week 5! Let's get you to Level 4. True or false: Bullies are mean to others because they are bad people. Text me back your answer.
Those who responded correctly received the following reinforcing message:
Bullies are mean for lots of reasons but that doesn't mean they are bad people. Maybe they just need help learning how to share their feelings. Hello, Level 4!
Those who responded incorrectly received the following response:
Actually, bullies can be mean for lots of reasons. That doesn't mean they are bad people. They might just need some help learning how to share their feelings.
Youth who answered incorrectly were given additional chances to answer a new question correctly before they were automatically moved on to the next level:
Let's try again. For Level 4: True or false: Young people who act “differently” are asking to be bullied.
Correct responses received the following:
Right! We're all different. Kids who act or look differently are just being themselves. They want to have friends just like everyone else. Hello, Level 4!
Incorrect responses resulted in this text:
We're all different. Teens who act or look differently are just being themselves. They want to have friends just like everyone else. Onward to Level 4!
Following a successful internal team test that confirmed the program software performed as intended and that the content was readable on cell phone screens, we conducted a beta test to confirm the feasibility of the protocol and technology in a school setting. To facilitate resolution of technology problems if needed, the beta test was conducted with participants from a low-income middle school in Illinois. Research staff went to the school in early December 2014 to screen 7th and 8th grade students from four preselected classrooms. Staff issued parental permission forms to eligible students to return in 1 week. During subsequent visits, research staff picked up the signed permission forms and facilitated enrollment (ie, obtaining informed assent from the students, helping the students complete the baseline Web-based survey in the school computer room, and verifying the students’ phones for compatibility with the software program).
Participants were randomized to either the 7.5-week intervention or control group at a 2:1 ratio. This maximized the amount of information gathered about the intervention, while also gathering sufficient data about the control group experience to determine its feasibility and acceptability. To avoid school hours, participants received one program message in the morning (7:15 AM) and the remainder of messages after school hours (between 4:00 and 9:00 PM). Intervention participants received three to six messages per day for a total of 214 messages, with an additional 15 to 29 messages from the level-up feature; were randomly assigned a Text Buddy; and had access to Happy Genie. Control participants received two messages weekly, one message encouraging thoughtful behavior (eg, A message from BullyDown: “Try doing one nice thing for someone today”) and the other thanking them for their participation, for a total of 14 messages. Control group participants did not have access to the Text Buddy and Happy Genie components.
In addition to the Web-based baseline survey, all participants were asked to complete a brief survey about their program experience, conducted via text message, every other week. We also conducted a Web-based follow-up survey 1 month post-intervention: youth received a link to the survey and two subsequent reminders via text message to facilitate self-completion. Those who did not complete the follow-up survey on their own could choose to complete the survey at school with research staff.
Participants received a US$10 cash incentive for returning the parent permission form, irrespective of whether they received permission. Participants received a US$25 Amazon gift card upon completion of the 1 month post-intervention follow-up survey.
Demographic characteristics of participants in the FGs and CAT, which were enrolled nationally; and the beta test, which was fielded in Illinois, are shown in
FG activities occurred between March and May 2014. Although the Facebook ads generated a high number of impressions (343,148) and unique clicks (2334) after 10 days of advertising, only 13 screeners were received. Of these, four were from eligible youth. Reasons for ineligibility included not owning a cell phone (7/13, 54%), not being enrolled in an unlimited text messaging plan (1/13, 8%), and being too young (1/13, 8%).
The recruitment firm identified 18 participants to take part in FG1 and 19 participants in FG2. Two participants from FG1 and one from FG2 assented and enrolled, but neither completed the FG nor provided a reason for their nonparticipation.
In total, the FG moderators posted 23 threads, each containing between one and five questions. As predicted, participant age was an important process variable: 6th grade youth had difficulty responding to questions. Homeschooled youth did as well.
For example, in response to familiarity with bullying prevention programming at school, a 12-year-old male said: “Im homeshooled so we don't have any of that.” (Note here, and throughout, that youth quotes are presented exactly as they were typed by participants.)
Participants overwhelmingly voted for the program name “BullyDown.” As one participant shared: “I like BullyDown.com the best. It states the purpose of the website and it's easy to remember!” (13-year-old female). Youth also expressed a preference for receiving two to four program text messages per day and felt that more would be too many.
While some youth reported that their schools had not discussed bullying, most youth were able to articulate and also agreed with school definitions of bullying: “Our school considers any type of picking on or hurting someone's feelings bullying and I feel the same.” (12-year-old male). Overall, youth related to the definition of bullying as verbal, physical, and relational forms of aggression that can happen among friends and nonfriends:
Bullying comes in many forms. Physical, mental and cyber are all kinds of bullying. To me bullying is picking on or hurting someone you know can't defend themselves. A person bullies others to feel better or more important about themselves. Being aggressive towards another person is not the same as bullying. To me aggression is more about anger and the anger one feels towards another.
Youth perceptions of victims varied, and in some cases, youth felt that some victims behaved in ways that contributed to their victimization. Youth also recognized that some students who are socially popular engage in bullying, making it more difficult to stand up to these bullies. Because emotion regulation is a core component of bullying prevention, we also asked FG participants about coping strategies they used when stressed or upset. Answers included: reading/writing, listening to music, counting, belly breathing, and talking to siblings and parents.
Based upon the SEL model, we drafted seven modules: (1) communication, (2) coping with stress and problem solving, (3) empathy, perspective-taking, and respect for diversity, (4) recognizing bullying, attitudes toward bullying, and attitudes supportive of aggression, (5) emotion regulation: anger, hostility, and impulsivity, (6) bystanders and intentions to intervene to help others, and (7) resources and support.
Demographic characteristics of BullyDown participants by development activity.
Demographic characteristics | Focus groups (n=37) |
CATa (n=9) |
Beta testb (n=22) |
||||
Sex | |||||||
Male | 18 (48.7) | 6 (66.7) | 8 (36.4) | ||||
Female | 19 (51.4) | 3 (33.3) | 14 (63.6) | ||||
Agec | |||||||
11 years | 1 (2.7) | NA | NA | ||||
12 years | 17 (46.0) | 4 (44.4) | 10 (45.5) | ||||
13 years | 16 (43.2) | 4 (44.4) | 8 (36.4) | ||||
14 years | 3 (8.1) | 1 (11.1) | 4 (18.2) | ||||
Race/ethnicityd | |||||||
Caucasian | 18 (48.7) | 6 (66.7) | 10 (45.5) | ||||
Black/African American | 10 (27.0) | 1 (11.1) | 7 (31.8) | ||||
Asian | 1 (2.7) | 0 (0.0) | 0 (0.0) | ||||
Mixed racial background | 1 (2.7) | 0 (0.0) | 4 (18.2) | ||||
Native American or Alaskan Native | 0 (0.0) | 1 (11.1) | 0 (0.0) | ||||
Hispanic | 7 (18.9) | 1 (11.1) | 4 (18.2) | ||||
Do not want to answer | 0 (0.0) | 0 (0.0) | 1 (4.56) | ||||
Gradec | |||||||
6th grade | 12 (32.4) | NA | NA | ||||
7th grade | 14 (37.8) | 4 (44.4) | 11 (50.0) | ||||
8th grade | 11 (29.7) | 5 (55.6) | 11 (50.0) | ||||
Region | |||||||
Northeast | 9 (24.3) | 2 (22.2) | 0 (0.0) | ||||
South | 12 (32.4) | 3 (33.3) | 0 (0.0) | ||||
Midwest | 8 (21.6) | 2 (22.2) | 22 (100.0) | ||||
West | 8 (21.6) | 2 (22.2) | 0 (0.0) | ||||
Type of residence | |||||||
Urban | 12 (32.4) | 3 (33.3) | Not asked | ||||
Suburban | 15 (40.5) | 3 (33.3) | Not asked | ||||
Rural | 10 (27.0) | 3 (33.3) | Not asked | ||||
Been bulliede | 10 (27.0) | 6 (66.7) | 14 (63.6) | ||||
Bullied someonee | 5 (13.5) | 2 (22.2) | 5 (22.7) |
aContent Advisory Team.
bBeta test participants were enrolled from a partner school in Illinois, as such all participants are from the Midwest region.
cGrade eligibility criterion was modified prior to the CAT to include grades 7 and 8 only.
dHispanic ethnicity was queried as part of race for the focus groups and CAT, and as a separate identity in the beta test. As such, race/ethnicity sums to more than 22 for the latter activity.
eFor the focus groups and CAT, youth were asked if they had ever bullied and if they had ever been bullied. In the beta test, a more complex series of questions were asked, including different ways in which youth could be bullied, the mode through which they could be bullied (eg, in-person, via the Internet), and the place that they could be bullied (eg, at school, on the way to and from school). The timeframe was limited to the past 30 days.
Example BullyDown program messages by SEL component.
SEL Component | BullyDown program text message | ||
Social | |||
Communication | Passive communication is when you don't really share an opinion: “I don't care;” “Whatever you want is fine;” and things like that. |
||
Communication | It is also passive if you don't say anything. If someone teases you and you don't say anything, people may think you are ok with it or that it doesn’t upset you. |
||
Problem-solving | Step 1 in problem-solving is figuring out what the problem is. While this may sound easy, sometimes it can be harder than we think. |
||
Respect for diversity | We're all different. Teens who act or look differently are just being themselves. They want to have friends just like everyone else. |
||
Recognizing bullying | It can be a fine line between bullying and messing around. So how do you know where the line is? |
||
Recognizing bullying | One teen said that it is no longer drama, but becomes bullying when the other person doesn't fight back - or they fight back but it doesn't work. |
||
Attitudes supportive of bullying and aggression | Peer pressure is hard: maybe your friends dare you; or you think your friends will like you better; or your friends are teasing someone and you join in. |
||
Bystanders and intentions to intervene | Good news! You don't have to do it alone. Let's talk about ways you can help kids who are in trouble. |
||
Emotional | |||
Coping with stress | Let’s talk more about what to do when you’re down. Ask for help. Staying quiet doesn't make you stronger, often times it just makes you feel alone. |
||
Empathy and perspective taking | It can be easy to make bad decisions when we're angry. This is really true if we don't try to understand what we're feeling and why. |
||
Emotion regulation: anger, hostility | It's okay to get angry - we all get angry sometimes. It's what you do with that anger that is good or bad. |
||
Emotion regulation: impulsivity | Listening to music, writing about it, or talking to someone can help. Even just sleeping on it can help you see that it is not such a big deal the next day. |
We incorporated quotes, as well as personal experiences shared by youth, from the FGs into program messages. For example, youth responses included:
Bullying is something done to someone weaker then you that you know cant defend themselves. When you go behind the other persons back it usally means your scared to say it to there face.
I saw a boy making fun of a handicapped student right in front of a teacher and they did not say anything, so I did and then they stopped it and the boy who was bullying was sent to the office.
These became adapted into the respective program messages:
Bullying can even be spreading a rumor about someone (true or not). It can happen face-to-face or behind your back. It can be online or through text messaging.
Another teen said: A boy was making fun of a handicapped teen in front of a teacher, but she didn't say anything. So I did. The boy was sent to the office.
In response to a question about how bullying might make a person feel, a participant shared:
it makes you feel badly about yourself and that feeling lasts for a long time. It doesn't go away quickly, even if others try to tell you it's not true. Deep inside you feel like it is true what they are saying. It stays with you and keeps coming back and being in your mind.
This was translated into the following program message:
A girl who was bullied told me: It makes you feel badly about yourself and that feeling lasts for a long time. It doesn't go away quickly……even if others try to tell you it is not true. Deep inside you feel like what they’re saying is true. It stays with you and keeps coming back into your mind.
We conducted the CAT between August and September 2014. Given the success using a recruitment firm during the FGs, we determined this strategy to be the most efficient for subsequent research activities. We excluded 6th graders and homeschooled youth in these subsequent activities because of their difficulty responding to questions during the FGs. The recruitment firm recruited 12 youth for the CAT (
Engagement with the CAT was low in part because of participants’ confusion about the protocol. Both the recruitment firm and the research staff received multiple requests from parents for clarification about the process. This confusion seemed to persist even after research staff explained the process to parents over the phone. Following up with participants and/or their parents was also difficult because messages from research staff often went unanswered. In many cases, the recruitment firm needed to contact parents and/or participants on behalf of research staff to resolve the issue. Response rates may also have been low because the CAT field period overlapped with the start of a new school year. After yielding little feedback, we extended the 7-day deadline by a week. To help invigorate completion, research staff provided participants with examples of the type of feedback that would be informative to program development. Interestingly, two participants requested that the messages be made available via the Internet through Google Docs. One did not have access to Word, and the other requested it for anticipated convenience.
Atypical of our past experience with adolescents [
As shown in
During the 2-day online group discussion, participants generally rated the program modules positively: between 80% (4/5) and 100% (5/5 or 4/4, depending on the number of youth who responded to the poll) of youth liked/strongly liked them. As an exception, 75% (3/4) of youth disliked the problem-solving module. Only one youth provided details on his negative reaction to the module:
I liked the how to deal with stress messages, the tips on what to do. The examples about starting rumors was nothing I would ever do, so did not like that.
When asked about their overall impression of the program, many participants responded positively:
the program is perfect no changes needed.
Less enthusiastic feedback was also received:
some of the messages were too long and boring and some of them were kinda confusing
it was jumping from topic to topic.
Four youth had negative reactions to the tone of some messages. For example, one youth shared:
[the messages] sounded like an adult trying to be ‘cool’ - the messages were too long and wordy and sounded like things my mom would say.
When asked for further detail, youth specifically disliked the use of the phrase “rock star” in one of the messages. Although many participants liked the inclusion of quotes from other teens, one youth questioned the authenticity of the quotes (ie, emotion regulation: anger, hostility message;
The level-up feature was well received, although it too was not immune to criticism. For example, some thought that while the feature was acceptable, the questions needed to be more challenging:
the questions to get to the next level were way too easy. the second try was even more easy. you should probably make it a little harder.
Another participant thought that the questions felt like homework:
… I don't like having to 'earn' levels - it makes it more like a school assignment, not something that should be have to be done. why do you have to pass a level to move on? is there a prize or something to get through the levels. If I was doing this in real life and did not pass a level, I would just stop and not continue anymore.
Regarding program features, participants were enthusiastic about Happy Genie:
This genie feature looks amazing this is just what is needed when their life feels like a bore and nothing is great about it. The feature that it sends them a positive quote is very nice indeed. Overall great feature really perked up my spirits with those quotes i am sure it will do just the same for the participants.
This sentiment was echoed by a 12-year-old female:
I think this feature is great! It could be really helpful for a person whose having a bad day.
Youth also suggested several name alternatives, including Forever Friend, Good Day Genie, and the Happy Doctor.
Example per-message feedback from the Content Advisory Team.
Message type | Specific program message | Participant feedbacka | |||
Communication | Assertive communication is when you’re clear about how you feel or what you want. Say someone is being mean to your friend. You could say: “I think the way you put him down was mean.” | ||||
Recognizing bullying | A teen told me that it's no longer drama, but becomes bullying when the other person doesn't fight back - or she fights back but it doesn't work. | ||||
Attitudes supportive of bullying and aggression | Remember: There are lots of ways to make friends without bullying. If your friends are pressuring you to be mean to others, maybe they're not such good friends. | ||||
Coping with stress | There are lots of ways to deal with stress. You could go on a long walk, play basketball, read a book, go for a run or a bike ride, or even write a poem! | ||||
Empathy and perspective taking | Maybe someone has called you a bully but you didn't know you hurt the person. This can be frustrating. Take a moment to think about why the other person may have felt bullied. | ||||
Emotion regulation: anger, hostility | Imagine you’re in a fight with your friend. To deal with it, you could: stop talking to her. Shove her to show her you're mad. Make up a rumor about her. Or, tell her how you feel. | ||||
Emotion regulation: anger, hostility | A teen told me: “I try to think about the situation before acting. If I can remember to do that, it works. But when I'm angry I usually forget those things.” | ||||
Emotion regulation: impulsivity | You don’t want to say something you’ll regret. Trying to get even with a bully continues the cycle. And it might put you at risk for something unsafe to happen. | ||||
Quote | Great advice: When bad things happen, I know you want to believe they are a joke, but sometimes, life is scary and dark. That is why we must find the light.” - BMO) | ||||
Communication | Assertive communication is when you’re clear about how you feel or what you want. Say someone is being mean to your friend. You could say: “I think the way you put him down was mean.” | ||||
Recognizing bullying | A teen told me that it's no longer drama, but becomes bullying when the other person doesn't fight back - or she fights back but it doesn't work. | ||||
Attitudes supportive of bullying and aggression | Remember: There are lots of ways to make friends without bullying. If your friends are pressuring you to be mean to others, maybe they're not such good friends. | ||||
Coping with stress | There are lots of ways to deal with stress. You could go on a long walk, play basketball, read a book, go for a run or a bike ride, or even write a poem! | ||||
Empathy and perspective taking | Maybe someone has called you a bully but you didn't know you hurt the person. This can be frustrating. Take a moment to think about why the other person may have felt bullied. | ||||
Emotion regulation: anger, hostility | Imagine you’re in a fight with your friend. To deal with it, you could: stop talking to her. Shove her to show her you're mad. Make up a rumor about her. Or, tell her how you feel. | ||||
Emotion regulation: anger, hostility | A teen told me: “I try to think about the situation before acting. If I can remember to do that, it works. But when I'm angry I usually forget those things.” | ||||
Emotion regulation: impulsivity | You don’t want to say something you’ll regret. Trying to get even with a bully continues the cycle. And it might put you at risk for something unsafe to happen. | ||||
Quote | Great advice: When bad things happen, I know you want to believe they are a joke, but sometimes, life is scary and dark. That is why we must find the light.” - BMO) |
aTo maintain participant voice, quotes are verbatim. As such, grammar and spelling errors exist.
Overlapping feedback provided by two or more youth were integrated into the program content where possible. For example, we deleted the word “rock star” and messages deemed particularly confusing (eg, the quote from BMO). We also modified the tone of messages to sound less motherly or adult-like. The message:
You can tell your friend how you feel. It might be hard. And it might not go well. But if you’re in a healthy friendship and you use assertive communication, it could go great!
Subsequently became:
Tell your friend how you feel. It might be hard, and it might not go well. But if you’re in a healthy friendship, and you talk assertively, it might go great!
We also reviewed the messages scheduled for the beginning and end of each program day for their flow and transitions, based on youth feedback.
We chose not to make the level-up questions more challenging as this seemed likely to make the feature further simulate homework. The level-up protocol was already designed to automatically move nonresponsive participants to the next module if they did not respond to the questions after a certain period. We established a plan to closely monitor participant reactions to the level-up feature during the beta test for further indication of acceptability.
We deemed participants’ positive review of Happy Genie as supportive of its acceptability. We renamed the feature “Forever Friend” based on the names suggested by participants and added inspirational messages (eg, A new day = a new beginning) and quotes from famous people (eg, “I’ve failed over and over again in my life. And that is why I succeed.” – Michael Jordan).
Enrollment spanned 3 months and required research staff to visit the partnered middle school four times. Research staff screened 78 youth, 45% (35/78) of whom were eligible. Reasons for ineligibility included: not having a cell phone (21/78, 27%), not or uncertain if they were enrolled in an unlimited text messaging plan (9/78, 12%), and not planning to or uncertain if they would have the same phone number for at least 6 months (21/78, 27%), including the possibility that parents may take away phone privileges as a form of punishment.
Thirty-three eligible youth returned their permission forms, all of whom had parental permission to participate. Twenty-four eligible students provided written assent to participate, of which 22 completed the registration process and were randomized. Of the nine youth who had parental permission but did not enroll, three changed their mind, two moved, one was absent every time the research staff went to the school, and three had problems with phone access (eg, one shared the phone with a parent). Reasons for not completing the registration process included no longer being interested (n=1) and no longer having a cell phone (n=1). Participants were successfully randomized, resulting in 14 students assigned to BullyDown and eight to the control group (
We conducted the beta test between December 2014 and May 2015. No concerns from the school administration or staff were expressed during the field period. Participants encountered some technology difficulties, however. The school’s firewall initially blocked the project enrollment website, but school technology staff quickly resolved the issue. Additionally, many students forgot to bring their phones to verify its compatibility with the program software. These students completed the Web-based baseline survey and were instructed to complete enrollment at home by responding with the word “verify” to the text message verification. Those randomized to the intervention group received a link to the Text Buddy Code of Conduct to read and accept. Some students had trouble completing the tasks, requiring research staff to return to the school to assist them in person.
Thirty-six percent (8/22) of participants completed the Web-based follow-up survey on their own time, and 14 completed it at school. The primary reason for not completing the survey independently was forgetting to do it. One participant thought the survey link did not look legitimate and so was hesitant to click on the link.
During the program delivery period, 86% (12/14) of intervention participants sent at least one message to their Text Buddy (range, 2-52), and 29% (4/14) sent at least one message to Happy Genie (range, 1-8). Half of intervention participants (7/14) responded to all seven weekly level-up questions, and 21% (3/14) responded to two or fewer level-up questions.
Feedback from the biweekly brief survey (
Beta test intervention group participant biweekly qualitative feedback.
Week | Program Message | Feedback per participanta,b | |||
2 | What do you think of the messages? Are they confusing? Boring? Interesting? Fun? Text me back what you think and why. The more details, the better. | ||||
Is there anything about your experience in BullyDown that you'd like me to know at this point? Text your feedback or text “no.” | |||||
4 | What one BullyDown text message sticks out in your mind, and why? This could be any message you've received since the start of the program. | ||||
Is there anything about your experience in the program that you'd like me to know at this point? Text your feedback or text “no”. | |||||
6 | What is one thing that you really like about BullyDown and why? The more detailed you can be, the better. | ||||
And what is one thing that you really do not like about BullyDown or think that we need to make better, and why? Again, detail is helpful. | |||||
What else about your experience in the program would you like me to know at this point? Text your feedback or text “nothing”. | |||||
8 | How would you rate the number of messages that we send you each week? Too few, too many, or just right? Text me what you think. | ||||
Is there anything about the program that you'd like me to know about your experience at this point? Text “no” or text your feedback. | |||||
2 | What do you think of the messages? Are they confusing? Boring? Interesting? Fun? Text me back what you think and why. The more details, the better. | ||||
Is there anything about your experience in BullyDown that you'd like me to know at this point? Text your feedback or text “no”. | |||||
4 | What one BullyDown text message sticks out in your mind, and why? This could be any message you've received since the start of the program. | ||||
Is there anything about your experience in the program that you'd like me to know at this point? Text your feedback or text “no”. | |||||
6 | What is one thing that you really like about BullyDown and why? The more detailed you can be, the better. | ||||
And what is one thing that you really do not like about BullyDown or think that we need to make better, and why? Again, detail is helpful. | |||||
What else about your experience in the program would you like me to know at this point? Text your feedback or text “nothing”. | |||||
8 | How would you rate the number of messages that we send you each week? Too few, too many, or just right? Text me what you think. | ||||
Is there anything about the program that you'd like me to know about your experience at this point? Text “no” or text your feedback. | |||||
2 | What do you think of the messages? Are they confusing? Boring? Interesting? Fun? Text me back what you think and why. The more details, the better. | ||||
Is there anything about your experience in BullyDown that you'd like me to know at this point? Text your feedback or text “no”. | |||||
4 | What one BullyDown text message sticks out in your mind, and why? This could be any message you've received since the start of the program. | ||||
Is there anything about your experience in the program that you'd like me to know at this point? Text your feedback or text “no”. | |||||
6 | What is one thing that you really like about BullyDown and why? The more detailed you can be, the better. | ||||
And what is one thing that you really do not like about BullyDown or think that we need to make better, and why? Again, detail is helpful. | |||||
What else about your experience in the program would you like me to know at this point? Text your feedback or text “nothing”. | |||||
8 | How would you rate the number of messages that we send you each week? Too few, too many, or just right? Text me what you think. | ||||
Is there anything about the program that you'd like me to know about your experience at this point? Text “no” or text your feedback. |
aTo maintain participant voice, quotes are verbatim. As such, grammar and spelling errors exist.
bBlank spaces indicate a lack of response.
c“Forever Friend” refers to the Happy Genie feature.
Moreover, all participants said they would recommend the program to a friend when asked at the end of week 6 (
Feedback from intervention participants suggested that they were enthusiastic about having a Text Buddy. This feature also drew at least one negative response each week, however (
As shown in
Acceptability and feasibility data from pilot test participants.
Youth responses | Control |
Intervention |
||||
Biweekly survey in field | ||||||
(n=6) | (n=10) | |||||
How much are you liking BullyDown? (Week 2) | NAb,c | 8 (80.0) | ||||
How easy or hard has it been to read your texts the *same* day we send them to you? (Week 4) | 5 (83.3) | 8 (80.0) | ||||
(n=5) | (n=8) | |||||
How likely are you to recommend BullyDown to your friends? (Week 6) | 3 (60.0) | 8 (100.0) | ||||
One-month follow-up survey | (n=8) | (n=14) | ||||
Acceptability (Agree/Strongly Agree) | ||||||
I like the program | 7 (87.5) | 12 (85.7) | ||||
I learned things in BullyDown… | ||||||
That will help me not bully others in the future | 7 (87.5) | 14 (100.0) | ||||
That will help me not be bullied by others in the future | 6 (75.0) | 12 (85.7) | ||||
That will help me stop bullying when I see it happening to others | 6 (75.0) | 13 (92.9) | ||||
I do not think people like me should go through the BullyDown program (Disagree/Strongly Disagree) | 7 (87.5) | 10 (71.4) | ||||
BullyDown talked too much about… (disagree/strongly disagree) | ||||||
Feelings | 7 (87.5) | 8 (57.1) | ||||
Bullying | 7 (87.5) | 11 (78.6) | ||||
The text messages were easy to understand | 6 (75.0) | 12 (85.7) | ||||
BullyDown talked about things my friends and I experience in our lives | 6 (75.0) | 11 (78.6) | ||||
Feasibility (disagree/strongly disagree) | ||||||
BullyDown sent too many messages | 7 (87.5) | 10 (71.4) | ||||
I stopped reading the messages by the end of the BullyDown program | 8 (100.0) | 10 (71.4) | ||||
BullyDown got in the way of my daily schedule | 8 (100.0) | 10 (71.4) |
aPercentages reflect those in the extreme categories: agree or strongly agree (4 or 5 on a 5-point Likert scale; 7-10 on a 10-point ordinal scale); disagree or strongly disagree (1 or 2 on a 5-point Likert scale; 1 or 2 on a 5-point ordinal scale).
bThe control group inadvertently did not receive the Week 2 biweekly survey.
cnot asked.
The enrollment process seemed to require in-person facilitation by research staff at a level that is at least commensurate with an in-person intervention. Both interesting and potentially problematic, 33% (11/33) of youth who were eligible and had parental permission lost interest in participating before enrollment. Given the small sample size, this may be an anomaly. Further investigation of this feasibility issue in a larger trial is needed before conclusions can be drawn. Also, some participants shared that they would have been more likely to respond to the follow-up survey if the link had been sent by email instead of text message. Future efforts should consider collecting multiple forms of contact (eg, Facebook, email address).
Despite CAT participants’ expressed interest in the concept of Text Buddy, most beta test participants only sent one or two messages to their buddy. Based upon their feedback, future trials could ideally match Text Buddies who are attending different schools and, when possible, of the same sex and grade level. An interactive guide, either in-person or via the Internet, instructing how to use the feature could also be incorporated into the content. Additionally, only participants with expressed interest in the Text Buddy feature could be matched to reduce the likelihood of unresponsive buddies.
Several intervention youth expressed concern about the intensity and/or timing of the daily messages during the field period but not when it was assessed at 1-month follow-up. This feedback most likely arose on days that had five or six messages, resulting in participants receiving four or five messages across a 4-hour window after school. To ameliorate this, the total number of messages per day in future iterations of the program will be limited to four where possible.
Consistent with feedback from some CAT participants who thought the level-up questions were too easy, one beta test participant thought that the questions should be more meaningful. Given the lack of negative feedback from other beta test participants about the level-up experience and the desire to avoid simulating homework however, more feedback from randomized controlled trial participants will be solicited before making significant changes.
Stepwise development of BullyDown over a 1-year period helped ensure the program was generally well received by middle school students. Findings reveal important insights about developing mHealth interventions for younger youth, particularly those focused on bullying prevention, including the preferred number of messages per day (ie, 2-4 messages), a desire for shorter messages, and the ideal age range (ie, 7th grade and older). In interventions aimed at older adolescents (e.g. 14- to 17-years old), others have noted a similar preference among participants for a low-intensity experience in interventions [
Also contrasting other mHealth adolescent recruitment efforts [
Several additional differences between the experiences in the BullyDown beta test with the implementation of other text messaging–based health behavior change programs should be noted. For example, compared with high school-aged youth [
Although control and intervention participants attended classes side by side, the control arm appears to have been blinded: they were equally likely as the intervention arm to say they liked the program, and a similar percent of youth said they learned valuable things about bullying during the program. This suggests that randomization at the individual-level may not pose a serious threat to contamination; however, results should be replicated in a larger sample size.
Specific program recommendations that emerged from this work may not generalize to other health behavior change topics or youth populations. Although the national recruitment strategy is a strength, FG and CAT participants may not be representative of all youth given they were identified through a recruitment firm. Beta test participants were recruited from a disadvantaged public middle school in Illinois and may not be representative of students in other settings. More generally, youth who have a cell phone and are enrolled in an unlimited text messaging plan may be different from youth who do not meet these criteria. It seems unlikely however that youth who lack a cell phone or are not on an unlimited text messaging plan would opt into such a program if it was publicly available. Additionally, confirming that youth actually read the text messages is not possible beyond their self-report; therefore, actual exposure is unknown.
Increasingly, SEL programs are being implemented in schools across the United States to address a wide-range of problematic behaviors (eg, bullying, delinquency) and to promote academic success. However, the majority of these programs include curricula that range from 15 to 20 or more lessons, requiring significant instructional time and resources that some school districts may not be able to provide. BullyDown harnesses the benefits of text messaging, such as being able to go where youth are and communicating important information in consumable chunks, to offer a bullying prevention program that can be delivered outside of school and without the need for facilitator time or resources. While expensive in time and money, iterative intervention refinement increases the likelihood that the resulting intervention is salient to the target population while retaining its adherence to theory.
Given the positive feedback from beta test participants about the program content and the experience as well as the 100% (22/22) retention rate, the BullyDown mHealth bullying prevention program appears to be both feasible and acceptable for implementation in a middle school setting with 7th and 8th grade youth. Our next step will be to test the intervention in a large randomized controlled trial to see if exposure is associated with reduction in bullying behavior.
content advisory team
focus group
social-emotional-learning
short message service
Drs. Ybarra and Espelage conceived the study. Dr. Ybarra identified the iterative development steps. Dr. Espelage mapped the SEL model to content areas and Dr. Ybarra led the writing of the content, including the integration of findings into the messaging. Drs. Ybarra and Espelage co-moderated the focus group discussions. Dr. Espelage and Ms. Prescott co-moderated the CAT. Ms. Prescott managed participant recruitment and retention activities. All three authors helped finalize the program content and contributed to the fielding of the beta test. The manuscript was drafted by Dr. Ybarra and Ms. Prescott, and significantly edited by Dr. Espelage. All authors approved the final version of the manuscript.
The project described was supported in part by a grant from HopeLab. The content is solely the responsibility of the authors and does not necessarily represent the official views of the HopeLab. We would like to thank the entire study team at the Center for Innovative Public Health Research and the University of Illinois at Urbana-Champaign for their contributions to the research and the participants for their time in the study. We also thank Ms. Emilie Chen for her assistance proofreading the article.
None declared.