Published on 23.04.18 in Vol 6, No 4 (2018): April
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/8155, first published Jun 08, 2017.
A Mobile Gaming Intervention to Increase Adherence to Antiretroviral Treatment for Youth Living With HIV: Development Guided by the Information, Motivation, and Behavioral Skills Model
Background: Highly active combination antiretroviral treatment has been shown to markedly improve the health of HIV-infected adolescents and young adults. Adherence to antiretroviral treatment leads to decreased morbidity and mortality and decreases the number of hospitalizations. However, these clinical achievements can only occur when young persons with HIV are adherent to care. Unfortunately, adolescents and young adults have poorer rates of adherence to antiretroviral medications and poorer rates of retention in care than older adults. Novel and engaging digital approaches are needed to help adolescents and young adults living with HIV be adherent to treatment.
Objective: The aim of this study was to develop an immersive, action-oriented iPhone gaming intervention to improve adherence to antiretroviral medication and treatment.
Methods: Game development was guided by social learning theory, taking into consideration the perspectives of adolescents and young adults living with HIV. A total of 20 adolescents and young adults were recruited from an HIV care clinic in Rhode Island, and they participated in qualitative interviews guided by the information-motivation-behavioral skills model of behavior change. The mean age of participants was 22 years, 60% (12/20) of the participants identified as male, and 60% (12/20) of the sample reported missing a dose of antiretroviral medication in the previous week. Acceptability of the game was assessed with client service questionnaire and session evaluation form.
Results: A number of themes emerged that informed game development. Adolescents and young adults living with HIV desired informational game content that included new and comprehensive details about HIV, details about HIV as it relates to doctors’ visits, and general health information. Motivational themes that emerged were the desire for enhancement of future orientation; reinforcement of positive influences from partners, parents, and friends; collaboration with health care providers; decreasing stigma; and increasing personal relevance of HIV care. Behavioral skills themes centered on self-efficacy and strategies for medical adherence and self-care. On the client service questionnaire, 10 out of the 11 participants indicated they were “satisfied with the game activities,” and 9 out of 11 “would recommend it to a friend.” On the session evaluation form, 9 out of 11 agreed that they “learned a lot from the game.”
Conclusions: We utilized youth feedback, social learning theory (information-motivation-behavioral skills), and agile software development to create a multilevel, immersive, action-oriented iPhone gaming intervention to measure and improve treatment adherence for adolescents and young adults living with HIV. There is a dearth of gaming interventions for this population, and this study is a significant step in working toward the development and testing of an iPhone gaming app intervention to promote adherence to antiretroviral treatment.
Trial Registration: ClinicalTrials.gov NCT01887210; http://clinicaltrials.gov/ct2/show/NCT01887210 (Archived by WebCite at http://www.webcitation.org/6xHMW0NI1)
JMIR Mhealth Uhealth 2018;6(4):e96
According to the Centers for Disease Control and Prevention, young persons aged 13-29 years accounted for 41% of the new HIV infections in the United States in 2015 , and an estimated 99,463 adolescents and young adults in the United States are living with HIV/AIDS (acquired immunodeficiency syndrome) [ ]. Advances in treatments for HIV can allow those infected to manage their HIV infection as a chronic, rather than imminently life-threatening, disease [ ]. However, these achievements can only be made when persons living with HIV take their medications as prescribed and maintain consistent medical care [ - ]. Unfortunately, adolescents and young adults with HIV are the least likely out of any age group to be adherent to care and have a suppressed viral load [ , ].
The percentage of prescribed doses of antiretroviral medications taken by adolescents and young adults ranges from 50% to 75% in the United States [- ], and studies, with adults, adolescents, and children, show that achieving and maintaining an extremely high level of medication adherence (approximately 90%) is needed to obtain the full benefits of antiretroviral treatment (ART) [ , , ]. Many of the barriers to ART adherence are similar for adults and youth; however, there are additional barriers for adolescents and young adults living with HIV [ ]. Developmentally, late adolescence and young adulthood is an age-related period characterized by less inhibition, increased risk-taking, decreased motivation for planning, and less parental monitoring [ , ]. Adolescents and young adults can feel invulnerable to consequences, and this can explain the risk-taking and limit-testing behaviors seen during adolescence. Often, adolescents and young adults do not have fully developed risk assessment skills, impulse control, or organizational abilities. These characteristics are believed to contribute to lower rates of adherence to care among this age group [ - ].
The Efficacy of Digital Interventions
Digital interventions to improve adherence to ART for adolescents and young adults hold particular promise . Adolescents and young adults aged 18-29 years have high rates of mobile phone use with 98% owning a mobile phone or a smartphone [ ]. Technologies, such as the mobile phone, play an increasingly significant role in adolescents’ and young adults’ interpersonal and environmental life, as they communicate information, reinforce cultural norms, and influence personal identity and behaviors [ ]. Furthermore, gaming is popular among youth. In the United States, 99% of teenage males and 94% of teenage females play video games, and 46% of all video gaming occurs on mobile phones or portable devices [ ]. The widespread appeal of digital game playing among all adolescents and young adults creates a unique opportunity to deliver health education during leisure time, outside of the clinic, and in a manner that is cost-effective and easily scalable [ ]. Games can attract and maintain attention, which is a key component for effective behavior change [ ]. Compelling interactive games can expose players to essential health-related content repeatedly and also give players unlimited opportunities to rehearse new skills and receive personalized feedback on health choices made within the game [ , ]. Finally, gaming has been shown to improve motivation for healthy behaviors. Motivation, defined as the process that initiates, guides, and maintains goal-oriented actions, is key to maintaining ART adherence [ , , ].
There is a paucity of data on gaming interventions to improve adherence to ART. However, before the widespread availability of the internet and cell phones, offline games were found (in randomized controlled trials, RCTs) to impact other health behaviors among youth living with asthma, diabetes, and cancer. A diabetes game for children, called Packy and Marlon, indicated that a well-designed, educational video game can be effective in terms of improving diabetes-related self-efficacy (P=.07), communication with parents about diabetes (P=.03), and self-care behaviors (P=.003). These changes occurred after diabetic youth played the game at home for 6 months, compared to a control group of diabetic youth who took home an entertainment video game that had no health content . In a game called Bronkie the Bronchiasaurus, youth engage in play with a dinosaur character with asthma and help him save his homeland while trying to avoid asthma triggers (pollen, cold viruses, dust) and keep his asthma under control. An empirical study showed that playing the game for less than an hour resulted in significant improvements in a player’s asthma knowledge, self-efficacy for asthma self-management, and self-efficacy for talking with friends about asthma [ , ]. Another video game, named Re-Mission, designed for a wide age range of adolescents and young adults (13-29 years) with acute leukemia, lymphoma, and soft-tissue sarcoma showed promising effects as well. Re-Mission was designed as an action-adventure game with the main character or protagonist shooting cancer-causing agents in the bloodstream. Players gain points and strength by adhering to medications in the game fantasy world. In a randomized control study with a 3-month follow-up, 375 male and female participants who played Re-Mission had significantly improved adherence to trimethoprim-sulfamethoxazole (P=.01) and 6-mercaptopurine (P=.002) compared with controls after an average of only 10.7 hours of play. Adherence to trimethoprim-sulfamethoxazole was tracked by electronic pill monitoring devices (n=200), and the proportion of doses taken correctly by those playing Re-Mission was 19% greater than those in the control group. Self-efficacy (P=.01) and knowledge (P=.04) also increased in the Re-Mission game intervention group compared with the control group [ , ]. The successful games reviewed above provided interactive environments where players could improve motivation and engage in behavioral rehearsal [ - , - ].
Gaps in Literature
Despite the promise of digital games, reviews describe that there is a paucity of published or presented abstracts related to gaming for adolescents and young adults living with HIV . Hightow-Weidman et al identified 5 digital games in development to improve ART adherence (from National Institutes of Health’s RePORTER, including our intervention described here). Published outcomes or descriptions of these interventions are sparse [ ]. LeGrand et al has published a description of the development phase of a game entitled Epic Allies. The game is designed to improve ART uptake, engagement in care, and adherence among young men who have sex with men (YMSM) and transgender women who have sex with men. During game play, users can earn medals and tokens for taking medications and reading health-related studies. These tokens can then be used to earn access to fun, non-HIV-related games in the app. Users can interact with one another in the app and send each other positive messages [ ]. We could not find other manuscripts or published descriptions of an iPhone gaming app to improve adherence to ART for adolescents and young adults.
There are published descriptions of gaming interventions targeted to HIV negative youth, who are at risk for acquiring HIV. An evidence-based gaming intervention called PlayForward aims to reduce risk for HIV among at-risk, ethnic, and racial minority adolescents aged 11-14 years. This tablet-based game provides an interactive world using an avatar where players face challenges such as peer pressure to drink alcohol or engage in other risky sexual behaviors. Players can experience how their choices affect their health and are able to go back in time to change their choices to create different, healthier outcomes . The smartphone game SwaziYolo is an interactive game for HIV-negative young adults (18-29 years) living in Swaziland, Africa. In this app, players can practice relationship and health choices while looking for love [ ]. A mobile phone–optimized intervention entitled healthMpowerment is designed to reduce sexual risk behaviors among YMSM. In this intervention, YMSM can acquire reputation points through reading information about HIV, playing sexually transmitted infection-related games, and positive interactions with other users. These points can be used to purchase T-shirts and iPod shuffles [ ].
There are no gaming interventions for older adults living with HIV. However, interventions to improve adherence to ART among older adults have tested the usefulness of less-complex technologies such as electronic reminders and/or pill bottle opening measurements. Reviews show that the most successful interventions couple these less-complex technologies with in-person interventions to improve motivation for treatment [- ]. Measuring adherence alone (through an electronic method such as a micro-electro-mechanical systems cap) or merely reminding patients about pill taking (with an alarm) does not significantly improve adherence in the long term [ , ].
Among older adults living with HIV, there are also promising studies that have examined interactive text messaging to improve ART . A meta-analysis of 8 studies, reporting 9 interventions, shows that text messaging interventions yielded significantly higher adherence than control conditions (odds ratio [OR] 1.39, 95% CI 1.18-1.64). The mean age of participants in the studies was 40 years (range 36-42 years). Sensitivity analyses of intervention characteristics suggested that studies had larger effects when intervention texts were sent less frequently than daily and included personalized communication. Text message interventions among adults were associated with improved viral load and/or CD4+ count (k=3; OR 1.56, 95% CI 1.11-2.20) [ ]. Less data are available on text messaging interventions for younger populations living with HIV. Dowshen et al did report that personalized, interactive, daily short message service reminders were feasible and acceptable among youth living with HIV who were between the ages of 14 and 29 years (mean age 23 years). Participants in that study (N=25) had significantly increased self-report of adherence at 12 and 24 weeks in comparison with baseline (week 0: 74.7; week 12: 93.3, P<.001; week 24: 93.1, P<.001) [ ].
Building on this knowledge, we developed a multilevel gaming intervention to improve adherence to ART for adolescents and young adults aged 18-26 years. This intervention integrates a smart pill bottle cap (that measures adherence) with an immersive iPhone game and personalized text messaging (see). The gaming intervention was informed by the information-motivation-behavioral skills (IMB) theory of learning [ - ]. The iPhone gaming app was designed for participants to experience absorbing action-oriented adventures that increase information about their health (eg, knowledge about HIV treatment, transmission, adherence), improve motivation (eg, action figures experience health benefits of adherence), and build skills (interact with clinicians at appointments, take medications as prescribed). Adherence (measured by the smart pill bottle cap) and game-related text messages are integrated into the gaming intervention. This multilevel approach integrates sound theoretical principles with novel, but intuitive, technology. The aim of this paper is to describe the development of this multilevel iPhone gaming app entitled Battle Viro.
Gaming App Development
Development of Battle Viro was accomplished using iterative and collaborative procedures to fully integrate the clinical experiences of adolescents and young adults living with HIV, academic researchers, and technology partners. Game development was guided by qualitative interviews with a diverse group of adolescents and young adults living with HIV between the ages of 18 and 26 years. Guided by the principles of agile software development , the qualitative interviews and the game/app programming were synergistic. Agile software development aims for continuous design testing and adaptation based on continuous feedback [ ].
As discussed in the Introduction, the adherence gaming app, Battle Viro, was designed to be consistent with the IMB model of health . The IMB model is a well-established conceptualization for improving adherence to antiretroviral medication and engagement in treatment. Nongaming interventions based on IMB have demonstrated efficacy [ , , ]. Reviews have suggested that interventions guided by accepted theories of change are more efficacious than those not driven by theory [ ]. According to the IMB model, health information, motivation, and behavioral skills are the fundamental determinants of health behavior. Information that is directly relevant to adherence and HIV transmission and easily applied to an individual’s cultural and social setting is a prerequisite for success. Motivation to engage in HIV preventive behavior, including personal motivation (favorable attitudes toward adherence) and social motivation (perceived social and cultural support for performing these acts), is also essential for healthy behavior. Finally, skills for performing adherence behaviors and a sense of self-efficacy are critical components. The IMB model, consistent with social learning theory, is broadly applicable and can be used to guide game development and create theoretically consistent gaming content [ , ]. Multiple reviews have demonstrated that behavioral interventions shown to be most efficacious are those tailored for the target population and preceded by formative research to inform intervention development [ - ].
Sample and Recruitment
Males and females, 14 to 26 years old, were eligible for enrollment in the study according to the following criteria: (1) English-speaking, (2) in medical care for HIV and receiving ART, (3) aware of their HIV status as per clinician and clinical record, (4) able to give consent/assent and not impaired by cognitive or medical limitations as per clinical assessment, and (5) adolescent assent and consent of a parent/legal guardian if under 18 years of age or consent of youth if 18 years of age or older. Those who did not meet the above-mentioned inclusion criteria were excluded.
We recruited 20 adolescents and young adults living with HIV for qualitative interviews to guide game development after institutional review board’s approval. Subjects were recruited from a convenience sample in the HIV care clinic in Rhode Island. Subjects were approached by research staff with an institutional review board-approved flyer, and written consent was obtained upon meeting with study staff for the qualitative interview. Overall, 20 subjects were approached over the course of the interviews, and all of them consented and completed the interview. Subjects were recruited until data saturation was achieved and a relative balance in the sample was achieved based on gender, age (<22 vs ≥22), race, and sexual orientation. We were not able to recruit participants younger than 18 years (as originally planned), as the vast majority of patients in our state who are diagnosed and living with HIV are older adolescents and young adults. The mean age of participants was 22 years (range 18-26 years; 8 out of the 20 were older than 22 years). Of the total participants, 60% (6/12) identified as male, and 60% (12/20) completed 12th grade. Of the participants who identified as African American (10/20, 50%), 10% (2/20) identified as Hispanic and 30% (6/20) identified as white. In total, 40% (8/20) identified as heterosexual, 40% (8/20) identified as homosexual, and 20% (4/20) identified as bisexual. Out of these 20 participants, 12 (60%) reported missing a dose of antiretroviral medication in the previous week.
Adaptation of the Information-Motivation-Behavioral Skills Adherence Gaming Intervention
A preliminary storyboard for the IMB gaming app proposed, entitled Battle Viro, was drafted based on the popular Mission Critical Studios game entitled Dr. Nano X: Incredible Voyage Inside the Body . Dr. Nano X is a 5-star-rated mobile game (the highest rating possible) in the iTunes app store and is available on both Android and iPhone. We worked directly with the development team at Mission Critical Studios to develop Battle Viro using Dr. Nano X as a framework. Adapting our game to promote ART adherence from an already existing game greatly decreased the cost of the project. Characters, actions, and IMB messaging were built specifically for Battle Viro; however, we were able to reuse backgrounds, mechanisms of game play/controls, and many sound effects from Dr. Nano X. The adapted gaming app for this project, Battle Viro, takes place inside the human body similar to Dr. Nano.
An initial storyboard was developed for Battle Viro. The storyboard starts with a short narrative movie that explains that the player is becoming miniaturized in order to enter his or her body and destroy attacking viruses and infections (see). Weapons and tools that help the player destroy virus and infections in the body can be earned by taking medications and building an alliance with medical staff. The first level begins on the surface of the skin. If players successfully battle virus, engage with providers, take medication, and make healthy decisions, they move to the next exciting level. As players become expert nanobots, they move onward through the arterial system, the lungs, kidneys, brain, and other new, vibrant, and distinctive organ systems (see - ). Throughout Battle Viro, multiple messages from doctors, clinicians, and friends are integrated into play as the nanobot/protagonist successfully destroys virus and other opponents (ie, opportunistic infections) (see ). At the end of each level or each mission, the player’s score (health status and pill count) is shown (see ). Each level provides new challenges in colorful body organs; however, the mission stays the same: kill the virus and build strength through taking medicine, learning information, improving motivation, and engaging with healthy characters to build skills.
The interview guide consisted of focused, but open-ended, questions aimed at maximizing participant responses (see). Participants were asked about information relevant to medical adherence, motivating factors to adherence, and behavioral skills needed for adherence. The IMB model guided interview content, and participants were also queried about their general gaming experience and their reactions to the storyboard and gaming content. The qualitative interviews and app programming were synergistic [ ]. Therefore, 11 participants were shown storyboards of the game and feedback was elicited, and additional 9 participants were shown an interactive iPhone version of the game as it became available, and feedback was elicited to inform further game design and development.
Information Needed for Adherence
Participants were asked about knowledge and information that have influenced their adherence to medication and engagement in medical appointments. Questions included “What type of information from doctors or friends makes it easier to take medications for HIV?” and “What information makes it easier to come to appointments?” This part of the interview aimed to understand the specific knowledge about HIV and ART that promotes adherence behaviors. For example, probes focused on how appropriate administration, expected side effects, and drug interactions can influence adherence to medication and care (for more examples, see).
Motivation for Adherence
Participants were queried about motivational issues related to adherence with probes such as “I would like to hear about what you think the serious issues are surrounding taking HIV medications and coming to medical appointments” and “What are the things that make it hard to take HIV medications?” This part of the interview was dedicated to understanding both personal and social motivations for adherence. Queries were focused on the positive and negative attitudes toward taking antiretroviral medications, perceived negative effects of nonadherence, and the individual’s perceptions of social support from significant others, family, friends, and medical care providers (for more examples, see).
Behavioral Skills for Adherence
Participants were asked about the behavioral skills needed for adherence. Participants were also asked about their ability to perform necessary adherence-related tasks and his/her perceived self-efficacy for these tasks. Questions included “What are the ways that you remember to take medications and remember your appointments?” and “What events in your life make it harder to remember to take medication? Or remember your appointments?” We also asked participants about strategies for self-reinforcement for adherence over time and across different situations. We asked questions such as “Do you consciously think about your medication schedule on a long-term basis?” and “What strategies have you used or developed to remember medication or appointments based on your activities?” This part of the interview aimed to assess perceived abilities and strategies to store, obtain, and self-cue the use of medications despite challenges and across situations (for more examples, see).
General Gaming Attitudes
Participants were also asked about their general attitudes and experiences with games. Participants were asked questions such as “What games do you, or people you know, play on the cellphone?”; “What types of graphics, avatars, and rewards do you like? And what do you not like?”; and “How are games useful? Do you develop any skills when you play games?” These queries elicited descriptions of popular game activities and attitudes about gaming. The responses were used to make the format and game mechanics of Battle Viro engaging and immersive (for more examples, see).
Battle Viro Storyboard and iPhone Game
Participants were asked for feedback about the storyboard or the iPhone game (once the mobile game was ready) with the probes such as “What was the main point of this activity?,” “What could you learn from this activity?,” “Would you recommend this type of game to your friends?,” and “What is your reaction to having some HIV information and skills in an iPhone game?” After the first version of the game was developed on the iPhone, participants were asked additional and modified probes such as “Is the game easy to navigate and easy to understand?,” “Did any part of the game not work?,” and “Are there other topics that the game should cover that it does not?” Answers to these questions guided the iterative development of the game levels, actions, characters, and graphics (for more examples, see).
Medication Adherence Monitoring Tracking and Game-Related Text Messages
Participants were also asked about the electronic pill monitoring organizers and game-related text messages. We queried participants about a 7-day per week electronic device and a smart pill bottle cap. Both the smart pill bottle cap and the 7-day organizer can electronically monitor, measure, and securely relay adherence pill bottle openings to our research team. Each time a participant opens his or her smart cap organizer, this information can be wirelessly relayed to a secure network. Our gaming intervention is designed so that, if a participant misses a dose, a message is sent from the pill dispenser to study the investigator’s database on a secure server. Study investigators can then send a game graphic with an adherence-related text message to the participant. Messages were designed to encourage players if a dose was missed with phrases such as “Missing you” and “Get in the game.” If doses were taken on time, participants would receive texts with game messages that were congratulatory such as “Great job in battle” and “You are fighting well!!” Low-cost programs exist that allow text messages to be sent automatically, without research staff involvement, based on wireless adherence readings from smart pill caps or 7-day organizers. However, at this time, integrating the game-related graphic into the adherence-based text message is costlier than research staff effort to send the messages individually. Therefore, for this stage of research (game development and an upcoming small exploratory RCT), research staff will be texting participants. For a larger RCT, the cost of programming automated text messages with game graphics would be reassessed, as the technology would become scalable.
Quantitative Feasibility and Acceptability Data
After the development of the first version of Battle Viro, 9 of the 20 participants played the game on an iPhone and provided both qualitative and written/quantitative feedback. Quantitative feedback was collected using adapted versions of the client service questionnaire (CSQ) and the session evaluation form (SEF). The SEF contains 13 items that assess the feasibility and perceived utility of the game. For example, the SEF states “I will be able to apply what I learned from this game in my life” (for which the response options are 1=“Strongly agree”; 2=“Agree”; 3=“Disagree”; and 4=“Strongly disagree”). The CSQ consists of 8 items that assess general satisfaction with the game. An example query from the CSQ is “In an overall, general sense, how satisfied are you with the amount of activities in the game?” (for which the response options are 4=“Very satisfied”; 3=“Mostly satisfied”; 2=“Indifferent or mildly dissatisfied”; and 1=“Quite dissatisfied”).
Participant consent and interviews were conducted in a private room located in the HIV clinic. Interviews were conducted by either an MD (psychiatrist) or a PhD (psychologist) with support from a trained research assistant. The research staff who conducted interviews did not provide medical or clinical services in the HIV clinic. Interviews lasted between 45 and 60 min and were digitally recorded. Because we adapted our gaming intervention from a game that was already developed (Dr. Nano X), the system and the framework (eg, code, database, design) were already in place at the beginning of the project. Adaptations to the game occurred as themes emerged from the interviews. The qualitative interviews and game development happened concurrently . This process allowed for continuous game design changes and improvements based on participant feedback and emerging themes. As part of the iterative process, biweekly meetings were held with the programmers to discuss all adaptations, including changes to content, game graphics framework, and game messaging.
Trained research assistants transcribed verbatim the digital audio recordings of each interview. Then the MD- or PhD-level research team member reviewed the transcripts with the digital recording for accuracy. Qualitative data analysis followed the tenets of thematic analysis, which consisted of sequential steps [, ], and interviews continued until data saturation was achieved. The research team familiarized themselves with the data, reviewing each transcription. Next, the research team met weekly and generated a list of codes as they emerged. The team generated a thematic table of the analyses and checked the extent to which the emerging themes reflected the coded data [ , ]. The team grouped the themes under the general categories of the interview guide (ART information, ART motivation, ART behavioral skills, general game attitudes, and reactions to Battle Viro). Themes were examined in their relationship to perceived utility of the game and for factors that would improve or detract from the game’s impact. Team discussion and interviews continued until discrepancies were resolved.
Participant responses on the CSQ and SEF were entered into an Excel file, and responses were verified with a second entry. Categorical response frequencies were calculated for each item of both scales. General acceptability of the intervention is illustrated using individual items from the scales. CSQ items are reported using the proportion of participants endorsing “satisfaction” with the intervention (response options “Very satisfied” and “Mostly satisfied” were combined). SEF items are reported using the proportion endorsing “agreement” with feasibility and utility of the game (response options “Strongly agree” and “Agree” were combined).
Reactions to Battle Viro Storyboard and iPhone
A total of 20 qualitative interviews were completed. Of the 20 participants, 11 were shown the storyboard of the gaming intervention during qualitative interviews. After feedback on the storyboard from these 11 participants, the preliminary iPhone game was developed directly from the storyboard. Then, the other 9 participants were interviewed after seeing and playing the game on the iPhone. Interviews were conducted until data saturation was achieved. Interviews from both the storyboard and iPhone game revealed a number of themes that guided game development. Participants desired informational game content that included new and comprehensive details about HIV, details about HIV as it relates to doctors’ visits, and general health information. Motivational themes that emerged were the desire for enhancement of future orientation; reinforcement of positive influences from peers, partners, and friends; collaboration with health care providers; decreasing stigma; and increasing personal relevance of HIV care. Behavioral skills themes centered on self-efficacy and strategies for medical adherence and self-care (see).
highlights the barriers and facilitators to adherence expressed by our participants and the corresponding gaming action or message that was adapted or used to enhance facilitators or challenge barriers. also includes general gaming attitudes that influenced the development of Battle Viro and specific reactions to the Battle Viro storyboard and iPhone game. In addition to the themes in , participants who played the game on the phone said that important gaming characteristics included directly destroying HIV in game play, improving health by taking pills, a prologue/introduction with a dramatic voice-over, and images that were HIV-relevant. Participants wanted levels that become increasingly difficult (for a sense of accomplishment). Participants did not want HIV in the title of the game due to concerns about privacy and stigma but wanted to fight HIV in the game action. Participants also commented that their older friends (at least to age 26) frequently played iPhone games and those who also had HIV would like and benefit from this product. For example, a 24-year-old black male participant said, “I like the progression through organ systems.” A 19-year-old white male participant said, “It was cool fighting what’s inside the body and shooting and killing the HIV viruses.” An 18-year-old black female stated, “I liked taking pills and fighting HIV; it mimicked real-life experience.” A 25-year-old Hispanic male stated, “It was cool that I am playing a game about HIV, that it was like tailored to me,” “I think my other friends who are positive would like this game,” and “The sound effects and music were cool” (see ).
The game was iteratively changed as comments were received that indicated a need for alteration. For example, facts about HIV and adherence were made more sophisticated when multiple participants gave feedback such that they knew most of the information given in the game, and they wanted more detailed information about side effects in the game. Many participants also asked for information about general health and substance use. A representative comment was from a 19-year-old white male who said, “I think there should be facts in the game about other health stuff, about smoking, exercise, and diet.” Many participants also wanted more guidance through the levels. For example, a 25-year-old Hispanic female participant stated, “I would like better orientation to the levels,” and an 18-year-old black male said, “There needs to be instructions or hints when it gets hard” (see).
|IMB construct and themes||Resulting game adaptations or actions|
|New and comprehensive details of HIV||Game includes complex and realistic information about opportunistic infections and HIV. Participants fight off infections in each organ. Opportunistic infections are graphically represented. Facts about HIV, CD4 counts, immunity, and viral loads are imparted at every level. HIV is pictured.|
|HIV as it relates to doctors\' visits||Terms and verbiage often used at doctors’ visits are used and defined in the game frequently.|
|General health information||Participants in game receive messages about how exercise and healthy eating also effects health. Participants also receive messages about avoiding cigarettes and illicit substances throughout each level.|
|Enhancement of future orientation||Messages about staying alive for family, friends, and children scroll through game. As gaming participant takes more pills, and builds more health, they are able to move through levels, receive more artillery, and have more success.|
|Personal relevance of HIV care||Participants are shrunken down to enter into their own body in order to fight HIV. Gaming participants see how HIV affects their organs during play.|
|Collaborating with health care providers||Throughout the game, the participant has to partner with doctors to advance to the next level, build strength, and collect artillery.|
|Reinforcement of influences from peers, partners, and friends||Scrolling messages remind gamers that staying alive for partners, friends, and family is meaningful for themselves and loved ones in their lives.|
|Decreasing stigma||Participant is empowered to kill HIV and feel stronger with each healthy decision. Adherence to care is valued as healthy, not as a consequence of being sick.|
|Self-Efficacy for medical adherence and self-care||Solving problems and collecting pills or swallowing pills, in the game leads to higher “Immune Status,” more health, and more artillery. This leads to more game play. Perseverance throughout levels leads to success in game.|
|Strategies for medical adherence and self-care||Scrolling messages encourage participants to use 7 day pill organizers, schedule routine doctors’ appointments, and ask providers/doctors questions about topics relevant to them.|
|General gaming attitudes|
|Desire for games with levels, sound effects, colorful graphics. Ability to earn points in game, and choose avatars||Levels/ organ systems become increasingly difficult (for a sense of accomplishment). Background music, sound effects, and dramatic voice-overs included. Colorful graphics are included and change often. Choice of avatars is available. Participants earn points in game by swallowing pills.|
|Reactions to Battle Viro|
|Desire for game action that is realistic with relevant info about HIV.|
Concerns about stigma.
|Participants can directly destroy HIV in game play, and graphics look like HIV. Participants improve health, and gain points in game by taking virtual pills. Participants liked progression through organ systems, with info about HIV that is pertinent to that organ system. Participants learn health facts about HIV that are complex (ie, information about opportunistic infections) during play.|
Monitoring Pill Bottle Opening and Game-Related Text Messages
We asked participants about the text messages with gaming graphics and the use of a smart pill cap that measured adherence. During the interviews, we demonstrated how openings of the pill bottle were measured wirelessly, and we showed participants sample adherence-informed text messages. When looking at the smart pill cap, an 18-year-old black male participant stated, “It’s cool how it links with game,” and “It’s awesome that there is a bottle that knows what you are doing.” A 19-year-old Hispanic female participant stated, “I hate this pill bottle cap, it’s clunky.” A 26-year-old white male described, “It was annoying because I can’t just carry it; it’s too big.” Multiple participants stated they would rather use their 7-day organizer. For example, a 21-year-old black female stated, “If you gave this to me, I would never use it; I would just open it every time I took a pill out of my normal 7-day organizer.” A 22-year-old black male said, “I would not use this because I would have to empty all my different pills into the same bottle, I like a daily organizer better.” Participants were shown text messages that corresponded to adherence data from the smart pill cap. Participants liked the proposed text messages and an 18-year-old black female described, “These messages will remind me to take my medications.” A 22-year-old Hispanic male stated, “I like the pictures” and “the texts seemed upbeat and cheerful.” A 25-year-old black male participant described, “These texts are good and they make me kinda want to play the game again,” and “I am glad they did not say HIV in them.” Of the participants, 3 described that texts “that always say the same thing are boring” (25-year-old black male, 23-year-old black female, and a 19-year-old black male),” and an 18-year-old African American male stated, “I would like more messages to have more about the game.”
Acceptability and Feasibility
CSQ and SEF scores were available from participants who played the game on the iPhone for 45-50 min. In addition, 90% (10/11) of the participants were satisfied with the activities in the game; 82% (9/11) learned a lot from this game; 73% (8/11) thought the game was well organized; 82% (9/11) felt game topics were interesting; 82% (9/11) felt they would recommend the game to a friend; 64% (7/11) felt game topics stimulated their interest in the material; 55% (6/11) felt that game topics were relevant to their lives; and 55% (6/11) felt they were able to do the activities in the game.
The gaming intervention was improved based on the above acceptability and feasibility feedback from the CSQ and SEF and also on the feedback from the iterative, qualitative interviews (see). Specifically, game play was made easier with written messages and hints throughout each level on how to move forward. We also improved narrated instructions at the beginning of each level to assist players. General health facts about smoking, eating healthy, and avoiding substances such as drugs and alcohol were incorporated into the game. To improve text messages, we included emojis and designed 10 different text messages utilizing phrases based on participant feedback. To improve on the electronic device used for measurement of medication adherence, we moved from a smart bottle pill cap to an electronic 7-day organizer made by Wisepill.
In this project, we utilized qualitative interviewing, focused by social learning theory (IMB), to create an iPhone gaming intervention to measure and improve treatment adherence for HIV-infected adolescents and young adults [- , ]. A number of themes emerged through qualitative interviews with youth that informed game development. We found that youth desired informational game content that included comprehensive details about HIV, doctors’ visits, and general health information. Motivational themes or findings that emerged were the desire for enhancement of future orientation; the need for reinforcement of positive influences from peers, partners, and friends; and the promotion of collaboration with health care providers. Motivational themes also included decreasing stigma and increasing personal relevance of HIV care. Behavioral skills themes or findings centered around self-efficacy and strategies for medical adherence and self-care.
Using the IMB theory in the development of this game ensured that the intervention was informed by decades of prevention research. This study demonstrates that qualitative assessment, social learning theory, and agile software development can complement each other and are important components to the development of a culturally tailored and clinically relevant app. Participant data were used throughout the development of the game and informed the informational, motivational, and behavioral skill-building components of the game. Using a storyboard provided the research team with opportunities to share concept models with participants early on in the design process, and gather feedback with respect to necessary modifications. Sharing the iPhone game with participants as it was developed also allowed for necessary, incremental improvements. Adolescents and young adults living with HIV provided key qualitative insights with respect to the content and design and process of the game. Culturally tailored games that are informed by those who will use them have more potential for effective integration and uptake in clinical settings.
Although iPhone games are pervasive in popular culture, few gaming apps have been developed to improve health outcomes for persons living with chronic illnesses. Findings of this study highlight several important barriers and facilitators to adherence to medication and treatment for young adults and adolescents living with HIV. Mobile interventions have the potential to reinforce skills learned in the clinic and require fewer resources to deliver patient-centered, evidence-based interventions . Furthermore, apps and mobile phone games have the potential to engage adolescents in interventions who otherwise may not be willing or able to participate in prevention programs.
Gaming and mobile apps also have the potential to advance the delivery of information and promote healthy decision making in disproportionately affected populations, including disadvantaged urban and minority youth who often have less access to medical care and support . National data from Pew Research Center indicate that younger, ethnic and racial minority populations use smartphones frequently, and some data show that African American youth are more likely to be mobile phone users than their white peers [ ]. The adolescents and young adults in this study repeatedly expressed having access to, and familiarity with, iPhones. This widespread use of iPhones facilitates the uptake of gaming apps in clinical populations. Therefore, mobile technologies, such as smartphone games and apps, have a great potential to enhance medical care for populations who are disproportionately affected by HIV and other sexually transmitted infections.
Findings should be interpreted in light of study limitations. First, our participants were recruited from a single HIV clinic in New England. This clinic may not be representative of all HIV clinics in the United States or internationally. Therefore, the generalizability of the data collected to inform the development of the app is unknown and may be limited. Second, this study focused on adolescent and young adult patient perspectives. It may be equally important to integrate clinician and caregiver perspectives into the game. In the future, including friends and social networks into the app/gaming prevention programs could be novel and effective. Perspectives of family, friends, and clinicians could also lead to a more robust understanding of barriers and facilitators to adherence to medication and treatment for those living with HIV. Therefore, future research could examine the utility of integrating feedback from clinicians, caregivers, and friends into the gaming app. Finally, this app was developed for the iPhone. Development of the app for Android devices could allow for greater availability of the game and could be a forthcoming step in the future phases of research.
This study is a significant step in working toward the development and testing of an iPhone gaming app intervention to promote adherence to ART. The long-term goal of this research program is to test our mobile game, Battle Viro, in a randomized trial and, if effective, disseminate the intervention to other clinical sites. There are many advantages to using newer interactive technology to improve adherence, rather than traditional face-to-face counseling, including scalability, efficiency, and cost-effectiveness. As electronic games are highly appealing to adolescents and young adults , they are a natural opportunity to deliver health education during leisure time and outside of the clinic [ , , - ]. Games can attract and maintain attention, which is a key component for effective behavior change. Compelling interactive games can expose players to essential health-related content thousands of times and also give players unlimited opportunities to rehearse new skills and receive personalized feedback on health choices made within the game [ , ]. We are not aware of other adherence interventions that integrate medication adherence monitoring technology, text messaging, and a theoretically informed game to improve information, motivation, and behavioral skills for ART adherence. An intervention with these components may empower and engage HIV-infected adolescents and young adults, aid overburdened clinics, and result in improvements in health for youth.
This publication was made possible with help from the Lifespan/Tufts/Brown Center for AIDS Research. The project described was supported by grant number RO1 HD074846 (PI: Brown) from National Institute of Child Health and Human Development. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Child Health and Human Development.
Conflicts of Interest
Multimedia Appendix 1
The iPhone gaming intervention.PNG File, 82KB
Multimedia Appendix 2
Short narrative movie at the beginning of game.PDF File (Adobe PDF File), 57KB
Multimedia Appendix 3
Players can design and individualize their game character.PDF File (Adobe PDF File), 139KB
Multimedia Appendix 4
Players can improve their immune status by picking up pills in the arteries and other organs.PDF File (Adobe PDF File), 45KB
Multimedia Appendix 5
Examples of gaming environments: the kidney, liver, and brain levels.PDF File (Adobe PDF File), 199KB
Multimedia Appendix 6
Answering questions with allied doctors, and building knowledge, helps each player successfully move to the next level or area of the body.PDF File (Adobe PDF File), 79KB
Multimedia Appendix 7
Summary of points earned at the end of each level.PDF File (Adobe PDF File), 73KB
- Centers for Disease Control and Prevention. Diagnoses of HIV infection in the United States and dependent areas, 2015. HIV surveillance report 2016;27.
- Patel K, Hernán MA, Williams PL, Seeger JD, McIntosh K, Van DRB, et al. Long-term effectiveness of highly active antiretroviral therapy on the survival of children and adolescents with HIV infection: a 10-year follow-up study. Clin Infect Dis 2008 Feb 15;46(4):507-515. [CrossRef] [Medline]
- Watson DC, Farley JJ. Efficacy of and adherence to highly active antiretroviral therapy in children infected with human immunodeficiency virus type 1. Pediatr Infect Dis J 1999 Aug;18(8):682-689. [Medline]
- Resino S, Resino R, Maria BJ, Micheloud D, Gutiérrez MDG, de JMI, et al. Clinical outcomes improve with highly active antiretroviral therapy in vertically HIV type-1-infected children. Clin Infect Dis 2006 Jul 15;43(2):243-252. [CrossRef] [Medline]
- Gibb DM, Goodall RL, Giacomet V, McGee L, Compagnucci A, Lyall H, et al. Adherence to prescribed antiretroviral therapy in human immunodeficiency virus-infected children in the PENTA 5 trial. Pediatr Infect Dis J 2003 Jan;22(1):56-62. [CrossRef] [Medline]
- Hogg RS, Heath K, Bangsberg D, Yip B, Press N, O'Shaughnessy MV, et al. Intermittent use of triple-combination therapy is predictive of mortality at baseline and after 1 year of follow-up. AIDS 2002 May 03;16(7):1051-1058. [Medline]
- Van DRB, Lee S, Johnson GM, Wiznia A, Mohan K, Stanley K, et al. Reported adherence as a determinant of response to highly active antiretroviral therapy in children who have human immunodeficiency virus infection. Pediatrics 2002 Apr;109(4):e61. [Medline]
- Simoni JM, Frick PA, Pantalone DW, Turner BJ. Antiretroviral adherence interventions: a review of current literature and ongoing studies. Top HIV Med 2003;11(6):185-198 [FREE Full text] [Medline]
- Bain-Brickley D, Butler LM, Kennedy GE, Rutherford GW. Interventions to improve adherence to antiretroviral therapy in children with HIV infection. Cochrane Database Syst Rev 2011 Dec 07(12):CD009513. [CrossRef] [Medline]
- Lima VD, Harrigan R, Murray M, Moore DM, Wood E, Hogg RS, et al. Differential impact of adherence on long-term treatment response among naive HIV-infected individuals. AIDS 2008 Nov 12;22(17):2371-2380. [CrossRef] [Medline]
- Harrigan PR, Hogg RS, Dong WWY, Yip B, Wynhoven B, Woodward J, et al. Predictors of HIV drug-resistance mutations in a large antiretroviral-naive cohort initiating triple antiretroviral therapy. J Infect Dis 2005 Feb 01;191(3):339-347. [CrossRef] [Medline]
- MacDonell K, Naar-King S, Huszti H, Belzer M. Barriers to medication adherence in behaviorally and perinatally infected youth living with HIV. AIDS Behav 2013 Jan;17(1):86-93 [FREE Full text] [CrossRef] [Medline]
- Haynes RB, Ackloo E, Sahota N, McDonald HP, Yao X. Interventions for enhancing medication adherence. Cochrane Database Syst Rev 2008(2):CD000011. [CrossRef] [Medline]
- Belzer ME, Fuchs DN, Luftman GS, Tucker DJ. Antiretroviral adherence issues among HIV-positive adolescents and young adults. J Adolesc Health 1999 Nov;25(5):316-319. [Medline]
- Naar-King S, Templin T, Wright K, Frey M, Parsons JT, Lam P. Psychosocial factors and medication adherence in HIV-positive youth. AIDS Patient Care STDS 2006 Jan;20(1):44-47. [CrossRef] [Medline]
- Hightow-Weidman LB, Muessig KE, Bauermeister JA, LeGrand S, Fiellin LE. The future of digital games for HIV prevention and care. Curr Opin HIV AIDS 2017 Sep;12(5):501-507. [CrossRef] [Medline]
- Anderson M. Pew Research Center. Technology device ownership. Pew Research Center 2015 Oct 2015 Oct 1:2015.
- Lenhart A, Duggan M, Perrin A, Stepler R, Rainie L, Parker K. Teens, social media & technology overview 2015. Pew Research Center 2015 Apr 2015 Apr.
- Lenhart A, Kahne J, Middaugh E, Macgill A, Evans C, Vitak J. Teens, videogames, and civics. Pew Internet & American Life Project 2008 Sep 16:2008.
- Baranowski T, Buday R, Thompson DI, Baranowski J. Playing for real: video games and stories for health-related behavior change. Am J Prev Med 2008 Jan;34(1):74-82 [FREE Full text] [CrossRef] [Medline]
- Schunk DH. Vicarious Influences on Self-Efficacy for Cognitive Skill Learning. Journal of Social and Clinical Psychology 1986 Sep;4(3):316-327. [CrossRef]
- Thompson D, Baranowski T, Buday R, Baranowski J, Thompson V, Jago R, et al. Serious Video Games for Health How Behavioral Science Guided the Development of a Serious Video Game. Simul Gaming 2010 Aug 1;41(4):587-606 [FREE Full text] [CrossRef] [Medline]
- Wideman R, Owston R, Brown C, Kushniruk A, Ho F, Pitts K. Simul Gaming(1)?30. 2007. Unpacking the potential of educational gaming: A new tool for gaming research URL: https://doi.org/10.1177/1046878106297650%5C [accessed 2018-04-06] [WebCite Cache]
- Brown SJ, Lieberman DA, Germeny BA, Fan YC, Wilson DM, Pasta DJ. Educational video game for juvenile diabetes: results of a controlled trial. Med Inform (Lond) 1997;22(1):77-89. [Medline]
- Homer C, Susskind O, Alpert HR, Owusu MS, Schneider L, Rappaport LA, et al. An evaluation of an innovative multimedia educational software program for asthma management: report of a randomized, controlled trial. Pediatrics 2000 Jul;106(1 Pt 2):210-215. [Medline]
- Shames RS, Sharek P, Mayer M, Robinson TN, Hoyte EG, Gonzalez-Hensley F, et al. Effectiveness of a multicomponent self-management program in at-risk, school-aged children with asthma. Ann Allergy Asthma Immunol 2004 Jun;92(6):611-618. [CrossRef] [Medline]
- Cole S, Kato P, Marin-Bowling V, Dahl G, Pollock B. Clinical trial of Re-Mission: A video game for young people with cancer. Cyberpsychology & Behavior ? 2006;9:666.
- Kato P, Cole S, Bradlyn A, Pollock B. A video game improves behavioral outcomes in adolescents and young adults with cancer: A randomized trial. Pediatrics ? 2008;122:317.
- LeGrand S, Muessig KE, McNulty T, Soni K, Knudtson K, Lemann A, et al. Epic Allies: Development of a Gaming App to Improve Antiretroviral Therapy Adherence Among Young HIV-Positive Men Who Have Sex With Men. JMIR Serious Games 2016 May 13;4(1):e6 [FREE Full text] [CrossRef] [Medline]
- Fiellin LE, Kyriakides TC, Hieftje KD, Pendergrass TM, Duncan LR, Dziura JD, et al. The design and implementation of a randomized controlled trial of a risk reduction and human immunodeficiency virus prevention videogame intervention in minority adolescents: PlayForward: Elm City Stories. Clin Trials 2016 Aug;13(4):400-408 [FREE Full text] [CrossRef] [Medline]
- Lukhele B, Musumari P, El-Saaidi C, Techasrivichien T, Suguimoto S, Kihara M, et al. Efficacy of mobile serious games in increasing HIV risk perception in Swaziland: a randomized control trial (SGprev Trial) research protocol. JMIR Res Protoc 2016 Oct/Dec 2016 Oct;5(4):a.
- Baltierra NB, Muessig KE, Pike EC, LeGrand S, Bull SS, Hightow-Weidman LB. More than just tracking time: Complex measures of user engagement with an internet-based health promotion intervention. J Biomed Inform 2016 Feb;59:299-307. [CrossRef] [Medline]
- Hugen PWH, Langebeek N, Burger DM, Zomer B, van LR, Schuurman R, et al. Assessment of adherence to HIV protease inhibitors: comparison and combination of various methods, including MEMS (electronic monitoring), patient and nurse report, and therapeutic drug monitoring. J Acquir Immune Defic Syndr 2002 Jul 01;30(3):324-334. [Medline]
- Farmer KC. Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clin Ther 1999 Jun;21(6):1074-90; discussion 1073. [CrossRef] [Medline]
- Wise J, Operario D. Use of electronic reminder devices to improve adherence to antiretroviral therapy: a systematic review. AIDS Patient Care STDS 2008 Jun;22(6):495-504. [CrossRef] [Medline]
- Mannheimer SB, Morse E, Matts JP, Andrews L, Child C, Schmetter B, et al. Sustained benefit from a long-term antiretroviral adherence intervention. Results of a large randomized clinical trial. J Acquir Immune Defic Syndr 2006 Dec 01;43 Suppl 1:S41-S47. [CrossRef] [Medline]
- Dunbar PJ, Madigan D, Grohskopf LA, Revere D, Woodward J, Minstrell J, et al. A two-way messaging system to enhance antiretroviral adherence. J Am Med Inform Assoc 2003;10(1):11-15 [FREE Full text] [Medline]
- Finitsis DJ, Pellowski JA, Johnson BT. Text message intervention designs to promote adherence to antiretroviral therapy (ART): a meta-analysis of randomized controlled trials. PLoS One 2014;9(2):e88166 [FREE Full text] [CrossRef] [Medline]
- Dowshen N, Kuhns LM, Johnson A, Holoyda BJ, Garofalo R. Improving adherence to antiretroviral therapy for youth living with HIV/AIDS: a pilot study using personalized, interactive, daily text message reminders. J Med Internet Res 2012;14(2):e51 [FREE Full text] [CrossRef] [Medline]
- Fisher W, Williams S, Fisher J, Malloy T. Understanding AIDS risk behavior among sexually active urban adolescents: an empirical test of the information-motivation-behavioral skills model. AIDS Behav 1999 Mar 1999 Mar;3(1):23. [CrossRef]
- Fisher JD, Fisher WA, Misovich SJ, Kimble DL, Malloy TE. Changing AIDS risk behavior: effects of an intervention emphasizing AIDS risk reduction information, motivation, and behavioral skills in a college student population. Health Psychol 1996 Mar;15(2):114-123. [Medline]
- Fisher JD, Fisher WA, Amico KR, Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol 2006 Jul;25(4):462-473. [CrossRef] [Medline]
- Dybå T, Dingsøyr T. Empirical studies of agile software development: A systematic review. Information and Software Technology 2008 Aug;50(9-10):833-859. [CrossRef]
- Paperny DM, Starn JR. Adolescent pregnancy prevention by health education computer games: computer-assisted instruction of knowledge and attitudes. Pediatrics 1989 May;83(5):742-752. [Medline]
- Starace F, Massa A, Amico KR, Fisher JD. Adherence to antiretroviral therapy: an empirical test of the information-motivation-behavioral skills model. Health Psychol 2006 Mar;25(2):153-162. [CrossRef] [Medline]
- Simoni JM, Montgomery A, Martin E, New M, Demas PA, Rana S. Adherence to antiretroviral therapy for pediatric HIV infection: a qualitative systematic review with recommendations for research and clinical management. Pediatrics 2007 Jun;119(6):e1371-e1383 [FREE Full text] [CrossRef] [Medline]
- Bandura A. Social foundations of thought and action: A Social cognitive theory. In: Englewood Cliffs, NJ: Prentice Hall. NJ: Prentice Hall; 1986:A.
- Romero LM, Galbraith JS, Wilson-Williams L, Gloppen KM. HIV prevention among African American youth: how well have evidence-based interventions addressed key theoretical constructs? AIDS Behav 2011 Jul;15(5):976-991. [CrossRef] [Medline]
- Resnicow K, Diiorio C, Davis R. Culturedevelopment of HIV preventiontreatment programs. In: Indgar TM, Noar SM, Freimuth VS. editors. Communication perspectives for Aids in the 21st Century?. Mahwah, NJ: Lawrence Erlbaum Associates; Aug 19, 2007:220.
- Bailey J, Mann S, Wayal S. Sexual health promotion for young people delivered via digital media: a scoping review. Southampton (UK): NIHR Journals Library; 2015.
- Mission CS. Nano X: Incredible Voyage Inside The Body. Dr URL: https://www.youtube.com/watch?v=lyHzSZFzU1Q [accessed 2017-06-06] [WebCite Cache]
- Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006 Jan;3(2):77-101. [CrossRef]
- Arya M, Kumar D, Patel S, Street RL, Giordano TP, Viswanath K. Mitigating HIV health disparities: the promise of mobile health for a patient-initiated solution. Am J Public Health 2014 Dec;104(12):2251-2255. [CrossRef] [Medline]
- Guest G, MacQueen K, Namey E. Applied Thematic Analysis. In: Thousand Oaks, SAGE Publications, Inc. Thousand Oaks, California: SAGE Publications; 2012:9781412971676.
- Gerbert B, Berg-Smith S, Mancuso M, Caspers N, McPhee S, Null D, et al. Using innovative video doctor technology in primary care to deliver brief smoking and alcohol intervention. Health Promot Pract 2003 Jul;4(3):249-261. [Medline]
- Smith A, McGeeney K, Duggan M, Rainie L, Keeter S. The Smartphone Difference. Pew Research Center 2015 Apr 01:2015.
- Raney A, Smith J, Baker K. Adolescentsthe appeal of video games. In: Vorderer P, Bryant J. editors. Playing video games: Motives, responses and consequences. Mahwah, NJ: Lawrence Erlbaum; 2006:165.
- Lee K, Peng W. What do we know about socialpsychological effects of computer games? A comprehensive review of the current literature. In: Vorderer P, Bryant J. editors. Playing video games: Motives, responses and consequences. Mahwah, NJ: Lawrence Erlbaum; 2006:325.
- Lieberman D. Interactive video games for health promotionffects on knowledge, self-efficacy, social support,health. Intreet RL, Gold WR, Manning T. editors. Health promotion and interactive technology: Theoretical applications and future directions. Mahwah, NJ: Lawrence Erlbaum; 1997:E-120.
- Lieberman D. What can we learn from playing interactive games? In: Vorderer P, Bryant J. editors. Playing video games: Motives, responses and consequences. Mahwah, NJ: Lawrence Erlbaum; 2006:9780805853216.
|ART: antiretroviral treatment|
|CSQ: client service questionnaire|
|IMB: information-motivation-behavioral skills|
|RCT: randomized control trial|
|SEF: session evaluation form|
|YMSM: young men who have sex with men|
Edited by C Dias; submitted 08.06.17; peer-reviewed by K Claborn, D Finitsis, L Dayer, RN Robbins; comments to author 29.08.17; revised version received 23.10.17; accepted 28.11.17; published 23.04.18
©Laura Whiteley, Larry Brown, Michelle Lally, Nicholas Heck, Jacob J van den Berg. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 23.04.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.