Published on 13.02.20 in Vol 8, No 2 (2020): February
Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/16062, first published Aug 29, 2019.
From “Step Away” to “Stand Down”: Tailoring a Smartphone App for Self-Management of Hazardous Drinking for Veterans
Background: US military veterans who screen positive for hazardous drinking during primary care visits may benefit from a mobile app. Step Away is an evidence-based mobile intervention system for the self-management of hazardous drinking. However, Step Away was not designed for veterans, and differences between veterans and civilians could limit the reach and effectiveness of the app with this population.
Objective: The primary objective of this study was to repurpose Step Away to address the needs and preferences of the veteran primary care population. The Method for Program Adaptation through Community Engagement (M-PACE) model was used to guide the adaptation process. This model can serve as a generalizable approach that other researchers and intervention developers can follow to systematically tailor mobile health tools for a new population.
Methods: Veteran patients who screened positive for hazardous drinking during a primary care visit (n=12) and peer providers employed by the US Veterans Health Administration (n=11) were recruited to systematically review Step Away and provide feedback on its content and presentation via Web-based surveys and a semistructured interview. Participant feedback was reviewed through an iterative process by key stakeholders who adjudicated which suggested modifications to the app could enhance engagement and effectiveness with veterans while maintaining program integrity.
Results: Usability ratings of the individual modules of Step Away were uniformly positive across patients and peers, as was the perceived utility of the app overall. Personalized feedback on the health consequences and costs of drinking, options for customization, and the measurement-based care capabilities of the app were viewed as facilitators of engagement. Conversely, lengthy text, small font, and a lack of interactive features were viewed as potential barriers with the older primary care population. Modifications to create a veteran version of the app (Stand Down: Think Before You Drink) included altering the appearance of the app to incorporate more veteran-centric content, adding links and options for resources and activities for veterans, and reducing the amount of text and adding veteran-specific references and common concerns and triggers for drinking in this population.
Conclusions: The M-PACE model provided a systematic approach to repurpose Step Away to fit the needs and preferences of veteran primary care patients who engage in hazardous drinking. Stand Down may serve as an innovative, low-cost means of expanding access to care for veterans who engage in hazardous drinking.
JMIR Mhealth Uhealth 2020;8(2):e16062
The evidence for various smartphone apps to improve users’ mental health is expanding [, ]. Nonetheless, for many mobile apps, the patient population for which they were developed and validated differs in important ways from subpopulations that may also benefit from these apps. To cite one of many possible examples, an app developed on a young adult sample would not necessarily be as appealing to the elderly population, thus creating barriers to implementation [ ]. Consequently, tailoring mobile apps to maximize reach and effectiveness with new populations defined by demographic and life experience factors is essential. In this paper, we outline a systematic approach to tailoring a mobile app for populations for which it was not originally designed [ ]. To illustrate this approach, we describe a mobile-based intervention system for self-management of drinking problems–Step Away [ - ]–that was originally developed for the general population. We outline our efforts to repurpose Step Away’s content and presentation to fit the characteristics, needs, and preferences of US veterans to maximize engagement and effectiveness with this population.
Hazardous Drinking Among US Veterans: Using Mobile Apps to Improve Access to Care
Hazardous drinking—a pattern of alcohol use that increases an individual’s risk for negative health consequences—is a significant public health problem and financial burden on health care systems and the society at large . Hazardous drinking is commonly assessed using the Alcohol Use Disorder Identification Test for Consumption (AUDIT-C) [ ], which is also an effective screening questionnaire for identifying those with a past-year diagnosis of an alcohol use disorder (AUD) [ ]. In the US Veterans Health Administration (VHA), 15% to 30% of veterans screen positive for hazardous drinking on the AUDIT-C during a primary care visit; however, most of these patients do not obtain treatment for their drinking despite wide availability of these services in VHA [ ].
Mobile apps offer an innovative and low cost means of expanding access to care for veterans who engage in hazardous drinking. Such apps have the benefit of overcoming key barriers to care access. For example, an app would make appointments unnecessary, thus reducing the burden on both the VHA (eg, staffing) and patients (eg, travel costs) [, ]. Providing veterans with an option for care that is private, self-directed, and available as needed can also address the stigma that many patients have in regard to alcohol use treatment [ ]. There is a need for an evidence-based mobile app that is customized and targeted toward veterans to help them reduce or abstain from alcohol use.
Step Away: Mobile App for Hazardous Drinking
Step Away is a mobile-based intervention program for individuals who want to reduce or abstain from drinking but are unable or unwilling to receive in-person care [, ]. Development of Step Away was guided by the empirically supported alcohol interventions of motivational enhancement and cognitive-behavioral therapies, particularly relapse prevention and community reinforcement approaches [ , ]. For example, the app provides a comprehensive assessment of drinking patterns, personalized feedback to enhance awareness of drinking-related problems, tracking progress toward drinking goals, and in-the-moment tools to help manage cravings and triggers. Step Away is applicable to individuals with a range of alcohol use severity as it allows a choice of either moderation or abstinence as a drinking goal.
Published research on Step Away has focused exclusively on the prototype version. In a pilot randomized controlled trial of 54 adults with an AUD diagnosis, the use of the prototype version was associated with significant reductions in heavy drinking days and drinks per drinking day over 6 weeks and more percentage days abstinent relative to use of a Web-based program (Drinker’s Check-up) plus bibliotherapy . On the basis of the usability ratings and system usage data, the prototype was subsequently modified, named “Step Away,” and is now available for download on iPhones and other iOS (iPhone Operating System) platform mobile devices [ ]. Step Away contains 10 modules: (1) drinking patterns, (2) goals, (3) rewards, (4) cravings, (5) strategies, (6) support persons, (7) reminders, (8) high risk, (9) moods, and (10) new activities. Step Away also includes a Get Help feature that provides strategies and resources for individuals who need immediate assistance to manage cravings or other problems, and Daily and Weekly Interviews to help individuals monitor their alcohol use and track progress toward their drinking goal [ ]. The acceptability and usability of the current version of Step Away has not been reported, though recent unpublished work from New Zealand and the United States indicates that acceptability of this version is high (P Dulin, personal communication).
Tailoring Step Away for US Veterans
Step Away was not designed for US veterans per se. Differences between veterans and civilians highlight the value of repurposing Step Away to address the characteristics, needs, and preferences of veterans. For example, modifications to the appearance of mobile health (mHealth) tools to include references to military culture can increase the likelihood that a veteran will identify with the program’s content and engage with the tool . Differences in the demographics and mental health needs of veterans (vs civilians) may also be relevant. To date, the development and validation of Step Away has been limited to adults aged 18 to 45 years, whereas the majority of veterans who screen positive for hazardous drinking during a primary care visit are in mid-to-late life [ , ]. Consequently, less text or larger font may be needed to facilitate usability among the older veteran population [ ]. Relative to young adults, older adults who engage in hazardous drinking also struggle with more comorbid medical conditions [ ] and may therefore benefit from feedback on the health consequences of drinking. Finally, the rate of trauma exposure and the comorbidity between hazardous drinking and posttraumatic stress disorder is higher among veterans than in the general population [ ]. Therefore, references to the association between trauma and alcohol use may enhance the relevance of the app for some veterans.
The objective of this study was to repurpose the Step Away mobile intervention system to create a version of the app that would maximize engagement and effectiveness with US veterans. To achieve this objective, we applied the Method for Program Adaptation through Community Engagement (M-PACE) model . M-PACE is a systematic approach to soliciting comprehensive feedback from stakeholders on potential changes to an intervention to address differences in a target population and deciding on which changes to implement. Per the M-PACE model, incorporating consumer feedback into the adaptation of an intervention, along with feedback from relevant experts and program facilitators, results in improved acceptability and utility of the intervention while maintaining program fidelity. The goal is to enhance engagement of the intervention in a new population without diluting the intervention’s effectiveness.
The long-term goal of this research program is to integrate the use of a veteran version of Step Away with telephone support from peer providers. Such providers would be veterans who are currently in recovery from addiction and have been trained to serve other veterans who are actively struggling with these issues. By encouraging utilization of the app and monitoring Veterans’ progress toward their recovery goals, a peer provider can provide supportive accountability to patients who use Step Away, thus increasing the reach and sustainability of this app with the veteran population . Therefore, the samples for this study were veteran patients who screened positive for hazardous drinking during a VHA primary care visit and peer providers who serve veterans with addiction.
VHA administrative data were used to identify eligible patients who received a positive screen on the AUDIT-C (scores of >4 for women and >5 for men) during a visit to a VHA primary care clinic between July and October 2017. A total of 150 eligible patients were sent a study invitation letter in the mail; patients who did not respond to the letter were called approximately 1 week later. Overall, 81 patients (81/150, 54.0%) opted out, 33 (33/150, 22.0%) did not respond to a voicemail message, and 13 (13/150, 8.7%) were unable to be reached because of their message inbox being full or the phone number listed in patients’ medical records being incorrect or not in service. Furthermore, 23 patients (23/150, 15.3%) were able to be reached and expressed interest in the study. Among this pool of interested patients, quota sampling based on gender, age, and race and ethnicity was used to select a representative sample of VHA primary care patients who screened positive for hazardous drinking. A total of 13 patients were initially enrolled, and 1 withdrew, yielding a sample of 12 (see sample characteristics in). Patient participants were mostly male (11/12, 92%), white (non-Hispanic; 7/12, 58%), college educated (9/12, 75%), and 58.9 years old, on average (SD 19.4). At the time of enrollment, 4 participants reported that they were currently receiving mental health treatment for mental health issues; none of them reported receiving treatment for addiction. Per AUDIT-C scores from the administrative records, patients reported a moderate level of drinking problems (mean 6.3, SD 2.4; out of a possible score of 12).
|Variable||Patients (N=12)||Peers (N=11)|
|Gender, n (%)|
|Male||11 (92)||9 (82)|
|Female||1 (8)||2 (18)|
|Race and ethnicity, n (%)|
|African American/black||4 (33)||1 (9)|
|White (non-Hispanic)||7 (58)||7 (64)|
|Hispanic||0 (0)||3 (27)|
|Other||3 (25)||0 (0)|
|Age, mean (SD)||58.9 (19.4)||48.3 (9.4)|
|Education (highest level), n (%)|
|High school diploma/General Equivalency Degree||1 (8)||5 (46)|
|Some college||2 (17)||3 (27)|
|4-year college/university||5 (42)||2 (18)|
|Graduate degree||4 (33)||1 (9)|
|Smartphone comfort level (1-5 scale), mean (SD)||4.5 (0.8)||4.4 (0.9)|
|1=very uncomfortable, n (%)||0 (0)||0 (0)|
|5=very comfortable, n (%)||8 (67)||7 (64)|
|Device used during the study, n (%)|
|Personal device||8 (67)||5 (46)|
|Study-provided iPod Touch||4 (33)||6 (55)|
A convenience sample of peer providers was recruited across 5 VHA facilities. Peer providers were eligible if they were veterans, worked with other veterans with alcohol use problems, and had been employed by VHA for at least 6 months. Approximately 71 peer providers were informed about the study through passive recruitment strategies, that is, mass emails to provider listservs and phone and in-person meetings with teams of peer support specialists and addiction therapists) rather than individual contacts. Overall, 13 peer providers contacted the study staff to express an interest and were enrolled, 2 of whom ultimately withdrew, yielding a sample of 11 (see sample characteristics in). Provider participants were mostly male (9/11, 82%), were white (non-Hispanic; 7/11, 64%), had at least some college education (6/11, 55%), and were 48.3 years old on average (SD 9.4 years). In addition, 3 participants were employed by the VHA as addiction therapists, and 8 were employed as peer support specialists. Among them, 5 participants reported working in their position for less than 3 years, and 6 participants reported working in their position for 3 years or more.
Method for Program Adaptation Through Community Engagement Steps
The M-PACE model guided adaptation of Step Away for the veteran population . The first step involved creation of a steering committee with members who represented key stakeholders in adaptation of the intervention—that is, researchers, practitioners, program developers, and patients. Specifically, the committee for this study comprised 12 individuals: 6 research investigators (experts on addiction among veterans, evidence-based treatments for AUD, and mHealth), the app developer (PLD), 4 VHA administrators and clinicians in primary care–mental health integration and peer support services at both the local and national levels, and a representative of a Veteran and Family Engagement Council. The second step of the M-PACE model involved implementing the program as originally designed with patients and providers who will ultimately use the adapted intervention. This study’s target users were veterans who screened positive for hazardous drinking during primary care visits and peer providers who would provide telephone-based care to veterans using the app. The third step involved systematically soliciting feedback and suggestions for change from patients and providers, using both qualitative and quantitative research methods. Surveys with open-ended and scaled items were used to assess likes, dislikes, and recommendations for change during the course of the intervention and soon after its completion. The fourth step involved summarizing the quantitative and qualitative data and reviewing results with the steering committee. During the fifth step—adjudicating participants’ feedback—each suggestion for change was evaluated using 3 criteria: importance (degree to which the change will improve effectiveness of the intervention), feasibility (degree to which additional burden would be placed on patients, providers, or program developers), and congruence (degree to which the suggested changes work with or against the core change features of the intervention). After evaluating each suggestion, the steering committee decided whether to implement the suggested change.
After enrollment, patients and providers completed a baseline interview to assess demographics and rate their comfort using a smartphone on a 5-point scale (1=very uncomfortable, 5=very comfortable). Both patients and peer providers reported a high level of comfort in using these devices (see). Patients were also queried on their current receipt of mental health or addiction treatment, and peer providers were queried on their length of employment in the VHA. A research assistant (RA) then assisted participants with downloading the unmodified Step Away app to their iPhone (the app was only available on the iOS platform at the time) or a study-provided iPod Touch. The majority of patients used their personal device (8/12, 67%), whereas the majority of peer providers borrowed an iPod Touch (6/11, 55%). Participants were asked to use the app daily for 10 days, reviewing an average of 1 of the 10 modules per day, and to complete daily surveys (sent via email using REDCap) to provide feedback on each module’s usability. Participants were provided with a written guide to direct their review of the individual modules. This approach to data collection increases participants’ accurate recall of feelings and reactions to program material and obtains suggestions that are specific to elements of a program (eg, a given module) rather than generalized across program material [ ]. The daily surveys consisted of 5 items involving quantitative ratings of the usefulness (1=not at all useful, 5=very useful) and difficulty of using a certain Step Away module (1=not at all difficult, 5=very difficult), and 3 open-ended questions asking what the participant liked, what they disliked, and what they would change about the module. With exception of the Reminders module, usability ratings for modules were completed by a majority of participants (see ). Semistructured interviews were conducted by phone 2 weeks after enrollment to obtain global feedback from participants on the perceived utility of Step Away, facilitators and barriers to engagement with the app, and suggested modifications to enhance engagement and effectiveness with veteran primary care patients. App usage data were extracted from the app during the 2-week period to validate participants’ use of the app during their participation. Owing to limited Wi-Fi availability, 5 participants who borrowed an iPod Touch did not have their app usage data uploaded to the Cloud-based storage system where app usage was being tracked. Therefore, data on app usage were available for 18 participants (9 patients and 9 peers). These participants launched an average of 5.3 (SD=4.2) of the 10 app modules and reviewed the app for an average of 150.2 min (SD=43.3) over the 2-week period. Patient participants were paid US $25 for the follow-up interview, US $5 per Web-based survey, and an additional US $25 if they completed all 10 Web-based surveys. All participant procedures were approved by the local institutional review board.
|Module||Total sample||Patients||Peers||Sample responses to free-text questions|
|Usefulness, mean (SD)||Difficulty, mean (SD)||Usefulness, mean (SD)||Difficulty, mean (SD)||Usefulness, mean (SD)||Difficulty, mean (SD)||Liked||Disliked/would change|
|Drinking patternsa||3.76 (0.9)||1.88 (1.1)||3.63 (0.9)||1.86 (1.2)||3.90 (0.7)||1.90 (0.9)||“Allows user to see how much they drink and the financial cost of drinking.” [Peer-004]||“Shorten, simplify. I find alcoholics are impatient while actively drinking.” [Peer-002]|
|Goalsb||4.06 (0.9)||1.88 (0.8)||3.67 (1.1)||1.86 (0.9)||4.44 (0.7)||1.89 (0.8)||“Health and money as benefits of not drinking.” [Patient-001]||“Explanations were very long.” [Patient-005]|
|Rewardsc||3.76 (1.3)||1.62 (1.2)||3.38 (1.1)||1.06 (0.2)||4.11 (1.4)||2.11 (1.5)||“Allows for custom input for rewards.” [Peer-012]||“Add reward choices that will appeal to lower income vets.” [Patient-008]|
|Cravingsd||4.21 (1.1)||1.64 (1.2)||3.83 (1.3)||1.50 (0.8)||4.50 (0.8)||1.75 (1.4)||“You can isolate and identify specific triggers.” [Peer-013]||“A lot of reading and less interacting in this step.” [Peer-012]|
|Strategiese||4.00 (1.2)||1.93 (1.3)||4.00 (1.0)||1.57 (1.1)||4.00 (1.4)||2.29 (1.5)||“Good variety of strategies.” [Patient-003]||“Seemed aimed at heavy drinkers.” [Patient-008]|
|Support personsf||3.81 (1.0)||1.41 (0.7)||3.63 (1.1)||1.56 (0.9)||4.00 (1.1)||1.25 (0.5)||“Actually putting in the number and email of the person.” [Peer-001]||“Prompt user to check in with someone every day.” [Peer-006]|
|Remindersg||3.60 (1.1)||1.60 (1.1)||3.29 (1.1)||1.71 (1.3)||4.33 (0.6)||1.33 (0.6)||“Different ways to personalize reasons for change (eg, photos of loved ones).” [Patient-003]||“More text space to provide reasons for change.” [Patient-010]|
|High riskh||3.75 (0.9)||1.33 (0.8)||3.67 (0.8)||1.00 (0.0)||3.83 (0.9)||1.67 (1.0)||“Allows me to see what time I am more at risk to drink.” [Peer-009]||“Provide link to supports as way to deal with high-risk times.” [Patient-013]|
|Moodsi||3.94 (0.7)||1.34 (0.6)||3.86 (0.7)||1.14 (0.4)||4.00 (0.7)||1.50 (0.7)||“Keeps weekly track of fluctuations in mood.” [Patient-011]||“More information regarding graphs.” [Patient-010]|
|New activitiesj||4.23 (0.8)||1.54 (0.9)||4.40 (0.9)||1.40 (0.9)||4.13 (0.8)||1.63 (0.9)||“Easy to customize. Allows you to plan ahead for a high-risk time.” [Peer-013]||“Expand to track activities taken up instead of drinking.” [Patient-010]|
aNTotal sample=21, NPatients=11, NPeers=10.
bNTotal sample=18, NPatients=9, NPeers=9.
cNTotal sample=17, NPatients=8, NPeers=9.
dNTotal sample=14, NPatients=6, NPeers=8.
eNTotal sample=15, NPatients=8, NPeers=7.
fNTotal sample=16, NPatients=8, NPeers=8.
gNTotal sample=10, NPatients=7, NPeers=3.
hNTotal sample=12, NPatients=6, NPeers=6.
iNTotal sample=16, NPatients=7, NPeers=9.
jNTotal sample=13, NPatients=5, NPeers=8.
Descriptive statistics were calculated for quantitative items from the daily surveys and the follow-up interview. Textual data from the open-ended questions of the daily surveys and the follow-up interviews were analyzed using techniques for rapid qualitative analysis recommended by Hamilton . Specifically, 2 RAs reviewed responses to the daily surveys, listened to the audio recordings from the semistructured interviews, and took detailed notes using a template to summarize responses to the survey and interview questions and document preliminary themes related to facilitators and barriers to engagement and suggested modifications. The RAs then copied these notes into an Excel matrix to compare the preliminary themes for each question (columns) across participants (rows). This matrix was organized such that the summary of participants’ response to a question and the preliminary theme were entered into each cell. Furthermore, 2 study investigators independently reviewed the composite matrix of these summarized responses to identify global themes in the data corresponding to domains of facilitators and barriers to engagement and suggested modifications to enhance the engagement and effectiveness of Step Away among veterans who engaged in hazardous drinking. The investigators then met to review their independently derived lists of themes and engaged in a consensus process to rectify disagreements and finalize the themes in each of these domains.
Usability Ratings of the Step Away Modules
Perceived Utility of Step Away
When asked, “Will the app help veterans reduce the amount they drink or how often they drink?,” 22 of the 23 participants answered affirmatively. When asked, “How likely are you to continue using Step Away?,” 15 of the 23 participants indicated plans to continue using the app or said they would continue to use it if it were available on their personal phone. Patient participants noted the benefits for increasing their access to care and ability to monitor their drinking over time:
It would help. Not a lot of people are in an area where there’s availability for help. It’s helped me already.
It's good because you can put in the exact number of drinks and track your progress. I think I personally drank less while using it.
Peer participants also noted the benefits of the app for reducing a patients’ drinking, although they added that this would depend on another person keeping the patient accountable for using it:
If they have a solid support system that reminds them of using the app it would be helpful.
Yes, it will hold them accountable and help them. As long as they’re reminded to use it, it would help.
Facilitators and Barriers to Engagement With Step Away
Regarding facilitators to their engagement with Step Away, participants noted the (1) reminders on high-risk times and situations; (2) encouraging rewards for reaching one’s drinking goals; (3) personalized feedback, particularly around the financial costs and health consequences of alcohol use; (4) ability to customize one’s drinking goal and other features of the app; and (5) measurement capabilities, such as the app’s ability to track one’s drinking and progress toward their drinking goal (see). In terms of barriers to engagement with the app, participants noted (1) key features of Step Away that felt hidden or insufficiently highlighted (eg, the Get Help feature and options for customization), (2) concerns that some aspects of the interface could limit user engagement (eg, not user-friendly for elderly patients because of the small font size, not enough visuals and graphics and the app is text heavy), and (3) privacy concerns that were insufficiently acknowledged.
|Reminders on high-risk times and situations||“[I like that it] provides education and awareness of risky situations and times of drinking.” [Patient-005]; “I like that it reminded me when I was coming up on a time I was going to drink.” [Peer-006]|
|Encouraging rewards for reaching drinking goals||“Rewards for milestones reached were helpful. Sometimes veterans don’t know what to do for themselves as a reward instead of drinking. Typically, they would go out and have a drink.” [Peer-001]|
|Personalized feedback regarding financial costs and health consequences of alcohol use||“Information about amount of money spent on alcohol was useful” [Peer-004]; “Good information in terms of caloric impact of my drinking and how alcohol may be affecting my problems with weight control.” [Patient-013]|
|Ability to customize one’s drinking goal and various app features||“It allowed me to set up a schedule to be notified; it engaged me and helped me be involved…it made it more personalized.” [Patient-008]; “I like that it gives a choice of moderation. That makes it helpful for people who might not want to fully quit but still want help.” [Peer-011]|
|Measurement capabilities (eg, ability of app to track drinking and progress toward goals)||“If you want to change something, measure it. [The app] helps vets pay attention to what they’re doing and keep track of progress so they have an idea of what they need to change.” [Patient-010)|
|Key features appeared hidden or were insufficiently highlighted: The “Get Help” feature; more options for customization||“I tend not to click on help icons. One reason is because I typically don’t find the answer I’m looking for and another is because I did not feel the need to use the feature.” [Patient-010]|
|Concerns that some aspects of the interface could limit user engagement: Not user-friendly for elderly patients (eg, small font size); “Text heavy”—not enough visuals and graphics.||“It was hard to read. It would be nice if they could be magnified for people with older eyes.” [Patient-013]; “I don’t think it would be helpful for vets who are actively drinking. Too much reading.” [Peer-004]|
|Privacy concerns insufficiently acknowledged||“Trust issues with government and VA. Fear of breach of confidentiality. Those who are new to sobriety might be overwhelmed.” [Peer-002]; “Provide a rationale as to why we are putting contacts in, for example, including that [providing personal information] typically helps people for reasons x, y, and z; otherwise it could be triggering.” [Patient-005]|
Suggested Modifications to Enhance Engagement and Effectiveness of Step Away Among Veterans
The qualitative interviews yielded themes to enhance engagement and effectiveness of Step Away with the veteran population (see, Column 1 for a bulleted outline of the suggested modifications). One theme focused on modifying the appearance and design of the app to include more veteran-centric content. Another theme related to the need to revise the text to increase the usability and relevance of the app for older and/or lower-income veterans. Other veteran-centric modifications included adding links to resources and services for veterans in crisis or seeking treatment; adding information on drinking problems among veterans; adding more preprogrammed response options relevant to veteran preferences and needs; and addressing potential privacy concerns among veterans such as whether the app data would be entered into patients’ Veterans Affairs (VA) medical records. Although many suggestions focused on how to repurpose the app for veterans per se, other modification themes focused on ways to increase the usability of the app more generally. For example, participants suggested more orientation to the app when users set up the program; more use of graphics to track progress toward one’s drinking goal over time; more interactive features, and increasing access and engagement with the app content more generally.
Adjudicating the Suggested Modifications to Step Away
Following the analyses, 2 meetings with the steering committee were convened. The first meeting (daylong) was held in person, with remote members attending by phone. Committee members downloaded Step Away to their iPhones or a study-provided iPod Touch to review the app before the meeting. During the meeting, a demonstration of Step Away’s content and functionality was given by the lead author. Participant feedback and other summary findings were then reviewed, and the suggested modifications to Step Away were discussed and operationalized. Committee members were asked to evaluate the suggested modifications according to 3 criteria: importance—the degree to which the change could enhance Step Away’s effectiveness and engagement with the veteran primary care population; feasibility—how burdensome the change would be to veterans, peer providers, and/or app developers; and congruence—did the change work with or against the core components of Step Away.
Following this meeting, a list of proposed changes to Step Away to create a veteran version of the app was drafted by the lead (DMB) and senior authors (PLD) and emailed to committee members. Members rated each of the proposed changes in terms of whether it was (1) essential, (2) nice to have, but not essential, or (3) not essential to incorporate into the veteran version of the app. Proposed changes that received a rating of essential by a majority of the committee members and were retained in the final list are shown in. Changes that were not viewed as essential by a majority of committee members included many of the interactive features that were suggested by participants—for example, voiceover audio, educational games, and link to an online forum. These changes were viewed by the committee as too costly and resource intensive to incorporate into the app’s format and were therefore not included into the final list of changes.
Creating the Veteran Version of Step Away (Stand Down: Think Before You Drink)
The lead and senior authors had a series of meetings to review the progress on the planned changes, discuss and edit wireframes of the changes, and conduct beta testing of the initial versions of the revised app. To solicit feedback on specific content that was being considered for inclusion in the app, a meeting was convened with members of the Veteran and Family Engagement Council at the local facility. This council assists with the development and implementation of projects and initiatives at the facility to ensure that the perspectives of veterans and their family members are incorporated. After all the changes to Step Away were completed and beta tested by the lead author and an RA, a second meeting of the steering committee was convened by phone. Before this meeting, an initial version of the repurposed app—Stand Down: Think Before You Drink (or Stand Down)—was made available to members for review. In the military, Stand Down refers to a period of recovery after a state of high alert; and among veterans, the term also refers to multiday events in providing them with resources, services, and referrals to additional services. During the meeting, major changes to the app were reviewed and feedback was solicited from the committee members regarding additional changes to be made to the app. On the basis of this feedback, minor changes were made to Stand Down, and the app was launched for research use in the iTunes store .
The goal of this study was to use the M-PACE model to repurpose the Step Away mobile intervention system for US veterans. In particular, we sought to create a version of the app that would enhance engagement and effectiveness with veterans in primary care who screened positive for hazardous drinking, many of whom are in mid-to-late life [, ]. A key advantage of using the M-PACE model was the ability to obtain feedback on both the individual modules of Step Away and global feedback on the app. Feedback on both facilitators and barriers to engagement with Step Away was critical to this process, which helped the Steering Committee adjudicate which modifications to this app were most essential and how best to implement them.
One strength of the app from the perspective of participants was the personalized feedback provided via the DrinkingPatterns module. Personalized, norm-based feedback is viewed as an essential component of brief interventions for drinking problems  and is a feature not currently incorporated into other mobile apps for drinking problems for veterans [ ]. Feedback on the financial costs and health consequences of drinking were highlighted by participants and the steering committee as common concerns of the target population that should be emphasized in the veteran version of the app. These issues are now highlighted to a greater extent in Stand Down than in Step Away. Another perceived strength of the app was the ability of users to customize the content to fit their needs and preferences, such as their drinking goal, rewards, nondrinking activities, and reminders. Such a personalized approach to care is consistent with a broader shift in addiction and mental health treatment toward more recovery-oriented and patient-centered approaches to care designed around patients’ unique goals and priorities for care [ , ]. To this end, Stand Down includes more preprogrammed recreational activities for veterans including a list of organizations that support veteran activities and events. The measurement-based care capabilities of Step Away in terms of tracking users’ alcohol use over time and progress toward their drinking goal was also viewed as a strength by participants. Consistent with a broader effort in VHA and the addiction treatment literature of measurement-based care [ ], this feature of Step Away was expanded in Stand Down by showing users’ weekly progress toward their drinking goal on the home screen.
One potential barrier to engagement with Step Away was both the size and the amount of text throughout the app’s modules. Small font size may be particularly salient for the target population, given that many veterans treated in primary care in VHA are mid-to-late life and may have visual impairments . Consistent with this, prior research has recommended using larger font on mHealth tools targeted toward older adults [ ]. Relatedly, other research has suggested minimizing text in mHealth tools when possible to mitigate the potential for fatigue [ ]. Accordingly, a key modification to create the Stand Down app involved a careful review of the app’s text files to identify opportunities to shorten descriptions, break up long paragraphs, and enumerate key information. These changes, along with more proactive presentation of app content via the Message of the Day, were intended to enhance the overall usability of the app for the target population.
In addition to modifying Step Away to address the characteristics of the veteran population and increase its usability, other changes involved alterations to the look and feel of the app to incorporate more veteran-centric content. For example, we replaced both the app icon and the splash page that users would see when they launch the app with an image of an American flag (see), embedded videos of veterans describing their recovery from drinking problems, and rebranded the app name. These changes are unlikely to have an adverse impact on the effectiveness of the Step Away program and may have the benefit of increasing the extent to which veterans identify with the app. This approach to tailoring an mHealth tool for the veteran population was employed successfully in a recent study that modified a Web-based intervention for anxiety and depression for rural veterans [ ]. This study extends this principle of adapting mHealth tools for veterans to mobile apps and to the treatment of hazardous drinking in this population.
Strengths, Limitations, and Future Directions
The design of this study included a number of strengths, many of which are inherent to the M-PACE model. Specifically, we used a mixed-methods approach to data collection to obtain comprehensive feedback from key consumers (veteran patients and peer providers) on the unmodified app. Drawing on the principles of community-based participatory research , a diverse set of stakeholders with expertise in addiction, evidence-based treatments for AUD, primary care–mental health integration, and mHealth was used to adjudicate the suggested changes to Step Away. Furthermore, the iterative and systematic approach to adjudication helped to identify which changes to the app can maximize its effectiveness with veterans while also being feasible to implement and congruent with the core components of Step Sway [ ]. These benefits of the M-PACE model notwithstanding, there may be opportunities to extend the model by incorporating elements from the field of instructional design [ ]. For example, cycles of formative evaluation are common in instructional design models as they allow for revisions of mHealth tools before implementation. Such an approach could be incorporated into the M-PACE model after the adjudication process by soliciting feedback on the new tool with the same consumers who initially reviewed the unmodified tool. This would permit assessment of whether the suggested modifications were incorporated sufficiently or whether further revisions to the mHealth tool are needed prior to its implementation with the target population.
Among the study limitations, the patient feedback was based on a sample from a single VA Health Care System, which may not generalize to the perceptions of veterans in other clinics and geographical regions. This limitation was somewhat countered by soliciting feedback from peer providers across multiple geographical regions in the United States. Furthermore, a minority of both the patient and peer providers who were contacted about the study agreed to participate; for patients, this may reflect the fact that many veterans who screen positive for alcohol use problems are not interested in receiving help. Consequently, feedback from these highly self-selected groups may not be representative of the larger samples of veterans and providers who could conceivably use the app. A larger sample size of both patients and peer providers may also have been advantageous. However, a substantial portion of the participant feedback involved qualitative data, and sample sizes of 10 or more are often sufficient for reaching thematic saturation and conducting group comparisons in qualitative research [, ]. The modest launch rate (5.3 out of the 10 app modules) and the lower number of daily surveys completed for some of the Step Away modules (eg, Reminders) also suggests that participants may not have reviewed the full menu of tools available in Step Away. Potentially, this modest level of app usage may be improved by the modifications that were made through the M-PACE process. Finally, this study did not provide data on the acceptability or utility of the Stand Down app. Pilot testing of this veteran-focused app is a critical next step in this program research. Should it prove effective in future research for improving the drinking outcomes of veterans engaging in hazardous drinking, subsequent implementation research should focus on how best to enhance the reach of the app with the target population. Our involvement of key stakeholders at the national level of VHA in this early stage of this program of research should ultimately facilitate such efforts.
We used the M-PACE model to repurpose the Step Away mobile intervention system to target the characteristics, needs, and preferences of veterans who are identified in primary care settings as engaging in hazardous drinking. We envision the approach outlined here as a generalizable method that other researchers can follow to systematically tailor an mHealth tool to maximize engagement and effectiveness of an app with a patient population for which the app was not originally designed. Ultimately, the use of Stand Down may serve as an innovative, low-cost means of overcoming barriers to access and engagement in alcohol use treatment among veteran primary care patients. Our preliminary study of this app would benefit from a follow-up study in a larger population and involving patients across multiple VA medical centers.
This work was supported by a Department of Veterans Affairs Health Services Research and Development grant awarded to DMB (PPO 16-305). CT and KH were supported by Senior Research Career Scientist Awards from the Department of Veterans Affairs Health Services Research and Development (RCS-00-001 and RCS-14-141, respectively). The views expressed are those of the authors and do not necessarily reflect those of the VHA.
Conflicts of Interest
PLD is the primary owner of Here & Now Systems, LLC—the company that developed the Step Away mobile app as well as the veteran version of this app (Stand Down: Think Before You Drink). There are no other conflicts to report.
Modifications to Step Away to create a veteran version of the app (“Stand Down: Think Before You Drink”).DOCX File , 6242 KB
- Armstrong CM, Hoyt T, Kinn JT, Ciulla RP, Bush NE. Mobile behavioral health applications for the military community: evaluating the emerging evidence base. Best Pract Mental Health 2017;13(1):106-119 [FREE Full text]
- Kuhn E, Kanuri N, Hoffman JE, Garvert DW, Ruzek JI, Taylor CB. A randomized controlled trial of a smartphone app for posttraumatic stress disorder symptoms. J Consult Clin Psychol 2017 Mar;85(3):267-273. [CrossRef] [Medline]
- Kuerbis A, Mulliken A, Muench F, A. Moore A, Gardner D. Older adults and mobile technology: factors that enhance and inhibit utilization in the context of behavioral health. Ment Health Addict Res 2017;2(2). [CrossRef]
- Chen EK, Reid MC, Parker SJ, Pillemer K. Tailoring evidence-based interventions for new populations: a method for program adaptation through community engagement. Eval Health Prof 2013 Mar;36(1):73-92 [FREE Full text] [CrossRef] [Medline]
- Dulin PL, Gonzalez VM, King DK, Giroux D, Bacon S. Smartphone-based, self-administered intervention system for alcohol use disorders: theory and empirical evidence basis. Alcohol Treat Q 2013;31(3) [FREE Full text] [CrossRef] [Medline]
- Dulin PL, Gonzalez VM, Campbell K. Results of a pilot test of a self-administered smartphone-based treatment system for alcohol use disorders: usability and early outcomes. Subst Abus 2014;35(2):168-175 [FREE Full text] [CrossRef] [Medline]
- Gonzalez VM, Dulin PL. Comparison of a smartphone app for alcohol use disorders with an internet-based intervention plus bibliotherapy: a pilot study. J Consult Clin Psychol 2015 Apr;83(2):335-345 [FREE Full text] [CrossRef] [Medline]
- Giroux D, Bacon S, King DK, Dulin P, Gonzalez V. Examining perceptions of a smartphone-based intervention system for alcohol use disorders. Telemed J E Health 2014 Oct;20(10):923-929 [FREE Full text] [CrossRef] [Medline]
- Whiteford HA, Degenhardt L, Rehm J, Baxter AJ, Ferrari AJ, Erskine HE, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet 2013 Nov 9;382(9904):1575-1586. [CrossRef] [Medline]
- Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Arch Intern Med 1998 Sep 14;158(16):1789-1795. [CrossRef] [Medline]
- Bradley KA, DeBenedetti AF, Volk RJ, Williams EC, Frank D, Kivlahan DR. AUDIT-C as a brief screen for alcohol misuse in primary care. Alcohol Clin Exp Res 2007 Jul;31(7):1208-1217. [CrossRef] [Medline]
- Bradley KA, Ludman EJ, Chavez LJ, Bobb JF, Ruedebusch SJ, Achtmeyer CE, et al. Patient-centered primary care for adults at high risk for AUDs: the Choosing Healthier Drinking Options In primary CarE (CHOICE) trial. Addict Sci Clin Pract 2017 May 17;12(1):15 [FREE Full text] [CrossRef] [Medline]
- Donker T, Petrie K, Proudfoot J, Clarke J, Birch M, Christensen H. Smartphones for smarter delivery of mental health programs: a systematic review. J Med Internet Res 2013 Nov 15;15(11):e247 [FREE Full text] [CrossRef] [Medline]
- Prentice JL, Dobson KS. A review of the risks and benefits associated with mobile phone applications for psychological interventions. Can Psychol 2014;55(4):282-290. [CrossRef]
- Rapp RC, Xu J, Carr CA, Lane DT, Wang J, Carlson R. Treatment barriers identified by substance abusers assessed at a centralized intake unit. J Subst Abuse Treat 2006 Apr;30(3):227-235 [FREE Full text] [CrossRef] [Medline]
- Blonigen DM, Harris-Olenak B, Haber JR, Kuhn E, Timko C, Humphreys K, et al. Customizing a clinical app to reduce hazardous drinking among veterans in primary care. Psychol Serv 2019 May;16(2):250-254. [CrossRef] [Medline]
- Miller WR, Rollnick S. Motivational Interviewing. Third Edition. New York, NY: Guilford Press; 2013.
- Blonigen DM, Finney JW, Wilbourne P, Moos RH. Psychosocial treatments for substance use disorders. In: Nathan PE, Gorman JM, editors. A Guide to Treatments that Work. Fourth Edition. New York: Oxford University Press; 2015:731-761.
- Here and Now Systems LLC. Step Away: Mobile Intervention for Alcohol Addiction URL: http://itunes.apple.com [accessed 2019-12-17]
- Abraham TH, Marchant-Miros K, McCarther MB, Craske MG, Curran GM, Kearney LK, et al. Adapting coordinated anxiety learning and management for veterans affairs community-based outpatient clinics: iterative approach. JMIR Ment Health 2018 Aug 22;5(3):e10277 [FREE Full text] [CrossRef] [Medline]
- Funderburk JS, Maisto SA, Wade MJ, Kenneson A, Campbell CE. Clinical course of alcohol use in veterans following an AUDIT-C Positive Screen. Mil Med 2014 Nov;179(11):1198-1206. [CrossRef] [Medline]
- Takahashi T, Lapham G, Chavez LJ, Lee AK, Williams EC, Richards JE, et al. Comparison of DSM-IV and DSM-5 criteria for alcohol use disorders in VA primary care patients with frequent heavy drinking enrolled in a trial. Addict Sci Clin Pract 2017 Jul 18;12(1):17 [FREE Full text] [CrossRef] [Medline]
- Timko C, Kong C, Vittorio L, Cucciare MA. Screening and brief intervention for unhealthy substance use in patients with chronic medical conditions: a systematic review. J Clin Nurs 2016 Nov;25(21-22):3131-3143 [FREE Full text] [CrossRef] [Medline]
- Fuehrlein BS, Mota N, Arias AJ, Trevisan LA, Kachadourian LK, Krystal JH, et al. The burden of alcohol use disorders in US military veterans: results from the National Health and Resilience in Veterans Study. Addiction 2016 Oct;111(10):1786-1794. [CrossRef] [Medline]
- Ray JM, Kemp LL, Hubbard A, Cucciare MA. Developing a peer support protocol for improving veterans' engagement to computer-delivered cognitive behavioural therapy. Behav Cogn Psychother 2017 May;45(3):253-265. [CrossRef] [Medline]
- Hamilton A. Health Services Research & Development - Veterans Affairs. 2013 Dec 11. Qualitative Methods in Rapid Turn-Around Health Services Research URL: https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780 [accessed 2019-12-17]
- Here & Now Systems LLC. mobile application software. Stand Down: Think Before You Drink URL: http://itunes.apple.com
- US Department of Veterans Affairs (VA). App Store - Apple. VetChange URL: http://itunes.apple.com [accessed 2019-12-17]
- Kuehn BM. Veterans health system cited by experts as a model for patient-centered care. J Am Med Assoc 2012 Feb 1;307(5):442-443. [CrossRef] [Medline]
- Marchand K, Oviedo-Joekes E. Prioritizing the patient in patient-centered addictions treatment. Addiction 2017 Mar;112(3):466-467. [CrossRef] [Medline]
- Dollar KM, Kirchner JE, DePhilippis D, Ritchie MJ, McGee-Vincent P, Burden JL, et al. Steps for implementing measurement-based care: implementation planning guide development and use in quality improvement. Psychol Serv 2019 Jul 18. [CrossRef] [Medline]
- Schubert MC, Gimmon Y, Millar J, Brewer KJ, Roberts D, Shelhamer M, et al. Veterans have greater variability in their perception of binocular alignment. PLoS One 2018;13(12):e0209622 [FREE Full text] [CrossRef] [Medline]
- Batterham PJ, Calear AL. Preferences for internet-based mental health interventions in an adult online sample: findings from an online community survey. JMIR Ment Health 2017 Jun 30;4(2):e26 [FREE Full text] [CrossRef] [Medline]
- Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health 2010 Apr 1;100(Suppl 1):S40-S46. [CrossRef] [Medline]
- Hilgart MM, Ritterband LM, Thorndike FP, Kinzie MB. Using instructional design process to improve design and development of internet interventions. J Med Internet Res 2012 Jun 28;14(3):e89 [FREE Full text] [CrossRef] [Medline]
- Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods 2006;18(1):59-82. [CrossRef]
- Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: How many interviews are enough? Qual Health Res 2017 Mar;27(4):591-608. [CrossRef] [Medline]
|AUD: alcohol use disorder|
|AUDIT-C: Alcohol Use Disorder Identification Test for Consumption|
|iOS: iPhone Operating System|
|M-PACE: Method for Program Adaptation through Community Engagement|
|mHealth: mobile health|
|RA: research assistant|
|VA: Veterans Affairs|
|VHA: Veterans Health Administration|
Edited by C Dias; submitted 29.08.19; peer-reviewed by J McKay, A Gual, J Taylor, I Savolainen; comments to author 30.09.19; revised version received 11.11.19; accepted 14.11.19; published 13.02.20
©Daniel Blonigen, Brooke Harris-Olenak, Eric Kuhn, Keith Humphreys, Christine Timko, Patrick Dulin. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 13.02.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.