Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 07.03.19 in Vol 7, No 3 (2019): March

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/10750, first published Apr 10, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    Comparison of Developers’ and End-Users’ Perspectives About Smoking Cessation Support Through the Crush the Crave App

    1Propel Centre for Population Health Impact, Faculty of Applied Health Sciences, University of Waterloo, Waterloo, ON, Canada

    2Institute for Health Living and Chronic Disease Prevention, University of British Columbia, Kelowna, BC, Canada

    3Propel Centre for Population Health Impact, Waterloo, ON, Canada

    4School of Nursing, University of British Columbia, Vancouver, BC, Canada

    5Innovative Learning Centre, Faculty of Education, University of British Columbia, Kelowna, BC, Canada

    *these authors contributed equally

    Corresponding Author:

    Laura Louise Struik, PhD, RN

    Propel Centre for Population Health Impact

    Faculty of Applied Health Sciences

    University of Waterloo

    200 University Avenue West

    Waterloo, ON, N2L 3G1

    Canada

    Phone: 1 519 888 4520

    Email:


    ABSTRACT

    Background: High smoking rates among end-users, combined with their high rates of app use, render this age group as a particularly captive audience for quit smoking apps. There is emerging evidence that apps are an effective way to support smoking cessation among end-users. How the expectations behind the design of apps align with the needs and preferences of end-users, and if this differs by gender, is poorly understood, limiting the ability to evaluate and scale these interventions.

    Objective: The objective of this qualitative case study was to detail how the overall design approach of Crush the Crave (CTC), a quit smoking app that targets end-users, compares with young adult women’s and men’s perspectives and experiences, with consideration for the influence of gender.

    Methods: Semistructured interviews were conducted with 15 developers involved in the development of CTC and 31 young adult CTC users. Data were analyzed inductively to derive thematic findings of the perceived pros and cons of CTC by both developers and end-users. Findings were grouped under 4 categories (1) technology and platforms utilized for the app, (2) foundation of app content, (3) underlying focus of the app, and (4) look, feel and functionality of the app.

    Results: Under the category, technology and platforms utilized for the app, it was found that both developers and end-users agreed that apps aligned with the needs and preferences of young adult smokers. Major limitations with the technology identified by end-users were the frequent “glitches” and requirement for internet or data. For the category, foundation of app content,developers agreed that the strength of CTC was in its strong evidence-base. What mattered to end-users, however, was that the content was packaged positively, focusing on the benefits of quitting versus the consequences of smoking. It was found under the category, underlying focus of the app, that the individually-led focus of the app resonated with both developers and end-users, especially young men. Under the final category, look, feel and functionality of the app, it was found that developers were very positive about the app's aesthetics but end-users thought that the aesthetics incited a negative effect. Also, while end-users found it easy to use, they did not find the app intuitive. Finally, end-users thought that, because the app functions were largely based on a user’s quit date versus their ongoing efforts, this often lent to unmeaningful data.

    Conclusions: The current study findings highlight the importance of understanding multiple perspectives of stakeholders involved in a mobile-based intervention. By gathering the viewpoints of developers and end-users, both problematic and effective approaches that underlie development goals were revealed as a means of informing the development, implementation, and evaluation of future electronic health (eHealth) interventions.

    JMIR Mhealth Uhealth 2019;7(3):e10750

    doi:10.2196/10750

    KEYWORDS



    Introduction

    Smoking remains the number one public health concern, especially among young adults because they continue to maintain the highest smoking rates [1]. Despite comprehensive evidence that the younger a smoker quits (before 34 years of age), the greater the health benefits, a significant number of smokers don’t successfully quit until after the age of 45 [2,3], when the adverse health impacts of smoking are not completely reversible. This is due, in part, to low uptake of cessation interventions by the young adult population [4-6]. This evidence underscores the urgent need to reach young adults with effective smoking cessation interventions, and mobile phone apps present as an attractive means to do so, with young adults representing a particularly receptive audience for quit smoking apps [7,8].

    The benefits of mobile phone apps over traditional cessation interventions include the ability to access support anytime, anywhere, via complex functions, including interactive self-monitoring activities, diverse multimedia, and tailored support via context sensors [9,10]. As a result, the quit smoking app market has exploded, with hundreds (over 500) of smoking cessation apps now available on the Apple and Android platforms [11,12]. Despite the plethora of quit smoking apps, few are evidence-informed, theoretically-based, or include the perspectives of end-users [13-15]. Regardless, consumers are downloading these apps to help them quit smoking, with popularity especially high among the young adult population [8]. While emerging evidence indicates that apps may be an effective way to assist individuals with quitting smoking [16-20], and young adults in particular [18], little is known about which aspects of these apps are well designed and address the needs and preferences of young adults.

    Researchers have expressed concern about how little detail is provided in relation to the underlying principles of development and design processes of mobile health (mHealth) smoking cessation interventions [10,21,22]. Tomlinson and colleagues [21] have described the current influx of mHealth interventions as a wave of “black boxes” because there is a lack of research detailing the developmental processes of and subsequent expectations for these interventions. The paucity of research on the development of these initiatives reflects a primary concern for health outcomes in the mHealth intervention literature despite the fact that the processes of development are essential to the establishment of optimal and scalable interventions [21].

    In addition to the lack of research in relation to the design and development of cessation apps, there is also a lack of research on end-user (those who actually use the apps) perspectives and experiences and if they align with the expectations of development. While some studies have harnessed end-user input during the development of an app [23], the actual experiences of end-users after app roll-out has yet to be interrogated. We don’t know if users’ actual experiences align with the design expectations of the developers. This is a critical step toward enhancing uptake and engagement with interventions. For example, while Ploderer and colleagues [24] did not make explicit the underlying development principles of the DistractMe app, they did describe which end-user experiences aligned with the design intentions of the app. As a result, the authors were able to draw conclusions about which aspects of the app worked well and how development practices can be improved to optimize its effectiveness.

    Furthermore, it is well established in the literature that there are significant gender influences on smoking behavior, and that gender-sensitive approaches to quitting smoking have been found to positively influence receptivity to and use of the interventions, as well as mobilize smoking cessation [25-27]. Yet, no efforts to investigate the influence of gender and ways to incorporate a gender-sensitive approach into mobile-based smoking cessation interventions specifically have been found. Because the ways in which young women and men use and benefit from smoking cessation interventions as they engage in quitting smoking may differ, attention should be paid to potential gender-related influences in the ways that an app is developed, as well as how women and men perceive and experience these interventions.

    To our knowledge, no studies to date have explicitly aligned developers’ perspectives and end-user perspectives on using an app for cessation. Juxtaposing the processes of development and design that underpin an app against user perspectives and experiences is essential to exposing misalignments and optimizing design practices. The aim of this study was to address an important knowledge gap by comparing the overall design approach of Crush the Crave (CTC), a quit smoking app that targets young adults motivated to quit smoking, with young adult women’s and men’s perspectives. As described in Baskerville and colleagues [23], the features and functions in the app are underpinned by principles of persuasive technology for behavior change and the US Clinical Practice Guidelines on what works to support quitting smoking. This study specifically compares the pros and cons of the app as perceived by the developers of CTC with the perceptions and experiences of young adult smokers who have used the app.


    Methods

    Design

    The CTC app was being evaluated through a randomized controlled trial (RCT) [28] at the time of the current study. In the RCT, baseline, 3-month, and 6-month follow-up surveys were collected to assess the effect of CTC on smoking cessation compared to the quit guide. The present study served as a companion study to this RCT. Young adult participants for the present study were drawn from the individuals in the intervention group that completed all 3 surveys (n=307).

    Through a qualitative case study design [29], the processes of development that underpin CTC were juxtaposed with end-users’ perspectives and experiences in using the app, a knowledge area that we know little about. A qualitative case study is ideal for engaging in a holistic investigation of a phenomenon on which little is known [29]. Congruent with this methodology, data collection and analysis were qualitative in nature.

    Ethics approval for this study was obtained from the University of British Columbia (Okanagan campus) Behavioral Research Ethics Board (Certificate H15-00466).

    Participants

    Developers

    Using a purposive sampling strategy, developers were derived from those who were directly involved in the decision making in relation to the development, design, and implementation of CTC. Developers were put in touch with the principal investigator via an introductory email sent by the senior scientist of CTC and subsequently recruited via email. Altogether, 15 developers provided informed consent and participated in the study. Table 1 provides a description of the key informant sample.

    Using a purposive sampling strategy, young adults (ages 19-29) who had been assigned to the RCT intervention group and were either recent quits or still smoking were recruited into the current study. Participants recruited were those who completed the 6-month questionnaire in the RCT and selected “yes” to receiving information about this qualitative companion study. These participants provided their contact information (email and phone number) and, therefore, recruitment was conducted via email, phone calls, and texting. Altogether, 31 young adult end-users provided informed consent and participated in this study. Table 2 provides demographic and smoking behavior data for young adult participants.

    Data Collection

    Semistructured interviews were held with developers and end-users, which are less directive and more open-ended [29] to ensure that the perspectives of each sample were captured. Except for one Skype interview with a developer, all interviews were conducted via telephone. Interview questions for developers focused on perceived strengths and limitations in the design of the app, while interview questions for end-users focused on their likes and dislikes related to app design. These conversations were about the overall features and functions of the app rather than the specific content (eg, for developers: “In your opinion, what are some of the best things about the app?” and for end-users: “Looking back over the time you have used the app, what were some of the things you liked best about using CTC?”).

    All interviews were audio-recorded and lasted between 30 and 80 minutes. Young adult participants received an honorarium (Can $50/interview) to acknowledge time spent on the study.

    Table 1. Key informant sample.
    View this table
    Table 2. Young adult study population (N=31).
    View this table

    Data Analysis

    Data analysis was conducted simultaneously with data collection, with a minimum requirement of six interviews for saturation [30]. Saturation was ultimately driven by saturation of themes within the overall framework of the study. All interviews were transcribed verbatim. Transcripts and documents were uploaded onto NVivo(QSR International (Americas) Inc.). An analysis was then guided by the framework approach, which consisted of a series of interconnected stages (familiarization, identifying an analytic framework, indexing, charting, and mapping and interpretation), enabling a coherent and transparent account of the analysis [31].

    Data from interviews with young women were kept as a separate dataset from those with young men to enable the lead author to compare and contrast young women’s and men’s experiences and identify notable gender-related influences in the datasets and findings. After the first four interviews with developers, young women, and young men, a coding framework was developed, identifying key themes in relation to their overall perspectives on the app. The thematic frameworks for each dataset were then reviewed and approved by all authors. The frameworks were subsequently used to code the remaining transcripts and revised as added to as new data emerged. The frameworks for young women and men were compared and then combined with important similarities and differences noted. The final frameworks were then transferred to tables. Representative quotes were selected to illustrate key themes and subthemes.


    Results

    The perceived pros and cons of CTC fell into 4 categories: (1) technology and platforms utilized for the intervention (relates to perspectives on delivering a cessation intervention via a mobile app, with the additional support of social media, (2) foundation of app content (relates to perspectives on the principles that underpin the app design), (3) underlying focus of the app (relates to perspectives on the implied focus of the app design based on the technology used and the dominant features and functions that were built into the app), and (4) look, feel and functionality of the app (relates to perspectives on the overall design of the app and how it is packaged for young adults).

    Technology and Platforms Utilized in the Intervention

    Developers described the use of mobile technology as a natural evolution of tobacco control efforts—that it was necessary to keep up with current trends of using digital media in tobacco control, and health care more broadly. Along with this vein, there was often a sense of urgency to take tobacco control efforts into the realm of electronic health (eHealth). Notably, this urgency was underpinned by a desire to get it right and to not just put something out there that wasn’t given much thought. It was clear that the developers were invested in designing an eHealth approach that would work rather than something that was simply novel:

    This is going to be the app or some version of something like that—some portable, accessible, customizable, personalized thing. Every trend is going to that; this is the future…We need to understand how to get this right because if we don’t, the tobacco companies will and other people will, and we [will be] competing against all kinds of other things. If we can get this right, this [will become] a frontline for public health…If we don’t go there, we’re losing an enormous opportunity to make a big difference in people’s lives.
    [Informant #9, male]

    Both developers and end-users thought that a cessation app has an edge on other cessation intervention formats because an app would meet diverse populations of young adults where they are at, both in terms of being ready at hand and in terms of the type of support they’d want to receive. For example, a key informant reflected on how young adult smokers have been neglected in relation to tobacco control efforts, mainly because the health-seeking behaviors of young adults are diverse and can differ from their older counterparts, with whom young adults are often grouped together in cessation initiatives (eg, going to a physician, counseling). She described CTC as an intervention that was designed to align with how young adults seek health information and support, saying that CTC “is a place that they can go to get that kind of support that is not totally out of their comfort zone”. End-users agreed that the use of mobile technologies for a smoking cessation intervention fit with the needs and preferences of their age group. They particularly described the portability of CTC, since it is delivered via an app, as one of the most liked aspects of using the CTC app:

    Just the fact that it is an app and I can bring it with me instead of like having a chart at home or something like that where it’s not very portable. The app you can bring with you wherever you go ‘cause chances are you’re gonna have your phone on you.
    [Participant #2, female smoker]

    Developers and end-users agreed that integrating the app with social media platforms enabled easier access and opened opportunities to reach young adults with cessation support. While end-users did not perceive any downfalls to delivering cessation support via an app, developers were concerned about the ever-changing nature of technologies and platforms, and user preferences and implications this had for keeping interventions relevant:

    Things go by the wayside so fast that by the time we probably tried to work something out, Snapchat would be no longer the thing that young adults are using. They would be on to the next thing. So, it’s so hard to keep up with technology just because it’s such a quick pace nowadays. Like phones—new phones come out every six months now…similar to apps and similar to the different platforms that people are using.
    [Informant 11, male]

    While developers didn’t anticipate technical problems, this was a frequent problem encountered by end-users (eg, freezing), leaving them feeling empty-handed in trying to get help with quitting smoking. These glitches frequently drove users away from the app. A few end-users also lamented the fact that the app could only be used if they had an internet or data connection. This often resulted in limited use of the app, as well as disappointment in not being able to access the app and its features during times that they needed it (eg, during a craving).

    Foundation of App Content

    The most talked about strength underlying the design of the app among developers was that its content and development was informed by evidence and theory. In particular, developers described the inclusion of end-users via focus groups during the development of the design concept, as well as to test the app prototype, as a key strength. This led many developers to suggest that the app would be relevant to various subpopulations of young adults, and it was often described as a “one-stop shop” or a “Cadillac” design because it included so many evidence-informed features. As demonstrated in the statement below, developers thought that the evidence-informed nature of the app gave CTC an edge on the growing number of quit smoking apps available:

    There were plenty of stop smoking apps out there but they weren't particularly good ones. They weren’t…[based] on the evidence of what we know works, and they weren't based on good theory around behavior change…There was an obvious gap there and we wanted to try and make sure there was something good available for young people who would be looking for apps.
    [Informant #14, male]

    A few developers, however, had counterviews about the staying power and transferability of clinical practice guidelines and long-standing theories related to behavior change in the eHealth context. Developers were cautious about taking guidelines that were designed for a clinician in a clinic and putting them into an app for users to support self-management of health behavior. Some developers were also concerned that the app was built upon the notion of a successful quit trajectory. Rather than making room for the relapses that frequently occur during quitting, the app implies that quitting is straightforward (eg, that end-users can expect to be smoke-free by the time they reach their quit date). A challenge during development was balancing an idealized notion of quitting with the reality of a quitting trajectory:

    The biggest limitation that I see is that they [the features of the app] are based on a generalized model of quitting smoking, even if you think about the theoretical model, it just kind of goes forward right? And they will acknowledge that yes people go back and forth…[But] the reason you don’t draw…anything other than an arrow, aside from maybe something going backward occasionally, is that it looks like spaghetti when you actually think about how people change...It’s a complete mess...people go forward, backward, sideways...But that’s almost impossible to put on a program. And so, we run this risk of creating a bit of a fiction, an evidence-informed fiction, if we may.
    [Informant #9, male]

    When asked about how gender was considered during the development of the app, key informants explained that the app was designed to be “gender neutral” and therefore, was not underpinned by consideration for gender. Some key informants thought that this led to them defaulting to a rather male-centric app:

    Definitely, the app is very much branded to your white male. Especially with the default images at the front with the rock climbing and that sort of thing so—that initial white male [feel] is unfortunately very prominent when you look at it through that lens.
    [Informant #10, female]

    What end-users liked about the foundation of the app content was that, despite it being developed and informed by evidence and scientific institutions, the content was delivered in a fun and positive way. Many end-users described how they typically expect to receive messages about the negative consequence of smoking versus the positives of quitting. Because the app focused on the latter, end-users were more receptive to it. One young woman who still smoked provided a detailed description of why the ways in which the content was delivered was appealing to those her age:

    [The app] is not just pure scientific, “this is what you have to do, this is why you need to quit smoking, these are all the chemicals that are in it,” you know what I mean? There was more of a fun aspect to it. Where it seemed like it wasn't so serious…And so I liked that about it, right? 'Cause it wasn't so stuffy, it wasn't so clinical. It was more like, “okay, we know that you wanna quit, but we're not gonna judge you too severely if you don’t,” right? I don’t wanna say that it didn’t seem as serious, it just didn’t seem as clinical.
    [Participant #22, female smoker]

    The Underlying Focus of the App

    A strength of the underlying focus in the design of CTC according to both developers and end-users was the individually-led nature of the intervention. In this respect, the CTC app was designed to enable end-users to quit on their own and track and modify their smoking behaviors without consultation with others, health professionals and personal networks alike. It was agreed by both developers and end-users that this individually-led focus was compatible with how young adults generally approach quitting smoking, which is on their own. In particular, this design feature was viewed as a strength for reaching young men. One female key informant described how the self-driven nature of the app played to many men’s preferences for self-management when it comes to their health:

    I mean, men like tools…it’s like a tool, it’s a do it yourself, I don’t have to tell anybody, I don’t have to ask anyone [thing]…Yeah, I think the fact that it’s a do it myself; [I don’t have to] ask for help sort of thing, and it’s like one-stop shop.
    [Informant #5, female]

    In keeping with these sentiments, young men were particularly vocal about liking the self-led design of CTC, explaining that they are very private and like to take on quitting smoking independently. Young men frequently stated that they did not like to share personal things, such as the decision to quit smoking, with anyone—friends and family alike. Young women, on the other hand, were more inclined to draw on social support options within the app (eg, the quit buddy).

    An aspect that women liked, but men were not keen about, was the “give and take” focus in the app. While men wanted to be less engaged with data entry, women appreciated the need to be engaged throughout the process because it provided them with opportunities to reflect on their smoking and to develop new ways of coping:

    Yeah, so it’s not just like documenting data, it’s giving you something to do…And, it’s like—it also takes and gives to you… it’s like, “what are you doing when you crave or when you do have a cigarette…what you’re feeling then,”…Some of the other apps, most of them, you have to buy that information. Like you have to pay, God only knows how much, and this app, it’s just like, okay it’s there for you.
    [Participant #12, female, smoker]

    Look, Feel, and Functionality of the App

    In relation to the app’s aesthetics, developers thought that they were done well:

    There is something that has to catch their eye, and we talked a lot about the design of it—like literally the design, like the logo, that sort of stuff because that sort of stuff does matter...It’s a nice logo, it’s different, it’s a cool name…I think this team got it.
    [Informant #9, male]

    Interestingly, almost all end-users, women and men alike, did not like the way the app was aesthetically packaged. In keeping with the one key informant’s thoughts, they described the app as too dark, espousing a negative effect, which contradicts the otherwise positive orientation of the app. This is captured in the following statement:

    One thing that I did notice is that the background colors; they’re a little dreary. Like the black and the orange and like when you first click on it….it could be a little more like brighter and happier….A different color scheme I think would work a lot better…And like black, from a psychological standpoint, black and red, they’re like angry negative colors…So if you put more like blues and greens and like yellows and like summer colors and things like that in it, it might change people’s you know mood a little bit more, psychologically without them even knowing.
    [Participant #2, female smoker]

    One thing that end-users stated that they liked about the look and feel of the app was that it was easy to use. This is demonstrated in the following quote by a young woman:

    It was easy to get to, easy to use. Especially like being a mom…it was easy and simple. It wasn’t overly complicated—like to start, like the start-up was [easy to] enter stuff…it wasn’t overly long…[My son] only lets me use my phone for like two seconds at a time.
    [Participant #25, female smoker]

    Despite that the app was often described as easy to use, end-users agreed that navigating the app was not very intuitive. They often questioned why there were so many subpages of pages, which led to “hidden” features they were unaware of, or found out about late in their quit smoking journey. They also expressed frustration with being sent out of the app to access certain tools (eg, craving distractions).

    A limitation identified by end-users in relation to the functionality of the app was that the functions (awards, leaderboard, health calculators) were based on a point in time—the user’s quit date. One young woman described how the calculators (money saved, health regained) could be very powerful for quitting smoking but that the glitches in the app prevented her from logging her smoking and cravings, which lent to inaccurate statistics displayed on the app:

    I couldn’t actually log how many cigarettes and stuff I had …[so] it’s not accurate. But if it was accurate it [would be] cool to see like, you know, money saved, like, oh hey, I saved $100 smoking so far. Like you know, it’s something to be proud of.
    [Participant #30, female smoker]

    Discussion

    Principal Findings and Implications

    To date, there has been a primary reliance on quantitative research evidence for evaluating eHealth interventions. The rich findings as a result of harnessing perspectives of both developers involved in app design and end-users using CTC hold potential for contextualizing why certain aspects of the app worked well and others did not work well, and in doing so extend the results of the quantitatively focused evaluations of CTC [28]. The importance of gaining knowledge about the implementation processes and experiences associated with interventions has been recognized [32-34] and is supported by the findings in the present study.

    This qualitative study is novel in that it provides a formal comparison of the developers’ and end-users’ perspectives providing much needed empirical evidence to the eHealth literature. By gathering the viewpoints of developers, both problematic and effective approaches that underlie development goals were revealed. Indeed, developers and end-users in the current study findings can advance the development and implementation of eHealth interventions, holding great promise to improve their uptake and impact compared to their current overall status, which is often poor or undecided [35-37].

    The positive feedback by both sample groups on entering the mobile market for supporting smoking cessation demonstrates that the use of mobile phone technology is a much-needed and relevant approach for supporting quitting smoking among young adults. Emerging evidence of the efficacy of some evidence-informed apps for quitting smoking [11,18,19,38] confirms that smoking cessation interventions are appropriately positioned in the mobile context. In the recent RCT of CTC (Baskerville et al in press), it was found that, while CTC was not superior to the control condition, the prolonged abstinent rate and thirty-day abstinent rate for CTC was comparable to other research on smoking cessation smartphone apps [11,38]. The widespread support by end-users in this study confirms support for entering the mobile space to reach young adult smokers specifically.

    While it is established that we are on the right track with using mobile technology, the differential and sometimes problematic experiences with CTC among end-users bring forward questions about how and when the target end-users are appropriately engaged in intervention research. Although CTC was designed and developed with input from young adult focus groups, the findings of the study revealed gaps between the developers’ perspectives and the perspectives and experiences of the end-users. As previously described [23], apart from one pilot test run with end-users, engagement with end-users primarily consisted of preintervention focus groups. This raises questions about the value in relying primarily on preintervention focus groups in the development of mHealth interventions. Recent research has detailed the problematic position of end-users in the development of eHealth interventions—they are often peripheral stakeholders that have marginal engagement during the development [37], which was the case for CTC. Others have argued that positioning end-users in this way have the potential to lead to a mismatch between technologies and end-users’ daily lives, habits, and rituals [37], lending to usability problems and high attrition rates [35,39,40]. The findings of this study provide further support for these concerns. For example, many end-users complained about technology glitches (eg, freezing) and the lack of intuitive design (eg, features and functions were not easily accessed or located). Users often cited these issues as contributing to their disinterest and eventual disengagement with the app.

    Researchers in eHealth have highlighted the need for and benefits of harnessing end-user perspectives during the design and development of eHealth behavior interventions. For example, involving end-users has been shown to improve usability [41], prevents the inclusion of superfluous features [42], and can be more economical in that money is not put into bad design aspects [41]. However, what is lacking in the literature is how and when to engage end-users in eHealth intervention research. This raises questions about the appropriateness and effectiveness of the ways in which end-users were engaged during the development of the app. Recently, eHealth researchers have begun to pay close attention to the developmental requirements of health behavior interventions so that these interventions can be more effectively developed and subsequently scaled up. In this vein, it has been suggested that multiple formative evaluations be conducted with end-users to test design assumptions and prototypes [37,43]. One way to address this need would be the inclusion of end-users on the development team in addition to conducting feature-level analyses based on log data (eg, google analytics), such as that used by Heffner and colleagues [44]. These practical strategies may help address the need for more comprehensive and frequent end-user input, while also addressing issues in relation to time, resources and funding that are commonly associated with evaluating eHealth interventions.

    A recently published qualitative investigation of CTC found that end-users did not engage with the social support aspects of CTC due to fear of judgment, failure in quitting, and shame in smoking [16]. Considering the current study finding in relation to end-users’ preference for an individually-led intervention, it can be reasonably argued that the social support features were not designed in a way that aligned with the nature of the intervention. This highlights the need to pay attention to how content is presented and if it aligns with the overall focus of the intervention (eg, an app-based forum versus a public social media page).

    Furthermore, while developers focused on the scientific background of the app content, end-users focused, again, on how this content was presented. Positive framing of the app and its content appeared to play an important role in uptake and use of the app. Despite some developers concerns about uptake, end-users described how the apps focus on the benefits of quitting versus the consequences of smoking largely influenced their desire to download and use the app. Given traditional approaches that frequently played on fear (eg, pictures of negative health consequences on cigarette packs), guilt (eg, neonatal health consequences), or judgment [45], end-users welcomed the positive and encouraging nature of the app. In the debate between positive and negative framing of content for tobacco control efforts, the findings of this study extend existing evidence that positive message framing resonates with smokers [45,46] and specifically resonates with the young adult population [47].

    CTC was designed to be “gender-neutral.” A gender-neutral approach in cessation interventions can be understood as gender-blind, running counter to best practice frameworks and guidelines for treating tobacco dependence [48,49]. Researchers have raised concerns about the lack of attention to gender in cessation interventions given evidence that gender-related factors play a significant role in tobacco use [50,51]. For example, men have a long history with tobacco use and dependence that has been linked to masculinities and gender roles. Similarly, gender-related factors have been implicated in women’s smoking, with femininities and attractiveness associated with women’s smoking and gendered factors such as concern for weight gain contributing to smoking maintenance [52]. Furthermore, a gender-neutral approach puts emphasis on the end-goal (quitting among end-users), ignoring gender-related factors that may limit ones’ ability to quit smoking. For example, oftentimes, young women have been reported to take up and maintain smoking/substance use to cope with current trauma or past trauma (eg, domestic violence) [53]. Given reports that 1 in 3 women experience trauma [54], a gender-neutral approach fails to account for and address this issue. Along with this vein, discourses in relation to gender roles and norms (eg, women are responsible for their personal and familial health) may be reinforced through a gender-neutral approach because gender-related diversity and differences are ignored and unaddressed. It is naïve to focus on the end-goal (in this case, smoking cessation) and not account for established factors that may prevent one from achieving important health behavior changes, like quitting smoking. Indeed, in keeping with the positive impacts of gender-sensitive interventions [25-27,55], it is strongly recommended that mobile-based smoking cessation interventions be designed to address gender-related factors influencing smoking and quit efforts.

    That end-users did not like the aesthetics of the app because it stimulated negative emotions, brings attention to the emotional side of end-user experiences, something that has been neglected by researchers investigating human-technology interactions [56]. Thuring and Mahlke [56] assert that researchers are primarily focused on effectiveness, efficiency, and satisfaction at the neglect of other aspects, such as the aesthetics of system design and emotional experiences during system usage. In addition, researchers have found that the visual attractiveness of an app also influences perceived usability [56,57]. It is urged, therefore, that future development practices in the area of mHealth foreground consideration of the aesthetics of apps alongside effectiveness and efficiency.

    Limitations

    There are several limitations to this study. As with all self-report research, there may be perspectives and experiences that were not captured during the interviews. Also, interviews with developers were conducted a couple of years after the app was developed between 2012 and 2014. In this regard, perspectives of developers may have been influenced by current advancements in technologies, as well as their knowledge of what aspects of the app worked well and which ones did not work well. In the same way, some end-users were interviewed up to a year after they entered the RCT study, potentially limiting their ability to recall their experiences. To address this potential limitation, reflective questions were posed during interviews with both samples to assist participants in recalling events and experiences, and when necessary follow-up questions and probes were used to capture additional details. Also, the young adult sample largely consisted of those who continued to smoke and were primarily Caucasian, thus affecting the generalizability of the findings. Finally, evolutions in technology inherently challenge the transferability of eHealth research.

    Conclusion

    Given the rich findings of this study, particularly in relation to some of the stark differences found between developers and end-users, the inclusion of multiple perspectives is a much-needed addition to the eHealth literature. By gathering the viewpoints of developers, both problematic and effective approaches that underlie development goals were revealed. Incorporating a gender-based lens in this study brought forward nuances between young women’s and men’s perspectives and experiences with the app. Indeed, by harnessing data from both developers and end-users, the current study findings can advance the development and implementation of eHealth interventions.

    Conflicts of Interest

    None declared.

    References

    1. Surgeon General. 2018   URL: https://www.surgeongeneral.gov/library/reports/preventing-youth-tobacco-use/factsheet.html [accessed 2018-04-10] [WebCite Cache]
    2. Jha P, Ramasundarahettige C, Landsman V, Rostron B, Thun M, Anderson RN, et al. 21st-century hazards of smoking and benefits of cessation in the United States. N Engl J Med 2013 Jan 24;368(4):341-350. [CrossRef] [Medline]
    3. Reid J, Hammond D, Rynard V, Burkhalter R. Tobacco use in Canada. In: Patterns and trends, 2015 edition. Waterloo, Ontario: Propel Centre for Population Health Impact; 2015.
    4. Bader P, Travis HE, Skinner HA. Knowledge synthesis of smoking cessation among employed and unemployed young adults. Am J Public Health 2007 Aug;97(8):1434-1443. [CrossRef] [Medline]
    5. Curry SJ, Sporer AK, Pugach O, Campbell RT, Emery S. Use of Tobacco Cessation Treatments Among Young Adult Smokers: 2005 National Health Interview Survey. Am J Public Health 2007 Aug;97(8):1464-1469. [CrossRef]
    6. Hughes JR, Cohen B, Callas PW. Treatment seeking for smoking cessation among young adults. J Subst Abuse Treat 2009 Sep;37(2):211-213 [FREE Full text] [CrossRef] [Medline]
    7. Ybarra ML, Jiang Y, Free C, Abroms LC, Whittaker R. Participant-level meta-analysis of mobile phone-based interventions for smoking cessation across different countries. Prev Med 2016 Aug;89:90-97 [FREE Full text] [CrossRef] [Medline]
    8. BinDhim NF, McGeechan K, Trevena L. Who Uses Smoking Cessation Apps? A Feasibility Study Across Three Countries via Smartphones. JMIR Mhealth Uhealth 2014 Feb 06;2(1):e4 [FREE Full text] [CrossRef] [Medline]
    9. BinDhim NF, McGeechan K, Trevena L. Assessing the effect of an interactive decision-aid smartphone smoking cessation application (app) on quit rates: a double-blind automated randomised control trial protocol. BMJ Open 2014 Jul 18;4(7):e005371 [FREE Full text] [CrossRef] [Medline]
    10. Dennison L, Morrison L, Conway G, Yardley L. Opportunities and challenges for smartphone applications in supporting health behavior change: qualitative study. J Med Internet Res 2013 Apr;15(4):e86 [FREE Full text] [CrossRef] [Medline]
    11. Bricker JB, Mull KE, Kientz JA, Vilardaga R, Mercer LD, Akioka KJ, et al. Randomized, controlled pilot trial of a smartphone app for smoking cessation using acceptance and commitment therapy. Drug Alcohol Depend 2014 Oct 01;143:87-94 [FREE Full text] [CrossRef] [Medline]
    12. Hoeppner BB, Hoeppner SS, Seaboyer L, Schick MR, Wu GWY, Bergman BG, et al. How Smart are Smartphone Apps for Smoking Cessation? A Content Analysis. Nicotine Tob Res 2016 May;18(5):1025-1031. [CrossRef] [Medline]
    13. Abroms LC, Lee WJ, Bontemps-Jones J, Ramani R, Mellerson J. A content analysis of popular smartphone apps for smoking cessation. Am J Prev Med 2013 Dec;45(6):732-736 [FREE Full text] [CrossRef] [Medline]
    14. Abroms LC, Padmanabhan N, Thaweethai L, Phillips T. iPhone apps for smoking cessation: a content analysis. Am J Prev Med 2011 Mar;40(3):279-285 [FREE Full text] [CrossRef] [Medline]
    15. Choi J, Noh G, Park D. Smoking cessation apps for smartphones: content analysis with the self-determination theory. J Med Internet Res 2014 Feb;16(2):e44 [FREE Full text] [CrossRef] [Medline]
    16. Struik LL, Bottorff JL, Baskerville NB, Oliffe JL. The Crush the Crave Quit Smoking App and Young Adult Smokers: Qualitative Case Study of Affordances. JMIR Mhealth Uhealth 2018 Jun 08;6(6):e134 [FREE Full text] [CrossRef] [Medline]
    17. BinDhim NF, McGeechan K, Trevena L. Smartphone Smoking Cessation Application (SSC App) trial: a multicountry double-blind automated randomised controlled trial of a smoking cessation decision-aid 'app'. BMJ Open 2018 Dec 21;8(1):e017105 [FREE Full text] [CrossRef] [Medline]
    18. Buller DB, Borland R, Bettinghaus EP, Shane JH, Zimmerman DE. Randomized trial of a smartphone mobile application compared to text messaging to support smoking cessation. Telemed J E Health 2014 Mar;20(3):206-214 [FREE Full text] [CrossRef] [Medline]
    19. Ubhi HK, Michie S, Kotz D, Wong WC, West R. A mobile app to aid smoking cessation: preliminary evaluation of SmokeFree28. J Med Internet Res 2015 Jan 16;17(1):e17 [FREE Full text] [CrossRef] [Medline]
    20. Iacoviello BM, Steinerman JR, Klein DB, Silver TL, Berger AG, Luo SX, et al. Clickotine, A Personalized Smartphone App for Smoking Cessation: Initial Evaluation. JMIR Mhealth Uhealth 2017 Apr 25;5(4):e56 [FREE Full text] [CrossRef] [Medline]
    21. Tomlinson M, Rotheram-Borus MJ, Swartz L, Tsai AC. Scaling up mHealth: where is the evidence? PLoS Med 2013;10(2):e1001382 [FREE Full text] [CrossRef] [Medline]
    22. Ghorai K, Akter S, Khatun F, Ray P. mHealth for Smoking Cessation Programs: A Systematic Review. J Pers Med 2014;4(3):412-423 [FREE Full text] [CrossRef] [Medline]
    23. Baskerville NB, Struik LL, Dash D. Crush the Crave: Development and Formative Evaluation of a Smartphone App for Smoking Cessation. JMIR Mhealth Uhealth 2018 Mar 02;6(3):e52 [FREE Full text] [CrossRef] [Medline]
    24. Ploderer B, Smith W, Pearce J, Borland R. A mobile app offering distractions and tips to cope with cigarette craving: a qualitative study. JMIR Mhealth Uhealth 2014 May 07;2(2):e23 [FREE Full text] [CrossRef] [Medline]
    25. Bottorff JL, Oliffe JL, Sarbit G, Sharp P, Kelly MT. Smoke-Free Men: Competing and Connecting to Quit. Am J Health Promot 2018 Jan;32(1):135-142. [CrossRef] [Medline]
    26. Haines-Saah R, Kelly M, Oliffe J, Bottorff J. Picture Me Smokefree: a qualitative study using social media and digital photography to engage young adults in tobacco reduction and cessation. J Med Internet Res 2015 Jan 26;17(1):e27 [FREE Full text] [CrossRef] [Medline]
    27. Schwartz J, Bottorff JL, Ratner PA, Gotay C, Johnson KC, Memetovic J, et al. Effect of web-based messages on girls' knowledge and risk perceptions related to cigarette smoke and breast cancer: 6-month follow-up of a randomized controlled trial. JMIR Res Protoc 2014 Sep 30;3(3):e53 [FREE Full text] [CrossRef] [Medline]
    28. Baskerville N, Struik L, Hammond D, Guindon GE, Norman CD, Whittaker R, et al. Effect of a mobile phone intervention on quitting smoking in a young adult population of smokers: randomized controlled trial study protocol. JMIR Res Protoc 2015 Jan 19;4(1):e10 [FREE Full text] [CrossRef] [Medline]
    29. Stake R. The art of case study research. Thousand Oaks, California: Sage; 1995.
    30. Guest G, Bunce A, Johnson L. How Many Interviews Are Enough? Field Methods 2016 Jul 21;18(1):59-82. [CrossRef]
    31. Ritchie J, Lewis J. Qualitative research practice. London: Sage; 2003.
    32. Brendryen H, Kraft P. Happy ending: a randomized controlled trial of a digital multi-media smoking cessation intervention. Addiction 2008 Mar;103(3):478-84; discussion 485. [CrossRef] [Medline]
    33. Brindal E. The POWeR of looking into the black box. The Lancet Diabetes & Endocrinology 2016 Oct;4(10):800-801. [CrossRef]
    34. Danaher BG, Brendryen H, Seeley JR, Tyler MS, Woolley T. From black box to toolbox: Outlining device functionality, engagement activities, and the pervasive information architecture of mHealth interventions. Internet Interv 2015 Mar 1;2(1):91-101. [CrossRef] [Medline]
    35. Baker TB, Gustafson DH, Shah D. How can research keep up with eHealth? Ten strategies for increasing the timeliness and usefulness of eHealth research. J Med Internet Res 2014;16(2):e36 [FREE Full text] [CrossRef] [Medline]
    36. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, et al. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med 2011 Jan 18;8(1):e1000387 [FREE Full text] [CrossRef] [Medline]
    37. Van Gemert-Pijnen JEWC, Nijland N, van Limburg M, Ossebaard HC, Kelders SM, Eysenbach G, et al. A holistic framework to improve the uptake and impact of eHealth technologies. J Med Internet Res 2011 Dec 05;13(4):e111 [FREE Full text] [CrossRef] [Medline]
    38. BinDhim NF, McGeechan K, Trevena L. Smartphone Smoking Cessation Application (SSC App) trial: a multicountry double-blind automated randomised controlled trial of a smoking cessation decision-aid 'app'. BMJ Open 2018 Dec 21;8(1):e017105 [FREE Full text] [CrossRef] [Medline]
    39. Birnbaum F, Lewis D, Rosen RK, Ranney ML. Patient engagement and the design of digital health. Acad Emerg Med 2015 Jun;22(6):754-756 [FREE Full text] [CrossRef] [Medline]
    40. Lupton D. The digitally engaged patient: Self-monitoring and self-care in the digital health era. Soc Theory Health 2013 Jun 19;11(3):256-270. [CrossRef]
    41. Karat C. A Business Case Approach to Usability Cost Justification for the Web. In: Cost-Justifying Usability 2nd ed. San Francisco: Morgan Kaufmann; 2005:103.
    42. Kujala S. User involvement: A review of the benefits and challenges. Behaviour & Information Technology 2003 Jan;22(1):1-16. [CrossRef]
    43. Van Velsen L, Wentzel J, Van Gemert-Pijnen JE. Designing eHealth that Matters via a Multidisciplinary Requirements Development Approach. JMIR Res Protoc 2013 Jun 24;2(1):e21 [FREE Full text] [CrossRef] [Medline]
    44. Heffner JL, Vilardaga R, Mercer LD, Kientz JA, Bricker JB. Feature-level analysis of a novel smartphone application for smoking cessation. Am J Drug Alcohol Abuse 2015 Jan;41(1):68-73 [FREE Full text] [CrossRef] [Medline]
    45. Moorman M, van den Putte B. The influence of message framing, intention to quit smoking, and nicotine dependence on the persuasiveness of smoking cessation messages. Addict Behav 2008 Oct;33(10):1267-1275. [CrossRef] [Medline]
    46. Yang D. The Communication Effects of Audience Situation and Message Framing on Smoking Cessation. Rochester, NY: Social Science Research Network; 2013.
    47. Mays D, Niaura RS, Evans WD, Hammond D, Luta G, Tercyak KP. Cigarette packaging and health warnings: the impact of plain packaging and message framing on young smokers. Tob Control 2015 Mar;24(e1):e87-e92 [FREE Full text] [CrossRef] [Medline]
    48. Fiore M, Jaen C, Baker T, Bailey W, Benowitz N, Curry S, et al. Treating tobacco use and dependence: Update. US Dep Health Hum Serv 2008 [FREE Full text]
    49. World Health Organization. Policy recommendations for smoking cessation and treatment of tobacco dependence. 2018   URL: http://www.who.int/tobacco/resources/publications/tobacco_dependence/en/ [accessed 2018-03-22] [WebCite Cache]
    50. Bottorff JL, Haines-Saah R, Kelly MT, Oliffe JL, Torchalla I, Poole N, et al. Gender, smoking and tobacco reduction and cessation: a scoping review. Int J Equity Health 2014 Dec 12;13:114 [FREE Full text] [CrossRef] [Medline]
    51. Pederson A, Greaves L, Poole N. Gender-transformative health promotion for women: a framework for action. Health Promot Int 2015 Mar;30(1):140-150 [FREE Full text] [CrossRef] [Medline]
    52. Alexander SA, Frohlich KL, Poland BD, Haines RJ, Maule C. I’m a young student, I’m a girl … and for some reason they are hard on me for smoking: The role of gender and social context for smoking behaviour. Critical Public Health 2010 Aug 25;20(3):323-338. [CrossRef]
    53. Greaves L. Why Put Gender and Sex into Health Research? In: Oliffe J, Greaves L, editors. Designing and conducting gender, sex, & health research. Thousand Oaks, CA: SAGE Publications, Inc; 2012:3.
    54. World Health Organization. Gender, equity and human rights. 2018   URL: http://www.who.int/gender-equity-rights/understanding/gender-definition/en/ [accessed 2018-03-22] [WebCite Cache]
    55. Oliffe JL, Bottorff JL, Sarbit G. Supporting fathers' efforts to be smoke-free: program principles. Can J Nurs Res 2012 Sep;44(3):64-82. [Medline]
    56. Thüring M, Mahlke S. Usability, aesthetics and emotions in human–technology interaction. International Journal of Psychology 2007 Aug;42(4):253-264. [CrossRef]
    57. Tractinsky N, Katz A, Ikar D. What is beautiful is usable. Interacting with Computers 2000 Dec;13(2):127-145. [CrossRef]


    Abbreviations

    CTC: Crush the Crave
    eHealth: electronic health
    mHealth: mobile health
    RCT: randomized controlled trial


    Edited by G Eysenbach; submitted 10.04.18; peer-reviewed by E Bergner, E Edwards; comments to author 10.10.18; revised version received 15.10.18; accepted 17.10.18; published 07.03.19

    ©Laura Louise Struik, Joan L Bottorff, N Bruce Baskerville, John Oliffe, Susan Crichton. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 07.03.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.