Feasibility and Engagement of a Mobile App Preparation Program (Kwit) for Smoking Cessation in an Ecological Context: Quantitative Study

Introduction

Tobacco Consumption

The World Health Organization (WHO) considers smoking an epidemic that affects >1 billion people worldwide. It is a public health issue for many countries because tobacco use is one of the leading risk factors for a significant number of deaths and disabilities worldwide, and although preventable, the economic and social costs of disease burden remain very high [1]. On the basis of WHO recommendations, France’s national tobacco reduction program successfully addressed tobacco consumption, as the quit rate has been decreasing significantly since implementing several tobacco control policies in 2014 (ie, the reimbursement of approximately EUR €150 [US $166] per year for nicotine replacement products, monitoring of health warnings, media restrictions, and taxation of cigarette packs) [2].

Although all these policies have had some success in increasing smoking cessation, the long-term abstinence rate is still extremely low despite 60% of worldwide smokers expressing their willingness to quit. In France, at least 29.9% of daily smokers tried to stop for at least a week the previous year without long-term success [2]. In addition, results from double-masked clinical trials show that the willingness to quit and actual reductions in smoking behavior are limited by the content of nicotine present in cigarettes, even in participants who were not initially interested in quitting [3,4]. Complex factors must be considered for long-term abstinence, such as the social influence of marketing misinformation about tobacco consumption, nicotine pharmacokinetics, everyday cue conditioning, withdrawal symptoms, and a comprehensive discussion of treatment options and goals [5].

Treatment for Smoking Cessation

Multiple treatment approaches have been explored because of the complexity of tobacco dependence. To date, 2 major treatment categories have been used to treat tobacco use disorder: pharmacological treatments (eg, nicotine replacement therapy, bupropion, and varenicline) and nonpharmacological approaches (eg, motivational interviewing, cognitive and behavioral therapies, and acceptance and commitment therapy) [6]. In addition, new treatment goals have been proposed as alternatives to total and abrupt abstinence. These include gradually reducing cigarette consumption or using methods like snus and vaping to lower the risks associated with smoking, particularly for individuals with high nicotine dependence [4,5].

Both WHO and the French Ministry of Health recognize the impact of mobile health (mHealth) by promoting communication on social networks and using new technologies (eg, helplines, government mobile apps, and specific websites) to facilitate access to information and, thus, health care for smoking cessation [7,8].

mHealth Apps for Smoking Cessation

Overview

mHealth refers to emerging technologies for accessing health and medical services through mobile devices [9]. This includes mobile eHealth apps (known as mHealth apps), which are software that provide health and wellness services, designed exclusively for mobile devices (smartphones and tablets) [10]. With nearly 70% of the world’s population having a smartphone and access to the internet, mHealth apps can promote access to smoking cessation treatments [11]. mHealth apps can facilitate access to effective treatments and therapeutic information services by (1) adapting and translating “active principles” (the term used to refer to the various strategies and practices of evidence-based behavioral and cognitive therapies) into a digital format; (2) facilitating communication with health professionals, allowing a more flexible and personalized relationship (chat and social groups); and (3) encouraging a sense of responsibility and commitment to one’s health through “nudges,” such as positive reinforcement through messages or notifications, habit tracking, regular feedback, and audiovisual support [12].

Despite the many benefits of using mobile apps for smoking cessation, research is still in its infancy. It faces specific challenges: the quality of content, the range of potential uses, and engagement with mHealth apps for smoking cessation.

Content Quality of Mobile Apps for Smoking Cessation

According to a systematic review and meta-analysis, there is a relationship between the number of cognitive and behavioral techniques used in smoking cessation programs and short- and long-term effectiveness [9]. However, according to a recent review of the content analysis of mobile apps for smoking cessation in France, in 2020, (1) the average quality of mobile apps was 3.5 out of 5 (median 3.1; range 1-3) based on the Mobile App Rating Scale, (2) most of the apps made little use of the “active ingredients” of evidence-based therapies (between 4 and 38 out of 93 indexed cognitive behavioral therapy [CBT] techniques), and (3) there was a lack of information that delivers proper advice regarding the use of approved pharmacotherapy or the implementation of behavioral techniques specific to helping people prepare for smoking cessation[10]. In contrast, only some apps have been extensively studied and made available in France [11]; however, they still need to be presented using a proper taxonomy of behavior change techniques to facilitate further research on their mobile app content quality.

The Range of Potential Uses of an mHealth App

Current studies primarily address smoking cessation mobile apps as self-help material used as a stand-alone treatment by their users [12-14], not focusing on other treatment goals of smoking cessation interventions such as facilitating risk reduction and relapse prevention [5,15,16]. Moreover, mobile apps can also be used as a complementary tool that enables continual accompaniment of the patient throughout their treatment by health professionals. The transtheoretical change model framework developed by framework of Prochaska and Prochaska [17] will be used to illustrate this purpose.

This model states that a long-term change (ie, smoking cessation) results from different stages. Each stage has specific challenges; once addressed successfully, the person can continue their change journey by following the next stage. The first stages of change (precontemplation, contemplation, and preparation) are about raising awareness of the target behavior and creating an action plan by identifying the benefits of quitting, decreasing the perceived difficulties of change, and acknowledging the pharmacological and psychological therapy. Hence, mobile apps can highlight therapeutic information on dependency and how pharmaceutical aids work. Mobile apps can also be used as an ecological momentary assessment tool through which it is possible to monitor cravings in real time to collect information that helps to formulate hypotheses on the precipitating and maintaining factors that lead to tobacco consumption [18]. On the basis of the previous assessment, it is possible to individualize therapeutic interventions by understanding the individual experience of craving through repeated and ecological measures [15].

In the action phase, the person can change the target behavior [17]. Therefore, to manage cravings differently instead of smoking, a person can experiment with new methods that can be transitional and complementary, such as the use of pharmacological (ie, nicotine replacement therapy or medications) and psychological substitute strategies (ie, avoiding a specific place or person). In this phase, mobile apps can be a tool that prompts experimentation of new strategies by giving different types of feedback on behavior (ie, monitoring nicotine consumption, meditation exercises, distraction activities, etc) as well as being a motivational levier that highlights the benefits of smoking cessation or its reduction in different aspects of the user’s life (health benefits and savings).

In the maintenance phase, “relapse” is perceived as the norm and not the exception, so it can be a learning opportunity in which the goal is to anticipate and prevent high-risk situations as well as navigate through the moment of the onset of craving by creating a relapse prevention plan [17]. Mobile apps can be a tool that helps monitor emotional changes and track withdrawal symptoms, both very common to relapse. On the basis of previous use, the mobile app can also provide feedback indicating treatment effectiveness and the risk of relapses.

As a result of these insights, Kwit app—a French mobile app initially conceived for maintaining smoking cessation—released the Kwit’s 9-step preparation program for smoking cessation (9s-Kwit’s program) in July 2021. A new feature specifically designed for users self-identifying as being in the preparation phase of smoking cessation according to the transtheoretical model of change developed by Prochaska and Prochaska [17], this program is based on cognitive and behavioral therapy for smoking cessation as well as dialectical and behavioral therapy, motivational interviewing, and self-determination theory.

Engagement With Digital Interventions Such as mHealth Apps for Smoking Cessation in Real Settings

As in any treatment protocol, “adherence to therapies is an important determinant of treatment success” [19]. The smoking cessation intervention attrition rate could be as high as 77% [20]. Conversely, mobile apps for chronic diseases face similar challenges as other mHealth apps, with attrition rates ranging from 43% (95% CI 16-63) in randomized controlled trials to 49% (95% CI 27-70) in observational studies [21]. To date, the dropout rate of some smoking cessation apps tested in a controlled scenario could vary from 19.3% after 28 days of use [22] to 39.5% after 87.3 days [23]. However, it remains to be seen what the attrition rate of smoking cessation mobile apps in ecological contexts would be [24-27], particularly with organic users. In this study, “organic users” or “organic installs”—terms primarily used in mobile app marketing—refers to individuals who discover and install a mobile app on their own, without any influence from paid marketing. Crucially, for the purposes of our research, these organic users are characterized by their autonomy to download the app and pursue smoking cessation. This autonomy distinguishes them significantly from users who participate in studies for compensation or who are mandated to use the app by health care institutions. Therefore, it is hypothesized that organic users will exhibit different behaviors and levels of engagement in smoking cessation compared to their nonorganic counterparts.

In conclusion, even if an app includes all the therapeutic guidelines, it can only be effective if used regularly [10,24,27]. Understanding how users engage periodically with their health treatment is challenging for mHealth apps [24,27,28]. Therefore, it is essential to identify and assess the main factors that favor using mobile apps for smoking cessation. To this end, the various research methods should ensure ecological validity in a design of evaluation setup that matches the user’s real work context [29] and ensures that the behavioral responses obtained represent the natural behaviors of people who want to quit smoking with the help of a mobile app [30].

The aims of this study were, therefore, to (1) explore, in an ecological context, the engagement rate of a 9s-Kwit’s program on a mobile app and (2) estimate the impact of the program on the motivation level among users who finished the program.

We hypothesized that (1) given the ecological context of this study, we will observe a remarkably high dropout rate between 80% and 95% of the users participating in the study. In addition, we sought to examine some key factors’ role in the Kwit program’s engagement rate. Specifically, (2) we hypothesized that the attrition rate within the intervention could be explained by the version of the program (basic and premium), the lack of relevance and understandability of the program content, the primary motivation to quitting smoking (ie, “money” or “family”), the lower levels of internalization to stop smoking (external motivation), and the lower level of motivation to quit before starting the program. Finally, we hypothesized that (3) users’ perception of their willingness, ability, and readiness to stop smoking will significantly improve after finishing the 9s-Kwit’s program.

Methods

Overview

This observational study was conducted from July 4, 2021, to July 28, 2021, in an ecological context. It represents the natural behaviors of people who downloaded and used the app to prepare for smoking cessation. In detail, this ecological context includes the following (1) the participants were organic users (users that installed the app because of their search results and without having encountered paid advertisements before onboarding); (2) only information essential for the everyday use of the mobile app was collected, and only data required for the study were exported to be analyzed (ie, no demographic data were collected before onboarding); (3) no compensation was offered to the users for their participation in the study; and (4) they checked a nonpreemptive box to give their consent to the collection and analysis of data related to their use of the app for research and improvement purposes. If an organic user chooses not to allow Kwit app to use their data for research, this preference is recorded in the server. This choice does not affect the quality of the services offered by the Kwit app. All users, regardless of their data use choice, have the option to access the premium version of the app. This ensures that there is no ethical compromise in the provision of services.

Ethical Considerations

The local French Ethical Committee (South-East) validated the protocol study on March 25, 2021. The research identification number is 2020-A02733-36, and the committee’s reference is CPP 20.10.02.44945 [31]. This study adheres to the ethical principles outlined in the Declaration of Helsinki. This feasibility study is part of a prospective study registered on April 8, 2022 (ClinicalTrials.gov NCT05318651). It aims to identify critical determinants of smoking cessation mobile app use among smokers seeking to quit.

Participants

Participants were aged at least 18 years and had a compatible smartphone (iOS 13.3 and above) with regular internet service. In total, 2 groups of participants were followed according to the type of program version they had access to, basic or premium.

Before participating in the program, participants were required to create an account that pseudonymized their personal information, including billing information where applicable. The Kwit app offered 2 versions: premium and basic. The choice between them was influenced by whether the participant opted for a 7-day free trial.

The premium version was available to users who opted for the free trial and provided full access to all features of the program. In contrast, the basic version limited the user’s access to only 4 levels of the program (steps 0, 1, 2, and 8). It is important to note that both versions visually presented the same program dashboard screen to ensure a consistent user experience. The user journey leading to the 9s-Kwit’s program dashboard is detailed in Multimedia Appendix 1.

In terms of pricing structure, the premium version of the Kwit app was offered through a 7-day free trial during the study period from July 4, 2021, to July 28, 2021. This trial period gave users the flexibility to discontinue the service if they wished. After the trial period, if the subscription was continued, it was considered an agreement to a monthly membership fee of EUR €9 (US $9.96). This pricing was strategically set to approximate the cost of a pack of cigarettes in France at the time, aligning the cost of the app with a tangible smoking expense and making the value proposition relatable and practical for users.

Intervention

The 9-step Kwit’s program was developed after 2 years of research and development, as part of a PhD thesis, in collaboration with the University of Paris Nanterre. The content of this program was defined and coordinated by the cognitive behavioral psychologist and PhD student to be then executed by a team composed of a user experience designer, 3 software developers with at least 10 years of experience, a researcher in public health and clinical research, and 4 interns with a psychology master’s degree. The 9s-Kwit’s program has been developed for commercialization and designed for direct delivery as a mobile app intervention on Apple and Google Stores. This program is based primarily on cognitive and behavioral therapy for smoking cessation but is also inspired by dialectical and behavioral therapy, motivational interviewing, and the self-determination theory.

The 9s-Kwit’s program comprises 9 steps from step 0 (s0) to step 8 (s8). Through different activities, each step aims to explore a specific aspect of tobacco use that has already been identified as essential to consider in treating smoking cessation [9,32]. A presentation of the 9s-Kwit’s program’s home screen, a summary of its contents, and an example of the activity screen per step are presented in Multimedia Appendix 2.

The presentation of the activity starts as follows: each activity starts with an informative screen that introduces the activity’s main goals, then the activity itself, and ends with a feedback screen (Multimedia Appendix 3). When the user finishes an activity, the next one is unblocked. Different types of feedback coexist to inform the user that an activity is completed: (1) the color of the activity title changes (ie, in s0 from gray to blue), (2) information about the date and hour is presented instead of the average duration of the activity, (3) the icon next to the title activity changes from a lock to an arrow to show that the next activity is then accessible, and (4) the activities that are already done have a summary card if you click on it. Once all activities of the same step are completed, the next step is unlocked.

The initial step (s0) consists of becoming aware of one’s motivation to engage in a change process, but to change the target behavior, motivation alone is not enough; it is crucial to get out of “self-pilot” mode and adopt an “observer attitude” to understand the association between the target behavior, its antecedents, and its consequences. Therefore, in the first step (s1), users are encouraged to monitor their behavior and create a baseline of the contexts and intensity of cravings and the inner strategies that have helped them cope. The behavioral baseline provides users with an objective measure that assists them in recognizing patterns of cravings and helpful strategies. The second step (s2) focuses on the different components of dependency, and by using the Horn scale, it is possible to explain how opposing stimuli can induce people to smoke (ie, relaxation vs stimulation). On the basis of the results, users receive advice on new behavioral strategies. The third step (s3) proposes that users categorize cravings according to their short-term benefits (pleasure or relief) and then some mindfulness exercises. The fourth step (s4) defines a goal and a road map adapted to the user’s resources. Steps 5, 6, and 7 teach new coping strategies for the 3 dependence types (s5: environmental, s7: psychical, and s7: psychological). The final step (s8) invites users to define a quit date and acknowledge the cognitive barriers to quitting. The program encourages being kind to oneself when slips and relapses occur and celebrating each small step toward the desired change. None of the program versions used in this study have the gamification layer where users earn points at the end of each activity, have their avatar, or any educational reading content or community layout that now exists in the platform. Table 1 details all activities of each of the 9s-Kwit’s program according to the taxonomy on behavior change techniques developed by Michie et al [33].

Outcome Variables and Measurements

In total, 4 different types of measurements were used: ecological momentary assessment to measure engagement rate and version of the program (“a method of data collection whereby a record is made each time a predefined event occurs” [18]); a visual analog scale (VAS) to measure user’s perception of their willingness, ability, and readiness to quit smoking (the VAS slider ranged from 0 to 10); a Likert-type scale to measure the internalization degree to stop smoking, which can range from none (amotivation) to completely internalized (intrinsic motivation); and a multiple-choice question to measure user’s relevance and understandability of the program content and the users’ main reason to quit. Examples can be found in Multimedia Appendix 3.

Engagement Rate Toward the 9s-Kwit’s program

The engagement rate is the ratio of users who completed the program from the first use (step 0) to the last activity proposed in 9s-Kwit’s program (step 8).

Motives for Quitting Smoking

On the basis of previous studies on reasons for quitting smoking [34-36], users were asked to choose 1 of 5 reasons: health, wellness, money, family, and planning a pregnancy, using the following statement: “My main reason for quitting smoking is...” (Multimedia Appendix 3).

Motivation Level: The Willingness, Ability, and Readiness to Quit Scale

Using a VAS, users rated their perception of their willingness, ability, and readiness to quit smoking at the beginning (S0) and the end of the program (S0). For this purpose, each question was presented on a single screen and in the following order: (1) “To what extent this change is a priority for you right now?” (willingness), (2) “To what extent are you confident in your ability to change right now?” (ability), and (3) “To what extent do you feel ready to take action?” (readiness). This way, users could move a slider from 0 (lowest) to 10 (highest) on each screen (Multimedia Appendix 3). For statistical analysis purposes, scores <3 are defined as “low,” scores between 4 and 7 are defined as “moderate,” and scores >7 are defined as “high.”

Nature of Motivation to Quit Smoking

The French Smoking Cessation Motivation Scale (F-SCMS) is a self-reported measure based on the self-determination theory, demonstrating good internal consistency (α=.86; ωh=0.7; ωt=0.89) and content validity (common-fit index=0.905, standardized root mean square residual=0.045, and root mean square error of approximation=0.087). The 18 items are divided into 6 subscales (each composed of 3 items) corresponding to the degree of internalization that participants have toward quitting smoking. The subscales are presented from no internalization to complete internalization of a behavioral change process: (1) amotivation, (2) external, (3) introjected, (4) mixed, (5) identified, (6) integrated, and (7) intrinsic motivation. We have previously validated the scale (F-SCMS) used in this study and explained the internalization process in a separate publication [37].

The scale was presented as follows within the app. Each screen was composed from top to bottom in the following order: (1) the statement “Right now,” (2) the item appearing as a card, and (3) the list with 5 Likert-type response options ranging from 1 (does not match at all) to 5 (matches exactly). Once the user answered all the items, they could receive predefined feedback corresponding to their degree of internalization [37].

Perceived Content Relevance of s0

The last activity of s0 of the program was the evaluation by the users of the comprehensibility and relevance of the contents presented during this step. The question was presented as “You have completed Step X. How would you define it?” with the following 3 options: (a) understandable and relevant, (b) understandable but irrelevant, and (c) not understandable and irrelevant.

Statistical Analysis

We performed descriptive analyses to characterize the 2 groups’ baseline samples and outcomes of interest. We conducted the Mann-Whitney test and a χ² test of association to compare differences between both groups at the beginning of the program in terms of the main reason to quit, the motivation level to quit smoking, the nature of the motivation, and the perceived content relevance at the beginning of the program. We conducted a Student t test (2-tailed) to estimate the program’s impact on the motivation levels of the program completion for each group. For each test, a P value <.05 will indicate statistical significance. Analyses were conducted using Jamovi V2.3.8.

Results

Overview

A total of 2331 users started the preparation program. Overall, 91.89% (2142/2331) of the initial sample used the basic version of the program, and 8.1% (189/2331) used the premium version. As a reminder, the app’s premium version allowed access to each program step’s activities (from s0 to s8), whereas participants with the basic version had access only to s0, s1, s2, and s8 activities.

Table 2 presents the distribution of participants of both versions of the program in terms of (1) motives for quitting smoking, (2) motivation level to quit smoking, (3) motivation nature, and (4) perceived content relevance of step 0. In general, participants’ main reason for quitting smoking was health (1173/2331, 50.32%), followed by savings (450/2331, 19.31%), well-being (428/2331, 18.36%), family (172/2331, 7.38%), and planning a pregnancy (108/2331, 4.63%). At baseline, 80.65% (1880/2331) of the total sample was moderately motivated to quit smoking according to the Willingness, Ability, and Readiness to Quit (WAR) scale mean score, and 50.45% (1176/2331) fell into the 2 highest categories of internalization level for smoking cessation—integrated motivation (687/2331, 29.47%) and intrinsic motivation (489/2331, 20.98%)—according to the F-SCMS scale. Upon completion of step 0, the content of this step was considered by 71.55% (1668/2331) of the users as understandable and relevant (option A), by 10.98% (256/2331) as understandable but irrelevant (option B), by 0.82% (19/2331) as incomprehensible and irrelevant (option C), and 16.65% (388/2331) of the initial sample did not answer the question.

Motives for Quitting Smoking

According to the χ² test of association and Cramér value (n=2331; χ²₄=19; P≤.001; V=0.08), users of each version of the program have reported proportionally different reasons for quitting smoking. Those who choose to get the premium version of the program were less likely to choose savings as their main reason to quit smoking (premium=8.5% vs basic=20.3%) but more likely to choose the well-being (premium=23.3% vs basic=17.9%) and planning a pregnancy (premium=7.4% vs basic=4.4%) options.

Level of Motivation to Quit Smoking

Concerning the total score of the WAR scale, 3 levels of motivation were calculated based on the average score of 3 subscales: low (0-3), medium (4-7), and high (8-10). Most users (1880/2331, 80.7%) reported a moderate motivation to quit smoking at the beginning of the preparation program (s0; 1880/2331, 80.65%; mean 5.85 out of 10, SD 1.68). There was no significant difference between basic (n=2076) and premium (n=183) users when comparing these 3 categories (n=2331; χ²₃=6.48; P=.09) at s0a2. Nevertheless, when the ordinal data were compared using the Mann-Whitney test, the 2 groups differed significantly but with a very slight effect within the total score and every subscale. Users who had the premium version of the app reported feeling significantly more willing (z score=56,055; P≤.001; Cohen d=0.15), capable (z score=172,905; P=.04; Cohen d=0.09), and ready (z score=166,390; P=.005; Cohen d=0.12) to stop smoking than users who had the basic version before completion of the preparation program.

Nature of Motivation to Engage With a Smoking Cessation Process

The internalization degree of motivation to quit smoking differs significantly between basic and premium users according to the χ² test of association (n=2331; χ²₇=14.7; P=.04). Premium users are less likely to score in the “amotivated” profile (premium=1.6% vs basic=3.6%) and more likely to be driven by intrinsic motivation (premium=28% vs basic=20.4%) to engage in a smoking cessation process. In addition, the proportion of participants for whom no answer was received is lower in the premium group than in the basic group (premium=17/183, 9% vs basic=340/2076, 15.9%).

Perceived Content Relevance of s0

The content assessment of s0 differs between premium and basic users according to the χ² test of association (n=2331; χ²₃=9.7; P≤.02), and by contrast, there are fewer premium users for whom a response is missing (–7.2%).

Engagement Rate Toward the 9s-Kwit’s program for Smoking Cessation in an Ecological Context

Overview

As shown in Figure 1, the engagement rate with the program drops significantly at 3 different moments. The biggest dropout concerns both samples (basic and premium) in between activities s0a4 and s1a1 (basic=–90.3% and premium=–70.1%). The second dropout was observed in the basic user sample between s2a2 and S8a1 (–4.2%), and the third dropout was within the premium sample from S4a2 to S5a1 (–12.5%). The proportion of participants who started each program activity is detailed in Table 3.

‎

In addition, based on the χ² test of association and Cramér value, there is a significant difference between basic and premium users in terms of the engagement rate at the end of step 0 (s0a4; n=2331; χ²₁=7.8; P=.005), the beginning of step 1 (s1a1; n=2331; χ²₁=68.1; P≤.001), and the end of the preparation program (S8a3; n=2331; χ²₁=5.43; P=.02). Premium users were more likely than basic users to finish the last activity of step 0 (s0a4; premium=91.5% vs basic=85.2%), start the new step (s1a1; premium=26.9% vs basic=9.7%), and complete all the program’s activity until the definition of the quit date (S8a3; premium=4.7% vs basic=2%).

The Impact of the Program Completion on Users Who Finished the Program on Motivation Levels Toward Quitting

From an initial sample of 2331 ecological users, only 57 (2.4%) completed activity s8a2, which assesses the motivation levels regarding WAR scale. Table 4 presents the WAR scale and subscales scores from the beginning (s0a2) to the end of the program (s8a2) of the 2 users’ groups who finished the program (57/2331, 2.4%) as well as the Mann-Whitney score.

To estimate the program’s impact on the motivation levels of the program completion, Student t tests were conducted within each group.

At the beginning of the program, users who finished (57/2331, 2.4%) reported feeling, on average, highly willing to quit smoking (7.46/10), moderately confident in their abilities to do so (5.33/10), and moderately ready to start the quitting journey (6.30/10). A Mann-Whitney U test was performed to assess whether scores on the WAR scale and its subscales differed significantly between users of each group both at the beginning and at the end of the program. The results indicated that only subscale Ability differs significantly between both groups’ samples (basic and premium), with a moderate effect size (U=6.86; z score=118; P=.02; d=0.45) before (s0a2) and after the program completion (s8a2). Users who had the app’s premium version reported feeling significantly more capable of smoking cessation than users who had the basic version before (basic=5.02; premium=7) and after (basic=5.98; premium=7.33) the completion of the preparation program.

Premium users who finished 9s-Kwit’s program reported, with a high effect size, significantly higher scores than at the beginning of the program on the Willingness subscale (t₈=2.83; P=.02; Cohen d=0.9) and the total score of the WAR scale (t₈=3.16; P=.01; Cohen d=1.05). Conversely, users who completed the basic version of the program (4 steps) reported, with moderate effect size, significantly higher end-of-program scores in the Ability subscale (t₄₇=3.17; P=.003; Cohen d=0.46) and a total score of the WAR scale (t₄₇=2.92; P=.005; Cohen d=0.42), and with low effect size, significantly higher end-of-program scores in the Readiness subscale (t₄₇=1.857; P=.07; Cohen d=0.26).

Discussion

Overview

This study aimed, in an ecological context, to explore the engagement rate of a 9s-Kwit’s program on a mobile app (Kwit app); to examine some moderating agents that could contribute to the engagement rate we observed; and to estimate the impact of the program on the motivation level among users who finished the program. An ecological context refers to research conducted in natural, real-world settings as opposed to controlled laboratory environments. This approach is crucial for several reasons. First, it embeds our investigation in real-life scenarios where participants interact with the Kwit app as part of their daily routines. This allows us to gain authentic insights into user behaviors, app engagement levels, and the program’s impact, mirroring genuine use patterns more accurately than laboratory conditions could. Such insights are invaluable for designing interventions that are not only effective in theory but also practical and beneficial for users in their real-world context.

Principal Findings

Overview

A total of 2331 users, all of whom were active smokers without a set quit date, started the 9s-Kwit’s program. Most users (2142/2331, 91.89%) opted for the basic version of the program, whereas 8% (n=189) selected the premium one. Only 2.3% (54/189) of the initial sample reached the last step of the program (s8), which consisted of defining a quit date.

Notably, there was a marked difference in completion rates between the 2 versions: 4.7% (9/189) of premium users finished all program steps, which was higher than the 2.95% (44/2142) completion rate observed in the basic version. This significant disparity in engagement rates (n=2331; χ²₁=5.4; P=.02) highlights a greater commitment among premium users, despite them having more steps to complete.

To better understand the attrition phenomenon, we monitored user engagement at each program activity and observed 3 prominent dropouts. The first and most significant was observed between the last activity of step 0 (s0a4) and the beginning of step 1 (s1a1), with higher attrition rate among basic users compared to premium users (Table 2). The second dropout was observed in the basic user sample between s2a2 and s8a1 (–4.2%), and the third dropout was within the premium sample from activity s4a2 to s5a1 (–12.5%). These dropout patterns can be attributed to 3 main factors: (1) specific user characteristics (ie, areas of motivation and perceived content relevance), (2) the specific attributes of the app’s user experience and the cognitive effort required to engage with different activities, and (3) typical attrition rates for smoking cessation interventions and the regular use of mHealth solutions.

Specific User Characteristics at the Beginning of the Program

In this study, data collection was confined to what was essential for the program’s use, focusing on motivation areas and perceived content relevance, without collecting demographic information or insights into users’ health and technology literacy. Following the rationale that user motivation to quit and user satisfaction with the app determines user engagement, higher engagement rates were anticipated. Only 26.9% (56/2142) of premium users and 9.7% (209/189) of basic users progressed beyond the initial stage of the program (from s0a4 to s1a2). Notably, while >70% (1668/2331) of both groups rated the content as “understandable and relevant,” there was a significant perceived relevance gap, with premium users rating the app content more relevant by >10 points compared to basic users, as detailed in Table 2. This discrepancy suggests that content relevance may play a substantial role in influencing engagement levels, particularly among different user groups.

Significant differences were also observed in the initial sample (N=2331) between the 2 groups in all motivational measures: the reasons for quitting smoking, the WAR scale, and the internalization level to quit smoking. Premium users, who were more likely to choose wellness and family planning as their primary reason for quitting, also showed greater willingness, ability, and readiness to quit smoking at baseline compared to basic users, who were primarily motivated by financial savings. Interestingly, there is a difference between the willingness to quit and the user’s self-perception of their ability and readiness to quit. On average, both groups reported high levels of willingness to quit smoking (premium=7.78; basic=7.40) but a moderate perception of their ability (premium=7; basic=5.02) and readiness (premium=6.67; basic=6.23) to do so. In addition, higher levels of internalization were observed in the premium sample with fewer incomplete responses (premium=15.9%; basic=9%) and lower amotivation profile (premium=1.6%; basic=3.6%). They were more represented in profiles of identified (premium=9%; basic=6.4%) and intrinsic motivation (premium=28%; basic=20.4%).

In summary, while both groups reported being highly willing to quit smoking, premium users reported feeling more confident and prepared to start their smoking cessation process, more driven by internalized motivation and reasons to quit (ie, well-being vs savings), and more committed from the early stages of the program.

The Specific Attributes of the Mobile App User Experience

Despite being user centered and clinically guided, the 9s-Kwit’s program design might contribute to user disengagement at specific points: (1) for both versions in the transition between the last activity of step 0 (s0a4) and the beginning of step 1 (s1a1); (2) for the basic version, between step 2 (s2a2) and step 8 (s8a1); and (3) for the premium version, between step 4 and step 5. The AIM-ACT framework helps explain this attrition: emotional and motivational mismatches (AIM) and attentional or contextual misalignments (ACT) could hinder user engagement, especially during key step changes. This suggests the need for aligning app activities more closely with users’ affective states, goals, and contextual realities to reduce dropout rates [27].

The first problem arises in the transition of the app between the dashboard and the activity screen (Multimedia Appendix 2). When users complete all activities in a step, they are redirected to the dashboard, where an animation (an opening of a small padlock) indicates the next available step. This transition can create friction rather than facilitate smooth progression, potentially discouraging motivated users from starting new activities. Alternative design elements, such as more prominent visual cues (eg, a red circle above the start step) or direct navigation to the next activity screens, could improve user engagement. Second, for basic version users, the dashboard could be improved by more clearly indicating accessible steps, perhaps by visually distinguishing available and unavailable steps, thus reducing confusion and friction in progression. For example, it would have been possible to slightly blur the steps to which they did not have full access to highlight the user’s path and decrease friction between step 2 and step 8.

The third issue involves the cognitive load of certain activities, particularly those that require more active engagement, such as reading or logging cravings for a long period. This could be the case for the dropout in between step 4 (s4) and step 5 (s5); 3 types of exercises were proposed in s4 to be performed for 24 hours, setting goals for craving management. The design of these activities, including their duration and nature, might impact users’ perception of the effort-to-reward ratio as insufficient to continue engaging with the app program.

In general, using any game element, such as achieving certain milestones, will encourage learning new skills and knowledge by making learning more enjoyable and engaging and by enhancing self-esteem and motivation, which are highly associated with long-term smoking cessation [25,38]. The version of the Kwit app used in this review represents the first stage of an evolving series of modifications. This baseline version was designed to establish a solid evidence-based core, filling a gap where many mobile smoking cessation apps have lacked a scientific foundation [10,39].

The Standardized Attrition Rate for Smoking Cessation and mHealth Solutions

The dropout rate observed in this study could be because of the nature of tobacco use disorders and the selection criteria. Tobacco use disorders are well studied, with various effective treatments available; however, they typically exhibit dropout rates ranging from 10.8% to 77% in participants who had already defined a quit date as part of the selection criteria [20]. According to model developed by Prochaska and Prochaska [17], our participants would be at a lower stage because they would be just in the preparation phase.

Moreover, attrition is a known challenge in mHealth, with an average dropout rate of 43%, with higher rates in research under ecological conditions (98%) and lower rates in randomized controlled trials (up to 30%) [21].

In the case of studies on smoking cessation apps, the reporting of quit rates has yet to be standardized and is not always reported. This threatens the validity of some studies by introducing selection bias and making it difficult to compare the effect of such apps [11]. For instance, dropout rates in such studies range from 19.3% at 28 days [22] to 39.5% at 87.3 days of use [23] in controlled settings with English-speaking participants, while studies in ecological settings with French-speaking participants report rates of 65% at 30 days and 80% at 90 days of use [39] with a 1-year data collection period (vs less than a month for our study).

The Impact of the Program Completion on Users’ Motivation Levels Toward Quitting

To estimate the program’s impact on the motivation level, the analysis was based on 57 users who completed the activity s8a2 of the program: 2.2% of the users from the basic version (n=48) and 4.7% from the premium version (n=9). At the beginning of the program, both groups reported high levels of willingness to quit smoking, with an average willingness score of 7.46 out of 10. However, they reported only moderate levels of ability and preparedness, with average scores of 5.33 out of 10 for their perceived ability to quit and 6.30 out of 10 for their readiness to begin the quitting process.

After the program completion, substantial improvements in the WAR total score were observed in both groups, with different areas of improvement for each group. The premium users improved their willingness levels, whereas users with the basic version benefited in the ability and readiness category. Given the difficulty of assessing the effect of each individual component, we can only hypothesize about the program content’s effect on WAR scores. It is worth noting that throughout her work, Michie has linked several techniques based on behavioral theories to different behavior change as smoking cessation to the point that there is now an ontology and a whole system that would allow this question to be answered in future studies [40,41]. Therefore, the content of the 9s-Kwit’s program was mainly described through the taxonomy developed by Michie et al [32,33].

Given the intricate challenge of isolating and evaluating the effect of individual components within the program, our analysis regarding its impact on WAR scale scores must be approached with caution. As such, we posit that the program’s influence on users’ sense of competence and readiness likely stems from the implementation of various CBT techniques previously recognized as efficacious for smoking cessation in personal intervention settings [42].

Extensive research into personal interventions across varied demographics and environments has pinpointed specific CBT techniques—and their synergies—to be instrumental in elevating smoking cessation success rates. The 9s-Kwit’s program incorporates these proven techniques, including goal setting for behaviors and outcomes, using problem-solving strategies, revisiting and refining behavioral goals, fostering commitment, delivering feedback on behavioral outcomes, enabling behavior substitution, facilitating behavioral practice and rehearsal, and reconfiguring both physical and social contexts to bolster behavior change efforts [42]. These components are strategically combined to amplify the probability of cessation among users.

Moreover, the program integrates principles from acceptance and commitment therapy right from the outset, urging users to align with their core values, such as family, well-being, or personal health. Using metaphors, these values are presented as guiding beacons during moments of temptation (cravings) [43]. The “plus” button feature distinctively separates sensations from actions, offering various strategies such as the 4Ds (Delay, Deep breath, Drink water, Do something else) and meditation to manage cravings effectively [44]. In the second stage, the Horn scale demystifies the multifaceted reasons behind smoking habits, proposing tailored strategies for each [6]. Culminating in stage 8, the program prompts users to confront their quitting fears, providing customized solutions to navigate these apprehensions successfully and asking them to set a quit date.

These results suggest that Kwit’s preparation program can address ambivalence by increasing willingness to change, self-confidence, and readiness to quit among its users, especially those who felt less able to do so (users with the basic version). Self-efficacy or confidence in one’s abilities has been identified as a determinant in long-term abstinence [45].

Strengths and Limitations

Strengths

The strength of this type of study is simultaneously one of its major limitations; being observational in an ecological context, it offers a unique “snapshot” of real-life behavior of a group of users of a mobile smoking cessation app, providing insight into how its use may impact motivation outcomes. This approach not only sheds light on the “science of attrition” by recognizing when intervention dropouts occur but also aids in identifying real-life adoption challenges, thereby facilitating the generation of hypotheses to enhance health care treatment adherence, particularly with the evolution of commercial mHealth apps [21,29]. To the best of our knowledge, only 2 other studies have explored using mHealth tools [46,47] to treat smokers who are ambivalent about quitting, and these studies focused on French-speaking smokers. This is the third study to examine this topic within an ecological context (ie, popular mobile apps) and with a large sample size [39,48].

Limitations

The low generalizability of the findings was a limitation for several reasons, outlined in the following sections.

Lack of Individual-Level Measures

The absence of detailed individual-level data, particularly on smoking habits, dependency levels, and mental health issues such as anxiety and depression, is a significant limitation. These factors are known to be associated with lower adherence to treatment [48,49]. In additional, the study did not collect detailed demographic information, such as age and gender, before the use of the 0-step preparation program. Conducted within an ecological context and adhering to General Data Protection Regulation, the study’s data collection was limited to what was essential for app functionality, excluding pre-use demographic details.

Sample Size and Demographics

The study’s sample is limited to iOS users, excluding Android users, who may differ in terms of quit attempts, quit date settings [50], and sociodemographics [51]. Given the global market share and diverse user base of Android, this limitation means the findings may not fully represent the broader population of smartphone users. The decision to focus on iOS users was driven by specific constraints at the time of the research design, making the results more reflective of the iOS user group.

Short Recruitment and Follow-Up Duration

The short duration of the recruitment phase and follow-up period is another limitation. Smoking cessation is a complex process often involving cycles of relapse and recovery. A brief follow-up may only capture a snapshot of this process, potentially overlooking critical data on sustained abstinence or long-term relapse rates. It may also fail to adequately assess dropout rates and reasons for disengagement, which are crucial for understanding and improving app efficacy.

Small Sample Size of Premium Users

The explanatory power of the study is also limited by the small sample size of premium users who completed the 9s-Kwit program. Although similar attrition rates have been reported in other ecological studies [39,48], the small number of premium users reduces the generalizability of the findings.

Future Perspectives

To address all these limitations and investigate user engagement over extended periods, we have established a new protocol in collaboration with the University Paris Nanterre, registered in ClinicalTrials.gov (NCT05318651). This initiative aims to provide a more comprehensive understanding of the program’s impact over time.

Despite the limitations outlined, studies such as ours are critical for shedding light on 1 stage of the mHealth app design process. This research provided the Kwit design team with critical early insights that allowed for rapid iterative adjustments through A/B testing before launching the program on the Android platform and localizing it for different language audiences. As for 2024, the newly developed program integrates game-like features (awarding users points for successfully completing activities, offering different levels, and allowing for avatar customization), educational reading materials related to the topic of each step of the program, and a community design that was not previously accessible. It also introduces educational reading materials and a community design, which were not previously available on the platform. It remains to be seen how this proactive and responsive approach to app design not only optimizes user engagement but also avoids significant costs. It underscores the critical role of early and ongoing user feedback in refining mHealth interventions to ensure that they are both effective and engaging.

Rather than focusing on increasing abstinence as a stand-alone intervention, new studies should aim to improve the use of these and other interventions (as a complementary treatment rather than a comparative one) [48] and consider dropout rates as a measure of intervention quality. A meager retention rate highlights the current inability to satisfy users’ expectations and needs of those seeking to quit [21]. To increase the duration and intensity of app use, qualitative studies are needed to obtain a user-centered perspective. In addition, small trials with users can help guide decision-making on design tweaks or new features. In contrast, through a quasi-experimental observational study based on the technology acceptance model, it would be possible to identify the determinants of engagement with a smoking cessation app [31].

Conclusions

This study on the first version of the 9s-Kwit’s program provides crucial insights into the challenges and potential of mHealth apps in facilitating smoking cessation efforts. Despite a modest completion rate—with only 2.3% (57/2331) of the participants reaching the program’s final step—the data unveil a significant difference in engagement levels between users of the basic and premium versions.

At the program’s onset, participants displayed a strong willingness to quit, yet their self-assessed ability and readiness were only moderate. Notably, premium users reported feeling more capable of quitting than basic users—a sentiment that not only persisted but also intensified by the program’s end. Subsequent evaluations indicated significant enhancements in willingness, ability, and readiness to quit across both user groups, with premium users showing an increased willingness and basic users experiencing considerable gains in self-assessed ability and readiness to quit.

These results imply that, despite the study’s limitations, the 9s-Kwit’s program may address user ambivalence, thereby enhancing willingness, confidence, and readiness to quit, particularly among those who initially felt less capable. Such improvements in self-efficacy are vital, as they play a pivotal role in achieving long-term smoking cessation success [45].

Ultimately, this research adds to the broader discourse on digital health interventions, underscoring the critical importance of adaptive design and user-centric approaches in developing effective mHealth solutions [24,26,27]. The observed attrition patterns highlight the intricate interaction between user characteristics, the cognitive demands of the app, and the inherent challenges of smoking cessation. This emphasizes the essential need for early, iterative design processes that are informed by user feedback.

However, the study’s conclusions must be approached with caution because of limitations such as the small sample size, the exclusive focus on iOS users, and the lack of comprehensive demographic data. By acknowledging these limitations, we have initiated a new research protocol with the University of Paris Nanterre (NCT05318651) to extend our investigation. This study aims to assess the long-term impact of the 9s-Kwit’s program on a more diverse and extensive participant base.