To be eligible, participants had to be aged 18 to 65 years, smoke at least five cigarettes daily, want to quit smoking in the next 30 days, own an iPhone with iOS 8 or higher capabilities, be willing and able to receive text messages, be able to comprehend the English language, live in the United States, and provide informed consent. The study aimed to include participants who were current daily smokers. We chose the daily cigarette cutoff of 5 to be consistent with other studies of comparable apps (eg, Bricker et al [16]). We also included desire to quit smoking as an inclusion criterion based on other cessation app studies’ inclusion criteria (eg, Bricker et al [16]). However, we assumed desire to quit based on participants responding to the digital advertisement, and we asked them about it during the phone screening call although we did not formally measure it.

We recruited potential participants from May to July 2016. Digital advertisements were posted to social media outlets (Facebook, Craigslist, Twitter, Instagram, and Reddit), targeting users who searched for “quit smoking” where possible. Digital advertisements contained IRB-approved copy: “Ready to quit? We’re ready to help. If you have an iPhone and want to become smokefree while earning up to $100, we may have a great solution for you.” The study director or a study team member under his supervision contacted respondents by telephone to prescreen for eligibility and, if appropriate, directed them to a Web portal for the study. After providing online informed consent and confirming eligibility, participants were sent an email with a secure link to download the app. Providing informed consent and downloading the app constituted enrollment in the study; these respondents constituted the intention-to-treat (ITT) sample for the study.

Clickotine adheres to the USCPG essential content features for smoking cessation. The USCPG was developed for in-person clinical settings and, as such, not all parts of the guideline will apply for mobile phone apps [14]. In designing the Clickotine program, we followed the parts of the guideline that are appropriate and amenable for inclusion in a digital therapeutic application: advise and encourage to quit; assess willingness to quit and enhance motivation; assist with quit planning and connect with intervention, including counseling and medications, advice on pharmacotherapy, provision of social support, and connection with a quitline; and arrange or provide follow-up. These USCPG features were developed in consideration of empirical support for their efficacy. For advising and encouraging to quit, evidence exists that even brief advice to quit from a counselor or health care provider significantly increases long-term smoking abstinence rates [24]. Support for assessing willingness to quit and enhancing motivation exists, as evidence suggests that a variety of motivational interventions can increase motivation for behavior change, including smoking cessation [25-27]. The effectiveness of encouragement and support as part of smoking cessation treatment (assisting with quit planning and connecting with appropriate interventions) is consistent with the literature regarding the importance of providing a caring, empathic, and understanding context in making health behavior changes [28-30]. Evidence for the effectiveness of arranging and providing follow-up in smoking cessation treatment also exists—the USCPG recommends that assessments within the first week after quitting should be encouraged, as these can minimize relapse in quitters or encourage abstinence in prequitters [31,32].

Clickotine follows these important USCPG guidelines and personalizes these as much as possible via the Clickometrics platform, which is engineered to enhance engagement. Upon downloading the Clickotine app to their iPhone, users are prompted to create a user profile and answer a brief questionnaire on smoking behavior. The users then self-select a quit date between 7 and 21 days after creating their user profile. Based on the chosen quit date, personal characteristics, and smoking characteristics (eg, name, age, sex, location, quit motives, smoking history, and even the desired number of messages per day as indicated by the user), users receive a tailored plan of missions and messages. Within the context of the USCPG features identified above, Clickotine delivers intervention components that have demonstrated efficacy in promoting smoking cessation: controlled breathing [33], personalized messaging [34,35], social engagement [36], encouragement of pharmacotherapy for cessation and of medication adherence [37,38], and digital diversions, including targeted strategies to cope with cravings, withdrawal symptoms, and lapses (such as reviewing positive moments or previous successes in weathering cravings) [39]. Delivery of these components in adherence to the USCPG results in a set of interactions for the user: controlling breathing, using digital diversions, logging cravings, receiving personalized messages, responding to messages, participating with “quit teams,” completing missions, logging sentiments and feelings, logging cigarettes smoked, journaling, learning about and using quit aids, and interacting with supporters linked through the app. Table 1 demonstrates the relationship between the USCPG features and the set of interactions experienced by the Clickotine user, and Figure 1 provides representative screenshots of Clickotine interactions. The Clickotine interaction categories do not map 1-to-1 onto the USCPG features, and some interaction categories apply to multiple USCPG features or stages of the Clickotine program. For example, some Clickotine missions are designed to advise and encourage quitting (eg, learn about the health effects of smoking), while others are relevant for assessing willingness and enhancing motivation to quit (eg, exploring the user’s quit motives); interacting with supporters is relevant for enhancing motivation to quit in the early stages (eg, connecting with supporters in-app, sharing your desire to quit, and asking for support), as well as being part of cessation intervention in the later stage of the program (eg, updating supporters on progress); and logging cravings or cigarettes smoked can be a mechanism for enhancing motivation to quit and could also be a component of effective intervention. Over the course of a user’s quit journey, in-app interactions are emphasized until the quit date, whereas personalized messaging is designed to become the primary engagement modality following the quit date. In general, in this study, we sent a minimum of 1 personalized message per day to participants as long as they had not deleted the app. If participants responded to a personalized message or completed a mission in response, they were more likely to receive another; thus, participants received different numbers of personalized messages.

We collected baseline demographic and smoking characteristic data via an online survey for participants who met the eligibility criteria and gave informed consent. We also collected data on smoking behavior, as well as engagement metrics including the number of app opens and interactions with the Clickotine program components in-app after downloading. We administered a Web-based outcome survey 8 weeks after enrollment and, in return for completing the 8-week survey, provided participants with a US $25 Amazon gift certificate. Participants were notified to take the online survey via text message with a link to the survey on day 53 (3 days before reaching 8 weeks after initial consent). Each day thereafter until participants completed the survey, up to 7 days after the target date, they were contacted via text message, email, or phone call. A sample text message is: “Good Morning [name]! Your 8-week survey is ready! Fill it out online to get your first Amazon gift card: http://xxxx.” These in-app and online survey data were collected in Amazon Simple Storage Service (Amazon S3; Amazon Web Services, Seattle, WA, USA) with no manual data entry, and access to all data was maximally restrictive, logged, and auditable.

A priori, the study data management plan defined 3 categories of study team members: (1) participant-facing (having some interaction with participants; eg, recruitment or in-app), (2) data team (maintaining the study database), and (3) study leadership (not interacting directly with participants, and not having access to the study database; only conducting analyses on relevant variables provided by the data team after the database lock). This means that access to data was restricted only to the minimum set of actors in the minimal set of use cases needed to complete the study. As per the approved study protocol, no one outside of the designated data team had access to the data and even the data team could not modify it. This procedure follows the principle of least privilege [40]. This was just one of the steps taken to ensure maximal objectivity and transparency in this study conducted by the sponsor. Moreover, the process that wrote the data from users to the database was restricted to write only those files that were relevant to it and had no ability to read data. All app and survey data were stored in an isolated network. All survey data were collected via preprogrammed, automated processes, and all collected data were immediately made immutable and could not be changed by the sponsor. These procedures were approved by an independent IRB and were designed and implemented to minimize potential biases unconsciously introduced by the sponsor, which might have influenced the study data.

We calculated descriptive statistics to estimate engagement with the app and smoking cessation rates. Engagement was quantified in three ways: the number of app opens during the 8-week study, the number of Clickotine program interactions experienced by the user (in total, and for each interaction category), and the number of weeks the user remained actively engaged with Clickotine. We estimated population-level engagement as the mean (SD) values across participants or median (interquartile range) values for instances of nonnormally distributed variables. Smoking abstinence rates were calculated as the proportion of the ITT sample that self-reported not smoking, not even a puff, for at least 7 days and at least 30 days at the 8-week study end point. In this ITT analysis, nonresponders to the 8-week survey were imputed as smokers. This method has been shown to yield potentially biased results, by potentially inflating the proportion of smokers compared with a completer analysis [48]. Despite the potential for bias in smoking cessation trials, this method for ITT analysis is expected for traditional clinical trials and was conducted in this study. However, we also conducted completer analyses on the sample that completed the 8-week outcome survey for comparison with the ITT results.

We conducted post hoc analyses using logistic regression, due to the nonnormal distributions of most of the predictor variables and the dichotomous outcome variables, to explore associations between baseline characteristics and engagement indicators, between baseline characteristics and cessation outcomes, and between engagement indicators and cessation outcomes in this sample. As these were exploratory, hypothesis-generating analyses, we evaluated each independent variable (baseline sex, age, race/ethnicity, nicotine dependence level, years smoked, cigarettes per day, and number of previous quit attempts) as a predictor in a separate model, with app opens, Clickotine interactions, and number of weeks active dichotomized into high/low groups according to median split. We corrected critical P values for multiple comparisons familywise (eg, .05/7=.007 for the analyses of 7 baseline predictors of each engagement indicator). Similar models were run with 7-day and 30-day cessation as the binary outcome to explore associations between baseline characteristics and outcomes, with the same P value correction (.05/7). Logistic regression analyses were also conducted with app opens, Clickotine interaction counts, and active week counts in separate models as engagement predictors of cessation outcomes, with P values corrected for 3 comparisons for each outcome (.05/3=.02). Visual inspection and analyses were conducted using IBM SPSS Statistics for Mac version 24.0 software (IBM Corporation).

After 63 days of social media advertising and recruitment, we received 2050 contacts and conducted 617 telephone prescreens of potentially eligible participants. Many of the 2050 respondents did not schedule or respond to the prescreening call. Screening calls were conducted with participants until approximately 600 were completed, which was the target to yield a targeted ITT sample of >400. This resulted in 452 participants invited to provide online informed consent and who were emailed a secure link to download the app. Of these, 416 participants ultimately downloaded the app and constituted the ITT population. Of the 416 participants in the ITT sample, 365 completed the 8-week outcome questionnaire, yielding an 87.7% retention rate. Figure 2 depicts the study flow diagram for this trial.

Table 2 provides the demographic and smoking characteristics of the study sample. At baseline, this sample reported a high degree of nicotine dependence: the mean score on the Fagerstrӧm Test for Nicotine Dependence was 6.1 (SD 2.2), which falls in the high range for nicotine dependence (scores between 6 and 7) [49]. On average, participants had been smoking for 18.1 (SD 10.6) years, and were smoking 16.7 (SD 7.8) cigarettes per day. At baseline, 29 participants (7.0% of the ITT sample) were using a pharmacotherapy smoking cessation aid (nicotine replacement therapy or medication). At study outcome, 66 participants (15.9% of the ITT sample) were using a pharmacotherapy smoking cessation aid (nicotine replacement therapy or medication).

The distributions were significantly nonnormal for app opens (W₄₁₆=.773, P<.001), Clickotine program interactions (W₄₁₆=.809, P<.001), and the number of active weeks with Clickotine (W₄₁₆=.892, P<.001) according to Shapiro-Wilk tests of departure from normality. Table 3 provides the measures of central tendency of these data: mean (SD), range, median, and interquartile range. Table 4 describes the frequency with which users encountered each interaction category, as well as the proportion of program interactions each category represented.

Table 5 summarizes the results of the ITT and completer analyses of cessation outcomes. In the ITT sample (n=416), at the end of the 8-week study period, 45.2% (n=188) of participants reported achieving 7-day abstinence and 26.2% (n=109) of participants achieved 30-day abstinence. Among the 365 study completers, 51.5% (n=188) of participants reported achieving 7-day abstinence and 29.9% (n=109) of participants achieved 30-day abstinence. In this study, participants did not appear to reliably provide the in-app weekly smoking data. Only 47 of the 416 participants (11.3%) provided complete data in the weekly cigarette counts; across all users, only 822 of the 3328 (24.7%) weekly cigarette counts were completed. Thus, we derived smoking status from self-report in the baseline and outcome surveys, and we did not use the in-app smoking data in the analyses of cessation outcomes.

Participants reported 4 negative health events spontaneously, and reported an additional 32 in response to the focused safety question in the 8-week questionnaire. Of these 36 negative health events, 19 were considered clinically significant and documented as adverse events. Of the 19 adverse events, 2 were considered possibly related to Clickotine use in the judgment of the medical monitor (nightmare, depressed mood). The most common adverse events were fatigue (reported by 3 participants) and mood change (reported by 2 participants); no additional adverse events occurred in more than 1 participant.

Table 6 provides the results of the logistic regression analyses of baseline characteristics predicting engagement indicators. The only association surviving correction for multiple comparisons was female sex predicting an increased likelihood of being in the high app opens group.

Table 7 provides the results of the logistic regression analyses of baseline characteristics predicting cessation outcomes. Older age and greater number of years smoking appeared to predict a decreased likelihood of reporting 7-day abstinence and 30-day abstinence. Greater number of cigarettes smoked per day predicted a decreased likelihood of reporting 30-day abstinence.

Table 8 provides the results of the logistic regression analyses of engagement indicators predicting cessation outcomes. Greater number of weeks active with Clickotine was associated with an increased likelihood of reporting 7-day and 30-day abstinence.