A total of 101 relevant iPhone apps were identified by searching for keywords “quit smoking,” “stop smoking,” and “smoking cessation” in Mandarin Chinese in the iPhone app store in February 2016. A total of 74 apps—53 irrelevant apps (eg, tobacco knowledge apps, male health apps and medical health apps), 5 duplicate apps, 2 apps which could not be opened, and 14 English language apps—were excluded by checking the content after download. The remaining 27 iPhone smoking cessation apps, including 26 free and 1 paid app, were evaluated for the study.

A total of 145 Android apps were identified from the 360 Assistant app store (86 apps) and Baidu Assistant app store (59 apps), which account for 87.4% of all Android app downloads in China, by searching for the keywords “quit smoking,” “stop smoking,” and “smoking cessation” in Mandarin Chinese in February 2016. A total of 108 apps—21 irrelevant apps (eg, tobacco advertisement apps, expertise in the tobacco industry apps), 15 duplicate apps, 7 prior version apps, and 65 English language apps—were excluded by checking the content after download. The remaining 37 free Android smoking cessation apps were evaluated for this study. In total, the analytical sample included 64 apps (Figure 1).

For each app, descriptive information was retrieved regarding its characteristics (ie, release date, size, type), popularity (frequency of downloads), and user rating (ie, app was rated ranging from 1 to 5 stars) (Table 1). Apps were categorized into different types: (1) calculators, that calculate number of days, amount of money saved, health improvements since quitting; (2) calendars, that track number of cigarettes per day, smoking pattern, and mood; (3) hypnosis, that use hypnosis techniques for smoking cessation; (4) rationing, that ration the number of cigarettes for a specific time period; and (5) other, for apps that did not fall into the previous categories or fell into multiple categories.

Apps were also coded for their level of adherence to the CCSCG [14]. To measure adherence to the CCSCG, an index of 23 items was developed that closely followed the guidelines. Each app was independently coded by two reviewers on each of the 23 items in the adherence index using a score ranging from 0 to 2. A score of 2 indicated that the feature was fully present, a score of 1 indicated that the feature was partially present, and a score of 0 indicated that it was not present at all. For example, for the guideline to recommend the use of medicine, apps that did not mention medicine received a score of 0, apps that partially mentioned medicine received a score of 1, and a clear and strong mention (fully presented the specific drug, proper use, dosing, and common side effects) for medicine received a score of 2.

A total of 87% (20/23) of items were ratings of concrete features, which by their very nature did not lend themselves to rater judgment and interpretation. For example, the Ask criterion measures whether or not the app asks the user if he or she smokes, and if so, how many cigarettes per day. The concrete quality of these ratings was reflected in the fact that there was 100% interrater agreement for these 20 items.

In 13% (3/23) of items, the guidelines were subjective (eg, presence of practical advice for quitting smoking). For these 3 items, there was a small fraction of interrater disagreement: 23% of the time for the iPhone apps and 15% for the Android apps. In these cases, we minimized subjectivity of rating with two coders discussing and resolved disagreement when the score differed by 1 (19% for iPhone and 8% for Android) and averaged the scores. When the score differed by 2 (5% for iPhone and 1% for Android), a gold standard outside coder resolved score discrepancies 100% of the time. The maximum possible score on the index was 46 for each app.

The statistical analysis was performed using SPSS version 20.0 software (IBM Corp). Total n, percentages, mean, and standard deviation were calculated to describe the features of apps. Weighted Cohen’s kappa and intraclass correlation coefficient were used to evaluate interrater agreement. In addition, to identify features of the apps which were associated with the popularity and user-rated quality, linear regression was used to test predictors of popularity (frequency of download) and user-rated quality (user rating with number of stars). Analyses regarding the user-rated quality of the apps were restricted to the apps that had star ratings (73% of the apps), and analyses regarding the popularity of apps were limited to the Android apps because iOS does not release data on the frequency of downloads for iPhone apps. The sample of 64 had 99% power to observe an effect size of 36.8% for the analysis of predictors of quality and downloads.

The characteristics of the 64 smoking cessation apps included in the analysis are presented in Table 1. Smartphone apps for smoking cessation started to appear in China in 2009. The largest number of app releases (12, 19%) was in the year 2010. The average size for all apps in the sample was 11.05 MB, and the frequency of download for 83% of apps was less than 10,000 times. The modal user rating for most apps (44%) was 3 or 4 stars (out of 5). Most apps used a calculator approach (53%), followed by calendar (42%), other (33%), rationing (13%), and hypnosis (2%). Most apps (91%) were designed and developed by individuals or technology companies rather than government agencies.

With a maximum score on the index of 46, the average score was 14.6 for iPhone apps and 11.1 for Android apps (Multimedia Appendix 1). For iPhone, the Quit Smoking at Once app had the highest score of 38, while apps 30 Day Messages for Quitting and Common Medical Knowledge received the lowest score of 2.5. For Android, the Quit Smoking App had the highest score of 25, while the apps 30 Day Quit Message, Quitting App, and Self-Awareness of Smoking Habits had the lowest score of 3.

There was no significant association between popularity, user rating, and the characteristics of apps (P>.05). However, there was a positive relationship between popularity, user rating, and adherence score as seen in Table 2.

This was the first study to analyze the content of all publicly available smoking cessation apps in China. The results showed that Chinese apps generally lack the content that is consistent with the CCSCG [14]. The number of existing smoking cessation apps in China is smaller than that of America because of the poorer development of eHealth interventions in China [33]. However, the adherence score of Chinese apps to the CCSCG is consistent with analyses of English apps (average adherence scores for iPhone and Android are 14.2 and 11.7, respectively, on 0 to 42 score) [33,34]. One reason may be that the developers of Chinese and English apps are not aware of or do not understand the importance of clinical practice guidelines, as most designers of smoking cessation apps are individuals or companies without any background in clinical practice for tobacco cessation. Indeed, the adherence of smoking cessation apps designed by government agencies was higher than that of other designers in the study.

The results showed that the content of Chinese apps is primarily limited to calculators (eg, money saved by not smoking) and calendars (eg, showing smoking patterns over time). While such tracking-derived information is potentially useful, it vastly underutilizes the potential of smartphone apps to provide evidence-based advice, motivation, and training in skills to cope with cravings and recover from relapses. In addition, the sensors (eg, Global Positioning System) already built into most smartphones make it possible for smartphone apps to provide sophisticated just-in-time and in-the-moment intervention for smoking cessation. In sum, the content of current Chinese apps was generally simplistic, and the potential for improvement is enormous.

Compared to the large population of smokers and smartphone owners, Chinese language smoking cessation apps had relatively few downloads. Indeed, only 6 Android apps in Chinese had more than 10,000 total downloads. However, 35.2% of US smoking cessation apps had more than 10,000 downloads with 7.8% having more than 100,000 downloads [33]. This disparity may be due to Chinese smokers’ lack of awareness of smoking cessation apps and smoking cessation advice in general. In traditional Chinese culture, smoking functions as a way to make social connections, smooth social interactions, and express one’s social and economic position. Furthermore, in the comments on smoking cessation apps, many users said that stopping smoking was very easy and they will be lucky to avoid the health consequences of smoking, which implies they do not want or understand the value of smoking cessation assistance [35]. Once they download an app, users appear to respond positively to it if it follows the CCSCG, as the adherence score was positively related to app popularity and user-used quality. However, more research is needed to understand what Chinese smokers know about smoking cessation advice, their interest in receiving it, and methods to overcome barriers they may have to using it. When smoking cessation apps that follow the CCSCG are developed that are appropriate and motivating to Chinese smokers, it would be highly valuable to test their efficacy. Indeed, the need for efficacy data is underscored by the fact that to date only randomized trials of smoking cessation apps developed in English have been published [36]. When Chinese apps show promise in randomized trials, public health media campaigns to widely publicize their existence and potential value for smokers would serve the role of promoting the adoption of proven interventions on a broad scale.

An important future research question is how to effectively reach a general population of smokers. A recent US survey of smokers showed that 91% had experience using smartphone apps [37], which suggests that a smoking cessation app in the United States could potentially reach users with demographics that are generally representative of the overall population of US smokers. Research is now needed to determine what fraction of the Chinese smokers have experience using apps and would be interested in using one to quit smoking, with the aim of determining demographic characteristics of the target population for quit smoking apps in China. As the purchase of iPhone and Android smartphones is rising among old and young people with lower socioeconomic status in China [38], the potential utility of smoking cessation apps for Chinese smokers will grow.

The results of the current studies in the United States and China indicate that despite the recent expansion of smartphone platforms and increased availability of apps for cessation, existing apps still lack many elements that are generally recommended for quitting smoking. We recommend that future research integrate the content of the clinical practice guidelines that have been proven effective in randomized clinical trails [39-41]. Once guidelines are integrated into to the content of Chinese language smartphone apps, these apps can be tested in randomized trials of Chinese smokers.

The proliferation of smartphones can now make them valuable health promotion tools. This study presents an evidence-based evaluation of a tool (smoking cessation apps) for health promotion in China that could inform health policy makers of an eHealth intervention with promise for widespread public health impact. As these apps proliferate in China, there is a great potential that data scientists can analyze large databases of users to understand patterns of use to inform personal tailoring of app content to specific users.

This study has limitations. First, like other content analysis, the data were rated by trained raters. Our method for minimizing rater bias was having 87% of all items be of concrete criterion with the remaining 13% having a small fraction of disagreement that was resolved 100% of the time via a rigorous process described in the methods. Second, the coding measured the presence or absence of the clinical guideline, not the degree to which that guideline was followed or presented effectively. Finally, not all app users provided quality ratings. In the study, an app’s number of stars was used as an indicator of its user-perceived quality, so the generalizability of those results is unknown.

Chinese apps for smoking cessation lack content that is known to be effective for smoking cessation. Including content based on China’s clinical practice guidelines would be extremely valuable for improving existing apps and informing the development of new apps and thereby increasing the chances they will make a significant impact on the smoking epidemic in China.