The apps analyzed in this study were searched for and downloaded in France using both the iOS and Android app stores. The names of all apps and their descriptions were initially screened by the first author (LAB). Most of the apps were downloaded for a second screening, and only those that met all criteria were recorded on video as if the user had downloaded them for the first time. Based on these videos, two independent raters assessed the presence of BCTs and mobile app quality. The raters were all trained in health behavior change, and they are both behavioral and cognitive psychologists and researchers. Since subjects were not recruited, no ethics approval was required. The recorded videos were necessary to ensure that both raters assessed the same versions of the apps and to facilitate discussion.

The study included both free and paid apps that support the French language and that intend to assist users with smoking cessation. The apps were identified on two occasions by the first author (LAB) (December 1, 2019, and April 20, 2020), using the app search keywords “smoking cessation,” “stop smoking,” and “quitting tobacco” (in French, “sevrage tabagique,” “arrêter de fumer,” and “arrêt tabac,” respectively). To be included in the full review, apps had to be designed to target smoking cessation only and support the French language. Excluded apps were those that were last updated before January 2019 and had ratings of less than 3/5 points in the app stores. Since each store provides different information on the number of downloads, we used different characteristics to select the most popular apps for a detailed content analysis. For android apps, where the number of downloads is available, an arbitrary threshold of 50,000 downloads was required to meet our popularity criteria, and since Apple Store does not share the number of app downloads, for iOS apps, an arbitrary threshold of 50 raters per app was set. Smokers wanting to quit with an app seem to favor apps with a strong “social proof,” meaning they care about the rating as well as the number of raters and number of downloads [24].

After the first author used and recorded each app, both raters assessed the apps’ content independently using both the MARS and behavior change technique taxonomy for smoking cessation. Prior to evaluation, all raters read each type of measure and had the opportunity to clarify and discuss the definitions in order to ensure clear differentiation between items.

The statistical analysis was performed using IBM SPSS statistics version 26.0 (IBM Corp). Following the suggestion of “issues and best practices in content analysis” [28], we decided to calculate three measures of reliability for each scale. To assess the level of agreement between raters (IRR), we used the ICC and Krippendorff alpha for both scales, weighted kappa for the MARS, and prevalence and adjusted kappa for the behavior change technique taxonomy. The ICC was assessed in a two-way random model for an agreement level. The weighted kappa was assessed by pulling quadratic weights for each value. All reliability tests were performed per dimension of each scale and for all apps. Descriptive analysis was used to identify the presence of app characteristics (ie, mean price and frequency of BCTs), and one-way analysis of variance (ANOVA) and the post-hoc Tukey honestly significant difference (HSD) test were used to determine any significant difference observed between the scales. The mean score by dimension of each scale was used in the Spearman correlation test to examine the relationship among the price per month, user ratings, and both mobile app qualities (general and therapeutic).

A total of 688 apps were initially identified from the French Google Play Store (n=603) and Apple Store (n=85). Figure 1 shows the results of the key stages of the mobile app review. After preliminary inclusion and exclusion criteria were applied, 74 Android and 33 iOS apps remained. Further screening based on evaluation of the app product page and the last update resulted in 61 Android and 14 iOS apps. Finally, further exclusion of apps upon download and during the coding procedure resulted in a total of 14 apps. Among these 14 apps, seven were accessible in both stores, six were accessible in the Google Play Store only, and one was accessible in the Apple Store only (Multimedia Appendix 1).

Table 1 shows that most of the included apps (12/14, 86%) were affiliated with a commercial company, while 14% (n=2) were affiliated with a university (n=1) and a government department (n=1). The mean user rating was very good (mean 4.5, range 3.7-4.8) in both stores without a particular difference. All apps were free to download and to use for a limited time or with limited features, but half of them had in-app purchases with different payment options that offered full access to all the contents of the app. The mean monthly price was €3.47 (US$ 4.19) in both stores, without a difference between both stores. Most apps included calculator (14/14, 100%), tracker (13/14, 93%), and information (13/14, 93%) features, as well as media connectors and reminders to use the app or to stay a nonsmoker. Very few apps (4/14, 29%) allowed users to protect their information with a password. All the apps needed internet access to download, and later, most of the features could be used offline.

The general quality was acceptable. The mean MARS score was 3.5 (median 3.6, IQR 0.6 [3.2-3.8]), with a maximum score of 4.3 and a minimum score of 2.4 (Table 2). Since only one app was not available in both the Google Play Store and Apple Store, no comparison between these distribution services was conducted. The IRR between the two raters was substantial. The ICC was 0.79 (95% CI 0.74-0.84), weighted kappa was 0.79 (95% CI 0.74-0.84), and Krippendorf alpha was .88 (95% CI .85-.91). Detailed results are presented in Multimedia Appendix 2.

The mean scores of the dimensions of the MARS are presented in Table 2. The values vary from “low” to “good.” The results of the one-way ANOVA reveal that there was at least one significant difference between the five dimensions regarding the score (F_5,78=13.51, P<.001). In order to determine which mean values differed more specifically from each other, the Tukey test (HSD) was performed. The results showed that the functionality dimension value was significantly higher than the values of the other dimensions (P=.009). The results also showed that the subjective dimension value was significantly lower than the values of all other dimensions (P=.001). However, the difference in the mean values of the engagement, information, esthetic, and total dimensions was not significant (P=.09). The engagement dimension had an average score but had the most variability (median 3.1, IQR 1.3). Figure 2 shows box plots with the median, first and third quartiles, and minimum and maximum scores. Each point represents the score for each app.

The mean number of BCTs identified in the apps was 22 (SD 10), with a maximum of 38 techniques and a minimum of 4 techniques. The IRR between the two raters was almost perfect. The ICC was 0.92 (95% CI 0.89-0.93), prevalence and adjusted kappa was 0.85 (95% CI 0.84-0.85), and Krippendorf alpha was .85 (95% CI .79-.9). Detailed results are presented in Multimedia Appendix 3.

The significance in the Shapiro-Wilk test (P=.11) indicated the inability to assess the strengths of the different prevalences of BCTs observed in each app. Yet, delivery of the intervention (D), addressing motivation to stay a nonsmoker (M), specific behavior change techniques focused on smoking behavior (B), and maximizing self-regulatory capacity and skills (S) were observed most frequently (between 62% [8.7/14] and 64% [9/14] of the apps), whereas proposing adjuvant strategies (A) was least frequently observed (31% [4.3/14] of the apps) (Table 3) [27].

Mean price per month, mean user rating, and number of BCTs were tested for correlation with the MARS score (Table 4). The mean MARS score (mean 3.5, SD 0.6) was positively correlated with price (mean 3.0, SD 3.4) (r=0.70, P=.007) and the number of implemented BCTs (mean 22, SD 11) (r=0.66, P=.01). User rating was not correlated with any quality scale (Table 4). The mean price per month (mean 3.0, SD 3.4) was positively correlated with the mean user rating (mean 4.5, SD 0.3) (r=0.58, P=.03) and the number of BCTs in the app (mean 22, SD 11) (r=0.60, P=.03).

The current review aimed to examine the content quality of popular mobile apps for smoking cessation in the French market. This type of content analysis is, to our knowledge, the first of its kind for the following two main reasons: the target market of the study and the methodology. French mHealth apps were reviewed in this study, with the aim to examine both of the following aspects of content quality: the general quality via the MARS scale and the therapeutic dimension though the behavior change technique taxonomy.

Based on the established behavior change technique taxonomy and the MARS, we analyzed a total of 14 apps. It appears that the French mobile app market is less developed than the English one, where the number of reviewed apps is much higher. Indeed, 252 apps were identified in the US market in 2013 [11] and 225 apps were identified in 2016 [14]. Similar to these findings, 112 apps were examined in Australia [15] and 140 apps in England [16]. Consequently, even if several apps are available on the market, they are not accessible to most of the native French-speaking population speaking only French. This accessibility limitation could be overcome by translating existing apps on the market. However, the translation process would need to take into account the cultural context and be periodically adapted to each update to ensure users can access and benefit from the app content. Since this process would require human time and financial resources, the translation of apps could be decided based on supply and demand.

The general quality of popular apps in France varies from “acceptable” to “good.” In spite of these results, health professional judges do not recommend most of the apps. It seems the general quality threshold needed to be recommended is not met by most of the popular apps on the French market.

Our findings suggest that popular apps focused primarily on the functionality dimension that is composed of the following four aspects: performance, ease of use, navigation, and gestural design. The priority on the functionality dimension is a trend already observed for smoking cessation [29] and weight management [21] apps. Enhancing only this dimension will not be enough to improve the general quality of content. Nevertheless, we recognize the importance of the functionality dimension as a facilitator for the use of mHealth solutions. Better integration of clinical expertise seems to be necessary to create engaging and informative content.

The use of BCTs in our mobile app sample was not normally distributed, indicating that evidence-based techniques are not properly implemented in the French market. Our findings show that the market is still in the initial stages, most likely driven by technical expertise, and suggest that there is a lack of theory-driven behavioral change techniques, as proposed previously [16,29]. In France, mobile app development is not driven by BCT theory because its techniques are rarely invoked in popular apps. Indeed, even the most widely used BCT functions were absent in more than 65% of the sample.

The strength of current apps is their focus on the target behavior through addressing motivation and maximizing self-regulatory capacity and skills. In contrast, our results suggest that most apps failed to include features from the adjuvant activities dimension and focus on the general aspect of interaction. Failure to promote activities that indirectly facilitate abstinence (ie, inform or advise on medication to stop smoking) is consistent with the tendency in English-speaking countries [10,30]. A lack of adjuvant activities results in harsh deprivation considering that adherence to medications that help to stop smoking increases the likelihood of successful abstinence by 50% compared to “cold turkey” [2]. Additionally, our findings indicate a deficit of techniques necessary for effective delivery (ie, acquire and communicate relevant information needed to adapt the intervention). In addition, some authors pointed out the importance of technique interactions for an effective behavior change, promoting the idea that some BCTs can be effective solely under specific combinations (used at their best under specific conditions). As reported, the least effective interventions were those providing feedback on performance without providing instructions [31]. We therefore believe that the current strength of the French mHealth market can be severely undermined owing to the limited information and communication techniques used.

A noteworthy result was the positive relationship between both general and therapeutic qualities. This association is relevant since efficacy is influenced by not only the content of a therapy (in this case the therapeutic quality), but also adherence to treatment [18].

The second interesting result was the absence of correlation between the app store’s user rating and the qualities assessed, suggesting that user ratings may not be a good predictor of general or therapeutic quality. These results may give rise to concerns as users choose their apps based primarily on the ratings on the app store [32]. This highlights the importance of creating standards and accreditation for mHealth apps to protect users.

The third interesting result was the relation between price and both quality measurements (the cheaper the app, the poorer the qualities, with the exception of two apps financed by public institutions). This result is at odds with the supposed accessibility benefits of mobile apps [6], which casts a doubt on the benefits of affordability. To better understand this relationship, more studies that address the issue of intervention efficacy should address the issue of the cost-benefit ratio.

An important strength of this study is that it is the first to examine smoking cessation apps on both the Apple Store and Google Play Store in the French market. The IRR was evaluated following the best practice recommendations for content analysis [28], and in all the tests, the IRR varied from substantial to almost perfect for each scale.

The findings of this study should be interpreted in light of some limitations. First, the apps were rated from the first and only use. There is a probability that some BCTs were not accessible to the raters and therefore were underrated. On the contrary, after prolonged use, apps could be seen as less engaging than in the first use, as the engagement attrition rate of mHealth apps is very high [33,34]. Second, we are aware of critics questioning the link between the quality and popularity of smoking cessation apps (whether the quality is therapeutic or general). Indeed, the few apps that exhibited high adherence to therapeutic guidelines were not necessarily the most popular [16], and about 17% of the high general quality apps identified appeared in the top 10 recommended smoking cessation apps in New Zealand app stores [15]. The aim of our study was not to identify the best solution in the French market, but to establish the quality of the most used apps available for French users.

The general and therapeutic content quality of popular smoking cessation apps in France varied by app type and price. General and therapeutic contents are positively correlated. The user rating on app stores is not an indicator of the general and therapeutic content quality. The findings suggest that popular apps focused primarily on the functionality dimension. At least half of the apps addressed motivation and advised on using behavioral skills to quit smoking or stay a nonsmoker; however, only a handful of apps gathered important information and delivered proper advice regarding the use of approved medication or the implementation of behavioral techniques. Overall, the findings provide the first snapshot of the quality of popular smoking cessation apps in France. This review will need to be revised in order to examine whether the content quality of smoking cessation apps will evolve in the French market. Further research is needed to understand how users engage and benefit from these apps in the real world.