A systematic review of smartphone apps with an analysis of content quality was carried out by 4 researchers. One general practitioner resident (CB) and 1 experienced general practitioner (EM) carried out the search for apps on search engines and the initial selection. Two general practitioner residents (MK and LP) and 1 experienced general practitioner (EM) evaluated the apps using the French version of the Mobile App Rating Scale (MARS-F) [23]. The study protocol was designed following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (items not relevant to a systematic search of apps were considered not applicable; Checklist 1) and the PRISMA-S (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Literature Search Extension) guidelines (Checklist 2) [24,25]. We took into account the specialized reporting items suggested by the PASSR (Protocol for App Store Systematic Reviews) guidelines (Checklist 3) to address the unique challenges of searching and analyzing mobile app stores [26]. The complete and detailed study protocol is provided in Multimedia Appendix 1.

A systematic search was performed on the 2 major digital platforms: the Apple App Store and the Google Play Store. These platforms are used by approximately 99% of smartphone users to download their apps [27]. Two similar searches, using the same method, were conducted between August 2022 and March 2023, and then between March 2023 and May 2023. The platforms were queried in French and English using keywords related to adolescent prevention areas derived from MeSH terms [28]. MeSH terms were preidentified in a literature review according to prevention themes (Table 1). MeSH terms were replaced with non-MeSH synonyms when a search did not find any relevant results. The choice of words was discussed between the 2 researchers, CB and EM. Each keyword was used individually on the app platforms. CB and EM carried out the search on an iPhone 11 smartphone running iOS 15 (Apple), an ASUS VivoBook computer (ASUSTeK Computer Inc), and a Google Pixel 7 smartphone with the S91xBXXS1AWD1 update. The search ended when the search engines no longer provided new apps: keywords were entered into the search tool, followed by an exhaustive review of the descriptions of the apps offered.

The selection by EM and CB followed a 2-stage process. First, EM and CB screened the apps based on their titles, descriptions, and metadata provided in the app stores. Second, the shortlisted apps were downloaded and tested for full eligibility. The initial selection of apps was based on inclusion and exclusion criteria from digital platforms. In cases of doubt about an exclusion criterion, the decision was made through discussions between CB and EM.

To meet our inclusion criteria, the apps had to (1) relate to one of the selected areas of prevention (orthopedics, mental health, substance abuse, risk behaviors, sexual health, vaccinations, social relations, nutrition); (2) be relevant to facilitating the preventive approach of health care professionals during consultations with adolescents; (3) be available in French or English; (4) have been scientifically validated (developed by health care professionals or a learned medical society and/or based on scientific or government recommendations); (5) be free (apps free for only a trial period were not selected); (6) have been updated within the 3 years preceding the end of the search, at least by 2020 (since 2022, the Google Play Store has removed apps that have not been updated for more than 2 years [29], and for the App Store, this period is 3 years for apps with few downloads in the last 12 months [30] but not all catalogs have yet been updated).

The exclusion criteria were as follows: (1) lack of information about the scientific data used to design the app’s content, (2) games without medicoeducational content, (3) wallpaper apps, (4) religious apps, (5) apps that offer links to medical appointments, and (6) advertising apps.

Inclusion criteria were based on the data available on each app’s presentation page: the title, description, images, and general information about the app. If any data were missing, the apps were opened to search for the missing information. Apps were excluded after download if they were no longer available in app stores at the time of evaluation by one of the evaluators. We chose to focus on free apps to ensure that the tools identified are accessible to all health care professionals, regardless of institutional funding or individual socioeconomic status, thereby promoting digital health equity regardless of which health care professional a patient consults [31]. Health care professionals have already invested in costly practice tools such as business software and telecare tools. We therefore wanted to offer them apps with no impact on their professional workloads. The high cost of an app is one of the main reasons why it is not downloaded [32,33]. Selecting free apps also limits the risk of commercial bias or conflicts of interest that can be associated with paid models.

As these digital tools are intended for use in clinical consultations, scientific rigor is nonnegotiable. Using unvalidated tools poses an ethical risk to the patient-provider relationship and the safety of care.

MK, LP, and EM evaluated apps using the MARS-F [23]. The MARS and its French version (MARS-F) have been validated for the overall quality assessment of apps in the field of mHealth [23,34,35]. The app evaluations took place between December 2023 and January 2024.

The MARS-F assesses four objective quality dimensions: (1) Engagement (entertainment, interest, customization, interactivity, and target group), (2) Functionality (performance, ease of use, navigation, and gestural design), (3) Esthetics (layout, graphics, and visual appeal), and (4) Information (description accuracy, goals, information quality or quantity, visual information, credibility, and evidence base). A subjective quality score and an app-specific section were also proposed. Each item is rated on a Likert scale from 1 (“inadequate”) to 5 (“excellent”).

When specific objectives were not delineated and information was lacking, the respective items were rated as “not attributable” (NA). Consequently, these items were excluded from consideration in the overall scoring process. In total, MARS-F is a total score out of 5, corresponding to the average of these 4 sections. For each evaluator, an average for each section (A, B, C, and D) from 1 to 5 was calculated, followed by an interevaluator average for each section. An overall average of the interrater MARS-F score was used to evaluate the apps. To ensure the selected apps were suitable for clinical practice and met the needs of health care professionals working with adolescents, specific quality cut-off scores were applied during the screening phase. Apps were excluded if they failed to meet the following minimum thresholds across the MARS-F sections (Multimedia Appendix 1). Apps were excluded if MARS-F ratings were below 3 for target group relevance (Item 5, Section A), below 2 for all technical usability items (Section B: Functionality), or if ratings were insufficient within the Information Quality domain (Section D: Item 15 <2 or NA, Item 16 <2, Item 17 <4). During the quality appraisal, if an exclusion criterion was found by only 1 of the 2 evaluators for the final evaluation score, the third researcher (BL) was asked to discuss the results, and consensus was sought.

From the apps selected on digital platforms by EM and CB, 3 researchers took part in evaluating them: the 2 general practitioner residents (MK and LP) divided up the prevention themes for analysis, and the experienced general practitioner (EM) analyzed all the apps in a blinded manner. All 3 completed the MARS-F [23] separately. A common table was then filled based on these MARS-F results. To ensure rating consistency, LP and MK underwent a structured calibration process led by the veteran expert (EM). During the 2-session calibration process, informal reliability checks were conducted to ensure rater alignment. The 3 researchers independently scored a pilot sample of test applications. Discrepancies and borderline cases were thoroughly discussed until a 100% consensus was reached on all criteria. However, no formal statistical reliability metrics, such as a training intraclass correlation coefficient (ICC), were computed during this preliminary phase.” The training consisted of two 2-hour sessions. The first session involved a comprehensive review of each MARS-F item and its scoring criteria. In the second session, conducted 3 weeks later, the 3 researchers independently and blindly evaluated 2 apps randomly selected from the initial search results. Discrepancies were discussed collectively, and scoring rules were clarified under the supervision of the researcher EM. Throughout the formal evaluation period, raters had access to expert guidance to resolve technical queries. Specific apps were not discussed until the final consensus meeting to maintain blinded assessment integrity. While a formal training ICC was not calculated, this iterative calibration was designed to minimize subjective bias.

MARS-F scores were presented as means and SDs. To assess the reliability of the MARS-F ratings (global and subscales), we used a multimetric approach. Interrater reliability was calculated using the ICC, model (2,1) (2-way random effects, absolute agreement, single measures), along with the SD. To address measurement precision, we calculated the standard error of measurement (SEM) using the formula SEM=SD×√(1–ICC). Descriptive statistics, including the mean and SD, were generated for each app and for each global scale or subscale to characterize the distribution of quality ratings. We analyzed the expert’s ratings (EM) against the residents’ ratings (MK or LP). Since the ICC is a ratio of intersubject variance to total variance, the lack of variability between subjects in a small sample could mathematically deflate the ICC, even when raters are in close agreement. To provide a more accurate assessment of rater reliability, we also calculated the mean absolute error (MAE). The MAE represents the actual average point difference between raters on the 5-point scale.

An ICC above 0.75 was considered excellent, good between 0.60 and 0.74, fair between 0.40 and 0.59, and poor if below 0.40. We have applied an interpretation threshold for MAE: excellent (<0.5), good (0.5‐0.75), fair (0.75‐1.0), and poor (>1.0).

ICC analyses were performed using JASP (version 0.96; 2026; University of Amsterdam), and MAE, SD, mean, and SEM were calculated in Microsoft Excel.

A request for authorization has been made to the Data Protection Officer of the University of Montpellier. In addition, the study did not require authorization from the “Comité de la Protection des Personnes” (Personal Protection Committee), as it did not involve experimentation on individuals and did not fall within the scope of the General Data Protection Regulation and, therefore, of MR004. This study complies with the rules of the Helsinki Declaration [36].

Using the preidentified keywords, the double screening identified 976 apps. Forty-seven apps had already disappeared from the platforms between the 2 screenings and at the time of the first analysis by CB and EM; 2 more disappeared during the final evaluations by LP, MK, and EM, giving a total of 52 apps that had disappeared during the search. In the end, 9 apps were retained. After evaluation with the MARS-F, 7 were included (Figure 1). The mean MARS-F interrater score ranged from 2.5/5 to 3.8/5 (Table 2).

The evaluation scores for the 9 evaluated apps were listed in a table (Multimedia Appendix 2), and a summary of the content of the apps was produced for the apps not excluded from the analysis (Multimedia Appendix 3). We provide a summary of the MARS-F scores for the 7 included apps (Table 2) and a short summary of these apps.

Some items in MARS-F have been rated NA. We describe the NA items present in the apps evaluated by MARS-F in Table 3.

The global MARS-F score demonstrated a pooled SD of 0.60 and an ICC (2,1) of 0.0003, resulting in a calculated SEM of 0.60. The average discrepancy between raters remained low, with an MAE of 0.53. Across subscales, the SEM ranged from 0.41 (Engagement) to 0.79 (Functionality) (Table 4).

The interrater reliability analysis comparing the veteran expert (EM) and general practitioner residents (MK and LP) yielded variable results across the MARS-F subscales (Table 4). As expected, the ICC (2,1) for the overall MARS-F score was poor at 0.0003 (95% CI −0.51 to 0.61). While ICC values for several subscales remained poor, ranging from 0.000 to 0.540, these findings are largely attributable to range restriction caused by the high homogeneity of app quality scores in our small sample (n=9). To account for this statistical artifact, we calculated an MAE to assess absolute agreement. The MAE for the global score was 0.53 (good), indicating that ratings differed by approximately half a point on the 5-point scale, which represents good clinical agreement.

Of the 976 apps identified, 37.9% (ie, 370) and 21.6% (ie, 211) were found using mood-related keywords and toxic consumption-related keywords, respectively, compared with only 13.6% (ie, 133) and 3.9% (ie, 38) for orthopedics and vaccination. Nine apps were subjected to double-blind analysis using the MARS-F, and 7 were included. The “Bariatrique Québec” app was excluded from Section D (Information); the “USPSTF Prevention TaskForce” app was excluded from 3 sections: A (Engagement), B (Functionality), and D (Information). Among the 7 apps selected from the 976 apps identified (0.72%), 71.4% (ie, 5) were related to sexual health, all of these sexual health apps (100%) dealt with contraception, and 60% of these apps (ie, 3) dealt very briefly with sexually transmitted infections (1 dealt with antiretrovirals and 2 with condom use). About 28.6% of the selected apps (ie, 2) were relevant to mental health. No apps were selected for the following themes: vaccination, nutrition, orthopedics, puberty, social or toxic relationships, and addictions.

We summarized the 7 apps selected for use by health care professionals to support prevention initiatives during consultation with adolescents as follows:

This digital review of apps available on the Android and iOS platforms identified 976 apps addressing areas of prevention relevant to consultations between health care professionals and adolescents: vaccinations, orthopedics, drug use and addiction, mental health, nutrition, puberty, sexual health, social relations, and violence. Of these 976 apps, 7 (0.72%) were retained after analysis as recommendable to health care professionals for use in consultations to help them implement prevention measures with adolescents: 2 on mental health (including 1 sleep) and 5 on sexual health (including 3 contraception only). No apps specifically dedicated to adolescent prevention in any of the recommended areas were identified.

In the MARS-F, NA ratings are described in great detail for each item within the subscales [23]. Since NA ratings are not included in the final scores, these ratings could artificially inflate the scores and lead to the inclusion of apps of insufficient quality. In our study, NA ratings were found only in subsection D (Information), items 14, 17, and 19, which could increase this risk. However, the researchers were rigorous in their analyses, and this did not affect the selection of apps. We have analyzed these NA ratings found in the following:

The integration of Clinical Decision Support Systems, such as the WHO Contraception Tool or mhGAP-IG 2.0 App, into practice carries significant medicolegal and ethical implications. These tools are designed to support, not replace, professional expertise. The health care professional remains solely liable for the final clinical decision. Most apps include disclaimers stating that recommendations are based on global guidelines and may not account for individual patient complexities. A critical risk is “automation bias,” in which a professional might overlook rare comorbidities or contraindications that are not programmed into the app’s algorithms.

A scoping review published in 2023 examined studies of apps used by health care professionals in clinical practice [37]. This scoping review aimed to describe how physicians use smartphones and mobile apps in clinical settings. Ten studies were included, mainly using survey or focus group methods. Smartphones and mobile apps were used for communication, medical education and training, clinical decision-making, and accessing drug compendia in most studies. Health care professionals were concerned about patient privacy and confidentiality. In our study, only 1 app (Agenda du sommeil antifatigue) involved personal data, but it was nonnominative and confidentiality was guaranteed. In addition, this app stands out as a robust model, being managed by the Bordeaux University Hospital, validated by a Data Protection Officer, and hosted within a secure institutional framework ensuring General Data Protection Regulation compliance. While other apps lack transparency regarding data encryption and remote storage, they do not collect or store any personally identifiable information or nominative data within the app.

The absence of age-verification and crisis-response features represents a limitation in adolescent-focused care apps. However, as the apps in our study are professional-facing tools, they lack autonomous safety triggers by design. Consequently, the safety of vulnerable users relies on practitioners’ expertise, with the clinical framework compensating for the absence of built-in safeguarding features.

Analysis of the MARS-F subdomains reveals that the apps had higher scores in Information and Functionality (mean scores of 3.5 for both), while Engagement and Esthetics received lower ratings (mean scores of 2.0 and 2.5, respectively). This suggests that current apps are often developed as “digital brochures” rather than interactive tools. This situation is frequently observed in mHealth literature [49], where institution- or professional-facing apps prioritize clinical evidence and technical reliability over visual appeal or gamification. For a health care provider, a tool that is difficult to navigate during a consultation is unlikely to be adopted, regardless of the quality of its medical evidence. This implies the need for development frameworks. These frameworks have to integrate optimal user-centered design and esthetics to encourage users to engage with an app, so that tools can be more effective in clinical settings. Our findings align with the recent update by the High Authority for Health on the evaluation framework for mHealth apps [21]. The High Authority for Health confirmed that medical expertise and scientific evidence are mandatory for digital tools to be integrated into official health care pathways. However, our study shows that very few prevention-focused apps currently meet these high standards, particularly for the adolescent population. Medical societies and health authorities should establish certification labels or “official app libraries” to guide health care professionals. This would reduce the “search burden” for clinicians and ensure the quality of tools used with adolescents.

Our study conducted a rigorous selection of existing apps covering all areas of prevention recommended for adolescents during consultations with a health care professional. This type of review should be carried out systematically before any new tool is developed to avoid duplication and focus creative energy on innovative digital tools. Another study will look at existing websites that are relevant to helping health care professionals with their prevention work with adolescents during consultations. Similar to this study, it will examine websites in all the recommended areas of prevention, and the sites will be analyzed using a validated tool, Netscoring [50]. These 2 research projects (reviews of apps and websites) could be used as models for similar digital reviews prior to the creation of any new app or website. Finally, whatever tool is identified or designed, feasibility and acceptability studies by the professionals themselves will be necessary.

While the MARS-F scale offers a robust evaluation of information quality and credibility, it lacks explicit criteria for assessing clinical safety features. Future research should integrate specific safety frameworks to determine whether apps incorporate essential emergency protocols or risk-warning systems, particularly for users presenting with severe mental health symptoms.

Pending assessment of the acceptability and feasibility in clinical practice of the 7 apps selected in our study, we recommend 4 of these apps: “Agenda du sommeil antifatigue” and “mhGAP-IG 2.0 App” (with certain limitations linked to the technical shortcomings of “mgGAP-IG 2.0 App”); of the 5 sexual health apps, we recommend “WHO Contraception Tool,” available in several languages, and “Practice Family Planning,” which is richer in video content for health care professionals. These 5 apps could be included in the recommended tools for health care professionals on the planned global website.

To provide a comprehensive operational framework for the 4 recommended apps, Table 5 categorizes them based on the following 6 key implementation parameters:

There are many areas of prevention that need to be addressed with adolescents during consultations. Only 7 apps were relevant for helping health care professionals in their prevention work with adolescents. Five deal with sexual health, particularly contraception, and 2 with mental health. No apps have been validated for orthopedics, social relations, addictions, or vaccinations. Health care professionals are connected, especially online during consultations. A general online tool to help health care professionals with prevention among adolescents during consultations is relevant but currently nonexistent. Our study will contribute to the selection of relevant content for the development of such a tool, but further research is needed to ensure that it meets the needs of health care professionals.