Our approach was structured in two stages: first, the systematic identification of relevant apps and, second, their assessment based on predefined quality, usability, and evidence-based criteria. This sequence aims to be a realistic, reproducible way to support both health care professionals and end users in the navigation and evaluation of the growing mHealth landscape.

An initial exploratory analysis was conducted to identify projects, initiatives, and organizations involved in the evaluation of health apps. Most of these resources have been catalogued in two documents by the Spanish Ministry of Health [12] and the European project mHealth-Hub [13,14]. Both include governmental and nongovernmental efforts. Each initiative was then individually reviewed to examine its content and structure. Notable heterogeneity among the initiatives was observed, allowing them to be classified into two main groups: (1) platforms offering pre-evaluated apps, used as databases for the search, and (2) those focused on normative aspects, standardization, and quality assessment. Given the diversity in the approaches and structures of these platforms, the search began with general terms related to the study’s focus, such as healthy eating, staying healthy, diet, lifestyle, preventive medicine, weight or healthy habits, which were later adapted to the navigation logic, taxonomy, or filtering system of each initiative (for more details, see Multimedia Appendix 1).

From August to December 2021, we searched for health apps that met the following inclusion criteria: aimed at adults (over 18 years old), available in English or Spanish, and offering nutritional guidance (eg, recommendations for dietary changes or meal planning). Apps that solely functioned as food diaries, calorie counters, or barcode readers but did not include any specific recommendations were excluded. We examined app descriptions and features to confirm eligibility.

In the absence of established guidelines for health app evaluation, we followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist (Multimedia Appendix 2) [15,16] adapted to our research objectives. We reviewed 28 certification bodies (see Table S1 in Multimedia Appendix 1), of which only 5 (17.9%) contained apps meeting our inclusion criteria (see Table 1 for additional details):

Healthy Living AppsGuide [17] assesses app functionality using the Mobile App Rating Scale (MARS) [18] and behavior change using the App Behavior Change Scale (ABACUS) [19]. Each criterion is rated on a scale of 0-5 stars, with additional considerations, such as app cost and data export capabilities.

MyHealthApps [20] focused on user preferences and developer-related data. This resource operated as a catalog, rather than assigning individual scores to apps. It assessed transparency regarding pricing, contact details, geographical location, and security measures. Unfortunately, it closed in May 2022.

ORCHA (Organization for the Review of Care and Health Applications) [21] uses a seven-step evaluation system comprising three main domains: data, professional guarantee, and usability and accessibility. App functionalities and features are also considered. A maximum score of 100 points is attainable for each domain, with ratings above 65 considered good quality. Scores between 45 and 65 suggest areas for further investigation, and scores below 45 deem the app (or domain) potentially unsafe or ineffective.

Gemeentelijke Gezondheidsdienst (GGD) AppStore [22] follows an evaluation method considering app availability, pricing, compatibility with different operating systems, provider contact details, promotion of healthy behavior, data management, and goal setting. To enter evaluation, apps must focus on self-care and include at least two methods for behavior change. The apps are also assessed for usability, reliability, privacy, safety, and relevance, with each characteristic rated as good, sufficient, or inadequate. Apps are scored up to a maximum of 5 stars.

Health Navigator [23] is supported by the New Zealand Ministry of Health and provides a library of reliable apps. The evaluation process encompasses app features, functionalities, quality of information, and relevance to users. Apps must fulfil specific criteria, including evidence-based content and an evaluation of effectiveness, acceptability, and usability, and include a privacy statement. Clinicians with diverse expertise rate the apps from 1 (very poor, not recommended) to a maximum of 5 stars (excellent). End users also provide feedback. Finally, apps deemed clinically unsafe or potentially harmful are excluded.

As health apps are frequently updated, a follow-up search was performed in May 2022. By that time, MyHealthApps was no longer active. From this point on, three rounds of app evaluations followed (see Figure 1).

The apps underwent three rounds of evaluation involving a total of seven reviewers who downloaded and assessed them on the devices described later. An endocrinologist (specialized in endocrinology and nutrition), one dietician, two psychologists, and three epidemiology and public health professionals participated. A second endocrinologist led the evaluation process and coordinated and summarized meetings.

Given the nature of this study, no ethics committee approval or informed consent to participate was needed. All authors declared their consent for publication.

Of the 28 certification bodies identified (see Table S1 in Multimedia Appendix 1), 14 (50%) catalogued a total of 557 apps across various categories, encompassing terms such as “healthy eating,” “nutrition,” “diet”, “healthy practices,” and “lifestyle”. Only 5 (17.9%) included apps that fulfilled our inclusion criteria (see Table 1). These certification bodies used a variety of criteria in their evaluation processes, in terms of both the elements assessed and the tools used. Nevertheless, there was a significant overlap in the factors considered key for determining app quality, including functionality, usability, engagement, aesthetics, privacy, data protection, and effectiveness in promoting behavior change.

A total of 41 (7.4%) apps met the inclusion criteria. After eliminating duplicates, 30 (73.2%) apps remained, of which 11 (36.7%) met the exclusion criteria (see Figure 2).

The characteristics of the selected 19 (63.3%) apps are summarized in Table 2. Of these 19 apps, 5 (26.3%) were included in two or more of the certification bodies: MyFitnessPal, FatSecret, MyNetDiary, Noom, and 8fit Workout & Meal Planner. Only the apps included in ORCHA had undergone formal evaluation of their content.

As per the definition, all apps included nutritional interventions, but most also combined different strategies or features, making their categorization challenging. Overall, 14 (73.7%) apps included weight control or tracking (numbered 1-6, 8, 11, 12, 14, 16-19 in Table 2), 12 (63.2%) allowed for calorie counting (numbered 1-4, 6-8, 11-13, 17, 19 in Table 2), and 13 (68.4%) facilitated the recording of physical activity (numbered 1-4, 6-9, 14-17, 19 in Table 2). Furthermore, 12 (63.2%) apps (numbered 1-8, 10, 11, 16, 18 in Table 2) incorporated behavior change techniques, such as motivational feedback, health education promotion, and goal setting.

Table 3 presents the scores given to each app by the reviewers during the three rounds of assessment, by the certification bodies, and by a commercial app store.

In this paper, we highlighted the need for a common evaluation framework to assess the quality of health apps and proposed a robust approach to this end. Our use case focused on apps promoting healthy eating. A total of 19 apps were included, which had already been positively assessed by dedicated certification bodies. The latter used a variety of evaluation criteria, and most did not include content quality among them. Following a rigorous, three-step evaluation process by seven experts, only 5 apps were recommended. The selection was based on a comprehensive assessment of the quality of their contents (using QUEST), as well as their usability and the use of behavior change techniques.

Health apps have emerged as promising tools to promote healthier lifestyles, due to their widespread accessibility, user friendliness, and cost-effectiveness [26-28]. However, the app market’s diversity and the lack of systematic evaluation processes leave consumers with limited tools to judge which apps are effective, safe, and suited to their needs.

To address this issue, several national and international actions have attempted to assess health app quality and safety [17,20-23]. Our investigation identified 28 initiatives dedicated to the evaluation of health apps, 5 including apps fulfilling our inclusion criteria. Although each of these initiatives uses distinct criteria for app evaluation, they all generally emphasize similar key factors, such as data security, privacy, ease of use, accessibility, usability, support for user-health care professional communication, personalization options, and capacity to induce behavior change. In a recent review, the importance of adherence to scientific evidence, as well as additional features (eg, gamification and cocreation of the app with health professionals and users), were supported [29]. Although a multitude of tools and scores have been proposed to evaluate different aspects of app quality, as of today, one single robust procedure or set of criteria does not exist [30-32], although some national and international evaluation efforts have been developed [33-35]. Additionally, all the certification bodies reviewed in this study have cautioned that they cannot guarantee the precision, quality, trustworthiness, and effectiveness of the health apps they assess, increasing uncertainty for the users.

Acknowledging these challenges, our study sought to evaluate nutrition-focused health apps using a tool aimed at the health information within the apps, assuming that health information based on evidence would support safety and effectiveness. We selected QUEST to assess the apps, since it has been specifically designed to evaluate health information and incorporates items such as attribution or study type to evaluate the quality of information sources. Additionally, it assesses items such as conflict of interest and tone to evaluate how information is presented to the user, where any sort of advertising is penalized. We found no significant correlations among the Play Store scores for the apps, the QUEST total mean score, or any of the certifier scores, probably because they assess different dimensions. QUEST is a robust tool for the assessment of content quality but does not address usability, engagement, or behavior change techniques, all factors known to influence the perceived quality of an app [11]. Therefore, the research team decided to complement this assessment with a subjective score, which also included these aspects. Such factors may influence the app store ratings, and many are directly evaluated by the certification bodies but are overlooked by QUEST. In contrast, Play Store scores are based on user opinions, which can be influenced not only by all these factors but also by popularity, endorsement by influencers, aesthetics, or alignment with their own nutritional preconceptions, which might not be backed by scientific evidence.

Furthermore, it is important to acknowledge that apps are dynamic and subject to constant updates. Although the latter can be beneficial if they enhance the quality of app contents, they also pose a challenge for app evaluation, as acknowledged by a recent consensus [36]. Thus, continuous reassessment would be necessary to account for the changes. Access to older app versions may not always be possible, and certain apps may become completely unavailable over time. Any app marketed as a health app should be subject to special scrutiny, particularly regarding collection of sensitive data and potential for harmful effects on users. Previously proposed improvements include establishing national lists of tested and trusted health apps, creating a catalog of health apps accessible to patients only if prescribed by professionals and guidelines for app developers toward evidence-based, unbiased, high-quality apps, with wider assessment requirements for higher-risk tools [14,33,36-39].

Our work has several strengths that contribute to the field of health app evaluation. First, the adoption of QUEST to assess all the apps constitutes a rigorous approach for the evaluation of content quality. This systematic assessment of the content was complemented by a subjective evaluation by the panel of usability and behavior change techniques. Additionally, the study’s methodological approach, involving multiple, independent reviewers and the evaluation of a diverse range of apps, strengthens the validity and comprehensiveness of the results. Finally, the study’s alignment with the existing literature and national and international certification/assessment bodies highlights its relevance and potential impact on improving health app quality and user safety.

We acknowledge that the study also has some limitations. The small sample size of apps reviewed and their focus on nutrition-related interventions may limit the generalizability of our results, since they represent an incomplete picture of the overall quality and safety of available health apps. The decision to exclude apps that solely function as food diaries and calorie counters may be controversial. We chose apps offering recommendations, meal plans, and recipes, because we focused on the quality of the information provided. Nevertheless, we acknowledge that food diaries and calorie counting could also help the users become more aware of their eating habits and induce a change in behavior despite the lack of any other specific nutritional advice and therefore be considered a type of intervention. Another limitation is the subjective nature (eg, tone) or the scoring difficulty (eg, attribution) of some of the items in QUEST. Discrepancies in the first round of evaluations, reflected by rather low κ coefficient values, were mainly attributed to the fact that the information to score some items might be difficult to find in the app or might even be found behind a link and thus not be obvious. Hence, in the second round of evaluations using QUEST, we standardized the time spent on the assessment of each app. In addition, some aspects of app quality are overlooked by QUEST (eg, usability, engagement, and capacity to induce positive behavior changes). Thus, QUEST is not a comprehensive enough tool to judge app quality on its own, but it is complementary to other measures. Finally, to overcome these difficulties, several rounds of revision were made, which made the evaluation process longer than initially planned.

To summarize, during this review, we encountered several challenges. In the first place, numerous certification bodies were scanned for evaluated apps. Creating a common repository collecting their criteria and decisions would facilitate this task [40]. In addition, the rapid change in mHealth apps poses a challenge to the timeliness of evaluation. This might be mitigated by the development of living systematic reviews, which are continuously updated [39] or the use of rapid reviews [41] and evidence maps [42]. Lastly, there is currently no standardized procedure to evaluate app quality. To address this, we included what we considered a critical and often overlooked dimension in previous evaluation efforts: a thorough assessment of the quality of the health information provided by the apps using QUEST scoring.

In conclusion, despite their previous evaluations by various certification bodies, only 5 of 19 apps promoting healthy eating met the quality standards set by our experts. Our study calls for enhanced scrutiny and regulatory measures to ensure that health apps meet rigorous standards of accuracy, reliability, and user safety. Guidelines for developers for the design of evidence-based, unbiased, high-quality health apps, as well as technological solutions for real-time monitoring of the apps, would address these challenges and improve reliability.