This paper is in the following e-collection/theme issue:

Quality Evaluation and Descriptive Analysis/Reviews of Multiple Existing Mobile Apps (362) mHealth for Wellness, Behavior Change and Prevention (2728) mHealth for Data Collection and Research (907) Usability of Apps and User Perceptions of mHealth (892) Evaluation and Research Methodology for mHealth (193) mHealth for Health Administration (136)

Published on in Vol 10, No 3 (2022): March

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/35879, first published .
Nutrition-Related Mobile Apps in the French App Stores: Assessment of Functionality and Quality

Nutrition-Related Mobile Apps in the French App Stores: Assessment of Functionality and Quality

Nutrition-Related Mobile Apps in the French App Stores: Assessment of Functionality and Quality

Authors of this article:

Prescilla Martinon1 Author Orcid Image ;   Ina Saliasi1 Author Orcid Image ;   Denis Bourgeois1 Author Orcid Image ;   Colette Smentek1 Author Orcid Image ;   Claude Dussart1 Author Orcid Image ;   Laurie Fraticelli1 Author Orcid Image ;   Florence Carrouel1 Author Orcid Image
Article Authors Cited by (13) Tweetations (4) Metrics

    Original Paper

    Health, Systemic, Process UR 4129 Research Unit, University Claude Bernard, University of Lyon, Lyon, France

    *these authors contributed equally

    Corresponding Author:

    Florence Carrouel, MPh-PhD

    Health, Systemic, Process UR 4129 Research Unit

    University Claude Bernard

    University of Lyon

    11 Rue Guillaume Paradin

    Lyon, 69008

    France

    Phone: 33 478785745

    Email: florence.carrouel@univ-lyon1.fr


    Background: The global burden of disease attributes 20% of deaths to poor nutrition. Although hundreds of nutrition-related mobile apps have been created, and these have been downloaded by millions of users, the effectiveness of these technologies on the adoption of healthy eating has had mixed

    Objective: The aim of this study was to review which nutrition-related mobile apps are currently available on the French market and assess their quality.

    Methods: We screened apps on the Google Play Store and the French Apple App Store, from March 10 to 17, 2021, to identify those related to nutritional health. A shortlist of 15 apps was identified, and each was assessed using the French version of the Mobile App Rating Scale: 8 dietitians and nutritionists assessed 7 apps, and the remaining apps were randomly allocated to ensure 4 assessments per app. Intraclass correlation was used to evaluate interrater agreement. Means and standard deviations of scores for each section and each item were calculated.

    Results: The top scores for overall quality were obtained by Yazio - Régime et Calories (mean 3.84, SD 0.32), FeelEat (mean 3.71, SD 0.47), and Bonne App (mean 3.65, SD 0.09). Engagement scores ranged from a mean of 1.95 (SD 0.5) for iEatBetter: Journal alimentaire to a mean of 3.85 (SD 0.44) for FeelEat. Functionality scores ranged from a mean of 2.25 (SD 0.54) for Naor to a mean of 4.25 (SD 0.46) for Yazio. Aesthetics scores ranged from a mean of 2.17 (SD 0.34) for Naor to a mean of 3.88 (SD 0.47) for Yazio. Information scores ranged from a mean of 2.38 (SD 0.60) for iEatBetter to a mean of 3.73 (SD 0.29) for Yazio. Subjective quality scores ranged from a mean of 1.13 (SD 0.25) for iEatBetter to a mean of 2.28 (SD 0.88) for Compteur de calories FatSecret. Specificity scores ranged from a mean of 1.38 (SD 0.64) for iEatBetter to a mean of 3.50 (SD 0.91) for FeelEat. The app-specific score was always lower than the subjective quality score, which was always lower than the quality score, which was lower than the rating from the iOS or Android app stores.

    Conclusions: Although prevention and information messages in apps regarding nutritional habits are not scientifically verified before marketing, we found that app quality was good. Subjective quality and specificity were associated with lower ratings. Further investigations are needed to assess whether information from these apps is consistent with recommendations and to determine the long-term impacts of these apps on users.

    JMIR Mhealth Uhealth 2022;10(3):e35879

    doi:10.2196/35879

    Keywords

    mobile apps; behavior change; diet; healthy food; nutrition; prevention; mHealth; mobile health; lifestyle; French 


    Worldwide, the burden of noncommunicable diseases continues to rise [1]. The Global Burden of Disease study [2] found that 1 in 5 deaths was due to poor diet; thus, dietary factors were responsible for 11 million deaths per year, which was more than those from any other risk factor included in the study. Several forms of malnutrition, including obesity and undernutrition, can coexist in the same population and have a significant impact on health systems. Primary health care services and lifestyle behavior improvement based on education and behavior change have great potential to decrease the global burden of noncommunicable diseases, improve health throughout the life course, and enhance well-being [3]. Thus, counseling on healthy diets and proper nutrition are among the most important nutritional interventions for promotion, prevention, treatment, and rehabilitation [4].

    Mobile health is defined by the World Health Organization’s Global Observatory for eHealth as “medical and public health practice supported by mobile devices, such as mobile phones, patient monitoring devices, personal digital assistants, and other wireless devices [5].” In recent years, the number of web-based mobile health apps has increased exponentially. Currently, there are more than 325,000 mobile health apps available on major app stores. These apps are in addition to web-based health apps available on other platforms such as websites, PC software, and game consoles [6].

    Furthermore, the number of apps for improving nutrition and fitness continues to grow [7]. Hundreds of nutrition-related mobile apps have been created and downloaded by millions of users over the past few years [8]. The fact that some of these apps have been downloaded numerous times indicates that people want to monitor and control their diet [9]. Access to these mobile health apps is primarily via smartphones [10]. However, it has been shown that web and mobile technologies related to nutrition have a greater impact if combined with personalized advice from a dietitian [11]. Although other prevention approaches are required, the development of effective and equitable nutrition programs is a prerequisite [12]. Since the number of apps is growing exponentially every year, it is essential to update them regularly [13]. The main industry-wide challenge is to provide credible evidence for these apps [14]. To date, little usability testing of these apps has been conducted [6]. Only a small number of English-language digital health apps have reported their usability evaluation results [6]. Although the usefulness of technologies has been demonstrated, results on the effectiveness of technology integration on the adoption of healthy eating habits are conflicting [15]. In 2018, French was spoken in 29 countries on all continents, by approximately 300 million people; 235 million people use it daily, and 90 million people are native speakers [16]; however, no overall evaluation of French-language nutrition apps has been identified in the literature.

    The aim of this study was to review which nutrition-related mobile apps were available on French App stores and to evaluate their quality.


    Selection of the French Mobile Health Apps

    Two academic researchers searched for nutritional health–related apps from March 10-17, 2021 on the French Apple App Store (for iOS) and the French Google Play Store (for Android) using the following search terms: “nutrition” (nutrition), “diététique” (dietetics), “alimentation” (food intake), “régime alimentaire” (diet), and “manger sain” (healthy eating). Because the use of truncation and logic operators (such as AND, OR, and NOT) were not possible in the App Store and Google Play Store, each search term was provided separately.

    The 2 researchers individually eliminated duplicate apps by cross-checking name of the app and the developer before comparing their respective lists. The download pages of the remaining apps were screened, and then, apps were downloaded for in-depth screening using the inclusion criteria: (1) French language, (2) targeting adult users, (3) nutrition, diet or eating habits as subject matter, (4) self-personalized programs, and (5) free (or free for at least 14 days). Mobile health apps focusing on the following topics were excluded: sports, shopping, water alert notifications only, special diet (diabetic, baby, pregnancy, vegan, religious, abstain from eating, weight gain), recipes, apps created specifically for nutritionists’ patients, meal delivery, pollution trackers, allergy and intolerance trackers, and barcode scanners.

    Selection of a Standardized Rating Scale for Mobile Apps

    We used the French version of the Mobile App Rating Scale (MARS-F). The MARS-F includes 19 objective items rated with a 5-point Likert scale that are divided into 4 sections [17-21]: the engagement section (5 items) evaluates if the app is fun, interesting, customizable, and interactive (eg, sends alerts, messages, reminders, feedback, or allows sharing); the functionality section (4 items) focuses on app operation, easy to learn, navigation, flow logic, and gestural design of the app; the aesthetics section (3 items) evaluates the graphic design, the overall visual appeal, the color scheme, and the stylistic consistency; and the information quality section (7 items) determines if the app contains high-quality information (eg, text, feedback, measurements, and references) from a credible source. The mean scores and distributions for each section were calculated. The overall MARS-F mean score was the mean score of the engagement, functionality, aesthetics, and information quality sections. Additionally, there is a subjective quality section (4 items), which evaluates the user’s interest for the app, and a specificity section, which assesses perceived effect on the user’s knowledge, attitudes, and intentions to change as well as likelihood of changing the identified targeted behaviors (we used daily habits).

    Evaluation

    Training the Raters for Evaluation

    We asked 8 dieticians and nutritionists (Multimedia Appendix 1) to rate the apps. All raters viewed a training video in French (available upon request to the corresponding author) developed for the MARS-F [22], adapted from the English-language training video [20]. To train, all raters evaluated 2 apps that had been excluded. For this, the raters downloaded and tested each app for at least 15 minutes and fulfilled the questionnaire of MARS-F. When an individual item’s rating score differed by at least 2 points, raters discussed until consensus was reached to ensure similar understanding of the item.

    App Selection

    Among the 15 apps that were included, we randomly selected 7 apps for evaluation by all raters (Compteur de calories FatSecret, Yazio - Régime et Calories, MyFitnessPal, Macros - Compteur de calories, Foodvisor, Lose It! - Compteur de calories, and Compteur de calories), and the remaining 8 apps were assigned to 4 raters (Lifesum: Compteur de calories, Naor, iEatBetter: Journal alimentaire, Le secret du poids, Compteur de calories ScanFood, FeelEat, Kalipi, and Bonne App).

    The evaluation process took place from April to May 2021. The raters independently used each app for 15 minutes, and then immediately evaluated the app using a web-based MARS-F questionnaire.

    Statistical Analysis

    To evaluate the interrater reliability, intraclass correlations (2-way random, average measures, absolute agreement) [23,24] and their 95% confidence intervals were calculated for the 7 common apps (for each item, section, and overall). The mean values and standard deviations were calculated for each item and for each section. Item 19 was excluded from all analyses due to missing values.

    Scatter plots were used to compare differences between the quality of the apps (for each item and for each section).

    The correlation between the overall quality mean and subjective item 23 (“What is your overall star rating of the app?”) was evaluated through the Pearson correlation coefficient.

    Statistical analyses were performed using Stata (version 15; StataCorp LLC) and using dplyr (version 1.0.8) and ggplot2 (version 3.3.5) packages with R software (version 4.1.1; The R Project for Statistical Computing).


    Selection of Mobile Apps

    A total of 226 apps in the Apple App Store and 971 apps in the Google Play Store were identified (Figure 1), with 78 apps available on both systems. After screening, 18 apps were preliminarily identified. After downloading, 15 apps were included.

    ‎
    Figure 1. Selection flowchart.
    View this figure

    Characteristics of Mobile Apps

    No common developer was identified for the 15 apps (Multimedia Appendix 2). Only 2 apps were fully free of charge; 1 app was free for 30 days, and the others required in-app purchases to function completely. MyFitnessPal was the most downloaded app (n=26,804 in the Apple App store and n=2,397,052 downloads in the Android App store), followed by Yazio (n=51,674 in the Apple App store and n=373,162 downloads in the Android App store) and FatSecret (n=3711 in the Apple App store and n=394,958 downloads in the Android App store).

    All 15 apps targeted behavior change, goal setting, and physical health (Table 1). Most apps (10/15, 67%) focused on increasing happiness and well-being. The theoretical background and strategies used were (1) information and education, (2) monitoring and tracking, and (3) goal setting. The apps were designed for adults (15/15, 100%), young adults (15/15, 100%), adolescents (13/15, 87%), and children under 12 years (11/15, 73%). All 15 apps (100%) sent reminders, and 10 apps (10/15, 67%) required internet access to function.

    Table 1. Characteristics of the 15 nutrition mobile apps.
    CharacteristicApp (n=15), n (%)a
    Focus or target

    		Increase happiness or well-being10 (67)

    		Mindfulness, meditation, or relaxation3 (20)

    		Anxiety or stress3 (20)

    		Behavior change15 (100)

    		Goal setting15 (100)

    		Relationships1 (7)

    		Physical health15 (100)
    Theoretical background or strategies

    		Assessment10 (67)

    		Feedback10 (67)

    		Information or education15 (100)

    		Monitoring or tracking15 (100)

    		Goal setting15 (100)

    		Advice, tips, strategies, and skills training9 (60)

    		Cognitive behavioral therapy - Behavioral (positive events)5 (33)

    		Cognitive behavioral therapy - Cognitive (thought challenging)5 (33)

    		Acceptance commitment therapy4 (27)

    		Mindfulness or meditation1 (7)

    		Relaxation1 (7)

    		Gratitude0 (0)

    		Strengths based6 (40)

    		Other0 (0)
    Age group

    		Children (under 12 years)11 (73)

    		Adolescents (13-17 years)13 (87)

    		Young adults (18-25 years)15 (100)

    		Adults15 (100)
    Technical aspects of app

    		Allows sharing (Facebook, Twitter, etc)4 (27)

    		Has an app community5 (33)

    		Allows password-protection9 (60)

    		Requires log-in2 (13)

    		Sends reminders15 (100)

    		Needs web access to function10 (67)

    aMore than one could be applicable; therefore, percentages do not add to 100%.

    Reliability of the Evaluation

    The reliability of the evaluations of the 7 common apps was considered good for overall quality (ICC 0.89, 95% CI 0.70-0.98) and for engagement (ICC 0.83, 95% CI 0.57-0.96), functionality (ICC 0.77, 95% CI 0.45-0.95), and aesthetics (ICC 0.83, 95% CI 0.57-0.97) sections individually. The reliability was excellent for the information quality section (ICC 0.92, 95% CI 0.78-0.98).

    Quality of the Content of the Nutrition-Related Mobile Apps

    The best quality scores (Figure 2; Multimedia Appendix 3) were obtained by Yazio (mean 3.84, SD 0.32), FeelEat (mean 3.71, SD 0.47), and Bonne App (mean 3.65, SD 09); whereas, the worst quality scores were obtained by Naor (mean 2.34, SD 0.39), iEatBetter (mean 2.59, SD 0.40), and Lose It! (mean 2.79, SD 0.29).

    ‎
    Figure 2. Qualitative evaluation of nutrition-related apps. Section A: Engagement; Section B: Functionality; Section C: Aesthetics; Section D: Information; Section E: Quality.
    View this figure

    The engagement scores ranged from a mean of 1.95 (SD 0.5) for iEatBetter to a mean of 3.85 (SD 0.44) for FeelEat. The functionality scores ranged from a mean of 2.25 (SD 0.54) for Naor to a mean of4.25 (SD 0.46) for Yazio. The aesthetics scores ranged from a mean of 2.17 (SD 0.34) for Naor to a mean of 3.88 (SD 0.47) for Yazio. The information quality scores ranged from a mean of 2.38 (SD 0.60) for iEatBetter to a mean of 3.73 (SD 0.29) for Yazio. For all apps, except Naor, the functionality mean score was always higher than the engagement mean score.

    The subjective quality scores (Figure 3) ranged from a mean of 1.13 (SD 0.25) for iEatBetter to a mean of 2.28 (SD 0.88) for FatSecret. The best subjective quality scores were obtained by FatSecret (mean 2.28, SD 0.88), FeelEat (mean 2.13, SD 0.48), and ScanFood (mean 2.13, SD 0.92); whereas, the worst quality scores were obtained by iEatBetter (mean 1.13, SD 0.26), Naor (mean 1.13, SD 0.25), and Lose It! (mean 1.41, SD 0.48).

    ‎
    Figure 3. Subjective qualitative evaluation of nutrition-related apps (Section E).
    View this figure

    Specificity of the Content of the Nutrition-Related Mobile Apps

    Scores for specificity of the content of the apps (Figure 4) ranged from a mean of 1.38 (SD 0.64) for iEatBetter to a mean of 3.50 (SD 0.91) for FeelEat. The best subjective quality scores were obtained by iEatBetter (mean 1.38, SD 0.64), Compteur de calories (mean 2.92, SD 0.79), Foodvisor (mean 2.83, SD 0.62), and ScanFood (mean 2.83, SD 1.25); whereas, the worst quality scores were obtained by Lifesum (mean 1.88, SD 1.18), Lose It! (mean 1.79, SD 0.48), and iEatBetter (mean 1.38, SD 0.64).

    ‎
    Figure 4. App-specific scores (Section F).
    View this figure

    Strengths and Weaknesses of Each App

    The app-specific score was always lower than the subjective quality score, which was always lower than the quality score. This score was lower than the rating score from the iOS or Android app stores (Multimedia Appendix 2; Table S1 in Multimedia Appendix 3).

    Low overall quality scores (Figure 5) were due to the information quality scores, for all apps except FatSecret, iEatBetter, Naor, and Yazio. In the information quality section, the worst score was observed for the item regarding the credibility of the app for all apps except for Bonne App, which obtained the worst score for goals and quality of information items, and Lifesum, which obtained the worst score for the goals item. In the subjective quality section, low scores were the result of the item indicating whether people be willing to pay for this app. The specificity scores were very close between items for the same app.

    ‎
    Figure 5. Heatmap of the average scores per item and per app, from yellow (1: worst score) to green (5: best score).
    View this figure

    Correlation Between MARS and Stars Ratings

    The correlation between the quality mean and the subjective item 23 (“What is your overall star rating of the app?”) was considered to be good (r=0.67, P<.001) and indicated that the quality score (overall) was generally higher than that of subjective item 23.

    Correlation analysis between overall MARS-F scores of the apps and their respective store ratings was limited by the availability of store ratings and the discrepancies among the number of raters. The store ratings were higher than overall MARS-F mean scores. Store ratings ranged from 3.0 (Le secret du poids) to 4.9 (ScanFood) for the iOS store and from 3.0 (ScanFood) to 4.7 (FatSecret) for the Android store.


    Principal Findings

    The increasing public consciousness and high comorbidity burden related to unhealthy nutrition has highlighted the necessity of a healthy diet [25]. Nutrition behaviors can be improved by using mobile health apps, which have become very popular [26]. For diabetes [27], renal disease [28], weight loss [29], and age-related macular degeneration [30], health and nutrition professionals have used mobile health apps to monitor and encourage better lifestyle and dietary choices. The use of mobile health apps has also been found to increase adherence to dietary monitoring [29,31].

    Screening of nutrition-related apps available in the French Apple App and Google Play App stores yielded 15 apps. In another study, screening of the Korean Apple app and Google Play stores yielded 29 nutrition-related apps [32]; the study [32] used 2 criteria—including only apps rated 4 stars or higher and the top 100 most reviewed apps. Another study, screening of the US Google Play Store yielded 86 apps, but the criterion for inclusion (only apps rated 4 stars or higher) was less restrictive [33].

    All 15 apps targeted behavior change, physical health, and goal setting via information and education and monitoring and tracking. A previous study [34] showed that diet monitoring and education were the most frequently used functions in diet and nutrition apps [34]. All ages were targeted for 73.3% (11/15) of the apps; this finding is consistent with that from another study [33], which found that 94% of diet and nutrition apps appealed to users of all ages.

    Ratings in the iOS and Android stores were higher than the MARS-F quality scores. Star ratings and user comments are valuable to users because they provide insight into the effectiveness and popularity of apps [33], but star ratings do not provide objective assessment of quality. In contrast to other studies [32,33], our study did not used star ratings as an inclusion criteria; however, for the 15 apps included (except Bonne App for which the number of raters or downloads was not sufficient) the star score was greater than 4.

    Quality scores were greater than 2.5, except for Naor (mean 2.34, SD 0.39). Functionality was the strength for all apps, except for Naor. The high scores could be explained by the inclusion of scroll and zoom features to increase readability. The maximum score of 5 was obtained for the ease of use, navigation, and gestural design (FeelEat, iEatBetter, Yazio, and FatSecret). In contrast, Naor navigation was rated low (mean 2, SD 0.82), which could be explained by difficulty in accessing the menu, the amount of data, and the design. The weakness of all the apps, except FatSecret, iEatBetter, Naor, and Yazio, was information quality. The worst score in the information quality section was typically for credibility of the app. This corresponded to the fact that the source of information was identified, but the source’s validity or reliability was questionable (eg, commercial enterprise with vested interest). Moreover, the level of scientific evidence was difficult to evaluate. The evaluators selected “N/A The app has not been tested” in most cases; therefore, this item was not included in the statistical analysis. To the best of our knowledge, only 5 of the 15 apps (FatSecret [8,35-38], Lifesum [8,35,38], MyFitnessPal [8,35-45], Yazio [35,36,38], and Lose It! [8,35,37]) are indexed in PubMed. On the other hand, the information contained in these nutrition-related apps may have errors. For example, FatSecret, Lifesum, MyFitnessPal, and Yazio tended to underestimate total energy intake [38].

    The subjective quality score were always lower than the star rating scores from the iOS and Android stores. This can be explained because the evaluations in the stores are made by all the users; whereas, in our study, dieticians or nutritionists assessed the apps using the MARS-F. Indeed, the use of user version of MARS can show different results [21].

    Moreover, subjective quality scores were also lower than quality scores. This indicates that even if engagement, functionality, aesthetics, and information quality for an app were good, professionals did not think that they would use the app often in the next 12 months, and they would not be willing to pay for the app. This finding can be compared with the results of an international survey of health care professionals’ opinions on nutrition and diet apps [46]. Among 1001 health care professionals questioned, only 45.5% recommended these types apps to their patients. Surprisingly, 22.5% of people who had not yet recommended the use of these types of apps did not know of their existence. Health care professionals who have recommended apps may have used them as supplementary tools to broaden their daily practice, engage patients, enhance care, and possibly contribute to the reduction in health care costs [47]. Additionally, patients living with diseases such as diabetes or obesity may use apps for self-monitoring of their diet and physical activity [48].

    Generally, raters shared a common negative opinion on the potential impact of the nutrition-related apps on the behavior change (macro micronutrients intakes), even if these apps have already demonstrated positive results in with respect to the prevention of being overweight or other chronic disease [34,49-51].

    Limitations

    This study has several limitations. First, only nutrition-related apps available on both Apple and Android French stores, were included. Other stores, such as the Huawei store, the Samsung store, the Windows phone store, or BlackBerry, could have been investigated. Second, we chose to use the French version of the MARS because this scale is the most commonly used in scientific literature for mobile health app evaluation to date [52-57]. However, other scales, such as ENLIGHT or Application Quality Evaluation, which was initially specifically developed for the evaluation of mobile health app linked to nutrition purpose [58], could have been used. Third, the assessment was conducted by dieticians and nutritionists; whereas mobile health apps are intended for the general public. In further investigations, a comparison between ratings with the user version of MARS [21] and those from our study could be interesting.

    Perspectives

    In a recent study [48], clinicians mentioned that nutrition apps may improve patient outcomes when compared to traditional methods of monitoring dietary and physical activity behaviors [48]. Nutrition-related apps are appealing to users, based on the high number of downloads, which supports the fact that diet intake monitoring and recommendations could be managed through these tools [8,59,60]. Thus, the findings of our study could help French users of mobile apps and professionals to select the best nutrition-related apps in terms of quality and to choose the most appropriate health literacy elements. Furthermore, when used as part of an empowerment strategy, the app must adapt to the user's chronic disease.

    The implementation of new therapeutic programs that integrate mobile apps associated with follow-up with health professionals could be a key element in changing behavior. On one hand, it is important to remain vigilant with respect to the ethical issues surrounding the use of health data and the development of apps for commercial purposes. On the other hand, the discrepancy between scores obtained for the subjective quality section and for those for the specificity of the apps demonstrated that, although nutrition-related apps could be a key element in modifying the nutritional behavior of patients, for this, it is necessary to integrate the nutrition-related apps in professional practice. It would be interesting to conduct randomized clinical trials or longitudinal studies, using the 15 nutrition-related mobile apps identified in this study, to analyze nutritional behavioral modification from use of the apps and impacts on noncommunicable diseases.

    Acknowledgments

    We would like to thank the 8 dieticians for their participation (Brigitte Boniteau, Hélène Quelennec, Sarah Achatoui, Anais Chouin, Laure Poindextre, Florian Kwasniewski, Claire Neveu, and NRO), and Stéphane Béchet for statistical analysis expertise.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Characteristics of the raters, the hardware, and the software used.

    PDF File (Adobe PDF File), 61 KB
    ‎
    Multimedia Appendix 2

    Descriptive and technical information of the nutrition mobile apps.

    PDF File (Adobe PDF File), 148 KB
    ‎
    Multimedia Appendix 3

    Mobile App Rating Scale (MARS) scoring.

    PDF File (Adobe PDF File), 132 KB
    1. Bennett JE, Stevens GA, Mathers CD, Bonita R, Rehm J, Kruk ME, et al. NCD Countdown 2030: worldwide trends in non-communicable disease mortality and progress towards sustainable development goal target 3.4. Lancet 2018 Sep;392(10152):1072-1088. [CrossRef]
    2. GBD 2017 Diet Collaborators. Health effects of dietary risks in 195 countries, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2019 May 11;393(10184):1958-1972 [FREE Full text] [CrossRef] [Medline]
    3. Celis-Morales C, Lara J, Mathers JC. Personalising nutritional guidance for more effective behaviour change. Proc Nutr Soc 2015 May;74(2):130-138. [CrossRef] [Medline]
    4. Kraef C, Wood B, von Philipsborn P, Singh S, Peterson SS, Kallestrup P. Primary health care and nutrition. Bull World Health Organ 2020 Dec 01;98(12):886-893 [FREE Full text] [CrossRef] [Medline]
    5. Ryu S. Book review: mhealth: new horizons for health through mobile technologies: based on the findings of the second global survey on ehealth (global observatory for ehealth series, Volume 3). Healthc Inform Res 2012;18(3):231. [CrossRef]
    6. Maramba I, Chatterjee A, Newman C. Methods of usability testing in the development of ehealth applications: a scoping review. Int J Med Inform 2019 Dec;126:95-104. [CrossRef] [Medline]
    7. König LM, Sproesser G, Schupp HT, Renner B. Describing the process of adopting nutrition and fitness apps: behavior stage model approach. JMIR Mhealth Uhealth 2018 Mar 13;6(3):e55 [FREE Full text] [CrossRef] [Medline]
    8. Franco RZ, Fallaize R, Lovegrove JA, Hwang F. Popular nutrition-related mobile apps: a feature assessment. JMIR Mhealth Uhealth 2016 Aug 01;4(3):e85 [FREE Full text] [CrossRef] [Medline]
    9. Vasiloglou MF, van der Horst K, Stathopoulou T, Jaeggi MP, Tedde GS, Lu Y, et al. The human factor in automated image-based nutrition apps: analysis of common mistakes using the goFOOD lite app. JMIR Mhealth Uhealth 2021 Jan 13;9(1):e24467 [FREE Full text] [CrossRef] [Medline]
    10. Jusoh S. A survey on trend, opportunities and challenges of mhealth apps. Int J Interact Mob Technol 2017 Nov 27;11(6):73. [CrossRef]
    11. Rollo ME, Haslam RL, Collins CE. Impact on dietary intake of two levels of technology-assisted personalized nutrition: a randomized trial. Nutrients 2020 Oct 29;12(11):3334 [FREE Full text] [CrossRef] [Medline]
    12. Attwood S, van Sluijs E, Sutton S. Exploring equity in primary-care-based physical activity interventions using PROGRESS-Plus: a systematic review and evidence synthesis. Int J Behav Nutr Phys Act 2016 May 20;13(1):60 [FREE Full text] [CrossRef] [Medline]
    13. DiFilippo KN, Huang W, Andrade JE, Chapman-Novakofski KM. The use of mobile apps to improve nutrition outcomes: a systematic literature review. J Telemed Telecare 2015 Feb 12:243-253. [CrossRef] [Medline]
    14. Guo C, Ashrafian H, Ghafur S, Fontana G, Gardner C, Prime M. Challenges for the evaluation of digital health solutions-a call for innovative evidence generation approaches. NPJ Digit Med 2020;3:110 [FREE Full text] [CrossRef] [Medline]
    15. Lim SY, Lee KW, Seow W, Mohamed NA, Devaraj NK, Amin-Nordin S. Effectiveness of integrated technology apps for supporting healthy food purchasing and consumption: a systematic review. Foods 2021 Aug 12;10(8):1861 [FREE Full text] [CrossRef] [Medline]
    16. La langue française dans le monde. Gallimard.   URL: https:/​/www.​gallimard.fr/​Catalogue/​GALLIMARD/​Gallimard-Education/​La-Bibliotheque-Gallimard/​Hors-serie/​La-langue-francaise-dans-le-monde# [accessed 2021-12-01]
    17. Grainger R, Townsley H, White B, Langlotz T, Taylor WJ. Apps for people with rheumatoid arthritis to monitor their disease activity: a review of apps for best practice and quality. JMIR Mhealth Uhealth 2017 Feb 21;5(2):e7 [FREE Full text] [CrossRef] [Medline]
    18. Masterson CRM, Maurer MS, Reading M, Hiraldo G, Hickey KT, Iribarren S. Review and analysis of existing mobile phone apps to support heart failure symptom monitoring and self-care management using the mobile application rating scale (MARS). JMIR Mhealth Uhealth 2016 Jun 14;4(2):e74. [CrossRef] [Medline]
    19. Salazar A, de Sola H, Failde I, Moral-Munoz JA. Measuring the quality of mobile apps for the management of pain: systematic search and evaluation using the mobile app rating scale. JMIR Mhealth Uhealth 2018 Oct 25;6(10):e10718 [FREE Full text] [CrossRef] [Medline]
    20. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015;3(1):e27 [FREE Full text] [CrossRef] [Medline]
    21. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the mobile application rating scale (uMARS). JMIR Mhealth Uhealth 2016;4(2):e72 [FREE Full text] [CrossRef] [Medline]
    22. Saliasi I, Martinon P, Darlington E, Smentek C, Tardivo D, Bourgeois D, et al. Promoting health via mhealth applications using a french version of the mobile app rating scale: adaptation and validation study. JMIR Mhealth Uhealth 2021 Aug 31;9(8):e30480 [FREE Full text] [CrossRef] [Medline]
    23. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979 Mar;86(2):420-428. [Medline]
    24. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016 Jun;15(2):155-163 [FREE Full text] [CrossRef] [Medline]
    25. Martinon P, Fraticelli L, Giboreau A, Dussart C, Bourgeois D, Carrouel F. Nutrition as a key modifiable factor for periodontitis and main chronic diseases. J Clin Med 2021 Jan 07;10(2):197 [FREE Full text] [CrossRef] [Medline]
    26. Goodman S, Morrongiello B, Randall Simpson J, Meckling K. Vitamin D intake among young Canadian adults: validation of a mobile vitamin D calculator app. J Nutr Educ Behav 2015;47(3):242-247. [CrossRef] [Medline]
    27. Block G, Azar KM, Romanelli RJ, Block TJ, Hopkins D, Carpenter HA, et al. Diabetes prevention and weight loss with a fully automated behavioral intervention by email, web, and mobile phone: a randomized controlled trial among persons with prediabetes. J Med Internet Res 2015 Oct 23;17(10):e240 [FREE Full text] [CrossRef] [Medline]
    28. Campbell J, Porter J. Dietary mobile apps and their effect on nutritional indicators in chronic renal disease: a systematic review. Nephrology (Carlton) 2015 Oct;20(10):744-751. [CrossRef] [Medline]
    29. Wharton CM, Johnston CS, Cunningham BK, Sterner D. Dietary self-monitoring, but not dietary quality, improves with use of smartphone app technology in an 8-week weight loss trial. J Nutr Educ Behav 2014 Oct;46(5):440-444. [CrossRef] [Medline]
    30. Ali ZC, Silvioli R, Rajai A, Aslam TM. Feasibility of use of a mobile application for nutrition assessment pertinent to age-related macular degeneration (MANAGER2). Transl Vis Sci Technol 2017 Jan;6(1):4 [FREE Full text] [CrossRef] [Medline]
    31. Carter MC, Burley VJ, Nykjaer C, Cade JE. Adherence to a smartphone application for weight loss compared to website and paper diary: pilot randomized controlled trial. J Med Internet Res 2013;15(4):e32 [FREE Full text] [CrossRef] [Medline]
    32. Choi J, Chung C, Woo H. Diet-related mobile apps to promote healthy eating and proper nutrition: a content analysis and quality assessment. Int J Environ Res Public Health 2021 Mar 28;18(7):3496 [FREE Full text] [CrossRef] [Medline]
    33. Schumer H, Amadi C, Joshi A. Evaluating the dietary and nutritional apps in the Google Play Store. Healthc Inform Res 2018 Jan;24(1):38-45 [FREE Full text] [CrossRef] [Medline]
    34. Coughlin SS, Whitehead M, Sheats JQ, Mastromonico J, Hardy D, Smith SA. Smartphone applications for promoting healthy diet and nutrition: a literature review. Jacobs J Food Nutr 2015;2(3):021 [FREE Full text] [Medline]
    35. Maringer M, Wisse-Voorwinden N, Veer P, Geelen A. Food identification by barcode scanning in the Netherlands: a quality assessment of labelled food product databases underlying popular nutrition applications. Public Health Nutr 2019 May;22(7):1215-1222. [CrossRef] [Medline]
    36. Bzikowska-Jura A, Sobieraj P, Raciborski F. Low comparability of nutrition-related mobile apps against the Polish reference method-a validity study. Nutrients 2021 Aug 20;13(8):2868 [FREE Full text] [CrossRef] [Medline]
    37. Fallaize R, Zenun Franco R, Pasang J, Hwang F, Lovegrove JA. Popular nutrition-related mobile apps: an agreement assessment against a UK reference method. JMIR Mhealth Uhealth 2019 Feb 20;7(2):e9838 [FREE Full text] [CrossRef] [Medline]
    38. Tosi M, Radice D, Carioni G, Vecchiati T, Fiori F, Parpinel M, et al. Accuracy of applications to monitor food intake: evaluation by comparison with 3-d food diary. Nutrition 2021 Apr;84:111018. [CrossRef] [Medline]
    39. Patel ML, Hopkins CM, Brooks TL, Bennett GG. Comparing self-monitoring strategies for weight loss in a smartphone app: randomized controlled trial. JMIR Mhealth Uhealth 2019 Feb 28;7(2):e12209. [CrossRef] [Medline]
    40. Mangieri CW, Johnson RJ, Sweeney LB, Choi YU, Wood JC. Mobile health applications enhance weight loss efficacy following bariatric surgery. Obes Res Clin Pract 2019;13(2):176-179. [CrossRef] [Medline]
    41. Patel ML, Brooks TL, Bennett GG. Consistent self-monitoring in a commercial app-based intervention for weight loss: results from a randomized trial. J Behav Med 2020 Jun;43(3):391-401. [CrossRef] [Medline]
    42. Evenepoel C, Clevers E, Deroover L, Van Loo W, Matthys C, Verbeke K. Accuracy of nutrient calculations using the consumer-focused online app MyFitnessPal: validation study. J Med Internet Res 2020 Oct 21;22(10):e18237 [FREE Full text] [CrossRef] [Medline]
    43. Stehr P, Karnowski V, Rossmann C. The multi-faceted usage patterns of nutrition apps: a survey on the appropriation of nutrition apps among German-speaking users of MyFitnessPal. BMC Med Inform Decis Mak 2020 Oct 28;20(1):279 [FREE Full text] [CrossRef] [Medline]
    44. Downs DS, Savage JS, Rivera DE, Pauley AM, Leonard KS, Hohman EE, et al. Adaptive, behavioral intervention impact on weight gain, physical activity, energy intake, and motivational determinants: results of a feasibility trial in pregnant women with overweight/obesity. J Behav Med 2021 Oct;44(5):605-621. [CrossRef] [Medline]
    45. Bardus M, Borgi C, El-Harakeh M, Gherbal T, Kharroubi S, Fares E. Exploring the use of mobile and wearable technology among university student athletes in Lebanon: a cross-sectional study. Sensors (Basel) 2021 Jun 30;21(13):4472 [FREE Full text] [CrossRef] [Medline]
    46. Vasiloglou MF, Christodoulidis S, Reber E, Stathopoulou T, Lu Y, Stanga Z, et al. Nutrients 2020 Jul 24;12(8):2214 [FREE Full text] [CrossRef] [Medline]
    47. Ward R, Stevens C, Brentnall P, Briddon J. The attitudes of health care staff to information technology: a comprehensive review of the research literature. Health Info Libr J 2008 Jun;25(2):81-97 [FREE Full text] [CrossRef] [Medline]
    48. Karduck J, Chapman-Novakofski K. Results of the clinician apps survey, how clinicians working with patients with diabetes and obesity use mobile health apps. J Nutr Educ Behav 2018 Jan;50(1):62-69.e1. [CrossRef] [Medline]
    49. Fakih El Khoury C, Karavetian M, Halfens RJG, Crutzen R, Khoja L, Schols JMGA. The effects of dietary mobile apps on nutritional outcomes in adults with chronic diseases: a systematic review and meta-analysis. J Acad Nutr Diet 2019 Apr;119(4):626-651. [CrossRef] [Medline]
    50. Villinger K, Wahl DR, Boeing H, Schupp HT, Renner B. The effectiveness of app-based mobile interventions on nutrition behaviours and nutrition-related health outcomes: a systematic review and meta-analysis. Obes Rev 2019 Oct;20(10):1465-1484. [CrossRef] [Medline]
    51. Recio-Rodriguez JI, Agudo Conde C, Calvo-Aponte MJ, Gonzalez-Viejo N, Fernandez-Alonso C, Mendizabal-Gallastegui N, EVIDENT Investigators. The effectiveness of a smartphone application on modifying the intakes of macro and micronutrients in primary care: a randomized controlled trial. the EVIDENT II study. Nutrients 2018 Oct 10;10(10):1473 [FREE Full text] [CrossRef] [Medline]
    52. Talwar D, Yeh Y, Chen W, Chen L. Characteristics and quality of genetics and genomics mobile apps: a systematic review. Eur J Hum Genet 2019 Jun;27(6):833-840 [FREE Full text] [CrossRef] [Medline]
    53. Chen R, Santo K, Wong G, Sohn W, Spallek H, Chow C, et al. Mobile apps for dental caries prevention: systematic search and quality evaluation. JMIR Mhealth Uhealth 2021 Jan 13;9(1):e19958 [FREE Full text] [CrossRef] [Medline]
    54. Ciapponi A, Donato M, Gülmezoglu AM, Alconada T, Bardach A. Mobile apps for detecting falsified and substandard drugs: a systematic review. PLoS One 2021;16(2):e0246061 [FREE Full text] [CrossRef] [Medline]
    55. Mandracchia F, Llauradó E, Tarro L, Valls RM, Solà R. Mobile phone apps for food allergies or intolerances in app stores: systematic search and quality assessment using the mobile app rating scale (MARS). JMIR Mhealth Uhealth 2020 Sep 16;8(9):e18339 [FREE Full text] [CrossRef] [Medline]
    56. Romero RL, Kates F, Hart M, Ojeda A, Meirom I, Hardy S. Quality of deaf and hard-of-hearing mobile apps: evaluation using the mobile app rating scale (MARS) with additional criteria from a content expert. JMIR Mhealth Uhealth 2019 Oct 30;7(10):e14198 [FREE Full text] [CrossRef] [Medline]
    57. Escriche-Escuder A, De-Torres I, Roldán-Jiménez C, Martín-Martín J, Muro-Culebras A, González-Sánchez M, et al. Assessment of the quality of mobile applications (apps) for management of low back pain using the mobile app rating scale (MARS). Int J Environ Res Public Health 2020 Dec 09;17(24):9209 [FREE Full text] [CrossRef] [Medline]
    58. DiFilippo KN, Huang W, Chapman-Novakofski KM. A new tool for nutrition app quality evaluation (AQEL): development, validation, and reliability testing. JMIR Mhealth Uhealth 2017 Oct 27;5(10):e163 [FREE Full text] [CrossRef] [Medline]
    59. Li Y, Ding J, Wang Y, Tang C, Zhang P. Nutrition-related mobile apps in the china app store: assessment of functionality and quality. JMIR Mhealth Uhealth 2019 Jul 30;7(7):e13261 [FREE Full text] [CrossRef] [Medline]
    60. Braz VN, de Moraes Lopes MHB. Evaluation of mobile applications related to nutrition. Public Health Nutr 2019 May;22(7):1209-1214. [CrossRef] [Medline]

    ‎
    ICC: intraclass correlation
    MARS: Mobile Application Rating Scale
    MARS-F: Mobile Application Rating Scale–French

    Edited by L Buis; submitted 21.12.21; peer-reviewed by F Denis, H Mehdizadeh; comments to author 29.12.21; revised version received 01.01.22; accepted 27.01.22; published 14.03.22

    Copyright

    ©Prescilla Martinon, Ina Saliasi, Denis Bourgeois, Colette Smentek, Claude Dussart, Laurie Fraticelli, Florence Carrouel. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 14.03.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.