This paper is in the following e-collection/theme issue:

mHealth for Wellness, Behavior Change and Prevention (2922) Innovations and Technology for Healthy Eating Education (425) Food Sciences and Nutrition Engineering (54) Instruments and Questionnaires for Nutrition and Food Intake (169)

Published on in Vol 8, No 9 (2020): September

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/16953, first published .
Validity and Usability of a Smartphone Image-Based Dietary Assessment App Compared to 3-Day Food Diaries in Assessing Dietary Intake Among Canadian Adults: Randomized Controlled Trial

Validity and Usability of a Smartphone Image-Based Dietary Assessment App Compared to 3-Day Food Diaries in Assessing Dietary Intake Among Canadian Adults: Randomized Controlled Trial

Validity and Usability of a Smartphone Image-Based Dietary Assessment App Compared to 3-Day Food Diaries in Assessing Dietary Intake Among Canadian Adults: Randomized Controlled Trial

Authors of this article:

Yuwei Ji1 Author Orcid Image ;   Hugues Plourde1 Author Orcid Image ;   Valerie Bouzo1 Author Orcid Image ;   Robert D Kilgour2, 3 Author Orcid Image ;   Tamara R Cohen3, 4 Author Orcid Image
Article Authors Cited by (56) Tweetations (6) Metrics

    Original Paper

    1School of Human Nutrition, McGill University, Montreal, QC, Canada

    2Department of Health, Kinesiology, and Applied Physiology, Concordia University, Montreal, QC, Canada

    3PERFORM Centre, Concordia University, Montreal, QC, Canada

    4Faculty of Land and Food Systems, Food, Nutrition and Health, University of British Columbia, Vancouver, BC, Canada

    Corresponding Author:

    Tamara R Cohen, PhD, RD

    Faculty of Land and Food Systems

    Food, Nutrition and Health

    University of British Columbia

    218-2205 East Mall

    Vancouver, BC, V6T 1Z4

    Canada

    Phone: 1 604 827 0362

    Email: tamara.cohen@ubc.ca


    Background: Accurate dietary assessment is needed in studies that include analysis of nutritional intake. Image-based dietary assessment apps have gained in popularity for assessing diet, which may ease researcher and participant burden compared to traditional pen-to-paper methods. However, few studies report the validity of these apps for use in research. Keenoa is a smartphone image-based dietary assessment app that recognizes and identifies food items using artificial intelligence and permits real-time editing of food journals.

    Objective: This study aimed to assess the relative validity of an image-based dietary assessment app — Keenoa — against a 3-day food diary (3DFD) and to test its usability in a sample of healthy Canadian adults.

    Methods: We recruited 102 participants to complete two 3-day food records. For 2 weeks, on 2 non-consecutive days and 1 weekend day, in random order, participants completed a traditional pen-to-paper 3DFD and the Keenoa app. At the end of the study, participants completed the System Usability Scale. The nutrient analyses of the 3DFD and Keenoa data before (Keenoa-participant) and after they were reviewed by dietitians (Keenoa-dietitian) were analyzed using analysis of variance. Multiple tests, including the Pearson coefficient, cross-classification, kappa score, % difference, paired t test, and Bland-Altman test, were performed to analyze the validity of Keenoa (Keenoa-dietitian).

    Results: The study was completed by 72 subjects. Most variables were significantly different between Keenoa-participant and Keenoa-dietitian (P<.05) except for energy, protein, carbohydrates, fiber, vitamin B1, vitamin B12, vitamin C, vitamin D, and potassium. Significant differences in total energy, protein, carbohydrates, % fat, saturated fatty acids, iron, and potassium were found between the 3DFD and Keenoa-dietitian data (P<.05). The Pearson correlation coefficients between the Keenoa-dietitian and 3DFD ranged from .04 to .51. Differences between the mean intakes assessed by the 3DFD and Keenoa-dietitian were within 10% except for vitamin D (misclassification rate=33.8%). The majority of nutrients were within an acceptable range of agreement in the Bland-Altman analysis; no agreements were seen for total energy, protein, carbohydrates, fat (%), saturated fatty acids, iron, potassium, and sodium (P<.05). According to the System Usability Scale, 34.2% of the participants preferred using Keenoa, while 9.6% preferred the 3DFD.

    Conclusions: The Keenoa app provides acceptable relative validity for some nutrients compared to the 3DFD. However, the average intake of some nutrients, including energy, protein, carbohydrates, % fat, saturated fatty acids, and iron, differed from the average obtained using the 3DFD. These findings highlight the importance of verifying data entries of participants before proceeding with nutrient analysis. Overall, Keenoa showed better validity at the group level than the individual level, suggesting it can be used when focusing on the dietary intake of the general population. Further research is recommended with larger sample sizes and objective dietary assessment approaches.

    JMIR Mhealth Uhealth 2020;8(9):e16953

    doi:10.2196/16953

    Keywords

    mobile food record; validity; image-based dietary assessment; healthy adults; 3-day food diary; diet; application; nutrition; mHealth; Canada 


    Assessment of dietary intake, in particular habitual dietary intake, remains a major challenge for researchers [1]. Limitations of dietary assessment have been well documented and vary by the method chosen. Research typically relies on traditional pen-and-paper methods to assess dietary intake: the 3-day food diary (3DFD), 24-h recall, or food frequency questionnaire (FFQ) [2]. However, researchers face many challenges when deciding which method is best [2]. Issues of participant burden, motivation and willingness to accurately report diet, and participant literacy and memory should be considered. Moreover, time to enter and analyze diet data, and therefore the availability of resources to correctly analyze dietary recalls, should be accounted for before commencing a study.

    All methods of dietary assessment have their limitations [3]. For instance, a single 24-h recall only reflects the food consumed from a single random day and may be less representative of an estimated individual’s intake. However, two or more 24-h recalls or food records are needed to estimate usual dietary intake distributions [4]. A limitation of food records is that they can lead to reactivity bias [5]. FFQs may lead to over reporting of average dietary intakes and, similar to 24-h recalls, rely on the participant’s ability to correctly recall portion sizes and frequencies [5]. Further, the FFQ can be lengthy; therefore, time and ability to remain focused can be demanding for participants [6]. For these reasons, new methods of dietary assessment are needed to benefit participants and researchers.

    With the development of technology, image-based dietary records have increased in popularity. There are different technology-based methods of tracking diet, including taking pictures [7] or using digital food databases [2,7,8]. Relevant to this study, mobile photo meal apps engage the user to take a picture of their food item(s) using their mobile device before it is consumed and require the user to input the quantity and other pertinent details, as needed. Although mobile phone–based methods still rely on user input, an advantage of this method is that food entries are time-stamped, which identifies when food items were consumed [9]. The use of dietary assessment applications, such as those that use a mobile phone app, has shown an increase in participant satisfaction and preference compared with conventional methods (eg, 24-h recall, written food diary) [2]. Moreover, using mobile devices has the potential to reduce costs and diet data entry errors by researchers [7,10]. However, assessment of the validity of these apps against traditional methods of dietary assessment is needed to ensure participant satisfaction and therefore compliance to recording intakes.

    Keenoa (Montreal, Quebec, Canada) is a smartphone imaged–based dietary assessment app that recognizes and identifies food items using artificial intelligence and permits editing of food journals in real time. Unique to other apps, Keenoa is accessed only by registered dietitians licensed to practice in Canada with the idea that dietitians are trained to identify food items that were missed or misidentified by the user. Therefore, the advantage of Keenoa is that dietitians can adjust the food items to generate accurate nutrient profiles of an individual’s dietary intake. Currently, the app is being used by practicing dietitians in Canada. From a researcher’s perspective, using Keenoa to assess dietary intakes would reduce systematic errors associated with data entry [5]. It is currently unknown if the app is appropriate for use in research.

    This study aimed to assess the relative validity of Keenoa against the 3DFD and to test its usability in a sample of healthy Canadian adults.


    Participants and Recruitment

    From February to April 2019, we recruited 102 participants at the PERFORM Centre (Concordia University, Montreal, Quebec, Canada). Inclusion criteria included adults (>18 years of age) who owned a smartphone (Android or Apple) and would be able to download the Keenoa app without assistance. Exclusion criteria included individuals with a previously diagnosed disease affecting their dietary intake (ie, Type 1 or Type 2 diabetes, renal disease, inflammatory or immunity disorders); currently following a diet or weight-loss regime; who, as an adult, suffered from, had a history of, or was being treated for an eating disorder; or who have completed or are in the process of completing a nutrition or related degree (eg, a dietetics major). Finally, individuals who were not French or English speaking and who could not understand written English were also excluded. Compensation for completing the study included a detailed dietary assessment by a registered dietitian that was emailed to them. This study was approved by the University Human Research Ethics Committee of Concordia University, and written consent was obtained from all participants during their first visit at the PERFORM Centre.

    Procedure

    Each participant met with a trained researcher (VB, TC) at the PERFORM Centre. Participants completed a brief sociodemographic questionnaire with questions related to total family income, ethnicity, highest level of education, age, and sex. Body weight and height were measured using a balance scale and a wall-mounted height rod, respectively. BMI was calculated using the measured weight and height (kg/m²).

    Using a computer-generated list, participants were randomly assigned to start with either the Keenoa application or 3DFD for the first week and then switch methods of dietary assessment for the second week. Both 3DFD and Keenoa recordings needed to include 2 non-consecutive weekdays and 1 weekend day. Participants were highly encouraged to maintain their typical eating habits throughout the study.

    Participants were provided a calendar to help them keep track of the specific days they should record their diet using the respective methods. A trained researcher also reviewed how to estimate portion sizes using the Dietitian’s of Canada Handy Guide to Servings Sizes [11].

    3-Day Food Diary (3DFD)

    All participants were provided a hard-copy, single-page printout of the standard 3DFD to record their diet. On the 3DFD, there were prompts for participants to record and estimate the details of all meals and snacks they consumed including portion size, cooking methods, and any add-ons (eg, cream, oil, butter, jam). If they had meals in restaurants, participants were told that the restaurant name and dishes needed to be recorded in detail as well. Participants were given the option to either mail the 3DFD with a prestamped posted envelope directed to the corresponding author or scan and email their 3DFD to the study email. All 3DFD were coded for confidentially; those emailed were immediately printed, and the email was deleted to maintain confidentially.

    Keenoa App

    Instructions on how to use the app were provided to the participant, and online tutorials were available from the company. In the case of this study, the researcher used the participant’s email to send an invitation to download the app on their smartphone, which, once downloaded, automatically connected the user to the study dietitian. Participants were asked to take pictures of the food items before they were consumed using their smartphone. Items could range from a single item (eg, an apple, French fries, a cup of coffee) to composite items (eg, a serving of lasagna, bowl of soup, or slice of pizza). If the app recognized the food item(s), it would display options for the users to choose from to correctly identify the food(s). Otherwise, participants could search and record the foods manually from a database that was linked to the Canadian Nutrient File (2015) [12]. The Canadian Nutrient File is a bilingual (English and French) food composition database that is managed by the Government of Canada that includes foods available only on the Canadian market.

    Once the food item was identified, the app then prompted participants to estimate and enter the serving size. Using visual aids (ie, a tennis ball, picture of a measuring cup), the participant scrolled to the correct unit (ie, unit count, volume, or weight) and identified a number corresponding to the unit. If a participant did not consume the entire food item, they had the option to record the information in text in the app. If a participant forgot to take a picture prior to consuming a food item or meal, they also had the option to manually enter foods and text in the app.

    Once a meal was complete, the user finalized the day, at which point the image and corresponding information were immediately uploaded to the research dietitians’ private page. Food items’ nutrient values were automatically computed using nutrient information from the Canadian Nutrient File (2015) database. Once participants completed their 3 days using Keenoa, the nutrient analyses were exported to Excel and coded to maintain confidentiality.

    Nutrient Analyses

    All the 3DFD data were reviewed and recorded in the Food Processor software (ESHA Research version 11.1, Oak Brook, IL) by a trained researcher. The 3-day average nutrient intake content for each subject was computed by the software for total energy (kcal), fat (g), protein (g), carbohydrate (g), saturated fatty acids (g), cholesterol (g), dietary fiber (g), and micronutrients, including vitamin A (μg), vitamin B1 (mg), vitamin B2 (mg), vitamin B12 (μg), vitamin C (mg), vitamin D (μg), calcium (mg), iron (mg), magnesium (mg), phosphorus (mg), potassium (mg), and sodium (mg). All foods were entered into the software using the Canadian database, which uses the Canadian Nutrient File (2015) dataset [12]. Data were exported to Excel for statistical analysis.

    The food records from Keenoa were also exported exactly as they were entered by the participant (Keenoa-participant). A research dietitian then reviewed each food record for missing items or misrecorded portion sizes as per the images (Keenoa-dietitian) and exported these data for analysis to Excel. The nutrition assessment and recommendations were sent to the participants by the research dietitian after the reports were corrected and reviewed by the dietitian. Similar to the Food Processor software (ESHA) and as mentioned, Keenoa also uses the Canadian Nutrient File (2015) database for diet analysis; the same nutrients were exported as for the 3DFD analysis.

    Exit Survey

    After completing both methods of dietary assessment, the participants were sent a link to complete an online survey. This English survey included the System Usability Scale (SUS) questionnaire [13] with 3 additional questions related to using the 3DFD method and Keenoa app. The SUS has been used in previous research to examine the user’s perspective of a mobile app [14,15]. This questionnaire includes 10 items and uses a 5-item rating scale. Specifically, this questionnaire surveys participants on different aspects of the Keenoa app (eg, adoption and complexity).

    Statistical Analyses

    The final analyses were restricted to those who completed both methods of diet recall (n=72), thereby excluding participants who withdrew (n=30). The characteristics of participants, as well as the results of the survey, are presented as percentages, means, and SD. Chi-square tests and Student’s t tests were performed to identify the differences in demographic and socioeconomic characteristics between participants who completed the study and those who dropped out. The mean and SD of the nutrition intake of the Keenoa-participant data, Keenoa-dietitian data, 3DFD data, and percentage of energy present in the macronutrients (protein, carbohydrates, and fat) were calculated.

    Repeated analyses of variance (ANOVA) with post hoc tests were used to compare the differences in nutrient consumption among the 3 groups. Percentage differences (% difference) between Keenoa-participant versus Keenoa-dietitian and Keenoa-dietitian versus 3DFD were also calculated.

    As the purpose of using the Keenoa app is to analyze participants’ or clients’ dietary intake after adjustment by registered dietitians, 6 cross-classification analyses were performed (ie, Pearson coefficient, cross-classification, kappa score, % difference, t test, Bland-Altman test) to assess the validity of the Keenoa app (Keenoa-dietitian). Weighted Cohen kappa and cross-classification tests were performed to evaluate the interrater agreement between the diet data from Keenoa-dietitian and 3DFD. This was analyzed by calculating the chance of misclassification between the 2 methods (eg, a participant being classified in the first quartile by 3DFD but classified in the fourth quartile by Keenoa or vice versa). Pearson correlations and Bland-Altman tests were also used to test for associations between Keenoa-dietitian and 3DFD. Validity assessments were performed as suggested by Lombard and colleagues [16], which combines the results of the 6 tests mentioned. The result of each test was classified as “good,” “acceptable,” or “poor,” and the total number of poor outcomes was calculated. All analyses were performed in SPSS version 23 (IBM Inc, Armonk, NY) with P values <.05 considered statistically significant.


    We recruited 102 participants in this study; 6 individuals did not attend the baseline visit, while 21 participants did not complete the study protocol as directed (ie, using one of two methods on 3 non-consecutive days to record diet, did not complete full days of recording food items on Keenoa, or did not return their 3DFD as instructed.) Both 3DFD and Keenoa diet reports were completed per the study protocol by 75 participants. Due to outlying diet data that could not be edited by the system, 3 participants were excluded from the analysis. This study reports on women (n=47) and men (n=25) with a combined mean age of 38.5 years and a mean BMI of 27.0 kg/m2 (SD 4.9 kg/m2; Table 1). Most participants (80.6%) held a university degree, and 62.5% of the participants had a family income of more than Can $30,000 per year. The sample population was 56.9% Caucasian, 19.4% Asian, and 23.6% other ethnicities, including Arab, African American, and Latin American. Only 2 participants (2.8%) reported having a vegan or vegetarian diet. There was no significant difference in characteristics in those who completed the study and those who withdrew.

    Table 1. Participant demographic characteristics (n=72).
    CharacteristicsValues
    Age (years), mean (SD)38.5 (16.7)
    Age (years), n (%)

    		18-3031 (43.1)

    		31-5018 (25.0)

    		51-6517 (23.6)

    		≥656 (8.3)
    BMI (kg/m2), mean (SD)27.0 (4.9)
    BMI (kg/m2), n (%)

    		<2524 (33.3)

    		25 to <3031 (43.1)

    		≥3017 (23.6)
    Sex, n (%)

    		Female47 (65.3)

    		Male25 (34.7)
    Education, n (%)

    		College and below12 (16.7)

    		University58 (80.6)

    		Refused to answer2 (2.8)
    Familyincome (Can $), n (%)

    		<30 00014 (19.4)

    		≥30 00045 (62.5)

    		Refused to answer13 (18.1)
    Ethnicity, n (%)

    		White41 (56.9)

    		Asian14 (19.4)

    		Other17 (23.6)
    Vegetarian or vegan diet, n (%)

    		Yes2 (2.8)

    		No70 (97.2)

    The differences between Keenoa-participant, Keenoa-dietitian, and 3DFD data are presented in Table 2. The percentage mean intakes of protein (P=.001), carbohydrates (P<.001), and fat (P<.001) were significantly different between Keenoa-participant and Keenoa-dietitian data as were grams of fat (P<.001), saturated fatty acids (P<.001), cholesterol (P<.001), vitamin A (P<.001), vitamin B2 (P<.001), magnesium (P=.009), phosphorus (P<.001), iron (P<.001), and sodium (P<.001). The majority of nutrients from Keenoa-participant were under-recorded compared with Keenoa-dietitian, excluding % protein, carbohydrates, % carbohydrates, and calcium. Vitamin A showed the highest percentage difference (134.2%, data not shown), and the lowest percentage difference was observed for potassium (0.7%, data not shown).

    Table 2. Differences between nutrients from Keenoa-participant, Keenoa-dietitian, and 3-day food diary (3DFD; n=72).
    NutrientsKeenoa-participant, mean (SD)Keenoa-dietitian, mean (SD)3DFD, mean (SD)P value
    Energy (kcal)1615.3 (1664.4)1693.0 (593.2)a2006.3 (540.5).000
    Protein (g)65.0 (40.7)68.8 (24.9)a85.6 (26.0)b.000
    % Protein18.0 (5.4)c16.5 (3.6)17.4 (4.2).001
    Carbohydrate (g)225.5 (372.5)181.8 (65.1)a224.6 (71.8).000
    % Carbohydrate50.1 (12.6)c43.5 (7.7)45 (8.8)b.000
    Fat (g)52.6 (29.5)c77.7 (32.6)84.9 (28.9)b.000
    % Fat33.5 (9.0)c37.7 (7.4)a40.8 (7.5)b.000
    Saturated fatty acids (g)17.5 (11.9)c23.2 (10.6)a27.7 (10.6).000
    Cholesterol (g)242.0 (190.8)c283.8 (192.1)328.3 (185.7)b.000
    Dietary fiber (g)20.1 (16.6)20.6 (8.6)22.2 (7.8).323
    Vitamin A (μg)216.5 (163.8)c260.6 (142.0)270.0 (170.0).001
    Vitamin B1 (mg)1.1 (0.6)1.2 (0.5)1.3 (0.6).084
    Vitamin B2 (mg)1.4 (0.7)c1.6 (0.7)1.7 (0.5)b.000
    Vitamin B12 (μg)2.8 (2.1)2.9 (1.4)3.8 (2.6)b.030
    Vitamin C (mg)244.1 (901.3)98.3 (62.4)112.6 (62.7).125
    Vitamin D (μg)3.1 (2.8)3.2 (2.0)3.5 (3.0).579
    Calcium (mg)792.7 (1304.3)691.8 (304.2)889.4 (966.6).174
    Iron (mg)10.2 (5.0)c11.6 (4.4)a13.3 (4.1)b.000
    Magnesium (mg)253.4 (146.0)c278.9 (133.4)319.2 (328.5).025
    Phosphorus (mg)902.1 (422.0)c1023.6 (357.5)1108.2 (370.0)b.000
    Potassium (mg)2402.6 (2162.7)2391.2 (910.7)2553.1 (766.2).560
    Sodium (mg)1729.0 (1059.2)c2333.9 (1203.1)2969.0 (1621.4)b.000

    aPost-hoc tests between Keenoa-dietitian and 3DFD with P<.05.

    bPost-hoc tests between Keenoa-participant and 3DFD with P<.05.

    cPost-hoc tests between Keenoa-participant and Keenoa-dietitian data with P<.05.

    The nutrient intake between Keenoa-dietitian and 3DFD were significantly different for mean intakes of energy (P<.001), protein (P<.001), carbohydrates (P=.001), % fat (P=.041), saturated fatty acids (P<.001), iron (P=.045), and sodium (P<.001), with fat and cholesterol showing a statistical trend (P=.069 and P=.052, respectively).

    Results of the validity analysis of the Keenoa app (Keenoa-dietitian) are summarized in Table 3 and are based on the classification method by Lombard et al [16]. The highest numbers were observed for sodium and vitamin D (n=5), and the lowest numbers were observed for vitamin B1 and phosphorous (n=1). Pearson coefficient coefficients between Keenoa-dietitian and 3DFD ranged from .38 to .51 for macronutrients and .42 to .47 for micronutrients.

    Table 3. Validity analysis of the Keenoa application (Keenoa-dietitian), based on criteria levels for good (G), acceptable (A), and poor (P) outcomes.
    NutrientsIndividual levelGroup levelTotal number of poor outcomes
    AssociationAgreementAgreementPresence of bias
    Pearson coefficientaCross-classificationbKappa scorec% differencedt testeBland-Altmanf
    Energy (kcal)AP-GPAPP4
    Protein (g)GP-GPAPP4
    % proteinAP-GPGGG2
    Carbohydrate (g)AP-GPAPP4
    % carbohydrateAP-GPGGG2
    Fat (g)AP-GPGGG2
    % fatAP-GPGPP4
    SFAg (g)AP-GPGPP4
    Cholesterol (g)AP-GPGGG2
    Dietary fiber (g)AP-GAGGG1
    Vitamin A (μg)AP-GAPGP3
    Vitamin B1 (mg)AP-GAGGG1
    Vitamin B2 (mg)AP-GPGGG2
    Vitamin B12 (μg)PP-GPPGG4
    Vitamin C (mg)PP-GPPGG4
    Vitamin D (μg)PP-PPPGG5
    Calcium (mg)PP-GPGGG3
    Iron (mg)AP-GPGPP4
    Magnesium (mg)PP-GAGGG2
    Phosphorus (mg)AP-GAGGG1
    Potassium (mg)PP-GAGGG2
    Sodium (mg)PP-GPGPP5
    Total number of poor outcomes722 (P), 1 (G)1647965
    AverageN/AhN/AN/AN/AN/AN/A2.95

    aGood, r>.05; acceptable, r=.20-.49; poor, r<.20.

    bGood, ≥50% in the same quartile and <10% in the opposite quartile; poor, <50% in the same quartile and ≥10% in the opposite quartile.

    cGood, ≥0.61; acceptable, 0.20-0.60; poor, <0.20.

    dGood, 0%-10.9%; acceptable, 11.0%-20.0%; poor, >20.0%.

    eGood, P value <.05; poor, P value ≤.05.

    fGood, P value <.05; poor, P value ≤.05.

    gSFA: saturated fatty acids.

    hN/A: not applicable.

    Cross-classification results revealed that the chance of misclassification was <10% for all nutrients, except for vitamin D (misclassification rate=33.8%). Weighted kappa scores ranged from 0.000 to 0.585, with an average of 0.143. Bland-Altman plots were used to compare the differences between mean intake of each nutrient between Keenoa-dietitian and 3DFD. The results showed that the majority of nutrients were in an acceptable range of agreement, with the exception of energy (P<.001), protein (P<.001), carbohydrates (P<.001), % fat (P<.001), saturated fatty acids (P=.001), vitamin A (P<.001), iron (P=.03), and sodium (P=.005).

    Finally, results from the SUS showed that the mean overall score was 61.6 points (SD 19.1 points). Data were divided and analyzed by positive (questions with odd numbers) and negative (questions with even numbers) statements. The positive statement responses ranged from “neutral” to “agree,” while the negative statement responses ranged from “disagree” to “neutral.” The majority of participants believed that Keenoa was easy to use (38/72, 52.8%) and reported they did not need the assistance of a technical person (54/72, 75.0%). There was no significant difference in acceptance between Keenoa and 3DFD (P=.28). However, 34.7% (25/72) of participants said that they would like to use Keenoa to track their diets, compared with only 9.7% (7/72) stating that they want to keep using 3DFD; 16.7% (12/72) would use both methods again, 27.8% (20/72) were not sure, and 11.1% (8/72) stated they would not use either method.


    Principal Findings

    This study assessed the use of an image-based app for assessment of dietary intake in healthy adults. Similar to other studies [8,16-20], the results from Keenoa-dietitian produced similar mean nutrient profiles for more than half of nutrients when compared to the 3DFD method. However, this study included relative validation; therefore, it is impossible to conclude that one method is closer to “true dietary intake” than the other, as true dietary intake is not known.

    In this study, we found significant differences between Keenoa-participant and Keenoa-dietitian data for 12 of the 22 nutrients analyzed. This suggests that the adjustments made by the dietitian were necessary to obtain the most accurate assessment of the participant’s diet. Specifically, this study found that participant reports of dietary fat and protein (% difference: +31.9% and +6.9%, respectively) were lower, while the report of carbohydrates was higher (% difference: –24.7%), compared to the edited version by the dietitian. These results suggest that Keenoa is appropriate for dietitians in clinical settings; however, dietitians should review the food entries prior to generating final reports.

    Despite the advantages of image-based diet-tracking apps, food identification from the user remains a challenge unless they are highly motivated to capture all food item details. An example of this lies in proper estimation of percentages of milk fat found in dairy products that is impossible to estimate from an image unless a picture of the milk carton is taken and recorded. Items that are not easily identifiable, such as milk fat from fluid milk, become problematic if diets are high in dairy-containing sauces or are included in sandwiches and other mixed-pasta dishes such as ravioli [19-22].

    We speculate this to be the case in our study, as suggested by the lower reports of dietary fat and protein. Previous studies have tried to overcome this issue by inviting participants to review their image-based food diaries with a trained researcher [23,24]. However, these studies had significantly smaller sample sizes (ie, n=40 [23] and n=20 [24]) compared to our study’s baseline participant pool (n=102), for which the time and resources needed to do these types of reviews were limited.

    Compared with 3DFD, the energy and nutrient intakes reported via the Keenoa app (Keenoa-dietitian) were all low (average % difference, 22.5%). The highest % difference was observed for vitamin A (222%). When we excluded vitamin A from the analysis, the average adjustment decreased to 13%. These results are similar to that of other studies. In pregnant women (n=60), Savard et al [20] found an average 12.2% difference when comparing 3DFD with another web-based dietary assessment tool (24-h recall for 3 days). Similar to their study, we also compared a traditional dietary record method to a new app-based method; therefore, a relatively higher % difference is acceptable [16]. Nevertheless, in this study, the majority of the tests (4/7) scored poorly on 10 of the 22 nutrients when comparing the 3DFD and Keenoa-dietitian, including energy, protein, carbohydrates, % fat, saturated fatty acids, and iron. To our knowledge, only a few research groups have performed such an in-depth analysis as seen with the Keenoa-dietitian data, and although our findings cannot speak to reliability, the average number of poor outcomes is similar but slightly higher than those found by Savard et al [20] and Lafrenière et al [22].

    It is suggested that a good Pearson coefficient should equal or surpass .5 [25]. Analysis in our study showed a weak association between 3DFD and Keenoa-dietitian since all values were <.5, except for protein. However, a relatively better association of energy and macronutrients between the two methods was observed with coefficient values closer to .5. Similarly, others have shown correlation coefficients between the 3DFD and web-based 24-h recall ranging from .03 to .76 [20]. A comparable trend was found in another study comparing a 4-day food record to two web-based 24-h recalls among 93 university-affiliated adults; the correlation coefficients varied between .06 and .76 [26]. Conversely, Lafrenière et al [22] observed a positive relative validity outcome with a mean adjusted correlation of .52 in their web-based 24-h dietary recall validation study. Notably, this research required participants to weigh their food and provide food labels or recipes, which may promote more accurate results. It also has been argued that a larger sample size could lead to a weaker correlation [19], since a good correlation (r=.46 to r=.93) was found by Wang et al [27] when studying a sample of 20 participants.

    In order to assess the validity of Keenoa, the total number of poor outcomes based on 7 methods for each nutrient was counted. Among the 22 variables, only vitamin D and sodium had 5 poor outcomes, reflecting poor validity. Specifically, the vitamin D findings are similar to those found by others [20], which may be due to the fact that the majority of the Canadian population do not consume high volumes of vitamin D–enriched foods such as fatty fish and fortified dairy products daily [28]. In this study, vitamin D was the only nutrient with a higher rate of misclassification in the cross-classification analysis. By contrast, the higher SD of sodium intake indicates significant variability in average sodium intake, which could contribute to its considerable number of poor outcomes. This is understandable since sodium intake could vary from day to day. The average weighted kappa score was 0.143, representing slightly higher agreement and reliability between the two assessment tools at the individual level. This average is similar to that found by Landis and Koch [29] but is lower in comparison to the findings of Savard et al [20] and Lafrenière et al [22] who obtained average weighted kappa scores of 0.32 and 0.33, respectively. Overall, the outcomes at the group level (paired t tests and Bland-Altman tests) were better than those at the individual level (Pearson coefficient, cross-classification, kappa score, and % difference). These findings are supported by those of Savard et al [20] and Lafrenière et al [22] who used the same validation tests. Therefore, the validity at the group level was stronger than at the individual level, implying that the Keenoa app is more robust at analyzing group level nutrient intakes or the general population’s dietary intake. However, it should be kept in mind that bias could still exist, and the data should be interpreted carefully [16].

    In this study, the average SUS score was 61.6 points, suggesting the Keenoa app “was generally well accepted; however, system users experienced usability issues” [14,30]. This is in agreement with 53.4% of participants stating that the Keenoa app was easy to use. Our results indicated that participants felt Keenoa was easy to understand and would be accepted by the general population; however, inconsistencies were found in the app, and not all participants were willing to continue using Keenoa. Indeed, compared to a written food diary, taking photos is less resource intensive, which eases the burden on participants. In line with other studies [19,31], more participants preferred using the Keenoa app over the traditional pen-and-paper method (ie, 3DFD).

    Image-based dietary assessment apps, especially the Keenoa app in this study, can simplify the diet assessment process in multiple ways. Most importantly, it significantly decreases the burden on both researchers and participants by easing much of the data entry process and therefore reduces the possibility of errors [7]. In addition, because the app can be accessed remotely, it benefits people who eat out and do not take a hard copy of the food record with them [32]. Another major strength of this app is that it is linked to the Canadian Nutrient File (2015) database, which significantly reduces the data entry workload for researchers. However, some errors were still found in both Keenoa and 3DFD exported data when generating the report, which we treated as outlier data and therefore excluded those participants from this study. Although it is common to find mistakes with a new app that are often resolved with time, data should still be audited carefully in order to ensure a valid output [33]. Canada’s population consists of people with different cultural backgrounds; hence, it is hard to include every food in the database. Also, estimation error is always a challenge for image-based assessment [17]; underestimation or overestimation has frequently been observed in multiple studies. Williamson et al [34] reported that estimation error was significantly decreased by employing three analysts; however, time and budget costs are a concern with this solution. Computer-aided estimation of portion sizes is an alternative direction. In the study by Fang et al [35], a machine was able to assess the portion size with a minimum range of error. Thus, estimation error may possibly be eliminated by more capable machine learning in the future.

    “Technology generation” has always been a concern with the coming of new technologies [36]. Studies with younger populations were observed to have better overall outcomes and higher usability scores compared with our study [8,17,19,31,33,37]. Younger adults are more adaptable to new technologies, while elderly users are considered to have lower adoption speeds [38-40]. However, there was no significant difference found when age was included as a covariate in our research. Thus, future research is needed to expand the validity of image-based dietary assessment to all age groups, especially the elderly.

    Limitations

    Compared to other similar research [8,17,20,22], this study has relatively strong generalizability, since it included adults of different age groups, ethnicities, and both sexes. The diet data entry and analysis were all conducted by dietitians. However, this research has some limitations. First, our research has a relatively high dropout rate (27/102, 26.5%). One possible reason is that all participants only received a dietary report at the end of the study as compensation, which may have led to reduced willingness to cooperate. Future work may wish to assess self-efficacy or motivation to track dietary intake prior to commencing a study of a similar nature. However, the majority of participants who withdrew did not complete both methods, and the percentage of participants who failed to complete one of the two methods (ie, either 3DFD or Keenoa) was similar. This means that the dropout rate was more likely due to time or interest and was not related to the use of the Keenoa app itself. Second, the majority (58/72, 80.6%) of participants who completed the study held at least a university degree, which may have impacted their proficiency in using the mobile app [41]. In addition, one may argue that using the same 3-day dietary record for both methods would give a better estimation of validity. However, such an approach would lead to a higher workload for participants, and therefore, increase the drop-out rate and worsen compliance. Besides, nutrient intakes were reported by the participants in both methods. The use of unbiased reference measures, such as nutrition biomarkers or feeding studies, would have been the preferred method of validation against Keenoa; however, these are expensive. The possibility of systematic bias should be addressed when analyzing dietary intake based on images, and although 7 methods were used to assess the validity of Keenoa, we cannot eliminate the potential bias when interpreting the data. Finally, portion sizes of mixed items and sauces were challenging to estimate based on images, which might have contributed to the possibility of underestimating or overestimating the nutrient intake [19].

    Conclusion

    This study assessed the relative validity of Keenoa, an image-based mobile app, against a 3DFD in healthy adults. Our results suggest that the Keenoa app has the potential to provide accurate dietary assessment information to dietitians in a more cost-efficient and time-efficient way. Furthermore, it was well-accepted by users compared with traditional methods. However, the prediction for energy, protein, carbohydrates, % fat, saturated fatty acids, and iron intake remains questionable and should be interpreted with caution. Compared to 3DFDs, Keenoa resulted in better validity at the group level than at the individual level; thereby, it may be more effective in analyzing the dietary intake of the general population. However, a study with greater representation of older adults is needed. While participants found Keenoa easier to use compared to 3DFD, further research is needed in understanding how the app can be improved from the user perspective.

    Acknowledgments

    This study was funded by the R. Howard Webster Foundation through the PERFORM Centre, Concordia University (Montreal, Quebec, Canada). The authors would like to thank Dr. Stephanie Chevalier, PhD, RD for her assistance with the study design, Rainer Molla for his assistance with the study visits, and Sara Sorrini, RD for her work on the data entry and dietary analysis. The authors would also like to acknowledge Anne-Julie Tessier, RD and Anthony Grant for their assistance with the Keenoa® app.

    Authors' Contributions

    TC, HP, and RK conceptualized the overall study idea. TC and HP created the research questions and study design. TC and VB conducted the study and data collection. YJ analyzed the data, conducted the data analyses, and prepared the manuscript. All authors have read, edited, and approved the final manuscript.

    Conflicts of Interest

    None declared.

    This randomized study was not registered, as the authors specified that it did not meet the criteria deeming it necessary to formally register the study as a clinical trial based on the ClinicalTrials.gov checklist. The editor granted an exception of ICMJE rules for prospective registration of randomized trials because the risk of bias appears low. However, readers are advised to carefully assess the validity of any potential explicit or implicit claims related to their primary outcomes or effectiveness, as the lack of registration means that authors could change their outcome measures retrospectively.

    ‎
    Multimedia Appendix 1

    CONSORT-eHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 2193 KB
    1. Bingham SA, Gill C, Welch A, Cassidy A, Runswick SA, Oakes S, et al. Validation of dietary assessment methods in the UK arm of EPIC using weighed records, and 24-hour urinary nitrogen and potassium and serum vitamin C and carotenoids as biomarkers. Int J Epidemiol 1997;26 Suppl 1:S137-S151. [CrossRef] [Medline]
    2. Sharp DB, Allman-Farinelli M. Feasibility and validity of mobile phones to assess dietary intake. Nutrition 2014 Nov;30(11-12):1257-1266. [CrossRef] [Medline]
    3. Thompson FE, Kirkpatrick SI, Subar AF, Reedy J, Schap TE, Wilson MM, et al. The National Cancer Institute's Dietary Assessment Primer: A Resource for Diet Research. J Acad Nutr Diet 2015 Dec;115(12):1986-1995 [FREE Full text] [CrossRef] [Medline]
    4. Subar A, Freedman L, Tooze J, Kirkpatrick SI, Boushey C, Neuhouser ML, et al. Addressing Current Criticism Regarding the Value of Self-Report Dietary Data. J Nutr 2015 Dec;145(12):2639-2645 [FREE Full text] [CrossRef] [Medline]
    5. Kirkpatrick SI, Vanderlee L, Raffoul A, Stapleton J, Csizmadi I, Boucher BA, et al. Self-Report Dietary Assessment Tools Used in Canadian Research: A Scoping Review. Adv Nutr 2017 Mar 15;8(2):276-289 [FREE Full text] [CrossRef] [Medline]
    6. Institute of Medicine. Dietary Risk Assessment in the WIC Program. In: Food-Based Assessment of Dietary Intake. Washington, DC: National Academies Press; 2002:1-184.
    7. Illner A, Freisling H, Boeing H, Huybrechts I, Crispim S, Slimani N. Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. Int J Epidemiol 2012;41(4):1187-1203. [CrossRef]
    8. Chen Y, Wong J, Ayob A, Othman N, Poh B. Can Malaysian Young Adults Report Dietary Intake Using a Food Diary Mobile Application? A Pilot Study on Acceptability and Compliance. Nutrients 2017 Jan 13;9(1):62. [CrossRef]
    9. Turner-McGrievy GM, Dunn CG, Wilcox S, Boutté AK, Hutto B, Hoover A, et al. Defining Adherence to Mobile Dietary Self-Monitoring and Assessing Tracking Over Time: Tracking at Least Two Eating Occasions per Day Is Best Marker of Adherence within Two Different Mobile Health Randomized Weight Loss Interventions. Journal of the Academy of Nutrition and Dietetics 2019 Sep;119(9):1516-1524. [CrossRef]
    10. Raatz SK, Scheett AJ, Johnson LK, Jahns L. Validity of electronic diet recording nutrient estimates compared to dietitian analysis of diet records: randomized controlled trial. J Med Internet Res 2015 Jan 20;17(1):e21 [FREE Full text] [CrossRef] [Medline]
    11. Dietitians of Canada. Handy Serving Guide. Unlock Foods.: Dietitians of Canada   URL: https:/​/www.​unlockfood.ca/​EatRightOntario/​media/​PDFs-new-website/​Portions%20Toolkit/​Handy-Servings-Guide-EN-v04-July-2018.​pdf [accessed 2020-04-10]
    12. Health Canada. Canadian Nutrient File, 2015 version. Published February 6. 2018.   URL: https://food-nutrition.canada.ca/cnf-fce/index-eng.jsp [accessed 2020-04-10]
    13. Brooke J. SUS - A quick and dirty usability scale. In: Jordan PW, Thomas B, McClelland IL, Weerdmeester B, editors. Usability evaluation in industry. Great Britain, UK: CRC Press; Jun 11, 1996:1-252.
    14. Benyamin S, Rutter M, Smith S. The utilization of system usability scale in learning management systems: A case study of Jeddah Cummunity College. Poster presented at the 9th International Conference of Education, Research and Innovation (ICERI2016) 2016 Nov [FREE Full text] [CrossRef]
    15. Kaya A, Ozturk R, Altin GC. Usability Measurement of Mobile Applications with System Usability Scale (SUS). In: Industrial Engineering in the Big Data Era. Switzerland AG: Springer International Publishing; 2019:389-400.
    16. Lombard MJ, Steyn NP, Charlton KE, Senekal M. Application and interpretation of multiple statistical tests to evaluate validity of dietary intake assessment methods. Nutr J 2015 Apr 22;14(1):40 [FREE Full text] [CrossRef] [Medline]
    17. Howes E, Boushey C, Kerr D, Tomayko E, Cluskey M. Image-Based Dietary Assessment Ability of Dietetics Students and Interns. Nutrients 2017 Feb 07;9(2):114 [FREE Full text] [CrossRef] [Medline]
    18. Park Y, Dodd K, Kipnis V, Thompson FE, Potischman N, Schoeller DA, et al. Comparison of self-reported dietary intakes from the Automated Self-Administered 24-h recall, 4-d food records, and food-frequency questionnaires against recovery biomarkers. Am J Clin Nutr 2018 Jan 01;107(1):80-93 [FREE Full text] [CrossRef] [Medline]
    19. Prinz N, Bohn B, Kern A, Püngel D, Pollatos O, Holl RW. Feasibility and relative validity of a digital photo-based dietary assessment: results from the Nutris-Phone study. Public Health Nutr 2018 Mar 6:1-8. [CrossRef] [Medline]
    20. Savard C, Lemieux S, Lafrenière J, Laramée C, Robitaille J, Morisset A. Validation of a self-administered web-based 24-hour dietary recall among pregnant women. BMC Pregnancy Childbirth 2018 Apr 23;18(1):112 [FREE Full text] [CrossRef] [Medline]
    21. Lafrenière J, Lamarche B, Laramée C, Robitaille J, Lemieux S. Validation of a newly automated web-based 24-hour dietary recall using fully controlled feeding studies. BMC Nutr 2017 Apr 5;3(1):34 [FREE Full text] [CrossRef] [Medline]
    22. Lafrenière J, Laramée C, Robitaille J, Lamarche B, Lemieux S. Assessing the relative validity of a new, web-based, self-administered 24 h dietary recall in a French-Canadian population. Public Health Nutr 2018 Jul 06;21(15):2744-2752. [CrossRef] [Medline]
    23. Gemming L, Rush E, Maddison R, Doherty A, Gant N, Utter J, et al. Wearable cameras can reduce dietary under-reporting: doubly labelled water validation of a camera-assisted 24 h recall. Br J Nutr 2014 Nov 28;113(2):284-291. [CrossRef] [Medline]
    24. Ptomey LT, Willis EA, Goetz JR, Lee J, Sullivan DK, Donnelly JE. Digital photography improves estimates of dietary intake in adolescents with intellectual and developmental disabilities. Disabil Health J 2015 Jan;8(1):146-150. [CrossRef] [Medline]
    25. Masson L, MCNeill G, Tomany J, Simpson J, Peace H, Wei L, et al. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr 2007 Jan 02;6(3):313-321. [CrossRef] [Medline]
    26. Frankenfeld CL, Poudrier JK, Waters NM, Gillevet PM, Xu Y. Dietary intake measured from a self-administered, online 24-hour recall system compared with 4-day diet records in an adult US population. J Acad Nutr Diet 2012 Oct;112(10):1642-1647. [CrossRef] [Medline]
    27. Wang DH, Kogashiwa M, Ohta S, Kira S. Validity and reliability of a dietary assessment method: the application of a digital camera with a mobile phone card attachment. J Nutr Sci Vitaminol (Tokyo) 2002 Dec;48(6):498-504. [CrossRef] [Medline]
    28. Hanley D, Davison K. Vitamin D insufficiency in North America. J Nutr 2005 Feb;135(2):332-337. [CrossRef] [Medline]
    29. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics 1977 Mar;33(1):159. [CrossRef] [Medline]
    30. Bangor A, Kortum PT, Miller JT. An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction 2008 Jul 30;24(6):574-594. [CrossRef]
    31. Simpson A, Gemming L, Baker D, Braakhuis A. Do Image-Assisted Mobile Applications Improve Dietary Habits, Knowledge, and Behaviours in Elite Athletes? A Pilot Study. Sports (Basel) 2017 Aug 11;5(3):60 [FREE Full text] [CrossRef] [Medline]
    32. Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res 2014 Jun;16(6):e150 [FREE Full text] [CrossRef] [Medline]
    33. Rangan AM, Tieleman L, Louie JCY, Tang LM, Hebden L, Roy R, et al. Electronic Dietary Intake Assessment (e-DIA): relative validity of a mobile phone application to measure intake of food groups. Br J Nutr 2016 Jun;115(12):2219-2226. [CrossRef] [Medline]
    34. Williamson DA, Allen H, Martin PD, Alfonso AJ, Gerald B, Hunt A. Comparison of digital photography to weighed and visual estimation of portion sizes. Journal of the American Dietetic Association 2003 Sep;103(9):1139-1145. [CrossRef]
    35. Fang S, Liu C, Zhu F, Delp E, Boushey C. Single-View Food Portion Estimation Based on Geometric Models. 2015 Nov Presented at: 2015 IEEE International Symposium on Multimedia; 2015; Miami, Florida p. 385-390. [CrossRef]
    36. Paul G, Stegbauer C. Is the digital divide between young and elderly people increasing? First Monday 2005 Oct 03:3 [FREE Full text] [CrossRef]
    37. Shoneye C, Dhaliwal S, Pollard C, Boushey C, Delp E, Harray A, et al. Image-Based Dietary Assessment and Tailored Feedback Using Mobile Technology: Mediating Behavior Change in Young Adults. Nutrients 2019 Feb 19;11(2). [CrossRef] [Medline]
    38. Casperson SL, Sieling J, Moon J, Johnson L, Roemmich JN, Whigham L. A mobile phone food record app to digitally capture dietary intake for adolescents in a free-living environment: usability study. JMIR Mhealth Uhealth 2015 Mar 13;3(1):e30. [CrossRef] [Medline]
    39. Hongu N, Pope BT, Bilgiç P, Orr BJ, Suzuki A, Kim AS, et al. Usability of a smartphone food picture app for assisting 24-hour dietary recall: a pilot study. Nutr Res Pract 2015 Apr;9(2):207-212 [FREE Full text] [CrossRef] [Medline]
    40. Daugherty BL, Schap TE, Ettienne-Gittens R, Zhu FM, Bosch M, Delp EJ, et al. Novel technologies for assessing dietary intake: evaluating the usability of a mobile telephone food record among adults and adolescents. J Med Internet Res 2012 Apr 13;14(2):e58 [FREE Full text] [CrossRef] [Medline]
    41. Robinson J, Neustadtl A, Kestnbaum M. The Online “Diversity Divide”: Public Opinion Differences Among Internet Users and Nonusers. IT & Society 2002;1(1):284-302 [FREE Full text]

    ‎
    3DFD: 3-day food diary
    A: average
    FFQ: food frequency questionnaire
    G: good
    P: poor
    SFA: saturated fatty acid
    SUS: System Usability Scale

    Edited by G Eysenbach; submitted 26.11.19; peer-reviewed by L Catherine, R Abd Jalil; comments to author 15.03.20; revised version received 06.05.20; accepted 13.06.20; published 09.09.20

    Copyright

    ©Yuwei Ji, Hugues Plourde, Valerie Bouzo, Robert D Kilgour, Tamara R Cohen. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 09.09.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.