This paper is in the following e-collection/theme issue:

mHealth for Wellness, Behavior Change and Prevention (2871) Games and Gamification for Health (650) Dental Sciences (119) Quality Evaluation and Descriptive Analysis/Reviews of Multiple Existing Mobile Apps (374) Gamification (128) Serious Games for Health and Medicine (683) Behavior Change (712) Games in Pediatrics (32)

Published on in Vol 8, No 7 (2020): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/16365, first published .
The Effects of Gamification and Oral Self-Care on Oral Hygiene in Children: Systematic Search in App Stores and Evaluation of Apps

The Effects of Gamification and Oral Self-Care on Oral Hygiene in Children: Systematic Search in App Stores and Evaluation of Apps

The Effects of Gamification and Oral Self-Care on Oral Hygiene in Children: Systematic Search in App Stores and Evaluation of Apps

Authors of this article:

Nino Fijačko1 Author Orcid Image ;   Lucija Gosak1 Author Orcid Image ;   Leona Cilar1 Author Orcid Image ;   Alenka Novšak2 Author Orcid Image ;   Ruth Masterson Creber3 Author Orcid Image ;   Pavel Skok4 Author Orcid Image ;   Gregor Štiglic1, 5 Author Orcid Image
Article Authors Cited by (33) Tweetations (11) Metrics

    Original Paper

    1Faculty of Health Sciences, University of Maribor, Maribor, Slovenia

    2Dr. Novsak storitve d.o.o., Ljubljana, Slovenia

    3Healthcare Policy and Research, Division of Health Informatics, Weill Cornell Medicine, New York, NY, United States

    4Medical Faculty, University of Maribor, Maribor, Slovenia

    5Faculty of Electrical Engineering and Computer Science, University of Maribor, Maribor, Slovenia

    *all authors contributed equally

    Corresponding Author:

    Nino Fijačko, MSc

    Faculty of Health Sciences

    University of Maribor

    Žitna 15

    Maribor

    Slovenia

    Phone: 386 23004764

    Email: nino.fijacko@um.si


    Background: Poor oral hygiene is a great public health problem worldwide. Oral health care education is a public health priority as the maintenance of oral hygiene is integral to overall health. Maintaining optimal oral hygiene among children is challenging and can be supported by using relevant motivational approaches.

    Objective: The primary aim of this study was to identify mobile smartphone apps that include gamification features focused on motivating children to learn, perform, and maintain optimal oral hygiene.

    Methods: We searched six online app stores using four search terms (“oral hygiene game,” “oral hygiene gamification,” “oral hygiene brush game,” and “oral hygiene brush gamification”). We identified gamification features, identified whether apps were consistent with evidence-based dentistry, performed a quality appraisal with the Mobile App Rating Scale user version (uMARS), and quantified behavior scores (Behavior Change score, uMARS score, and Coventry, Aberdeen, and London-Refined [CALO-RE] score) using three different instruments that measure behavior change.

    Results: Of 612 potentially relevant apps included in the analysis, 17 met the inclusion criteria. On average, apps included 6.87 (SD 4.18) out of 31 possible gamification features. The most frequently used gamification features were time pressure (16/17, 94%), virtual characters (14/17, 82%), and fantasy (13/17, 76%). The most common oral hygiene evidence-based recommendation was brushing time (2-3 minutes), which was identified in 94% (16/17) of apps. The overall mean uMARS score for app quality was high (4.30, SD 0.36), with good mean subjective quality (3.79, SD 0.71) and perceived impact (3.58, SD 0.44). Sufficient behavior change techniques based on three taxonomies were detected in each app.

    Conclusions: The majority of the analyzed oral hygiene apps included gamification features and behavior change techniques to perform and maintain oral hygiene in children. Overall, the apps contained some educational content consistent with evidence-based dentistry and high-quality background for oral self-care in children; however, there is scope for improvement.

    JMIR Mhealth Uhealth 2020;8(7):e16365

    doi:10.2196/16365

    Keywords

    mobile health; oral health care; gamification; mobile store; evidence-based dentistry; behavior change technique; Mobile Application Rating Scale user version 


    Oral diseases caused by oral hygiene are a major problem worldwide. In some countries, more than 80% of school children are affected [1-3]. Oral diseases can cause severe pain and loss of teeth, both of which affect appearance, dietary intake, and consequently the growth and development of children [4,5]. Poor oral hygiene is associated with poor quality of life and increased morbidity and mortality [1-3,6]. The maintenance of oral hygiene or oral self-care, which includes the use of toothbrushes, dental floss, and other interdental aids for healthy teeth, gums, oral soft tissues, palate, tongue, lips, and salivary glands, is important for quality of life, socialization, overall health, and well-being [7-9].

    Brushing at least twice a day with toothpaste containing fluoride is considered an important aspect of the prevention and promotion of good brushing habits at an early age to prevent early childhood dental decay [10-12]. Oral health care providers play a key role in promoting oral health care among children [13]. Maintaining optimal oral hygiene can be challenging owing to different factors contributing to the lack of motivation and the need to initiate initial dental assessments between 6 and 12 months of age [14,15].

    As children get older, there are new opportunities to use mobile health (mHealth) apps to support pediatric oral hygiene. Gamification is defined as the use of features to increase target behaviors and engagement. The purpose of gamification is to enable users to perform tasks more effectively while making them more enjoyable [16-18]. Gamification features, including badges, levels, and leader boards [17-19], offer novel approaches in dentistry [20]. Gamification has also been included in asthma apps [21] for similar reasons and has been incorporated into other app reviews for fitness apps [22,23], chronic disease management [24], smoking cessation [25], and health promotion [26]. The aim of our study was to identify apps that include gamification features and evidence-based dentistry (EBD) to support the maintenance of oral self-care in children.


    Systematic Search Criteria and Selection

    All apps were searched in June 2019 across six different smartphone app stores (Google Play Store [27], Apple App Store [28], Windows Phone Store [29], Amazon Appstore [30], BlackBerry World [31], and Samsung Galaxy Apps [32]). Our search strategy had two stages. The first evaluation stage was performed in each app store by using the same four search strings (“oral hygiene game,” “oral hygiene gamification,” “oral hygiene brush game,” and “oral hygiene brush gamification”). The PICO criteria [33] were used to define the search criteria as follows: population (children below 13 years), intervention (free and paid apps that contain enlightenment gamification features and that allow users to be part of the interaction), comparison (app contents), and outcome (suitable apps for learning, performing, and maintaining oral health). Apps were excluded if they were duplicate (from multiple search strings and from multiple official web stores), were games, had a non-English interface, and were not related to the oral health care of children above 13 years. The second stage was performed using smartphones. In this stage, the relevant apps were downloaded and evaluated independently by two reviewers (NF and LG). The same reviewers evaluated the eligibility of the apps against preset criteria. Any issues were resolved by discussion among the other members of the study team. We included all free and paid apps that had two or more gamification features in English. Downloaded apps were analyzed on Samsung Galaxy S8 (Samsung, Seoul, South Korea) running Android 9.0 Pie (Google Inc, Mountain View, California, USA) for Android apps and iPhone 7 running iOS 12.3.1 (Apple Inc, Cupertino, California, USA) for iOS apps. If the same app was found for both Android and iOS, we reviewed the app available on Android, as the Google Play Store provides more information about each app than that available on the Apple App Store [34].

    Measures and Rating Tools

    All ratings and rankings were conducted individually and independently by authors (NF, LG, AN, and LC) with experience, knowledge, and skills in the field of mHealth and experience in different fields of health care (nursing, bioinformatics, and dentistry) to support the synthesis of the search results.

    Rating Tool for Gamification Features

    Two authors (LC and NF) identified gamification features by using the modified taxonomy of 31 gamification components (Multimedia Appendix 1) [19,35]. The scoring for gamification features was 1 point for the full implementation of gamification features, 0.5 points for partial implementation, and 0 points if the feature was not implemented.

    Rating Tool for Oral Hygiene-Related Content Based on Dentistry Evidence

    We evaluated EBD in each app using criteria and scored them according to the following four groups: (1) preparing to brush (two items); (2) before brushing (one item); (3) brushing (four items); and (4) after brushing (three items) (Multimedia Appendix 2) [36-40].

    Mobile App Rating Scale User Version and Quantifying Behavior Change

    Based on the Mobile App Rating Scale user version (uMARS) [41], we performed quality appraisal of apps. Originally, the MARS tool was developed with the aim that researchers could use it to determine whether apps satisfied certain quality criteria instead of relying on a subjective five-star rating system [41,42]. The uMARS is a simplified version of MARS [42], which allows multidimensional measurements (sections A-F) of performance indicators (section A), functionality (section B), esthetics (section C), quality of information (section D), subjective quality of the app (section E), and perceived impact (section F). The uMARS tool was developed to eliminate the need for trained professionals, and according to a study by Stoyanov et al, it has good internal consistency (α=.09), and high reliability [41]. All items were rated on a 5-point Likert scale from 1 (inadequate) to 5 (excellent) and were already used in a similar study [43]. Behavior change was also rated and ranked according to three taxonomies (Behavior Change score [44], uMARS score [41], and Coventry, Aberdeen, and London-Refined [CALO-RE] score [45]) related to behavior change (Multimedia Appendix 3 and Multimedia Appendix 4). Similar to the rating tool for gamification features, the scoring system allowed 0, 0.5, or 1 point according to implementation. The exception was the uMARS score where items were scored based on a Likert scale from 1 (strongly disagree) to 5 (strongly agree).

    Data Analysis

    All data analyses and visualizations were prepared using Microsoft Office Professional 2016 (Microsoft Excel 2016; Microsoft Corp, Redmond, Washington, USA) and IBM SPSS Statistics version 25 for Windows (IBM Corp, Armonk, New York, USA). Intraclass correlation coefficients (ICCs) were calculated to provide results on consistency among observational ratings provided by multiple raters [46].


    Descriptive Characteristics

    We identified 612 potentially relevant apps, and only 17 (3%) met the inclusion criteria. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram [47] presents the process of scanning the apps using the inclusion and exclusion criteria. The two largest groups of apps that were excluded in the PRISMA diagram (Figure 1) were apps that required specific equipment or accessories [48] and apps that included gamification but did not include any interaction with the user [49].

    ‎
    Figure 1. PRISMA flow diagram of the selection of the included apps.
    View this figure

    The vast majority of apps were free to download (11/17, 65%) and were available in both the Google Play Store and Apple App Store (14/17, 82%). Most apps did not require registration (14/17, 82%) and were updated in the last 2 years (13/17, 76%). Apps were developed in multiple countries, including the United States (6/17, 35%), Japan (2/17, 12%), Australia (1/17, 6%), Germany (1/17, 6%), England (1/17, 6%), Lithuania (1/17, 6%), North Korea (1/17, 6%), South Korea (1/17, 6%), Russia (1/17, 6%), Sweden (1/17, 6%), and Ukraine (1/17, 6%). Apps were developed by small and medium-size enterprises (14/17, 82%), health care–related agencies (2/17, 12%), and an individual (1/17, 6%).

    Apps were targeted at different age groups of children. Among the 17 apps, 1 (6%) app was for children aged 5 years or less, 7 (41%) were for children aged 5 to 8 years (41%), 2 (12%) were for children aged 6 to 12 years, and 7 (41%) did not provide information about the target age of children but had a Pan European Game Information 3 (PEGI) certificate.

    Table 1 presents all the relevant apps and their basic characteristics [50-80]. We assessed the subjective quality of the apps by answering the following question included in the uMARS framework: “What is your overall (star) rating of the app?” The quality of the apps was rated on a 5-point Likert scale from 1 (the worst app I have used) to 5 (the best app I have used). The mean app score was 3.79 (SD 0.69). The lowest score was 3, and the highest score was 5. We compared the results with the ratings of users in the smartphone app stores. If the app ratings were available on both smartphone platforms (Android and iOS), we used the mean score. The mean user rating was 4.26 (SD 0.29) (Android: 4.21, SD 0.34; iOS: 4.33, SD 0.54), and the mean number of raters was 4981.79 (SD 12023.77) for Android apps and 359.62 (SD 934.17) for iOS apps. Based on a t test, there was a statistically significant difference between uMARS estimates (mean 3.85, SD 0.16) and user ratings (mean 4.26, SD 0.30; t32=−2.27; P=.03).

    Table 1. Description of the included apps.
    Full app name

    		Smartphone platformRating in the form of stars (number of raters)
    Google Play StoreApple App Store
    Brush DJ [50,51]Android & iOS4.2 (1363)4.7 (948)
    Brush Hero [52,53]Android & iOS—a4.2 (24)
    Brushing Hero [54,55]Android & iOS4.5 (371)4.2 (‎24)
    Brush Monster [56,57]Android & iOS4.4 (199)5.0 (2)
    Brush My Teeth [58,59]Android & iOS4.6 (5)—
    Brush\'n\'Save [60,61]Android & iOS4.0 (87)—
    Brush Teeth with The Wiggles [62,63]Android & iOS4.3 (132)4.8 (‎8)
    Brush Up [64,65]Android & iOS3.8 (736)4.1 (64)
    Chomper Chums [66,67]Android & iOS4.1 (355)5.0 (7)
    Disney Magic Timer by Oral-B [68,69]Android & iOS4.0 (36,434)3.9 (3348)
    Mimizavr Clean Teeth [70,71]Android & iOS4.0 (14)—
    MyTeeth [72,73]Android & iOS5 (1)3.0 (9)
    My Virtual Tooth - Virtual Pet [74,75]Android & iOS4.3 (29,945)4.3 (197)
    Timo Kids Routine Timer [76,77]Android & iOS4.1 (100)4.0 (1)
    Tooth Hero [78]Android3.7 (3)—
    Toothsavers Brushing Game [79]iOS—4.6 (8)
    Toothy: Toothbrush Timer [80]iOS—4.5 (34)

    aNot available.

    Gamification Features

    On average, apps included 6.87 (SD 4.18) out of 31 possible gamification features. The most frequent gamification features were time pressure (16/17, 94%), virtual characters (14/17, 82%), and fantasy (13/17, 76%). The less frequently represented gamification features were conforming behavior and leaderboards (each 0.5/17, 3%) (Figure 2).

    ‎
    Figure 2. Gamification component categories of gamification features.
    View this figure

    We identified all apps based on the gamification features (17/17, 100%) from the system design and challenges part of gamification component categories (Figure 2).

    Apps in which we identified the most gamification component categories of gamification features were Toothsavers Brushing Game [79], Chomper Chums [66,67], Brush Monster [56,57], and Brush Up [64,65] (Figure 3).

    ‎
    Figure 3. Percentage of gamification component categories for each app.
    View this figure

    Oral Hygiene-Related Content Based on Dentistry Evidence

    On average, we identified 4.9 (SD 4.98) oral hygiene-related contents in 10 groups based on the level of EBD (EDB1-EDB10). The most common EBD content focused on brushing time (Figure 4) from the EBD group category “Brushing.” Most apps (14/17, 82%) had brushing time set to 2 minutes whereas the app Brushing Hero [54,55] had it set to 1 minute. Only two apps (MyTeeth [72,73] and Brush Up [64,65]) had it set to 3 minutes.

    ‎
    Figure 4. Group categories of evidence-based dentistry (EBD) oral hygiene-related content.
    View this figure

    The most common brushing technique was the scrub brush technique (5/17, 29%). One app (MyTeeth [72,73]) allowed the user to choose among different brushing techniques (scrub brush technique; circular technique; Bass technique; and vertical technique from red [gums] to white [teeth], or inside, chewing surface, and outside or chewing surface, outside, and inside). Over one-third of the apps did not mention any EBD brushing technique (Figure 5).

    On average, the apps included almost 3 out of 10 EBD contents (Figure 6). The MyTeeth app included the most EBD contents.

    ‎
    Figure 5. Brushing techniques used in the apps.
    View this figure
    ‎
    Figure 6. Number of evidence-based dentistry (EBD) contents in the apps.
    View this figure

    uMARS App Quality Scores and Oral Hygiene Care Behavior Change Techniques

    The total mean score for the section “app quality” was 4.30 (SD 0.36), which is considered a good result. The ICC among app reviewers was high for uMARS ratings (ICC=0.80; 95% CI 0.63-0.92). Toothsavers Brushing Game [79] had the highest score in each section of the uMARS (Table 2).

    Table 2. App scores for each section of the Mobile App Rating Scale user version.
    Full app nameuMARSa section
    EngagementFunctionalityEstheticsInformationApp qualityApp subjective qualityPerceived impact
    Brush DJ3.54.53.54.13.93.53.8
    Brush Hero3.93.14.7—b3.92.52.9
    Brushing Hero4.44.84.7—4.64.03.3
    Brush Monster5.04.85.04.44.84.94.0
    Brush My Teeth3.74.44.33.13.93.33.2
    Brush\'n\'Save3.85.02.54.33.83.93.6
    Brush Teeth with The Wiggles4.44.84.0—4.43.53.2
    Brush Up4.34.64.34.54.45.04.2
    Chomper Chums4.74.64.24.34.44.14.1
    Disney Magic Timer by Oral-B4.34.84.23.04.33.13.2
    Mimizavr Clean Teeth4.55.05.0—4.84.33.5
    MyTeeth3.24.64.0—3.93.33.7
    My Virtual Tooth - Virtual Pet4.44.64.0—4.33.63.1
    Timo Kids Routine Timer4.04.94.2—4.43.13.6
    Tooth Hero4.44.04.02.54.54.03.7
    Toothsavers Brushing Game4.94.94.84.84.95.04.5
    Toothy: Toothbrush Timer3.14.44.24.03.93.43.3
    Mean score (SD)4.15 (0.55)4.58 (0.46)4.21 (0.60)3.90 (0.76)4.30 (0.36)3.79 (0.71)3.58 (0.44)
    ICCc,d0.850.710.780.980.560.880.84

    auMARS: Mobile App Rating Scale user version.

    bNot available; in 7 out of 17 (41%) apps the “information” quality section could not be assessed because data were not included.

    cICC: intraclass correlation coefficient.

    dA random effects average measures model with absolute agreement was used for calculated ICCs among two sets of ratings.

    We also evaluated apps based on “subjective quality,” where the mean estimate was good (mean 3.79, SD 0.71) and on “perceived impact,” where the mean estimate was also good (mean 3.58, SD 0.44) (Table 2).

    Results from three behavior change scores across three instruments indicated that Brush Up [64,65], Toothsavers Brushing Game [79], and Chomper Chums [66,67] included the most strategies targeting behavior change (Figure 7). The most common oral hygiene care behavior change techniques were “prompt intention formation” and “model or demonstrate the behavior.”

    ‎
    Figure 7. Number of oral hygiene care behavior change techniques. BC: Behavior Change; CALO-RE: Coventry, Aberdeen, and London-Refined; uMARS: Mobile App Rating Scale user version.
    View this figure

    Gamification Features

    The primary objective of this study was to identify apps that include gamification features focused on motivating children for behavior change to maintain oral self-care and optimal oral hygiene. The study results showed that most of the oral hygiene apps included a relevant number of gamification features and behavior change techniques to perform and maintain oral hygiene in children. Additionally, many apps contained EBD-supported educational content and achieved relatively high scores in app quality ratings. The most commonly identified gamification features were “time pressure” and “audible feedback” in the form of songs that help children to practice oral health care correctly. We compared our gamification results with a study conducted in a pediatric population [81] and found similar results with a focus on “feedback” and “goals” versus “information provision (general)” and “goal setting (behavior).” The authors also concluded that using apps for oral health care can improve motivation for oral self-care in children. Moreover, recent research found that knowledge provision increased motivation in children and that gamification might improve engagement with apps [49].

    Apps Content Based on Dentistry Evidence

    Our second aim was to determine what should be included in EBD health care apps. A small proportion of apps included EBD and the most frequently included EBD content was brushing time. Oral health care professionals generally recommend at least 2 or 3 minutes for brushing teeth. A study from 2019 showed increased duration of toothbrushing when using the Brush Up app [81].

    The brushing technique is important for optimal oral self-care. It is recommended that children use a modified Bass technique because of efficient plaque removal compared with other toothbrushing techniques [82,83]. The scrub brushing technique was the most frequently suggested in the reviewed apps. The scrub brushing technique is the easiest to program into an app and the most commonly used technique in the general population [82,84].

    Children should perform oral hygiene activities more than two times a day, and yet, less than half of the apps included proper EBD warnings or reminders for users to brush their teeth at least twice a day. We also found that only a few apps emphasized brushing with a proper toothbrush and toothpaste quantity, as supported by the literature to reduce the incidence of decay [82,85]. The recommended “pea-sized amount,” which represents approximately 1000 parts per million (ppm) fluoride, is generally recommended, yet only two apps visualized the correct quantity of toothpaste.

    Another important aspect of oral hygiene is the use of a soft or extra-soft toothbrush for safe oral health care [86]. Older brushes lose their plaque removal ability [87,88] and are often contaminated with microbes [89]. Guidelines recommend replacing toothbrushes when they show signs of bristle splaying tear. In addition to oral self-care, less than 20% of apps included the use of dental floss or other interdental brushing approaches. None of the apps promoted the EBD recommendation to first floss and then brush [90,91].

    Although the usage of apps to monitor and manage health is increasing, it is difficult to find information about the sources used in the development of the apps [92]. In addition, many app developers focus on the usefulness and ease of use of the app instead of the quality of the content included in the app.

    App Quality and Behavior Change Techniques

    Our results showed that average users tended to give more positive feedback to apps based on functionality [93] and popularity [94], and there was less focus on usability or quality, which can potentially represent dangerous health information for future users. Therefore, researchers often rely on more complex evaluation tools like the uMARS [95] to obtain a better estimation of app quality. Overall, the Toothsavers Brushing Game [79] had the highest quality score according to the uMARS. Additionally, Toothsavers Brushing Game [79] had the highest scores in all categories of gamification components, EBD oral hygiene-related content, and all three behavior change technique taxonomies, which is consistent with our results. Brush Monster [56,57] and Chomper Chums [66,67] were also visually well designed, included EBD content, and did not have any specific technical problems during their use.

    One of the most important behaviors in oral hygiene care change techniques is user interaction with the app [96]. Our results showed that one app included learning of oral self-care with virtual model teeth and enabled the user to perform self-based oral health care interaction in the app. In the apps Brushing Hero [54,55] and Brush Monster [57,58], augmented reality is added to the self-based oral health care interaction, where the user is a role model. Other apps use one or more different teaching approaches for oral self-care in the form of virtual teeth or a virtual tooth, gradually revealing a picture, virtual model teeth, video, and games.

    We also must consider the risk of using apps as an intervention for oral hygiene care in children, as the youngest users seem to be most highly affected by and at risk for behavioral addictions [97]. The pattern of smartphone abuse is greatest among young people, and problematic use is linked to self-esteem, impulsivity, self-identity, and self-image. Problematic use is also associated with sleep disturbance, anxiety, stress, and, to a lesser extent, depression [98], as well as impaired psychological well-being, impaired parent and school relationships, and additional behavioral problems [99]. As interventions using apps or games for increasing oral hygiene care quality can be successful, judicious use is advised. Our observation is that using apps for learning, performing, and maintaining oral self-care requires less than 10 minutes every day, and this is much lower than the average time spent on a smartphone.

    Observation of Apps and Tips for Future Oral Health Care App Developers

    Our study provides app reviewers and developers of oral self-care with structured information on how future apps should be implemented and which EBD content should be included. Apps for supporting oral self-care in oral hygiene should have a combination of gamification features and EBD. Collaboration with oral health care professionals (eg, oral health care organizations and dentists) can improve the content provided in the app and bring EBD self-care information to targeted end users.

    We found the following two secondary app payment strategies: “freemium” (eg, in-app purchases) and “free trial” (eg, free to premium version). Three apps had a trial version and required payment over time (Toothy: Toothbrush Timer [80], €1.32/month [US $1.42/month]; Brush Up [64,65], €1.11/month [US $1.20/month] or buy €11.14 [US $12.02]; My Virtual Tooth-Virtual Pet [74,75], €5.49 [US $5.92]). In-app purchases in different forms were offered in five apps, with a minimum price of €0.99 (US $1.07) and maximum price of €11.14 (US $12.02) per item. To increase usage, many mobile game companies provide ‘‘freemium” services, which cost the player nothing for basic usage but need payment with real money for advanced functionality or virtual goods [100]. Liu et al found that the freemium strategy is positively associated with increased sales of paid apps [101]. The most popular in-app purchases were in the form of getting virtual goods (ie, coins), disabling in-app advertisements, getting virtual characters (eg, avatars), and achieving virtual goals (eg, levels). Only three apps in the smartphone app stores provided information that the apps contain advertisements and offer in-app purchases. Affective states, such as playfulness, seem to be one of the most influential factors positively affecting consumers’ intention to pay for mobile game services [102]. Additionally, “stickiness,” that is, the trait of a game that engages users, is found to greatly and positively influence in-app purchases [103].

    Positive effective states associated with in-game rewards seem to help the user to play the game better and can enhance both perceived value and customer loyalty [104]. Many smartphone games that provide “freemium“ services are intended to be used by children. Hence, it seems that smartphone game service providers try to use the affective states of children to increase their revenue. This is especially concerning since the preadolescent (and adolescent) brain is still very much developing, especially in terms of reasoning and anticipating what will happen in the future [105]. Only one app (Disney Magic Timer by Oral-B [68,69]) alerted users that real money is needed to make in-app purchases; therefore, parental supervision is recommended. Google Play Store and Apple App Store provide users the opportunity to get a refund of the money paid but within a limited period.

    Limitations

    As with any review of commercially available tools in app stores, there is time sensitivity of the results because apps are being added and removed from app stores daily. Given that caveat, this review is meant to provide an overview of the state of apps available in this field, with less emphasis on individual apps. Another limitation of this study is that it focused only on features that were freely available in the apps. As such, we may have missed gamification features, EBD contents, and other features in the apps. We also excluded apps that required subscription over time. The last limitation in this study is that we did not make any in-app purchases in paid apps, so we could not present additional results for in-app purchases.

    Future Research

    Future work should evaluate one of the highest scoring apps (eg, Toothsavers Brushing Game and Chomper Chums) in a randomized controlled trial to evaluate oral hygiene outcomes and motivation for learning, performing, and maintaining oral self-care among children.

    Conclusions

    In this systematic review, we did not find any app that had all the segments for learning, performing, and maintaining proper oral health care. The Toothsavers Brushing Game app had the highest scores, and if future updates introduce more EBD content, it could be the most appropriate app for learning, performing, and maintaining good oral health care in children. Gamification features with EBD have good potential as new approaches for health care providers to change behavior in the form of learning, performing, and maintaining proper oral hygiene with EBD in the clinical environment.

    Acknowledgments

    This study was partially supported by the “knowledge through creative pathways 2016-2020” scheme cofunded by the European Union from the European Social Fund and the Republic of Slovenia and the Slovenian Research Agency (grant numbers N2-0101 and P2-0057). In addition, Dr. Masterson Creber is supported by NIH/National Institute of Nursing Research (R00NR016275).

    Authors' Contributions

    The systematic review presented here was carried out through collaboration among all authors. NF developed a study design and supervised the study. NF, LC, and RMC drafted the manuscript. NF, LG, GS, and PS conducted data collection and analysis. AN interpreted results from the dental hygiene point of view. RMC conducted a comprehensive content review. All authors read, revised, and approved the final manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Exemplary gamification features rating criteria for oral hygiene apps.

    DOCX File , 45 KB
    ‎
    Multimedia Appendix 2

    Oral self-care app segments and evidence-based dentistry.

    DOCX File , 44 KB
    ‎
    Multimedia Appendix 3

    Exemplary oral health care behavior change techniques according to the Behavior Change score.

    DOCX File , 45 KB
    ‎
    Multimedia Appendix 4

    Exemplary oral health care behavior change techniques according to the Coventry, Aberdeen, and London-Refined behavior change score.

    DOCX File , 47 KB
    1. Kwan SY, Petersen PE, Pine CM, Borutta A. Health-promoting schools: an opportunity for oral health promotion. Bull World Health Organ 2005 Sep;83(9):677-685 [FREE Full text] [CrossRef] [Medline]
    2. Parmar P, Radha G, Rekha R, Pallavi S, Nagashree S. Promoting oral hygiene and health through school. Int J Oral Health Sci 2016;6(2):70-77. [CrossRef]
    3. Azodo CC, Unamatokpa B. Gender difference in oral health perception and practices among Medical House Officers. RusOMJ 2012 Sep;1(2):0208. [CrossRef]
    4. Petersen P. The World Oral Health Report 2003: continuous improvement of oral health in the 21st century--the approach of the WHO Global Oral Health Programme. Community Dent Oral Epidemiol 2003 Dec;31 Suppl 1:3-23 [FREE Full text] [CrossRef] [Medline]
    5. Nunn JH. The burden of oral ill health for children. Arch Dis Child 2006 Mar;91(3):251-253 [FREE Full text] [CrossRef] [Medline]
    6. Rabiei S, Mohebbi SZ, Patja K, Virtanen JI. Physicians' knowledge of and adherence to improving oral health. BMC Public Health 2012 Oct 09;12:855 [FREE Full text] [CrossRef] [Medline]
    7. Bashiru BO, Anthony IN. Oral self-care practices among university students in Port Harcourt, Rivers State. Niger Med J 2014 Nov;55(6):486-489 [FREE Full text] [CrossRef] [Medline]
    8. Naseem S, Fatima SH, Ghazanfar H, Haq S, Khan NA, Mehmood M, et al. Oral Hygiene Practices and Teeth Cleaning Techniques Among Medical Students. Cureus 2017 Jul 18;9(7):e1487 [FREE Full text] [CrossRef] [Medline]
    9. Jabeen C, Umbreen G. Oral Hygiene: Knowledge, Attitude and Practice among School Children, Lahore. J Liaquat Uni Med Health Sci 2017 Sep 30;16(03):176-180. [CrossRef]
    10. Wilson M, Morgan M. A survey of children's toothbrushing habits in Wales: Are parents following the guidelines? Journal of Health Visiting 2016 Feb;4(2):95-99. [CrossRef]
    11. Melo P, Fine C, Malone S, Frencken JE, Horn V. The effectiveness of the Brush Day and Night programme in improving children's toothbrushing knowledge and behaviour. Int Dent J 2018 May;68 Suppl 1:7-16. [CrossRef] [Medline]
    12. Pollick H. The Role of Fluoride in the Prevention of Tooth Decay. Pediatr Clin North Am 2018 Oct;65(5):923-940. [CrossRef] [Medline]
    13. Soltani R, Ali EA, Mahaki B, Alipoor M, Sharifirad G. Do maternal oral health-related self-efficacy and knowledge influence oral hygiene behavior of their children? International Journal of Pediatrics 2016;4(7):2035-2042. [CrossRef]
    14. Acharya S, Goyal A, Utreja AK, Mohanty U. Effect of three different motivational techniques on oral hygiene and gingival health of patients undergoing multibracketed orthodontics. Angle Orthod 2011 Sep;81(5):884-888. [CrossRef] [Medline]
    15. Meera R, Muthu MS, Phanibabu M, Rathnaprabhu V. First dental visit of a child. J Indian Soc Pedod Prev Dent 2008;26 Suppl 2:S68-S71 [FREE Full text] [Medline]
    16. Khaleel FL, Ashaari NS, Wook TS, Ismail A. User-enjoyable learning environment based on Gamification elements. 2015 Presented at: 2015 International Conference on Computer, Communications, and Control Technology (I4CT); April 21-23, 2015; Kuching, Malaysia   URL: https://www.researchgate.net/publication/304627718_User-enjoyable_learning_environment [CrossRef]
    17. Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining gamification. Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments 2011 Sep 29 [FREE Full text] [CrossRef]
    18. Subhash S, Cudney EA. Gamified learning in higher education: A systematic review of the literature. Computers in Human Behavior 2018 Oct;87:192-206. [CrossRef]
    19. Scott T, Sebastian L, Dirk B. Gamifying information systems-a synthesis of gamification mechanics and dynamics. 2014 Jun 9 Presented at: European Conference on Information Systems (ECIS) 2014; June 9-11, 2014; Tel Aviv, Israel p. 1-17   URL: https://pdfs.semanticscholar.org/a02f/7cc91d26671e591e27b241ed6b17debc2d89.pdf
    20. Alaa HQ. Gamification in dentistry: A new era and direction in dental education, awareness and business. 2014 Apr 21 Presented at: 2nd International Conference and Exhibition on Dental & Oral Health; April 21-23, 2014; Crown Plaza Dubai, UAE p. 194-194   URL: https:/​/www.​longdom.org/​proceedings/​gamification-in-dentistry-a-new-era-and-direction-in-dental-education-awareness-and-business-24562.​html
    21. Tinschert P, Jakob R, Barata F, Kramer J, Kowatsch T. The Potential of Mobile Apps for Improving Asthma Self-Management: A Review of Publicly Available and Well-Adopted Asthma Apps. JMIR Mhealth Uhealth 2017 Aug;5(8):e113 [FREE Full text] [CrossRef] [Medline]
    22. Cotton V, Patel MS. Gamification Use and Design in Popular Health and Fitness Mobile Applications. Am J Health Promot 2019 Mar;33(3):448-451 [FREE Full text] [CrossRef] [Medline]
    23. Lister C, West JH, Cannon B, Sax T, Brodegard D. Just a fad? Gamification in health and fitness apps. JMIR Serious Games 2014 Aug 04;2(2):e9 [FREE Full text] [CrossRef] [Medline]
    24. Miller AS, Cafazzo JA, Seto E. A game plan: Gamification design principles in mHealth applications for chronic disease management. Health Informatics J 2016 Jun;22(2):184-193. [CrossRef] [Medline]
    25. El-Hilly AA, Iqbal SS, Ahmed M, Sherwani Y, Muntasir M, Siddiqui S, et al. Game On? Smoking Cessation Through the Gamification of mHealth: A Longitudinal Qualitative Study. JMIR Serious Games 2016 Oct 24;4(2):e18 [FREE Full text] [CrossRef] [Medline]
    26. Edwards EA, Lumsden J, Rivas C, Steed L, Edwards LA, Thiyagarajan A, et al. Gamification for health promotion: systematic review of behaviour change techniques in smartphone apps. BMJ Open 2016 Oct 04;6(10):e012447 [FREE Full text] [CrossRef] [Medline]
    27. Google Play.   URL: https://play.google.com/store/apps?hl=en_USA [accessed 2019-09-11]
    28. App Store.   URL: https://www.apple.com/uk/ios/app-store/ [accessed 2019-09-11]
    29. Windows Phone Store.   URL: https://www.microsoft.com/sl-si/store/apps/windows-phone [accessed 2019-09-11]
    30. Amazon Appstore.   URL: https://www.amazon.com/mobile-apps/b?ie=UTF8&node=2350149011
    31. App World. BlackBerry World   URL: https://appworld.blackberry.com/ [accessed 2019-09-11]
    32. Galaxy Store.   URL: https://www.samsung.com/global/galaxy/apps/galaxy-store/ [accessed 2019-09-11]
    33. Stone PW. Popping the (PICO) question in research and evidence-based practice. Appl Nurs Res 2002 Aug;15(3):197-198. [CrossRef] [Medline]
    34. Cruz Zapata B, Hernández Niñirola A, Idri A, Fernández-Alemán JL, Toval A. Mobile PHRs compliance with Android and iOS usability guidelines. J Med Syst 2014 Aug 24;38(8):81. [CrossRef] [Medline]
    35. Tiffany B, Blasi P, Catz SL, McClure JB. Mobile Apps for Oral Health Promotion: Content Review and Heuristic Usability Analysis. JMIR Mhealth Uhealth 2018 Sep 04;6(9):e11432 [FREE Full text] [CrossRef] [Medline]
    36. Pujar P, Subbareddy VV. Evaluation of the tooth brushing skills in children aged 6-12 years. Eur Arch Paediatr Dent 2013 Aug 5;14(4):213-219. [CrossRef] [Medline]
    37. Muller-Bolla M, Courson F. Toothbrushing methods to use in children: a systematic review. Oral Health Prev Dent 2013;11(4):341-347. [CrossRef] [Medline]
    38. Matsui M, Chosa N, Shimoyama Y, Minami K, Kimura S, Kishi M. Effects of tongue cleaning on bacterial flora in tongue coating and dental plaque: a crossover study. BMC Oral Health 2014 Jan 14;14:4 [FREE Full text] [CrossRef] [Medline]
    39. Pitts N, Duckworth RM, Marsh P, Mutti B, Parnell C, Zero D. Post-brushing rinsing for the control of dental caries: exploration of the available evidence to establish what advice we should give our patients. Br Dent J 2012 Apr 13;212(7):315-320. [CrossRef] [Medline]
    40. ADA Division of Communications. For the dental patient. Toothbrush care, cleaning and replacement. J Am Dent Assoc 2006 Mar;137(3):415. [CrossRef] [Medline]
    41. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016;4(2):e72 [FREE Full text] [CrossRef] [Medline]
    42. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015 Mar;3(1):e27 [FREE Full text] [CrossRef] [Medline]
    43. Adam A, Hellig JC, Perera M, Bolton D, Lawrentschuk N. 'Prostate Cancer Risk Calculator' mobile applications (Apps): a systematic review and scoring using the validated user version of the Mobile Application Rating Scale (uMARS). World J Urol 2018 Apr;36(4):565-573. [CrossRef] [Medline]
    44. Abraham C, Michie S. A taxonomy of behavior change techniques used in interventions. Health Psychol 2008 May;27(3):379-387. [CrossRef] [Medline]
    45. Michie S, Ashford S, Sniehotta FF, Dombrowski SU, Bishop A, French DP. A refined taxonomy of behaviour change techniques to help people change their physical activity and healthy eating behaviours: the CALO-RE taxonomy. Psychology & health 2011 Nov;26(11):1479-1498. [CrossRef] [Medline]
    46. Hallgren KA. Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. Tutor Quant Methods Psychol 2012;8(1):23-34 [FREE Full text] [Medline]
    47. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    48. Fijacko N, Stiglic G, Kocbek P, Cilar L, Gosak L. Learning oral hygiene with gamification and smart accessories: a systematic review of mobile phone apps. In: Book of Abstracts International Scientific Conference. 2019 Presented at: International Scientific Conference; June 13, 2019; Maribor, Slovenia. [CrossRef]
    49. Parker K, Bharmal RV, Sharif MO. The availability and characteristics of patient-focused oral hygiene apps. Br Dent J 2019 Apr;226(8):600-604. [CrossRef] [Medline]
    50. Google Play Store. Brush DJ   URL: https://play.google.com/store/apps/details?id=uk.co.appware.brushdj&hl=sl [accessed 2020-02-13]
    51. Apple App Store. Brush DJ   URL: https://apps.apple.com/us/app/brush-dj/id475739913 [accessed 2020-02-13]
    52. Google Play Store. Brush Hero   URL: https://play.google.com/store/apps/details?id=com.brushheroapp.brushhero [accessed 2020-02-13]
    53. Apple App Store. Brush Hero   URL: https://apps.apple.com/us/app/brush-hero/id1380844743 [accessed 2020-02-13]
    54. Google Play Store. Brushing Hero   URL: https://play.google.com/store/apps/details?id=jp.co.litalico.brushinghero [accessed 2020-02-13]
    55. Apple App Store. Brushing Hero   URL: https://apps.apple.com/us/app/brushing-hero/id1317198419 [accessed 2020-02-13]
    56. Google Play Store. Brush Monster - AR Toothbrushing Guide App   URL: https://play.google.com/store/apps/details?id=com.kittenplanet.brushmon&hl=sl [accessed 2020-02-13]
    57. Apple App Store. Brush Monster - AR Toothbrush   URL: https://apps.apple.com/au/app/brush-monster-artoothbrush/id1320759633 [accessed 2020-02-13]
    58. Google Play Store. Brush my teeth   URL: https://play.google.com/store/apps/details?id=se.skuggan.brush.en&hl=sl [accessed 2020-02-13]
    59. Apple App Store. Brush My Teeth   URL: https://apps.apple.com/au/app/brush-my-teeth/id974298516 [accessed 2020-02-13]
    60. Google Play Store. Brush'n'Save   URL: https://play.google.com/store/apps/details?id=com.amanodental.www.HCFM02&hl=sl [accessed 2020-02-13]
    61. Apple App Store. Brush'n'Save   URL: https://apps.apple.com/us/app/brushnsave/id916125398 [accessed 2020-02-13]
    62. Google Play Store. Brush Teeth with The Wiggles   URL: https://play.google.com/store/apps/details?id=au.com.australianunity.wigglesbrush&hl=sl [accessed 2020-02-13]
    63. Apple App Store. Brush Teeth with The Wiggles   URL: https://apps.apple.com/us/app/brush-teeth-with-the-wiggles/id1113805211 [accessed 2020-02-13]
    64. Google Play Store. Brush Up   URL: https://play.google.com/store/apps/details?id=com.gtw.BrushUp&hl=sl [accessed 2020-02-13]
    65. Apple App Store. Brush Up: Toothbrush Trainer   URL: https://apps.apple.com/us/app/brush-up-toothbrush-trainer/id967009000 [accessed 2020-02-13]
    66. Google Play Store. Chomper Chums   URL: https://play.google.com/store/apps/details?id=com.ucci.dtt&hl=sl [accessed 2020-02-13]
    67. Apple App Store. Chomper Chums   URL: https://apps.apple.com/us/app/chomper-chums/id579421399 [accessed 2020-02-13]
    68. Google Play Store. Disney Magic Timer by Oral-B   URL: https://play.google.com/store/apps/details?id=com.disneydigitalbooks.DisneyMagicBrushTimer_goo&hl=sl [accessed 2020-02-13]
    69. Apple App Store. Disney Magic Timer by Oral-B   URL: https://apps.apple.com/us/app/disney-magic-timer-by-oral-b/id747541884 [accessed 2020-02-13]
    70. Google Play Store. Mimizavr clean teeth   URL: https://play.google.com/store/apps/details?id=ru.mimizavr&hl=sl [accessed 2020-02-13]
    71. Apple App Store. Mimizavr Brush Teeth   URL: https://apps.apple.com/us/app/mimizavr-brush-teeth/id1242740400 [accessed 2020-02-13]
    72. Google Play Store. MyTeeth   URL: https://play.google.com/store/apps/details?id=com.thekeptpromise.myteeth&hl=sl [accessed 2020-02-13]
    73. Apple App Store. MyTeeth   URL: https://apps.apple.com/us/app/myteeth/id715668236 [accessed 2020-02-13]
    74. Google Play Store. My Virtual Tooth - Virtual Pet   URL: https://play.google.com/store/apps/details?id=com.mytooth.virtualpet&hl=sl [accessed 2020-02-13]
    75. Apple App Store. My Virtual Tooth - Virtual Pet   URL: https://apps.apple.com/us/app/my-virtual-tooth-virtual-pet/id1129665919 [accessed 2020-02-13]
    76. Google Play Store. TIMO Kids Visual Timer - Game for Daily Routine   URL: https://play.google.com/store/apps/details?id=com.timo.time&hl=sl [accessed 2020-02-13]
    77. Apple App Store. TIMO Kids Visual Timer   URL: https://apps.apple.com/us/app/timo-kids-routine-timer/id1417830395 [accessed 2020-02-13]
    78. APKPure. Tooth Hero - Brushing Timer Game, Stop Cavities!   URL: https://apkpure.com/tooth-hero-brushing-timer-game-stop-cavities/com.newrealmtech.toothhero [accessed 2020-02-13]
    79. Apple App Store. Toothsavers Brushing Game   URL: https://apps.apple.com/us/app/toothsavers-brushing-game/id734501468 [accessed 2020-02-13]
    80. Apple App Store. Toothy: Tooth Brushing Timer   URL: https://apps.apple.com/us/app/toothy-tooth-brushing-timer/id1453139747 [accessed 2020-02-13]
    81. Jacobson D, Jacobson J, Leong T, Lourenco S, Mancl L, Chi DL. Evaluating Child Toothbrushing Behavior Changes Associated with a Mobile Game App: A Single Arm PrePost Pilot Study. Pediatr Dent 2019 Jul 15;41(4):299-303 [FREE Full text] [Medline]
    82. Wainwright J, Sheiham A. An analysis of methods of toothbrushing recommended by dental associations, toothpaste and toothbrush companies and in dental texts. Br Dent J 2014 Aug;217(3):E5. [CrossRef] [Medline]
    83. Janakiram C, Taha F, Joe J. The Efficacy of Plaque Control by Various Toothbrushing Techniques-A Systematic Review and Meta-Analysis. Journal of Clinical & Diagnostic Research 2018;12(11). [CrossRef]
    84. Baruah K, Thumpala VK, Khetani P, Baruah Q, Tiwari RV, Dixit H. A review on toothbrushes and tooth brushing methods. Int J Pharm Sci Invention 2017 May;6(5):29-38.
    85. Valkenburg C, Van der Weijden F, Slot DE. Is plaque regrowth inhibited by dentifrice?: A systematic review and meta-analysis with trial sequential analysis. Int J Dent Hyg 2019 Feb;17(1):27-38. [CrossRef] [Medline]
    86. Ranzan N, Muniz FW, Rösing CK. Are bristle stiffness and bristle end-shape related to adverse effects on soft tissues during toothbrushing? A systematic review. Int Dent J 2019 Jun;69(3):171-182. [CrossRef] [Medline]
    87. Van Leeuwen MP, Van der Weijden FA, Slot DE, Rosema MA. Toothbrush wear in relation to toothbrushing effectiveness. Int J Dent Hyg 2019 Feb;17(1):77-84. [CrossRef] [Medline]
    88. Gundavarapu KC, Ramachandra SS, Dicksit DD. An investigation into toothbrush wear related to months of use among university students. Can J Dent Hyg 2015 Sep 13;49(2):76-80 [FREE Full text]
    89. Karibasappa GN, Nagesh L, Sujatha BK. Assessment of microbial contamination of toothbrush head: an in vitro study. Indian J Dent Res 2011;22(1):2-5 [FREE Full text] [CrossRef] [Medline]
    90. Torkzaban P, Arabi SR, Sabounchi SS, Roshanaei G. The Efficacy of Brushing and Flossing Sequence on Control of Plaque and Gingival Inflammation. Oral Health Prev Dent 2015;13(3):267-273. [CrossRef] [Medline]
    91. Mazhari F, Boskabady M, Moeintaghavi A, Habibi A. The effect of toothbrushing and flossing sequence on interdental plaque reduction and fluoride retention: A randomized controlled clinical trial. J Periodontol 2018;89(7):824-832. [CrossRef]
    92. Armstrong S. Which app should I use? BMJ 2015 Sep 09;351:h4597. [CrossRef] [Medline]
    93. Iacob C, Veerappa V, Harrison R. What are you complaining about?: a study of online reviews of mobile applications. Station Road, Swindon, United Kingdom: BCS Learning & Development Ltd; 2013 Presented at: The 27th International BCS Human Computer Interaction Conference; September 9-13, 2013; Brunel University, United Kingdom   URL: https://dl.acm.org/doi/proceedings/10.5555/2578048 [CrossRef]
    94. Singh K, Drouin K, Newmark LP, Lee J, Faxvaag A, Rozenblum R, et al. Many Mobile Health Apps Target High-Need, High-Cost Populations, But Gaps Remain. Health Aff (Millwood) 2016 Dec 01;35(12):2310-2318. [CrossRef] [Medline]
    95. LeBeau K, Huey LG, Hart M. Assessing the Quality of Mobile Apps Used by Occupational Therapists: Evaluation Using the User Version of the Mobile Application Rating Scale. JMIR Mhealth Uhealth 2019 May 01;7(5):e13019 [FREE Full text] [CrossRef] [Medline]
    96. Madan Kumar PD, Mohandoss AA, Walls T, Rooban T, Vernon LT. Using smartphone video "selfies" to monitor change in toothbrushing behavior after a brief intervention: A pilot study. Indian J Dent Res 2016;27(3):268-277 [FREE Full text] [CrossRef] [Medline]
    97. Alavi SS, Maracy MR, Jannatifard F, Ojaghi R, Rezapour H. The psychometric properties of cellular phone dependency questionnaire in students of Isfahan: A pilot study. J Educ Health Promot 2014;3:71 [FREE Full text] [CrossRef] [Medline]
    98. De-Sola GJ, Rodríguez DF, Rubio G. Cell-Phone Addiction: A Review. Front Psychiatry 2016;7:175 [FREE Full text] [CrossRef] [Medline]
    99. Foerster M, Roser K, Schoeni A, Röösli M. Problematic mobile phone use in adolescents: derivation of a short scale MPPUS-10. Int J Public Health 2015 Feb;60(2):277-286. [CrossRef] [Medline]
    100. Staykova KS, Damsgaard J. The race to dominate the mobile payments platform: Entry and expansion strategies. Electronic Commerce Research and Applications 2015 Sep;14(5):319-330. [CrossRef]
    101. Liu CZ, Au YA, Choi HS. Effects of Freemium Strategy in the Mobile App Market: An Empirical Study of Google Play. Journal of Management Information Systems 2015 Mar 09;31(3):326-354. [CrossRef]
    102. Wang W, Li H. Factors influencing mobile services adoption: a brand‐equity perspective. Internet Research 2012 Mar 30;22(2):142-179. [CrossRef]
    103. Hsu CL, Lin JCC. Effect of perceived value and social influences on mobile app stickiness and in-app purchase intention. Technological Forecasting and Social Change 2016 Jul;108:42-53. [CrossRef]
    104. Yi Y, Jeon H. Effects of Loyalty Programs on Value Perception, Program Loyalty, and Brand Loyalty. J Acad Mark Sci 2003;31(3):229-240. [CrossRef]
    105. Giedd JN, Blumenthal J, Jeffries NO, Castellanos FX, Liu H, Zijdenbos A, et al. Brain development during childhood and adolescence: a longitudinal MRI study. Nat Neurosci 1999 Oct;2(10):861-863. [CrossRef] [Medline]

    ‎
    EBD: evidence-based dentistry
    ICC: intraclass correlation coefficient
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    uMARS: Mobile Application Rating Scale user version

    Edited by G Eysenbach; submitted 25.09.19; peer-reviewed by N Alshaiban, MA Vicente, C Fernández, L Grepo; comments to author 11.12.19; accepted 23.03.20; published 08.07.20

    Copyright

    ©Nino Fijačko, Lucija Gosak, Leona Cilar, Alenka Novšak, Ruth Masterson Creber, Pavel Skok, Gregor Štiglic. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 08.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.