This paper is in the following e-collection/theme issue:

Quality Evaluation and Descriptive Analysis/Reviews of Multiple Existing Mobile Apps (362) mHealth in Medical Education and Training (66) Games and Gamification for Health (609) Serious Games for Health and Medicine (644) mHealth for Wellness, Behavior Change and Prevention (2743) Mobile Apps for Cardiology (127) Innovations in Cardiopulmonary Resuscitation (31) Patient Education in Cardiology (22) Gamification (117)

Published on in Vol 9, No 7 (2021): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/25437, first published .
Evaluating Quality, Usability, Evidence-Based Content, and Gamification Features in Mobile Learning Apps Designed to Teach Children Basic Life Support: Systematic Search in App Stores and Content Analysis

Evaluating Quality, Usability, Evidence-Based Content, and Gamification Features in Mobile Learning Apps Designed to Teach Children Basic Life Support: Systematic Search in App Stores and Content Analysis

Evaluating Quality, Usability, Evidence-Based Content, and Gamification Features in Mobile Learning Apps Designed to Teach Children Basic Life Support: Systematic Search in App Stores and Content Analysis

Authors of this article:

Nino Fijačko1 Author Orcid Image ;   Ruth Masterson Creber2 Author Orcid Image ;   Lucija Gosak1 Author Orcid Image ;   Gregor Štiglic1, 3, 4 Author Orcid Image ;   Dominic Egan5 Author Orcid Image ;   Brian Chaka6 Author Orcid Image ;   Nika Debeljak1 Author Orcid Image ;   Matej Strnad7 Author Orcid Image ;   Pavel Skok7 Author Orcid Image
Article Authors Cited by (12) Tweetations (4) Metrics

    Original Paper

    1Faculty of Health Sciences, University of Maribor, Maribor, Slovenia

    2Healthcare Policy and Research, Division of Health Informatics, Weill Cornell Medicine, New York, NY, United States

    3Faculty of Electrical Engineering and Computer Science, University of Maribor, Maribor, Slovenia

    4Usher Institute, University of Edinburgh, Edinburgh, United Kingdom

    5School of Nursing and Healthcare Leadership, University of Bradford, Bradford, United Kingdom

    6School of Allied Health Professions and Midwifery, University of Bradford, Bradford, United Kingdom

    7Faculty of Medicine, University of Maribor, Maribor, Slovenia

    *all authors contributed equally

    Corresponding Author:

    Nino Fijačko, MSc

    Faculty of Health Sciences

    University of Maribor

    Žitna 15

    Maribor, 2000

    Slovenia

    Phone: 386 23004764

    Fax:386 23004747

    Email: nino.fijacko@um.si


    Background: Globally, 3.7 million people die of sudden cardiac death annually. Following the World Health Organization endorsement of the Kids Save Lives statements, initiatives to train school-age children in basic life support (BLS) have been widespread. Mobile phone apps, combined with gamification, represent an opportunity for including mobile learning (m-learning) in teaching schoolchildren BLS as an additional teaching method; however, the quality of these apps is questionable.

    Objective: This study aims to systematically evaluate the quality, usability, evidence-based content, and gamification features (GFs) of commercially available m-learning apps for teaching guideline-directed BLS knowledge and skills to school-aged children.

    Methods: We searched the Google Play Store and Apple iOS App Store using multiple terms (eg, cardiopulmonary resuscitation [CPR] or BLS). Apps meeting the inclusion criteria were evaluated by 15 emergency health care professionals using the user version of the Mobile Application Rating Scale and System Usability Scale. We modified a five-finger mnemonic for teaching schoolchildren BLS and reviewed the apps’ BLS content using standardized criteria based on three CPR guidelines. GFs in the apps were evaluated using a gamification taxonomy.

    Results: Of the 1207 potentially relevant apps, only 6 (0.49%) met the inclusion criteria. Most apps were excluded because the content was not related to teaching schoolchildren BLS. The mean total scores for the user version of the Mobile Application Rating Scale and System Usability Scale score were 3.2/5 points (95% CI 3.0-3.4) and 47.1/100 points (95% CI 42.1-52.1), respectively. Half of the apps taught hands-only CPR, whereas the other half also included ventilation. All the apps indicated when to start chest compressions, and only 1 app taught BLS using an automated external defibrillator. Gamification was well integrated into the m-learning apps for teaching schoolchildren BLS, whereas the personal and fictional, educational, and performance gamification groups represented most GFs.

    Conclusions: Improving the quality and usability of BLS content in apps and combining them with GFs can offer educators novel m-learning tools to teach schoolchildren BLS skills.

    JMIR Mhealth Uhealth 2021;9(7):e25437

    doi:10.2196/25437

    Keywords

    cardiopulmonary resuscitation; basic life support; mobile learning; mobile phone; gamification; schoolchildren 


    Background

    Sudden cardiac arrest is a leading cause of mortality, responsible for 3.7 million deaths per year [1-5]. Most deaths occur in the community and can be prevented with basic life support (BLS) [6], specifically cardiopulmonary resuscitation (CPR), which doubles the chances of survival [6-8]. The European Resuscitation Council (ERC) [9] and American Heart Association (AHA) [10] guidelines recommend that lay persons respond immediately after a patient collapses and before the arrival of emergency paramedic personnel; however, CPR bystander response rates are <50%, primarily because of low self-efficacy and knowledge of safely conducting CPR by the lay public [6,11,12].

    Following the World Health Organization endorsement of the Kids Save Lives statement [13], initiatives to include BLS training in primary and secondary schools have been implemented in the hope of increasing rates of bystander CPR [14-16]. Early findings demonstrate that when schoolchildren were educated in BLS, bystander rates of CPR have doubled [17]. Proponents of educating schoolchildren in BLS can do so using interactive digital technologies, including mobile learning (m-learning) [18,19] with gamification features (GFs) [20,21]. Gamification, popularly defined as “the use of game design elements in non-game contexts,” [20] has emerged as a means of harnessing competitiveness by integrating gamification elements such as leaderboards, rewards, badges, avatars, and competitions to engage and motivate consumers [21,22]. The most common tools for teaching BLS to schoolchildren include self-made games [23-25], posters [26], songs [27], and manikins [23,28,29].

    Objectives

    The ERC guidelines for resuscitation recommend that the use and development of technology and social media should be encouraged and the impact, assessed [30]. Reviews of apps offering real-time instructions for adult learning and bystander CPR have been published [31,32]; however, they exclude school-aged children. This study aims to systematically evaluate the quality, usability, evidence-based content, and gamification features of commercially available m-learning apps for teaching guideline-directed BLS knowledge and skills to school-aged children.


    Searching, Screening, and Reviewing of Commercially Available Apps

    We conducted a systematic search of commercially available apps using a rigorous methodology that has been previously published [33,34]. The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist [35] is available in Multimedia Appendix 1. We searched the Google Play Store [36] (for Android apps) and the App Store (for Apple iOS apps) [37] in May 2020. We created inclusion criteria including the population (schoolchildren aged 6-13 years), intervention (free apps without in-app purchases that contain GFs for schoolchildren to learn CPR), and outcomes (app contents for teaching CPR by emergency health care professionals) [38].

    Our search strategy was conducted in three rounds. First, the apps were searched using search strings (cardiopulmonary resuscitation, CPR BLS, CPR BLS kids game, CPR BLS game, and CPR BLS kids). After removing duplicates, the apps were screened based on their title, icons, screenshots, photography, pictures, videos, and descriptions by 2 independent investigators. During this round, apps were excluded based on three criteria: (1) irrelevant to BLS, m-learning, and gamification; (2) irrelevant to BLS, relevant to m-learning and gamification; and (3) irrelevant to m-learning and gamification, relevant to BLS. In the third round, the apps were downloaded onto the Samsung Galaxy S8 (Android 9.0 Pie) and iPhone 7 (iOS 12.3.1 Apple Inc) mobile phones for Android and iOS apps, respectively, and content was fully reviewed. During this round, apps were excluded based on nine criteria: (1) without or only one GF, (2) irrelevant to BLS, (3) need of specific equipment, (4) without app interaction, (5) technical problems, (6) not for free, (7) not available, (8) not targeting schoolchildren age, and (9) not in English. To ensure consistency, when discrepancies arose, a consensus was reached through discussion by the researchers. The PRISMA flow diagram [35] was used to represent the selection of the included and excluded apps (Figure 1). If the apps were found in both the Google Play Store [36] and Apple App Store [37], they were reviewed in the Google Play Store [39].

    ‎
    Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram of app selection. BLS: basic life support.
    View this figure

    Evaluator’s Recruitment

    The inclusion criteria for the selection of emergency health care professionals were as follows: above 18 years, CPR training certification by an established medical association, and more than 5 years of experience teaching emergency medicine. A total of 15 emergency health care professionals rated each app independently in a laboratory environment using two validated rating tools, the user version of the Mobile Application Rating Scale (uMARS) [40] and System Usability Scale (SUS) [41]. The duration of each app review was recorded, and 3 additional investigators with expertise in consumer health informatics and emergency medicine reviewed each app and rated them for BLS content.

    Ethics Approval

    The study was conducted in a central European country (Slovenia). All emergency health care professionals signed informed consent to participate, and ethics approval was obtained from the two Health Care Centre in the north-eastern part of Slovenia.

    Measures and Rating Tools

    Rating Tools for App Quality and Usability

    uMARS provides a multidimensional measure of performance indicators, functionality, esthetics, and quality of information. Apps’ subjective quality was not assessed. All items were rated on a 5-point Likert scale ranging from 1 (inadequate) to 5 (excellent). The SUS includes 10 statements on a 5-point Likert scale with both positive and negative statements about usability. The total score for the SUS is 100 points, which is divided into six usability categories including worst imaginable (0-25 SUS score), poor (25.1-51.6 SUS score), ok (51.7-62.6 SUS score), good (62.7-72.5 SUS score), excellent (72.6-84.0 SUS score), and best imaginable (84.1-100 SUS score) [42-44]. The uMARS and SUS have been used in similar studies [31,32,34,45-47].

    Rating Tool for BLS-Related Content Based on CPR Evidence

    In total, 3 investigators evaluated evidence-based BLS in each app using the ERC [9,48], AHA [10,49], and Australian Resuscitation Council [50] guidelines for teaching BLS. On the basis of these guidelines, the team identified 17 discrete BLS contents and divided them into five groups based on the Slovenian Resuscitation Council five-finger BLS teaching mnemonic for teaching first responders BLS [51]: (1) safety (1 item, BLS1), (2) consciousness (2 items, BLS2-3), (3) breathing and call (4 items, BLS4-7), (4) CPR (9 items, BLS8-16), and (5) defibrillation (1 item, BLS17), as shown in Multimedia Appendix 2 [9,10,48,49,51]. The five-finger BLS teaching mnemonic is based on two well-known ways of remembering CPR: (1) DRSABCD (pronounced drs A-B-C-D; danger, responses, send, airway, breathing, CPR, defibrillation) [52] action plan and (2) chain of survival [53]. The scoring system included one point if the BLS content was correctly implemented based on the BLS guidelines. We used a digital metronome to compare the frequency of chest compressions in apps.

    Rating Tool for GFs in Apps

    We modified gamification taxonomy [54,55] into five gamification groups, where each group represented different GFs: (1) ecological (4 GFs, GF1-4), (2) social (3 GFs, GF5-7), (3) personal and fictional (7 GFs, GF8-14), (4) performance (8 GFs, GF15-22), and (5) educational (4 GFs, GF23-26). The purpose of this gamification taxonomy is to evaluate m-learning environments such as apps. Each gamification group has a different relationship with the environment and learners in the form of implementation (ecological group), interaction (social group), usage (personal and fictional group), response (performance group), and knowledge (educational group). A total of 26 GFs were included in the final gamification taxonomy, as presented in Multimedia Appendix 3 [54,55]. The gamification rating was classified using a dummy coding [56] by 2 investigators, one point for inclusion of gamification taxonomy, no points for no gamification taxonomy, and 0.5 points for partial implementation of gamification taxonomy.

    Data Analysis

    We used Microsoft Office Professional 2016, R (version 3.6.0, R Foundation for Statistical Computing), SPSS Statistics for Windows (version 27.0, IBM Corp), and Inkscape 1.0 (Inkscape Developers, GNU General Public License) to analyze and visualize the results. The interrater reliability for uMARS and SUS was calculated using the intraclass correlation coefficient (ICC2, k; intraclass correlation coefficient, two-way random, average measures, and absolute agreement) [57,58].


    Overview

    We identified 1207 apps. The PRISMA flow diagram presents the process of selecting and scanning apps using the inclusion and exclusion criteria. After removing duplicates from multiple search strings from web-based mobile smartphone stores, 63.29% (764/1207) apps remained; 4.88% (59/1207) apps were relevant to BLS, m-learning, and gamification. After applying all the inclusion and exclusion criteria, 0.49% (6/1207) apps were included in the final evaluation (Figure 1).

    All apps were classified into the educational category by the Google Play Store. Half of the apps had a disclaimer that the app was made for educational purposes only and was not a substitute for accredited BLS training. Only 2 apps required registration. Apps were developed across multiple countries, including Australia, Italy, Finland, China, and the United States. All apps were developed in collaboration with one or more health care organizations (Table 1 and Multimedia Appendix 4).

    Table 1. Description of the included apps.
    Full App NameHealth care organization collaboratorCountry, BLSa guideline organizationDescription of BLS scenarios
    First Aid Action Hero [59]St John Ambulance Australia (Victoria)Australia (St John Ambulance Australia)One scenario in which the user performed BLS, first on a conscious and second on an unconscious animated cartoon figure
    CPR APP [60]Emergency Medicine Unit, Li Ka Shing Faculty of Medicine, The University of Hong KongUnited States (American Heart Association)No scenario. In the simulation environment, the user performed BLS on an animated human figure
    Everyday Lifesaver [61]Life Saving Victoria LimitedAustralia (St John Ambulance Australia)Three scenarios in which the user performed BLS on a drowned adult, a drowned child, and an unconscious animated cartoon figure
    A Breathtaking Picnic [62]The Italian Resuscitation CouncilItaly (The Italian Resuscitation Council)Two scenarios in which the user performed BLS on an animated animal that was choking and one that experienced cardiac arrest
    ReLIVe Responder [63]The University of Pittsburgh, Department of Emergency MedicineUnited States (American Heart Association)Two scenarios in which the user performed BLS on an unconscious and a conscious animated human figure
    Responder Rescuebusters: Fire and First-Aid [64]Emergency Response Centre Agency Finland, Finnish Recovery Council, Finnish Fire Officers’ Association’sFinland (Finnish Recovery Council)One scenario in which the user performed BLS on an animated human figure that experienced cardiac arrest

    aBLS: basic life support.

    Most apps targeted children aged above 4 years of age, and 1 app—A Breathtaking Picnic [62]—targeted schoolchildren aged between 6 and 8 years. All the apps had a Pan European Game Information 3 certificate [65]. According to game genres [65], 3 apps were developed as animated tutorials, 2 apps were developed as simulations, and 1 app was developed as a virtual world; 2 apps represented the first responder as virtual characters (eg, animated boy or animal), and an animated victim was included in each app (Table 1).

    uMARS Quality and SUS Usability Rating

    A total of 15 emergency health care professionals participated (3 females and 12 males) in evaluating apps using uMARS and SUS. In total, 40% (6/15) of the participants were nurses, 27% (4/15) were nurses with a master’s degree, and 33% (5/15) were physicians. Overall, the mean age of emergency health care professionals was 36 years. All emergency health care professionals had an Advanced Life Support (ALS) certificate provided by the ERC, and their mean professional experience was 13 years. All emergency health care professionals own and were proficient daily users of mobile smartphones.

    The mean total uMARS rating of apps was 3.2/5 (95% CI 3.0-3.4), and the details across the four domains are shown in Multimedia Appendix 5. The mean testing app time was 9.5 minutes. The most time-consuming app was the Everyday Lifesaver [61] app (mean 24 minutes) because the app included multiple features for evaluation. Interrater reliability between emergency health care professionals was good for the overall uMARS score (ICC2,k 0.8, 95% CI 0.8-0.9; Tables 2 and 3; Multimedia Appendix 6) but poor for overall SUS score (ICC2,k 0.3, 95% CI 0.03-0.5). The A Breathtaking Picnic [62] app had the highest mean SUS score (54.8 points). The mean SUS score of all assessed apps was 47.1/100 (95% CI 42.1-52.1) points. The usability of the apps was rated from poor to ok. The mean SUS score of the apps is indicated by a red dashed line. The six bands in the Figure 2 indicate the six levels of SUS categories of usability.

    Table 2. User version of the Mobile Application Rating Scale results and time spent on each app.
    Full app nameuMARSa sectionTimeb

    		EngagementFunctionalityAestheticsInformationOverall app quality
    First Aid Action Hero [59]3.74.04.03.93.912
    CPR APP [60]3.13.73.23.73.47
    Everyday Lifesaver [61]3.53.03.73.53.424
    A Breathtaking Picnic [62]2.93.54.13.73.55
    ReLIVe Responder [63]2.83.63.23.53.35
    Responder Rescuebusters: Fire and First-Aid [64]3.43.43.63.23.44

    auMARS: user version of the Mobile Application Rating Scale.

    bMean time for testing apps (in minutes).

    Table 3. Mean scores and intraclass correlation coefficients of the user version of the Mobile Application Rating Scale and time spent on each app.
    VariableScore, mean (95% CI)ICC2, ka (95% CI)
    uMARSb section

    		Engagement3.2 (3.0-3.4)0.9 (0.8-0.9)

    		Functionality3.5 (3.4-3.7)0.7 (0.5-0.8)

    		Aesthetics3.6 (3.4-3.8)0.8 (0.6-0.9)

    		Information3.6 (3.4-3.8)0.8 (0.6-0.9)

    		Overall app quality3.2 (3.0-3.4)0.9 (0.8-0.9)
    Time for testing apps (min)9.2 (7.7-10.7)N/Ac

    aICC2,k: intraclass correlation coefficient; two-way random, average measures, absolute agreement.

    buMARS: user version of the Mobile Application Rating Scale.

    cN/A: not applicable.

    ‎
    Figure 2. System Usability Scale results in the form of a box plot for each review app. SUS: System Usability Scale.
    View this figure

    Evidence-Based BLS Content

    The overall evaluation of evidence-based BLS content in the apps was poor to average based on expert consensus. Within each of the five categories, there were inconsistencies regarding mapping to the ERC [9,48], AHA [10,49], and Australian Resuscitation Council [50] guidelines for teaching BLS (Figure 3 and Multimedia Appendix 2). In the safety category, 50% (3/6) of the apps included checking for safety. For consciousness, most apps (4/6, 67%) included checking for responsiveness. Within the breathing and call category, the vast majority (5/6, 83%) included calling the emergency number or asking someone to call them; however, none of the apps included placing the patient in the right recovery position. In the CPR category, half of the apps taught hands-only CPR, whereas the other half also included ventilation, which is inconsistent with the most recent BLS guidelines internationally. Only 1 app included teaching BLS using an automated external defibrillator (AED). In the 4 apps, the chest compression frequency was set to 100 beats per minute.

    ‎
    Figure 3. Basic life support groups and percentages of basic life support contents in the apps. AED: automated external defibrillator; BLS: basic life support; CPR: cardiopulmonary resuscitation.
    View this figure

    Gamification Features

    The most common GFs in apps were in personal and fictional (32%), performance (28%), and educational (22%; Figure 4 and Multimedia Appendix 3) gamification groups. One of the most integrated GFs in the apps for teaching schoolchildren BLS was feedback and sensation or stimulation (both 8%).

    ‎
    Figure 4. Gamification groups and features in all the apps. GF: gamification feature.
    View this figure

    Principal Findings

    We conducted a systematic evaluation of the quality, usability, evidence-based content, and GFs of commercially available m-learning apps for teaching guideline-directed BLS knowledge and skills to school-aged children. Overall, the quality of the apps was average based on the uMARS, the usability was poor to ok based on the SUS, and the quality of the content was poor to average in terms of alignment with international BLS guidelines, and GFs were well represented across the gamification taxonomy.

    Quality and Usability of Apps

    Many of the apps analyzed in this review were not high-quality apps according to the uMARS tool. Overall, the lowest mean uMARS score was represented in the engagement section, evaluating entertainment, interest, customization, interactivity, and target group. Future apps could learn from this review by ensuring that user engagement is prioritized during the development phase. From a customization perspective, some personal app options (eg, selecting gender or adding names of a player) or BLS options such as changing BLS victims or scenarios, including or excluding ventilation as a part of CPR, and varying chest compression frequency should be added. Nevertheless, the information section represented the highest uMARS score, and app developers should consider adding more relevant evidence-based BLS contents to BLS m-learning apps.

    Relevant to functionality and esthetics, schoolchildren prefer visually attractive apps [22] with high levels of interactivity [66-68] and a relatable storyline (eg, bicycle accident) that evokes empathy for the victim [26]. However, in this study, the quality of visual esthetics was inversely proportional to the app’s learnability and usability. Of the 6 apps examined with the SUS tool, the app A Breathtaking Picnic [62] had the highest mean usability score. Educators must consider a few critical aspects of choosing the most usable m-learning app for teaching BLS to schoolchildren. Currently, there is no single app available that is appropriate for teaching BLS to all children aged 8-18 years. On the basis of this review, we recommend using the A Breathtaking Picnic [62] app (54.8 points) for teaching schoolchildren aged 6-8 years, the First Aid Action Hero [59] app (40.1 points) for children aged 8-10 years, and the Everyday Lifesaver [61] app (45.7 points) for children aged 11-18 years. Apps with SUS scores higher than 82 have a considerable chance of being recommended to a colleague [32]; in this review, none of the apps achieved this score.

    Evaluation of Evidence-Based BLS Content

    This review represented the poor-to-average quality of BLS content regarding international BLS guidelines [9,10,48-50].

    Five-Finger BLS Teaching Mnemonic

    We classified BLS training for schoolchildren using a modified version of the five-finger BLS teaching mnemonic [26,49], including (1) safety, (2) consciousness, (3) breathing and call, (4) CPR, and (5) defibrillation (Figure 3).

    Safety

    In terms of safety, only half of the apps were designed to check whether the area was safe before approaching the victim. All BLS guidelines reinforce the importance of ensuring safety for first responders, victims, and bystanders [9,10,48-50].

    Consciousness

    Consciousness, in most apps, was assessed by checking the victim’s responsiveness to the question, “Are you all right?” and gently shaking the victim. For example, in the Everyday Lifesaver [61] app, responsiveness is taught using the acronym COWS [50,69] (“Can you hear me?”, “Open your eyes,” “What’s your name?”, and “Squeeze my hand”). Studies have shown that schoolchildren do not have problems in correctly assessing consciousness [24,26,70,71].

    Breathing and Call

    As recommended in the CPR guidelines [9,10,48-50], for the breathing and call category, the head tilt–chin lift maneuver is generally well taught in the apps, except when the jaw has to be lifted upward to bring the chin forward and the teeth almost to occlusion. Most apps included calling the emergency number or asking somebody to call them; however, none of the apps correctly showed the process of moving the victim into the right recovery position or turning away from the rescuer (Figure 3). A problem with most apps is teaching the look, listen, and feel method for signs of breathing discretely. Importantly, in the context of the COVID-19 pandemic, the head tilt–chin lift maneuver and look, listen, and feel method is no longer recommended in the 2020 guidelines [72]. Most apps do not emphasize abnormal types of breath; only the ReLIVe Responder [63] app and Everyday Lifesaver [61] app provide information about agonal breathing or gasping or gurgling, as was shown in a study conducted in 2018 [32].

    Most apps correctly demonstrated how to make an emergency service call using the speakerphone function. For example, in the Everyday Lifesaver [61] app game scenario, the mock operator’s questions are based on the 5 Ps (place, phone number, problem, people, and progress). Most schoolchildren can correctly remember the information needed to make emergency calls [73-75].

    CPR
    Chest Compression

    The most common BLS content included in apps was chest compressions. According to the modified CPR guidelines [72], hands-only CPR is recommended to decrease the risk of COVID-19 infection. Chest compressions were typically indicated by a circle on the chest for each compression. The major problem is that the area for pressing frequently does not correspond with the correct anatomical location, and this could provide users with misleading visual information, especially schoolchildren. As such, the biggest limitation of the apps overall is the gestural design of chest compressions, especially because there is no universal gestural design. In addition, GFs focus on users’ attention to pressing a specific circle rather than focusing on the victim. Similarly, in a real scenario, there are no clear indicators of where to compress, and not knowing appropriate anatomical landmarks could create confusion. This is further complicated by the fact that most of the victims were cartoon animals, so specific locations for chest compressions are unclear when translated to humans.

    An alternative approach to indicating the point for compression using a circle is to interact with the phone by holding the smartphone in the palm, facing up, and moving it up and down with the chest compressions. This method facilitates immediate visual feedback on how chest compressions should be performed. A limitation of the apps was that the compression site was inconsistently labeled, leading to inaccurate hand and arm positions. According to a study conducted in 2019, consistency with hand and arm positions is critical for chest compression accuracy [76]. According to the CPR guidelines [9,10,48-50], chest compressions should be at least 5-6 cm in depth, at a rate of 100-120 compressions per minute (2 per second). In general, the apps provided appropriate BLS information regarding the depth and rate of chest compression. On average, the frequency of chest compressions was set to 100 beats per minute and could not be changed to higher frequency rates. Only a few apps emphasize chest recoil and fraction. One of the challenges for schoolchildren is having the strength to perform chest compressions [77]. Even if a child is not physically able to perform chest compressions, they can still learn the fundamentals of BLS and are capable of learning comprehensive BLS content and selecting skills [29,78].

    Ventilation

    Our results indicate that half of the apps do not include steps for ventilation when teaching CPR. Those that do include ventilation provide accurate BLS contents about how to perform mouth-to-mouth ventilation. However, ventilation volume and verification of chest rising are poorly integrated into the ventilation part of CPR. A study from 2019 indicated that teaching schoolchildren ventilation requires more teaching time, and it is harder to establish good quality BLS results [76]. Apps that teach ventilation as a part of CPR are more time-consuming; however, they also adhere more closely to the BLS guidelines. In addition, according to the modified CPR guidelines, mouth-to-mouth ventilation is not recommended to decrease the risk of COVID-19 infection [72].

    Defibrillation

    Overall, the use of AEDs was poorly represented. Only the Everyday Lifesaver [61] app included AEDs for teaching schoolchildren BLS training. A study reported that using an app that provides AEDs may be beneficial in terms of performance and security but at the cost of delivering a shock [79]. However, a small proportion of the schoolchildren without previous training could use an AED correctly in less than 3 minutes following the device’s acoustic and visual instructions [80].

    GFs in Apps

    Recently, gamified m-learning has become increasingly popular in various medical and educational contexts, including BLS training [81-83]. Through gamification, not only can apps create a mindset that encourages schoolchildren to try new things without being afraid of failing [84,85] but it also enables schoolchildren to engage in the learning experience.

    Personal and fictional, educational, and performance GFs were the most represented in apps for teaching BLS to schoolchildren. Gamification audio or visual BLS feedback features, levels and stages in the way of BLS steps, and sensation in a sort of stimulation were well integrated into apps for teaching schoolchildren BLS. Knowledge retention plays an important role in teaching BLS [22,86-88]; only two apps (eg, the Everyday Lifesaver [61] app) included knowledge retention in the form of repeated BLS content. It is recommended that retention content be integrated into apps because of the rapid deterioration of BLS skills after training.

    Schoolchildren today have high smartphone literacy, but less is known about educators. To use m-learning to teach BLS to schoolchildren, educators must feel confident about the platform. Some resuscitation councils, such as the Italian Resuscitation Council [89] or recently ERC [90], have already recognized that m-learning is a new trend in education and are starting to emphasize m-learning in the future resuscitation teaching guidelines. The most recent ERC draft guidelines [91] recommend that schoolchildren need supervision for learning BLS and that schoolteachers are more appropriately positioned to teach schoolchildren than health care professionals [92,93].

    Using gamified learning features, educators can expect changes in psychomotor, cognitive, and affective learning outcomes [82,83,94]. A systematic review [65] demonstrated that knowledge acquisition and retention of content, productive learning experience, and motor skills are all improved when GFs are incorporated into m-learning. Therefore, learning BLS should include both knowledge transfer and the motivation to perform BLS. We conclude that m-learning has the potential to be used to enhance BLS education for schoolchildren by improving the retention of BLS knowledge.

    Limitations

    We deliberately selected emergency health care professionals and not schoolteachers to review the apps because we were focused on adherence to evidence-based guidelines for educational purposes. There were also a small number of final apps, probably as a result of our prespecified criteria, including focusing on a target population below 13 years of age and free apps. In addition, a few apps were excluded because of limitations in language design (eg, Held: Reanimatie Game, in Dutch) [95] and the area where the study was conducted (eg, the First Aid Skills app is available only in Australia) [96]. Finally, all of the apps were developed before the COVID-19 pandemic; therefore, COVID-19–specific modifications to BLS content were not included.

    Future Research

    The results of this study provide opportunities for developing an app for teaching BLS to schoolchildren. The First Aid Action Hero [59] app and A Breathtaking Picnic [62] app have potential to be part of a randomized controlled study in which the effects of m-learning on knowledge retention, motor skills, and motivation to perform BLS can be evaluated.

    Conclusions

    Our study represents an opportunity to include m-learning apps for teaching BLS to schoolchildren. Using an adapted five-finger BLS teaching mnemonic and m-learning with GFs, there is tremendous potential for teaching BLS to schoolchildren to improve survival rates of cardiac arrest.

    Acknowledgments

    This study was partially supported by the Digital toolbox for innovation in nursing education (I-BOX), Improving Health Care Students’ Competences for Behaviour Change to Effectively Support Self-care in Chronic Diseases (Train4Health), which is cofunded by the European Union Erasmus+ program and the Slovenian Research Agency (grant N2-0101 and P2-0057). In addition, RMC is supported by the National Institute of Nursing Research (R00NR016275). The authors would like to thank all the emergency health care professionals (Vrečar V, Borovnik Lesjak V, Vec M, Kramberger A, Vihar D, Moravs D, Koželj A, Lešnik D, Lešnik B, Mažič M, Nežmah J, Zabukovšek D, Vitka V, and Petrovčič R) for participating in the app evaluation.

    Authors' Contributions

    This study was conducted with collaboration among all authors. NF developed the study design and supervised the study. NF, LG, and RMC drafted the manuscript. NF, LG, ND, and GS collected and analyzed the data. NF, MS, and PS interpreted the results from a BLS point of view. RMC, DE, and BC conducted a comprehensive review of the content. All authors read, revised, and approved the final manuscript.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

    DOCX File , 35 KB
    ‎
    Multimedia Appendix 2

    Basic life support groups and contents.

    DOCX File , 27 KB
    ‎
    Multimedia Appendix 3

    Gamification groups and features.

    DOCX File , 26 KB
    ‎
    Multimedia Appendix 4

    Description of the included apps.

    DOCX File , 14 KB
    ‎
    Multimedia Appendix 5

    User version of the Mobile Application Rating Scale scores and sections.

    DOCX File , 20 KB
    ‎
    Multimedia Appendix 6

    User version of the Mobile Application Rating Scale and time spent on each app results.

    DOCX File , 15 KB

    References

    1. Beck B, Bray J, Cameron P, Smith K, Walker T, Grantham H, Aus-ROC Steering Committee. Regional variation in the characteristics, incidence and outcomes of out-of-hospital cardiac arrest in Australia and New Zealand: results from the Aus-ROC Epistry. Resuscitation 2018 May;126:49-57 [FREE Full text] [CrossRef] [Medline]
    2. Kudenchuk PJ, Sandroni C, Drinhaus HR, Böttiger BW, Cariou A, Sunde K, et al. Breakthrough in cardiac arrest: reports from the 4th Paris International Conference. Ann Intensive Care 2015 Dec 17;5(1):22 [FREE Full text] [CrossRef] [Medline]
    3. Wong CX, Brown A, Lau DH, Chugh SS, Albert CM, Kalman JM, et al. Epidemiology of sudden cardiac death: global and regional perspectives. Heart Lung Circ 2019 Jan;28(1):6-14. [CrossRef] [Medline]
    4. Abhilash S, Namboodiri N. Sudden cardiac death - historical perspectives. Indian Heart J 2014 Jan;66 Suppl 1:4-9 [FREE Full text] [CrossRef] [Medline]
    5. Murugappan M, Boon H, Raj AN, Krishnan G, Palanisamy K. ECG morphological features based sudden cardiac arrest (SCA) prediction using nonlinear classifiers. In: Sengodan T, Murugappan M, Misra S, editors. Advances in Electrical and Computer Technologies. Singapore: Springer; 2020:789-801.
    6. Hasselqvist-Ax I, Riva G, Herlitz J, Rosenqvist M, Hollenberg J, Nordberg P, et al. Early cardiopulmonary resuscitation in out-of-hospital cardiac arrest. N Engl J Med 2015 Jun 11;372(24):2307-2315. [CrossRef]
    7. Perkins GD, Lockey AS, de Belder MA, Moore F, Weissberg P, Gray H, Community Resuscitation Group. National initiatives to improve outcomes from out-of-hospital cardiac arrest in England. Emerg Med J 2016 Jul 23;33(7):448-451 [FREE Full text] [CrossRef] [Medline]
    8. Sasson C, Rogers MA, Dahl J, Kellermann AL. Predictors of survival from out-of-hospital cardiac arrest: a systematic review and meta-analysis. Circ Cardiovasc Qual Outcomes 2010 Jan;3(1):63-81. [CrossRef]
    9. Monsieurs KG, Nolan JP, Bossaert LL, Greif R, Maconochie IK, Nikolaou NI, ERC Guidelines 2015 Writing Group. European Resuscitation Council Guidelines for Resuscitation 2015: Section 1. Executive summary. Resuscitation 2015 Oct;95:1-80. [CrossRef] [Medline]
    10. Neumar RW, Shuster M, Callaway CW, Gent LM, Atkins DL, Bhanji F, et al. Part 1: Executive Summary: 2015 American Heart Association Guidelines update for cardiopulmonary resuscitation and emergency cardiovascular care. Circulation 2015 Oct 14;132(18 suppl 2):315-367. [CrossRef]
    11. Perkins GD, Brace-McDonnell SJ, OHCAO Project Group. The UK Out of Hospital Cardiac Arrest Outcome (OHCAO) project. BMJ Open 2015 Oct 01;5(10):e008736 [FREE Full text] [CrossRef] [Medline]
    12. Meaney PA, Bobrow BJ, Mancini ME, Christenson J, de Caen AR, Bhanji F, et al. Cardiopulmonary Resuscitation Quality: improving cardiac resuscitation outcomes both inside and outside the hospital a consensus statement from the American Heart Association. Circulation 2013 Jul 23;128(4):417-435 [FREE Full text] [CrossRef] [Medline]
    13. Böttiger BW, Van Aken H. Kids save lives - Training school children in cardiopulmonary resuscitation worldwide is now endorsed by the World Health Organization (WHO). Resuscitation 2015 Sep;94:5-7. [CrossRef] [Medline]
    14. Böttiger BW, Van Aken H. Training children in cardiopulmonary resuscitation worldwide. Lancet 2015 Jun;385(9985):2353. [CrossRef]
    15. Böttiger BW, Bossaert LL, Castrén M, Cimpoesu D, Georgiou M, Greif R, Board of European Resuscitation Council (ERC). Kids Save Lives - ERC position statement on school children education in CPR.: "Hands that help - Training children is training for life". Resuscitation 2016 Aug;105:1-3. [CrossRef] [Medline]
    16. Bohn A, Van Aken HK, Möllhoff T, Wienzek H, Kimmeyer P, Wild E, et al. Teaching resuscitation in schools: annual tuition by trained teachers is effective starting at age 10. A four-year prospective cohort study. Resuscitation 2012 May;83(5):619-625. [CrossRef] [Medline]
    17. Wissenberg M, Lippert FK, Folke F, Weeke P, Hansen CM, Christensen EF, et al. Association of national initiatives to improve cardiac arrest management with rates of bystander intervention and patient survival after out-of-hospital cardiac arrest. J Am Med Assoc 2013 Oct 02;310(13):1377-1384. [CrossRef] [Medline]
    18. Liu Y, Li H, Carlsson C. Factors driving the adoption of m-learning: an empirical study. Comput Edu 2010 Nov;55(3):1211-1219. [CrossRef]
    19. Pedro LF, Barbosa CM, Santos CM. A critical review of mobile learning integration in formal educational contexts. Int J Educ Technol High Educ 2018 Mar 15;15(1):10. [CrossRef]
    20. Deterding S, Dixon D, Khaled R, Nacke L. From game design elements to gamefulness: defining "gamification". In: Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. 2011 Presented at: MindTrek '11: Academic MindTrek 2011; September, 2011; Tampere Finland p. 9-15. [CrossRef]
    21. King D, Greaves F, Exeter C, Darzi A. 'Gamification': influencing health behaviours with games. J R Soc Med 2013 Mar;106(3):76-78. [CrossRef] [Medline]
    22. Mkpojiogu EO, Hussain A, Hassan F. A systematic review of usability quality attributes for the evaluation of mobile learning applications for children. AIP Conf Proc 2016 2018:020092 [FREE Full text] [CrossRef]
    23. Nagy R, Bánfai-Csonka H, Musch J, Betlehem J, Bánfai B. Comparison of effectiveness of video-based only, video-based with self instruction kit and conventional resuscitation training for primary school children. Resuscitation 2020 Oct;155:S9. [CrossRef]
    24. Bánfai B, Deutsch K, Pandur A, Bánfai-Csonka H, Betlehem J. Preliminary results of teaching first aid to 5-6 year old children - a longitudinal study. Kontakt 2018 Jun 26;20(2):120-125. [CrossRef]
    25. Otero-Agra M, Varela-Casal C, Castillo-Pereiro N, Casillas-Cabana M, San Román-Mata S, Barcala-Furelos R, et al. Can we train the chain of survival while playing? Validation of the tool «Rescube». Anales de Pediatría (English Edition) 2021 Apr;94(4):213-222. [CrossRef]
    26. Bollig G, Myklebust AG, Østringen K. Effects of first aid training in the kindergarten - a pilot study. Scand J Trauma Resusc Emerg Med 2011 Feb 28;19(1):13 [FREE Full text] [CrossRef] [Medline]
    27. Del Pozo FJ, Alonso JV, Velis NB, Barahona MM, Siggers A, Lopera E. Basic life support knowledge of secondary school students in cardiopulmonary resuscitation training using a song. Int J Med Educ 2016 Jul 20;7:237-241 [FREE Full text] [CrossRef] [Medline]
    28. Nakagawa NK, Oliveira KM, Lockey A, Semeraro F, Aikawa P, Macchione M, et al. Effectiveness of the 40-minute handmade manikin program to teach hands-on cardiopulmonary resuscitation at school communities. Am J Cardiol 2021 Jan 15;139:126-130. [CrossRef] [Medline]
    29. Weidenauer D, Hamp T, Schriefl C, Holaubek C, Gattinger M, Krammel M, et al. The impact of cardiopulmonary resuscitation (CPR) manikin chest stiffness on motivation and CPR performance measures in children undergoing CPR training - a prospective, randomized, single-blind, controlled trial. PLoS One 2018 Aug 16;13(8):e0202430 [FREE Full text] [CrossRef] [Medline]
    30. Greif R, Lockey AS, Conaghan P, Lippert A, De Vries W, Monsieurs KG, Education and implementation of resuscitation section Collaborators, Collaborators. European Resuscitation Council Guidelines for Resuscitation 2015: Section 10. Education and implementation of resuscitation. Resuscitation 2015 Oct;95:288-301. [CrossRef] [Medline]
    31. Kalz M, Lenssen N, Felzen M, Rossaint R, Tabuenca B, Specht M, et al. Smartphone apps for cardiopulmonary resuscitation training and real incident support: a mixed-methods evaluation study. J Med Internet Res 2014 Mar 19;16(3):e89 [FREE Full text] [CrossRef] [Medline]
    32. Metelmann B, Metelmann C, Schuffert L, Hahnenkamp K, Brinkrolf P. Medical correctness and user friendliness of available apps for cardiopulmonary resuscitation: systematic search combined with guideline adherence and usability evaluation. JMIR Mhealth Uhealth 2018 Nov 06;6(11):e190 [FREE Full text] [CrossRef] [Medline]
    33. Creber RM, Maurer MS, Reading M, Hiraldo G, Hickey KT, Iribarren S. Review and analysis of existing mobile phone apps to support heart failure symptom monitoring and self-care management using the mobile application rating scale (MARS). JMIR Mhealth Uhealth 2016 Jun 14;4(2):e74 [FREE Full text] [CrossRef] [Medline]
    34. Fijačko N, Gosak L, Cilar L, Novšak A, Creber RM, Skok P, et al. The effects of gamification and oral self-care on oral hygiene in children: systematic search in app stores and evaluation of apps. JMIR Mhealth Uhealth 2020 Jul 08;8(7):e16365 [FREE Full text] [CrossRef] [Medline]
    35. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    36. Google Play Store.   URL: https://play.google.com/store/apps [accessed 2020-10-15]
    37. Apple Store – Apple.   URL: https://theappstore.org/ [accessed 2020-10-15]
    38. Polit D, Beck C. Essentials of Nursing Research: Appraising Evidence for Nursing Practice. Philadelphia, Pennsylvania, United States: Lippincott Williams and Wilkins; 2018:1-512.
    39. Zapata BC, Niñirola AH, Idri A, Fernández-Alemán JL, Toval A. Mobile PHRs compliance with Android and iOS usability guidelines. J Med Syst 2014 Aug 24;38(8):81. [CrossRef] [Medline]
    40. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016 Jun 10;4(2):e72 [FREE Full text] [CrossRef] [Medline]
    41. Brooke J. SUS: A quick and dirty usability scale. In: Usability Evaluation In Industry. Boca Raton, Florida, United States: CRC Press; 1996.
    42. Bangor A, Kortum P, Miller J. Determining what individual SUS scores mean: adding an adjective rating scale. J Usab Stud 2009;4(3):114-123 [FREE Full text]
    43. Peres SC, Pham T, Phillips R. Validation of the System Usability Scale (SUS): SUS in the wild. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2013 Sep 30 Presented at: Human Factors and Ergonomics Society Annual Meeting; September 30–October 4, 2013; San Diego, CA p. 192-196. [CrossRef]
    44. Kortum P, Sorber M. Measuring the usability of mobile applications for phones and tablets. Int J Hum Comput Interact 2015 Jul 31;31(8):518-529. [CrossRef]
    45. Ahn C, Cho Y, Oh J, Song Y, Lim TH, Kang H, et al. Evaluation of smartphone applications for cardiopulmonary resuscitation training in South Korea. Biomed Res Int 2016;2016:6418710-6418718 [FREE Full text] [CrossRef] [Medline]
    46. Adam A, Hellig JC, Perera M, Bolton D, Lawrentschuk N. 'Prostate Cancer Risk Calculator' mobile applications (Apps): a systematic review and scoring using the validated user version of the Mobile Application Rating Scale (uMARS). World J Urol 2018 Apr 8;36(4):565-573. [CrossRef] [Medline]
    47. LeBeau K, Huey LG, Hart M. Assessing the quality of mobile apps used by occupational therapists: evaluation using the user version of the mobile application rating scale. JMIR Mhealth Uhealth 2019 May 01;7(5):e13019 [FREE Full text] [CrossRef] [Medline]
    48. Perkins GD, Handley AJ, Koster RW, Castrén M, Smyth MA, Olasveengen T, Adult basic life supportautomated external defibrillation section Collaborators. European Resuscitation Council Guidelines for Resuscitation 2015: Section 2. Adult basic life support and automated external defibrillation. Resuscitation 2015 Oct;95(10):81-99 [FREE Full text] [CrossRef] [Medline]
    49. Kleinman ME, Brennan EE, Goldberger ZD, Swor RA, Terry M, Bobrow BJ, et al. Part 5: Adult basic life support and cardiopulmonary resuscitation quality. Circulation 2015 Oct 14;132(18 suppl 2):414-435. [CrossRef]
    50. Guidelines. New Zealand Resuscitation Council.   URL: https://www.nzrc.org.nz/guidelines/#Sections%202-8%20Guidelines [accessed 2020-10-15]
    51. Livk M, Mesec U, Mohor M. Pet prstov - nov način temeljnih postopkov oživljanja. In: Proceedings of the 25th International Symposium on Emergency Medicine. 2018 Presented at: 25th International Symposium on Emergency Medicine; June 14–6, 2018; Portorož, Slovenia p. 325   URL: https://www.szum.si/media/uploads/files/dokoncni-program-2018.pdf
    52. Wilks J, Pendergast D. Skills for life: First aid and cardiopulmonary resuscitation in schools. Health Edu J 2017 Sep 08;76(8):1009-1023. [CrossRef]
    53. Nolan J, Soar J, Eikeland H. The chain of survival. Resuscitation 2006 Dec;71(3):270-271. [CrossRef] [Medline]
    54. Toda AM, Klock AC, Oliveira W, Palomino PT, Rodrigues L, Shi L, et al. Analysing gamification elements in educational environments using an existing gamification taxonomy. Smart Learn Environ 2019 Dec 04;6(1):1-14. [CrossRef]
    55. Nah F, Zeng Q, Telaprolu V, Ayyappa A, Eschenbrenner B. Gamification of education: a review of literature. In: International Conference on HCI in Business. Switzerland: Springer; 2014:401-409.
    56. Tinschert P, Jakob R, Barata F, Kramer J, Kowatsch T. The potential of mobile apps for improving asthma self-management: a review of publicly available and well-adopted asthma apps. JMIR Mhealth Uhealth 2017 Aug 02;5(8):e113 [FREE Full text] [CrossRef] [Medline]
    57. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86(2):420-428. [CrossRef]
    58. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016 Jun;15(2):155-163 [FREE Full text] [CrossRef] [Medline]
    59. First aid action hero. App Store Preview.   URL: https://apps.apple.com/au/app/first-aid-action-hero/id1423366252 [accessed 2020-10-15]
    60. CPR APP.   URL: https://apkpure.com/cpr-app/cocpr.teli.hku.hk [accessed 2020-10-15]
    61. Everyday lifesaver. Google Play Store.   URL: https://play.google.com/store/apps/details?id=air.com.lsv.everydaylifesaver [accessed 2020-10-15]
    62. A breathtaking picnic. Google Play Store.   URL: https://play.google.com/store/apps/details?id=com.elastico.picnic [accessed 2020-10-15]
    63. ReLIVe responder. Google Play Store.   URL: https://play.google.com/store/apps/details?id=com.simcoachgames.relive [accessed 2020-10-15]
    64. Pan European Game Information.   URL: https://pegi.info/ [accessed 2020-10-15]
    65. Hainey T, Connolly TM, Boyle EA, Wilson A, Razak A. A systematic literature review of games-based learning empirical evidence in primary education. Comput Edu 2016 Nov;102:202-223. [CrossRef]
    66. Mohammed W, Husni H. Reading apps for children: readability from the design perspective. AIP Conf Proc 2017;1891(1):020095. [CrossRef]
    67. Kwok K, Ghrear S, Li V, Haddock T, Coleman P, Birch SA. Children can learn new facts equally well from interactive media versus face to face instruction. Front Psychol 2016 Oct 25;7:1603 [FREE Full text] [CrossRef] [Medline]
    68. Pitchford NJ, Kamchedzera E, Hubber PJ, Chigeda AL. Interactive apps promote learning of basic mathematics in children with special educational needs and disabilities. Front Psychol 2018 Mar 6;9:262 [FREE Full text] [CrossRef] [Medline]
    69. The SLSA Public Safety and Aquatic Rescue Training Manual - 35th Edition (PSAR35). Surf Life Saving NSW.   URL: https:/​/static1.​squarespace.com/​static/​57a457a5414fb54f51f98e49/​t/​5f911d2e43e34c13f599ac57/​1603345737542/​PSAR35+M6+RESUSCITATION.​pdf [accessed 2021-06-01]
    70. Lubrano R, Romero S, Scoppi P, Cocchi G, Baroncini S, Elli M, et al. How to become an under 11 rescuer: a practical method to teach first aid to primary schoolchildren. Resuscitation 2005 Mar;64(3):303-307. [CrossRef] [Medline]
    71. Bollig G, Wahl HA, Svendsen MV. Primary school children are able to perform basic life-saving first aid measures. Resuscitation 2009 Jun;80(6):689-692. [CrossRef] [Medline]
    72. European Resuscitation Council COVID-19 Guidelines. European Resuscitation Council. 2020.   URL: https:/​/erc.​edu/​sites/​5714e77d5e615861f00f7d18/​content_entry5ea884fa4c84867335e4d1ff/​5ea885f34c84867335e4d20e/​files/​ERC_covid19_pages.​pdf?1588257310 [accessed 2020-10-15]
    73. Fleischhackl R, Nuernberger A, Sterz F, Schoenberg C, Urso T, Habart T, et al. School children sufficiently apply life supporting first aid: a prospective investigation. Crit Care 2009;13(4):R127 [FREE Full text] [CrossRef] [Medline]
    74. Ammirati C, Gagnayre R, Amsallem C, Némitz B, Gignon M. Are schoolteachers able to teach first aid to children younger than 6 years? A comparative study. BMJ Open 2014 Sep 19;4(9):e005848 [FREE Full text] [CrossRef] [Medline]
    75. Nord A, Svensson L, Hult H, Kreitz-Sandberg S, Nilsson L. Effect of mobile application-based versus DVD-based CPR training on students' practical CPR skills and willingness to act: a cluster randomised study. BMJ Open 2016 Apr 29;6(4):e010717 [FREE Full text] [CrossRef] [Medline]
    76. Abelsson A, Nygårdh A. To enhance the quality of CPR performed by youth layman. Int J Emerg Med 2019 Oct 07;12(1):30 [FREE Full text] [CrossRef] [Medline]
    77. Mpotos N, Iserbyt P. Children saving lives: Training towards CPR excellence levels in chest compression based on age and physical characteristics. Resuscitation 2017 Dec;121:135-140. [CrossRef] [Medline]
    78. Yeung J, Kovic I, Vidacic M, Skilton E, Higgins D, Melody T, et al. The school Lifesavers study - a randomised controlled trial comparing the impact of Lifesaver only, face-to-face training only, and Lifesaver with face-to-face training on CPR knowledge, skills and attitudes in UK school children. Resuscitation 2017 Nov;120:138-145. [CrossRef] [Medline]
    79. Carballo-Fazanes A, Jorge-Soto C, Abelairas-Gómez C, Bello-Rodríguez J, Fernández-Méndez F, Rodríguez-Núñez A. Could mobile apps improve laypeople AED use? Resuscitation 2019 Jul;140:159-160. [CrossRef] [Medline]
    80. Jorge-Soto C, Abelairas-Gómez C, Barcala-Furelos R, Garrido-Viñas A, Navarro-Patón R, Muiño-Piñeiro M, et al. Automated external defibrillation skills by naive schoolchildren. Resuscitation 2016 Sep;106:37-41. [CrossRef] [Medline]
    81. Márquez-Hernández VV, Gutiérrez-Puertas L, Garrido-Molina JM, García-Viola A, Granados-Gámez G, Aguilera-Manrique G. Using a mobile phone application versus telephone assistance during cardiopulmonary resuscitation: a randomized comparative study. J Emerg Nurs 2020 Jul;46(4):460-467. [CrossRef] [Medline]
    82. Corkery G, Dawson-Howe K. A smartphone tool for evaluating cardiopulmonary resuscitation (CPR) delivery. Prague, Czech Republic: Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP; 2019:489-496.
    83. Leary M, McGovern SK, Chaudhary Z, Patel J, Abella BS, Blewer AL. Comparing bystander response to a sudden cardiac arrest using a virtual reality CPR training mobile app versus a standard CPR training mobile app. Resuscitation 2019 Jun;139:167-173. [CrossRef] [Medline]
    84. Lee J, Hammer J. Gamification in education: what, how, why bother? Acad Exch Q. 2011.   URL: https://www.researchgate.net/publication/258697764_Gamification_in_Education_What_How_Why_Bother [accessed 2021-06-01]
    85. Papadakis S, Kalogiannakis M. Using gamification for supporting an introductory programming course. The case of classcraft in a secondary education classroom. In: Brooks AL, Brooks E, Vidakis N, editors. Interactivity, Game Creation, Design, Learning, and Innovation. Switzerland: Springer; 2018:366-375.
    86. Watanabe K, Lopez-Colon D, Shuster JJ, Philip J. Efficacy and retention of Basic Life Support education including Automated External Defibrillator usage during a physical education period. Prev Med Rep 2017 Mar;5:263-267 [FREE Full text] [CrossRef] [Medline]
    87. Cutumisu M, Patel SD, Brown MR, Fray C, von Hauff P, Jeffery T, et al. RETAIN: A board game that improves neonatal resuscitation knowledge retention. Front Pediatr 2019 Jan 31;7:13 [FREE Full text] [CrossRef] [Medline]
    88. Roppolo LP, Pepe PE. Retention, retention, retention: targeting the young in CPR skills training!. Crit Care 2009;13(5):185 [FREE Full text] [CrossRef] [Medline]
    89. Semeraro F, Scapigliati A, Rota E, De Luca M, Tammaro G, Landini C, et al. A breathtaking picnic: a fairy tale to save lives. Resuscitation 2016 Sep;106:e47. [CrossRef]
    90. Greif R, Lockey A, Conaghan P. Guidelines 2020: Chapter 12 Education. European Resuscitation Council.   URL: https://cprguidelines.eu/ [accessed 2020-10-15]
    91. Greif R, Lockey A, Breckwoldt J, Carmona F, Conaghan P, Kuzovlev A, et al. European Resuscitation Council Guidelines 2021: Education for resuscitation. Resuscitation 2021 Apr;161:388-407 [FREE Full text] [CrossRef] [Medline]
    92. Lukas R, Van Aken H, Mölhoff T, Weber T, Rammert M, Wild E, et al. Kids save lives: a six-year longitudinal study of schoolchildren learning cardiopulmonary resuscitation: who should do the teaching and will the effects last? Resuscitation 2016 Apr;101:35-40 [FREE Full text] [CrossRef] [Medline]
    93. Bohn A, Lukas RP, Breckwoldt J, Böttiger BW, Van Aken H. ‘Kids save lives’: why schoolchildren should train in cardiopulmonary resuscitation. Curr Opin Crit Care 2015;21(3):220-225. [CrossRef]
    94. Sharda R, Romano Jr NC, Lucca JA, Weiser M, Scheets G, Chung J, et al. Foundation for the study of computer-supported collaborative learning requiring immersive presence. J Manag Info Syst 2014 Dec 08;20(4):31-64. [CrossRef]
    95. Held: Reanimatie game.   URL: https://play.google.com/store/apps/details?id=nl.fuego.held [accessed 2020-10-15]
    96. First Aid Skills. Google Play Store.   URL: https://play.google.com/store/apps/details?id=com.stjohnwa.firstaid&hl=en_US&gl=US [accessed 2020-10-15]

    ‎
    AED: automated external defibrillator
    AHA: American Heart Association
    ALS: Advanced Life Support
    BLS: basic life support
    CPR: cardiopulmonary resuscitation
    ERC: European Resuscitation Council
    GF: gamification feature
    ICC: intraclass correlation coefficient
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    SUS: system usability scale
    uMARS: user version of the Mobile Application Rating Scale

    Edited by L Buis; submitted 02.11.20; peer-reviewed by T Baranowski, X Ding; comments to author 23.11.20; revised version received 12.12.20; accepted 07.05.21; published 20.07.21

    Copyright

    ©Nino Fijačko, Ruth Masterson Creber, Lucija Gosak, Gregor Štiglic, Dominic Egan, Brian Chaka, Nika Debeljak, Matej Strnad, Pavel Skok. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 20.07.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.