Mobile health (mHealth) is an emerging field, with health care professionals increasingly using apps as part of clinical practice [1]. Apps geared toward well-being and health promotion have drastically increased in number and availability and may offer new opportunities for individuals seeking immediate and confidential health care [2,3]. One population that has the potential to benefit from the use of such apps is public safety personnel and military members [4]. Canadian Armed Forces (CAF) service members (SMs), similar to other global militaries, have significant mental and physical health challenges associated with unique occupational stressors associated with military duties [4,5]. Military service commonly involves high-risk activities during physical training, daily trade-related tasks, overseas deployment, and responses to natural disasters among others. Their involvement in such activities can increase their likelihood of sustaining physical and mental health injuries, including mild traumatic brain injury (mTBI) [5,6].

In Round 1, the writer screened descriptions from the app stores and excluded apps that were irrelevant, which included games and health and fitness apps unrelated to mTBI. Apps were also excluded if they had an associated cost, were specifically created for use by health care clinicians, were not in English or French, and were not accessible by the author on either the Apple App Store or the Google Play store. In Round 2, apps that were meant for peer support purposes, brain games, text messaging apps, for individuals aged 18 years or younger, and/or specific to 1 symptom of mTBI were also excluded. These included eye-tracking apps, balance apps, and apps aiming to diagnose an mTBI (such as sport-specific sideline apps) that did not have an associated psychoeducational component. Apps were excluded if they did not specifically address mTBI or concussion in the description or title and/or were designed for other purposes (ie, first aid apps). This was because of the assumption that if an app is not specifically designated for concussions or mTBI and does not list this in the name or description, it is unlikely that a person would recognize and choose the app. In Round 3, the remaining apps were compared from the Apple App Store with the Google Play Store to remove duplicates. In elimination Round 4, the writer opened the apps using an iPhone 8 iOS 12.3 (Apple Inc) and Google Pixel XL with Android 9.0. Apps were then excluded if they were intended to be used alongside an in-person therapist or sports coach, or if they otherwise did not include a psychoeducational component for a patient recovering from an mTBI. If the app remained and its purpose was for mTBI detection but it had an educational component, it was included for further consideration. Round 5 involved the use of the MARS for evaluation. A literature search of the 13 apps that remained at Round 5 was conducted using the Scopus, Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), and Cumulative Index of Nursing and Allied Health Literature (CINAHL) Plus databases. The aim of this literature search was to determine whether studies using the apps had been published in the evidence-based literature.

Once the remaining 13 apps were screened and selected for inclusion in this review, they were evaluated by 3 raters using the MARS [2]. The MARS was created by Stoyanov et al [2] in 2015 for developing an mHealth app rating tool that was reliable, multidimensional, and which could provide a framework to trial, classify, and rate apps. Stoyanov et al [2] developed the MARS and found a high level of interrater reliability for overall MARS scores after evaluating 50 health and well-being apps. The components of the MARS were selected following a literature review to determine existing websites and app evaluation tools and after consulting a multidisciplinary advisory team that included psychologists and mHealth developers [2]. The resulting 3 MARS categories, app quality, app subjective quality, and app specific, were chosen to merge existing mHealth evaluation criteria into a format that was not overly technical, difficult to use, or specific to any one health domain [2].

The app quality category is broken down into 4 subsections: (A) engagement, (B) functionality, (C) aesthetics, and (D) information (Table 1) [2]. Each subsection is averaged with a mean score out of 5. Engagement (A) is addressed using 5 questions and explores the ability of the apps to hold a user’s attention by rating (1) how interesting it was to use, (2) how well-tailored it was to the targeted users, (3) how fun or entertaining it was, and (4) how well it could be customized for each user. Functionality (B) has 4 questions that rate how well the app works for the user. This includes how intuitive the gestures and icons are, how well the components worked when trialed, and how easy it is for a new user to learn. Aesthetics (C) is the shortest subsection, with 3 questions addressing the quality of the visual appeal and graphics. Information (D) is the last and largest subsection, with 7 questions exploring the credibility of the developer and content in the app and whether the app’s efficacy has been tested and reported in published research. This section also addresses how the information within the app is presented to users (eg, is the information accessible to the target audience?) and if the information is of sound quality [2].

App-specific items are available to assess how effectively the app is perceived to address or impact a targeted health behavior [2]. In this study, the targeted health behavior was mTBI management. The 6 app-specific items indicated the rater’s belief that the assessed app would be able to increase the knowledge of, attitude toward, and intention to change health management strategies of someone with an mTBI. The app-specific items also address how strongly the rater felt that the use of the app would actually result in a change in this targeted health behavior. These items were rated on a scale from 1 (strongly disagree) to 5 (strongly agree).

Use of the MARS to rate apps has numerous benefits. Its consideration of a health app based both on design elements (ie, color, graphic resolution, and layout) and content is essential for evaluating quality [2]. Newly published research exploring the quality of apps intended to equip individuals to self-manage their disease reports that apps that have been tested and rated highly on the MARS tool are trustworthy for clinicians to recommend to clients [30]. The MARS was selected to facilitate evaluation of apps in this research project for these features and because it is more comprehensive than the subjective star ratings featured on app stores [4] and is customizable for this specific research focus [2].

Three raters were trained in the MARS by watching the MARS training video posted by Stoyanov on YouTube [31]. These raters included a PhD candidate and a CAF Occupational Therapist, an MSc Occupational Therapist, and an American College of Sports Medicine Certified Clinical Exercise Physiologist. The finalized app selection was evaluated by the writers using the MARS January 30, 2020, with apps downloaded onto a Samsung Galaxy Tab A SM-T350 with Android version 7.1.1 software, Google Pixel XL with Android 9.0 software, and/or iPhone 8 iOS 12.3. The results from the raters were averaged to create the final MARS score. The Ontario Neurotrauma Foundations’ Guideline for Concussion/Mild Traumatic Brain Injury and Persistent Symptoms: 3rd Edition, 5th International Conference on Concussion in Sport Concussion Consensus Statement and relevant military-specific evidence-based publications [15,18-20] were used to evaluate the quality and accuracy of the information (Information: Section D of MARS) provided within each app [7,8].

On searching both the Apple App Store and the Google Play Store, it was evident that the Google Play Store contained more apps; however, the Apple App Store produced a more refined and relevant search results (Figure 1). The requirements to publish apps vary between the Apple App Store and the Google Play Store, which influence the quality and volume of apps available. The Apple App Store has more rigorous review processes before allowing an app to be available to the public, which results in a more restrictive selection [32]. In addition, apps that score poorly in their customer five-star rating system risk being removed from the platform [32]. The Apple App Store may be a better option for finding adequate and usable apps for psychoeducation as a person with active symptoms of mTBI may struggle with the volume of results and the need to filter those that are not relevant [29]. Availability in both the Google Play Store and the Apple App Store will maximize the availability of the app to the target audience. It is evident that a clear title and description is key for finding and using an app [29].

Ideally, the developer of the app is an organization whose policies and protocols are based on evidence-based, peer-reviewed literature, such as a government or an academic institution. The most common evidence-based apps are those that were developed through nationally competitive government or research funding and have undergone rigorous research ideally with randomized control trials [2]. None of the results yielded apps that had been researched to this degree. Additionally, health care app developers do not always involve medical professionals in app development; therefore, apps may have limited referencing and make misleading claims [33]. When health providers recommend apps, they need to be competent in the evidence-based literature on the topic to evaluate the apps for the quality of the content and ensure that it is in line with the desired therapeutic interventions and outcomes [3]. Although this expectation may be realistic for health care professionals who are clinically trained in a military health care system, it could be an unattainable expectation for military members in distress from an mTBI who are exploring mobile resources for themselves [4].

Most of the apps reviewed did not require an internet connection to function once downloaded and installed. This may be an asset for those apps intended for the military population as access to the internet may not always be available, especially when on deployment, in rural areas, or on a military exercise. Additionally, many smartphone users have limited data and are reliant on Wi-Fi. The requirement of a log-in and an account may benefit users in that they can save more of their customizations and information within the app and virtually share their progress with health care professionals. Regular access to the internet, however, may be required as well as the sharing of more personal information within the app. The top-rated apps, Concussion Coach, World Rugby Concussion, CDC Heads Up!, Concussion Ed, and LifeArmor (Figure 3), did not require internet access after the app was initially downloaded to the device. In addition, they did not require a log-in or an account that negated the need for additional personal data to be stored and transmitted.

Cost is also a factor in selecting the right app for health care needs, and health care professionals recommending the use of apps should consider the benefits and drawbacks of free versus paid apps. Free apps are more accessible and more likely to be used but may come with intrusive advertisements that may be distracting and confusing for someone experiencing cognitive, vestibular, or visuospatial dysfunction from mTBI. Paid apps may have fewer or no advertisements but are less likely to be downloaded [34]. A 2015 study concluded that although 93% of smartphone users download apps, only 35.8% will purchase apps that have an associated cost [34]. All 13 apps in this study were free to download and did not provide an option to pay for an upgraded version of the app. Advertisements were either absent or subtle enough to not disrupt the user experience.

Several apps were outdated and had not undergone a version update in several years. All apps encouraged periods of complete rest after an mTBI; however, the evidence base is evolving toward a more active approach to recovery [7,17]. Although no regulations or standards exist regarding the frequency with which a developer should update an app, these apps should ideally be updated regularly enough to reflect the rapidly changing evidence base and recommendations regarding psychoeducation and mTBI management [28]. The 5 top-rated apps had received updates within 24 months of the app search, except the French version of Concussion Ed (last updated version in 2017).

Concussion Ed was the only app available in both of Canada’s official languages, English and French. All other apps reviewed were only available in English. If the CAF wishes to use a standard app across all regions, there is currently only 1 available option that will meet the official language needs of the country.

Of the 13 apps reviewed, 6 contained education specific to return to activities, such as sports, work, military duty, and/or school. Of the top 5 rated apps, the World Rugby Concussion, CDC Heads Up!, and Concussion Ed included this component and had concrete, specific examples for activity grading of frequency, volume, and intensity for both physical and cognitive tasks. The suggested return to play, school, or work guidance was consistent with evidence-based and best practice recommendations [7,8]. Guidance on return to activity is a common reason civilians and military members alike seek guidance from health care professionals [17]. This is also an area that is rapidly evolving in the literature as recent studies increasingly promote a more active recovery in contrast to previous recommendations that advocated for a longer period of complete rest immediately after sustaining an mTBI [7,8,17].

Of the 13 apps, 6 had a component of interaction with the app that allowed for customized day-to-day symptom tracking. Symptom tracking features addressed symptoms such as dizziness, headaches, balance issues, cognitive dysfunction, nausea, visual disturbances, sensitivity to noise, hearing difficulties, fatigue, sleep disturbances, and mood changes. Consistent with best practice recommendations, some apps utilized evidence-based outcome measures such as the Neurobehavioral Symptom Inventory to guide questions regarding symptom occurrence and severity [8,35]. Symptom tracking may have benefits such as allowing a patient with an mTBI to visualize improvement and better report their symptom status to their health care team. This may assist with decisions to re-engage with returning to certain activities. Conversely, frequent tracking could conceivably foster hypervigilance and preoccupation with symptoms, especially among anxious individuals [25].

To provide evidence-based best practice care for mTBI, it is important that a holistic multidisciplinary approach is encouraged because of the variable experiences of those who experience mTBI [10]. This requires that mental health distress and diagnoses, such as PTSD, anxiety, and depression, are acknowledged as potential contributors to mTBI and PCS, especially in the military context where evidence of these comorbidities is strong [15,17-20]. The majority of apps reviewed did not contain reference to mental health. Although mental health symptomology and its comorbid relationship with mTBI is a key piece of the psychoeducational strategies recommended for mTBI management and intervention, only 1 app, Concussion Coach, explicitly acknowledged the comorbidity of mTBI and mental health conditions [8,15]. Furthermore, only 2 apps, Concussion Coach and LifeArmor, provided resources for mindfulness, goal setting, and cognitive behavioral therapy, which have been identified as components of a psychoeducational or integrated behavioral health intervention for mTBI and comorbid mental health conditions [15].

Of the 13 apps, 7 had a component of acute mTBI assessment or recognition. Several were based on well-known and validated diagnostic tools utilized in a health care context such as the Glasgow Coma Scale and the Sport Concussion Assessment Tool 3 or 5 (SCAT3/SCAT5) [7,36,37]. The SCAT3/SCAT5 is a commonly used and evidence-based mTBI screen recommended for use during sport [7,37]. Of the top 5 rated apps, 3 (World Rugby Concussion, CDC Heads Up!, and Concussion Ed) included this component. As all the apps in this review are indicated for use by the general public, the mTBI assessment and recognition could carry significant legal liability, which is a concern that has been voiced in other reviews [24,25]. As stated earlier, none of the available smartphone apps in Canada and the United States have been federally regulated or approved for use in diagnosing mTBI [25,26]. The diagnostic utility of apps in mTBI detection is a controversial topic with emerging evidence-based literature [38-40].

Readability and function of a concussion app needs to be appropriate for an individual experiencing mTBI symptoms; however, most apps contain layouts and fonts that would be difficult to read and process for individuals with visuospatial or cognitive symptoms of mTBI. As symptoms of concussion may include light sensitivity, difficulty reading, and visual disturbances, larger fonts, high visibility text, pictures or diagrams, and an intuitive interface are very important for usability [41]. Only Concussion Ed had a customizable interface that allowed the user to easily enlarge the text. In the opinion of the authors, the apps that scored in the top 5 had the best visibility and usability of the 13 apps assessed. With mTBI and its symptomatology being multifaceted and complex, intervention and rehabilitation will vary widely depending on the needs of the person [20].

As mentioned earlier, the use of “military” as a search term confounded the app search with multiple games and other nonrelevant apps. It is unrealistic to expect someone to search through all the options, especially if they are experiencing distress and cognitive dysfunction, as may be the case with military members who sustained an mTBI or a PCS and/or have a mental health condition [42]. Given the rate at which new apps are entering the marketplace, the open access nature of app stores may impact the effectiveness of starting a search there; consequently, search results are not guaranteed to be consistent [42].

Only 2 apps, Concussion Coach and LifeArmor, provided military-specific resources. The majority of apps, as well as evidence-based literature on mTBI, focus on sport-related mTBI, which is common in the civilian population. Although military populations experience these types of mTBI, apps specific to military contexts would be an asset as there are multiple complexities within military organizations, cultures, occupational roles, and the environment to which civilians are not exposed [13].

Military members are exposed to a variety of physical and psychosocial variables, which either in isolation or in combination can exacerbate the severity, longevity, and dysfunctionality of mTBI symptoms [9-16,43]. Psychosocial factors that are prevalent at a higher rate in military populations include increased geographical isolation, alcohol consumption, mental health diagnoses (ie, depression, anxiety, and PTSD), chronic pain, TBI, and sleep disturbances, all of which can exacerbate mTBI symptoms [9-16]. In addition, military contexts necessitate higher levels of cognitive functioning. Cognitive dysfunction can potentially result in decreased efficiency and effectiveness, along with an increased risk of harm to self, the unit, and a mission [18]. Reduced physical, mental, and/or cognitive functioning may result in involuntary release from the CAF because of the inability to meet the Universality of Service criteria for employment, deployment, and fitness [44]. Moreover, as a CAF-SM transitions from the CAF to veteran status, PCS may continue to contribute to challenges within the transition processes, the family unit, civilian employment, leisure activities, and self-care [21].

Blast injuries are also more unique to military populations, with a portion of the mTBI sustained by military members during OEF and OIF being potentially attributable to members being in close proximity to explosions [5,9-13,43]. A blast mTBI is an injury to the brain leading to dysfunction that is the result of an explosion or a blast [13,43]. Despite differences among mechanisms of injury, no significant variations in mTBI symptoms and PCS caused by blast versus blunt force have been identified apart from a blast mTBI preceding more severe hearing loss [13]. Finally, repeated exposure to low-level head trauma, such as being in proximity to low-level blasts, use of a ram, or other weapon utilization, is another mechanism of possible head trauma that is unique to military and paramilitary personnel being explored in the literature [45]. More research is needed to determine whether these mechanisms cause differences in the presentation of mTBI and PCS, and whether military-specific mTBI needs to be addressed as a unique subset compared with civilian mTBI.

There is potential for psychoeducational apps for mTBI to be utilized in a clinical setting [29]. A health care provider could recommend a vetted app to assist with education, reassurance, and potentially behavioral change. This may reduce the need for multiple follow-up appointments and could assist with symptom reporting and monitoring [3]. Some apps have a feature that allows data to be electronically provided to the health care professional, which could reduce the administrative burden on both the health care professional and the patient as long as data sharing, privacy, and security are considered [26]. Although there could be some benefits, this area of mHealth remains novel and not without significant issues [1,27-29,33].

Currently, health-related mTBI apps, including those providing psychoeducation and included in this review, are not formally regulated by government agencies, although Concussion Ed is endorsed by Health Canada [46]. The purpose of such regulation would be to provide the consumer with confidence that the product can be safely used [26]. Until this is changed, the onus is on app developers and the app provider (ie, Apple or Google) to provide the consumer with a well-documented and described product and a clear indication as to the intended target group of the app [24]. The abundance of available health apps and the rapid rate at which they continue to be released indicate that it will take considerable time for agencies or regulatory bodies to monitor a virtual market for regulation [25,26].

None of the apps investigated were found to have rigorous peer-reviewed research published on the app-specific effectiveness of the psychoeducational or mTBI management components. The lack of empirical research to demonstrate effectiveness may be related to the short time frame during which mHealth apps have emerged, the speed at which their availability changes as well as the focus on diagnostic apps opposed to psychoeducational apps [25].

The type and volume of data gathered from an electronic device when an app is downloaded and the details of the electronically signed end user license vary [46]. App creation, app use, and data storage may all occur in different countries with different laws and regulations regarding data privacy and sharing of data collected through electronic means [47]. Data sharing and privacy is a consideration that requires attention from researchers, health care professionals, and the general public when deciding on which app to utilize or if app utilization is appropriate at all [33]. Future systems of app evaluation and research would benefit from adding a component that considers data sharing, storage, and privacy.

Technology acceptance and usability studies are also lacking for apps related to mTBI [24]. There are few early feasibility and compliance studies for mTBI apps published with small sample sizes that are not specific to the psychoeducational components of the apps [25]. Investigation into the feasibility, logistics, security, IT compatibility, and acceptance by health care providers and military members with mTBI is critical to the implementation of standardized practices in military health care systems. The rapid rate at which mHealth is evolving contrasts with the slower pace of traditional evidence-based research practices [25]. Exhaustive evaluation for effectiveness, efficacy, and usability through research may not be practical and new approaches to evidence may have to be considered [25]. New methods of research with novel tools, such as the MARS, may also need further consideration and acceptance to assist both with the rapid need for mHealth research and the regulation of health apps.

This study has a number of limitations. First, the search and identification of apps was limited to 1 day, and given the fast-paced release of new apps, it may not have captured all apps available till date. Second, there are specific concerns regarding the use of the MARS as a rating tool for health apps. The MARS does not address data sharing, security, and privacy, which are important components to consider when making decisions regarding mHealth utilization. Furthermore, the MARS involves several potentially subjective responses by the raters, most notably in the area of app subjective quality. For instance, question 2 asks, “Is the app interesting to use? Does it use any strategies to increase engagement by presenting its content in an interesting way?” [2]. As such questions may be interpreted subjectively, mitigation strategies were implemented. These included rater participation in standardized video training before rating the apps, engagement of multiple raters in the rating process, and averaging ratings across raters [2,29].

The main strength of this study is the systematic, methodological evidence-based approach undertaken to evaluate the apps, including the use of an evidence-based tool and multiple rounds of elimination. In addition, the researchers engaged in this project have clinical experience, are employed in clinical settings, and routinely work with military and civilian populations who have sustained mTBI. They are also skilled at providing psychoeducational content to address mTBI and support subsequent recovery. The execution of the a priori process for app rating, coupled with the educational and clinical experience of the researchers, contributed to the validity and rigor of the app evaluation results and subsequent knowledge synthesis of findings. As mHealth is a rapidly evolving field, the brief time from app search and evaluation to manuscript preparation facilitated timely knowledge translation and integration into clinical practice.

As a component of mHealth, smartphone apps have become widely available in recent years as app technology rapidly improves. Similar to civilians, military populations have also embraced the use of health apps, which may have advantages specific to the challenges and barriers faced by military personnel, including geographical isolation and stigma. Health apps have the potential to be an engaging and accessible means of providing psychoeducational information for mTBI management. Of the 13 apps reviewed in this study, 5 (Concussion Coach, World Rugby Concussion, CDC Heads Up!, Concussion Ed, and LifeArmor) were well-suited to provide evidence-based psychoeducational information on mTBI management. Only 2 (Concussion Coach and LifeArmor) contained information specific to military populations and addressed mental health information and strategies, which are a critical component of mTBI and PCS management and recovery [8]. Further research should investigate the applicability, technology acceptance, and usability specific to the utilization of psychoeducational apps among military members with mTBI at the patient and health professional level within military contexts such as garrison, training, and deployed environments.