This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.
Military personnel have an elevated risk of sustaining mild traumatic brain injuries (mTBI) and postconcussion symptoms (PCS). Smartphone apps that provide psychoeducation may assist those with mTBI or PCS to overcome unique barriers that military personnel experience with stigma and access to health care resources.
This study aims to (1) use the Mobile Application Rating Scale (MARS) to evaluate smartphone apps purporting to provide psychoeducation for those who have sustained an mTBI or a PCS; (2) explore the relevance, utility, and effectiveness of these apps in facilitating symptom management and overall recovery from mTBI and PCS among military personnel; and (3) discuss considerations pertinent to health care professionals and patients with mTBI when considering the use of mobile health (mHealth), including apps for mTBI psychoeducation.
A five-step systematic search for smartphone apps for military members with mTBI or PCS was conducted on January 31, 2020. Cost-free apps meeting the inclusion criteria were evaluated using the MARS and compared with evidence-based best practice management protocols for mTBI and PCS.
The search yielded a total of 347 smartphone apps. After applying the inclusion and exclusion criteria, 13 apps were subjected to evaluation. Two apps were endorsed by the US Department of Veterans Affairs and the US Department of Defense; all the others (n=11) were developed for civilians. When compared with evidence-based best practice resources, the apps provided various levels of psychoeducational content. There are multiple considerations that health care professionals and those who sustain an mTBI or a PCS have to consider when choosing to use mHealth and selecting a specific app for mTBI psychoeducation. These may include factors such as the app platform, developer, internet requirement, cost, frequency of updates, language, additional features, acknowledgment of mental health, accessibility, military specificity, and privacy and security of data.
Psychoeducational interventions have a good evidence base as a treatment for mTBI and PCS. The use of apps for this purpose may be clinically effective, cost-effective, confidential, user friendly, and accessible. However, more research is needed to explore the effectiveness, usability, safety, security, and accessibility of apps designed for mTBI management.
Mobile health (mHealth) is an emerging field, with health care professionals increasingly using apps as part of clinical practice [
mTBI, also known as concussion, is defined as a temporary change in brain functioning caused by an insult to the head with a period of posttraumatic amnesia lasting less than a day [
The cause of an mTBI varies among CAF-SMs, with some occurring as a result of motor vehicle collisions, falls, sports, explosions, or other forces related to combat and military training [
Military members experience a higher incidence of posttraumatic stress disorder (PTSD), anxiety, and depression, which can have significant functional implications when co-occurring with mTBI. mTBI or TBI and mental health disorders, such as PTSD, can co-occur from the same or separate traumatic incidents [
In addition to symptoms and stressors directly attributed to mTBI, psychosocial stressors may also be experienced by military members. Such stressors may include social and geographical isolation as well as concerns regarding medical employment limitations. It is widely acknowledged that mTBI is underreported both in the CAF and other global militaries because of several factors including stigma, fear of career implications, and ignorance of the potential seriousness of mTBI [
Various interventions for treating mTBI symptoms have been studied among military populations [
Military members require psychoeducational interventions that are clinically effective, cost-effective, user friendly, available in multiple environments, secure, and confidential [
As with the civilian population, the use of health apps is becoming more widespread within military populations [
Since the first appearance of an mTBI-based app in 2009, apps specific to mTBI have been rapidly produced and evolving [
There are both potential benefits and challenges associated with using apps for health and behavioral change. Benefits include decreased stigma and improved privacy, immediate access to psychoeducational content, reduced wait times to access resources, less administrative burden for appointment scheduling, and the ability to track symptoms and share information with health care providers [
The purpose of this evidence-based app review was to (1) use the Mobile Application Rating Scale (MARS) [
An app search was conducted on January 31, 2020. The Google Search Engine, the Google Play Store (Canadian) and the Apple App Store (Canadian) were the 3 platforms used for the search. Initial search terms employed with the Google search engine included “military” and “mtbi” or “mild traumatic brain injury” or “concussion” and “apps” or “applications” or “mobile device applications.” Google was selected as the database because of its familiarity, popularity as a search tool, and accessibility from CAF computers. A Google search also provides the user with peer recommendations, which can provide more information about the usefulness of the app for specific populations and may describe features in more detail than the description provided by the 2 app stores. Searches were then conducted on the Apple App Store and the Google Play Store using the terms “concussion” and “mTBI.” The Department of Defense Mobile Health Practice Guide (3rd ed) was also utilized as a starting point; however, it had not been recently updated and did not yield any apps that were not found via the Google search [
Summary of the 5 rounds of elimination to analyze the apps. mTBI: mild traumatic brain injury.
In Round 1, the writer screened descriptions from the app stores and excluded apps that were irrelevant, which included games and health and fitness apps unrelated to mTBI. Apps were also excluded if they had an associated cost, were specifically created for use by health care clinicians, were not in English or French, and were not accessible by the author on either the Apple App Store or the Google Play store. In Round 2, apps that were meant for peer support purposes, brain games, text messaging apps, for individuals aged 18 years or younger, and/or specific to 1 symptom of mTBI were also excluded. These included eye-tracking apps, balance apps, and apps aiming to diagnose an mTBI (such as sport-specific sideline apps) that did not have an associated psychoeducational component. Apps were excluded if they did not specifically address mTBI or concussion in the description or title and/or were designed for other purposes (ie, first aid apps). This was because of the assumption that if an app is not specifically designated for concussions or mTBI and does not list this in the name or description, it is unlikely that a person would recognize and choose the app. In Round 3, the remaining apps were compared from the Apple App Store with the Google Play Store to remove duplicates. In elimination Round 4, the writer opened the apps using an iPhone 8 iOS 12.3 (Apple Inc) and Google Pixel XL with Android 9.0. Apps were then excluded if they were intended to be used alongside an in-person therapist or sports coach, or if they otherwise did not include a psychoeducational component for a patient recovering from an mTBI. If the app remained and its purpose was for mTBI detection but it had an educational component, it was included for further consideration. Round 5 involved the use of the MARS for evaluation. A literature search of the 13 apps that remained at Round 5 was conducted using the Scopus, Medical Literature Analysis and Retrieval System Online (MEDLINE), Excerpta Medica dataBASE (EMBASE), and Cumulative Index of Nursing and Allied Health Literature (CINAHL) Plus databases. The aim of this literature search was to determine whether studies using the apps had been published in the evidence-based literature.
Once the remaining 13 apps were screened and selected for inclusion in this review, they were evaluated by 3 raters using the MARS [
The app quality category is broken down into 4 subsections: (A) engagement, (B) functionality, (C) aesthetics, and (D) information (
App-specific items are available to assess how effectively the app is perceived to address or impact a targeted health behavior [
Use of the MARS to rate apps has numerous benefits. Its consideration of a health app based both on design elements (ie, color, graphic resolution, and layout) and content is essential for evaluating quality [
Three raters were trained in the MARS by watching the
The Mobile Application Rating Scale.
Sections and subsections | Question number and headings | ||
|
|||
|
A. Engagement |
Entertainment Interest Customization Interactivity Target group |
|
|
B. Functionality |
Performance Ease of use Navigation Gestural design |
|
|
C. Aesthetics |
Layout Graphics Visual appeal |
|
|
D. Information |
Accuracy of app description Goals Quality of information Quantity of information Visual information Credibility Evidence base |
|
|
|||
|
N/Aa |
Would you recommend this app to people who might benefit from it? How many times do you think you would use this app in the next 12 months if it was relevant to you? Would you pay for this app? What is your overall star rating of the app? |
|
|
|||
|
N/A |
Awareness Knowledge Attitudes Intention to change Help seeking Behavior change |
aN/A: not applicable.
The search yielded 347 apps that were subjected to 5 rounds of elimination (
Scores for each individual section of the MARS are listed in
App search results. MARS: Mobile Application Rating Scale.
Individual features of the 13 apps.
App names | Developers | Availability | Log-in | Internet | mTBIa Ax. | Mindfulness | CBTb | Goal setting | Symptom tracking | Return to activity | French | Military |
Concussion Coach | US Department of Veterans Affairs | Google Play and Apple App Store | No | No | Yes | Yes | Yes | Yes | Yes | No | No | Yes |
World Rugby Concussion App | Mobanode | Google Play and Apple App Store | No | No | Yes | No | No | No | Yes | Yes | No | No |
CDC Heads Up! Concussion and Helmet Safety App | Centers for Disease Control (CDC) | Google Play and Apple App Store | No | No | No | No | No | No | No | Yes | No | No |
Concussion Awareness | Hockey Canada | Google Play and Apple App Store | No | Yes | No | No | No | No | No | Yes | No | No |
Concussion Ed | Parachute | Google Play and Apple App Store | No | No | Yes | No | No | No | Yes | Yes | Yes | No |
LifeArmor | National Centre for Telehealth and Technology | Google Play and Apple App Store | Yes | No | Yes | Yes | Yes | Yes | Yes | No | No | Yes |
CCMI Concussion Tracker | CCM Inc | Google Play and Apple App Store | Yes | Yes | Yes | No | No | No | Yes | No | No | No |
Concussion Smart | ABI Ireland/Medtronic | Apple App Store | No | No | Yes | No | No | No | Yes | Yes | No | No |
Concussion NI | Sport Northern Ireland | Google Play | No | Yes | No | No | No | No | No | Yes | No | No |
Concussion Quick Check | American Academy of Neurology | Google Play and Apple App Store | No | No | Yes | No | No | No | No | Yes | No | No |
Concussion Info | Programming is Fun | Google Play | No | No | No | No | No | No | No | No | No | No |
How to Treat a Concussion | nermine_92 | Google Play | No | No | No | No | No | No | No | No | No | No |
PACE Concussion | PACE Concussion | Google Play | Yes | N/Ac | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A |
amTBI: mild traumatic brain injury.
bCBT: cognitive behavioral therapy.
cN/A: not applicable.
Mobile Application Rating Scale scores for each of the 13 apps.
App names | Section A: engagement | Section B: functionality | Section C: aesthetics | Section D: information | App quality | Section E: subjective | Section F: app specific |
Concussion Coach | 3.8 | 4.3 | 4.7 | 4.6 | 4.4 | 5.0 | 4.3 |
World Rugby Concussion App | 4.4 | 5.0 | 4.8 | 4.3 | 4.6 | 4.5 | 4.0 |
CDC Heads Up! Concussion and Helmet Safety App | 3.8 | 5.0 | 5.0 | 4.3 | 4.5 | 4.0 | 3.8 |
Concussion Awareness | 2.4 | 2.8 | 2.3 | 2.7 | 2.6 | 1.8 | 2.7 |
Concussion Ed | 3.6 | 4.8 | 4.3 | 4.0 | 4.2 | 3.5 | 4.0 |
LifeArmor | 3.4 | 4.0 | 3.7 | 4.0 | 3.8 | 3.0 | 3.0 |
CCMI Concussion Tracker | 3.4 | 4.5 | 4.7 | 4.1 | 4.2 | 3.3 | 3.7 |
Concussion Smart | 3.2 | 4.8 | 4.3 | 3.4 | 3.9 | 3.0 | 3.7 |
Concussion NI | 2.8 | 4.0 | 4.7 | 4.0 | 3.9 | 3.0 | 3.7 |
Concussion Quick Check | 2.6 | 3.8 | 3.0 | 3.6 | 3.3 | 2.0 | 3.0 |
Concussion Info | 2.4 | 2.3 | 2.3 | 2.3 | 2.3 | 1.8 | 2.7 |
How to Treat a Concussion | 2.6 | 3.8 | 3.3 | 2.3 | 3.0 | 1.5 | 2.7 |
PACE Concussion | N/Aa | 1.0 | 3.0 | 1.3 | 1.8 | 1.0 | 1.0 |
aN/A: not applicable.
The decision of whether to use mHealth apps, as well as selecting which apps are appropriate, can be complex even without the addition of military health care, culture, and contexts. There are multiple considerations that need to be taken into account by both the person with an mTBI and/or a clinician using an app to assist with a psychoeducational intervention. To date, most of the available literature surrounding the use of mHealth and smartphone apps for mTBI focuses on diagnostic assessment or concussion recognition, and few studies have been published regarding app quality for psychoeducation and mTBI management. On the basis of the results of this search and subsequent MARS evaluations, 5 apps demonstrated superior app quality domains, including engagement, functionality, aesthetics, and information. The 5 highest-scoring apps included
Rating apps using the MARS highlighted features of the apps that may be beneficial to individuals with mTBI. Of the top apps,
Most app users will be familiar with the 5-star rating system of the Google Play Store and the Apple App Store as a subjective manner for users to provide feedback on apps. Within this star rating system, the scores of the aforementioned top apps, as rated by the MARS, varied greatly. In the Apple App Store,
Although scoring from the MARS was helpful in assessing these health apps and providing a numeric score and ranking, there are many additional factors that need to be considered. These factors will be further discussed, including app platform, developer, internet requirement, cost, frequency of updates, language, additional features, acknowledgment of mental health, accessibility, and military specificity.
The top 5 scoring apps based on the Mobile Application Rating Scale (MARS) evaluation. Concussion Coach, World Rugby Concussion, Concussion Ed, CDC Heads Up!, and LifeArmor.
On searching both the Apple App Store and the Google Play Store, it was evident that the Google Play Store contained more apps; however, the Apple App Store produced a more refined and relevant search results (
Ideally, the developer of the app is an organization whose policies and protocols are based on evidence-based, peer-reviewed literature, such as a government or an academic institution. The most common evidence-based apps are those that were developed through nationally competitive government or research funding and have undergone rigorous research ideally with randomized control trials [
Most of the apps reviewed did not require an internet connection to function once downloaded and installed. This may be an asset for those apps intended for the military population as access to the internet may not always be available, especially when on deployment, in rural areas, or on a military exercise. Additionally, many smartphone users have limited data and are reliant on Wi-Fi. The requirement of a log-in and an account may benefit users in that they can save more of their customizations and information within the app and virtually share their progress with health care professionals. Regular access to the internet, however, may be required as well as the sharing of more personal information within the app. The top-rated apps,
Cost is also a factor in selecting the right app for health care needs, and health care professionals recommending the use of apps should consider the benefits and drawbacks of free versus paid apps. Free apps are more accessible and more likely to be used but may come with intrusive advertisements that may be distracting and confusing for someone experiencing cognitive, vestibular, or visuospatial dysfunction from mTBI. Paid apps may have fewer or no advertisements but are less likely to be downloaded [
Several apps were outdated and had not undergone a version update in several years. All apps encouraged periods of complete rest after an mTBI; however, the evidence base is evolving toward a more active approach to recovery [
Of the 13 apps reviewed, 6 contained education specific to return to activities, such as sports, work, military duty, and/or school. Of the top 5 rated apps, the
Of the 13 apps, 6 had a component of interaction with the app that allowed for customized day-to-day symptom tracking. Symptom tracking features addressed symptoms such as dizziness, headaches, balance issues, cognitive dysfunction, nausea, visual disturbances, sensitivity to noise, hearing difficulties, fatigue, sleep disturbances, and mood changes. Consistent with best practice recommendations, some apps utilized evidence-based outcome measures such as the Neurobehavioral Symptom Inventory to guide questions regarding symptom occurrence and severity [
To provide evidence-based best practice care for mTBI, it is important that a holistic multidisciplinary approach is encouraged because of the variable experiences of those who experience mTBI [
Of the 13 apps, 7 had a component of acute mTBI assessment or recognition. Several were based on well-known and validated diagnostic tools utilized in a health care context such as the Glasgow Coma Scale and the Sport Concussion Assessment Tool 3 or 5 (SCAT3/SCAT5) [
Readability and function of a concussion app needs to be appropriate for an individual experiencing mTBI symptoms; however, most apps contain layouts and fonts that would be difficult to read and process for individuals with visuospatial or cognitive symptoms of mTBI. As symptoms of concussion may include light sensitivity, difficulty reading, and visual disturbances, larger fonts, high visibility text, pictures or diagrams, and an intuitive interface are very important for usability [
As mentioned earlier, the use of “military” as a search term confounded the app search with multiple games and other nonrelevant apps. It is unrealistic to expect someone to search through all the options, especially if they are experiencing distress and cognitive dysfunction, as may be the case with military members who sustained an mTBI or a PCS and/or have a mental health condition [
Only 2 apps,
Military members are exposed to a variety of physical and psychosocial variables, which either in isolation or in combination can exacerbate the severity, longevity, and dysfunctionality of mTBI symptoms [
Blast injuries are also more unique to military populations, with a portion of the mTBI sustained by military members during OEF and OIF being potentially attributable to members being in close proximity to explosions [
There is potential for psychoeducational apps for mTBI to be utilized in a clinical setting [
Currently, health-related mTBI apps, including those providing psychoeducation and included in this review, are not formally regulated by government agencies, although
None of the apps investigated were found to have rigorous peer-reviewed research published on the app-specific effectiveness of the psychoeducational or mTBI management components. The lack of empirical research to demonstrate effectiveness may be related to the short time frame during which mHealth apps have emerged, the speed at which their availability changes as well as the focus on diagnostic apps opposed to psychoeducational apps [
The type and volume of data gathered from an electronic device when an app is downloaded and the details of the electronically signed end user license vary [
Technology acceptance and usability studies are also lacking for apps related to mTBI [
This study has a number of limitations. First, the search and identification of apps was limited to 1 day, and given the fast-paced release of new apps, it may not have captured all apps available till date. Second, there are specific concerns regarding the use of the MARS as a rating tool for health apps. The MARS does not address data sharing, security, and privacy, which are important components to consider when making decisions regarding mHealth utilization. Furthermore, the MARS involves several potentially subjective responses by the raters, most notably in the area of app subjective quality. For instance, question 2 asks, “Is the app interesting to use? Does it use any strategies to increase engagement by presenting its content in an interesting way?” [
The main strength of this study is the systematic, methodological evidence-based approach undertaken to evaluate the apps, including the use of an evidence-based tool and multiple rounds of elimination. In addition, the researchers engaged in this project have clinical experience, are employed in clinical settings, and routinely work with military and civilian populations who have sustained mTBI. They are also skilled at providing psychoeducational content to address mTBI and support subsequent recovery. The execution of the
As a component of mHealth, smartphone apps have become widely available in recent years as app technology rapidly improves. Similar to civilians, military populations have also embraced the use of health apps, which may have advantages specific to the challenges and barriers faced by military personnel, including geographical isolation and stigma. Health apps have the potential to be an engaging and accessible means of providing psychoeducational information for mTBI management. Of the 13 apps reviewed in this study, 5 (
Canadian Armed Forces
Mobile Application Rating Scale
mobile health
mild traumatic brain injury
Operation Enduring Freedom
Operation Iraqi Freedom
postconcussion symptom
posttraumatic stress disorder
Sport Concussion Assessment Tool 3
Sport Concussion Assessment Tool 5
service member
traumatic brain injury
None declared.