This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.
Military members (MMs) and public safety personnel (PSP) are vulnerable to occupational stress injuries because of their job demands. When MMs and PSP transition out of these professions, they may continue to experience mental health challenges. The development and implementation of resilience-building mobile health (mHealth) apps as an emergent mental health intervention platform has allowed for targeted, cost-effective, and easily accessible treatment when in-person therapy may be limited or unavailable. However, current mHealth app development is not regulated, and often lacks both clear evidence-based research and the input of health care professionals.
This study aims to evaluate the evidence-based quality, efficacy, and effectiveness of resilience-building mobile apps targeted toward the MMs, PSP, and veteran populations via a scoping literature review of the current evidence base regarding resilience apps for these populations and an evaluation of free resilience apps designed for use among these populations.
The studies were selected using a comprehensive search of MEDLINE, CINAHL Plus, PsycINFO, SocINDEX, Academic Search Complete, Embase, and Google and were guided by PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). A narrative synthesis of the resulting papers was performed. The Alberta Rating Index for Apps was used to conduct a review of each of the identified apps. The inclusion criteria consisted of apps that were free to download in either the Google Play Store or the Apple App Store; updated within the last 3 years; available in English and in Canada; and intended for use by MMs, veterans, and PSP.
In total, 22 apps met the inclusion criteria for evaluation. The resilience strategies offered by most apps included psychoeducation, mindfulness, cognitive behavioral therapy, and acceptance and commitment therapy. Overall, 50% (11/22) of apps had been tested in randomized controlled trials, 7 (32%) apps had been evaluated using other research methods, and 5 (23%) apps had not been studied. Using the Alberta Rating Index for Apps, the app scores ranged from 37 to 56 out of 72, with higher rated apps demonstrating increased usability and security features.
The mHealth apps reviewed are well-suited to providing resilience strategies for MMs, PSP, and veterans. They offer easy accessibility to evidence-based tools while working to encourage the use of emotional and professional support with safety in mind. Although not intended to function as a substitute for professional services, research has demonstrated that mHealth apps have the potential to foster a significant reduction in symptom severity for posttraumatic stress disorder, depression, anxiety, and other mental health conditions. In clinical practice, apps can be used to supplement treatment and provide clients with population-specific confidential tools to increase engagement in the treatment process.
Globally, military members (MMs) and public safety personnel (PSP), for example, correctional workers, dispatchers, firefighters, paramedics, and police officers, experience increased exposure to trauma and stress in their daily activities, which can affect their mental health and well-being [
MMs in the Canadian Armed Forces (CAF) are at greater risk of mental health disorders and suicide risk compared with the Canadian civilian population [
Owing to the need for mental health support among MMs, veterans, and PSP, mobile health (mHealth) apps have emerged as a portable treatment modality option [
Evidence illustrates that resilience training and interventions, primarily those focused on coping skills and self-efficacy, can work to support a decrease in psychological distress and symptoms of PTSD [
Resilience has been identified as an important factor that enables individuals adapt to and recover from emotionally, physically, and psychologically distressing situations and trauma [
Pillars of mental resilience [
The activities and skills listed in this model have been attributed to the fostering of resilience, including those used to improve emotional regulation and coping, such as mindfulness, grounding, and self-talk; positive affect, such as purposeful leisure activities; and interpersonal relationships [
The term
Specific to the MMs, PSP, and veteran populations, health care professionals (HCPs) are well-suited to use mHealth tools, such as apps, to supplement treatment and provide clients with immediate tools to help them overcome psychological impairment related to their traumas [
Although mHealth tools have significant potential, several barriers limit their full uptake in the health care system. For example, some forms of technology, though widely used, may still not be available to everyone under all circumstances, for example, locations with unreliable or reduced cellular service, limited Wi-Fi access, and financial barriers may impede their use [
The aim of this study is to evaluate the evidence-based quality, efficacy, and effectiveness of resilience-building mobile apps targeted toward the MMs, PSP, and veteran populations. This is addressed through two objectives: (1) completion of a scoping literature review of the current evidence base regarding mental health apps for these populations and (2) evaluation of common free mental health apps designed for use among these populations. We then compare and triangulate the data from these 2 approaches. The determination of these factors will aid in improving the evidence base for mHealth apps to highlight their potential use for HCPs who may be providing mental health services.
A scoping literature review was completed to explore the available literature on mHealth apps and their cultivation of resilience in MMs, PSP, and veteran populations. The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) was used to guide this scoping review and app search, both of which were conducted between December 18, 2020, and December 20, 2020 [
This literature review aimed to answer the following research question: What is the efficacy, effectiveness, and quality of mHealth apps on increasing resilience and self-regulatory strategies among MMs, PSP, and veterans?
The literature included in the search encompassed studies published from the year 2000 onward to account for the development of technology during this time. The articles had to address resilience or self-regulatory strategies. In addition, the articles were required to pertain to military, veteran, or PSP populations (
The search was limited to studies published from the year 2000 onward to include more current technology.
Included articles focused on participants aged ≥16 years.
Articles addressing resiliency, hardiness, or coping.
Data not pertaining to military populations or public service personnel.
Studies published in languages other than English.
No outcome of interest.
Keywords for the search were determined using three main concepts: specific population, resilience, and games (refer to
After the removal of duplicate articles from the search results, a minimum of 2 researchers screened each article based on their titles and abstracts to determine further eligibility for the literature review. Articles that did not meet the eligibility criteria were excluded. Conflicts were discussed and resolved via team consensus.
The screened articles were then read in full by a minimum of 2 researchers. Conflicts were discussed and resolved via team meetings and final eliminations were made. The remaining articles were included in the scoping review.
The type of evidence, population, funding, interventions, outcomes, and recommendations were extracted from each remaining article and recorded on a spreadsheet.
A narrative synthesis was performed by 3 researchers to summarize the findings of the different studies and evaluation results. Narrative synthesis refers to an approach that relies primarily on the use of words and text to summarize and explain the findings of multiple studies associated with reviews [
The identified apps were chosen through the following steps: (1) identification of the apps addressed in the literature review, (2) establishment of eligibility criteria, and (3) search of the eligible apps by name in the Apple App Store or Google Play Store. Apps were then evaluated for overall quality using the Alberta Rating Index for Apps (ARIA) [
Apps included in the study were available for download in the Apple App Store or Google Play Store, could be set up in English, and were accessible within the geographic region of Canada. Apps chosen were intended for use primarily with MMs, PSP, and veterans; however, app use and availability could also extend to civilian populations. We included only free apps because evidence indicates that although 93% of smartphone users are likely to download an app, only 35.8% would be inclined to pay for an app [
Apps that were available on the Apple App Store and Google Play Store.
Apps that were free to download.
Apps that were intended for use by military members or public safety personnel.
Apps that were not free to download.
Apps that were not available in the English language.
Apps that were not available in Canada.
Apps that were not yet released for public use or access.
The ARIA (
The care-provider version of the ARIA was chosen to review the selected apps because of its ability to be implemented by users, caregivers, and HCPs. Each identified app that met the eligibility criteria was evaluated by 2 reviewers using the ARIA. Conflicts were resolved through discussion with all researchers to determine the final ARIA score.
After searching the databases and identifying records through additional sources, a total of 691 articles were identified. Seven additional records were identified from the other sources. After the removal of 252 duplicates, 63.5% (439/691) of articles remained for title and abstract screening. A total of 52.1% (360/691) of articles were determined to be irrelevant. Full-text reads were completed on the remaining 10.7% (74/691) of articles and 6.1% (42/691) of additional articles were excluded because of differences in outcomes of interest, irrelevancy to mHealth app use, limited qualitative or quantitative data, and repetitive publications. A total of 9 studies were excluded because they did not have a relevant outcome of interest. The reasons for exclusion were that the studies involved virtual reality without an app and were specifically for outcomes related to PTSD. From these 32 articles, 22 apps were identified as meeting the inclusion criteria.
A PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) chart for the systematic review study identification, selection, exclusion, and inclusion.
Of the 22 apps identified in the scoping literature review, 11 (50%) apps had been tested in randomized controlled trials (RCTs), 7 (32%) apps using other methods, and 5 (23%) apps had not been research trialed at the time of this study. Alternative methods used to evaluate apps included a nonrandomized quasi-experimental pre–post follow-up design, pre- and posttest questionnaires, qualitative focus groups, and the Mobile App Rating Scale. Apps that were not research trialed were identified through scoping reviews. Two apps had been tested using multiple methods.
The literature indicated that out of the 22 apps, 15 (68%) used evidence-based strategies or incorporated evidence-based components within them. Among these 15 apps, 11 (73%) apps were developed using evidence-based practices as their foundation (
The resilience strategies offered by most of the apps fit within the pillars of mental resilience, including mindfulness training, psychoeducation, cognitive behavioral therapy, and acceptance and commitment therapy. Other strategies included biofeedback, sleep strategies, social engagement, mood tracking, time scheduling, and muscle relaxation techniques, such as yoga. Many of the apps included more than one strategy. Of the total 22 apps, 8 (36%) apps used mindfulness strategies, including
The evidence-based literature demonstrated that many of the apps increased resilience strategies for users as well as improved the overall aspects of mental health [
Some apps, such as
Within the studies, there were important themes that arose through narrative synthesis around user preferences. One of those themes included apps that had a sense of progression, rhythm and routine, and elements of personal causation [
The ARIA scores ranged from 37 to 56 out of 72 (
Highest scoring apps on Alberta Rating Index for Apps.
Certain items of the ARIA were identified as being particularly relevant to the MMs and PSP populations. These included security and confidentiality, trustworthiness, and usefulness and satisfaction. As security and confidentiality are a high priority for the MMs and PSP populations, the security item of the ARIA is imperative to acknowledge [
Apps were rated higher on trustworthiness if developed by reliable sources with proof of evidence. In total, 15 apps were developed by government agencies in Canada and the United States, including the Department of Veteran Affairs, the Department of Defence, the National Centre for Telehealth and Technology, and VAC. For trustworthiness,
The aim of this study is to evaluate the evidence-based quality, efficacy, and effectiveness of resilience-building mobile apps targeted toward the MMs, PSP, and veteran populations. This involved the completion of a scoping literature review of the current evidence base regarding mental health apps for these populations and the evaluation of common free mental health apps designed for use among these populations. This study aims to provide some insight into the following research question: What is the efficacy, effectiveness, and quality of mHealth apps on increasing resilience and self-regulatory strategies among MMs, PSP, and veterans?
Overall, the results of this study indicated that most of the mHealth apps reviewed were well-suited to provide resilience strategies and skills for MMs, PSP, and veterans. These apps provided skills, strategies, and services, which could be categorized into the pillars of mental resilience and other commonly accepted definitions regarding psychoeducational interventions that can foster resilience. Common resilience strategies were well represented in many of the apps, often including mindfulness, psychoeducation, and positive coping or thinking skills. Our results indicated that no app fully addressed the 6 pillars of resilience identified by the armed forces of the United Kingdom and Australia. In total, 5 apps addressed 5 of the 6 pillars, whereas 12 apps addressed 4 or more of the 6 pillars, and 20 apps addressed 2 or more of the 6 pillars. The apps that rated the highest on the ARIA were
In the evaluation of the 22 apps,
When considering evidence around the apps, it was noted that out of the 22 apps, only 11 (50%) apps had undergone evidence-based evaluation through an RCT. Although from the total 22 apps, only 11 (50%) apps were determined to be evidence-based, 15 (68%) apps had used evidence-based strategies or components within them. Some of the apps selected for evaluation in this study have not been evaluated in the evidence-based literature. This is partially a result of the large creation and turnover of mHealth apps available for download as well as the currently limited regulations guiding their development [
Although these apps incorporate strategies and skills that may assist in facilitating resilience, it must be acknowledged that there are other factors that impact their ability to increase foster resilience, such as individual motivation, education on use of the app, access to social support, and the use of apps together with an HCP or independently. In addition, care should be taken regarding the specific designation of these apps as
Mental health can impact the daily functioning of an individual, and the concept of resilience is closely tied to mental health and well-being [
Smartphones and technology are part of daily habits in the modern era, and HCPs can identify how to incorporate mHealth apps into health care settings. As MMs, PSP, and veterans face many unique challenges in terms of sudden environmental, lifestyle, and role changes, mHealth tools can present a more feasible option for access [
HCPs have a responsibility to advocate for best practices; this can be challenging with mHealth because of the high rate of app development and the inability for evidence-based practice to keep up [
This study has several strengths. Both the scoping literature review and app evaluation were conducted following a planned a priori procedure, with attention to ensuring quality control and minimizing bias. The detailed search strategy in the literature review was extensive, including 6 databases. The inclusion and exclusion criteria were determined before the study onset and adhered to throughout for both the literature review and app evaluation. We also used appropriate calibration and at least two independent reviewers for all stages of the process.
There are certain limitations to this study, which should also be acknowledged. In the literature review, only studies written in English were included. The app selection criteria were limited to apps available for download in Canada, which excluded potentially beneficial apps available in other geographic locations. In addition, as the researchers only had access to iPhones, the apps were not tested on Android devices, which could have an impact on the usability criteria.
Although the authors identified the ARIA as an appropriate evaluation tool to use, it should be acknowledged that this is still relatively new and has not yet been extensively researched, used, or validated at this point. As such, there are currently no similar studies conducted with the ARIA, with which the present results could be compared. The results listed within this study only reflect app use from the perspective of clinicians, which could create bias. It will be important in future studies to invite users from the MMs, PSP, and veteran populations to complete the
Regarding the future of mHealth and resilience, there is much work to do in the areas of research, development, and policy. First, despite its use in health care contexts, future research is required to determine how the ARIA scores correlate to app adherence, acceptance, and adoption by users as well as to health outcomes. Further comparison of the ARIA with other app evaluation tools would be valuable in understanding the utility, criterion validity, and other concepts related to its ability to rate apps from the perspective of HCPs and clients as end users.
As previously mentioned, there is a paucity of evidence-based studies on the existing apps geared toward both resilience and the specific patient population of MMs, PSP, and veterans. Although this lack of research does not necessarily indicate that apps are of poor quality, it highlights the need for further research on health app development to ensure safety, effectiveness, and efficacy. It has been identified that traditional study design and research methods may be inappropriate for the study of mHealth as technology evolves much faster than traditional evidence-based research [
Another area of study related to mHealth that is important and specific to military and PSP users is privacy, security, and confidentiality. Although there is a higher expectation of privacy for apps that involve health care information, military and PSP organizations may be subject to other restrictions on internet access that may impede the use of the app or demand higher security. Data sharing and privacy are considerations that require attention from researchers, HCPs, and the general public when deciding on which app to use or if app use is appropriate at all [
Future initiatives to assist HCPs and their clients in navigating the world of mHealth would be an asset to balance client autonomy through app literacy and would assure that apps have some level of evidence-based merit. As mHealth use is on the rise among many HCP and client populations, training to use apps to support service delivery is of utmost importance for health care organizations. The establishment of clear practice guidelines is important for both HCPs and clients, so that expectations about the usefulness and effectiveness of the app are appropriately managed. Currently, clients may have overly enthusiastic ideas about the effectiveness of these apps to develop their resilience and support their mental health, even when seeking professional mental health treatment might be necessary. Similarly, issues of risk and safety in mental health apps (including resilience) also need to be addressed.
The question of what constitutes a resilience app versus, for example, a wellness app, is also a murky territory that requires further navigation. Until a universally operationalized definition has been established for resilience, it is likely that confusion will remain regarding the codification of specific resilience skills and strategies. For example, family- and community-level factors were seldom addressed in our identified apps, despite being indicated as an important component of resilience [
Finally, the evaluation of resilience apps will remain challenging for all stakeholders and can affect the quality of the product unless all stakeholders were included in the consultation process. With the exception of the apps designed by the United States Department of Defense of Veterans Affairs, it was difficult to determine if the end user’s perspective was incorporated into the development of the app. A collaborative approach to development, using both expert and user input, has been noted in recent studies as an effective approach to increasing the success of apps [
Resilience is often targeted by HCPs through interventions that strengthen social support systems, foster greater self-concept, encourage optimism, promote the ability to reflect, and build emotional strength. Although not intended to function as a substitute for professional services and interventions, mHealth apps have the potential to foster resilience and support a significant reduction in symptom severity for OSIs, including PTSD, depression, and anxiety, in populations affected by OSIs, such as MMs, veterans, and PSP. Apps provide easy accessibility to evidence-based tools and encourage users to initiate help-seeking behaviors when stigma or uncertainty may impede the use of direct care. In clinical practice, HCPs can assist clients in identifying apps that support their habits and values and bolster participation and engagement in activities of daily living. As accessible, novel, and evidence-based interventions and resources for fostering resilience and addressing mental health become available, MMs, veterans, and PSP may be able to facilitate their healing, recovery, and growth, which would have a positive effect on their families, communities, organizations, and the public they serve.
Literature search strategy for scoping review.
Alberta Rating Index for Apps care-provider and user versions.
Alberta Rating Index for Apps score table.
Alberta Rating Index for Apps
Canadian Armed Forces
health care professional
mobile health
military member
occupational stress injury
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews
public safety personnel
posttraumatic stress disorder
randomized controlled trial
Veterans Affairs Canada
The authors wish to thank Dr Peyman Azad Khaneghah for offering the use of the Alberta Rating Index for Apps and providing support for its implementation. This study would not have been possible without the support of the Heroes in Mind, Advocacy, and Consortium research laboratory.
The Recipient is providing this report of the project results on an "as is, where is" basis and makes no representations or warranties, either express or implied, as to any matter including, without limitation, whether the project results or any part or aspect of the same will be capable of statutory protection, the existence or non-existence of competing technology, the condition, quality or freedom from error of the project results or any part thereof, any merchantability, or its fitness for any particular purpose and all warranties and conditions expressed or implied, statutory or otherwise are hereby disclaimed. Neither the Recipient nor its officers, directors, employees, students or agents will be liable for any direct, consequential or other damage suffered by anyone resulting from the development or use of the project results or any inventions, technology or product produced in the course of or using the project results. The user of this report and/or any project results contained in the report uses the report and/or project results at the user's own risk.
None declared.