Besides Coulon et al’s [20] review of stress management apps, there have been no systematic reviews that focused on the evaluation of MBSR apps for individuals with chronic lung disease. Therefore, the objective of our review was to identify and evaluate apps available in the Google Play Store (Android devices) and/or the Apple Store (for iOS-based devices) for promoting mindfulness-based strategies specifically among adults with chronic lung disease. The primary aim was to evaluate whether and to what extent the content of these apps is evidence-based and transparent in its purpose, development, and content (eg, provides contact information for its developers). The secondary aim was to evaluate the usability, readability, and cultural sensitivity of these apps. Our ultimate goal is to determine if these apps can improve the quality of life among racially and ethnically diverse populations of lung disease survivors. Furthermore, we want to make recommendations to improve these apps, so that health information can be more accessible, accurate, and effective for these populations.

A total of 19 keyword combinations were created using COPD or Lung Cancer, followed by MSBR, meditation, breathing, diaphragmatic, stress management, progressive muscle relaxation, or yoga. Each combination was searched in Google and iOS app stores in May 2017 using compatible mobile devices (see Textbox 1).

We adapted existing procedures by Coulon and colleagues [20]. We evaluated the apps for content, usability heuristics, readability, and cultural sensitivity. Evaluation was conducted in a multidisciplinary group setting involving 5 to 6 reviewers (ie, group evaluation) for content and usability. Readability and cultural sensitivity was conducted independently by 1 and 2 reviewers, respectively (ie, individual evaluation). Drawing on the work of Coulon and colleagues [20], we used 1 level of inclusion criteria. We used app descriptions to determine whether an app met 1 of the following 4 criteria: (1) available in English, (2) targeted adults with chronic lung disease, (3) was not a duplicate within or across app stores, and (4) was not a service gateway, such as required subscriptions beyond the app. The apps were downloaded for further investigation if the 4 criteria could not be determined from the description (Figure 1).

The 19 keyword combination searches yielded 768 apps—508 apps in the Apple Store and 260 in the Google Play Store. Figure 2 illustrates the procedure for determining if apps met the inclusion criteria. Of these, 37 apps were not in English (criterion 1). Of the remaining 731 apps, 639 did not target adults with chronic lung disease (criterion 2). This left 92 apps, of which 69 were duplicates of each other (ie, failed criterion 3), and 2 required additional subscriptions or purchases (thereby failing criterion 4). Thus, after the inclusion assessment, 21 apps remained. Upon downloading these apps, 4 failed to meet criterion 2 (brief or generic descriptions previously prevented this determination). Moreover, 3 additional apps failed to meet criterion 3 (also because the nature of the content had been obscured until downloading). Of the remaining 14 apps, 5 were irretrievable or malfunctioned during download (ie, failed criterion 5). In total, 9 apps were selected for further evaluation. None of the 9 remaining apps were duplicated across both app stores. In addition, all apps were free, though 1 app had the option to purchase additional breathing exercises.

We established 4 content evaluation domains: (1) evidence of science, (2) scientific strategy and engagement, (3) evidence-based stress management, and (4) transparency in its purpose, development, and content. Although all of our criteria were adapted from Coulon et al’s study, we included 2 of their inclusion criteria as evaluation criteria [20] (see Textbox 2 for domain definitions and their respective criteria). Across the 4 domains, we developed a coding sheet (converted to a Google Form) to determine whether criteria were met, somewhat met, or not met.

The multidisciplinary research team of 5 to 6 reviewers met 5 times to conduct these evaluations in a group setting. We operationalized this evaluation by downloading a given app and projecting it onto a big screen to familiarize ourselves with the app content (for 10-20 min). One reviewer was responsible for navigating through the app. Some discussion ensued if reviewers had uncertainties or questions about a given app that warranted additional explanation. Each researcher then independently scored the content in Google Forms guided by the criteria in Textbox 2. The response options for each of these criteria were as follows: met, somewhat met, or did not meet. Reviewers were also provided with a space to justify their given response. The content evaluation took 20-45 min. All results (including comments) were concatenated using Google Forms, and these results are reported based on majority consensus.

An expert in human-computer interaction prepared the usability heuristics evaluation questionnaire based on Nielsen’s 10 usability heuristics (see Table 1) [21-23]. The questionnaire was administered via Google Forms. The review team, similar to the process described above for the content evaluation, evaluated usability heuristics of each app as a group and individually assigned a numerical value to rate the severity of each heuristic violation (see Table 1 for the heuristics on which each app was evaluated, accompanied by questions and response options. In addition, similar to our content evaluation, results were determined based on majority consensus).

Readability was evaluated using the approach employed by Smith and colleagues [20]. Readability was measured using Readibility.io (computer software by Added Bytes, Sussex, England). This software provides grade-level scores according to 5 standardized reading scales (Flesch-Kincaid Grade Level, Gunning Fog Score, Coleman-Liau Index, SMOG Index, and Automated Readability Index) [24] along with an average of the 5 scores. We retrieved the average score using 125 to 150 words of text from each app.

To our knowledge, there are no validated measures for evaluating the cultural sensitivity of commercially available apps. Therefore, we adapted the Cultural Sensitivity Checklist (CSC) developed by Friedman and Hoffman-Goetz [25] to evaluate our material for cultural sensitivity for African Americans. We are interested in the apps’ sensitivity among African Americans specifically because they have higher rates of lung cancer [26] and earlier onsets of COPD [27]. The CSC checklist was designed to evaluate printed material, but has also been used for online material [28]. The original checklist contains 8 items, of which only 5 were pertinent to our study. (Two items overlapped with our content and readability evaluations and 1 focused on cancer prevention instead of symptom management.) We scored each app on the basis of whether it met, somewhat met, or did not meet the CSC criteria. To establish intercoder reliability, 2 reviewers conducted separate evaluations for each app. Percent agreement was calculated by dividing the total number of agreements by the total possible items. The 2 reviewers reached 100% agreement.

Usability varied across the apps (Table 2). Of the 10 usability heuristics, half were critically violated by a majority of apps—visibility, match between system and real world, error prevention, recognition, and help and documentation. We will focus our discussion on these 5 (see definitions in Table 1).

Visibility was problematic in 8 of the 9 apps. In particular, some apps had a complex navigation structure, including a lack of headers and feedback, leading to feeling lost in the app. Moreover, 4 apps utilized a small font size, problematic for many cancer survivors over the age of 50 years.

A mismatch between the system and the real world was another usability issue, problematic in 7 apps. Examples included the use of technical jargon, a disorganized menu, and use of advanced yoga terms not obvious in meaning to novice users.

A third usability heuristic was error prevention. Of the 9 apps, 8 contained issues related to disorganized content or functions, or buttons too close together. Furthermore, some apps that required health information (eg, blood pressure), or self-report of medication practices, did not allow users to edit information they had entered or made it very difficult to enter information (due to font size or entry fields).

All 9 apps contained issues related to recognition rather than recall. Instructions were often hard to find (an issue related to 3 other usability heuristics—visibility, help, and documentation). Some apps had so much functionality that they were overwhelming to first-time users. This led to difficulty in navigation, how to use specific functionality (eg, videos, information trackers), and even how to figure out the app’s purpose. There were 2 apps that did offer a tutorial for first-time users, and this feature mitigated some of these recognition or recall issues.

Finally, 8 of the 9 apps contained help and documentation issues. Similar to recognition issues, many of these problems stemmed from a lack of, or difficult-to-find, instructions. Furthermore, developer contact information was often unavailable, making it difficult to request help or report an issue.

Table 3 shows the results from the grade-level readability evaluation. The grade-level readability results showed that 7 of the 8 apps contained content at the ninth-grade reading level or higher. Only 1 app had a reading level below eighth grade, whereas 1 did not have enough text to calculate a readability score.

Of the 9 apps evaluated, none met any of the CSC criteria. One of the apps was predominantly intended for tracking symptoms and thus contained minimal text, making it difficult to evaluate cultural sensitivity (same app for which we could not calculate a readability score).

Of the 9 apps evaluated, 3 focused on providing support for individuals with asthma (My Breathefree, Breathcount, Asthma Tracker and Log), 2 aimed to support individuals with COPD (COPD Disease, COPD), 2 provided general relaxation and breathing training (Pranayama Free, 7pranayama—Yoga Breath Calm), and 2 contained videos and exercises to support individuals with lung cancer (Loving Meditations—Bring Calm to Cancer, Lung+ Pioneering Healthcare).

Though all 9 apps were marketed as breathing management and stress reduction, only 3 met the criteria for having both an evidence-based stress management strategy and an evidence-based stress management structure (ie, assessment, self-monitoring, or psychoeducation features; see Textbox 2). For example, Loving Meditations —Bring Calm to Cancer included 5 evidence-based stress management strategies that we assessed—meditation or mindfulness, diaphragmatic breathing, cognitive restructuring, visualization and imagery, and active coping or behavioral activation.

None of the apps fully met the criteria for providing scientific evidence to support claims about information within, or the efficacy of, their app. The 2 apps that somewhat met criteria for having scientific evidence—Lung+ Pioneering Healthcare and COPD—both contained content linked to a corporation (eg, Roche), and neither referenced peer-reviewed literature or other scientific evidence. In addition, most apps either met or partially met the criteria for being interactive or engaging, and over half of these incorporated a skill-building component. For example, in addition to offering breathing exercises and general lung education, Lung+ Pioneering Healthcare featured a multistage, interactive saxophone player breathing game that challenged players to blow rhythmically into their mobile microphones to the tune of jazz music.

Transparency was variable across the apps with Lung+ Pioneering Healthcare meeting the most criteria (5 of 8). The average grade-level readability in our review was 10th grade, which is 2 grades above the acceptable level [23]. Many of the apps had usability challenges identified as critical violations. Finally, none of the apps met any of the criteria for being culturally sensitive to African Americans, who are more likely to experience lung cancer mortality [29] and may have earlier onsets of COPD [25].

This review focused on evaluating apps for the most common, evidence-based stress management techniques (ie, meditation and mindfulness, diaphragmatic breathing, and seeking social support). Therefore, less-common evidence-based stress management strategies may be used but were excluded in our review.

Compared with Coulon and colleagues’ findings [20], the apps reviewed were less likely to include at least one evidence-based stress management strategy. However, when apps employed these strategies (eg, meditation), they were similar to those found in the previous review. Our findings regarding apps’ scientific merits were comparable to those of Coulon and colleagues [20], but their review yielded more apps with scientific references (33% vs our 0%).

Regarding transparency, our review produced results consistent with those of Coulon and colleagues [20], but criteria most and least often met differed. Specifically, apps in both reviews were likely to provide contact information, but other criteria (eg, advertising policy) were satisfied less often in our review. Our assessment of apps for inclusion of skill-building instructions and opportunities for continued engagement demonstrated that most apps met or partially met these criteria. Although Coulon and colleagues did not assess cultural sensitivity or readability, our findings regarding an absence of cultural sensitivity and low readability were consistent with other prior work [28,30,31].

Finally, our review included a comprehensive heuristic evaluation to determine usability, which was more in-depth than the review by Coulon and colleagues [20]. Therefore, we identified critical design flaws that may affect users’ ease of use.

On the basis of our evaluation, we make 5 key recommendations for improving the quality of commercially available apps aimed at adults with a COPD or lung cancer history.

Few mobile apps exist for promoting mindfulness-based strategies among adults with a chronic lung disease. Among those available, few meet the criteria for the 4 content evaluation domains (evidence of science, scientific strategy and engagement, evidence-based stress management strategies and structures, and transparency). Although the usability of apps reviewed varied greatly, most had design flaws that may compromise their helpfulness to populations with low technology usage or self-efficacy or limited experience with apps. In addition, the app content was not culturally sensitive or written for audiences with lower reading levels. To enhance the accessibility of evidence-based, commercially available apps for promoting mindfulness-based strategies among adults with a chronic lung disease, we outlined 5 key recommendations. Future research should assess the feasibility and efficacy of implementing these recommended processes within an app development lifecycle. Additionally, future app reviews should also include an assessment of apps for potentially harmful strategies.