Background: There are thousands of apps for individuals struggling with headache, insomnia, and pain, but it is difficult to establish which of these apps are best suited for patients’ specific needs. If clinicians were to have access to a platform that would allow them to make an informed decision on the efficacy and feasibility of smartphone apps for patient care, they would feel confident in prescribing specific apps.
Objective: We sought to evaluate the quality of apps for some of the top common, disabling neurologic conditions (headache, insomnia, and pain) based on principles derived from the American Psychiatric Association’s (APA) app evaluation model.
Methods: We used the Mobile Health Index and Navigation database and expanded upon the database’s current supported conditions by adding 177 new app entries. Each app was rated for consistency with the APA’s app evaluation model, which includes 105 objective questions based on the following 5 major classes of consideration: (1) accessibility, (2) privacy and security, (3) clinical foundation, (4) engagement style, and (5) interoperability. These characteristics were evaluated to gain a broader understanding of the significant features of each app category in comparison against a control group.
Results: Approximately 90% (187/201) of all apps evaluated were free to download, but only 50% (63/201) of headache- and pain-related apps were truly free. Most (87/106, 81%) sleep apps were not truly free to use. The apps had similar limitations with limited privacy, accessibility, and crisis management resources. For example, only 17% (35/201) of the apps were available in Spanish. The apps offered mostly self-help tools with little tailoring; symptom tracking was the most common feature in headache- (32/48, 67%) and pain-related apps (21/47, 45%), whereas mindfulness was the most common feature in sleep-related apps (73/106, 69%).
Conclusions: Although there are many apps for headache, pain, and insomnia, all 3 types of apps have room for improvement around accessibility and privacy. Pain and headache apps share many common features, whereas insomnia apps offer mostly mindfulness-based resources. Given the many available apps to pick from, clinicians and patients should seek apps that offer the highest-quality features, such as complete privacy, remedial features, and the ability to download the app at no cost. These results suggest that there are many opportunities for the improvement of apps centered on headache, insomnia, and pain.
There is a health crisis in the United States whereby people cannot access neurologic care in a timely manner . As smartphones and digital tools increase in popularity, with 85% ownership as of 2021, compared to only 35% in 2011 [ ], many mobile health tools have been developed as a means to provide self-management and other strategies to patients. This is especially true for common and disabling neurologic conditions such as headache, sleep, and pain disorders [ ]. With 1 in every 6 American adults experiencing migraine and severe headache [ ], 70 million American adults experiencing sleep problems [ ], and 1 in 5 American adults experiencing chronic pain [ ], there is a clear need for treatment. Although a quick search in an app store for “headache,” “pain,” or “sleep” may reveal countless apps, the apps listed at the top of a search result do not necessarily offer benefits in terms of utility and efficacy compared to others [ ]. There have been many reviews of mental health apps in each of the app store marketplaces (ie, Apple iTunes and Google Play) [ ], but there have been fewer reviews for neurology-focused apps [ ]. Given the inherent risks of apps, including privacy concerns [ - ], mixed evidence around efficacy, and broad usability concerns [ , ], clinicians and patients need to be aware of the state of these public-facing apps and be able to understand their risks and benefits.
Despite the broad risks in the digital health space, emerging evidence suggests the potential benefits of apps for neurological conditions. Even simple headache tracking apps have been shown to help with the management of symptoms . In randomized controlled trials, apps for insomnia have shown benefits such as significant reduction in sleep-related impairment of quality of life and mental well-being [ ]. Presently, apps for pain management are expanding in scope, with features such as pain impact recording and medication tracking [ ]. To aid users in their efforts to discover apps that are accessible, safe, effective, and evidence based, several app libraries have been developed. One such publicly available tool that considers these and many other metrics when evaluating an app is the Mobile Health Index and Navigation Database (MIND) [ , ]. MIND is the largest open and publicly accessible database of mental health and neurology-focused apps—with over 600 apps, each rated across 105 criteria and updated at least every 6 months. Per recent research, MIND is unique as it also represents diversity, equity, and inclusion criteria, such as accessibility features and language options, which offer a more comprehensive window into apps utility [ ]. To understand the current state of mobile apps for neurological disorders, using headache and pain as the leading causes of nonfatal health loss [ ] and insomnia as the common sleep disorder [ ], we applied the 105 metrics found in MIND to the top neurological apps discoverable on iOS and Android devices.
We sought to (1) search app stores for headache-, pain-, and sleep-related apps and review them using the MIND database, and (2) evaluate the characteristics of these apps and compare them to a control group of apps across unique features not yet reported in the literature, including language (Spanish), crisis management, and the ability to connect with providers. We predict that if clinicians were to have access to a platform that would allow them to make an informed decision on the efficacy and feasibility of smartphone apps for patient care, they would feel confident in prescribing specific apps.
This study used the MIND database, published by the Division of Digital Psychiatry at Beth Israel Deaconess Medical Center . Details about MIND have been published previously [ , , ]. In brief, MIND is the largest publicly available database of mental health apps; it currently logs 656 mental health apps in the commercial market across a variety of supported conditions and acts as an open resource for users to filter apps by features of their personal preference. To add an app to the MIND database, there are 105 objective questions that are answered based on the following 5 major classes of consideration within the American Psychiatric Association’s app evaluation model: accessibility, privacy and security, clinical foundation, engagement style, and interoperability ( ). All data in the MIND database are publicly accessible through the MINDapps website [ ]. A screenshot of the MIND database is provided in .
The database does not include apps for which the cost of download exceeds US $10 or those not accessible to the public. Prior to this study, the database had 24 apps for pain-, headache-, or sleep-related conditions. The study expanded upon the current database by adding 177 new app entries for the supported conditions—47 apps for pain, 48 for headache, and 106 for sleep—accounting for overlap between categories, amounting to a total of 201 apps analyzed in this study. The selection of new apps was conducted as follows: to gain an understanding of the app marketplace for headache-, pain-, and sleep-related apps, terms such as “headache,” “pain,” and “insomnia” were searched in both the iOS and Google Play stores in June and July 2021. The first 50 apps that appeared on each platform (Apple App Store and Google Play Store) for each search term (headache, pain, and sleep) were compiled into a Google Sheet. Thus, 300 apps were discovered in our preliminary search. Some of these apps appeared within the first 50 apps searched on both platforms. Apps were assessed for relevance to the neurological conditions of interest. Given that there is no standard definition for these apps and many apps appearing in a search may not be related to wellness or health (eg, a gaming app), all apps selected for evaluation were agreed upon for relevance via consensus of all raters, all of whom are authors. Apps were excluded if they were irrelevant, clinician facing, nonfunctional, unavailable in English, and required an access code to use them. Upon removal of apps that did not meet the inclusion criteria, the remaining 177 relevant apps were evaluated and added to the MIND database.shows a flowchart detailing app selection and app rating process.
Given that all data were entered into the MIND, unique to this study, we sought to publicly share all app evaluations, so others can expand on these results and use this raw data to explore and find relevant apps today, as well as to aid neurologists in making informed decisions around choosing smartphone apps for patients with these conditions.
A total of 6 app raters underwent interrater reliability training , and 1 app rater evaluated each app. Interrater reliability was assessed using Cohen κ statistic, for which raters demonstrated very good interrater reliability (defined as a κ value above 0.750), with an average κ value of 0.859 across all apps rated. Discrepancies between the raters were addressed individually through discussion and subsequently resolved by clarifying any discrepancy in the description of each question.
As this was purely an exploratory study, we used Excel (Microsoft Corporation) and reported descriptive statistics.
Of the 656 apps available in MIND in July 2021, a total of 201 were related to headache, pain, or insomnia. Overall, these 201 apps focusing on headache, pain, and insomnia offered common features for tracking symptoms, tracking medication, journaling, and psychoeducation. Very few apps (26/201, 13%) used biological data, defined as metrics obtained from external devices (eg, wearables or built-in phone sensors), to monitor personal health. Examples of biodata collected include skin conductance, heart rate, and sleep quality. Across all 3 types of apps examined, we found similarities in terms of platform cost, special features (eg, Spanish language and accessibility features), clinician support, and privacy features as shown in bothand .
|Characteristics||Apps, n (%)|
|Headache and migraine (n=48)||Sleep and insomnia (n=106)||Chronic pain (n=47)|
|iOS||31 (65)||87 (82)||39 (83)|
|Android||32 (65)||83 (78)||22 (47)|
|Both iOS and Android||15 (31)||64 (60)||14 (30)|
|Web||10 (21)||19 (18)||6 (13)|
|Totally free||22 (46)||19 (18)||22 (47)|
|Free to download||43 (90)||100 (94)||44 (94)|
|In-app purchases||17 (21)||66 (62)||20 (43)|
|Subscription||10 (13)||65 (61)||9 (19)|
|Spanish||5 (11)||24 (23)||6 (13)|
|Offline||31 (65)||52 (49)||29 (62)|
|Accessibility features||20 (42)||42 (40)||22 (47)|
|Email or export data||25 (46)||23 (22)||17 (36)|
|Peer support||2 (4)||13 (12)||4 (9)|
|Collaboration with provider||7 (15)||8 (8)||4 (9)|
|Meets HIPAAa requirements||1 (2)||5 (5)||1 (2)|
|Crisis management feature||0 (0)||8 (8)||1 (2)|
aHIPAA: Health Insurance Portability and Accountability Act.
Apps were accessible on Apple, Android, and web browsers, although less than 50% (63/201) were truly free of cost. Over 90% (187/201) of apps in all categories were free to download, but this did not guarantee no-cost or even low-cost use. Over 50% (112/201) of apps across all disease states offered functionalities for working offline, that is, without an internet connection. Approximately 17% (35/201) of these apps supported Spanish.
Crisis Management and Privacy
Self-help and Hybrid Use With a Clinician
Most apps were self-help centered, but some included collaboration of a clinician (either from the app or outside the app). This was offered by 15% (7/46) of headache apps, 9% (8/47) of pain apps, and 8% (4/106) of sleep apps. Proportions of apps offering peer support were similarly low.
The reported functionality offered by these apps is shown in. Results show that headache and pain apps shared many common features, with tracking symptoms as the most used feature and mindfulness as one of the least used features. In contrast, the apps that focused on sleep had mindfulness as their most common feature and symptom tracking as one of the least common features.
|Apps and features||Values, n (%)|
|Headache and migraine apps (n=48)|
|Track symptoms||32 (67)|
|Track medication||30 (63)|
|Track sleep||6 (13)|
|Track mood||6 (13)|
|Physical health||5 (11)|
|Sleep and insomnia apps (n=106)|
|Deep breathing||58 (55)|
|iCBTa or sleep therapy||26 (25)|
|Track mood||26 (25)|
|Goal settings or habits||25 (24)|
|Track sleep||24 (23)|
|Physical health||15 (14)|
|Pain apps (n=47)|
|Track symptoms||21 (45)|
|Physical health||20 (43)|
|Track medication||16 (34)|
|Physical health exercises||12 (26)|
|Track sleep||8 (17)|
|Track mood||8 (17)|
aiCBT: internet-based cognitive behavioral therapy.
Each app was evaluated across 105 individual criteria, and all results are publicly accessible and searchable today through the MINDapps database . shows an example of how a reader can interactively explore and search these apps across individual questions. We do not provide scores for certain categories (privacy, functionality, etc), as MINDapps allows users to select their own filters and create their own criteria based on personal needs.
Our review of apps for headache, insomnia, and pain is the largest review of publicly available neurology-focused offerings to date, with results derived from over 200 apps each categorized across 105 dimensions. Pain and headache apps share many common features, whereas insomnia apps offer mostly mindfulness-based resources. We found that apps mostly offered self-help tools with little tailoring, and that symptom tracking was the most common feature in headache- (32/48, 67%) and pain-related apps (21/47, 45%), whereas mindfulness was the most common feature in sleep-related apps (73/106, 69%). Despite the number of apps and 3 unique conditions, we found numerous commonalities, including limited privacy, accessibility, and crisis management resources, in these mostly self-help tools. In terms of features offered, tracking and mindfulness-related features were most common, with individual apps offering varied ratios or types of these core features. These results suggest opportunities for innovation around the structure of apps themselves, as well as how they deliver tracking or mindfulness, with any innovation presenting transdiagnostic benefits. The numerous overlapping features offered by these apps also suggest that clinicians and patients today can be demanding in selecting an app, as there are likely minimal differences in their core functionality. Using MINDapps , they can explore which apps may offer the exact app features desired.
A recent review of mental health–focused apps found that almost 15% supported Spanish, which is consistent with our result of 17% (35/201) for the neurology-focused apps we reviewed . This result suggests an immediate opportunity to increase reach, while better supporting diversity and inclusion. Most apps we reviewed were self-help focused, with only a small fraction designed to be used in partnership with a peer or clinicians. Across the broader digital health field, there is growing evidence that apps used in partnership with others may be more engaging and effective than the self-help ones. Lessons already learned about low engagement with mental health apps [ ] may help these neurology-related apps develop as more engaging relationship-based tools that could offer more support for hybrid use.
Comparison With Prior Work
Our findings about the common features underlying apps for headache, insomnia, and pain suggest room for transdiagnostic innovation around all these apps. The common features of symptom tracking, and mindfulness are also the most common features in mental health apps , suggesting a potential synergy between these two fields. This result makes sense as apps offer a practical platform to deliver behavioral-based treatments and remotely capture symptoms, which are themselves shared aspects between psychiatry and neurology. Although the insomnia apps focused more on behavioral interventions, the headache and pain apps focused more on symptom tracking.
Furthermore, although smartphone apps may show promising results for patient care, it remains challenging to evaluate their effectiveness. Clinical studies do not often feature a valid control condition or simulate the challenges of real-world app engagement, which is frequently low . Thus, the MIND framework does not rate the quality of scientific evidence for apps nor their engagement, given the lack of consensus or data availability around these points. Our results are therefore best interpreted as signals of what these apps are claiming to offer, with the recommendations for personal use and exploration with the app itself to determine whether the feature meets the needs for each clinical use case. Clinicians should feel empowered to go to MINDapps [ ] and search for apps that they feel would meet the standards for their patient’s needs. The results of this paper can help calibrate expectations and guide clinicians in searching for apps that meet the unique demands of each patient served.
Although the number of headache, sleep, and pain apps on the market continues to expand, there are numerous opportunities for content improvement. Many of these apps were lacking in privacy, accessibility, and crisis management resources—features that would significantly improve the app platforms. The results of this study suggest an opportunity for improvement in app structure and the delivery of important features. Patient care may be improved with the incorporation of a transdiagnostic approach to health-based smartphone apps.
MTM designed and conceptualized the study, drafted the manuscript, as well as revised it. AG conducted data analysis and interpretation, drafted the manuscript, and revised it. EC conducted data analysis and interpretation, drafted and revised the manuscript, and was responsible for the acquisition of data. LY, AS, QH, DV, MS, and MC contributed to the acquisition, analysis, and interpretation of data.
Conflicts of Interest
MTM is supported by NCCIH (grant K23 AT009706-05). MTM also helped develop the RELAXaHEAD app, which has shared IP between New York University Langone Health and Irody Inc. The app was not used in this study, as it has been used as a research app to date and has not been available to the public. JT developed the MIND database with support from the Argosy Foundation.
App review questions.DOCX File , 27 KB
- Dorsey ER, George BP, Leff B, Willis AW. The coming crisis: obtaining care for the growing burden of neurodegenerative conditions. Neurology 2013 Apr 24;80(21):1989-1996. [CrossRef] [Medline]
- Mobile fact sheet. Pew Research Center. 2021. URL: https://www.pewresearch.org/internet/fact-sheet/mobile/ [accessed 2022-06-01]
- Minen MT, Stieglitz EJ. Wearables for neurologic conditions. Neurol Clin Pract 2020 Sep 28;11(4):e537-e543. [CrossRef] [Medline]
- Burch R, Rizzoli P, Loder E. The prevalence and impact of migraine and severe headache in the united states: figures and trends from government health studies. Headache 2018 Apr 12;58(4):496-505. [CrossRef] [Medline]
- Hafner M, Stepanek M, Taylor J, Troxel WM, Van Stolk C. Why sleep matters—the economic costs of insufficient sleep: a cross-country comparative analysis. RAND corporation. 2017. URL: https://www.rand.org/content/dam/rand/pubs/research_reports/RR1700/RR1791/RAND_RR1791.pdf [accessed 2022-06-01]
- Zelaya CE, Dahlhamer JM, Lucas JW, Connor EM. Chronic pain and high-impact chronic pain among U.S. adults, 2019. NCHS Data Brief 2020 Nov(390):1-8. [Medline]
- Wisniewski H, Liu G, Henson P, Vaidyam A, Hajratalli NK, Onnela J, et al. Understanding the quality, effectiveness and attributes of top-rated smartphone health apps. Evid Based Ment Health 2019 Feb 11;22(1):4-9 [FREE Full text] [CrossRef] [Medline]
- Lagan S, D'Mello R, Vaidyam A, Bilden R, Torous J. Assessing mental health apps marketplaces with objective metrics from 29,190 data points from 278 apps. Acta Psychiatr Scand 2021 Aug 29;144(2):201-210. [CrossRef] [Medline]
- Minen MT, Gopal A, Sahyoun G, Stieglitz E, Torous J. The functionality, evidence, and privacy issues around smartphone apps for the top neuropsychiatric conditions. J Neuropsychiatry Clin Neurosci 2021;33(1):72-79 [FREE Full text] [CrossRef] [Medline]
- Benjumea J, Ropero J, Rivera-Romero O, Dorronzoro-Zubiete E, Carrasco A. Privacy assessment in mobile health apps: scoping review. JMIR Mhealth Uhealth 2020 Jul 02;8(7):e18868 [FREE Full text] [CrossRef] [Medline]
- Minen MT, Stieglitz EJ, Sciortino R, Torous J. Privacy issues in smartphone applications: an analysis of headache/migraine applications. Headache 2018 Jul 04;58(7):1014-1027 [FREE Full text] [CrossRef] [Medline]
- Minen MT, Gopal A, Sahyoun G, Stieglitz E, Torous J. The functionality, evidence, and privacy issues around smartphone apps for the top neuropsychiatric conditions. J Neuropsychiatry Clin Neurosci 2021 Jan;33(1):72-79 [FREE Full text] [CrossRef] [Medline]
- Molloy A, Anderson PL. Engagement with mobile health interventions for depression: a systematic review. Internet Interv 2021 Dec;26:100454 [FREE Full text] [CrossRef] [Medline]
- Friis-Healy EA, Nagy GA, Kollins SH. It is time to REACT: opportunities for digital mental health apps to reduce mental health disparities in racially and ethnically minoritized groups. JMIR Ment Health 2021 Jan 26;8(1):e25456 [FREE Full text] [CrossRef] [Medline]
- Bensink M, Shah S, Shah N, Desai P, Khan F, Rubin A, et al. Tracking migraine digitally: the future of migraine management. J Nurse Pract 2021 Apr;17(4):462-470. [CrossRef]
- Espie CA, Emsley R, Kyle SD, Gordon C, Drake CL, Siriwardena AN, et al. Effect of digital cognitive behavioral therapy for insomnia on health, psychological well-being, and sleep-related quality of life: a randomized clinical trial. JAMA Psychiatry 2019 Jan 01;76(1):21-30 [FREE Full text] [CrossRef] [Medline]
- Zhao P, Yoo I, Lancey R, Varghese E. Mobile applications for pain management: an app analysis for clinical usage. BMC Med Inform Decis Mak 2019 May 30;19(106):1-10. [CrossRef]
- MIND: M-Health Index and Navigation Database. URL: https://mindapps.org/ [accessed 2022-06-01]
- Ramos G, Ponting C, Labao JP, Sobowale K. Considerations of diversity, equity, and inclusion in mental health apps: a scoping review of evaluation frameworks. Behav Res Ther 2021 Dec;147:103990 [FREE Full text] [CrossRef] [Medline]
- GBD 2017 US Neurological Disorders Collaborators, Feigin VL, Vos T, Alahdab F, Amit AML, Bärnighausen TW, et al. Burden of neurological disorders across the us from 1990-2017: a global burden of disease study. JAMA Neurol 2021 Feb 01;78(2):165-176 [FREE Full text] [CrossRef] [Medline]
- Dopheide JA. Insomnia overview: epidemiology, pathophysiology, diagnosis and monitoring, and nonpharmacologic therapy. Am J Manag Care 2020 Mar 12;26(4 Suppl):S76-S84 [FREE Full text] [CrossRef] [Medline]
- Lagan S, Ramakrishnan A, Lamont E, Ramakrishnan A, Frye M, Torous J. Digital health developments and drawbacks: a review and analysis of top-returned apps for bipolar disorder. Int J Bipolar Disord 2020 Dec 01;8(1):39 [FREE Full text] [CrossRef] [Medline]
- Lagan S, Aquino P, Emerson MR, Fortuna K, Walker R, Torous J. Actionable health app evaluation: translating expert frameworks into objective metrics. NPJ Digit Med 2020 Jul 30;3(1):1-8. [CrossRef]
- Tangari G, Ikram M, Ijaz K, Kaafar MA, Berkovsky S. Mobile health and privacy: cross sectional study. BMJ 2021 Jun 16;373:n1248 [FREE Full text] [CrossRef] [Medline]
- Albert Henry T. FTC warns health apps to comply with health data-breach rules. AMA. 2021 Nov 29. URL: https://www.ama-assn.org/practice-management/digital/ftc-warns-health-apps-comply-health-data-breach-rules#:~:text=Health%20apps %20and%20connected%20devices,Federal%20Trade%20Commission%20(FTC) [accessed 2022-06-01]
- Muñoz AO, Camacho E, Torous J. Marketplace and literature review of Spanish language mental health apps. Front Digit Health 2021 Feb 15;3:615366 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Muench F, Edan S, Kane JM. Objective user engagement with mental health apps: systematic search and panel-based usage analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [CrossRef] [Medline]
|FTC: Federal Trade Commission|
|HIPAA: Health Insurance Portability and Accountability Act|
|MIND: Mobile Health Index and Navigation|
Edited by L Buis; submitted 24.01.22; peer-reviewed by M Ainslie, M K., K Fadahunsi, G Noomen; comments to author 28.03.22; revised version received 17.05.22; accepted 25.05.22; published 21.06.22Copyright
©Mia T Minen, Alexis George, Erica Camacho, Leslie Yao, Ananya Sahu, Maya Campbell, Mia Soviero, Quazi Hossain, Deepti Verma, John Torous. Originally published in JMIR mHealth and uHealth (https://mhealth.jmir.org), 21.06.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on https://mhealth.jmir.org/, as well as this copyright and license information must be included.