RefCheck Maintenance Notice

On Monday, December 3, 2018, from 16:00-18:00 EST, RefCheck will be undergoing maintenance. RefCheck is the process where, during copyediting, all references are extracted from the manuscript file, parsed, matched against various databases (eg, PubMed and CrossRef), and automatically corrected. For more information on RefCheck, please visit our Knowledge Base.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 14.06.16 in Vol 4, No 2 (2016): Apr-Jun

This paper is in the following e-collection/theme issue:

    Original Paper

    Review and Analysis of Existing Mobile Phone Apps to Support Heart Failure Symptom Monitoring and Self-Care Management Using the Mobile Application Rating Scale (MARS)

    1Columbia University, School of Nursing, New York, NY, United States

    2College of Physicians & Surgeons, Columbia University, New York, NY, United States

    3NewYork-Presbyterian/Columbia University Medical Center, Division of Cardiology, New York, NY, United States

    Corresponding Author:

    Ruth M Masterson Creber, PhD, MSc, RN

    Columbia University

    School of Nursing

    617 W 168th St

    New York, NY, 10032

    United States

    Phone: 1 2123050391

    Fax:1 2123056937

    Email:


    ABSTRACT

    Background: Heart failure is the most common cause of hospital readmissions among Medicare beneficiaries and these hospitalizations are often driven by exacerbations in common heart failure symptoms. Patient collaboration with health care providers and decision making is a core component of increasing symptom monitoring and decreasing hospital use. Mobile phone apps offer a potentially cost-effective solution for symptom monitoring and self-care management at the point of need.

    Objective: The purpose of this review of commercially available apps was to identify and assess the functionalities of patient-facing mobile health apps targeted toward supporting heart failure symptom monitoring and self-care management.

    Methods: We searched 3 Web-based mobile app stores using multiple terms and combinations (eg, “heart failure,” “cardiology,” “heart failure and self-management”). Apps meeting inclusion criteria were evaluated using the Mobile Application Rating Scale (MARS), IMS Institute for Healthcare Informatics functionality scores, and Heart Failure Society of America (HFSA) guidelines for nonpharmacologic management. Apps were downloaded and assessed independently by 2-4 reviewers, interclass correlations between reviewers were calculated, and consensus was met by discussion.

    Results: Of 3636 potentially relevant apps searched, 34 met inclusion criteria. Most apps were excluded because they were unrelated to heart failure, not in English or Spanish, or were games. Interrater reliability between reviewers was high. AskMD app had the highest average MARS total (4.9/5). More than half of the apps (23/34, 68%) had acceptable MARS scores (>3.0). Heart Failure Health Storylines (4.6) and AskMD (4.5) had the highest scores for behavior change. Factoring MARS, functionality, and HFSA guideline scores, the highest performing apps included Heart Failure Health Storylines, Symple, ContinuousCare Health App, WebMD, and AskMD. Peer-reviewed publications were identified for only 3 of the 34 apps.

    Conclusions: This review suggests that few apps meet prespecified criteria for quality, content, or functionality, highlighting the need for further refinement and mapping to evidence-based guidelines and room for overall quality improvement in heart failure symptom monitoring and self-care related apps.

    JMIR Mhealth Uhealth 2016;4(2):e74

    doi:10.2196/mhealth.5882

    KEYWORDS



    Introduction

    Heart failure (HF) is a common, complex, and costly cardiovascular condition. Heart failure currently affects 5.7 million Americans [1], is the fastest growing cardiovascular condition in the United States [2], and the most common reason for hospitalization among older adults [3-6]. Worldwide, the prevalence of HF is estimated to be more than 23 million people [7]. The most common reason for HF-related hospitalizations is symptom exacerbations. Symptom changes are often insidious, making it difficult for patients to recognize and respond to changes early and resulting in need for hospital-based management of HF exacerbations. To reduce the societal and cost burden of HF, effective symptom management strategies are important for patients and also may help to reduce hospital admissions [8,9]. Major clinical guidelines recommend the inclusion of daily symptom monitoring as part of routine management of patients with HF [10].

    With an uptake of mobile phone ownership among adults in the United States [11], there is growing opportunity to capitalize on the use of mobile phone technology to enhance the management of HF. Mobile phones are an optimal vehicle for housing mobile health (mHealth) apps for symptom monitoring because they are accessible continuously, portable, and convenient. Mobile health apps are reported to be an ideal platform for behavior change because of popularity, connectivity, and increased sophistication [12]. Apps can support added functionalities and have the potential for real-time data collection, graphic feedback, interactivity, and links to social functionalities [12]. In addition, apps have the potential to be useful for symptom management because they can include behavioral prompts, reminders, illness monitoring, and self-management programs that extend far beyond the clinic walls.

    Currently, reviews of commercial mHealth apps exist to support patients undergoing bariatric surgery [13], those who are managing bipolar disorder [14], cancer [15], cardiovascular disorders [16], chronic pain [17,18], depression [19], diabetes [20], health care–associated infection prevention [21], human immunodeficiency virus [22,23], and schizophrenia [24]. A review has been conducted on published literature on mHealth apps for HF [25]; however, it did not include an evaluation of commercially available mHealth apps. To date no studies have assessed commercially available apps to support HF symptom monitoring and self-care. To address this gap, we conducted a thorough review of commercially available existing mobile apps focused specifically on self-management and symptom monitoring for patients with HF. Our objectives were to (1) identify HF-related apps available in the main app stores; (2) describe their characteristics; (3) identify if any of the available apps have been rigorously tested; and (4) rate the quality of the apps based on the Mobile Application Rating Scale (MARS) [26], functionality score from the IMS Institute for Healthcare Informatics report [27], and Heart Failure Society of America (HFSA) guidelines for nonpharmacologic management [10].


    Methods

    Systematic Search Criteria and Selection

    In January 2016, we conducted a thorough review of mobile apps across 3 mobile app stores: Apple iTunes Store, Android Google Play store, and Amazon Appstore. The following search terms were included: “heart failure,” “cardiology,” “heart failure and self-management,” “heart failure and symptom management,” “heart failure and symptom monitoring,” “heart failure and self-care,” “cardiology and symptom management,” “cardiology and symptom monitoring,” “heart,” “symptom,” “symptom management,” “self-care,” and “self-care and heart.” Each term was searched in each of the 3 app stores listed.

    Preliminary screening was conducted based on app titles, full marketing description, and screenshots of the potential apps for relevance and inclusion. Apps were excluded if they were games, unrelated to health, or not written in English or Spanish. The second round of exclusion criteria focused on removing (1) duplicate apps (those found in multiple stores or from multiple search terms), (2) highly similar versions of the same app (eg, “lite” or “pro” versions), (3) apps that are not patient-facing, (4) apps focused solely on health and fitness, (5) apps for continuing medical education or conference apps, and (6) apps that were no longer available (Figure 1). Team members reviewed the apps after each round of exclusion criteria were completed (almost 70% of the apps were rated by at least two reviewers). The remaining apps were downloaded, reviewed (iOS 9.2.1 on iPhone 6; iPad mini or Android phone), rated, and evaluated by 2 reviewers (GH and RMC).

    A data extraction form was built using a Google Docs survey that included the full MARS scale, IMS Institute for Healthcare Informatics functionality scoring system, and 8 questions related to specific self-care behaviors recommended in the “Nonpharmacologic Management and Health Care Maintenance in Patients with Chronic Heart Failure” published by the HFSA [10].

    Figure 1. Screening process flowchart.
    View this figure

    Measures or Rating Tool

    We rated and ranked the apps based on 3 scores: (1) MARS quality score [26], (2) IMS Institute for Healthcare Informatics functionality score [27], and (3) consistency with HFSA guideline recommendations [10] with an additional question related to the number of self-care behaviors that the apps addressed. The MARS was used to rate app quality and includes 3 sections and a modifiable app-specific section: classification, quality, and satisfaction [26]. The classification section provides descriptive information about the apps. The objective app quality section includes 19 items divided into 4 scales: engagement, functionality, aesthetics, and information quality. The subjective quality section contains 4 items evaluating the user’s overall satisfaction. MARS items are scored using a 5-point Likert scale (1-inadequate, 2-poor, 3-acceptable, 4-good, and 5-excellent). The final MARS scores include 4 subscale scores, a total mean score, subjective quality score, and an app-specific subscale that assesses perceived effect on the user’s knowledge, attitudes, and intentions to change as well as likelihood of changing the identified targeted behaviors.

    The IMS functionality score is based on 7 functionality criteria and 4 functional subcategories as described in detail in the IMS Institute for Healthcare Informatics report [27] (Table 1). Each app was assessed for having or not having 11 functionalities and given a functionality score (0-11) [27].

    Table 1. IMS Institute for Healthcare Informatics functionality scoring criteria.
    View this table

    Two functionality scores were used for this review because the functionality scores provide different types of information on app functionality. The MARS functionality score focuses on performance, ease of use, navigation, and gestural design of the app [26], whereas the IMS Institute for Healthcare Informatics functionality score focuses on scope of functions, including informing, instructing, recording, displaying, guiding, reminding, and communicating information [27].

    Each of the apps was also evaluated for whether it included 8 specific self-care behaviors recommended by HFSA guidelines [10]. These behaviors included daily weighing, checking extremities for swelling, doing physical activity or exercise, eating a low-salt diet, taking daily medications, attending doctor’s appointments, daily monitoring of HF symptoms, and actively responding to symptoms when they change, consistent with HFSA nonpharmacologic guidelines [10].

    Data Analysis

    Four reviewers (GH, RMC, MR, and SI) watched the accompanying MARS instructional videos for how to use the MARS scale. Each reviewer rated 4 randomly selected apps to evaluate interrater reliability. The interclass correlation coefficients (ICCs) were calculated between the 4 reviewers. On the basis of ICC guidelines by Shrout and Fleiss [28], we selected an individual consistency-of-agreement intraclass correlation (CA-ICC) for a two-way random-effects model. The assumptions of this model include that the variance of raters only adds noise to the mean estimate and that the mean rater error is zero. It also models both the effect of the individual rater as well as the average of the raters and assumes both are drawn randomly from a larger population [29].


    Results

    Descriptive Characteristics

    Android Google Play, Apple iTunes, and Amazon Appstore searches identified 3636 potentially relevant apps, of which 34 met our final inclusion criteria. The flow diagram (Figure 1) provides an overview of the selection process and categories for exclusion. Most apps were excluded because they were unrelated to HF (n=3120), not available in English or Spanish (n=162), or were games (n=190).

    Table 2 provides the full list of the included apps and their characteristics. Most apps were free to download (31/34, 91%) with costs up to US $4.99. Most of the apps have been updated within the last year (63%). The average consumer star rating across all of the apps was 3 out of 5 with a range of 0 to 5. The number of individual ratings ranged from 0 to more than 52,000. Most of the apps had been installed between 100 and 500 times, but WebMD and iTriage had been installed between 5 and 10 million times. Almost 50% of the apps included a privacy policy.

    Table 2. Description of included apps.
    View this table

    MARS App Quality Scores

    Table 3 presents the 4 subscale scores (engagement, functionality, aesthetics, and information), overall quality score, subjective quality score (satisfaction), and app-specific health behavior score from the MARS. It was not possible to rate item 19, because a PubMed search identified only 3 efficacy studies among the 34 apps. More than 2/3 of the apps were evaluated by 2 or more expert MARS raters, and there was excellent interrater reliability (two-way mixed CA-ICC=.93, 95% CI: 0.68-0.99). Of the 4 subscales, functionality had the highest score and median engagement had the lowest (2.9).

    The median overall MARS score was 3.4 out of 5, and 68% (23/34) had a minimum acceptability score of 3.0. Overall, the AskMD app had the highest average MARS total (4.9) followed by WebMD (4.4), Symple (4.3), Heart Failure Health Storylines (4.1), and ContinuousCare Health App (4.0). Heart Failure Health Storylines (4.6) and AskMD (4.5) had the highest scores for behavior change.

    Table 3. Mobile Application Rating Scale scores.
    View this table

    Functionality

    Figure 2 illustrates the functionalities of the apps and highlights that nearly all had a record function (29/34, 85%). The median number of functionalities was 5 and the majority of apps (66%) had less than 7. Twenty-four apps had the option to display, 18 to inform, 16 to communicate, 15 to instruct, 15 to guide, and 10 to remind/alert. Three apps had a total of 11 functionalities (WebMD, Symple, and ContinuousCare Health App) followed by Heart Failure Health Storylines, which had 10 functionalities.

    Of the 29 apps that had the function to record, 26 could collect data, 12 could share the data, and 10 had the function to intervene. Examples of data that were collected using these apps include medications, symptoms, daily moods, daily vital signs, and physical activity.

    Heart Failure Health Storylines and WebMD have the ability to sync (collect) with a wide variety of fitness devices, apps, and even some scales. Many of the apps sync with the Apple Health app for iPhone users. Examples of apps that included the option to share data included Heart Failure Health Storylines and ContinuousCare Health App. In Heart Failure Health Storylines, the user has the option to share or communicate each feature with certain “circles of support” to which the user can add friends and family through email or keep the data private, including symptoms, vital signs, moods, and journal entries. Heart Failure Health Storylines does not have an option to share data using a message feature.

    ContinuousCare Health App has a newsfeed feature that includes health-related articles and a customizable user profile. ContinuousCare Health App also includes the option for real-time consultation with a licensed health care provider, as well as a “Doctor Virtual Practice” that allows the user to invite his or her provider to virtually track data and communicate with the user in the app itself.

    Symple offers a data exportation (share feature) that allows the user to back up data recorded in the app to a personal email or a spreadsheet app. In Symple users can also share an overview of current symptoms with their doctor. This document saves a PDF attachment and can then be sent over email. In WebMD the user can share health data and providers can respond and share education materials through this feature, which is password-protected.

    Figure 2. Functionality of included apps based on IMS Institute for Healthcare Informatics functionality scores.
    View this figure

    Heart Failure Society of America Guidelines

    Table 4 includes the 8 HF-specific self-care behaviors evaluated. The most commonly addressed was daily monitoring of symptoms (21/34, 62%), followed by responding to symptoms (16/34, 47%), taking daily medications (13/34, 38%), following a low-salt diet (10/34, 29%), going to provider appointments (6/34, 18%), doing exercise (7/34, 21%), daily weighing (7/34, 21%), and checking extremities for swelling (8/34, 24%). The app that addressed all of these self-care behaviors was Heart Failure Health Storylines, which was developed in collaboration with the HFSA.

    Table 4. Heart Failure Society of America–recommended nonpharmacologic management behaviors included in the apps.
    View this table

    Overall App Quality

    Factoring in the MARS, IMS Institute for Healthcare Informatics functionality, and HFSA guideline scores, the highest performing apps included Heart Failure Health Storylines, Symple, ContinuousCare Health App, WebMD, and AskMD.

    A PubMed search of the apps in this review found that only 3 apps have been evaluated and published in peer-reviewed journals [30-32].


    Discussion

    This study is the first to comprehensively review commercially available mobile apps for HF symptom monitoring and self-care and independently evaluate their quality using validated multiple rating scales, including the MARS expert rating scale, IMS Institute for Healthcare Informatics functionality scale, and HFSA guidelines for nonpharmacologic management. The most common functionality among the 34 apps reviewed was being able to record information, typically syncing data from other sources. Few apps provided any guidance in response to reported input, reminders or alerts about medications or symptom tracking, or the ability to communicate with providers.

    The 2 apps that provided the most options for symptom tracking included Heart Failure Health Storylines and Symple (Figure 3). Heart Failure Health Storylines includes the feature of being able to track multiple symptoms simultaneously to allow the user to detect potential correlations between symptoms and time periods. Users can record symptom severity, moods, vital signs, and medications and the data are displayed using a color-coding scheme and weekly calendar format. The vital sign data are also displayed in a line graph to show daily fluctuations.

    In Symple, users enter symptoms by calendar date and select the time of day and severity of each symptom (none, mild, moderate, difficult, severe) from a color-coded graphic (Figure 3). Symptoms are displayed using blocks with the same color-coding scheme in a weekly calendar format. Users can choose to view symptoms from one time of day (eg, every morning this week) or the entire day. One drawback of this feature is that it only allows viewing of one symptom at a time; multiple symptoms cannot be plotted on the same calendar.

    Despite the pressing need that patients with HF have for better symptom and self-monitoring tools, most mHealth apps are designed to support healthy living rather than chronic disease management. Many apps focused on helping patients find a diagnosis for their symptoms (ie, AskMD and WebMD). Some of the apps supported self-care maintenance in terms of recording daily health behaviors or including reminders about taking medication, but were very limited with self-care management behaviors including more advanced symptom monitoring, tracking, and evaluation of whether specific behaviors improved health outcomes.

    A total of 3 peer-reviewed articles evaluated 3 of the Web-based apps. The first article was a brief review on the development and future directions of the ASCVD Risk Estimator app [30]. The second article was an evaluation of the Heartkeeper app using Google app usability standards (completed by the authors) and a quality of experience survey completed by 24 users who were recruited through the app itself [31]. This review found the app to be generally compliant with Google’s standards; it had mixed feedback on the quality of experience. The third article was a 4-month trial of the iTreat app in the hospital setting among 39 junior doctors in the United Kingdom [32]. Although participants reported some positive outcomes from using the app, the study highlighted many barriers to the use of mobile phones in the hospital setting [32].

    Many apps are being used with minimal knowledge of their functionality and ability to integrate data into health care systems [27], let alone efficacy for improving patient or clinical outcomes. The lack of efficacy testing in clinical trials is one of the biggest barriers to adoption of mHealth apps. Health care providers are reticent to prescribe apps without evidence of their benefit, guidelines regarding use in clinical practice, and confidence in the privacy and security of personal health information that is both stored and transmitted [27]. These barriers are the major reasons why apps need to undergo rigorous clinical trial testing before they can be fully integrated into clinical care.

    One good example of an mHealth app with demonstrated effectiveness for managing diabetes is BlueStar from WellDoc Diabetes Management. This app has been evaluated in a clinical trial and has demonstrated effectiveness for supporting diabetes management [33]; however, it is only accessible for patients with diabetes who have a prescription from their health care provider and it is not otherwise available to the public.

    These findings suggest that apps have not yet been readily adopted into routine clinical management and need further development to support comprehensive symptom management for patients with HF. The limited number of apps and functionalities of specific apps targeting HF behaviors suggests that the apps are in an early stage of development and that patients and providers who would be using them are at an early stage of adoption. This is also true for some older adults who have lagged behind in the adoption of smartphone technology as well [11]. One of the ongoing priorities for the adoption of mHealth apps into clinical practice will be the rigorous assessment of app quality as demonstrated in this study and effectiveness in rigorous comparative effectiveness studies. Improving the ability of apps to engage is also a targeted area for future improvement.

    Figure 3. Symptom tracking features. Heart Failure Health Storylines (left image) enables tracking symptoms over time and Symple (right image) enables reporting symptoms for a single day and visualization of individual graphs by symptom.
    View this figure

    Strengths and Limitations

    Strengths of this review include applying a rigorous multistep methodology to the evaluation of the apps and using the MARS rating scale. The star rating system can be misleading given the low numbers of ratings that some of these apps have. Systematic consolidation and rigorous evaluation beyond the star rating system and user comments are needed for patients to be able to evaluate which apps may be best for their symptom monitoring and self-care. The use of the MARS was a strength because it was rigorously developed [26] and has been used to evaluate apps related to mindfulness [34] and weight management [35]. One of the limitations of this review is that apps that were not publically available were not included, such as those that required a prescription or enrollment in a specific health care network or insurance plan.

    Future Research

    According to a report by the IMS Institute for Healthcare Informatics, one of the most important areas for future research will be the generation of evidence of value from the use of apps that will demonstrate the magnitude of behavior change and improved health outcomes [27]. Evaluation of existing apps should use rigorously tested tools, such as the MARS or IMS functionality score. In addition, future studies should test the effectiveness of apps with higher functionality and usability scores. Further mapping of HF-specific apps to evidence-based clinical guidelines is needed. Focusing on improving apps that are already commercially available is a viable option.

    In addition, there is also the need for enhanced interoperability between electronic health records and apps so that real-time data can inform clinician decision making and clinical management. Enhanced data integration should take place within the context of robust organizational governance frameworks that take into consideration the evaluation of clinical outcomes [36].

    Conclusions

    In general, mHealth apps offer a potentially cost-effective solution with 24/7 access to symptom monitoring at the point of need, promotion of patient engagement in their care, and can foster interactive care and communication with providers. Increasing the options for mHealth apps to support successful care management is critical. Patient collaboration with health care providers and decision making is a core component of patient engagement [37,38], improving quality of life and decreasing hospital use [39]. Our review highlights the need for further refinement and mapping to guidelines and room for overall quality improvement in HF symptom monitoring and self-care related apps. To ensure engagement, ease of use, and aesthetics, patients also need to be involved in the development of the mHealth apps.

    Acknowledgments

    The authors gratefully acknowledge funding for RMC by the National Institutes of Health (NIH)/National Institute of Nursing Research (NINR), K99NR016275, “mHealth for Heart Failure Symptom Monitoring.” Her post-doctoral funding was funded by the NINR (T32NR007969) and she received Columbia University School of Nursing pilot funds to support this study. Additional mentorship training was provided to RMC through the Agency for Healthcare Research and Quality (R01HS021816) at the Department of Biomedical Informatics at Columbia University. The authors also acknowledge the postdoctoral funding for senior author, SI, by the NINR through the Comparative and Cost-Effectiveness Research Training for Nurse Scientists Award (T32NR014205) at Columbia University School of Nursing. MSM is supported by a K24 Award from the NIA (AG036778) and KTH is supported by a R01 Award from the NINR (R01 NR014853: iPhone Helping Evaluate Atrial Fibrillation Rhythm Through Technology (iHEART)).

    Conflicts of Interest

    None declared.

    References

    1. Mozaffarian D, Benjamin EJ, Go AS, Arnett DK, Blaha MJ, Cushman M, American Heart Association Statistics CommitteeStroke Statistics Subcommittee. Heart disease and stroke statistics--2015 update: a report from the American Heart Association. Circulation 2015 Jan 27;131(4):e29-322. [CrossRef] [Medline]
    2. Roger VL, Go AS, Lloyd-Jones DM, Benjamin EJ, Berry JD, Borden WB, American Heart Association Statistics CommitteeStroke Statistics Subcommittee. Heart disease and stroke statistics--2012 update: a report from the American Heart Association. Circulation 2012 Jan 3;125(1):e2-e220 [FREE Full text] [CrossRef] [Medline]
    3. Heidenreich PA, Trogdon JG, Khavjou OA, Butler J, Dracup K, Ezekowitz MD, American Heart Association Advocacy Coordinating Committee, Council on Cardiovascular RadiologyIntervention, Council on Clinical Cardiology, Council OE, Council OA, ThrombosisVascular B, Council OC, Critical C, PerioperativeResuscitation, Council on Cardiovascular Nursing, Council on the Kidney in Cardiovascular Disease, Council on Cardiovascular SurgeryAnesthesia‚Interdisciplinary Council on Quality of CareOutcomes Research. Forecasting the future of cardiovascular disease in the United States: a policy statement from the American Heart Association. Circulation 2011 Mar 1;123(8):933-944 [FREE Full text] [CrossRef] [Medline]
    4. Lloyd-Jones D, Adams RJ, Brown TM, Carnethon M, Dai S, De SG, American Heart Association Statistics CommitteeStroke Statistics Subcommittee. Heart disease and stroke statistics--2010 update: a report from the American Heart Association. Circulation 2010 Feb 23;121(7):e46-e215 [FREE Full text] [CrossRef] [Medline]
    5. Ross JS, Chen J, Lin Z, Bueno H, Curtis JP, Keenan PS, et al. Recent national trends in readmission rates after heart failure hospitalization. Circ Heart Fail 2010 Jan;3(1):97-103 [FREE Full text] [CrossRef] [Medline]
    6. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med 2009 Apr 2;360(14):1418-1428. [CrossRef] [Medline]
    7. Roger VL. Epidemiology of Heart Failure. Circulation Research 2013 Aug 29;113(6):646-659. [CrossRef]
    8. Gheorghiade M, Zannad F, Sopko G, Klein L, Piña IL, Konstam MA, International Working Group on Acute Heart Failure Syndromes. Acute heart failure syndromes: current state and framework for future research. Circulation 2005 Dec 20;112(25):3958-3968 [FREE Full text] [CrossRef] [Medline]
    9. Felker GM, Leimberger JD, Califf RM, Cuffe MS, Massie BM, Adams KF, et al. Risk stratification after hospitalization for decompensated heart failure. J Card Fail 2004 Dec;10(6):460-466. [Medline]
    10. Heart Failure Society of America, Lindenfeld J, Albert NM, Boehmer JP, Collins SP, Ezekowitz JA, et al. HFSA 2010 Comprehensive Heart Failure Practice Guideline. J Card Fail 2010 Jun;16(6):e1-194. [CrossRef] [Medline]
    11. Pew Research Center.: Pew Research Center; 2014 Apr 03. Older adults and technology use   URL: http://www.pewinternet.org/files/2014/04/PIP_Seniors-and-Tech-Use_040314.pdf [accessed 2016-05-24] [WebCite Cache]
    12. Hale K, Capra S, Bauer J. A Framework to Assist Health Professionals in Recommending High-Quality Apps for Supporting Chronic Disease Self-Management: Illustrative Assessment of Type 2 Diabetes Apps. JMIR Mhealth Uhealth 2015;3(3):e87 [FREE Full text] [CrossRef] [Medline]
    13. Stevens DJ, Jackson JA, Howes N, Morgan J. Obesity surgery smartphone apps: a review. Obes Surg 2014 Jan;24(1):32-36. [CrossRef] [Medline]
    14. Nicholas J, Larsen ME, Proudfoot J, Christensen H. Mobile Apps for Bipolar Disorder: A Systematic Review of Features and Content Quality. J Med Internet Res 2015 Aug;17(8):e198 [FREE Full text] [CrossRef] [Medline]
    15. Bender JL, Yue Rossini Ying Kwan, To MJ, Deacken L, Jadad AR. A lot of action, but not in the right direction: systematic review and content analysis of smartphone applications for the prevention, detection, and management of cancer. J Med Internet Res 2013;15(12):e287 [FREE Full text] [CrossRef] [Medline]
    16. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M, Herreros-González J. Mobile apps in cardiology: review. JMIR Mhealth Uhealth 2013 Jul;1(2):e15 [FREE Full text] [CrossRef] [Medline]
    17. Wallace LS, Dhingra LK. A systematic review of smartphone applications for chronic pain available for download in the United States. J Opioid Manag 2014 Feb;10(1):63-68. [CrossRef] [Medline]
    18. de la Vega R, Miró J. mHealth: a strategic field without a solid scientific soul. a systematic review of pain-related apps. PLoS One 2014;9(7):e101312 [FREE Full text] [CrossRef] [Medline]
    19. Shen N, Levitan M, Johnson A, Bender JL, Hamilton-Page M, Jadad A, et al. Finding a depression app: a review and content analysis of the depression app marketplace. JMIR Mhealth Uhealth 2015;3(1):e16 [FREE Full text] [CrossRef] [Medline]
    20. Arnhold M, Quade M, Kirch W. Mobile applications for diabetics: a systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older. J Med Internet Res 2014;16(4):e104 [FREE Full text] [CrossRef] [Medline]
    21. Schnall R, Iribarren SJ. Review and analysis of existing mobile phone applications for health care-associated infection prevention. Am J Infect Control 2015 Jun;43(6):572-576. [CrossRef] [Medline]
    22. Schnall R, Mosley JP, Iribarren SJ, Bakken S, Carballo-Diéguez A, Brown IW. Comparison of a User-Centered Design, Self-Management App to Existing mHealth Apps for Persons Living With HIV. JMIR Mhealth Uhealth 2015;3(3):e91 [FREE Full text] [CrossRef] [Medline]
    23. Muessig KE, Pike EC, Legrand S, Hightow-Weidman LB. Mobile phone applications for the care and prevention of HIV and other sexually transmitted diseases: a review. J Med Internet Res 2013;15(1):e1 [FREE Full text] [CrossRef] [Medline]
    24. Firth J, Torous J. Smartphone Apps for Schizophrenia: A Systematic Review. JMIR Mhealth Uhealth 2015;3(4):e102 [FREE Full text] [CrossRef] [Medline]
    25. Cajita MI, Gleason KT, Han H. A Systematic Review of mHealth-Based Heart Failure Interventions. J Cardiovasc Nurs 2016;31(3):E10-E22. [CrossRef] [Medline]
    26. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015;3(1):e27 [FREE Full text] [CrossRef] [Medline]
    27. IMS Institute for Healthcare Informatics. Parsippany, NJ: IMS Institute for Healthcare Informatics; 2013 Oct. Patient Apps for Improved Healthcare: From Novelty to Mainstream   URL: http:/​/www.​imshealth.com/​en/​thought-leadership/​ims-institute/​reports/​patient-apps-for-improved-healthcare [accessed 2016-05-24] [WebCite Cache]
    28. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979 Mar;86(2):420-428. [Medline]
    29. McGraw KO, Wong SP. Forming inferences about some intraclass correlation coefficients. Psychological Methods 1996;1(1):30-46. [CrossRef]
    30. Gluckman TJ, Kovacs RJ, Stone NJ, Damalas D, Mullen JB, Oetgen WJ. The ASCVD Risk Estimator App: From Concept to the Current State. J Am Coll Cardiol 2016 Jan 26;67(3):350-352. [CrossRef] [Medline]
    31. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M. Experiences and Results of Applying Tools for Assessing the Quality of a mHealth App Named Heartkeeper. J Med Syst 2015 Nov;39(11):142. [CrossRef] [Medline]
    32. Payne KF, Weeks L, Dunning P. A mixed methods pilot study to investigate the impact of a hospital-specific iPhone application (iTreat) within a British junior doctor cohort. Health Informatics J 2014 Mar;20(1):59-73. [CrossRef] [Medline]
    33. Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, Gruber-Baldini A. WellDoc mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther 2008 Jun;10(3):160-168. [CrossRef] [Medline]
    34. Mani M, Kavanagh DJ, Hides L, Stoyanov SR. Review and Evaluation of Mindfulness-Based iPhone Apps. JMIR Mhealth Uhealth 2015;3(3):e82 [FREE Full text] [CrossRef] [Medline]
    35. Bardus M, van Beurden S, Smith JR, Abraham C. A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. Int J Behav Nutr Phys Act 2016;13(1):35 [FREE Full text] [CrossRef] [Medline]
    36. Charani E, Castro-Sánchez E, Moore S, Holmes A. Do smartphone applications in healthcare require a governance and legal framework? It depends on the application!. BMC Med 2014;12:29 [FREE Full text] [CrossRef] [Medline]
    37. Singh K, Kaitlin D, Newmark L, Rozenblum R, Lee J, Landman A, et al. The Commonwealth Fund.: The Commonwealth Fund; 2016 Feb. Developing a Framework for Evaluating the Patient Engagement, Quality, and Safety of Mobile Health Applications   URL: http://www.commonwealthfund.org/publications/issue-briefs/2016/feb/evaluating-mobile-health-apps [accessed 2016-05-24] [WebCite Cache]
    38. Brown SM, Rozenblum R, Aboumatar H, Fagan MB, Milic M, Lee BS, et al. Defining patient and family engagement in the intensive care unit. Am J Respir Crit Care Med 2015 Feb 1;191(3):358-360. [CrossRef] [Medline]
    39. Boult C, Green AF, Boult LB, Pacala JT, Snyder C, Leff B. Successful models of comprehensive care for older adults with chronic conditions: evidence for the Institute of Medicine's “retooling for an aging America” report. J Am Geriatr Soc 2009 Dec;57(12):2328-2337. [CrossRef] [Medline]


    Abbreviations

    CA-ICC: consistency-of-agreement intraclass correlation
    HF: heart failure
    HFSA: Heart Failure Society of America
    ICC: interclass correlation coefficient
    MARS: Mobile Application Rating Scale
    mHealth: mobile health


    Edited by G Eysenbach; submitted 15.04.16; peer-reviewed by J Torous, C Micallef; comments to author 03.05.16; revised version received 05.05.16; accepted 05.05.16; published 14.06.16

    ©Ruth M. Masterson Creber, Mathew S. Maurer, Meghan Reading, Grenny Hiraldo, Kathleen T. Hickey, Sarah Iribarren. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 14.06.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.