This paper is in the following e-collection/theme issue:

Quality Evaluation and Descriptive Analysis/Reviews of Multiple Existing Mobile Apps (363) Safety and Error Prevention in Health (187)

Published on in Vol 6, No 3 (2018): March

Consumer Mobile Apps for Potential Drug-Drug Interaction Check: Systematic Review and Content Analysis Using the Mobile App Rating Scale (MARS)

Consumer Mobile Apps for Potential Drug-Drug Interaction Check: Systematic Review and Content Analysis Using the Mobile App Rating Scale (MARS)

Consumer Mobile Apps for Potential Drug-Drug Interaction Check: Systematic Review and Content Analysis Using the Mobile App Rating Scale (MARS)

Authors of this article:

Ben YB Kim1 Author Orcid Image ;   Anis Sharafoddini1 Author Orcid Image ;   Nam Tran1 Author Orcid Image ;   Emily Y Wen1 Author Orcid Image ;   Joon Lee2 Author Orcid Image
Article Authors Cited by (66) Tweetations (16) Metrics

    Review

    Health Data Science Lab, School of Public Health and Health Systems, University of Waterloo, Waterloo, ON, Canada

    Corresponding Author:

    Joon Lee, PhD

    Health Data Science Lab

    School of Public Health and Health Systems

    University of Waterloo

    Lyle Hallman North, 3rd Floor

    200 University Avenue W

    Waterloo, ON, N2L 3G1

    Canada

    Phone: 1 519 888 4567 ext 31567

    Fax:1 519 746 6776

    Email: joon.lee@uwaterloo.ca


    Background: General consumers can now easily access drug information and quickly check for potential drug-drug interactions (PDDIs) through mobile health (mHealth) apps. With aging population in Canada, more people have chronic diseases and comorbidities leading to increasing numbers of medications. The use of mHealth apps for checking PDDIs can be helpful in ensuring patient safety and empowerment.

    Objective: The aim of this study was to review the characteristics and quality of publicly available mHealth apps that check for PDDIs.

    Methods: Apple App Store and Google Play were searched to identify apps with PDDI functionality. The apps’ general and feature characteristics were extracted. The Mobile App Rating Scale (MARS) was used to assess the quality.

    Results: A total of 23 apps were included for the review—12 from Apple App Store and 11 from Google Play. Only 5 of these were paid apps, with an average price of $7.19 CAD. The mean MARS score was 3.23 out of 5 (interquartile range 1.34). The mean MARS scores for the apps from Google Play and Apple App Store were not statistically different (P=.84). The information dimension was associated with the highest score (3.63), whereas the engagement dimension resulted in the lowest score (2.75). The total number of features per app, average rating, and price were significantly associated with the total MARS score.

    Conclusions: Some apps provided accurate and comprehensive information about potential adverse drug effects from PDDIs. Given the potentially severe consequences of incorrect drug information, there is a need for oversight to eliminate low quality and potentially harmful apps. Because managing PDDIs is complex in the absence of complete information, secondary features such as medication reminder, refill reminder, medication history tracking, and pill identification could help enhance the effectiveness of PDDI apps.

    JMIR Mhealth Uhealth 2018;6(3):e74

    doi:10.2196/mhealth.8613

    Keywords

    drug interactions; telemedicine; mobile applications; smartphone; consumer health informatics; consumer health information 


    Potential drug-drug interactions (PDDI) have been a prevalent source of preventable problems that can occur in any age group and increase costs to the health care systems [1]. A PDDI occurs when an individual is prescribed two drugs that are known to interact. An occurrence of drug-drug interaction (DDI) is defined as a clinical alteration of the exposure or response to a drug as a result of coadministration. DDIs can be clinically relevant when the result of the interaction warrants the attention of health care professionals (HCPs). When the outcome of the DDI is harmful, it is referred to as an adverse drug reaction (ADR) [2]. DDIs have a profound impact on the safety of patients, and it has been found to be involved in 26% of all ADR-related hospital admissions [3]. Furthermore, in the United States, emergency visits because of ADR cost in average US $3704 per patient [4,5], demonstrating a huge economical impact.

    Most PDDIs are preventable, but it remains a significant problem to patients and the health care system [3,6]. It has been observed that physicians are not always aware of clinically significant drug interactions [7,8] and may underestimate the effects of PDDIs [9]. Other factors such as high workload in pharmacy could also lead to higher risk of PDDIs for patients [10,11]. DDIs have also been identified as a significant portion of the overall ADRs resulting in hospitalization among older adults [12].

    One possible solution that has been proposed is to use a decision support system to detect and avoid PDDIs [7,9]. With the rise of smartphones and mobile apps, decision support systems for PDDIs are now within the reach of consumers and patients and no longer exclusive to HCPs. This is an opportunity that can engage and empower patients by providing necessary tools to detect, avoid, and report ADR events stemming from DDIs [13-17]. The potential benefit for older adults with polypharmacy—the use of multiple medications—is deemed greater because of multiple prescribing providers involved in the care, which is a substantial risk factor for medication errors and ADR events [18].

    Mobile health (mHealth) apps with PDDI decision support are not subject to the Food and Drug Administration regulation [19], and this may pose a substantial threat to the safety of consumers and patients. To our knowledge, the quantity, features, characteristics, or efficacy of the available PDDI mHealth apps on the market have never been systematically assessed. Therefore, understanding the characteristics of these mHealth apps is important in planning future interventions or policies aiming at patient-centered care and patient safety. This study systematically reviewed and assessed PDDI decision support mHealth apps available in Canada through the Google Play Store (Google Inc, Canada) and Apple’s App Store (Apple Inc, Canada) using the Mobile App Rating Scale (MARS) [20].


    Systematic Review Design

    This systematic review adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses systematic review protocol [21] as closely as possible, but it deviated in few instances because of the characteristics of mHealth app databases, which differ from scholarly reference databases for published articles. To ensure the review process is transparent and replicable, the detailed descriptions of each step are provided below.

    App Search Strategy

    Our review aimed to search apps that were publicly available to Canadians in English. Two most popular mobile app databases, Apple’s App Store and the Google Play Store, which account for over 80.0% of mobile apps market in 2016 [22], were searched in this study.

    This study developed a keyword search procedure to identify potentially eligible apps (Textbox 1). First, the searcher was instructed to log out from the Google account on a browser to prevent any personalized search results. Apple’s App Store and the Google Play Store were searched with the search terms related to drug interactions. The search terms were specifically developed to be in all lower case letters and in quotations for consistent and comprehensive search results. As operating systems and apps are updated routinely, searches on both stores were conducted on the same day in December 2016. Additionally, the searches were performed on a designated set of devices and the same network to obtain consistent search results and avoid deviations by personalized search results [23]. Search results were extracted and saved in a spreadsheet for the next stage of app selection.

    App Selection

    Following the search of the two databases, for each search term, all the identified apps were screened in two stages. First, the reviewers verified the eligibility of the apps against the inclusion criteria by reading the apps’ descriptions available in the app stores. This study included apps that claim to check for PDDIs in their description, published in English, and last updated in 2016 or later. Apps were excluded if they targeted nongeneral consumers, passively informed users of PDDIs (does not allow pair-wise or combinational interaction check), checked for drug interactions for pets and animals, and specific to a particular disease or drug class. After screening the results for each search term, the selected app names were aggregated. If an app was listed in both stores, this study considered them separately and examined both versions to capture potentially varying features and user reviews. Second, the authors downloaded and installed the remaining apps from the first step to verify their eligibility one more time. Apps that failed to launch after three attempts on the test devices were excluded. All Apple test devices ran iPhone operating system (iOS, Apple Inc) 10, and all Android test devices ran Android 6.0.

    Data Collection Process

    A set of general information about the apps were extracted following previous app review studies [24,25]. General app information provides contextual information such as availability, affordability, and user satisfaction level. A set of secondary features that can further empower end users beyond the PDDI check feature was identified from literature review [24,26-28].

    Search strategy with an example for Google Play Store.

    Preparing your device for the search:

    • Connected to the University of Waterloo network
    • Log out from Google in your browser

    Search procedure:

    • Search the following terms in the respective store
    • Search terms must be in quotation (eg, “drug interaction”)
    • All search terms should be entered in lower case letters

    Search terms (number of hits)

    1. drug interaction (66)
    2. drugs interaction (8)
    3. drug interactions (193)
    4. drugs interactions (16)
    5. drug-interaction (66)
    6. pill interaction (3)
    7. pills interaction (0)
    8. pill interactions (3)
    9. pills interactions (0)
    10. pill-interaction (3)
    11. medication interaction (10)
    12. medications interaction (0)
    13. medication interactions (192)
    14. medications interactions (0)
    15. medication-interaction (10)
    Textbox 1. Search strategy with an example for Google Play Store.

    In summary, the two extracted sets of information were as follows: (1) general information about the apps: last updated date, price, and user rating and (2) other relevant secondary features that the apps offered:

    • Medication management related features: reminder to take medication, reminder to refill medication, medication history tracking, pill identification, searching medication using generic or brand names, and access to medication database
    • Security and privacy related features: password protection for user data and multiple user support
    • Data sharing and social media: sharing user data with a third party
    • Clinician and technical support: customer support

    Multimedia Appendix 1 presents the secondary features extracted and examined for each app.

    Critical Appraisal of the Apps (Quality Assessment)

    The MARS, a 23-item, expert-based rating scale with a purpose of assessing the quality of mHealth apps, was used to critically and systematically evaluate the quality of the mHealth apps [20] (See Multimedia Appendix 2 for a detailed MARS score for all included apps). Each question from MARS used a 5-point scale (1=inadequate, 2=poor, 3=acceptable, 4=good, and 5=excellent). This expert scale consists of multiple dimensions that assess different quality aspects of apps, including end-user engagement, features, aesthetics, content quality, and subjective quality [20]. This expert rating scale has been increasingly adopted in recent years for evaluating mHealth apps such as mindfulness [29], weight loss [25,30], smoking cessation [30], self-care [31], online well-being [32], and medication adherence [24]. A previous study has shown high internal consistencies in the total score and subscales, as well as strong interrater reliability (IRR) [20]. Moreover, use of a standardized assessment scale such as MARS for evaluating mHealth apps has been recommended by various researchers [33-35]. The popularity of MARS led to the further development of an Italian version [36] and an end-user version for nonresearchers [37].

    The last dimension of MARS is app subjective quality, which takes the subjective opinions of the reviewers. To ensure the quality assessment process is as consistent and objective as possible, the subjective quality dimension was omitted from this review. A previous study that employed MARS as an objective method to assess quality also excluded the subjective quality dimension [25]. Instead, relevant information was captured from the app databases, including the price and app ratings.

    Before rating the apps, each rater read and familiarized themselves with the MARS protocol. A group discussion was followed to achieve a consensus on the rating criteria, and the first app was rated as a group. The need for an objective example of PDDIs arose for the MARS questions (#15 and #16) that assess comprehensiveness and accuracy of the content and information. On the basis of a careful review of the literature [38,39], we developed a list of PDDIs with 20 true positive and six false positive examples (Multimedia Appendix 3). The percentage of correctly identified and described PDDIs was scaled to a range from 1 to 5 for questions #15 and #16. No previous studies have reported on the details of how the accuracy and comprehensiveness of app content were assessed.

    Two raters assessed each app individually. Weighted kappa, Krippendorff alpha, and intraclass correlation (ICC) were used to estimate IRR for MARS tool. The kappa value was assessed by putting quadratic weights for differing values. The ICC coefficient was calculated with a two-way random model and for agreement level. The weighted kappa, Krippendorff alpha, and ICC were calculated per dimension and for all apps.

    Statistical Analyses

    Each dimension in MARS was analyzed using the mean value as recommended by the developers [20]. The difference in app quality between the two app stores was analyzed by t tests. The relationships among four dimensions of the MARS score—MARS total score, price, average rating, and number of features—were examined by the Spearman correlation. A significance level of .05 was used in this study. All analyses were performed in R version 3.3.2 (R Foundation for Statistical Computing, Vienna, Austria).


    Systematic Search Results

    The app store search was conducted in December 2016. This study identified 570 apps from Google Play and 582 apps from Apple App Store (Figure 1). After removing duplicates in each database, the authors reviewed the descriptions of 247 apps against the inclusion and exclusion criteria (Figure 1). Apps found to be eligible based on their descriptions (n=44) were installed for another round of review against the criteria (Figure 1). Review was initiated for 25 apps, but the authors excluded two additional apps identified as duplicates in multi-language versions, leaving a total of 23 apps for this study (Figure 1).

    ‎
    Figure 1. App selection process.
    View this figure

    General Information

    Twenty-three apps—12 from Apple App Store and 11 from Google Play—were developed by 15 developers. Seven apps were listed in both stores. Table 1 summarizes the general information of the reviewed apps and the mean MARS scores.

    There were five paid apps, three from Apple App Store and two from Google Play, with an average price of $7.19 CAD. The average prices of paid apps were $7.32 CAD and $6.99 CAD for Apple’s App Store and the Google Play Store, respectively. Four apps, two apps from each store, “Drug Interactions” and “Prescription Checker” by the same developer, were functionally identical but listed at two different prices: $10.99 CAD and $6.99 CAD in Apple’s App Store and $9.33 CAD and $4.65 CAD in the Google Play Store, respectively.

    The last updated dates for the Android apps were from April 2016 to December 2016, whereas the iOS apps ranged from July to December 2016.

    The average rating for the apps from the Google Play Store was 3.82, with a minimum of 2.1 and a maximum of 4.8 (interquartile range, IQR 0.85). On the other hand, the apps from Apple’s App Store averaged 4.5 based on the two apps with valid user ratings.

    App Features

    Secondary features, features other than PDDI check, were extracted and examined for each app. On average, they had 3.67 features with a minimum of zero for “DrugChecker— Interactions (Lite)” and a maximum of eight for GenieMD in both stores (IQR 3). The overall number of apps per secondary feature is shown in Figure 2. Medication refill reminder was among the least incorporated features (2/23). The option to search medications with their generic and brand name (20/23), multiple user support (17/23), access to the app’s medication database (16/23), password protection (14/23), and customer support (14/23) were the most common features.

    Table 1. General information about the eligible apps, developer, tested version, cost, average rating, and mean Mobile App Rating Scale (MARS) score. iOS: iPhone operating system. NA: not available.
    App number and namePlatformDeveloperTested app
    version    		Cost
    ($ CAD)    		Average rating
    (out of 5)    		Mean MARS
    score (out of 5)
    1Drug center—pediatric oncalliOSPediatric Oncall3FreeNA3.15
    2Drug interactionsiOSPierre Chaillet1.5.310.99NA2.29
    3DrugChecker—Interactions (Lite)iOSSYSTEM YOSHII1.2.1FreeNA2.00
    4Drugs.com Medication GuideiOSDrugsite Trust2.7.24Free44.06
    5GenieMDiOSGenieMD7.4Free53.92
    6MyRxProfileiOSMyRxProfile1.0.2FreeNA3.02
    7PharmaGuideiOSAsif Baig1.0.5FreeNA1.94
    8PharmazamiOSPharmazam2FreeNA2.94
    9Pharmacist Pro—Drug Interactions CheckeriOSYury Dubovoy23.99NA3.60
    10PillSync Drug Facts IdentifieriOSScanIDme1.2FreeNA3.23
    11Prescription CheckeriOSPierre Chaillet1.16.99NA2.29
    12ZibdyHealthiOSZibdy1.5FreeNA2.29
    13Assist IE—Drug InteractionsAndroidInfomed Mobile1.0.41Free3.93.60
    14Assist UK—Drug InteractionsAndroidInfomed Mobile1.0.41Free4.03.60
    15CVS CaremarkAndroidCVS Caremark4.15Free3.54.10
    16Drug Center—Pediatric OncallAndroidPediatric Oncall3.2Free4.43.33
    17Drug InteractionsAndroidPierre Chaillet1.5.4$9.332.72.29
    18Drugs.com Medication GuideAndroidDrugs.com2.0.7.28Free4.34.06
    19Epocrates PlusAndroidEpocrates17.1Free4.34.25
    20GenieMDAndroidGenieMD5.9.9.54Free4.83.75
    21PillSync Drug Facts IdentifierAndroidScanIDme4.3.0Free2.12.29
    22Prescription CheckerAndroidPierre Chaillet1.5.4$4.653.52.29
    23ZibdyHealthAndroidZibdy2.0Free4.53.60
    ‎
    Figure 2. The number of apps that contain the secondary features listed on the x-axis.
    View this figure
    ‎
    Figure 3. Mobile App Rating Scale (MARS) dimension scores. Each point represents the score for an individual app. The box plot shows the median, first, and third quartiles and minimum and maximum scores.
    View this figure

    Critical Appraisal of App Quality

    Overall App Quality

    The mean MARS score of the 23 apps was 3.05 (IQR 1.55), with a maximum of 4.40 for “Epocrates Plus” and a minimum of 1.87 for “PharmaGuide” (Table 1). The mean MARS score for the apps from Google Play and App Store were comparable at 3.25 and 2.86, respectively, with no statistical difference (P=.96). The IRR between two raters as assessed by the weighted kappa was .63 (95% CI 0.58-0.68), the ICC was .64 (95% CI 0.59-0.68), and the Krippendorff alpha was .63 (95% CI 0.58-0.66). Detailed IRR results are presented in Multimedia Appendix 4.

    The mean scores of the four dimensions of MARS were examined to investigate the magnitude of the differences in quality in each dimension. Functionality dimension resulted in the highest mean score (3.52), whereas engagement dimension showed the lowest average score (2.75). The functionality dimension had the most variability (Figure 3).

    Relationships Between App Characteristics and Quality

    General and functional characteristics of the 23 apps were examined for a correlation with the MARS score (Table 2). The general and functional characteristics, including average user rating and total number of features, were statistically significantly associated with the total MARS score (Table 2). Statistically significant associations were observed between the general and functional characteristics, including the total number of features and the price and average user rating (Table 2). Within the MARS dimensions, all were statistically significantly correlated with each other except for information dimension (Table 2).

    Table 2. Correlations among total Mobile App Rating Scale (MARS) score, four MARS dimension scores, price, rating, and number of features.
    CharacteristicsMARSPriceAverage ratingNumber of features


    		TotalEngagementFeatureAestheticsInformation


    MARS








    		Total1.00







    		Engagement.87a1.00






    		Feature.72a.49b1.00





    		Aesthetics.88a.91a.64a1.00




    		Information.43b.30−.08.121.00


    Price−.37−.49b−.47b−.55a.351.00

    Rating.61a.49b.39.41b.43b−.261.00
    Number of features.47b.70a.06.56a.16 −.43b.42b1.00

    aP<.01.

    bP<.05.


    Principal Findings

    In this app review study, a systematic search strategy was used to find PDDI apps. To our knowledge, this is the first systematic review on apps that offers decision support for PDDI checking. The 23 included apps were analyzed to extract general characteristics and functional characteristics, and their quality was assessed using MARS. Only five of the 23 apps (22%) were paid apps. This proportion of paid apps is consistent with other studies that systematically reviewed the Google Play Store and Apple’s App Store [24,25]. App price had statistically significant negative correlations with three of four MARS dimensions and number of features. This demonstrates that app quality is not always represented by the selling price. A plausible explanation for this counterintuitive and inverse relationship is that free apps may have been developed by companies and organizations with sufficient resources; hence, apps were developed to expand consumer reach, whereas individual developers who may have limited resources may rely on generating revenue from app sales, while the quality of app may not be as high as the apps developed by companies and organizations that can afford to hire a group of expert developers. Further research should investigate the relationship between the price of consumer mHealth apps and its quality, as well as its impact on consumer perception.

    The primary features of the examined mHealth apps were providing drug information to users and checking for PDDIs. Despite this aim of these apps, a low average score in the information dimension was found based on MARS. This indicates that the PDDI check feature is of low quality, delivering inaccurate and potentially unsafe information about PDDIs. In particular, MARS questions #15 and #16, which assessed the accuracy and comprehensiveness, scored on average 2.9 and 2.4, respectively. This is alarming as only slightly more than half of 26 investigated PDDIs (58%, 3/5) have been identified by the apps. To worsen the problem, less than half of the correctly identified PDDIs (2.4 out of 5) have correctly described the interactions. Inability to detect PDDIs and providing incomplete and incorrect information is a significant threat to patient safety. It also diminishes mHealth app’s value as an avenue for patient empowerment. It must be noted that there was a large variability in the accuracy of PDDIs among the tested apps, where 48% (11/23) apps scored 4 or higher out of 5 for question #15, whereas 30% (7 / 23) apps scored less than 1 out of 5. This polarized quality of information found in mHealth apps further raises the question about the tools available for consumers to evaluate and select high quality apps. The average user rating was significantly correlated with the information dimension, and it indicates that the average user rating can potentially be an important tool for selecting mHealth apps. There are other resources available such as app clearinghouses that make recommendations for mHealth apps to consumers based on the results from systematically evaluating the usability, quality, accuracy, or evidence of the app and its content [40]. Examples of app clearinghouses include National Health Service Health App Library and iMedicalApps [40]. These app clearinghouses hold promise to enhance consumer safety of mHealth apps, but they have not been investigated against MARS or other validated tools that assess the quality of mHealth apps.

    The low average MARS score for the engagement dimension can be partially explained by the primary purpose of the included apps. The investigated apps work as a reference to check for PDDIs, and these apps do not rely on user engagement to elicit behaviour change. On the other hand, other mHealth apps that focus on behaviour change tend to score higher in the engagement dimension, as the success of behaviour change may heavily depend on how successfully they engage the user [24].

    Most MARS dimensions were correlated with each other except information. This is consistent with the findings from Bardus and colleagues who assessed weight management mHealth apps [25], where all dimensions but the information and engagement dimensions were significantly associated. A very strong correlation between the aesthetics and engagement dimensions can be explained by many user interface design, and usability studies that found attractive and appealing aesthetics lead to greater user engagement and perceived usability [41-43]. Interpreting the correlation between the total MARS score and each dimension’s score should take caution as the total MARS score is derived from the scores from all dimension of MARS. The number of features was strongly correlated with the engagement dimension but not with the features dimension that measures functionality, performance, and ease of use [20]. This result may represent the trade-off between ease of use and the complexity of an app that attempts to provide more features at the cost of performance. A similar relationship has been found in a previous website design and usability study [44].

    Secondary Features Offered to Consumers

    Besides the PDDI check feature, maintaining medication adherence is a challenging problem in individuals taking medication, particularly for older adults [45,46] and those with chronic diseases [47]. Improving medication adherence can ensure the effectiveness of a treatment, thereby impacting maintaining health and managing chronic diseases [48]. There are many barriers for medication adherence, but forgetfulness has been reported as the most common cause, and much research has focused on overcoming this barrier [49,50]. A well-researched solution to overcoming forgetfulness is medication reminders and refill reminders [47,51]. Such reminders have increased patient medication adherence by encouraging timely refill and further demonstrated feasibility in cognitively impaired populations [47,51]. Therefore, these features can also be useful to individuals using PDDI apps. The usefulness of refill reminders has been acknowledged by the US government and made the refill reminder an exception to the Health Insurance Portability and Accountability Act [52]. Despite sufficient ground for implementing these features, only two apps featured a refill reminder (GenieMD in both stores), whereas five had a medication reminder.

    Patients with comorbidities are usually cared for by a general physician and several specialists, which tends to lead to a heterogeneous list of medications [53,54]. The PDDI check feature can inspect for possible adverse effects, but this would be accurate only when the medication list is complete. Unfortunately, only 30% (7/23) of the reviewed apps had a feature to track medication history (Multimedia Appendix 1). Medication history tracking is also important in understanding PDDIs for drugs with long half-lives or over-the-counter drugs [55]. Therefore, mHealth apps that can track the history of medication can further prevent other drug complications. Such a feature can empower patients by enabling them to take charge of their medication list and minimizing PDDIs stemming from many HCPs with multiple prescriptions.

    Every over-the-counter and prescription medication must have a unique appearance and imprint code for identification by the Food and Drug Administration [56]. Code imprint, size, color, and shape of the medication together permit identification of the product and manufacturer. However, using this identification system can be difficult for end users, and only 22% (5/23) of the apps had a feature to automatically identify pills from its physical attributes (Multimedia Appendix 1). Identification by drug name can also be difficult because of the discrepancies between generic and brand names. This review found that 87% (20/23) of the apps allow searching by both generic and brand names, and 70% (16/23) provide further drug information by allowing users to access a drug database. These features can help older adults who have developed polypharmacy to identify and distinguish drugs from one another, as a large number of medications and confusing names are often causes for medication error, even among trained clinicians [57].

    Another issue that PDDI app users may be concerned about is data security as privacy is a major concern for collecting personal health information [58]. Overall, 61% (14/23) of the PDDI apps had password protection, and 74% (17/23) had support for multiple users on the same device (Figure 2). Given that smartphones and tablets can be protected with a password, an additional app-level password protection provides another level of security. Information and Privacy Commissioner of Ontario [59] and the Health Insurance Portability and Accountability Act state that password protection is required, but this may not be secure enough. Data encryption is recommended for added security. This is an area that can be greatly improved with a more stringent guideline and oversight by regulators and governments. Moreover, future research should investigate the level of data encryption presented in mHealth apps and its implication for consumers.

    Our review also investigated availability of other features related to medication management in the apps. For instance, one study [60] has described data sharing via social networking sites as a potential communication platform to pharmaceutical companies to give feedback. Furthermore, social media can facilitate the interactions among patients, clinicians, researchers, and vendors [60]. The capabilities of data exporting, synthesized reporting for clinicians, and sharing on social media were found only in 22% (5/23) of the apps. In the context of mHealth apps that check for PDDIs, social media can provide a medium for consumers to interact with other drug users to share side effects and other relevant information. As Steele described [60], it may also help pharmaceutical companies interact with the users and gain insights into rare side effects, PDDIs, or high-risk subpopulations such as older adults.

    Finally, in the event that information provided by the apps is not satisfactory, users should be able to get additional help. Of all, 61% of the apps (14/23) provided some level of customer support. Given the seriousness of potential ADRs that can be caused from the exposure to the PDDIs, an option to contact a clinician, preferably a pharmacist, would be ideal. It is worthwhile to note that no apps have provided contact information for reporting ADRs to local regulatory bodies. Providing a formal way to report potential ADRs to regulatory bodies can enhance public health programs for monitor PDDIs and ADRs.

    Limitations

    This review is not without limitations. We limited our focus on English apps available in Canada, but other researchers may benefit from extending this review to other regions and languages. Moreover, mHealth apps are frequently updated, and new apps are published daily. Fast evolving app market can limit the generalizability of the results. Another limitation of reviewing app stores is the app databases’ nontransparent search algorithms. Although we reported our search strategy as transparent as possible, the underlying search algorithm can change without the public’s knowledge. This can undermine the reproducibility of our study. Finally, our review unveiled the quality of existing PDDI mHealth apps on the market, but this does not necessarily translate to how consumers use these apps in the real world. This knowledge gap should be further investigated in future research.

    Conclusions

    Checking for PDDIs has been a task reserved for clinicians and pharmacists. With the increased popularity of smartphones and other information technologies, they promise more features and functionalities to enhance our lives and well-being. In this study, we searched the most popular mobile app databases and found 23 apps that can check for PDDIs. Some of these apps provided high quality, accurate, and comprehensive information about PDDIs. However, not all apps conformed to high standards, and given the high stake of incorrect drug information, the need for oversight was clear to ensure end-user safety. We also identified secondary features that future apps should incorporate to further benefit the end users. These features can support medication management, improve data security and privacy, and facilitate communications.

    Acknowledgments

    This work was supported by a grant from the AGE-WELL Network of Centres of Excellence.

    Authors' Contributions

    The idea for the study was conceived, and the review design was established by AS and BK in 2016. The search and screening phases were performed by AS, BK, EW, and NT. AS, BK, and NT extracted data from included apps and performed the analysis. AS, BK, and NT contributed to preparing and developing the manuscript. EW generated the flowchart and organized references. JL reviewed and refined the manuscript.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Extracted secondary features and their presence among included apps.

    PDF File (Adobe PDF File), 357KB
    ‎

    Multimedia Appendix 2

    A detailed MARS score for all included apps.

    PDF File (Adobe PDF File), 382KB
    ‎

    Multimedia Appendix 3

    List of drug-drug interactions tested for the MARS #15 and #16.

    PDF File (Adobe PDF File), 313KB
    ‎

    Multimedia Appendix 4

    Detailed inter-rater reliability as analyzed by the weighted kappa, intraclass correlation, and Krippendorff alpha for each MARS dimension.

    PDF File (Adobe PDF File), 188KB
    1. Taylor Jr R, V Pergolizzi Jr J, Puenpatom RA, Summers KH. Economic implications of potential drug-drug interactions in chronic pain patients. Expert Rev Pharmacoecon Outcomes Res 2013 Dec;13(6):725-734. [CrossRef] [Medline]
    2. Scheife RT, Hines LE, Boyce RD, Chung SP, Momper JD, Sommer CD, et al. Consensus recommendations for systematic evaluation of drug-drug interaction evidence for clinical decision support. Drug Saf 2015 Feb;38(2):197-206 [FREE Full text] [CrossRef] [Medline]
    3. Miranda V, Fede A, Nobuo M, Ayres V, Giglio A, Miranda M, et al. Adverse drug reactions and drug interactions as causes of hospital admission in oncology. J Pain Symptom Manage 2011 Sep;42(3):342-353. [CrossRef] [Medline]
    4. Yee JL, Hasson NK, Schreiber DH. Drug-related emergency department visits in an elderly veteran population. Ann Pharmacother 2005 Dec;39(12):1990-1995. [CrossRef] [Medline]
    5. Dechanont S, Maphanta S, Butthum B, Kongkaew C. Hospital admissions/visits associated with drug-drug interactions: a systematic review and meta-analysis. Pharmacoepidemiol Drug Saf 2014 May;23(5):489-497. [CrossRef] [Medline]
    6. Juurlink DN, Mamdani M, Kopp A, Laupacis A, Redelmeier DA. Drug-drug interactions among elderly patients hospitalized for drug toxicity. J Am Med Assoc 2003 Apr 02;289(13):1652-1658. [CrossRef] [Medline]
    7. Glassman PA, Simon B, Belperio P, Lanto A. Improving recognition of drug interactions: benefits and barriers to using automated drug alerts. Med Care 2002 Dec;40(12):1161-1171. [CrossRef] [Medline]
    8. Langdorf MI, Fox JC, Marwah RS, Montague BJ, Hart MM. Physician versus computer knowledge of potential drug interactions in the emergency department. Acad Emerg Med 2000 Nov;7(11):1321-1329 [FREE Full text] [Medline]
    9. Ko Y, Malone DC, Skrepnek GH, Armstrong EP, Murphy JE, Abarca J, et al. Prescribers' knowledge of and sources of information for potential drug-drug interactions: a postal survey of US prescribers. Drug Saf 2008;31(6):525-536. [Medline]
    10. Becker ML, Caspers PW, Kallewaard M, Bruinink RJ, Kylstra NB, Heisterkamp S, et al. Determinants of potential drug-drug interaction associated dispensing in community pharmacies in the Netherlands. Pharm World Sci 2007 Apr;29(2):51-57 [FREE Full text] [CrossRef] [Medline]
    11. Malone DC, Abarca J, Skrepnek GH, Murphy JE, Armstrong EP, Grizzle AJ, et al. Pharmacist workload and pharmacy characteristics associated with the dispensing of potentially clinically important drug-drug interactions. Med Care 2007 May;45(5):456-462. [CrossRef] [Medline]
    12. Pirmohamed M, James S, Meakin S, Green C, Scott AK, Walley TJ, et al. Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820 patients. Br Med J 2004 Jul 3;329(7456):15-19 [FREE Full text] [CrossRef] [Medline]
    13. Awé C, Lin SJ. A patient empowerment model to prevent medication errors. J Med Syst 2003 Dec;27(6):503-517. [Medline]
    14. van Grootheest K, de Jong-van den Berg L. Patients' role in reporting adverse drug reactions. Expert Opin Drug Saf 2004 Jul;3(4):363-368. [Medline]
    15. Basch E. The missing voice of patients in drug-safety reporting. N Engl J Med 2010 Mar 11;362(10):865-869 [FREE Full text] [CrossRef] [Medline]
    16. Longtin Y, Sax H, Leape LL, Sheridan SE, Donaldson L, Pittet D. Patient participation: current knowledge and applicability to patient safety. Mayo Clin Proc 2010 Jan;85(1):53-62 [FREE Full text] [CrossRef] [Medline]
    17. Britten N. Medication errors: the role of the patient. Br J Clin Pharmacol 2009 Jun;67(6):646-650 [FREE Full text] [CrossRef] [Medline]
    18. Marek KD, Antle L. Medication management of the community-dwelling older adult. In: Hughes R, editor. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, Maryland: Agency for Healthcare Research and Quality (US); Apr 2008:499-536.
    19. University of Houston Law Center. 2015. FDA Regulation of Mobile Medical Applications: Current Changes and Potential Issues   URL: http:/​/www.​law.uh.edu/​Healthlaw/​perspectives/​2015/​Torres%20Cot_FDA%20Regulation%20of%20Mobile%20Medical%20Applications.​pdf [accessed 2017-01-25] [WebCite Cache]
    20. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015;3(1):e27 [FREE Full text] [CrossRef] [Medline]
    21. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015 Jan;4:1 [FREE Full text] [CrossRef] [Medline]
    22. Statista. App stores - Statistics & Facts   URL: https://www.statista.com/topics/1729/app-stores/ [accessed 2017-07-27] [WebCite Cache]
    23. Horing B, Kulick M. Googleblog. 2009. Personalized Search for everyone   URL: https://googleblog.blogspot.com/2009/12/personalized-search-for-everyone.html [accessed 2017-09-06] [WebCite Cache]
    24. Santo K, Richtering SS, Chalmers J, Thiagalingam A, Chow CK, Redfern J. Mobile phone apps to improve medication adherence: a systematic stepwise process to identify high-quality apps. JMIR Mhealth Uhealth 2016 Dec 02;4(4):e132 [FREE Full text] [CrossRef] [Medline]
    25. Bardus M, van Beurden SB, Smith JR, Abraham C. A review and content analysis of engagement, functionality, aesthetics, information quality, and change techniques in the most popular commercial apps for weight management. Int J Behav Nutr Phys Act 2016;13(1):35 [FREE Full text] [CrossRef] [Medline]
    26. Dayer L, Heldenbrand S, Anderson P, Gubbins PO, Martin BC. Smartphone medication adherence apps: potential benefits to patients and providers. J Am Pharm Assoc (2003) 2013;53(2):172-181 [FREE Full text] [CrossRef] [Medline]
    27. Krueger KP, Berger BA, Felkey B. Medication adherence and persistence: a comprehensive review. Adv Ther 2005;22(4):313-356. [Medline]
    28. Kim BY, Lee J. Smart devices for older adults managing chronic disease: a scoping review. JMIR Mhealth Uhealth 2017 May 23;5(5):e69 [FREE Full text] [CrossRef] [Medline]
    29. Mani M, Kavanagh DJ, Hides L, Stoyanov SR. Review and evaluation of mindfulness-based iPhone apps. JMIR Mhealth Uhealth 2015;3(3):e82 [FREE Full text] [CrossRef] [Medline]
    30. Patel R, Sulzberger L, Li G, Mair J, Morley H, Shing MN, et al. Smartphone apps for weight loss and smoking cessation: Quality ranking of 120 apps. N Z Med J 2015 Sep 04;128(1421):73-76. [Medline]
    31. Anderson K, Burford O, Emmerton L. Mobile health apps to facilitate self-care: a qualitative study of user experiences. PLoS One 2016;11(5):e0156164 [FREE Full text] [CrossRef] [Medline]
    32. Antezana G, Bidargaddi N, Blake V, Schrader G, Kaambwa B, Quinn S, et al. Development of an online well-being intervention for young people: an evaluation protocol. JMIR Res Protoc 2015;4(2):e48 [FREE Full text] [CrossRef] [Medline]
    33. Kenny R, Dooley B, Fitzgerald A. Feasibility of “CopeSmart”: a telemental health app for adolescents. JMIR Ment Health 2015 Aug 10;2(3):e22 [FREE Full text] [CrossRef] [Medline]
    34. Hingle M, Patrick H. There are thousands of apps for that: navigating mobile technology for nutrition education and behavior. J Nutr Educ Behav 2016 Mar;48(3):213-8.e1. [CrossRef] [Medline]
    35. Baumel A, Muench F. Heuristic evaluation of eHealth interventions: establishing standards that relate to the therapeutic process perspective. JMIR Ment Health 2016 Jan 13;3(1):e5 [FREE Full text] [CrossRef] [Medline]
    36. Domnich A, Arata L, Amicizia D, Signori A, Patrick B, Stoyanov S, et al. Development and validation of the Italian version of the Mobile Application Rating Scale and its generalisability to apps targeting primary prevention. BMC Med Inform Decis Mak 2016;16:83 [FREE Full text] [CrossRef] [Medline]
    37. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and validation of the user version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016;4(2):e72 [FREE Full text] [CrossRef] [Medline]
    38. Saverno KR, Hines LE, Warholak TL, Grizzle AJ, Babits L, Clark C, et al. Ability of pharmacy clinical decision-support software to alert users about clinically important drug-drug interactions. J Am Med Inform Assoc 2011 Jan;18(1):32-37 [FREE Full text] [CrossRef] [Medline]
    39. Phansalkar S, Desai AA, Bell D, Yoshida E, Doole J, Czochanski M, et al. High-priority drug-drug interactions for use in electronic health records. J Am Med Inform Assoc 2012 Sep;19(5):735-743. [CrossRef] [Medline]
    40. Boudreaux ED, Waring ME, Hayes RB, Sadasivam RS, Mullen S, Pagoto S. Evaluating and selecting mobile health apps: strategies for healthcare providers and healthcare organizations. Transl Behav Med 2014 Dec;4(4):363-371 [FREE Full text] [CrossRef] [Medline]
    41. Sonderegger A, Sauer J. The influence of design aesthetics in usability testing: effects on user performance and perceived usability. Appl Ergon 2010 May;41(3):403-410. [CrossRef] [Medline]
    42. Lindgaard G, Dudek C, Sen D, Sumegi L, Noonan P. An exploration of relations between visual appeal, trustworthiness and perceived usability of homepages. ACM Trans Comput Hum Interact 2011 Apr 01;18(1):1-30. [CrossRef]
    43. Hartmann J, Sutcliffe A, De Angeli A. Towards a theory of user judgment of aesthetics and user interface quality. ACM Trans Comput Hum Interact 2008 Nov 01;15(4):1-30. [CrossRef]
    44. Palmer JW. Web site usability, design, and performance metrics. Inf Syst Res 2002 Jun;13(2):151-167. [CrossRef] [Medline]
    45. MacLaughlin EJ, Raehl CL, Treadway AK, Sterling TL, Zoller DP, Bond CA. Assessing medication adherence in the elderly: which tools to use in clinical practice? Drugs Aging 2005;22(3):231-255. [Medline]
    46. Who.int. Geneva: World Health Organization; 2003. Adherence to Long-Term Therapies: Evidence for Action   URL: http://www.who.int/chp/knowledge/publications/adherence_full_report.pdf?ua=1 [WebCite Cache]
    47. Ascione FJ, Brown GH, Kirking DM. Evaluation of a medication refill reminder system for a community pharmacy. Patient Educ Couns 1985 Jun;7(2):157-165. [Medline]
    48. Stawarz K, Cox AL, Blandford A. Don't forget your pill!: designing effective medication reminder apps that support users' daily routines. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2014 Apr Presented at: CHI 2014; April 26-May 01, 2014; Toronto p. 2269-2278. [CrossRef]
    49. Tran N, Coffman JM, Sumino K, Cabana MD. Patient reminder systems and asthma medication adherence: a systematic review. J Asthma 2014 Jun;51(5):536-543. [CrossRef] [Medline]
    50. Ershad Sarabi R, Sadoughi F, Jamshidi Orak R, Bahaadinbeigy K. The effectiveness of mobile phone text messaging in improving medication adherence for patients with chronic diseases: a systematic review. Iran Red Crescent Med J 2016 May;18(5):e25183 [FREE Full text] [CrossRef] [Medline]
    51. Tao D, Xie L, Wang T, Wang T. A meta-analysis of the use of electronic reminders for patient adherence to medication in chronic disease care. J Telemed Telecare 2015 Jan;21(1):3-13. [CrossRef] [Medline]
    52. US Department of Health & Human Services. Washington D.C: US Department of Health & Human Services The HIPAA Privacy Rule and Refill Reminders and Other Communications about a Drug or Biologic Currently Being Prescribed for the Individual   URL: https://www.hhs.gov/hipaa/for-professionals/privacy/guidance/refill-reminders/index.html [WebCite Cache]
    53. Cios KJ, Moore GW. Uniqueness of medical data mining. Artif Intell Med 2002;26(1-2):1-24. [Medline]
    54. Iglesias JE, Rocks K, Jahanshad N, Frias-Martinez E, Andrada LP, Bui AA. Tracking medication information across medical records. AMIA Annu Symp Proc 2009 Nov 14;2009:266-270 [FREE Full text] [Medline]
    55. Fitzgerald RJ. Medication errors: the importance of an accurate drug history. Br J Clin Pharmacol 2009 Jun;67(6):671-675 [FREE Full text] [CrossRef] [Medline]
    56. Accessdata.fda. Silver Spring: U.S. Food & Drug Administration; 2017. CFR - Code of Federal Regulations Title 21   URL: http:/​/www.​accessdata.fda.gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm?CFRPart=50&showFR=1&subpartNode=21:1.​0.​1.​1.​20.​2 [accessed 2017-05-30] [WebCite Cache]
    57. Cheragi MA, Manoocheri H, Mohammadnejad E, Ehsani SR. Types and causes of medication errors from nurse's viewpoint. Iran J Nurs Midwifery Res 2013 May;18(3):228-231 [FREE Full text] [Medline]
    58. Liu LS, Shih PC, Hayes GR. Barriers to the adoption and use of personal health record systems. In: Proceedings of the 2011 iConference.: ACM; 2011 Presented at: iConference 2011; 2011; Washington, USA p. 363-370. [CrossRef]
    59. Information and Privacy Commissioner. Ipc.on.ca. 2014. Safeguarding Privacy on Mobile Devices   URL: https://www.ipc.on.ca/wp-content/uploads/Resources/safeguarding-privacy-on-mobile-devices-e.pdf [accessed 2017-05-20] [WebCite Cache]
    60. Steele R. Social media, mobile devices and sensors: categorizing new techniques for health communication. New York City: IEEE; 2011 Nov 28 Presented at: Fifth International Conference on Sensing Technology (ICST); 28 November-1 December, 2011; Palmerston North, New Zealand p. 187-192. [CrossRef]

    ‎
    ADR: adverse drug reaction
    DDI: drug-drug interaction
    HCP: health care professional
    ICC: intraclass correlation
    iOS: iPhone operating system
    IQR: interquartile range
    IRR: interrater reliability
    MARS: Mobile App Rating Scale
    mHealth: mobile health
    PDDI: potential drug-drug interactions

    Edited by G Eysenbach; submitted 31.07.17; peer-reviewed by M Bardus, R Boyce; comments to author 24.08.17; revised version received 07.12.17; accepted 21.01.18; published 28.03.18

    Copyright

    ©Ben YB Kim, Anis Sharafoddini, Nam Tran, Emily Y Wen, Joon Lee. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 28.03.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.