The NYSACC was created in 2012 by IPRO, the CMS-designated QIO for New York State, to advance the drug safety priorities identified in the CMS QIN-QIO 10th Statement of Work [26]. The NYSACC’s multidisciplinary membership consists of more than 150 representatives from clinical practice, academia, industry, and advocacy organizations with interest in anticoagulation management quality. The NYSACC identified inadequate availability of tools for periprocedural management as a barrier to quality anticoagulant management that warranted action.

In the spring of 2013, the group developed and disseminated a novel clinical tool, the MAPPP Tool. This tool, disseminated in hard copy and pdf formats, utilized a 3×3 matrix to help clinicians simultaneously categorize underlying thromboembolic risk (high, moderate, or low) and procedural bleeding risk (high, low, or minimal) and provided evidence-based guidance on (1) whether the anticoagulant should be interrupted; (2) timing for preprocedural discontinuation of anticoagulant (if necessary); (3) whether “bridging” with heparin products is warranted, with details of timing and laboratory monitoring; and (4) timing and dosing of re-initiation of anticoagulant in the postprocedural period. The tool included guidance for all oral anticoagulants available for use in the United States at the time of release, including apixaban, dabigatran, edoxaban, rivaroxaban, and warfarin.

Recognizing the barriers to disseminating and updating the MAPPP in its original form, and appreciating the limited number of mutually exclusive steps involved in navigating the tool, it was determined that the structure of the MAPPP lent itself to the creation of a mobile app that would potentially allow for broader dissemination, remote updates to reflect most current knowledge, and the collection of data on downloads and utilization. The MAPPP app design team was convened, which included clinical content experts and IPRO app developers, with the goal of creating an app that would provide a high-value proposition for the clinical user (evidence-based, accurate, and quick results) with simplicity of design (few pages). Construction of a wireframe and subsequent user interface sample screenshots using MAPPP app logic were designed to iteratively develop the app through use cases, preliminary usability testing, and clinical content expert feedback. The resulting prototype was refined into the final product via iterative cycles of clinical user testing.

The MAPPP app was developed using the open source framework Cordova (Apache Software Foundation, Wakefield, MA), which utilizes HTML, CSS, and JS to develop apps across multiple platforms from a single code base. The MAPPP app was made available through both the iOS App Store and Google Play; it functions with Web browsers that support modern Web technologies. The app was released in April 2016 and, based on user feedback, underwent minor modifications in May 2016 to enhance the disclaimers, improve the visibility of the button linking to more information on selection options, and expand flexibility of use through the addition of a “Back” button. While no major changes have been made to the clinical content of the app to date, a process is in place to support app updates as necessary based on the most current evidence-based recommendations in the field of perioperative antithrombotic therapy.

Data were extracted from Google Analytics (including iOS, Android, and Web browser data) to characterize new users and completed episodes temporally (by calendar quarter) and geographically (globally, United States, and the targeted US state of New York) for the period between April 1, 2016 and September 30, 2017. A completed episode was defined by the project team as a user interaction that resulted in the app presenting a recommendation page based on sequential selection of an anticoagulant drug, assignment of bleeding risk category for a procedure, and indication of underlying thromboembolic risk of the patient [21]. Additional analysis of completed episodes was performed to characterize utilization by individual drug, procedural bleeding risk category, and underlying thromboembolic category.

The MAPPP app was developed and disseminated to address a pressing public health need (ie, the rate of ADEs associated with anticoagulant use). The technology was perceived to have specific advantages over traditional distribution of clinical tools as hard copies or pdf files, including ease of access when providing direct patient care and the ability to reach an expanded recipient audience, update content, and passively collect data. This analysis of the initial uptake and utilization of the MAPPP app supports the premise that such apps can not only be a useful means of disseminating clinical tools but also help identify relevant weaknesses and opportunities for improvement.

The app demonstrated the ability to reach a broad audience of recipients within not only the state of New York (the target geography of the CMS-funded quality improvement project) but also across the United States and in other countries. Despite the absence of any formal dissemination plan beyond an initial launch press release on April 29, 2016, direct provider interactions in New York, and sharing by members of the NYACC, the app has continued to generate new users and a growing number of completed user episodes. Evaluation of utilization data also suggests that the app is frequently used to access guidance for patients undergoing procedures with the highest bleeding and thromboembolic risk. While no changes were made to the clinical content of the app during the period of this evaluation, several formatting changes were made successfully, demonstrating the ability to update information to providers without having to remove outdated hard copies or pdf files. Finally, the analysis was performed using data passively collected by Google Analytics, which is impossible with hard copy tools or pdf files without advanced formatting functionality.

However, the app and the current analysis are not without limitations. Because the app does not identify the end user or establish a connection with actual patient medical records, there is no way of knowing whether the observed episodes were utilized for actual patient cases. However, data from New York’s comprehensive ADE reduction quality improvement initiative do suggest that the app is a promising component of a successful intervention campaign to improve anticoagulation safety. In its routine quality improvement role for CMS, IPRO assessed the quarterly rates of bleeding and thromboembolic events resulting in hospitalization within 30 days of an elective surgical procedure among Medicare beneficiaries who had Part D claims for anticoagulants and who were residing within the metro areas of New York State where MAPPP app sessions were identified. In the first calendar quarter of 2016 (ie, prior to app release), IPRO identified 154 ADEs among all eligible cases (154/10,855, 1.42%). In the last quarter of that year (ie, post-MAPPP app launch), among 13,948 cases, 1.13% (157/13,948) ADEs were identified, indicating an approximately 20% relative reduction in the rate of ADEs.

While such improvements cannot be attributed directly to use of the app, efforts are currently underway under a CMS Special Innovation Project awarded to IPRO to facilitate the integration of the app’s logic into the electronic health record systems of 3 health systems as active clinical decision support using Substitutable Medical Applications reusable technologies on Fast Healthcare Interoperability Resources when possible, which will allow a direct assessment of the patient-level impact. Incorporation of the app into existing clinical workflows with executive-level adoption buy-in as facility policy has the potential to affect all patients undergoing relevant procedures and will provide access to the data needed to objectively evaluate the app’s clinical and financial impact. Training on MAPPP app use via recorded webinar and associated patient education materials can be found on the MAPPP app website [25]. Electronic health record integration is anticipated to scale MAPPP use globally. Results of the project are anticipated in 2019.

Similar apps have been developed by reputable and authoritative organizations such as the University of Michigan’s MAQI2 Anticoagulation Toolkit and the ManageAnticoag app developed by the American College of Cardiology [27,28].

The MAPPP app is a successful means of disseminating current guidance on periprocedural anticoagulant use, as indicated by broad global uptake and upward trends in utilization. Lack of access to provider- and patient-specific data precludes objective evaluation of the clinical impact of the app. An ongoing study incorporating app logic into the electronic health records of 3 health systems will provide a more definitive evaluation of the clinical impact of the app logic.