A systematic search of the New Zealand iTunes and Google Play stores was conducted on April 1, 2016, to identify all potentially relevant apps. The search was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for systematic reviews [20]. Search terms included “arthritis” OR “rheumatoid” OR “RA” OR “rheumatoid arthritis” OR “rheumatic.” The app store description of each identified app was read and compared with the inclusion and exclusion criteria. Apps were included if they were: (1) a smartphone-based app; (2) capable of running on Android or iOS operating systems; (3) in English language; (4) useful for people with RA or to assist clinical care of people with RA; and (5) available for download in the New Zealand app store (iTunes or Google Play). Apps were excluded if: (1) a condition other than RA was targeted; (2) app content was for information, education, or reference only (ie, no data entry); (3) the app included only treatment algorithms; or (4) it was explicitly only for clinician use. When an app was found in both the Google Play and iTunes store, both versions were included so any differences between operating systems could be identified. Android apps (New Zealand Google Play store) were downloaded and tested using 2 Samsung Galaxy J1 Ace phones equipped with Android version 5.1.1. iOS apps (New Zealand iTunes store) were downloaded and tested using iPhones (4s and 6) with iOS 9.1 installed.

Since the New Zealand app stores may not include all potentially relevant apps, the United States, the United Kingdom, Australia, and Canada iTunes stores were also searched for eligible apps by conducting the search using the terms “rheumatoid arthritis” on the website fnd.io [21]. “Rheumatoid arthritis” was used as the sole search term, as this returned almost all apps found in the main search and did not identify any additional apps.

The following data about all apps were recorded: app name, platform (Android, iOS), developer, current version, size, cost, number of installs, and user star ratings. Functional features were noted descriptively.

App adherence with relevant recommendations for monitoring of RA disease activity in clinical practice from the American College of Rheumatology (ACR) and the European League Against Rheumatism (EULAR) was evaluated [3,5]. This was determined by operationalizing the recommendations and determining whether present or not present in each app (Table 1).

All apps were rated by two independent reviewers (HT and BW) using the MARS [16]. Before app assessment, the two reviewers discussed the use of the MARS in the context of apps for people with RA. The target group was determined to be “all people with RA aged 18 years or older; some familiarity with smartphone technology.” As recommended by the developers of the MARS, the reviewers considered all items of the MARS and confirmed that all were applicable to apps for RA, and that no additional app-specific items were required [16]. The reviewers also viewed the training video developed by Stoyanov et al.

Before assessing all the apps identified in the search, both reviewers assessed and discussed an excluded app to ensure shared understanding of the MARS items and process. The reviewers then independently rated all apps using the MARS. Before scoring each app, the reviewers used each app for at least ten minutes to gain an adequate understanding of the app functionality. Apps were tested on April 11, 2016, using the app version downloaded on April 1, 2016. Any issues or uncertainties about specific apps were discussed, and consensus was reached.

Scores were calculated for each MARS item, along with a total mean score. The mean score from two reviewers was calculated. No apps had been tested in clinical studies. Therefore, MARS item 19 “evidence base” was excluded from calculations. Inter-rater reliability of the MARS subscales and total quality score were calculated using the intraclass correlation coefficient (ICC) in SPSS Version 20.0 (IBM Corp; 2-way random-effects model of absolute agreement between single ratings).

The search retrieved 721 Android apps from the Google Play store. Of these, 710 were excluded, leaving 11 apps for analysis (Figure 1). A total of 216 iOS apps were retrieved from the iTunes app store. After exclusion of 200 apps, 16 apps remained for analysis. No further apps were found in the fnd.io search of the United States, the United Kingdom, Australia, and Canada iTunes stores. As 8 apps were available in both operating systems, a total of 19 different apps were included, of which 18 were free apps (Rheumatoid Arthritis Diary was available for NZD $6.39 for Android and NZD $6.49 for iOS).

The information on app platform, developer, version, and size are shown in Table 2. Since no apps had different functionality between operating systems, the apps are presented only once in Tables 2-6. The app description, target user (as derived from the app store description), Android installs, and Android star rating are shown in Table 3. None of the iOS apps included had the minimum of 5 reviews from users in New Zealand required on the New Zealand iTunes store before a star rating is provided. Table 4 shows joint count data entry and main functionality in the apps. Eleven apps allowed users to enter a joint count, either by selecting joints on a homunculus (n=4) or by entering the number of joints (n=7). Seven of these apps included the standard 28 swollen and tender joint count and primarily functioned as disease activity measure calculators, with no capacity to store or track data. The remaining 4 apps with other joint counts (Cliexa-RA, myRA, RheumaTrack RA, and RAPA) all had additional patient-focused functions, such as recording fatigue, and storage and tracking of imputed data. Fourteen apps included calculation of a RA disease activity measure. Six apps allowed export of patient data, including via email (n=5), spreadsheet (n=2), or to a website (n=2).

App inclusion of the component measurement instruments, composite disease activity measures calculated, and app functionality to record and retrieve data over time (as recommended by ACR and EULAR [3,5]) are shown in Table 5. Eight apps included at least one recommended composite measure of RA disease activity. Only 1 of these 8 apps provided the formulae for calculation of the composite disease activity measures (RheumaHelper), which were confirmed to be the correct formulae. Ten apps included a function allowing data to be recorded and retrieved. One app, Arthritis Power, included both 1 composite disease activity measure and allowed data recording and retrieval, but this app did not have functionality to record a 28 tender or swollen joint count.

MARS ratings for included apps are shown in Table 6. The ICC for MARS ratings was greater than or equal to 0.69 for all MARS sections. For overall MARS ratings, the ICC was .93 (95% CI 0.76-0.98) for Android apps and .82 (95% CI 0.55-0.94) for iOS apps, confirming good inter-rater reliability. The overall MARS scores for the apps ranged from 1.98 to 4.62, indicating large variation in the quality of apps. Engagement (1.6-4.8) and aesthetics (1.17-4.67) showed greatest variability. Of the 6 apps that scored ≥4/5 on the overall MARS rating, only 1 (RheumaHelper) included a composite disease activity score endorsed by ACR and EULAR, but this app did not have a data tracking function. Of the other 5 apps scoring ≥4/5 on the overall MARS rating (myRA, RAISE, myRAteam, Rheumatoid Arthritis Patient Companion, and RheumaTrack RA), 2 allowed entry of CRP and ESR but no other validated RA disease activity instruments were included in these apps. Arthritis Power, the only app that included an ACR and EULAR–recommended composite disease activity score and tracked results had an overall MARS score of 3.41.

This review of apps for monitoring disease activity in people with RA showed that there are broadly two categories of apps available: apps for calculation of validated disease activity measures and those for people with arthritis to track symptoms. Many symptom-tracking apps did not use validated instruments. Apps that focused on calculations of a disease activity measure tended to only perform that function. One app, myRAteam, provided an environment in which people with RA could connect and share updates about their symptoms. Other less commonly encountered app functions included setting reminders and information sharing with a clinician, either via email or through a linked app. The latter is essential for an app to facilitate telehealth. Six apps allowed email or sharing of data, and only 1 of these apps provided a mechanism for sharing specifically with a clinician. This indicates a lack of apps suitable for large-scale telehealth management of RA.

Only one app, Arthritis Power, included both a symptom-tracking function and calculation of an ACR and EULAR-recommended composite measure of RA disease activity. However, Arthritis Power did not include a joint count function. Some apps appear to perform both functions, but include an incorrect version of a disease activity measure, for example, a 28 swollen joint count without a tender joint count (eg, RAPA). Overall, 14 apps provided a composite disease activity score, but only 8 apps used the correct component instruments to calculate the composite disease activity measure and therefore provided an ACR- and EULAR–recommended composite measure. A common reason for measures to not meet the latter criterion was the use of a joint count that did not specify tender and swollen as the joint abnormalities of interest or did not count the 28 joints required for a DAS-28. Some apps recording joint symptoms may be useful for people with RA to monitor their symptoms, but could not be used in a remote monitoring telehealth care service. People with RA wishing to monitor their own symptoms should be encouraged to choose apps, which use validated instruments and have a tracking function, such as Arthritis Power.

There were no apps that scored ≥4/5 on the overall MARS and included all ACR and EULAR endorsed disease activity instruments. This could be because apps are designed with either people with RA or rheumatologists as target users where patients do not usually perform joint counts and doctors would not usually need to store patient data in a mobile phone. The MARS scores had a wide range indicating highly variable quality of apps in terms of user experience. Future app development should occur with cooperation between software developers and key stakeholders. Software developers should optimize user experience in collaboration with people with RA, while doctors can ensure app adherence with best-practice evidence-based medicine. Item 19 of the MARS, “evidence base,” was excluded from all calculations because no apps had been studied in clinical trials, as specified by Stoyanov et al [16]. Therefore any future apps developed for RA disease activity monitoring should be assessed in clinical trials to determine the impact on clinical outcomes for people with RA and cost-effectiveness and undergo external quality review [22].

This study had a number of limitations. Only apps available in New Zealand app stores and in English language were included. An app for patient-led monitoring of RA disease activity has been developed in Japanese, which includes ACR and EULAR recommended instruments and diseases activity measures [12]. However, a preliminary search of the iTunes stores of 4 other English-speaking countries with the term “rheumatoid arthritis” suggested that the search of the New Zealand app stores has captured all relevant apps in the English language.

App quality was assessed using the MARS. The MARS is a recently developed tool and has not been extensively validated. However, it has now been used in several other app evaluations [17,23,24], and as in this study, the MARS has consistently proven good inter-rater reliability. App quality was also assessed by considering whether apps complied with ACR and EULAR RA management recommendations. There may have been other criteria that could have been used to assess app quality. Assessment of data security is not included in the MARS but is one commonly considered criterion of health software quality not included in this study [25]. Data security considerations are of utmost importance but will need to be considered within the regulatory requirements of the country in which the app is being used. The integration of health behavior theory concepts into app design and function, which has been used as a measure of quality, was also not considered in this study [26,27].

The recommended RA composite disease activity (CDA) scores include those with exclusively patient-reported outcomes (eg, RAPID 3 and PAS) and those that combine patient-reported outcomes and physician-performed tender and swollen joint counts. Remote telehealth monitoring of disease activity for people with RA assumes either that disease activity is derived from patient-reported outcomes or that patient self-performed joint counts will provide sufficiently accurate assessments of RA disease activity. Patient-performed joint counts do correlate moderately with physician-performed joint counts [28]. However, further validation of the assumption that patient-performed joint counts will be sufficient for longitudinal measurement of RA disease activity is required.

The findings of this study suggest that currently available RA apps for RA disease activity monitoring are of variable quality and generally do not comply with RA management guidelines. Many other studies of health apps have found that most apps do not comply with evidence-based guidelines [29-31]. Like RA, inflammatory bowel disease requires ongoing management by a specialist physician and has a variable, unpredictable clinical course. A comprehensive analysis of apps for inflammatory bowel disease (IBD) identified that only 54% (14/26) provided a symptom-tracking function and only 19% (5/26) had medical input during app development [32]. Eight apps were specifically for providing information about IBD. When information about IBD included in these apps was compared with the minimum information set of 14 statements recommended to be shared with people with IBD, only 38% of these statements had complete coverage in the apps [32]. Similarly, a review of apps for people with gout, another common form of chronic arthritis, found only 1 of 6 relevant apps included all recommendations for patient-focused quality care of gout [33]. These studies suggest that the lack of a comprehensive, guideline-compliant app for RA is part of a wider paucity of high-quality health apps available.

The inclusion of persuasive principles, aimed to support positive behavioral change, has been considered as a measure of app quality in a recent systematic analysis of apps for chronic arthritis [34]. In the 28 assessed apps, a mean of only 5.8 of 37 persuasive principles per app was found with social support techniques (eg, social media, user forums) and sophisticated dialogue support techniques (eg, praise, rewards) largely absent. This suggests that the design process for future for RA should consider evidence-based persuasive techniques.

This review indicated the lack of high-quality apps available to assist in the management of RA, particularly the longitudinal assessment of RA disease activity. Only 1 app of the 19 identified in this study had functionality to allow both calculation of a validated composite disease activity measure and tracking of the calculated patient data. No available apps meet the aforementioned criteria along with inclusion of 28 tender and swollen joint counts. Thus, current apps fall into two categories: simple calculators for rheumatologists and data tracking tools for people with RA. The latter do not uniformly collect data using validated composite disease activity measures. Apps that were rated highly according to the MARS tended to collect only patient-reported outcomes.

The rheumatology professional workforce is inadequate to meet current population rheumatology health care needs. Since demand for care is predicted to increase, adoption of different models of care provision will be necessary [18,35,36]. These are likely to include telehealth and an increased emphasis on participatory health care where people with RA are active agents in the management of RA. Developing apps that are attractive, engaging, simple to use, and having functionalities relevant to the clinical management of the health condition will require collaboration between rheumatologists, people with RA, app developers, and health systems, and due consideration of local regulatory environment, health service delivery, and user experience [22]. Once apps are developed, assessment of the validity and accuracy of self-performed joint counts will be required along with demonstration of equivalent health outcomes for people with RA whose care is provided with a mixed face-to-face and telehealth approach.