Published on in Vol 8, No 2 (2020): February

Preprints (earlier versions) of this paper are available at, first published .
A Possible Mobile Health Solution in Orthopedics and Trauma Surgery: Development Protocol and User Evaluation of the Ankle Joint App

A Possible Mobile Health Solution in Orthopedics and Trauma Surgery: Development Protocol and User Evaluation of the Ankle Joint App

A Possible Mobile Health Solution in Orthopedics and Trauma Surgery: Development Protocol and User Evaluation of the Ankle Joint App

Original Paper

1Department for Orthopaedics and Orthopaedic Surgery, Saarland University Medical Center and Saarland University Faculty of Medicine, Homburg, Germany

2Clinic of Traumatology and Orthopedics, Bundeswehr Hospital Berlin, Berlin, Germany

3Department of Plastic, Reconstructive & Aesthetic Surgery, Specialized Clinic Hornheide, Münster, Germany

4Department of Orthopaedics, Trauma and Recontructive Surgery, St. Marien Hospital Mülheim and Chair of Orthopaedics and Trauma Surgery, University of Duisburg-Essen, Essen, Germany

5Clinic of Trauma, Hand and Reconstructive Surgery, University Hospital Essen, Essen, Germany

6Sportsclinic Hellersen, Lüdenscheid, Germany

Corresponding Author:

Florian Dittrich, Dr med

Department for Orthopaedics and Orthopaedic Surgery

Saarland University Medical Center and Saarland University Faculty of Medicine

Kirrbergerstr. 100

Building 37-38



Phone: 49 68411624520


Background: Ankle sprains are one of the most frequent sports injuries. With respect to the high prevalence of ankle ligament injuries and patients’ young age, optimizing treatment and rehabilitation is mandatory to prevent future complications such as chronic ankle instability or osteoarthritis.

Objective: In modern times, an increasing amount of smartphone usage in patient care is evident. Studies investigating mobile health (mHealth)–based rehabilitation programs after ankle sprains are rare. The aim of this study was to expose any issues present in the development process of a medical app as well as associated risks and chances.

Methods: The development process of the Ankle Joint App was defined in chronological order using a protocol. The app’s quality was evaluated using the (user) German Mobile App Rating Scale (MARS-G) by voluntary foot and ankle surgeons (n=20) and voluntary athletes (n=20).

Results: A multidisciplinary development team built a hybrid app with a corresponding backend structure. The app’s content provides actual medical literature, training videos, and a log function. Excellent interrater reliability (interrater reliability=0.92; 95% CI 0.86-0.96) was obtained. The mean overall score for the Ankle Joint App was 4.4 (SD 0.5). The mean subjective quality scores were 3.6 (surgeons: SD 0.7) and 3.8 (athletes: SD 0.5). Behavioral change had mean scores of 4.1 (surgeons: SD 0.7) and 4.3 (athletes: SD 0.7). The medical gain value, rated by the surgeons only, was 3.9 (SD 0.6).

Conclusions: The data obtained demonstrate that mHealth-based rehabilitation programs might be a useful tool for patient education and collection of personal data. The achieved (user) MARS-G scores support a high quality of the tested app. Medical app development with an a priori defined target group and a precisely intended purpose, in a multidisciplinary team, is highly promising. Follow-up studies are required to obtain funded evidence for the ankle joints app’s effects on economical and medical aspects in comparison with established nondigital therapy paths.

JMIR Mhealth Uhealth 2020;8(2):e16403




An ankle sprain is one of the most frequent injuries, with an incidence of 1:10,000 individuals per day in amateur and high-performance sports in the United States [1]. With respect to the high prevalence of ankle ligament injuries and patients’ young age, optimizing aftercare and rehabilitation is mandatory [2]. Moreover, the economic burden of ankle sprains is enormous [3]. High medical, physiotherapeutic, and lost productivity costs burden health care systems and create the need for new, efficient diagnostic and therapeutic solutions [4].

To prevent long-term complications, complex ligament injuries and recurrent ankle sprains with progression to chronic ankle instability (CAI) have to be recognized. The development of ankle osteoarthritis (OA) as a long-term consequence of CAI was first shown by Harrington et al [5] in 1979. Following an ankle ligament injury, posttraumatic muscular insufficiency [6,7] and ankle OA were observed in 13% of the cases [8]. Therefore, the adequate and consequent treatment of an ankle sprain might prevent CAI and OA.

Nowadays, early functional treatment is considered the gold standard for the lateral ligament lesion of the ankle [2,9]. The latest national guideline published by the German Orthopedic Foot and Ankle Society (Deutsche Assoziation für Fuß und Sprunggelenk eV, DAF) also recommends a conservative approach to acute ligament tears of the lateral ankle joint [10].

In times of digitalization and emerging technologies, smartphones are regularly used to accomplish everyday tasks, such as Web-based banking and communication via messenger or email, and penetrate rapidly into more and more areas of life [11]. The portability and omnipresent accessibility of smartphones enable usage anywhere and anytime [12]. In general, the growing implementation of smartphones as a transfer media in medical context is evident [13].

It has already been shown that the patients’ acceptance is given for collecting personalized health-relevant data via software apps, to share these with their peers or the medical staff [14]. Moreover, mobile short message service text messages and apps can have a positive impact on the posttraumatic outcome by showing increased adherence to medications and protocols, improved clinic attendance, and decreased readmission rates and emergency room visits [15].

However, the implementation and use of mobile health (mHealth) in medical care, especially in the fields of orthopedics and trauma surgery, can still be regarded to be in an early stage. So far, only 13 serious medical apps in orthopedics and trauma surgery have been identified for regular use in outpatient and inpatient medical care in German-speaking countries [16]. In a survey among German orthopedic and trauma surgeons, the Ankle Joint App (Sprunggelenks-App, Mediploy GmbH, Langenfeld, Germany) was shown to be frequently chosen, although the medical usage rate was still very low at 2.3% [17].

Studies investigating mHealth-based diagnostics [18] or rehabilitation programs after lateral ankle sprains already exist, for example, in the Netherlands (app: Strengthen your Ankle) [19], where a positive influence on medical and economic aspects could be demonstrated [20-22]. To date, the application of posttraumatic mHealth solutions after ankle sprains has not been investigated in Germany.


To address this gap, this work outlines the methodology to develop and design an app for patient education as well as prevention and identification of CAI after ankle sprains (Ankle Joint App). The publication of an app development process might be the basis for future mHealth solutions to improve patient care.

The app’s content, usability, and styling were evaluated by German orthopedic or trauma surgeons and athletes who suffered from ankle sprain.

Development Protocol

Basic App Conception

A multidisciplinary team was involved in the development of the ankle joint app. The team members comprised 2 orthopedic and trauma surgeons (FD and SB), a physiotherapist, a lawyer, and a software and Web developer. Before programming the app, some general aspects regarding the software structure, design, and content had to be considered. At an early stage, the target group and the intended purpose needed to be defined precisely to clarify whether the app had to be defined as a medical device and therefore had to be regulated by medical products law [23].

Technical Specifications

The ankle joint app was developed using React Native (Facebook Inc) technology. React Native is a Javascript-based framework for software developers, building cross-platform mobile apps for Android or iOS devices. The framework features built-in components and application programming interfaces, which are essential for developing innovative and user-friendly mobile apps [24].

The backend server runs on a Web app based on the Hypertext Preprocessor framework Symfony and meets actual software security guidelines. Any data exchange between the backend server and the app runs via Secure Sockets Layer secured connection. All server structures are located in Germany. Patient-related data remain strictly on the mobile device.

Texts and Videos

The ankle joint app is based on the latest national guidelines published by the German Orthopedic Foot and Ankle Society (DAF) and related medical literature. The content is written in German. All relevant references are stored in the app and are hyperlinked to the primary source to facilitate search for the user. Special efforts have been made to ensure that the information communicated is short, clear, and easy to understand. To explain medical terms comprehensibly, a glossary function has been integrated to avoid overloaded main text pages. By answering frequently asked questions, personal data are collected and made available to the user at some key points. Thus, the content adapts to the individual healing process constantly.

With the cooperation of a physiotherapist and considering the current research data, a training program was created, which can be carried out without special equipment. In addition to giving some general information, for example, the PRICE-rule (P=protection, R=rest, I=ice, C=compression, and E=elevation) or the activation of the muscle-vein pump in the acute stage, patients are also provided with short video clips in the later stages (Figure 1). A total of 15 successive built-up exercises were made available to patients via the app. A special focus in this training circle was placed on early functional mobilization and proprioceptive training to prevent CAI.

Figure 1. App screen view: (a) timeline-based aftercare plan, (b) information, and (c) training videos.
View this figure
Patient-Generated Data and Log Function

Initially, to start the timeline, the timepoint of the injury has to be defined. In addition, the user is asked to evaluate whether it was the first or the second event of an ankle sprain. Moreover, the kind of trauma and prescribed aids are requested. At regular intervals, patients are asked questions via push messages about their current level of pain, using the visual analog scale; feeling of instability; and load-bearing capacity. The collected patient-related data are presented to the patient in an understandable graphical form in the diary function (Figure 2).

German Cumberland Ankle

The Cumberland Ankle Instability Tool (CAIT) was developed for measuring the severity of functional ankle instability [25]. Using a well-established 9-item 30-point scale, the CAIT shows an adequate correlation to performance tests. It is a valid and reliable instrument for assessing CAI [26-28]. The minimal detectable change, as well as the minimal clinical important difference, lies at ≥3 points [29]. We assessed the status of CAI using the validated German CAIT in a digital form for the first time [30,31]. The CAIT was surveyed on day 56 after trauma. A score of <25 indicates CAI, and the app user is informed that additional diagnostics are recommended [32].

Figure 2. App screen view: (a) collection of patient-related data and (b) log function of patient-related data.
View this figure
Styling, Design, and Testing

Special attention was paid to the development of an intuitive and user-friendly interface. To allow elderly patients to use the app, an onboarding feature was established to explain the main functions and interactions. Milestones in rehabilitation were presented graphically in a timeline to ensure clarity for patients about the progress of their rehabilitation. Information regarding the rehabilitation process was structured logically and linked to icons with a recognition factor.

After the app had been developed, an alpha test was carried out by the development team using the software TestFlight (Apple Inc). The following beta test was performed on persons who had already undergone an ankle distortion trauma. Some technical, content-related, and interactional improvements were made as a consequence to the test feedback under controlled conditions.

App Quality Testing

Study Design and Sample

The app was evaluated by 20 German orthopedic and trauma surgeons with a special interest in foot and ankle surgery as well as 20 athletes. All the physicians and athletes included were familiar with smartphone devices and used apps on a daily basis. The involved athletes already sustained an acute ankle sprain in their past sporting career. The study was conducted between June 2019 and August 2019. The link for the digital questionnaire on a Google Docs (Google LLC) platform was sent to the participants by email or Quick Response code scanning. The email addresses of the physicians were generated manually via the home pages of clinics or via established email distribution lists. The athletes were screened in local badminton or boxing clubs (FC Langenfeld, VFL Bochum, and Lanna martial arts Bochum).

German Mobile App Rating Scale

The MARS rating is a well-established assessment scale for medical app quality. It was developed for professionals, and it includes the sections classification, objective app quality, subjective app quality, and a modifiable app-specific section. MARS items are scored using a 5-point Likert scale (1=inadequate, 2=poor, 3=acceptable, 4=good, and 5=excellent). The objective app quality section includes 19 items divided into 4 subscales—engagement, functionality, esthetics, and information quality—and a separate subjective app quality section. The subjective app quality section contains four items evaluating the user’s overall satisfaction.

Calculating the mean scores of the engagement, functionality, esthetics, and information quality objective subscales, as well as an overall mean app quality, total score is how the MARS is scored. Mean scores instead of total scores are used because items can be rated as not applicable. The subjective quality items can be scored separately as a mean subjective quality score [33].

The English MARS version’s sections were extended in the MARS-G by an additional section focusing on the medical gain of an app. The 5 subscales and the overall score determine the app’s quality [34]. All surgeons watched the associated MARS-G instructional video on how to use the MARS-G scale before rating in case of doubt [35].

Data Analysis

The analog (user) MARS-G was converted into a digital questionnaire on a Google Docs platform (Google LLC). Data were saved and then transferred into an Excel table (Microsoft Corp). Descriptive statistics were calculated for all items. The intraclass correlation coefficient (ICC) was calculated among the reviewers. We selected an individual absolute agreement ICC (AA-ICC) for a two-way mixed model on the basis of ICC guidelines by Shrout and Fleiss [36]. All statistical analyses were conducted using SPSS (version 25, IBM Corp).


A total of 20 foot and ankle surgeons as well as 20 athletes who suffered an ankle sprain took part in the app rating, which is equivalent to a response rate of 65% (20/31) for the surgeons and 44% (20/46) for the athletes. Excellent interrater reliabilities (two-way mixed model AA-ICC=0.92; 95% CI 0.86-0.96 for surgeons and athletes) were shown following the guidelines for ICC interpretation established by Koo et al [37]. The surgeons’ group comprised 20% (4/20) Android and 80% (16/20) iOS users, and the athletes’ group comprised 59% (10/17) Android and 41% (7/17) iOS users.

German Mobile App Rating Scale

The mean overall score for the Ankle Joint App was 4.4 (SD 0.5), rated by both surgeons and athletes. It was derived from the mean scores on app functionality, engagement, esthetics, and information quality (Figure 3). The mean subjective quality scores were 3.6 (surgeons: SD 0.7) and 3.8 (athletes: SD 0.5). The section behavioral change, which included an assessment of the perceived impacts on disease-related knowledge, attitude, awareness, and behavior, had mean scores of 4.1 (surgeons: SD 0.7) and 4.3 (athletes: SD 0.7). The medical gain, rated by the surgeons only, was 3.9 (SD 0.6; Table 1, Figure 3).

Figure 3. Mean scores of the (user) German Mobile App Rating Scale for the Ankle Joint App (surgeons: n=20 and athletes: n=20).
View this figure
Table 1. Detailed results of the (user) German Mobile App Rating Scale.
SubscaleMARS-Ga surgeonsuMARS-Gb athletes

MinimumMaximumMean (SD)MinimumMaximumMean (SD)

Entertainment354.4 (0.9)354.1 (0.7)

Interest354.3 (0.9)254.4 (0.9)

Customization253.8 (0.8)253.6 (0.9)

Interactivity354.2 (0.7)254.3 (0.9)

Target group454.6 (0.5)454.6 (0.5)

Performance455.0 (0.2)354.8 (0.6)

Usability454.6 (0.5)354.6 (0.6)

Navigation454.6 (0.5)354.5 (0.7)

Gestural design454.5 (0.5)454.5 (0.5)

Layout354.4 (0.8)354.5 (0.6)

Graphics454.6 (0.5)354.5 (0.6)

Visual appeal354.1 (0.7)354.3 (0.7)

Accuracy of app description (in app store)354.8 (0.5)c

Goals354.3 (0.6)

Quality of information354.5 (0.8)454.7 (0.5)

Quantity of information454.8 (0.4)354.9 (0.5)

Visual information454.3 (0.5)354.5 (0.6)

Credibility354.0 (0.9)253.8 (1.0)

Evidence base254.0 (1.1)
Medical gain

Gain for patients354.3 (0.8)

Gain for physicians253.8 (0.9)

Risks, side and adverse effects354.5 (0.6)

Transferability into routine care253.3 (1.0)
Subjective quality

Would you recommend this app to people who might benefit from it?354.4 (0.9)354.3 (0.7)

How many times do you think you would use this app in the next 12 months if it was relevant to you?153.2 (1.2)253.9 (0.7)

Would you pay for this app?132.0 (0.7)132.4 (0.8)

What is your overall star rating of the app?454.6 (0.5)354.6 (0.6)

Awareness254.5 (0.8)354.6 (0.6)

Knowledge354.6 (0.6)354.8 (0.6)

Attitudes253.7 (0.9)354.3 (0.7)

Intention to change353.9 (0.8)454.4 (0.5)

Help seeking254.1 (1.0)253.8 (1.0)

Behavior change254.0 (0.9)354.3 (0.8)

aMARS-G: German Mobile App Rating Scale.

buMARS-G: (user) German Mobile App Rating Scale.

cNot applicable.

Principal Findings

Our research using the Ankle Joint App demonstrates that mHealth-based rehabilitation programs might be an adequate and innovative tool for patient education, prevention, and collection of personal data.

The achieved (user) MARS-G scores prove the app’s quality from a professional and user point of view, demonstrating a comparatively high overall mean (user) MARS-G value [33]. The highest scores were reached in the functionality section for both surgeons and athletes. In accordance with a recent survey among orthopedic and trauma surgeons, intuitive usability was considered the most important factor for the regular use and quality of an app. The integration of complex functions in an intuitive lean and secure user interface poses a great challenge to the development team. Moreover, the development of an intuitive frontend is complex and involves high development costs and test phases [38]. Multifunctional apps, for example, in the field of diabetes mellitus type II therapy for patients over 50 years showed limited usability with negative effects on compliance and therapy outcomes. Apps with basic functions provide enhanced usability [39], but the limitation of software features affects the app’s functionality. To address this divergence, trial runs with specific target groups and permanent reevaluation of the initial concept are mandatory during the app’s development process.

As a first step, when developing an app, the target group and the intended purpose have to be defined precisely. The Ankle Joint App was especially designed for young and active patients to optimize conservative rehabilitation following an acute ankle sprain without osseous lesions. With respect to this, the differences in evaluating the medical gain for physicians and patients, with individual requirements in their rehabilitation episode [40], can be explained.

Customization seems to be important to the target group and might be improved in our app. We believe that medical apps have to be adaptable not only to the specific users’ requirements but also to the varying hospital standards. The aspects to be considered in the development of medical apps are the limited areas of application in combination with varying standards of treatment, both national and international, and the legal and medical aspects of an app with regard to liability and data protection [21]. These aspects represent a challenge for financing the complex development and maintenance of an app, as the 10 most popular apps ranked by the number of users in Germany in 2018 were all available for free download [41].

Considering recent data scandals, which led to a fundamental distrust of apps that might be implemented in the context of Big Data, the secure and transparent collection of medical personal data can be challenging [42,43]. For this reason, we decided to store personal data exclusively on the mobile device to avoid cloud upload. In general, dichotomous scenarios about data exchange between patients and medical staff are possible. The recovery progress could be displayed analogously on the patient’s smartphone in the event of a doctor’s appointment (eg, CAIT). Alternatively, an upload of data into a secure cloud system can be taken into consideration [44]. In the event of deviations from the expected progression of the disease, patients could be informed about and provided with medical expertise more rapidly. Moreover, the collection of validated scores and surveys might be relevant for academic research.

In contrast to these positive effects, app users and providers (physicians and medical staff) should keep in mind that the collection and processing of personal data also represent cornerstones of app financing. This could lead to potential conflicts of interest as the collected data represent an immense value, for example, for the provision of personalized advertising [45]. Before downloading an app, the financing, development process, and data flow have to be completely and plausibly depicted by the publisher. Therefore, transparent and appropriate app store descriptions, data protection regulations, and terms and conditions of use are of utmost importance [46].

This study has some limitations. The evaluation of the app’s quality was carried out within a theoretical framework; thus, it only reflects its use to a limited extent in daily clinical practice. It has to be mentioned that the app was only evaluated by a relatively small number of users (patients) for a short period of 3 months. Thus, data on compliance, demographics, and usage behavior are hardly representative. Moreover, the economic aspects of the development process and app costs per user were not taken into account. In addition, the response rate was moderate, which might lead to a bias toward users with high digital affinity. Comparative randomized control trial studies are required to gain funded evidence on the app’s positive effects on patient education and treatment progress in comparison with established nondigital therapy paths to prevent CAI and reach a final scientific conclusion; this has to be addressed by future studies.

Nowadays, in many countries, an increasing number of patients visiting emergency units with minor complaints can be registered. Often, the treatment of these patients is time and staff consuming, compromising the medical attention of more severely injured individuals. This overcrowding may lead to negative consequences to the patients’ safety and their outcome [47-49]. Given the increasing workload, physicians are dissatisfied with highly time-consuming procedures, for example, electronic patient recording in the emergency department [50,51]. Apps might be used by the emergency staff to easily create and recommend digitally customized aftercare plans.

Moreover, the established discharge letter contains medical terminology, which offers very little benefit to a self-determined, competent patient. This practice does not meet the requirements of adequate patient involvement in the treatment process [52]. Improving patient education and optimizing the communication structures via apps on mobile devices have the potential to solve these issues. Individually designed and supervised aftercare treatments showed better outcomes [53]. The Ankle Joint App has a modular design and might be transferred to a wide range of aftercare treatments.

In contrast to the great potential of standardized medical app usage, there are also risks. However, medical resources and health care have to be distributed equally for everyone on the basis of moral and ethical obligations. This is why medical app usage also entails a particular risk of disadvantaging groups with low health competence and a high risk of disease [39]. In particular, the elderly patient might be disadvantaged by the use of medical apps, because in 2014, only about 17% of the individuals over 65 years regularly used a smartphone [54]. As degenerative diseases represent an important pillar of orthopedic and trauma surgery expertise, special attention has to be paid on the app development for these newcomers and their requirements in the future. Particularly in the area of app usability, the requirements of elder generations have to be addressed, for example, implementing an intuitive interface, a reading function, or a screen magnifier [55]. Self-endangerment because of incorrect app usage might occur, but the risk can be reduced by an onboarding function with an introduction of the app to new users and individual feedback mechanisms [56].


Working in a multidisciplinary team, using a backend structure to modify the app’s content and using React Native, proved to be efficient in the development process of medical apps. The success was proven by reaching high overall mean MARS-G scores for the Ankle Joint App in surgeons and athletes. Data obtained suggest that an mHealth-based rehabilitation program might be a useful tool for patient education and collection of personal data. The achieved (user) MARS-G scores prove the tested app’s high quality. Medical app development with an a priori defined target group and a precisely intended purpose, in a multidisciplinary team, is highly promising.


We acknowledge support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) and Saarland University within the funding programme Open Access Publishing.

Conflicts of Interest

FD and SB are active in Mediploy GmbH and in the development process of the Ankle Joint App.

  1. Waizy H, Harrasser N, Fehske K. [Lateral ligament injuries]. Unfallchirurg 2018 Sep;121(9):683-692. [CrossRef] [Medline]
  2. Vuurberg G, Hoorntje A, Wink LM, van der Doelen BF, van den Bekerom MP, Dekker R, et al. Diagnosis, treatment and prevention of ankle sprains: update of an evidence-based clinical guideline. Br J Sports Med 2018 Aug;52(15):956. [CrossRef] [Medline]
  3. Bielska IA, Wang X, Lee R, Johnson AP. The health economics of ankle and foot sprains and fractures: a systematic review of English-language published papers. Part 1: Overview and critical appraisal. Foot (Edinb) 2019 Jun;39:106-114. [CrossRef] [Medline]
  4. Hupperets MD, Verhagen EA, Heymans MW, Bosmans JE, van Tulder MW, van Mechelen W. Potential savings of a program to prevent ankle sprain recurrence: economic evaluation of a randomized controlled trial. Am J Sports Med 2010 Nov;38(11):2194-2200. [CrossRef] [Medline]
  5. Harrington KD. Degenerative arthritis of the ankle secondary to long-standing lateral ligament instability. J Bone Joint Surg Am 1979 Apr;61(3):354-361. [Medline]
  6. Kazemi K, Arab AM, Abdollahi I, López-López D, Calvo-Lobo C. Electromiography comparison of distal and proximal lower limb muscle activity patterns during external perturbation in subjects with and without functional ankle instability. Hum Mov Sci 2017 Oct;55:211-220. [CrossRef] [Medline]
  7. Lobo CC, Morales CR, Sanz DR, Corbalán IS, Marín AG, López DL. Ultrasonography comparison of peroneus muscle cross-sectional area in subjects with or without lateral ankle sprains. J Manipulative Physiol Ther 2016;39(9):635-644. [CrossRef] [Medline]
  8. Valderrabano V. Joint-preserving surgery of ankle osteoarthritis. Foot Ankle Clin 2013 Sep;18(3):xiii-xxiv. [CrossRef] [Medline]
  9. Kerkhoffs GM, Handoll HH, de Bie R, Rowe BH, Struijs PA. Surgical versus conservative treatment for acute injuries of the lateral ligament complex of the ankle in adults. Cochrane Database Syst Rev 2007 Apr 18(2):CD000380. [CrossRef] [Medline]
  10. Rammelt S, Stürmer KM, Richter M. AWMF: AWMF aktuell. 2017 Aug 8. Fresh Outer Ligament Rupture on the Upper Ankle   URL: https:/​/www.​​uploads/​tx_szleitlinien/​012-022l_S1_Aussenbandruptur_oberes_Sprunggelenk_2017-08.​pdf [accessed 2020-01-07]
  11. Krebs P, Duncan DT. Health app use among US mobile phone owners: a national survey. JMIR Mhealth Uhealth 2015 Nov 4;3(4):e101 [FREE Full text] [CrossRef] [Medline]
  12. Cha S, Seo B. Smartphone use and smartphone addiction in middle school students in Korea: prevalence, social networking service, and game use. Health Psychol Open 2018;5(1):2055102918755046 [FREE Full text] [CrossRef] [Medline]
  13. Meskó B, Drobni Z, Bényei É, Gergely B, Győrffy Z. Digital health is a cultural transformation of traditional healthcare. Mhealth 2017;3:38 [FREE Full text] [CrossRef] [Medline]
  14. Morris ME, Aguilera A. Mobile, social, and wearable computing and the evolution of psychological practice. Prof Psychol Res Pr 2012 Dec;43(6):622-626 [FREE Full text] [CrossRef] [Medline]
  15. Lu K, Marino NE, Russell D, Singareddy A, Zhang D, Hardi A, et al. Use of short message service and smartphone applications in the management of surgical patients: a systematic review. Telemed J E Health 2018 Jun;24(6):406-414. [CrossRef] [Medline]
  16. Dittrich F, Beck S, Busch A, Dudda M, Harren A, Sander A, et al. [Is the discharge letter still relevant? : Chances and risks of 'Medical apps' in orthopedics and traumatology]. Orthopade 2018 Oct;47(10):842-848. [CrossRef] [Medline]
  17. Dittrich F, Busch A, Harren K, Jäger M, Landgraeber S, Reinecke F, et al. [Apps in clinical use in orthopedics and trauma surgery : The status quo in Germany]. Unfallchirurg 2019 Sep;122(9):690-696. [CrossRef] [Medline]
  18. Morales CR, Lobo CC, Sanz DR, Corbalán IS, Ruiz BR, López DL. The concurrent validity and reliability of the Leg Motion system for measuring ankle dorsiflexion range of motion in older adults. PeerJ 2017;5:e2820 [FREE Full text] [CrossRef] [Medline]
  19. VeiligheidNL2. Google Play Store. Versterk je Enkel   URL: [accessed 2020-01-07]
  20. van Reijen M, Vriend II, Zuidema V, van Mechelen W, Verhagen EA. The implementation effectiveness of the 'Strengthen your ankle' smartphone application for the prevention of ankle sprains: design of a randomized controlled trial. BMC Musculoskelet Disord 2014 Jan 7;15:2 [FREE Full text] [CrossRef] [Medline]
  21. van Reijen M, Vriend I, van Mechelen W, Verhagen EA. Preventing recurrent ankle sprains: is the use of an app more cost-effective than a printed booklet? Results of a RCT. Scand J Med Sci Sports 2018 Feb;28(2):641-648. [CrossRef] [Medline]
  22. van Reijen M, Vriend I, Zuidema V, van Mechelen W, Verhagen EA. The 'Strengthen your ankle' program to prevent recurrent injuries: a randomized controlled trial aimed at long-term effectiveness. J Sci Med Sport 2017 Jun;20(6):549-554. [CrossRef] [Medline]
  23. Rämsch-Günther N, Stern S, Lauer W. [Qualification and classification of medical apps : what should be noted and what is BfArM's contribution?]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2018 Mar;61(3):304-313. [CrossRef] [Medline]
  24. Facebook. Facebook Open Source. 2019. React Native   URL: [accessed 2020-01-07]
  25. Hiller CE, Refshauge KM, Bundy AC, Herbert RD, Kilbreath SL. The Cumberland ankle instability tool: a report of validity and reliability testing. Arch Phys Med Rehabil 2006 Sep;87(9):1235-1241. [CrossRef] [Medline]
  26. Kunugi S, Masunari A, Yoshida N, Miyakawa S. Association between Cumberland Ankle Instability Tool score and postural stability in collegiate soccer players with and without functional ankle instability. Phys Ther Sport 2018 Jul;32:29-33. [CrossRef] [Medline]
  27. Sierra-Guzmán R, Jiménez F, Abián-Vicén J. Predictors of chronic ankle instability: analysis of peroneal reaction time, dynamic balance and isokinetic strength. Clin Biomech (Bristol, Avon) 2018 May;54:28-33. [CrossRef] [Medline]
  28. Dressler P, Gehring D, Zdzieblik D, Oesser S, Gollhofer A, König D. Improvement of functional ankle properties following supplementation with specific collagen peptides in athletes with chronic ankle instability. J Sports Sci Med 2018 Jun;17(2):298-304 [FREE Full text] [Medline]
  29. Wright CJ, Linens SW, Cain MS. Establishing the minimal clinical important difference and minimal detectable change for the Cumberland ankle instability tool. Arch Phys Med Rehabil 2017 Sep;98(9):1806-1811. [CrossRef] [Medline]
  30. Gehring D, Faschian K, Lauber B, Lohrer H, Nauck T, Gollhofer A. Mechanical instability destabilises the ankle joint directly in the ankle-sprain mechanism. Br J Sports Med 2014 Mar;48(5):377-382. [CrossRef] [Medline]
  31. FOMT - Fortbildungen für Orthopädische Medizin und Manuelle Therapie. Cumberland Ankle Instability Tool (validierte deutsche Version (Gehring et al 2016))   URL: [accessed 2020-01-07]
  32. Wright CJ, Arnold BL, Ross SE, Linens SW. Recalibration and validation of the Cumberland Ankle Instability Tool cutoff score for individuals with chronic ankle instability. Arch Phys Med Rehabil 2014 Oct;95(10):1853-1859. [CrossRef] [Medline]
  33. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015 Mar 11;3(1):e27 [FREE Full text] [CrossRef] [Medline]
  34. Messner ET, Barke A, Baumeister H, Stoyanov S, Hides L, Kavanagh D, et al. Development and Validation of the German Version of the Mobile Application Rating Scale (MARS-G). JMIR mHealth and uHealth;JMIR Preprints 2019. [CrossRef]
  35. MARS-Rating Universität Ulm. YouTube. 2017. MARS - Mobile Anwendungen Rating Skala   URL: [accessed 2020-01-07]
  36. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979 Mar;86(2):420-428. [CrossRef] [Medline]
  37. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016 Jun;15(2):155-163 [FREE Full text] [CrossRef] [Medline]
  38. Mortensen D. Interaction Design Foundation. 2019. How to Create an Intuitive Design   URL: [accessed 2019-05-10] [WebCite Cache]
  39. Arnhold M, Quade M, Kirch W. Mobile applications for diabetics: a systematic review and expert-based usability evaluation considering the special requirements of diabetes patients age 50 years or older. J Med Internet Res 2014 Apr 9;16(4):e104 [FREE Full text] [CrossRef] [Medline]
  40. Sanz DR, Lopez-Lopez D, Garcia DM, Medrano AS, Ponce AM, Lobo CC, et al. Effects of eccentric exercise in pressure pain threshold in subjects with functional ankle equinus condition. Rev Assoc Med Bras (1992) 2019 Mar;65(3):384-387 [FREE Full text] [CrossRef] [Medline]
  41. Statista. Most Popular Mobile Apps by Number of Users in Germany in 2018 (in Millions)   URL: https:/​/de.​​statistik/​daten/​studie/​872047/​umfrage/​beliebteste-mobile-apps-nach-der-anzahl-der-nutzer-in-deutschland/​ [accessed 2020-01-03]
  42. Stroud C. Forbes. 2018 Apr 30. Cambridge Analytica: The Turning Point In The Crisis About Big Data   URL: https:/​/www.​​sites/​courtstroud/​2018/​04/​30/​cambridge-analytica-the-turning-point-in-the-crisis-about-big-data/​#2ae8f43b48ec [accessed 2020-01-07] [WebCite Cache]
  43. Ancker JS, Mauer E, Kalish RB, Vest JR, Gossey JT. Early adopters of patient-generated health data upload in an electronic patient portal. Appl Clin Inform 2019 Mar;10(2):254-260. [CrossRef] [Medline]
  44. Car J, Sheikh A, Wicks P, Williams MS. Beyond the hype of big data and artificial intelligence: building foundations for knowledge and wisdom. BMC Med 2019 Jul 17;17(1):143 [FREE Full text] [CrossRef] [Medline]
  45. Househ M, Aldosari B, Alanazi A, Kushniruk A, Borycki E. Big data, big problems: a healthcare perspective. Stud Health Technol Inform 2017;238:36-39. [Medline]
  46. Albrecht U, Malinka C, Long S, Raupach T, Hasenfuß G, von Jan U. Quality principles of app description texts and their significance in deciding to use health apps as assessed by medical students: survey study. JMIR Mhealth Uhealth 2019 Feb 27;7(2):e13375 [FREE Full text] [CrossRef] [Medline]
  47. Scherer M, Lühmann D, Kazek A, Hansen H, Schäfer I. Patients attending emergency departments. Dtsch Arztebl Int 2017 Sep 29;114(39):645-652 [FREE Full text] [CrossRef] [Medline]
  48. Carter EJ, Pouch SM, Larson EL. The relationship between emergency department crowding and patient outcomes: a systematic review. J Nurs Scholarsh 2014 Mar;46(2):106-115 [FREE Full text] [CrossRef] [Medline]
  49. Gonçalves-Bradley D, Khangura JK, Flodgren G, Perera R, Rowe BH, Shepperd S. Primary care professionals providing non-urgent care in hospital emergency departments. Cochrane Database Syst Rev 2018 Feb 13;2:CD002097 [FREE Full text] [CrossRef] [Medline]
  50. Perry JJ, Sutherland J, Symington C, Dorland K, Mansour M, Stiell IG. Assessment of the impact on time to complete medical record using an electronic medical record versus a paper record on emergency department patients: a study. Emerg Med J 2014 Dec;31(12):980-985. [CrossRef] [Medline]
  51. Lorenzetti DL, Quan H, Lucyk K, Cunningham C, Hennessy D, Jiang J, et al. Strategies for improving physician documentation in the emergency department: a systematic review. BMC Emerg Med 2018 Oct 25;18(1):36 [FREE Full text] [CrossRef] [Medline]
  52. Lin R, Gallagher R, Spinaze M, Najoumian H, Dennis C, Clifton-Bligh R, et al. Effect of a patient-directed discharge letter on patient understanding of their hospitalisation. Intern Med J 2014 Sep;44(9):851-857. [CrossRef] [Medline]
  53. Hayden JA, van Tulder MW, Tomlinson G. Systematic review: strategies for using exercise therapy to improve outcomes in chronic low back pain. Ann Intern Med 2005 May 3;142(9):776-785. [CrossRef] [Medline]
  54. Berauk VL, Murugiah MK, Soh YC, Sheng YC, Wong TW, Ming LC. Mobile health applications for caring of older people: review and comparison. Ther Innov Regul Sci 2018 May;52(3):374-382. [CrossRef] [Medline]
  55. Wildenbos GA, Jaspers MW, Schijven MP, Dusseljee-Peute LW. Mobile health for older adult patients: Using an aging barriers framework to classify usability problems. Int J Med Inform 2019 Apr;124:68-77. [CrossRef] [Medline]
  56. Buus N, Juel A, Haskelberg H, Frandsen H, Larsen JL, River J, et al. User involvement in developing the MYPLAN mobile phone safety plan app for people in suicidal crisis: case study. JMIR Ment Health 2019 Apr 16;6(4):e11965 [FREE Full text] [CrossRef] [Medline]

AA-ICC: absolute agreement intraclass correlation coefficient
CAI: chronic ankle instability
CAIT: Cumberland Ankle Instability Tool
ICC: intraclass correlation coefficient
MARS-G: German Mobile App Rating Scale
mHealth: mobile health
OA: osteoarthritis

Edited by G Eysenbach; submitted 25.09.19; peer-reviewed by J Reponen, D López López, E Sadeghi-Demneh; comments to author 06.11.19; revised version received 29.11.19; accepted 16.12.19; published 26.02.20


©Florian Dittrich, David Alexander Back, Anna Katharina Harren, Marcus Jäger, Stefan Landgraeber, Felix Reinecke, Sascha Beck. Originally published in JMIR mHealth and uHealth (, 26.02.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.