Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Monday, March 11, 2019 at 4:00 PM to 4:30 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 28.03.19 in Vol 7, No 3 (2019): March

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/12207, first published Sep 14, 2018.

This paper is in the following e-collection/theme issue:

    Original Paper

    An Argument Against Cross-Platform Development: Lessons From an Augmented Reality App Prototype for Rural Emergency Responders

    1Marshfield Clinic Research Institute, National Farm Medicine Center, Marshfield, WI, United States

    2University of Wisconsin-Stevens Point, Stevens Point, WI, United States

    3linkedPIXEL LLC, Marshfield, WI, United States

    4Department of Environmental, Agricultural and Occupational Health, University of Nebraska Medical Center, University of Nebraska-Lincoln, Omaha, NE, United States

    Corresponding Author:

    Bryan Weichelt, PhD, MBA

    Marshfield Clinic Research Institute

    National Farm Medicine Center

    1000 N Oak Ave

    Marshfield, WI, 54449

    United States

    Phone: 1 7152217276

    Email: weichelt.bryan@marshfieldresearch.org


    ABSTRACT

    Background: Mobile augmented reality (MAR) apps offer potential support for emergency responders in rural areas.

    Objective: In this report, we described lessons learned from the development process of augmented reality (AR) Farm Mapping to Assist, Protect and Prepare Emergency Responders (MAPPER), a MAR app that provides emergency responders onsite information about the agricultural operation they enter.

    Methods: Cross-platform frameworks were used to create AR MAPPER to accommodate budget constraints and overcome issues with markerless MAR technologies. Although the single codebase and Web technologies streamlined development, cross-device hardware limitations impacted location accuracy, lengthened the development cycle, and required regular updates to third-party libraries.

    Results: A hybrid development approach of using Web-based technologies with native tie-ins for specialized components and enhanced performance cut time and costs. This also led to consistency across multiple platforms and ensured that there is only a single set of source files to modify for Android and iPhone operating systems. Meanwhile, active development was delayed by some major hurdles. Apple and Google both released new versions of their operating systems, and the Wikitude framework issued four major updates, each of which brought with it some important enhancements and also led to some new issues.

    Conclusions: Developers should consider single platform native development to benefit from platform-specific MAR implementations and to avoid development, testing, and maintenance costs associated with cross-platform implementation. Emergency response organizations may be more likely to utilize a single platform across the devices used by their command staff. This also reduces the benefits of cross-platform development. Furthermore, providing map-based, non-AR cross-platform apps for landowners, farmers, and ranchers would help improve and maintain data quality, which is crucial for the utility and user experience of MAR apps.

    JMIR Mhealth Uhealth 2019;7(3):e12207

    doi:10.2196/12207

    KEYWORDS



    Introduction

    Augmented reality (AR) [1] combines a view of the real world with digitally overlaid content. Recently, mobile augmented reality (MAR) apps have become increasingly popular, where virtual objects are combined with objects in the real environment in real time and aligned based on the user’s point of view through their mobile device display [2]. MAR games and improving support from mobile platform providers have made MAR accessible to end users and developers. However, users have several expectations toward MAR apps and that the underlying technology will influence the user experience (UX). MAR interfaces are expected to be able to provide valid, up-to-date, and relevant content to the user [3]. This may be particularly true in the context of apps created for medical education, health care service delivery, and other industries where unreliable and irrelevant content could have material and safety implications. Given these requirements, decisions made early in development have a significant effect on whether MAR apps provide reliable and relevant content. Developing a modern and cross-platform MAR app can easily accumulate significant expenses because of the sheer magnitude of resources required [4]. In practical terms, a key decision is whether to develop native apps for 1 or multiple mobile platforms or to use a cross-platform approach whereby hybrid app development frameworks are utilized to deliver the app to multiple platforms.

    Farm Mapping to Assist, Protect and Prepare Emergency Responders (MAPPER) provides emergency responders onsite information about the agricultural operation they are entering [5]. This case report describes development lessons learned in creating the AR Farm MAPPER, specifically addressing cross-platform development. The MAR prototype of Farm MAPPER improves on the earlier, static overhead version by incorporating a real-time depiction of icons, such as hazards, resources, and points of entry. This offers the possibility for on-scene commanders to have a heads-up display. This report, which builds upon a previously published manuscript [6], may be of more interest to researchers and practitioners working with MAR in the medical informatics field and, particularly, when considering issues related to the choice of approach—cross-platform development versus targeting single platforms. The specifics of the original Farm MAPPER app and its applicability to agricultural injury prevention interventions are the topic of another paper [6].


    Methods

    The project was led by a research team, which was administratively housed within a private rural health care system in the Upper Midwest. The skill set needed to develop and test MAR technology can be difficult to find. No internal resources were available. Thus, the lead developer (third author) recruited for this project was self-employed and subcontracted, although residing in the same community as the research team. The lead developer had significant experience in Web and mobile app development, with some familiarity with AR. The developer’s portfolio included first place awards for back-to-back public app development competitions run by Intel, the only developer to have done so. Few stakeholders were involved in this prototype’s development, which did not include formal usability testing.

    The benefit of a Web-based approach is the ability to leverage any of the thousands of freely available libraries and extensions, many of which are licensed under MIT, Apache, or GNU, which makes them fully available for distribution in any type of app, free or commercial. This can save tens of thousands of work hours to reinvent many already existing components. Moreover, the complexity of managing multiple software development kits with their differing application programming interfaces (APIs), build systems, and tools can be avoided [7]. This approach, as opposed to developing fully native apps for each platform, represents a more efficient way to deploy a product that will work on most modern mobile platforms. This can be particularly useful if the app is expected to be used by multiple organizations, some of which may have already committed to a specific platform.

    Although the window of time between our first written proposal and the final completion and delivery of the product extended more than a year, the fixed development budget only accommodated approximately 292 hours of actual labor (roughly equivalent to 7 full-time work weeks by a single person). Although remaining conscientious of the available budget and the relatively uncharted nature of markerless AR, we had to make some development decisions early on to effectively accommodate the various requirements. This led to the use of a third-party framework to simplify the AR mathematics and presentation as well as the use of Web-based technologies to retain a single universal codebase.


    Results

    What Went Right

    It appears that taking a hybrid app approach toward development by using Web-based technologies (with native tie-ins for specialized components and enhanced performance) was a decision that had some benefits. This strategy helped ensure consistency across multiple platforms and ensured that there is only a single set of source files to modify for Android and iPhone operating system (iOS, Apple Inc) combined. We were able to also borrow from various user interface (UI) libraries, notably OnsenUI [8] and the Wikitude [9] frameworks, to expedite the implementation of key features while ensuring a consistent UX across both Android and iOS devices. The key benefits of Wikitude are the ability to program AR content using basic Web technologies and easy porting of apps between platforms [10].

    Traditional development would have required use of native UI elements for all facets of the app; any changes made on one platform would then have to be manually worked into the source of the other platform. Requiring constant synchronization of 2 native apps would have effectively reduced the available work time by half. Conversely, the use of an existing and long-standing AR framework that supported hybrid apps within a universal API represented a substantial savings of time. Despite only an evaluation copy of the Wikitude framework being used throughout development and the considerable work that was still required to satisfactorily integrate it into the overall app, it offered a positive demonstration of the current capabilities and technical limitations of developing location-based AR.

    What Went Wrong

    Physical Hardware Limitations

    Going into this project, we knew that one of the largest setbacks we would encounter related to the technical limitations of modern-day hardware, particularly geolocation and global positioning system (GPS) accuracy. During testing across various devices, it was not uncommon for the variance in GPS accuracy to range anywhere from 30+ feet all the way up to 1400+ feet. This poses a problem when the objective of the app is to allow emergency responders the ability to quickly locate points of interest on a property. The nature of GPS also tends to require a delay between the initial GPS probe and an accurate fix. Although most modern cellular phones can typically lock onto GPS quite rapidly using assisted GPS technology, some devices we tested, including the LG G2, could take up to 10 min. Certain modern Android devices also lacked gyroscope functionality, which caused the AR framework to fail even though it was meant to be a universal solution. This then required additional correspondence with the framework developers who had to try and implement a workaround specifically for the affected devices.

    Lapse in Timeline of Development

    Despite a relatively short active development time (comparable with 1-2 months of dedicated work), the prolonged overall timeline of this project adversely impacted numerous aspects of it. There were multiple extended gaps between development from January to September, due in part to external occurrences, contractual matters, and other commitments. Although the generous timespan seemed appealing to better accommodate availability, it also led to struggles in having to make continual updates to the various frameworks and then retest all aspects to ensure no newfound issues emerged when subsequent operating system (OS) versions and libraries were released.

    Within the project’s timeframe, Apple and Google both released sweeping new versions of their OS (iOS 11.0 and Android 8.0, respectively). Meanwhile, the Wikitude framework issued 4 major updates (from 6.0.1 to 7.1), each of which brought with it some important enhancements and also led to some new issues.

    Given the basic prototype goals of this project, it would have been reasonable to develop and deploy the baseline product within a couple of months of dedicated work compared with stretching it over the course of a year and having to regularly update and recheck major components as the technologies evolved. A more focused, short-term deadline could have alleviated some of these obstacles.


    Discussion

    Arguments Against Cross-Platform Development

    Cross-platform development is a popular buzzword. In many instances, it is indeed beneficial to accommodate all major platforms—currently Android and iOS. However, that is not always the case, and in retrospect, we would propose that this prototype could have benefited more directly if it had been built only for a single target platform. Organizations that provide work-appointed devices almost always adhere to a single platform strategy, such as a health clinic’s laptop distribution. This way, they only need to be concerned that the apps being used function as intended on the single platform and OS that they support.

    The intent of this MAR project was to create a conceptual example of what emergency responders could use to locate areas of interest on a rural or agricultural property through visual cues such as hovering AR icons representing hazards, resources, and items of interest. Rather than relying on the emergency staff’s own personal phones for such activities, it is more probable that the facilities would supply identical, company-owned units to all personnel or at least individual shift leaders. In this respect, focusing on a single platform with a limited set of technology-related variables could help provide a more consistent UX.

    In related research, a survey of the Google Play Store revealed that hybrid development is more popular as an approach to creating data-intensive apps and correspondingly less popular in apps that require closer interaction with the underlying OS [11]. App reviews show that native apps are generally perceived as more performant and less buggy (ibid.), and hybrid apps on iOS and Android were more prone to complaints in user reviews [12]. Mobile app developers may also favor native development, particularly when better UX and access to device-specific features were needed [13]. Several case studies have shown that although a reasonable UX can be achieved by using a cross-platform framework, there are still limitations with performance, UX, and usability [14-16].

    Benefits of Developing for iPhone Operating System Only

    With the consideration to develop natively for a single platform, iOS would be an ideal choice if it is to be used as an internal company app. Although Android is equally capable and more versatile in many ways, from a development and industrial perspective, Android also has a vastly more diverse blend of devices, OS versions, and varying limitations that can impede development time and testing.

    iOS also has a more streamlined adoption rate in which new OS versions are generally installed soon after being released (more than 95% of all iOS users are within 1 version of the latest OS) [17]. In comparison, Android users often must stick to older versions because of their carrier and device restrictions; so the adoption rate to newer versions is quite low (less than 40% of all Android users are within 1 version of the latest OS) [18].

    Table 1. Potential app features.
    View this table

    Finally, although Android OS spans thousands of different devices with a highly varying list of specs, iOS devices all use the same underlying architecture and are very limited in variety. An app developed for a modern version of iOS is almost certain to function identically on all the latest iPhones, with minimal design or developmental changes required to compensate for device-specific issues.

    Perhaps most importantly, developing on only a single platform would allow the use of each platform’s respective AR framework. Apple shipped ARKit [19] alongside iOS 11 for the latest generation of iPhones, whereas Google debuted its competing system dubbed ARCore [20]. Neither of these frameworks are cross-platform, and they were not publicly available for much of this app’s lifecycle, but if developing a native AR app in the future, they should be highly considered. We anticipate that open-source libraries for hybrid apps will be developed to transparently support both AR toolsets depending on the platform, which will rival many features currently found in Wikitude.

    The Alternative iPhone Operating System–Only Approach—Heightened Focus on Functionality

    Developing a custom AR component may have been more achievable had the focus been devoted entirely to a single platform. If so, this could have alleviated the need to depend on a costly and proprietary third-party commercial framework, which itself includes many excess features not needed or used by AR MAPPER.

    During this project, a sizeable amount of time was spent troubleshooting and resolving device-specific issues while testing on Android. Even when using the popular hybrid approach, some native components still caused issues depending on the device. When Wikitude 7.0 was released, for example, it inadvertently broke the app on a couple of Android devices we used for testing that did not have certain sensors. This led to more time spent troubleshooting and ultimately having to point out the issues to the framework developers, who addressed the issue, but not formally for another 3 months.

    Other Potential Features for This Type of App

    The time and budget for this project elapsed before several considered features could be properly assessed or integrated. A summary of such imagined features is described in Table 1.

    Evaluating Mobile Augmented Reality Apps

    Although the proposed MAR app was not formally tested, several options exist to assess the success of mobile apps. Usability heuristics have been adapted to the design and evaluation of AR apps [21,22], but they are of somewhat limited utility when performance in the field is of interest. Success of AR app concepts can also be evaluated via surveys before design and development [23] or based on user reports on the use of an AR app [24]; however, these methods also suffer from lack of direct access to the real-world context of use. Although appropriate methods for field testing mobile apps are well understood [25], field evaluation of MAR has generally received less attention [26]. In situ, MAR evaluations have included various combinations of task-based assessment, observation, and subjective feedback collection through questionnaires and interviews [27-30]. Although such short-term, task-based evaluation can provide a baseline assessment of the immediate success of the MAR app, longitudinal studies would be more appropriate for uncovering issues that users would face in their day-to-day interactions.

    This brief report is a snapshot in time in the development of a better, more useful farm mapping tool for emergency responders. As a part of a Centers for Disease Control and Prevention–funded research project, the original Farm MAPPER has been integrated as a tool for high-quality emergency planning for rural firefighters and the agricultural operations in their coverage areas. During trainings, these emergency responders are given a preview of the MAPPER AR and have been highly receptive. Of the 50 or so trainees, nearly all have voiced support for the heads-up style display that MAR technology allows. In future development, it is anticipated that valuable usability feedback will be easily attained, yielding meaningful and actionable results. Despite the limitations of the development caused by accommodating multiple platforms, the purpose of the new technology appears to be valued enough by the end users to allow for additional studies and builds.

    Despite the rapid advancement and adoption of MAR, little is known about the effectiveness of these technologies in real-world rural emergency response [6]. Researchers in related disciplines, such as medical education, have also struggled to evaluate the technology’s place and effectiveness [31-36]. Next steps and future research must address the effectiveness and feasibility of this technology in the agricultural or occupational and patient care environments.

    Conclusions

    From the early design and mockup phases to the final steps of development and testing, this project presented challenges. Many different frameworks and design methodologies were considered to satisfy the original scope and requirements while remaining within the budget. The result is a cross-platform, multi-view prototype app with aerial and AR components.

    Undeniably, the biggest hurdle continues to be present-day hardware limitations, which cannot always guarantee a reliable connection to satellites or cellular towers required to parse data and track very granular and specific points of interest. This approach is not suitable for indoor navigation or other areas with heavy ground obstructions or tree foliage. An even greater concern exists in that many rural locations still do not have adequate cellphone reception. Although GPS signal is generally obtainable in these areas, the app still depends on network connectivity to download satellite imagery and initialize the various components correctly. Thankfully, the spread and improvement of carrier coverage and adoption of mobile devices continue to penetrate rural areas.

    Developers should consider single platform native development to benefit from platform-specific MAR implementations and to avoid development and testing costs associated with cross-platform implementation. Emergency response organizations may be more likely to utilize a single platform across their devices, reducing the benefit of cross-platform development. Furthermore, providing map-based, non-AR cross-platform apps for landowners, farmers, and ranchers would help improve and maintain the data quality of AR content.

    With the recently-launched Apple ARKit and Google’s ARCore framework and more broad concepts such as its Visual Positioning Service for indoor GPS tracking, we can assume that MAR will play increased roles in emergency response, health care delivery, and everyday life in the years to come. Enhancing everyday reality with accurate, meaningful, and visualized data can transform the training and execution of any number of skills, including emergency medical services and bedside health care. MAR technology can increase the amount of contextual data in emergency responses, thereby improving the decision-making capabilities of the users. In turn, this can expedite response times and protect responders. Combined, these elements may improve patient outcomes and increase other successes further downstream in health care delivery.

    Acknowledgments

    Funding support was provided through the Pilot and Feasibility Projects Program of Central States Center for Agricultural Health and Safety Center (NIOSH 5U54OH010162), the Rural Firefighters Delivering Agricultural Safety and Health project as a part of the Upper Midwest Agricultural Safety and Health Center (CDC-NIOSH U54 OH010170), the National Farm Medicine Center, and the Marshfield Clinic Research Institute. The authors would like to thank the project advisor David Schwebel, PhD, and collaborators Fire Chief Jerry Minor and Matthew Keifer, MD.

    Authors' Contributions

    All authors participated in the conception or design of the work; the acquisition, analysis, or interpretation of data for the work. BW acquired funding, drafted the proposal and early drafts of the manuscript with input from all authors. MP led the development of the MAR app. BW, TH, and MP drafted and revised the manuscript. All authors reviewed and revised it critically for important intellectual content. All authors provided final approval of the version to be submitted or published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

    Conflicts of Interest

    None declared.

    References

    1. Milgram P, Takemura H, Utsumi A, Kishino F. Augmented reality: a class of displays on the reality-virtuality continuum. 1995 Presented at: Telemanipulator and Telepresence Technologies; October 31-November 4, 1994; Boston, MA, United States   URL: http://etclab.mie.utoronto.ca/publication/1994/Milgram_Takemura_SPIE1994.pdf [CrossRef]
    2. Chatzopoulos D, Bermejo C, Huang Z, Hui P. Mobile augmented reality survey: from where we are to where we go. IEEE Access 2017;5:6917-6950. [CrossRef]
    3. Olsson T, Lagerstam E, Kärkkäinen T, Väänänen-Vainio-Mattila K. Expected user experience of mobile augmented reality services: a user study in the context of shopping centres. Personal Ubiquitous Comput 2013;17(2):287-304 [FREE Full text] [CrossRef]
    4. Niantic, Inc. 2015 Oct 15. Niantic Inc Raises $20 Million in Financing from The Pokémon Company, Google and Nintendo   URL: https://nianticlabs.com/blog/niantic-tpc-nintendo/ [accessed 2019-02-12] [WebCite Cache]
    5. Reyes I, Rollins T, Mahnke A, Kadolph C, Minor G, Keifer M. Farm mapping to assist, protect, and prepare emergency responders: farm MAPPER. J Agromedicine 2014 Jun;19(2):90-95. [CrossRef] [Medline]
    6. Weichelt B, Yoder A, Bendixsen C, Pilz M, Minor G, Keifer M. Augmented reality farm MAPPER development: lessons learned from an app designed to improve rural emergency response. J Agromedicine 2018;23(3):284-296. [CrossRef] [Medline]
    7. Charland A, Leroux B. Mobile application development: web vs native. Commun ACM 2011 May 1;54(5):49 [FREE Full text] [CrossRef]
    8. OnsenUI.   URL: https://onsen.io/ [accessed 2019-02-15]
    9. Wikitude.   URL: https://www.wikitude.com/ [accessed 2018-02-15]
    10. Amin D, Govilkar S. Comparative study of augmented reality Sdk's. Int J Comput Sci Appl 2015;5(1):11-26 [FREE Full text] [CrossRef]
    11. Malavolta I, Ruberto S, Soru T, Terragni V. End users' perception of hybrid mobile apps in the Google Play Store. 2015 Jun Presented at: IEEE International Conference on Mobile Services; June 27-July 2, 2015; New York, USA p. 25-32   URL: http://www.ivanomalavolta.com/files/papers/MS_2015.pdf
    12. Mercado I, Munaiah N, Meneely A. The impact of cross-platform development approaches for mobile applications from the user's perspective. In: Proceedings of the International Workshop on App Market Analytics. 2016 Nov Presented at: WAMA 2016; November 14, 2016; Seattle, USA p. 43-49. [CrossRef]
    13. Joorabchi M, Mesbah A, Kruchten P. Real challenges in mobile app development. 2013 Presented at: 2013 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement; October 10-11, 2013; Baltimore, Maryland, USA p. 15-24.
    14. Ahti V, Hyrynsalmi S, Nevalainen O. An Evaluation Framework for Cross-Platform Mobile App Development Tools: A case analysis of Adobe PhoneGap framework. In: Proceedings of the 17th International Conference on Computer Systems and Technologies 2016 (CompSysTech '16). 2016 Presented at: 17th International Conference on Computer Systems and Technologies 2016; June 23-24, 2016; New York, USA p. 41-48   URL: https://doi.org/10.1145/2983468.2983484 [CrossRef]
    15. Angulo E, Ferre X. A case study on cross-platform development frameworks for mobile applications and UX. In: Proceedings of the XV International Conference on Human Computer Interaction. 2014 Presented at: Interacción '14; September 10-12, 2014; Puerto de la Cruz, Tenerife, Spain.
    16. Sommer A, Krusche S. Evaluation of Cross-Platform Frameworks for Mobile Applications. 2013 Presented at: International Conference on Engineering and Applied Science; March 15-17, 2013; Tokyo, Japan p. 363-376   URL: https:/​/www.​researchgate.net/​publication/​327719390_Evaluation_of_Cross-Platform_Frameworks_for_Mobile_Applications
    17. Apple Inc. App Store   URL: https://developer.apple.com/support/app-store/ [accessed 2019-02-13] [WebCite Cache]
    18. Android Developers. Distribution dashboard   URL: https://developer.android.com/about/dashboards/ [accessed 2019-02-13] [WebCite Cache]
    19. Apple Inc. ARKit   URL: https://developer.apple.com/arkit/
    20. ARCore.   URL: https://developers.google.com/ar/ [accessed 2019-02-18] [WebCite Cache]
    21. de Paiva Guimarães M, Martins VF. A checklist to evaluate Augmented Reality Applications. In: Proceedings of the 2014 Fifth International Conference on Intelligent Systems Design and Engineering Applications. 2014 May Presented at: ISDEA'14; June 15-16, 2014; Salvador, Brazil p. 45-52.
    22. Lee W, Lee HK. The usability attributes and evaluation measurements of mobile media AR (augmented reality). Cogent Arts Humanit 2016;3(1) [FREE Full text]
    23. Olsson T, Kärkkäinen T, Lagerstam E, Ventä-Olkkonen L. User evaluation of mobile augmented reality scenarios. J Ambient Intell Smart Environ 2012;4(1):29-47. [CrossRef]
    24. Olsson T, Salo M. Online user survey on current mobile augmented reality applications. In: Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality. 2011 Presented at: 2011 10th IEEE International Symposium on Mixed and Augmented Reality; October 26-29, 2011; Basel, Switzerland p. 75-84.
    25. Kjeldskov J, Skov MB. Was it worth the hassle? Ten years of mobile HCI research discussions on lab and field evaluations. In: Proceedings of the 16th international conference on Human-Computer Interaction with Mobile Devices & Services. 2014 Sep Presented at: MobileHCI'14; September 23-26, 2014; Toronto, Ontario, Canada p. 43-52.
    26. Dey A, Billinghurst M, Lindeman RW, Swan J. A systematic review of 10 years of augmented reality usability studies: 2005 to 2014. Frontiers in Robotics and AI 2018;5(37) [FREE Full text] [CrossRef]
    27. Woodward C, Kuula T, Honkamaa P, Hakkarainen M, Kemppi P. Implementation and evaluation of a mobile augmented reality system for building maintenance. In: Proceedings of the 14th International Conference on Construction Applications of Virtual Reality.: Teesside University; 2014 Nov Presented at: 14th International Conference on Construction Applications of Virtual Reality; November 16-18, 2014; Sharjah, UAE p. 18   URL: http://virtual.vtt.fi/virtual/proj2/multimedia/media/publications/CONVR2014-Woodward_et_al.pdf
    28. Damala A, Cubaud P, Bationo A, Houlier P, Marchal I. Bridging the gap between the digital and the physical: design and evaluation of a mobile augmented reality guide for the museum visit. In: Proceedings of the 3rd international conference on Digital Interactive Media in Entertainment and Arts. 2008 Presented at: DIMEA '08; September 10-12, 2008; Athens, Greece p. 120-127.
    29. Kerr S, Rice MD, Teo Y, Wan M, Cheong YL, Ng J, et al. Wearable mobile augmented reality: evaluating outdoor user experience. In: Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry. 2011 Presented at: VRCAI '11; December 11-12, 2011; Hong Kong, China p. 209-216.
    30. Dhir A, Al-kahtani M. A case study on user experience (UX) evaluation of mobile augmented reality prototypes. J Univers Comput Sci 2013;19(8):1175-1196 [FREE Full text] [CrossRef]
    31. Noll C, von Jan U, Raap U, Albrecht U. Mobile augmented reality as a feature for self-oriented, blended learning in medicine: randomized controlled trial. JMIR Mhealth Uhealth 2017 Sep 14;5(9):e139. [CrossRef] [Medline]
    32. Stretton T, Cochrane T, Narayan V. Exploring mobile mixed reality in healthcare higher education: a systematic review. Res Learn Tech 2018 Nov 27;26 [FREE Full text] [CrossRef]
    33. Kobayashi L, Zhang XC, Collins SA, Karim N, Merck DL. Exploratory application of augmented reality/mixed reality devices for acute care procedure training. West J Emerg Med 2018 Jan;19(1):158-164 [FREE Full text] [CrossRef] [Medline]
    34. Dixon B, Daly MJ, Chan HH, Vescan A, Witterick IJ, Irish JC. Inattentional blindness increased with augmented reality surgical navigation. Am J Rhinol Allergy 2014;28(5):433-437 [FREE Full text] [CrossRef] [Medline]
    35. Moro C, Štromberga Z, Raikos A, Stirling A. The effectiveness of virtual and augmented reality in health sciences and medical anatomy. Anat Sci Educ 2017 Nov;10(6):549-559. [CrossRef] [Medline]
    36. Garrett BM, Anthony J, Jackson C. Using mobile augmented reality to enhance health professional practice education. Curr Iss Emerg eLearn 2018;4(1):224-247 [FREE Full text]


    Abbreviations

    API: application programming interface
    AR: augmented reality
    GPS: Global Positioning System
    iOS: iPhone operating system
    MAPPER: Mapping to Assist, Protect and Prepare Emergency Responders
    MAR: mobile augmented reality
    OS: operating system
    UI: user interface
    UX: user experience


    Edited by G Eysenbach; submitted 14.09.18; peer-reviewed by A Wang, A Mahnke, S Zheng; comments to author 27.10.18; revised version received 21.12.18; accepted 07.01.19; published 28.03.19

    ©Bryan Weichelt, Tomi Heimonen, Matthew Pilz, Aaron Yoder, Casper Bendixsen. Originally published in JMIR Mhealth and Uhealth (http://mhealth.jmir.org), 28.03.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mhealth and uhealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.