Augmented reality (AR) is an emerging technology that can enhance how the real world is experienced by the user. Compared with virtual reality (VR), in which the user is immersed in a completely synthesized world, AR combines both the virtual and real by overlaying the external world with computer-generated sensory data such as audio, video, and graphics. AR technology, often accessed through head-mounted devices (HMDs) or software on personal devices, can be used to display information and virtual objects that facilitate learning and navigation through tasks in the real world [1,2].

In medicine, VR and AR have been explored in educational, diagnostic, and treatment settings, with an increasing number of publications in the last decade [3-6]. A 2012 to 2017 review of 338 original studies using AR in medicine, most related to surgery and simulation learning, estimated the technology readiness level of AR to be at the stage of a prototype that has yet to be completed and tested in its intended environment [7]. AR has since appeared in the literature across many specialized fields, with reviews since 2019 describing AR technology in emergency medicine (EM) [8], dermatology [9], radiology [10,11], orthopedics [12], nursing [13], and many more.

A promising use of AR technology is in remote collaboration, an application seen in many industry- and engineering-related tasks over the last 2 decades [14,15]. As COVID-19 pushed health care to explore remote health solutions, providers, caregivers, and students have become increasingly aware of technology’s role in enabling access to care [16-18]. A cohort study of 36.5 million individuals in the United States found that 23.6% of ambulatory visits in 2020 were billed as telehealth visits compared with 0.3% in 2019 [19]. Although videoconferencing programs allow health care workers to remotely connect with each other, trainees, and patients, current systems limit the extent of care that can be delivered during such interactions [20]. Innovations using AR technology offer an opportunity to expand real-time remote health services such as consultation and telesurgery [21]. Recent literature on AR includes studies on caregiver perspectives on the technology [22], frameworks for AR-assisted remote medical communication [23], and remote health care delivery devices incorporating AR [24].

Remote medical communication is a defining feature of telemedicine, a concept that first arose with the use of telephones to share information across hospital systems [25]. Since the conception of the internet and personal devices, remote health care visits and interventions have become possible, with the scope of care expanding as technological advances are introduced. Currently, there is no literature summarizing the applications of AR in synchronous telemedicine, defined as the use of electronic devices for real-time communication in health care services related to patient encounters, treatment, and consultation [25,26]. Telementoring, a subcategory of telemedicine, is the real-time remote guidance of health care procedures or skills. Telementoring also plays a role in granting remote access to care and expertise, enabling the sharing of specialized knowledge and education [27]. Similar to telemedicine, it is also a topic scarcely studied in relation to AR and has the potential to evolve with technological innovation [28]. By exploring how AR is used for real-time telemedicine and telementoring, this review could better inform health care providers and developers of AR’s future potential and current limitations.

Scoping reviews entail a systematic selection of literature with the purpose of examining the extent and nature of an area of interest [29,30]. Compared with systematic reviews, scoping studies allow for the integration of a range of study designs, especially in fields with emerging evidence that may lack randomized controlled trials (RCTs). By mapping the existing evidence of AR in telemedicine, the scoping approach allows for the identification of gaps that may inform future studies and innovations.

Which devices and platforms using AR have been studied in the published literature in the context of real-time telemedicine and remote education? In which areas of medicine have these been integrated and for what purposes? How are outcomes evaluated and what variables have yet to be measured? What are the overall findings of existing studies?

The literature was reviewed using PubMed, Scopus, Embase, and MEDLINE for articles or trials published from January 1, 2012, to October 18, 2022, with search queries submitted and articles accessed on October 18, 2022. The search terms were “augmented reality” OR “AR” AND “remote” OR “telemedicine” OR “telehealth” OR “telementoring.” The PubMed search was performed using article titles and abstracts. The Scopus search was performed using article titles, abstracts, and keywords, with articles and conference papers in the areas of “medicine,” “health professions,” and “nursing” included. The Embase search was performed using article titles, abstracts, and keywords, with articles and conference papers included. The MEDLINE search was also performed using article titles, abstracts, and keywords. Only articles available in English were included.

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, the titles and abstracts of all articles were independently reviewed by 2 researchers (AD and AF) for relevance to AR and real-time communication between separated individuals. Articles were excluded if they were unrelated to medicine or were reviews or discussions. Any articles that were included by one reviewer but not the other were included in the full-text screening, which was also performed independently by the 2 researchers. As this review focused on the use of AR rather than the development of related equipment or software, articles were excluded if the technological design, rather than the implementation, was the focus or if the technology was not intended for remote interaction as described by the inclusion criteria. Articles that included both technological design and implementation data were included, with the review focusing on the latter. Articles that described mixed reality devices capable of both AR and VR were included if the implementation primarily used and studied AR features. Reviews, perspectives, discussion-based articles, and study proposals without results were excluded. Correspondence was sent to the authors of articles for which the full text was not available; a lack of response resulted in the exclusion of these articles. Any disagreements regarding an article’s inclusion were resolved through discussion between the 2 reviewers.

The articles were reviewed based on the context in which the AR was implemented. Articles describing mixed reality devices were analyzed for data relevant to AR use and not VR. Unique devices and platforms using AR across the articles were identified. The articles were later grouped into one of the 3 identified areas: patient evaluation, medical intervention, and education. “Patient evaluation” included articles that described the examination of patients and processes that obtained information for clinical decision-making. “Medical intervention” included articles that described procedures related to the initiation or provision of therapy. “Education” included articles that described the mentorship or training of a less experienced individual for a task or procedure. Subgroups for surgical versus nonsurgical tasks in the latter 2 groups were created. Articles that fell into more than one of the 3 identified areas were organized in the Results section of this paper based on which area was the primary focus. Finally, common objective and subjective end variables discussed in the articles were identified.

For this scoping review, we first provide an overview of the devices and platforms that use AR and the types of tasks in which they appear. We then summarize the implementation and measurement of these AR-capable tools within 3 areas: patient evaluation, medical intervention, and education. Finally, we review the common methodologies and end variables observed across the studies.

The PubMed, Scopus, Embase, and MEDLINE searches yielded 298, 195, 187, and 274 articles, respectively. This totaled 954 articles, 558 (58.5%) of which were identified as duplicates. The abstracts and titles of the remaining 396 articles were reviewed, with 62 (15.7%) identified as meeting the inclusion criteria. In total, 5% (3/62) of the articles, for which the full text was not available, were excluded after no response was received from the original authors. A total of 39 articles were included following full-text screening. The selection process is depicted in Figure 1. The publication years of the selected articles spanned 2014 to 2022, with 64% (25/39) published in the last 3 years.

From the included articles, 20 unique devices and platforms with AR features were identified, with 10 (50%) being commercial HMDs in which AR features were projected before the wearer’s eyes and another 4 (20%) being “virtual presence”–type platforms in which a remote viewer can superimpose video of their hand or tools over the live stream of a local site or procedure; the hybrid video is accessed by both local and remote users via a smartphone, tablet, computer, or monitor. The remaining 30% (6/20) involved systems with no commercial HMDs or virtual presence—these included a smartphone app and systems built specifically for tele-ultrasonography, physical rehabilitation, and surgical telementoring. Overall, all the identified devices and platforms (20/20, 100%) allowed remote individuals to view the perspective or environment of the local user. Common AR features of these devices included annotation (12/20, 60%) and graphical overlay over the local user’s view, specifically 2D or 3D images (9/20, 45%) and the remote viewer’s hands or tools (8/20, 40%). The identified devices and platforms are listed in Table 1, with similar device models included in the same row.

Of the 39 studies, 22 (56%) were related to surgery, 6 (15%) were related to EM, and 4 (10%) were related to hospital medicine. When looking at the setting and structure of the tasks from each study, 51% (20/39) involved an operation or technique used in the operating room, where a mentor figure used AR to remotely interact with a task performer. The remaining 19 studies involved nonoperative tasks, 11 (58%) of which also involved a mentor remotely interacting with a task performer, whereas 8 (42%) involved the mentor performing the task instead, with AR enhancing what their remote spectators saw.

The articles were divided into 3 sections based on the AR-assisted task performed: patient evaluation, medical intervention, and education (Multimedia Appendix 1-3). Medical intervention and education are further subdivided based on whether AR supported a nonsurgical or surgical task. Notably, some articles discussed in the medical intervention section were also relevant to education.

The 26% (10/39) of articles included in this section (Multimedia Appendix 1 [33,34,37,43-45,54,65-67]) described implementations of AR in remote triage, wound assessment, musculoskeletal examination, sonography, and hospital rounding.

The potential of AR to perform fast-paced triage assessments remotely was explored in 20% (2/10) of the studies. Broach et al [34] investigated the use of Google Glass to relay what paramedics saw to other remote providers, with the intention of allowing the remote EM physicians to perform secondary triage before the patient’s arrival at a hospital. The remote physicians accessed the perspective of the paramedics at a simulated disaster scene and could send instructional SMS text messages projected onto the paramedics’ Glass. When comparing the remote assessments of physicians with the in-person assessments by different EM physicians, the study found high interrater agreement (0.923), which was not significantly different from the interrater agreement within the same assessment condition (0.976; P=.41) [34]. Follmann et al [33] investigated how AR implementation could affect triage time and accuracy. The performance of non–AR-assisted first responders was compared with that of 2 groups using AR-capable glasses, one that displayed an interactive triage algorithm and the other that streamed footage to a remote EM physician who could verbally guide the on-site individual. The results revealed that AR assistance increased accuracy at the cost of time, with the accuracy of the 3 groups being 58%, 92% (P=.04), and 90% (P=.01), whereas the duration of triage was 16.6, 37.0 (P=.001), and 35.0 (P=.01) seconds, respectively [33].

Other studies focused on AR’s potential to enhance remote wound assessments. Ponce et al [54] demonstrated the use of AR with mobile devices to allow orthopedic surgeons and neurosurgeons to conduct remote postoperative visits. Using a virtual interactive presence (VIP) smartphone app, surgeons overlaid live camera footage of their hands over the camera footage of the patient’s postoperative wound. Surveys from users revealed that both patients and surgeons found utility (27 of 28 and 26 of 29 with positive responses) in the virtual experience [54]. Kaylor et al [37] and Hill [43] focused on the use of AR for consultations during inpatient wound assessments. Using Microsoft HoloLens, bedside nurses in the former study could send live video of a patient’s wound and communicate with a remote wound, ostomy, and continence (WOC) nurse. The remote WOC nurse could provide annotations or images to guide the local nurse during the assessment. When comparing remote assessments with in-person assessments performed by a different WOC nurse, the study found the interrater agreement of treatment plans to be 100% across 21 cases [37]. The bedside nurses in the study by Hill [43] used the Microsoft HoloLens 2 to consult for complications at night or over weekends for patients undergoing negative pressure wound therapy. Compared with a control group of previous cases that did not use AR, the study group underwent fewer unplanned surgical revisions (P=.002) and admissions related to wound infection (P=.004) [43].

Borresen et al [66,67] introduced the AR-based Telerehabilitation System with Haptics (ARTESH) to perform remote strength and range-of-motion examinations. Both the local and remote sites were equipped with a haptic device, a Kinect camera, and a 3D-capable television that allowed the physician and patient to view each other seated together at a virtual table. The 7-point Likert scale surveys from the pilot study showed positive ratings from 5 physicians on the ability to evaluate arm strength and visualize limb movement (6/7 and 5.87/7) [66]. A follow-up study measured interrater agreement between in-person examinations and remote evaluations using ARTESH for different components of the upper extremity examination. The highest levels of agreement were observed in strength testing of elbow flexion, shoulder abduction, and protraction (κ=0.63, 95% CI 0-1.0), with the percentage of interrater agreement across evaluations for all 15 patients ranging from 30 to 100 [67].

Rigamonti et al [44] and Carbone et al [65] conducted studies that integrated AR technology to allow for supervision and consultation while performing ultrasound examinations. Rigamonti et al [44] interviewed engineering and sports science professionals across 6 countries after they accessed live video footage from a Microsoft HoloLens 2 worn by an ultrasound operator in Germany. Users were able to annotate the stream, overlay pictures and videos onto the display, and communicate in real time as they watched the examination from the operator’s perspective. Interview responses from the spectators revealed AR’s potential in both education and enhancing remote examinations with the supplementation of live data and feedback [44]. Carbone et al [65] developed a tele-ultrasound system that allowed rural hospital clinicians to contact a consultant. The remote consultant, who viewed live video of the user’s environment and ultrasound sequences, could provide audio feedback and overlay annotations or a cursor on the ultrasound imaging seen at the local site. Although the connected parties primarily used the platform to discuss the diagnosis, in 5 of 12 cases, the consultant directed the user on the device’s probe position [65].

The study by Martin et al [45] used an AR-capable Microsoft HoloLens 2 in multiple COVID-19 wards. The senior member of a clinical team would wear the headset and personal protective equipment (PPE) while examining patients, allowing the remaining members of the team to watch and interact remotely. Viewers could remotely annotate objects as well as overlay patient imaging and data from the electronic health record onto the view of the headset user and the live video feed. In comparison with teams without the device, the AR-assisted teams saw less COVID-19 exposure time by 51.5% and decreased PPE use by 83.1% over a week (P=.002 and P=.02) [45].

The 41% (16/39) of articles included in this section (Multimedia Appendix 2 [31,35,38,39,46,52,53,55-57,59-61,63,64,68]) described AR in remote nonsurgical and surgical contexts. The latter included studies that focused on surgical efficiency, long-distance consultation, and differences between telesurgical systems.

The articles discussed in this section (Multimedia Appendix 3 [32,36,40-42,47-51,58,62,69]) are divided based on nonsurgical contexts, which include clinical skills, autopsy, and sonography, and surgical contexts, which include procedure observation, tool-specific training, simulations, and intraoperative learning. Of note, one-third (6/19, 32%) of the articles that involved education have been described in the previous section and will be briefly mentioned in this section.

In this scoping review, we discussed 39 studies that used AR in real-time telemedicine and telementoring. From these studies, 20 unique devices and platforms, most of which involved an HMD, were identified and found to have common features such as annotation, graphical references, and the ability to overlay a remote viewer’s hands or tools onto a local user’s screen.

AR builds on the remote examinations of current audio- and videoconferencing tools by enhancing the remote acquisition and exchange of information. AR technology can supply users with visual aids, such as electronic health record data and guidelines, or facilitate communication with specialists trained to identify specific conditions or manipulate diagnostic equipment. Studies on AR-assisted remote patient evaluations took place in both outpatient and inpatient settings, including emergency triage, wound evaluation, and hospital rounding. Although the devices vary across settings, the comparative studies in this review found noninferior accuracy and reliability of AR-assisted remote conditions compared with in-person controls. Devices dedicated to remote musculoskeletal and sonographic examinations were also developed, with positive user ratings. As technology improves and becomes more accessible, the types of examinations that can be performed remotely will expand and allow for greater access to care, especially by patients and providers in low-resource areas.

The research on AR in the remote delivery of medical interventions predominantly focused on surgery, with other treatment modalities such as physical therapy being less studied [70]. Telesurgery implemented AR features such as virtual presence and annotation to facilitate procedural guidance, mentorship, and consultation. The distance of such connections was tested, with several case reports describing AR in the telementoring of operations across continents. Various types of surgical procedures were telementored using AR, including minimally invasive surgery, orthopedic surgery, and pediatric neurosurgery. The literature surrounding AR and telesurgery focused primarily on the technology’s potential to improve procedural efficiency and communication with experts, with the differences across systems for surgical telementoring studies being investigated in more recent articles. From the data available so far, the performance of AR-assisted procedures appears equivalent to other remotely mentored conditions, with limited research suggesting AR’s potential to substitute in-person mentorship without sacrificing resources or short-term outcomes. Although such findings require further validation, AR plays a promising role in enabling less experienced providers to perform more specialized treatments and avoid the unnecessary transfer of patients across hospital systems. Research on AR-assisted telerehabilitation and psychotherapy could offer insight into whether these nonsurgical treatments could also be effectively performed as remote visits.

Medical education emerged as a common theme underlying AR’s integration into telemedicine and telementoring. In addition to surgery, remote training using AR was studied for clinical skills that are important to inpatient wards and procedures related to pathology and EM. Both surgical and nonsurgical studies implemented AR tools in similar ways, including observational learning, real-time audiovisual feedback, and teaching by demonstration. When in-person mentorship was not possible or inconvenient, the studies showed how AR-enhanced communication allowed distant learners to not just observe but also engage with clinicians and surgeons. Distance learning of surgical and sonographic equipment could also be achieved with AR systems, with few controlled trials thus far indicating noninferior performance results compared with in-person mentoring and superior results compared with unassisted conditions, particularly for less experienced trainees. As the equipment for telecollaboration in medicine becomes readily available, AR technology is expected to advance the sharing of knowledge related to clinical skills, health care devices, and procedural techniques, which directly and indirectly improves patient care outcomes.

Given the early stage of platform development, many studies included in this review (25/39, 64%) used small cohorts of ≤15 patients or participants per study group; although approximately half (21/39, 54%) of the studies included comparative data, far fewer were RCTs. This finding is expected as AR is an emerging technology in medicine, and its use in telemedicine and telementoring is still developing. Existing comparative trials that measured time consumption and performance have so far found that AR-assisted conditions mostly have noninferior results compared with other remote and in-person methods. However, as these studies are heterogeneous in methodology and equipment, dedicated RCTs with standardized designs are needed to understand whether certain AR systems for specific tasks can effectively substitute current remote alternatives or in-person methods. Feedback surveys and interviews were the most common form of data collection observed in this review; subjective variables such as perceived efficacy and ease ratings were commonly measured with consistently positive findings despite the novelty of the technology, suggesting interest across user groups, including providers, trainees, and patients.

Overall, interventions and learning that require active patient or family caregiver participation (eg, physical therapy, psychotherapy, preventative medicine, chemotherapy, and dialysis) have yet to achieve a similar level of investigation in the space of AR and telemedicine or telementoring as surgery. From the nonsurgical studies included in this review, there is an interest in AR for tele-sonography, EM, and hospital medicine, with many nonsurgical fields yet to be represented. In the realm of patient evaluation, AR’s role is still being separately investigated for electrograms [71], diagnostic procedures related to endoscopy [72], biopsies [73], and urology [74], as well as specialty-specific examinations pertinent to dentistry [75], ophthalmology [76], and dermatology [9], to name a few.

Considering the diversity of medical fields and levels of experience across users, AR-enabled remote interactions are likely to appear in certain settings or users sooner than in others. In the interventional and educational space, studies so far have primarily implemented AR tools for remote consultation between current or future medical professionals. Few trials using untrained individuals for AR-assisted procedural, nonsurgical tasks exist; it would be reasonable to anticipate future research with AR facilitating remote provider-to-patient or provider-to-caregiver interactions in the therapeutic context [77]. To support innovations focused on remote interactions with home caregivers and patients, perspectives from and qualitative studies on these particular end users, rather than solely clinician users, are necessary [22,78].

Although more dedicated RCTs would be needed to assess the efficacy of AR-assisted communication in the treatment setting, AR’s impact on variables related to costs beyond procedure time and outcome measures beyond complication rates remains relatively unexplored. These include hospital-based measurements such as length of stay, equipment costs, and intra- and postoperative pain medication use but also patient-centered variables such as treatment cost, posttreatment functional status, and quality of life. The literature in this review focused mainly on short-term variables, which reveals a lack of longitudinal research that could provide insight into both the long-term outcomes of patients and system-wide effects on productivity and sustainability of AR use. Longitudinal studies with dedicated comparisons, which are likely to increase as devices improve in wearability, usability, and affordability, are also needed to fully understand whether AR could enhance knowledge and skill retention in the remote learning environment.

This review was limited to studies from the last 10 years in 4 medical research databases. Many AR developments in telemedicine and telementoring that exist in the private sector have not been described in published articles and so cannot be systematically evaluated. Notably, a 2019 systematic review of AR in telementoring has also explored additional databases [28]; however, our review included “remote,” “telemedicine,” and “telehealth” as search terms to locate a wider variety of health care tasks that may not rely on equipment or procedures typically associated with “telementoring.” Furthermore, this scoping review placed more emphasis on the diversity of systems using AR and the nonsurgical specialties incorporating them. The small cohorts and predominant collection of subjective data were expected given the novel nature of this intersection and the technology.

Although the potential benefits of AR in telemedicine are promising, the challenges facing this technology are the early stage of research and prototype development for these application contexts and a lack of standardized devices. Outside the original and surgery-specific platforms, most of the hardware observed in this review is available to consumers but at costs that limit widespread use [79]. Furthermore, the adjunct programs and applications used with the hardware greatly varied across the studies. The diversity of tools and their availability could limit the design and generalizability of future trials, especially if the technology is custom-made and difficult to reproduce. Combined with low awareness and a lack of guidelines on how to evaluate AR technology, innovators face difficulty in developing appropriate tools and introducing them into current or even unexplored health care spaces. Future research focusing on the utility and feasibility of AR compared with current technology in the medical setting is paramount, but studies looking into the costs of implementation, user readiness, and user-friendly design are also necessary for successful adoption.

This scoping review discussed studies that combined AR with real-time telemedicine and telementoring, including patient evaluation, medical intervention, and education. Commonly explored applications for this novel intersection include consultation and procedural guidance, particularly in telesurgery. AR-assisted telecommunication was studied to complement or even improve the capability of remote visits, treatments, and training, but more RCTs are needed to validate task-specific benefits as well as understand the long-term effects for all users. As technology evolves and use at the consumer and industry levels becomes more widespread, research on AR in health care is expected to see larger cohorts, standardized equipment, and more rigorous methods of evaluation. Developing AR tools in medicine must balance user-friendly design with limited research and uptake; such challenges create an opportunity for institutional involvement and a need for perspectives from all those involved in health care, including but not limited to clinicians, caregivers, and patients.