Background

The first application of augmented reality was developed by Ivan Edward Sunderland in 1968 with a binocular system “kinetic depth effect” made of two cathode ray tubes. It wasn’t until 1991 that the definition of “augmented reality” was first described by Tom Caudell of the Boeing Company [1,2,3].

Since then, the popularity explosion of augmented reality has reached high levels in the last lustrum. Its applications are also easier since many existing devices are compatible with this technology while other are being developed in order to maximize its performances [1]. The gaming industry is predominant in the augmented reality area because of the expertise brought by virtual reality development [4]. The inherence from this specific field provided tools which are being used by some researchers, for example, virtual reality headset [5, 6].

The definition of augmented reality refers to: “a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view”. Augmented reality, however is commonly confused with virtual reality since both have many aspects in common, even though the outcomes are completely different. Virtual reality, as the name suggests, is a virtual immersive environment where the user’s senses are stimulated with computer-generated sensations and feedbacks generating an “interaction”. Augmented reality, instead, generates an interaction between the real environment and virtual objects. For example a virtual reality system would be a head worn helmet which simulates navigation inside human body and permits the user to explore it on the base of a virtual three-dimensional reconstruction. A similar example with the augmented reality would permit to directly observe a human body and to see virtual objects on it, or through it as the anatomy of the body was superimposed [1, 2, 7].

Immersive reality is similar to augmented reality but the user is interacting with a digital 3d world recreated through 360° real records. The user can navigate recordings which replace the real world in a convincing way. The 360 records recreate the continuity of the surrounding with no interruptions. There also might be physical interaction with the environment and physical feedback given by haptic response when interacting with an object. Other features can be added as 3D audio direction, freedom of movement in the environment and conformance to human vision, which permits correct sizing of object in distance [6].

Its application in dentistry begun with the development of new visualization system for anatomic exploration from the use of virtual-reality based software [5].

The growth in popularity has brought the use of augmented reality to the attention of the medical researchers and of the digital centers who are following two different methods: using already available systems or developing their own, customized combination of hardware and software.

However, substituting virtual reality with augmented reality means to superimpose virtual objects to the reality in a precise and reproducible way considering the three dimensions of space as well as the user’s and patient’s movements. This is still a controversial topic since it is highly affected by the system used. Most authors propose a handmade pre-operative calibration, instead of an automatic one. However the use of markers simplifies this tracking process. The most commonly used systems are head mounted displays and half, silvered mirror projections, both of them are valid systems for augmented reality and have a multitude of different setting as described by Azuma et al. [1].

The superimposed virtual objects are usually obtained with 3-dimensional X-rays as CT dental scans which are then manipulated with commercially available software for CBCT manipulation. Also MRI, angiography or any other three-dimensional data could be used in the same way. The most commonly used software is Mimics (Materialise, Leuven). The object is exported in a widely recognisable format (.stl for example) using “mask” function set with thresholding on the area of interest and 3D reconstruction function [8,9,10,11].

The revolutionary scope of developing an augmented reality based system is to solve one of the biggest issue in the structure of most digital dentistry commonly available systematics; in fact, the use digital technologies like the scanners is structured in a 3-step procedure which can be summarized as follows: the digital image is acquired by a scanning device, the changes are performed digitally from T0 to T1, the new information is transferred back to solid state. The use of augmented reality permits direct visualization bypassing the last transfer step, which means, on a large scale, to avoid data and time loss. Visualization of digital data directly on the patient means the possibility of achieving great advantages in digital procedures [12, 13].

The aim of this systematic review was to collect and to describe available literature about the use of augmented reality in different fields of dentistry and maxillofacial surgery. Collected data will be used to describe the current combinations of hardware and software proposed by the authors, with a focus on self-developement, the field of interest where augmented reality is being used, the primary outcomes which are being obtained by the use of different systems and the precision and timing of the procedures performed. Data about sample considered in the different studies and the designs of the protocol proposed will be also described.

Materials and methods

A prior research was made before the beginning of the study design. Manuscripts from 1968, the year when augmented reality was first described, to the end of 2018 were considered. A protocol for the research was structured by the authors after screening the titles and the abstracts of the articles found. After full accordance among the authors it was registered in PROSPERO with rn:CRD42019120058. The search strategy included the databases to be screened and the search query. The articles found were selected with the application of inclusion and exclusion criteria. The resulting full texts were analyzed by the authors for data extraction. Full text access has been granted by “Università Degli Studi di Milano” - University of Milan, Orthodontics department for the research.

Search strategy

The review was researched using the following electronic databases: Medline, Ovid, Pubmed, Embase, Cochrane Library and Google Scholar. The research refers to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA-P) 2015 [14].

The search query used is available in (Fig. 1) MeSH terms.

Fig. 1
figure 1

MeSH research strategy

Full size image

Grey literature was also screened according to Pisa declaration on Policy Development for Grey Literature Resources.

Inclusion criteria

Articles describing new or already existing applications or frameworks for augmented reality methodologies and relevant informations include: type of intervention, field of interest, clinical outcomes, precision and timing efficiency of the proposed system and combination of software and hardware used were considered. Articles in English referring to Dentistry, oral and maxillofacial surgery were included. No limit for study design was applied, the target of the studies considered are: humans, human parts (extracted teeth), phantoms, animals. Studies from 1968 were considered.

Exclusion criteria

All the articles describing virtual reality systems were discarded, like anatomical explorations, improper use or any concept which doesn’t refer to the exact definition of augmented reality as described in the introduction section.

All the articles lacking methodology description with at least less than 3 of the following were discarded: study design, sample size, hardware utilized, software installed.

All the descriptive methodologies, conference papers, patents, and all the publications in general not identified as “Articles” were discarded.

All the application areas not related to dentistry, oral surgery or cranio-facial district where discarded.

Qualitative analysis and quantitative synthesis

The research outcomes synthesis refers to SPIDER (Sample, Phenomenon of Interest, Design, Evaluation, Research type) tool [15].

Data regarding precision measurements and times of the procedure were collected. The data regarding precision described the error in millimeters or percentage between the markers and the digital image, the error between the real object and the superposed digital image [8, 16,17,18,19,20,21,22,23,24,25,26,27,28,29,30] and the degree of the orientation error [18, 30,31,32,33,34,35,36,37,38,39,40,41,42,43,44]. Time measurements were taken regarding the additional time required to fit the digital models or the gain in the operative procedures [8, 22, 26, 27, 29, 30]. The high number of variables made the data inconsistent for meta-analysis. The variables considered for qualitative synthesis were: the type of procedure and field of interest, primary outcome and results obtained, study design, software used and if custom made or already existing, type of hardware, sample size and target of the study: animal/human or phantom.

Risk of bias in individual studies

Due to the high heterogeneity of the studies design, which is common for new technologies independently developed with different features, common tools for risk of bias assessment were not applicable. In general, risk of bias was considered and judged by the authors low or null for data description but very high for analyzing effectiveness of such methodologies. All the studies found in literature presented high or unknown selection bias and reference standards. Also none of the studies refers to a specific protocol.

Results

The primary research gave a total of 17,652 records after duplicates removal, 17,603 results were excluded on the base of full title and abstract, 45 out of 49 studies were considered eligible. After full text reading by two among the authors, 33 articles were selected after application of exclusion and inclusion criteria (Fig. 2). Variables regarding the sample size and target of the study: animal/human or phantom, type of hardware, software used, field of interest of the proposed procedure and study design, were then extracted from the text, collected and discussed among all the authors. Out of the 33 articles 16 contained at least one quantitative description of the following variables regarding timing of the procedure and precision o the proposed system.

Fig. 2
figure 2

PRISMA flow chart

Full size image

Data extraction

Sample

Out of the selected studies we found that the 51% (n = 17) is performed on phantoms, with 2 of those performed also in one single volunteer: for video see-through on maxillofacial surgery and for the overlaying of computed tomography on the surgical area. Studies referring to experiments carried out on real human patients are 33% (n = 11) considering the two with a single volunteer [8, 21].

Out of in vivo studies on humans the ones referring to actual interventions carried out on patients with the use of augmented reality systems are: intra-oral distractor positioning on 10 cases with 10 controls (OMS/maxillofacial surgery F.O.I), 16 class III patients for waferless maxillary positioning (OMS/maxillofacial surgery F.O.I), one subject for orthodontic positioning of brackets (Orthodontics F.O.I), MASO on 15 patients (OMS/maxillofacial surgery F.O.I), maxillary positioning on 5 patients (OMS/maxillofacial surgery F.O.I), and on 148 patients for multiple operations (OMS F.O.I). All the interventions carried out on humans outcomes are positively described by the authors, with no exception [20, 22, 24, 26, 27, 30].

Other samples considered refer to animal with the studies in 2017 for MASO on dogs, in 2015 for vascular landmarks on one porcine tongue and in 2015 for dental implants on a pig corpse [19, 37, 39].

Other studies have been carried out in vitro on 126 human teeth for Endodontics F.O.I in 2013 (Table 1) [25].

Table 1 Overview
Full size table

Phenomenon of interest: hardware used

Out of the studies considered the majority (60% n = 20) refers to camera-display based systems although the most classical use of augmented reality refers to systems which are head-mounted used in 21% of the studies considered (n = 7). Other systems described are glass silvered mirrors or mirror based systems (n = 3) with 3 selected studies using other specific systems. One system described consists in an interactive portable display unit which can be defined as camera-display based, portable as the H.M.S. but not wearable (Table 2) [22].

Table 2 Software and Hardware
Full size table

Phenomenon of interest: software used

Extracted data about software used in the studies brings that 7 authors describe new custom-made software for a total of 9 studies. The authors involved into the development of the customized new software are describe using C++ programming language to develop the new software while Bogdan describes using C++ and Java language [16,17,18, 21, 25, 31, 39, 40, 45].

The majority of studies presents a variety of commercially available softwares as well as: Leap Motion® [5];, Ar Toolkits® [37], ITK Snap® [19], Hitlab NZ® [32], Aumentaty® [33], Maya® [34], Iplan® [22], Implant Smart® [23], Dentsim® [29], Xscope® [27], Medscan® [30] and multiple software [43].

The only software used by a multitude of authors is Mimics® from 4 authors for a total of 6 manuscripts [8, 20, 26, 36] (Table 2).

Field of interest: F.O.I

Out of 33 selected studies the majority refers to the OMS (Oral and Maxillofacial Surgery) area which can be divided into three specific areas: implantology, maxillofacial surgery and oral surgery; the following area were restorative dentistry, educational and learning and orthodontics.

Respectively OMS included 21 studies divided into 17 for maxillo-facial, 3 for implantology and 1 for oral surgery; Restorative dentistry included 5 studies; educational and learning 4 studies and orthodontics 1 study (Fig. 3).

figure 3

Fig. 3 Field of interest

The studies considered applied augmented reality technology for the following operations: Implant placing performed on 3D printed mandibular models with better better accuracy applicability and efficiency as outcome: < 1.5 mm as linear deviation and < 5.5 degree of angular deviation by Jiang et al. [16] Lefort 1 has been performed on models by surgical residents with more self-confidence and knowledge as overall resulting experience [5]. A very specific operation like orbital implant placement has been tested out on 3D printed mode which is very useful for the instant feedback and with a translation error of 1.12–1.15 mm and rotational of < 3° [18]. Inferior block nerve anesthesia have been tested on one phantom model with good results using just a camera and a laptop [36]. MASO have been carried on by two different authors, coauthors in one of the manuscripts [26, 37]; they described an increase of time needed of about 1 h of preparation before the surgery on human in their first study. In the second study they don’t report such data on MASO performed on dog mandibles, even by unexperienced operators, both of them landed good results and were judged helpful. Othe authors proposed the use of augmented reality with one of the most sophisticated hardware found in literature which is the Da Vinci si robot in 2015, their experiment involved the resection of a neoplasm on a porcine tongue using vascular landmarks. This is one of the only articles referring a clear failure of the experiment with a mean error of more than 5 mm [19]. Other authors proposed the positioning of distractors for hemifacial microsomia with the use of augmented reality in 2015, for their study they enrolled 10 randomized cases and 10 controls presenting microsomia. The aim was to transfer the surgical planning to the surgical site in hemifacial microsomia elongment using a Head Mounted Display predisposed with the use of Mimics and of the software AR Toolkits. They found the technology useful with difference between the vertical distances from the coronoid to the plane CP1 (AA′) and CP2 (AA′′) of 1.43 ± 0.13 mm in the AR group and 2.53 ± 0.39 mm in the control group [20]. Another interesting study proposed the use of NDI Polaris tracking system to solve the positioning issues related with the use of augmented reality. NDI Polaris is a tracking device which use spherical markers capture by a set of two rapid movement camera. The systems was implemented with self developed software with the use of a head-mounted device as described by the authors and it was used for implant placing in a pig corpse. The outcomes were evaluated through questionnaires which assessed ergonomic benefits and easier procedures, linear and angular error in the positioning were not assessed [39].

Outcomes

13 studies quantified the errors in the superposition of the virtual objects with reality or compared the outcomes with traditional set up, while 6 studies evaluated the changes in time needed for the intervention, a total of 16 studies considered at least one of the two variables as described in (Table 3). All the studies considered the results satisfactory for the quantification of the error/precision except for one but not many considered satisfactory the timing comparisons.

Table 3 Error and timing
Full size table

Authors considered the mean error of the tracking tool for vascular landmarks of the base of the tongue for neoplasm resection by using the Da Vinci si robot of 5 mm not acceptable [19].

Other authors evaluated the maxillary reposition with X-scope prolonged by approximately 1 h, while others considered MASO with computer aided tools needs approximately 1 h of registration before the start but they suggest that it can be improved with experience in the future [26, 27]. some authors in 2013 considered maxillary repositioning using a custom portable device 60 min longer than a conventional operation [22]. All the outcomes are collected in (Table 3).

Research types/design

The design proposed for the selected studies is experimental randomized clinical trial in one of the studies proposed [20], there are 3 Cohort studies [16, 33, 41] and three review studies [30, 38, 43].

Discussion

The first studies taken into account were published in 2005, 38 years after the publishing of the first head-mounted augmented reality system by Sunderland. Even though augmented reality is a specifically visual immersive system, most of the authors are proposing non-wearable display-camera systems. This reduces the efforts related to stabilization of overlapping two different dynamic systems, which is preponderant in head-mounted and portable systems but also reduces the scope of “augmenting” the perception of the operator [32].

The studies considered are rapidly growing from 2013 as can be seen by (Fig. 4) and the most productive state are China and Japan, which also collaborated between each other in different studies, followed by Germany, UK and Belgium.

figure 4

Fig. 4 Year of publication

The majority of the systems refer to OMS area specifically to maxillofacial surgery. Implantology and oral surgery, the two other subgroups, include just 4 studies out of 21 in the OMS, which means that 84% of OMS studies refers specifically to maxillofacial surgery. Educational and learning studies are almost equivalent to restorative dentistry respectively they include 5 and 4 studies. Orthodontics and endodontics are represented with one study each. There is lack of a system studied in different fields, this could be explained with the high customization and knowledge required for every system to adapt to a specific field. Even though some systems share the same hardware [22, 34].

The prevalence of studies in the maxillofacial area can be associated with the extension of the area of intervention. The larger is the subject to be seen in augmented reality the more applicability finds the system. This fact can be associated with contemporary availability of already existing hardware and components used for customized systems. High precision cameras with efficient stabilization and the possibility to zoom in a small area are still very expensive and big in size. Also the landmark of reference are highly influencing the predominant interest in the OMS area, in fact trials carried on using vascular references, even with high precision hardwares, obtained result where the outcomes were considered not satisfactory (mean error more than 5 mm) [19].

This could be a major limitation of this new technology in operations carried on exclusively on soft tissues since the lack of stability represents an obstacle to stabilization of the overlapping images.

Primary endpoints of the studies show general positivity for improvement, usefulness and even good outcomes in the precision of the proposed systems (higher than usual standards in some cases). Educational systems were evaluated through questionnaires and brought great response in the students. While other fields of interest might appear as they are making their first step on the augmented technologies, education seems already available for wider studies since navigation systems were already available with the use of virtual reality, having a low cost [7]. Also a good response is to be expected from young generations which are more prominent to adapt to new technologies. The use of this technology could simplify digital procedures with direct visualization of virtual informations (Fig. 5).

figure 5

Fig. 5 Digital dentistry procedures, conventional vs augmented reality

Timing, although, is more controversial and highly depends on the structure of the system proposed. The timing outcomes are very different between each other, in fact some relates to the setting and calibration time, some other refers to the duration of the intervention and the educational studies refer to the time needed for gaining a given skill in dental training and manual dexterity [17].

The positivity in the outcomes and primary endpoints of the studies considered (31/33) should be taken with caution since many of the systems described are self-developed by the same institutions of the authors.

Custom made software were not used by other authors except the first describing them, which is a major flaw and could represent conflict of interest in validating a new proposed system. Also, there is a lack of randomized clinical trials with a proper sample size calculation and other effort to avoid major bias.

Conclusions

Most recent technologies are being developed with custom software: 7 out of 9 were self-developed by the authors in the last 5 years. More efforts is needed to implement the hardware support. From what is known a simple, portable and accessible tool is needed. Timing is a controversial topic in different fields of interest since half of the authors (3 out of 6) report an increase of at least one hour while precision is judged satisfactory by most authors (12 out of 13).

Although the technologies proposed are not validated by external teams, customized augmented reality systems seems to provide great results in simple experimental models since most of the studies were carried on phantoms (51% n = 17). OMS area is referee of great advantages in interventions carried on medium sized surgical areas and its gaining the most benefits from this technology since superposition of digital images is easier on bony structures. Most of the studies were carried on this augmented reality field of application (21 out of 33).