Skip to main content

Advertisement

SpringerLink
Log in

RETRACTED ARTICLE: DVT: a recent review and a taxonomy for oral and maxillofacial visualization and tracking based augmented reality: image guided surgery

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

This article was retracted on 27 February 2024

A Correction to this article was published on 07 June 2023

This article has been updated

Abstract

Augmented reality (AR) navigation system is increasingly being integrated into Image Guided Surgery systems. The use of AR registration and tracking system in operating rooms (OR) for oral and maxillofacial surgery (OMS) can result in reducing medical errors and decreasing total operation times. As such, AR systems are assisting to reduce some of the surgical complexities associated with OMS, making it easier for the surgeons to view the operation. Although AR systems has been implemented in OR for many years, there are still several factors that are less than optimal and can cause complications, such as inadequate system accuracy, poor image and video quality, high operating time and cost, and significant complexity of the system required to achieve accurate views of the surgical target of OMS. The aim of this research is to improve the use of AR in OMS using the proposed review taxonomy which incorporated Data, Visualization, and Tracking (DVT). DVT taxonomy defines the major components that are required to implement in an AR navigation system. Those components are validated and evaluated considering the clear and accurate output or view during craniofacial surgery for the end user. The proposed DVT taxonomy have been considered comparison of system, completeness of system and acceptance of the system as the major criteria. DVT is evaluated and validated our DTV taxonomy by analysing and classifying the 33 state of art publications which work in the AR navigation. This work presents a review over navigational approach towards surgery through AR which highlights the features and usefulness of AR compared to the existing surgical process in terms of processing time, accuracy, efficiency, and feasibility in surgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Change history

References

  1. Badiali G, Ferrari V, Cutolo F, Freschi C, Caramella D, Bianchi A, Marchetti C (2014) Augmented reality as an aid in maxillofacial surgery: Validation of a wearable system allowing maxillary repositioning. J Cranio-Maxillofac Surg 42(8):1970–1976. https://doi.org/10.1016/j.jcms.2014.09.001

    Article  Google Scholar 

  2. Barber R, Fitoussi A, Hersant B, Dao TH, Meningaud JP (2018) Intraoperative augmented reality with heads-up displays in maxillofacial surgery: a systematic review of the literature and a classification of relevant technologies. Int J Oral Maxillofac Surg 48(1):132–139. https://doi.org/10.1016/j.ijom.2018.09.010

    Article  Google Scholar 

  3. Basnet BR, Alsadoon A, Withana C, Deva A, Paul M (2018) A novel noise filtered and occlusion removal: navigational accuracy in augmented reality-based constructive jaw surgery. Oral Maxillofac Surg 22(4):385–401. https://doi.org/10.1007/s10006-018-0719-5

    Article  PubMed  Google Scholar 

  4. Bong JH, Song H, Oh Y, Park N, Kim H, Park S (2018) Endoscopic navigation system with extended field of view using augmented reality technology. Int J Med Robot Comput Assist Surg 14(2). https://doi.org/10.1002/rcs.1886

  5. Bosc R, Fitoussi A, Hersant B, Dao TH, Meningaud JP (2019) Intraoperative augmented reality with heads-up displays in maxillofacial surgery: a systematic review of the literature and a classification of relevant technologies. Int J Oral Maxillofac Surg 48(1):132–139. https://doi.org/10.1016/j.ijom.2018.09.010

    Article  CAS  PubMed  Google Scholar 

  6. Chen X, Xu L, Wang Y, Wang H, Wang F, Zeng X et al (2015) Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. J Biomed Inform 55:124–131. https://doi.org/10.1016/j.jbi.2015.04.003

    Article  PubMed  Google Scholar 

  7. Chen X, Xu L, Wang Y, Hao Y, Wang L (2016) Image-guided installation of 3D-printed patient-specific implant and its application in pelvic tumor resection and reconstruction surgery. Comput Methods Programs Biomed 125:66–78. https://doi.org/10.1016/j.cmpb.2015.10.020

    Article  PubMed  Google Scholar 

  8. Chu Y, Yang J, Ma S, Ai D, Li W, Song H et al (2017) Registration and fusion quantification of augmented reality based nasal endoscopic surgery. Med Image Anal 42:241–256. https://doi.org/10.1016/j.media.2017.08.003

    Article  PubMed  Google Scholar 

  9. Crafts TD, Ellsperman SE, Wannemuehler TJ, Bellicchi TD, Shipchandler TZ, Mantravadi AV (2017) Three-dimensional printing and its applications in otorhinolaryngology-head and neck surgery. Otolaryngol Head Neck Surg 156(6):999–1010. https://doi.org/10.1177/0194599816678372

  10. Cutolo F, Badiali G, Ferrari V (2015) Human-PnP: Ergonomic AR interaction paradigm for manual placement of rigid bodies. In: Linte C, Yaniv Z, Fallavollita P (eds) Augmented Environments for Computer-Assisted Interventions. AE-CAI 2015. Lecture Notes in Computer Science, 9365. Springer, Cham. https://doi.org/10.1007/978-3-319-24601-7_6

  11. Dai J, Wu J, Wang X et al (2016) An excellent navigation system and experience in craniomaxillofacial navigation surgery: a double-center study. Sci Rep 6:28242. https://doi.org/10.1038/srep28242

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  12. Diotte B, Fallavollita P, Wang L, Weidert S, Euler E, Thaller P, Navab N (2015) Multi-modal intra-operative navigation during distal locking of intramedullary nails. IEEE Trans Med Imaging 34(2):487–495. https://doi.org/10.1109/tmi.2014.2361155

    Article  PubMed  Google Scholar 

  13. Dixon BJ, Daly MJ, Chan H et al (2013) Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc 27:454–461. https://doi.org/10.1007/s00464-012-2457-3

    Article  PubMed  Google Scholar 

  14. Dixon BJ, Daly MJ, Chan HH, Vescan A, Witterick IJ, Irish JC (2014) Inattentional blindness increased with augmented reality surgical navigation. Am J Rhinol Allergy 28(5):433–437

    Article  PubMed  Google Scholar 

  15. Edgerton MT, Marsh JL (1977) Surgical treatment of hemifacial microsomia (first and second branchial arch syndrome). Plast Reconstr Surg 59(5):653–666. https://doi.org/10.1097/00006534-197705000-00006

  16. Hassfeld S, Mühling J (2001) Computer assisted oral and maxillofacial surgery – a review and an assessment of technology. Int J Oral Maxillofac Surg 30(1):2–13. https://doi.org/10.1054/ijom.2000.0024

    Article  CAS  PubMed  Google Scholar 

  17. Hayashi Y, Misawa K, Oda M, Hawkes D, Mori K (2015) Clinical application of a surgical navigation system based on virtual laparoscopy in laparoscopic gastrectomy for gastric cancer. Int J Comput Assist Radiol Surg 11(5):827–836. https://doi.org/10.1007/s11548-015-1293-z

    Article  PubMed  Google Scholar 

  18. He C, Liu Y, Wang Y (2016) Sensor-fusion based augmented-reality surgical navigation system. 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings, Taipei, 1–5. https://doi.org/10.1109/I2MTC.2016.7520404

  19. Hung K, Wang F, Wang H, Zhou W, Huang W, Wu Y (2017) Accuracy of a real-time surgical navigation system for the placement of quad zygomatic implants in the severe atrophic maxilla: A pilot clinical study. Clin Implant Dent Relat Res 19(3):458–465. https://doi.org/10.1111/cid.12475

    Article  PubMed  Google Scholar 

  20. Kilgus T, Heim E, Haase S, Prüfer S, Müller M, Seitel A, Maier-Hein L et al (2015) Mobile markerless augmented reality and its application in forensic medicine. Int J Comput Assist Radiol Surg 10(5):573–586. https://doi.org/10.1007/s11548-014-1106-9

    Article  PubMed  Google Scholar 

  21. Kim GW, Bae YC, Bae SH, Nam SB, Lee DM (2018) A clinical review of reconstructive techniques for patients with multiple skin cancers on the face. Arch Craniofac Surg 19(3):194–199. https://doi.org/10.7181/acfs.2018.02012

  22. Liao H, Inomata T, Sakuma I, Dohi T (2010) 3-D augmented reality for MRI-guided surgery using integral videography autostereoscopic image overlay. IEEE Trans Biomed Eng 57(6):1476–1486. https://doi.org/10.1109/TBME.2010.2040278

    Article  PubMed  Google Scholar 

  23. Lin L, Shi Y, Tan A, Bogari M, Zhu M, Xin Y, Chai G (2016) Mandibular angle split osteotomy based on a novel augmented reality navigation using specialized robot-assisted arms—A feasibility study. J Cranio-Maxillofac Surg 44(2):215–223. https://doi.org/10.1016/j.jcms.2015.10.024

    Article  Google Scholar 

  24. Lin Y-K, Yau H-T, Wang IC, Zheng C, Chung K-H (2015) A novel dental implant guided surgery based on integration of surgical template and augmented reality. Clin Implant Dent Relat Res 17(3):543–553. https://doi.org/10.1111/cid.12119

    Article  PubMed  Google Scholar 

  25. Lin PT, Kartikeya S, Tom H, Harsha B, Ding-Yu F, Ou B (2013) A high performance MEG based BCI using single trial detection of human movement intention, functional brain mapping and the endeavor to understand the working brain, Dr. Francesco Signorelli (Ed.), InTech. https://doi.org/10.5772/54550. Available from: https://www.intechopen.com/books/functional-brain-mapping-and-the-endeavor-to-understand-the-working-brain/a-high-performance-meg-based-bci-using-single-trial-detection-of-human-movement-intention

  26. Luther N, Iorgulescu JB, Geannette C, Gebhard H, Saleh T, Tsiouris AJ, Härtl R (2015) Comparison of navigated versus non-navigated pedicle screw placement in 260 patients and 1434 screws. J Spinal Disord Tech 28(5):E298–E303. https://doi.org/10.1097/bsd.0b013e31828af33e

    Article  PubMed  Google Scholar 

  27. Ma L, Zhao Z, Chen F, Zhang B, Fu L, Liao H (2017) Augmented reality surgical navigation with ultrasound-assisted registration for pedicle screw placement: A pilot study. Int J Comput Assist Radiol Surg 12(12):2205–2215. https://doi.org/10.1007/s11548-017-1652-z

    Article  PubMed  Google Scholar 

  28. Ma L, Zhao Z, Zhang B, Jiang W, Fu L, Zhang X, et al (2018) Three-dimensional augmented reality surgical navigation with hybrid optical and electromagnetic tracking for distal intramedullary nail interlocking. Int J Med Robot 14:e1909. https://doi.org/10.1002/rcs.1909

  29. Ma L, Jiang W, Zhang B, Qu X, Ning G, Zhang X, Liao H (2019) Augmented reality surgical navigation with accurate CBCT-patient registration for dental implant placement. Med Biol Eng Comput 57(1):47–57. https://doi.org/10.1007/s11517-018-1861-9

    Article  PubMed  Google Scholar 

  30. Mahmoud N, Grasa Ó, Nicolau S, Doignon C, Soler L, Marescaux J, Montiel J (2016) On-patient see-through augmented reality based on visual SLAM. Int J Comput Assist Radiol Surg 12(1):1–11. https://doi.org/10.1007/s11548-016-1444-x

    Article  PubMed  Google Scholar 

  31. Meulstee JW, Nijsink J, Schreurs R et al (2019) Toward holographic-guided surgery. Surg Innov 26(1):86–94. https://doi.org/10.1177/1553350618799552

    Article  PubMed  Google Scholar 

  32. Murugesan YP, Alsadoon A, Manoranjan P, Prasad PWC (2018) A novel rotational matrix and translation vector algorithm: geometric accuracy for augmented reality in oral and maxillofacial surgeries. Int J Med Robot Comput Assist Surg 14(3):e1889. https://doi.org/10.1002/rcs.1889

    Article  Google Scholar 

  33. Nakao M, Endo S, Nakao S, Yoshida M, Matsuda T (2016) Augmented endoscopic images overlaying shape changes in bone cutting procedures. PLoS ONE 11(9):e0161815. https://doi.org/10.1371/journal.pone.0161815

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Pokhrel S, Alsadoon A, Prasad PWC, Paul M (2018) A novel augmented reality (AR) scheme for knee replacement surgery by considering cutting error accuracy. Int J Med Robot Comput Assist Surg. https://doi.org/10.1002/rcs.1958

    Article  Google Scholar 

  35. Profeta AC, Schilling C, Mcgurk M (2016) Augmented reality visualization in head and neck surgery: an overview of recent findings in sentinel node biopsy and future perspectives. Br J Oral Maxillofac Surg 54(6):694–696. https://doi.org/10.1016/j.bjoms.2015.11.008

    Article  PubMed  Google Scholar 

  36. Qu M, Hou Y, Xu Y, Shen C, Zhu M, Xie L, Chai G (2015) Precise positioning of an intraoral distractor using augmented reality in patients with hemifacial microsomia. J Cranio-Maxillofac Surg 43(1):106–112. https://doi.org/10.1016/j.jcms.2014.10.019

    Article  Google Scholar 

  37. Reichard D, Häntsch D, Bodenstedt S, Suwelack S, Wagner M, Kenngott H, Speidel S et al (2017) Projective biomechanical depth matching for soft tissue registration in laparoscopic surgery. Int J Comput Assist Radiol Surg 12(7):1101–1110. https://doi.org/10.1007/s11548-017-1613-6

    Article  PubMed  Google Scholar 

  38. Shi Y, Lin L, Zhou C, Zhu M, Xie L, Chai G (2017) A study of an assisting robot for mandible plastic surgery based on augmented reality. Minim Invasive Ther Allied Technol 26(1):23–30. https://doi.org/10.1080/13645706.2016.1216864

    Article  PubMed  Google Scholar 

  39. Suenaga H, Hoang Tran H, Liao H, Masamune K, Dohi T, Hoshi K, Takato T (2013) Real-time in situ three-dimensional integral videography and surgical navigation using augmented reality: a pilot study. Int J Oral Sci 5(98). https://doi.org/10.1038/ijos.2013.26, https://www.nature.com/articles/ijos201326#supplementary-information

  40. Suenaga H, Tran H, Liao H, Masamune K, Dohi T, Hoshi K, Takato T (2015) Vision-based markerless registration using stereo vision and an augmented reality surgical navigation system: a pilot study. BMC Med Imaging 15(1). https://doi.org/10.1186/s12880-015-0089-5

  41. Tuladhar S, AlSallami N, Alsadoon A, et al (2020) A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality [published online ahead of print, 2020 May 9]. Int J Med Robot :e2120. https://doi.org/10.1002/rcs.2120

  42. Valenti M, Ferrigno G, Martina D, Yu W, Zheng G, Shandiz M, Momi ED et al (2016) Gaussian mixture models based 2D–3D registration of bone shapes for orthopedic surgery planning. Med Biol Eng Compu 54(11):1727–1740. https://doi.org/10.1007/s11517-016-1460-6

    Article  Google Scholar 

  43. Wang J, Suenaga H, Hoshi K, Yang L, Kobayashi E, Sakuma I, Liao H (2014) Augmented reality navigation with automatic marker-free image registration using 3-D image overlay for dental surgery. IEEE Trans Biomed Eng 61(4):1295–1304. https://doi.org/10.1109/TBME.2014.2301191

    Article  PubMed  Google Scholar 

  44. Wang J, Suenaga H, Yang L, Kobayashi E, Sakuma I (2017) Video see-through augmented reality for oral and maxillofacial surgery. Int J Med Robot Comput Assist Surg 13(2):e1754. https://doi.org/10.1002/rcs.1754

    Article  Google Scholar 

  45. Wang J, Suenaga H, Yang L, Kobayashi E, Sakuma I (2016) Video see-through augmented reality for oral and maxillofacial surgery. Int J Med Robot Comput Assist Surg. https://doi.org/10.1002/rcs.1754

    Article  Google Scholar 

  46. Wang J, Suenaga H, Liao H, Hoshi K, Yang L, Kobayashi E, Sakuma I (2015) Real-time computer-generated integral imaging and 3D image calibration for augmented reality surgical navigation. Comput Med Imaging Graph 40:147–159. https://doi.org/10.1016/j.compmedimag.2014.11.003

    Article  PubMed  Google Scholar 

  47. Wild E, Teber D, Schmid D, Simpfendörfer T, Müller M, Baranski A, Maier-Hein L et al (2016) Robust augmented reality guidance with fluorescent markers in laparoscopic surgery. Int J Comput Assist Radiol Surg 11(6):899–907. https://doi.org/10.1007/s11548-016-1385-4

    Article  PubMed  Google Scholar 

  48. Xiao Y, Drouin S, Gerard IJ et al (2018) An augmented-reality system prototype for guiding transcranial Doppler ultrasound examination. Multimed Tools Appl 77:27789–27805. https://doi.org/10.1007/s11042-018-5990-9

    Article  Google Scholar 

  49. Zhang X, Chen G, Liao H (2017) High-quality see-through surgical guidance system using enhanced 3-D autostereoscopic augmented reality. IEEE Trans Biomed Eng 64(8):1815–1825. https://doi.org/10.1109/TBME.2016.2624632

    Article  ADS  PubMed  Google Scholar 

  50. Zhu M, Liu F, Chai G, Pan J, Jiang T, Lin L, Li Q (2017) A novel augmented reality system for displaying inferior alveolar nerve bundles in maxillofacial surgery. Sci Rep 7:42365. https://doi.org/10.1038/srep42365

    Article  ADS  CAS  PubMed  PubMed Central  Google Scholar 

  51. Zhu M, Liu F, Zhou C, Lin L, Zhang Y, Chai G, Li Q (2018) Does intraoperative navigation improve the accuracy of mandibular angle osteotomy: Comparison between augmented reality navigation, individualised templates and free-hand techniques. J Plast Reconstr Aesthet Surg 71(8):1188–1195. https://doi.org/10.1016/j.bjps.2018.03.018

    Article  PubMed  Google Scholar 

  52. Zhu M, Chai G, Zhang Y, Ma X, Gan J (2011) Registration strategy using occlusal splint based on augmented reality for mandibular angle oblique split osteotomy. J Craniofac Surg 22(5):1806–1809. https://doi.org/10.1097/SCS.0b013e31822e8064

    Article  PubMed  Google Scholar 

  53. Zinser MJ, Mischkowski RA, Dreiseidler T, Thamm OC, Rothamel D, Zöller JE (2013) Computer-assisted orthognathic surgery: waferless maxillary positioning, versatility, and accuracy of an image-guided visualisation display. Br J Oral Maxillofac Surg 51(8):827–833. https://doi.org/10.1016/j.bjoms.2013.06.014

    Article  PubMed  Google Scholar 

  54. Zorina ZA (2005) Animal intelligence: Laboratory experiments and observations in nature. (in Russian). Zoologichesky Zhurnal 84(1):134–148

    Google Scholar 

Download references

Acknowledgment

The authors would like to thank Amit Nepal, Shakti Shrestha, and Hazel Singh Shrestha in participating them of collecting some information related to this paper.

Author information

Authors and Affiliations

  1. School of Computing Mathematics and Engineering, Charles Sturt University (CSU), Albury-Wodonga, Australia

    Abeer Alsadoon, Jeffrey J. Gosper & P. W. C. Prasad

  2. School of Computer Data and Mathematical Sciences, Western Sydney University (WSU), Sydney, Australia

    Abeer Alsadoon

  3. Asia Pacific International College (APIC), Sydney, Australia

    Abeer Alsadoon

  4. Computer Science Department, Worcester State University (WSU), Worcester, MA, USA

    Nada AlSallami

  5. Computer Science and Engineering, University of Kurdistan Hewler, Erbil, KRG, Iraq

    Tarik A. Rashid

  6. Department of Oral and Maxillofacial Services, Greater Western Sydney Area Health Services, Mount Druitt, Australia

    Sami Haddad

  7. Department of Oral and Maxillofacial Services, Central Coast Area Health, Gosford, Australia

    Sami Haddad

Authors
  1. Abeer Alsadoon
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nada AlSallami
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tarik A. Rashid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jeffrey J. Gosper
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. P. W. C. Prasad
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sami Haddad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abeer Alsadoon.

Ethics declarations

No Funding for this work and no Conflicts of interests as well.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article has been retracted. Please see the retraction notice for more detail: https://doi.org/10.1007/s11042-024-18683-1"

The original online version of this article was revised: The author name "Jeffrey J. Gosper" was misspelled as "Jeff Gosper" in the original publication of this article.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alsadoon, A., AlSallami, N., Rashid, T.A. et al. RETRACTED ARTICLE: DVT: a recent review and a taxonomy for oral and maxillofacial visualization and tracking based augmented reality: image guided surgery. Multimed Tools Appl 83, 685–729 (2024). https://doi.org/10.1007/s11042-023-15581-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15581-w

Keywords

Search

Navigation