Skip to main content
SpringerLink
Log in

Twin-S: a digital twin for skull base surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose:

Digital twins are virtual replicas of real-world objects and processes, and they have potential applications in the field of surgical procedures, such as enhancing situational awareness. We introduce Twin-S, a digital twin framework designed specifically for skull base surgeries.

Methods:

Twin-S is a novel framework that combines high-precision optical tracking and real-time simulation, making it possible to integrate it into image-guided interventions. To guarantee accurate representation, Twin-S employs calibration routines to ensure that the virtual model precisely reflects all real-world processes. Twin-S models and tracks key elements of skull base surgery, including surgical tools, patient anatomy, and surgical cameras. Importantly, Twin-S mirrors real-world drilling and updates the virtual model at frame rate of 28.

Results:

Our evaluation of Twin-S demonstrates its accuracy, with an average error of 1.39 mm during the drilling process. Our study also highlights the benefits of Twin-S, such as its ability to provide augmented surgical views derived from the continuously updated virtual model, thus offering additional situational awareness to the surgeon.

Conclusion:

We present Twin-S, a digital twin environment for skull base surgery. Twin-S captures the real-world surgical progresses and updates the virtual model in real time through the use of modern tracking technologies. Future research that integrates vision-based techniques could further increase the accuracy of Twin-S.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://www.atracsys-measurement.com/products/fusiontrack-500/.

  2. https://www.jnjmedtech.com/en-EMEA/product/anspach-eg1-electric-system.

  3. https://www.brainlab.com/loop-x.

References

  1. Inc I (2022) What is a digital twin?. IBM. https://www.ibm.com/topics/what-is-a-digital-twin. Accessed 13 Aug 2022

  2. Jones D, Snider C, Nassehi A, Yon J, Hicks B (2020) Characterising the digital twin: a systematic literature review. CIRP J Manuf Sci Technol 29:36–52

    Article  Google Scholar 

  3. Martinez-Velazquez R, Gamez R, El Saddik A (2019) Cardio twin: a digital twin of the human heart running on the edge. In: 2019 IEEE international symposium on medical measurements and applications (MeMeA), pp 1–6. IEEE

  4. Breton MD, Kanapka LG, Beck RW, Ekhlaspour L, Forlenza GP, Cengiz E, Schoelwer M, Ruedy KJ, Jost E, Carria L et al (2020) A randomized trial of closed-loop control in children with type 1 diabetes. N Engl J Med 383(9):836–845

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Björnsson B, Borrebaeck C, Elander N, Gasslander T, Gawel DR, Gustafsson M, Jörnsten R, Lee EJ, Li X, Lilja S et al (2020) Digital twins to personalize medicine. Genome Med 12:1–4

    Article  Google Scholar 

  6. Laubenbacher R, Niarakis A, Helikar T, An G, Shapiro B, Malik-Sheriff R, Sego T, Knapp A, Macklin P, Glazier J (2022) Building digital twins of the human immune system: toward a roadmap. NPJ Digit Med 5(1):64

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Coelho G, Rabelo NN, Vieira E, Mendes K, Zagatto G, de Oliveira RS, Raposo-Amaral CE, Yoshida M, de Souza MR, Fagundes CF et al (2020) Augmented reality and physical hybrid model simulation for preoperative planning of metopic craniosynostosis surgery. Neurosurg Focus 48(3):19

    Article  Google Scholar 

  8. Chalasani P, Wang L, Roy R, Simaan N, Taylor RH, Kobilarov M (2016) Concurrent nonparametric estimation of organ geometry and tissue stiffness using continuous adaptive palpation. In: Proc. ICRA

  9. Wang L, Chen Z, Chalasani P, Yasin RM, Kazanzides P, Taylor RH, Simaan N (2017) Force-controlled exploration for updating virtual fixture geometry in model-mediated telemanipulation. J Mech Robot 9(2):021010

    Article  Google Scholar 

  10. Yasin R, Chalasani P, Zevallos N, Shahbazi M, Li Z, Deguet A, Kazanzides P, Choset H, Taylor RH, Simaan N (2020) Evaluation of hybrid control and palpation assistance for situational awareness in telemanipulated task execution. IEEE Trans Med Robot Bionics 3(1):31–43

    Article  Google Scholar 

  11. Taylor RH, Mittelstadt BD, Paul HA, Hanson W, Kazanzides P, Zuhars JF, Williamson B, Musits BL, Glassman E, Bargar WL (1994) An image-directed robotic system for precise orthopaedic surgery. IEEE Trans Robot Autom 10(3):261–275

    Article  Google Scholar 

  12. Kazanzides P (1999) Robot assisted surgery: the robodoc® experience. In: International symposium on robotics

  13. Chalasani P, Wang L, Yasin R, Simaan N, Taylor RH (2018) Preliminary evaluation of an online estimation method for organ geometry and tissue stiffness. IEEE Robot Autom Lett 3(3):1816–1823

    Article  Google Scholar 

  14. Yasin R, Chalasani P, Zevallos N, Shahbazi M, Li Z, Deguet A, Kazanzides P, Choset H, Taylor RH, Simaan N (2021) Evaluation of hybrid control and palpation assistance for situational awareness in telemanipulated task execution. TMRB

  15. Shi Y, Deng X, Tong Y, Li R, Zhang Y, Ren L, Si W (2022) Synergistic digital twin and holographic augmented-reality-guided percutaneous puncture of respiratory liver tumor. IEEE Trans Human Machine Syst 52(6):1364–1374

    Article  Google Scholar 

  16. Laaki H, Miche Y, Tammi K (2019) Prototyping a digital twin for real time remote control over mobile networks: application of remote surgery. IEEE Access 7:20325–20336

    Article  Google Scholar 

  17. Bonne S, Panitch W, Dharmarajan K, Srinivas K, Kincade J-L, Low T, Knoth B, Cowan C, Fer D, Thananjeyan B et al (2022) A digital twin framework for telesurgery in the presence of varying network quality of service. In: 2022 IEEE 18th international conference on automation science and engineering (CASE), pp 1325–1332. IEEE

  18. Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X (2021) A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Dev 18(1):47–62

    Article  CAS  Google Scholar 

  19. Aguilar-Salinas P, Gutierrez-Aguirre SF, Avila MJ, Nakaji P (2022) Current status of augmented reality in cerebrovascular surgery: a systematic review. Neurosurg Rev 45(3):1951–1964

    Article  PubMed  Google Scholar 

  20. Kockro RA, Tsai YT, Ng I, Hwang P, Zhu C, Agusanto K, Hong LX, Serra L (2009) Dex-ray: augmented reality neurosurgical navigation with a handheld video probe. Neurosurgery 65(4):795–808

    Article  PubMed  Google Scholar 

  21. Agha RA, Fowler AJ (2015) The role and validity of surgical simulation. Int Surg 100(2):350–357

    Article  PubMed  PubMed Central  Google Scholar 

  22. Munawar A, Li Z, Kunjam P, Nagururu N, Ding AS, Kazanzides P, Looi T, Creighton FX, Taylor RH, Unberath M (2022) Virtual reality for synergistic surgical training and data generation. Comput Methods Biomech Biomed Eng Imag Vis 10(4):366–374

    Article  Google Scholar 

  23. Munawar A, Li Z, Nagururu N, Trakimas D, Kazanzides P, Taylor RH, Creighton FX (2023) Fully immersive virtual reality for skull-base surgery: surgical training and beyond. arXiv preprint arXiv:2302.13878

  24. Razavi CR, Wilkening PR, Yin R, Barber SR, Taylor RH, Carey JP, Creighton FX (2019) Image-guided mastoidectomy with a cooperatively controlled ent microsurgery robot. Otolaryngol Head Neck Surg 161(5):852–855

    Article  PubMed  Google Scholar 

  25. Yaniv Z (2015) Which pivot calibration? In: Medical imaging 2015: imageguided procedures, robotic interventions, and modeling, vol 9415, pp 542–550. SPIE

  26. Kabsch W (1976) A solution for the best rotation to relate two sets of vectors. Acta Crystallogr Sect A Crystal Phys Diffract Theoretical General Crystallogr 32(5):922–923

  27. Feng L, Wilkening P, Sevimli Y, Balicki M, Olds KC, Taylor RH (2016) Accuracy assessment and kinematic calibration of the robotic endoscopic microsurgical system. In: 2016 38th EMBC, pp. 5091–5094. IEEE

  28. Schenker PS (1992) Sensor fusion IV: control paradigms and data structures; proceedings of the meeting, In: SPIE proceedings. Vol. 1611, Nov. 12–15, 1991. Technical report, Society of photo-optical instrumentation engineers, Boston, MA

  29. Zhang Z (2000) A flexible new technique for camera calibration. IEEE Transact Pattern Anal Machine Intell 22(11), 1330–1334

  30. Horaud R, Dornaika F (1995) Hand-eye calibration. Int J Robot Res 14(3), 195–210

  31. Furrer F, Fehr M, Novkovic T, Sommer H, Gilitschenski I, Siegwart R (2018) Evaluation of combined time-offset estimation and hand-eye calibration on robotic datasets. In: Field and service robotics results of the 11th international conference, pp 145–159. Springer

  32. Holland MT, Mansfield K, Mitchell A, Burchiel KJ (2021) Hidden error in optical stereotactic navigation systems and strategy to maximize accuracy. Stereotact Funct Neurosurg 99(5):369-376

    Article  PubMed  Google Scholar 

  33. Kennedy KL, L.J.: Mastoidectomy. https://www.ncbi.nlm.nih.gov/books/NBK559153. Accessed 5 Feb 2023

  34. Milletari F, Navab N, Ahmadi S-A (2016) V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 4th International conference on 3D vision (3DV), pp 565–571

  35. Li Z, Shu H, Liang R, Goodridge A, Sahu M, Creighton FX, Taylor RH, Unberath M (2022) Tatoo: vision-based joint tracking of anatomy and tool for skull-base surgery. arXiv preprint arXiv:2212.14131

  36. Li Z, Liu X, Drenkow N, Ding A, Creighton FX, Taylor RH, Unberath M (2021) Revisiting stereo depth estimation from a sequenceto-sequence perspective with transformers. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 6197–6206

  37. Li Z, Drenkow N, Ding H, Ding AS, Lu A, Creighton FX, Taylor RH, Unberath M (2021) On the sins of image synthesis loss for self-supervised depth estimation. arXiv preprint arXiv:2109.06163

  38. Li Z, Gordon A, Looi T, Drake J, Forrest C, Taylor RH (2020) Anatomical mesh-based virtual fixtures for surgical robots. In: 2020 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3267–3273. IEEE

Download references

Acknowledgements

This work was supported in part by Johns Hopkins University internal funds, an agreement between Johns Hopkins University and the Multi-Scale Medical Robotics Centre Ltd., and in part by NIDCD K08 Grant DC019708.

Author information

Author notes

  1. Hongchao Shu, Ruixing Liang and Zhaoshuo Li are joint first authors.

Authors and Affiliations

  1. Johns Hopkins University, Baltimore, MD, USA

    Hongchao Shu, Ruixing Liang, Zhaoshuo Li, Anna Goodridge, Xiangyu Zhang, Hao Ding, Manish Sahu, Russell H. Taylor, Adnan Munawar & Mathias Unberath

  2. Johns Hopkins Medicine, Baltimore, MD, USA

    Ruixing Liang, Nimesh Nagururu & Francis X. Creighton

Authors
  1. Hongchao Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruixing Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaoshuo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Anna Goodridge
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiangyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nimesh Nagururu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Manish Sahu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Francis X. Creighton
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Russell H. Taylor
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Adnan Munawar
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Mathias Unberath
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongchao Shu.

Ethics declarations

Conflict of interest

Russell Taylor and Johns Hopkins University (JHU) may be entitled to royalty payments related to technology discussed in this paper, and Dr. Taylor has received or may receive some portion of these royalties. Also, Dr. Taylor is a paid consultant to and owns equity in Galen Robotics, Inc. These arrangements have been reviewed and approved by JHU in accordance with its conflict of interest policy.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 1270 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shu, H., Liang, R., Li, Z. et al. Twin-S: a digital twin for skull base surgery. Int J CARS 18, 1077–1084 (2023). https://doi.org/10.1007/s11548-023-02863-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-02863-9

Keywords

Search

Navigation