Skip to main content
SpringerLink
Log in

Augmented Reality Imaging for Robot-Assisted Partial Nephrectomy Surgery

  • Conference paper
  • First Online:
Book cover Medical Imaging and Augmented Reality (MIAR 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9805))

Included in the following conference series:

Abstract

Laparoscopic partial nephrectomy (LPN) is a standard of care for small kidney cancer tumours. A successful LPN is the complete excision of the kidney tumour while preserving as much of the non-cancerous kidney as possible. This is a challenging procedure because the surgeon has a limited field of view and reduced or no haptic feedback while performing delicate excisions as fast as possible. This work introduces and evaluates a novel surgical navigation marker called the Dynamic Augmented Reality Tracker (DART). The DART is used in a novel intra-operative augmented reality ultrasound navigation system (ARUNS) for robot-assisted minimally invasive surgery to overcome some of these challenges. The DART is inserted into a kidney and the DART and pick-up laparoscopic ultrasound transducer are tracked during an intra-operative freehand ultrasound scan of the tumour. After ultrasound, the system continues to track the DART and display the segmented 3D tumour and location of surgical instruments relative to the tumour throughout the surgery. The ultrasound point reconstruction root mean squared error (RMSE) was 0.9 mm, the RMSE of tracking the da Vinci surgical instruments was 1.5 mm and the total system RMSE, which includes ultrasound imaging and da Vinci kinematic instrument tracking, was 5.1 mm. The system was evaluated by an expert surgeon who used the DART and ARUNS to excise a tumour from a kidney phantom. This work serves as a preliminary evaluation in anticipation of further refinement and validation in vivo.

P. Edgcumbe and R. Singla—These authors are both first authors and have contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Elhage, O., Murphy, D., Challacombe, B., Shortland, A., Dasgupta, P.: Ergonomics in minimally invasive surgery. Int. J. Clin. Pract. 61(2), 186–188 (2007)

    Article  Google Scholar 

  2. Sutherland, S.E., Resnick, M.I., Maclennan, G.T., Goldman, H.B.: Does the size of the surgical margin in partial nephrectomy for renal cell cancer really matter? J. Urol. 167(1), 61–64 (2002)

    Article  Google Scholar 

  3. Gill, I.S., Desai, M.M., Kaouk, J.H., Meraney, A.M., Murphy, D.P., Sung, G.T., Novick, A.C.: Laparoscopic partial nephrectomy for renal tumor: duplicating open surgical techniques. J. Urol. 167(2), 469–476 (2002)

    Article  Google Scholar 

  4. Hughes-Hallett, A., Mayer, E.K., Pratt, P., Mottrie, A., Darzi, A., Vale, J.: The current and future use of imaging in urological robotic surgery: a survey of the European Association of Robotic Urological Surgeons. Int. J. Med. Robot. Comput. Assist. Surg. 11(1), 8–14 (2015)

    Article  Google Scholar 

  5. Våpenstad, C., Rethy, A., Langø, T., Selbekk, T., Ystgaard, B., Hernes, T.A.N., Mårvik, R.: Laparoscopic ultrasound: a survey of its current and future use, requirements, and integration with navigation technology. Surg. Endosc. 24(12), 2944–2953 (2010)

    Article  Google Scholar 

  6. Su, L.M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4), 896–900 (2009)

    Article  Google Scholar 

  7. Schneider, C.M., Dachs II, G.W., Hasser, C.J., Choti, M.A., DiMaio, S.P., Taylor, R.H.: Robot-assisted laparoscopic ultrasound. In: Navab, N., Jannin, P. (eds.) IPCAI 2010. LNCS, vol. 6135, pp. 67–80. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Langø, T., Vijayan, S., Rethy, A., Våpenstad, C., Solberg, O.V., Mårvik, R., Johnsen, G., Hernes, T.N.: Navigated laparoscopic ultrasound in abdominal soft tissue surgery: technological overview and perspectives. Int. J. Comput. Assist. Radiol. Surg. 7(4), 585–599 (2012)

    Article  Google Scholar 

  9. Hughes-Hallett, A., Mayer, E.K., Marcus, H.J., Cundy, T.P., Pratt, P.J., Darzi, A.W.: Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 83(2), 266–273 (2014)

    Article  Google Scholar 

  10. Cheung, C.L., Wedlake, C., Moore, J., Pautler, S.E., Peters, T.M.: Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010. LNCS, vol. 6363, pp. 408–415. Springer, Heidelberg (2010)

    Google Scholar 

  11. Pratt, P., Di Marco, A., Payne, C., Darzi, A., Yang, G.Z.: Intraoperative ultrasound guidance for transanal endoscopic microsurgery. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2010. LNCS, vol. 7510, pp. 463–470. Springer, Heidelberg (2012)

    Google Scholar 

  12. Pratt, P., Jaeger, A., Hughes-Hallett, A., Mayer, E., Vale, J., Darzi, A., Peters, T., Yang, G.Z.: Robust ultrasound probe tracking: initial clinical experiences during robot-assisted partial nephrectomy. Int. J. Comput. Assist. Radiol. Surg. 10(12), 1905–1913 (2015)

    Article  Google Scholar 

  13. Hughes-Hallett, A., Pratt, P., Dilley, J., Vale, J., Darzi, A., Mayer, E.: Augmented reality: 3D image-guided surgery. Cancer Imaging 15(Suppl. 1), O8 (2015)

    Article  Google Scholar 

  14. Teber, D., Guven, S., Simpfendörfer, T., Baumhauer, M., Güven, E.O., Yencilek, F., Gözen, A.S., Rassweiler, J.: Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur. Urol. 56(2), 332–338 (2009)

    Article  Google Scholar 

  15. Schneider, C., Guerrero, J., Nguan, C., Rohling, R., Salcudean, S.: Intra-operative “Pick-Up” ultrasound for robot assisted surgery with vessel extraction and registration: a feasibility study. In: Taylor, R.H., Yang, G.-Z. (eds.) IPCAI 2011. LNCS, vol. 6689, pp. 122–132. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  16. Yushkevich, P.A., Piven, J., Hazlett, H.C., Smith, R.G., Ho, S., Gee, J.C., Gerig, G.: User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage 31(3), 1116–1128 (2006)

    Article  Google Scholar 

  17. Geuzaine, C., Remacle, J.F.: Gmsh: a 3-D finite element mesh generator with built-in pre- and post-processing facilities. Int. J. Numer. Methods Eng. 79(11), 1309–1331 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  18. Grenier, N., Gennisson, J.L., Cornelis, F., Le Bras, Y., Couzi, L.: Renal ultrasound elastography. Diagn. Interv. Imaging 94(5), 545–550 (2013)

    Article  Google Scholar 

  19. Bouguet, J.Y.: Camera calibration toolbox for matlab (2004)

    Google Scholar 

  20. Kwartowitz, D.M., Herrell, S.D., Galloway, R.L.: Toward image-guided robotic surgery: determining intrinsic accuracy of the da Vinci robot. Int. J. Comput. Assist. Radiol. Surg. 1(3), 157–165 (2006)

    Article  Google Scholar 

  21. Gooding, M.J., Kennedy, S., Noble, J.A.: Volume segmentation and reconstruction from freehand three-dimensional ultrasound data with application to ovarian follicle measurement. Ultrasound Med. Biol. 34(2), 183–195 (2008)

    Article  Google Scholar 

  22. Kutikov, A., Uzzo, R.G.: The RENAL nephrometry score: a comprehensive standardized system for quantitating renal tumor size, location and depth. J. Urol. 182(3), 844–853 (2009)

    Article  Google Scholar 

  23. Edgcumbe, P., Nguan, C., Rohling, R.: Calibration and stereo tracking of a laparoscopic ultrasound transducer for augmented reality in surgery. In: Liao, H., Linte, C.A., Masamune, K., Peters, T.M., Zheng, G. (eds.) MIAR 2013 and AE-CAI 2013. LNCS, vol. 8090, pp. 258–267. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  24. Camara, M., Mayer, E., Darzi, A., Pratt, P.: Soft tissue deformation for surgical simulation: a position-based dynamics approach. Int. J. Comput. Assist. Radiol. Surg. 11(6), 919–928 (2016)

    Article  Google Scholar 

  25. Leslie, D.C., Waterhouse, A., Berthet, J.B., Valentin, T.M., Watters, A.L., Jain, A., Kim, P., Hatton, B.D., Nedder, A., Donovan, K., Super, E.H.: A bioinspired omniphobic surface coating on medical devices prevents thrombosis and biofouling. Nat. Biotechnol. 32(11), 1134–1140 (2014)

    Article  Google Scholar 

  26. Lin, J., Clancy, N.T., Stoyanov, D., Elson, D.S.: Tissue surface reconstruction aided by local normal information using a self-calibrated endoscopic structured light system. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9349, pp. 405–412. Springer, Heidelberg (2015)

    Google Scholar 

  27. Edgcumbe, P., Pratt, P., Yang, G.Z., Nguan, C., Rohling, R.: Pico Lantern: surface reconstruction and augmented reality in laparoscopic surgery using a pick-up laser projector. Med. Image Anal. 25(1), 95–102 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

The authors gratefully thank Andrew Wiles of Northern Digital Inc. for support; Prof. Tim Salcudean for providing infrastructure and advice; Denise Kwok for graphics; and funding from the CIHR Vanier Scholarship, VCH-CIHR-UBC MD/PhD Studentship Award and the NSERC CGS-M Award.

Author information

Authors and Affiliations

  1. MD/PhD Program, University of British Columbia, Vancouver, British Columbia, Canada

    Philip Edgcumbe

  2. Electrical and Computer Engineering, University of British Columbia, Vancouver, British Columbia, Canada

    Rohit Singla, Caitlin Schneider & Robert Rohling

  3. Surgery and Cancer, Faculty of Medicine, Imperial College London, London, UK

    Philip Pratt

  4. Urologic Sciences, University of British Columbia, Vancouver, British Columbia, Canada

    Christopher Nguan

  5. Mechanical Engineering, University of British Columbia, Vancouver, British Columbia, Canada

    Robert Rohling

Authors
  1. Philip Edgcumbe
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rohit Singla
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philip Pratt
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Caitlin Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christopher Nguan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Robert Rohling
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rohit Singla .

Editor information

Editors and Affiliations

  1. Institute for Surgical Technology, University of Bern, Bern, Switzerland

    Guoyan Zheng

  2. Tsinghua University, Tokyo, Japan

    Hongen Liao

  3. Faculte de Medicine, Universite de Rennes 1, Rennes Cedex, France

    Pierre Jannin

  4. University of Basel, Allschwil, Switzerland

    Philippe Cattin

  5. Hamlyn Centre, Imperial College London, London, United Kingdom

    Su-Lin Lee

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Edgcumbe, P., Singla, R., Pratt, P., Schneider, C., Nguan, C., Rohling, R. (2016). Augmented Reality Imaging for Robot-Assisted Partial Nephrectomy Surgery. In: Zheng, G., Liao, H., Jannin, P., Cattin, P., Lee, SL. (eds) Medical Imaging and Augmented Reality. MIAR 2016. Lecture Notes in Computer Science(), vol 9805. Springer, Cham. https://doi.org/10.1007/978-3-319-43775-0_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-43775-0_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-43774-3

  • Online ISBN: 978-3-319-43775-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation