An Augmented Reality Framework for Soft Tissue Surgery

  • Peter Mountney
  • Johannes Fallert
  • Stephane Nicolau
  • Luc Soler
  • Philip W. Mewes
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8673)

Abstract

Augmented reality for soft tissue laparoscopic surgery is a growing topic of interest in the medical community and has potential application in intra-operative planning and image guidance. Delivery of such systems to the operating room remains complex with theoretical challenges related to tissue deformation and the practical limitations of imaging equipment. Current research in this area generally only solves part of the registration pipeline or relies on fiducials, manual model alignment or assumes that tissue is static. This paper proposes a novel augmented reality framework for intra-operative planning: the approach co-registers pre-operative CT with stereo laparoscopic images using cone beam CT and fluoroscopy as bridging modalities. It does not require fiducials or manual alignment and compensates for tissue deformation from insufflation and respiration while allowing the laparoscope to be navigated. The paper’s theoretical and practical contributions are validated using simulated, phantom, ex vivo, in vivo and non medical data.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Hughes-Hallett, A., Mayer, E.K., Marcus, H.J., Cundy, T.P., Pratt, P.J., Darzi, A.W., Vale, J.A.: Augmented reality partial nephrectomy: Examining the current status and future perspectives. Urology, 266–273 (2013)Google Scholar
  2. 2.
    Allard, J., Cotin, S., Faure, F., Bensoussan, P.J., Poyer, F., Duriez, C., Delingette, H., Grisoni, L.: Sofa-open framework for medical simulation. In: MMVR (2007)Google Scholar
  3. 3.
    Collins, T., Bartoli, A.: Towards live monocular 3d laparoscopy using shading and specularity information. In: Abolmaesumi, P., Joskowicz, L., Navab, N., Jannin, P. (eds.) IPCAI 2012. LNCS, vol. 7330, pp. 11–21. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  4. 4.
    Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  5. 5.
    Oktay, O., Zhang, L., Mansi, T., Mountney, P., Mewes, P., Nicolau, S., Soler, L., Chefd’hotel, C.: Biomechanically driven registration of pre- to intra-operative 3D images for laparoscopic surgery. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 1–9. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  6. 6.
    Puerto Souza, G.A., Adibi, M., Cadeddu, J.A., Mariottini, G.L.: Adaptive multi-affine (AMA) feature-matching algorithm and its application to minimally-invasive surgery images. In: IROS, pp. 2371–2376 (2011)Google Scholar
  7. 7.
    Maier-Hein, L., Franz, A., dos Santos, T., Schmidt, M., Fangerau, M., Meinzer, H., Fitzpatrick, J.: Convergent iterative closest-point algorithm to accomodate anisotropic and inhomogenous localization error. PAMI 34(8), 1520–1532 (2012)CrossRefGoogle Scholar
  8. 8.
    Mirota, D., Uneri, A., Schafer, S., Nithiananthan, S., Reh, D., Ishii, M., Gallia, G., Taylor, R., Hager, G., Siewerdsen, J.: Evaluation of a system for high-accuracy 3D image-based registration of endoscopic video to c-arm cone-beam CT for image-guided skull base surgery. Transactions on Medical Imaging 32, 1215–1226 (2013)CrossRefGoogle Scholar
  9. 9.
    Mountney, P., Yang, G.Z.: Motion compensated slam for image guided surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part II. LNCS, vol. 6362, pp. 496–504. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Nicolau, S.A., Pennec, X., Soler, L., Buy, X., Gangi, A., Ayache, N., Marescaux, J.: An augmented reality system for liver thermal ablation: Design and evaluation on clinical cases. Medical Image Analysis 13(3), 494–506 (2009)CrossRefGoogle Scholar
  11. 11.
    Teber, D., Guven, S., Simpfendrfer, T., Baumhauer, M., Gven, E.O., Yencilek, F., Gzen, A.S., Rassweiler, J.: Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? preliminary in vitro and in vivo results. European Urology 56(2), 332–338 (2009)CrossRefGoogle Scholar
  12. 12.
    Su, L.M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4), 896–900 (2009)CrossRefGoogle Scholar
  13. 13.
    Pratt, P., Mayer, E., Vale, J., Cohen, D., Edwards, E., Darzi, A., Yang, G.Z.: An effective visualisation and registration system for image-guided robotic partial nephrectomy. Journal of Robotic Surgery 6(1), 23–31 (2012)CrossRefGoogle Scholar
  14. 14.
    Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: RSS, p. 4 (2009)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Peter Mountney
    • 1
  • Johannes Fallert
    • 2
  • Stephane Nicolau
    • 3
  • Luc Soler
    • 3
    • 4
  • Philip W. Mewes
    • 5
  1. 1.Imaging and Computer VisionSiemens Corporate TechnologyPrincetonUSA
  2. 2.Imaging Technologies ResearchGermany
  3. 3.Institut de Recherche contre les Cancers de l’Appareil Digestif (IRCAD)StrasbourgFrance
  4. 4.Institut Hospitalo-Universitaire de Strasbourg (IHU Strasbourg)StrasbourgFrance
  5. 5.Angiography & Interventional X-Ray SystemsSiemens HealthcareGermany

Personalised recommendations