Skip to main content

An Augmented Reality Framework for Soft Tissue Surgery

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNIP,volume 8673)


Augmented reality for soft tissue laparoscopic surgery is a growing topic of interest in the medical community and has potential application in intra-operative planning and image guidance. Delivery of such systems to the operating room remains complex with theoretical challenges related to tissue deformation and the practical limitations of imaging equipment. Current research in this area generally only solves part of the registration pipeline or relies on fiducials, manual model alignment or assumes that tissue is static. This paper proposes a novel augmented reality framework for intra-operative planning: the approach co-registers pre-operative CT with stereo laparoscopic images using cone beam CT and fluoroscopy as bridging modalities. It does not require fiducials or manual alignment and compensates for tissue deformation from insufflation and respiration while allowing the laparoscope to be navigated. The paper’s theoretical and practical contributions are validated using simulated, phantom, ex vivo, in vivo and non medical data.


  • Augmented Reality
  • Iterative Close Point
  • Laparoscopic Partial Nephrectomy
  • Tissue Deformation
  • Target Registration Error

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. Hughes-Hallett, A., Mayer, E.K., Marcus, H.J., Cundy, T.P., Pratt, P.J., Darzi, A.W., Vale, J.A.: Augmented reality partial nephrectomy: Examining the current status and future perspectives. Urology, 266–273 (2013)

    Google Scholar 

  2. Allard, J., Cotin, S., Faure, F., Bensoussan, P.J., Poyer, F., Duriez, C., Delingette, H., Grisoni, L.: Sofa-open framework for medical simulation. In: MMVR (2007)

    Google Scholar 

  3. Collins, T., Bartoli, A.: Towards live monocular 3d laparoscopy using shading and specularity information. In: Abolmaesumi, P., Joskowicz, L., Navab, N., Jannin, P. (eds.) IPCAI 2012. LNCS, vol. 7330, pp. 11–21. Springer, Heidelberg (2012)

    CrossRef  Google Scholar 

  4. Stoyanov, D., Scarzanella, M.V., Pratt, P., Yang, G.-Z.: Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 275–282. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  5. Oktay, O., Zhang, L., Mansi, T., Mountney, P., Mewes, P., Nicolau, S., Soler, L., Chefd’hotel, C.: Biomechanically driven registration of pre- to intra-operative 3D images for laparoscopic surgery. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013, Part II. LNCS, vol. 8150, pp. 1–9. Springer, Heidelberg (2013)

    CrossRef  Google Scholar 

  6. Puerto Souza, G.A., Adibi, M., Cadeddu, J.A., Mariottini, G.L.: Adaptive multi-affine (AMA) feature-matching algorithm and its application to minimally-invasive surgery images. In: IROS, pp. 2371–2376 (2011)

    Google Scholar 

  7. Maier-Hein, L., Franz, A., dos Santos, T., Schmidt, M., Fangerau, M., Meinzer, H., Fitzpatrick, J.: Convergent iterative closest-point algorithm to accomodate anisotropic and inhomogenous localization error. PAMI 34(8), 1520–1532 (2012)

    CrossRef  Google Scholar 

  8. Mirota, D., Uneri, A., Schafer, S., Nithiananthan, S., Reh, D., Ishii, M., Gallia, G., Taylor, R., Hager, G., Siewerdsen, J.: Evaluation of a system for high-accuracy 3D image-based registration of endoscopic video to c-arm cone-beam CT for image-guided skull base surgery. Transactions on Medical Imaging 32, 1215–1226 (2013)

    CrossRef  Google Scholar 

  9. Mountney, P., Yang, G.Z.: Motion compensated slam for image guided surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part II. LNCS, vol. 6362, pp. 496–504. Springer, Heidelberg (2010)

    CrossRef  Google Scholar 

  10. Nicolau, S.A., Pennec, X., Soler, L., Buy, X., Gangi, A., Ayache, N., Marescaux, J.: An augmented reality system for liver thermal ablation: Design and evaluation on clinical cases. Medical Image Analysis 13(3), 494–506 (2009)

    CrossRef  Google Scholar 

  11. Teber, D., Guven, S., Simpfendrfer, T., Baumhauer, M., Gven, E.O., Yencilek, F., Gzen, A.S., Rassweiler, J.: Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? preliminary in vitro and in vivo results. European Urology 56(2), 332–338 (2009)

    CrossRef  Google Scholar 

  12. Su, L.M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: toward real-time 3D-CT to stereoscopic video registration. Urology 73(4), 896–900 (2009)

    CrossRef  Google Scholar 

  13. Pratt, P., Mayer, E., Vale, J., Cohen, D., Edwards, E., Darzi, A., Yang, G.Z.: An effective visualisation and registration system for image-guided robotic partial nephrectomy. Journal of Robotic Surgery 6(1), 23–31 (2012)

    CrossRef  Google Scholar 

  14. Segal, A., Haehnel, D., Thrun, S.: Generalized-ICP. In: RSS, p. 4 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations


Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Mountney, P., Fallert, J., Nicolau, S., Soler, L., Mewes, P.W. (2014). An Augmented Reality Framework for Soft Tissue Surgery. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014. MICCAI 2014. Lecture Notes in Computer Science, vol 8673. Springer, Cham.

Download citation

  • DOI:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-10403-4

  • Online ISBN: 978-3-319-10404-1

  • eBook Packages: Computer ScienceComputer Science (R0)