Dynamic Guidance for Robotic Surgery Using Image-Constrained Biomechanical Models

  • Philip Pratt
  • Danail Stoyanov
  • Marco Visentini-Scarzanella
  • Guang-Zhong Yang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6361)


The use of physically-based models combined with image constraints for intraoperative guidance is important for surgical procedures that involve large-scale tissue deformation. A biomechanical model of tissue deformation is described in which surface positional constraints and internally generated forces are derived from endoscopic images and preoperative 4D CT data, respectively. Considering cardiac motion, a novel technique is presented which minimises the average registration error over one or more complete cycles. Features tracked in the stereo video stream provide surface constraints, and an inverse finite element simulation is presented which allows internal forces to be recovered from known preoperative displacements. The accuracy of surface texture, segmented mesh and volumetrically rendered overlays is evaluated with detailed phantom experiments. Results indicate that by combining preoperative and intraoperative images in this manner, accurate intraoperative tissue deformation modelling can be achieved.


Augmented Reality Video Frame Robotic Surgery Partial Nephrectomy Laparoscopic Partial Nephrectomy 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Sun, H., Kennedy, F.E., Carlson, E.J., Hartov, A., Roberts, D.W., Paulsen, K.D.: Modeling of brain tissue retraction using intraoperative data. In: Barillot, C., Haynor, D.R., Hellier, P. (eds.) MICCAI 2004, Part II. LNCS, vol. 3217, pp. 225–233. Springer, Heidelberg (2004)Google Scholar
  2. 2.
    Szpala, S., Wierzbicki, M., Guiraudon, G., Peters, T.M.: Real-time fusion of endoscopic views with dynamic 3-D cardiac images: A phantom study. IEEE Transactions on Medical Imaging 24(9), 1207–1215 (2005)CrossRefGoogle Scholar
  3. 3.
    Figl, M., Rueckert, D., Hawkes, D., Casula, R., Hu, M., Pedro, O., Zhang, D.P., Penney, G., Bello, F., Edwards, P.: Image guidance for robotic minimally invasive coronary artery bypass. In: Dohi, T., Sakuma, I., Liao, H. (eds.) MIAR 2008. LNCS, vol. 5128, pp. 202–209. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  4. 4.
    Su, L.M., Vagvolgyi, B.P., Agarwal, R., Reiley, C.E., Taylor, R.H., Hager, G.D.: Augmented reality during robot-assisted laparoscopic partial nephrectomy: Toward real-time 3D-CT to stereoscopic video registration. Urology 73(4), 896–900 (2009)CrossRefGoogle Scholar
  5. 5.
    Lerotic, M., Lee, S.L., Keegan, J., Yang, G.Z.: Image constrained finite element modelling for real-time surgical simulation and guidance. In: Proceedings of the IEEE International Symposium on Biomedical Imaging, pp. 1063–1066 (2009)Google Scholar
  6. 6.
    Zhang, Z.: A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  7. 7.
    Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A., Yang, G.Z.: Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005, Part II. LNCS, vol. 3750, pp. 139–146. Springer, Heidelberg (2005)Google Scholar
  8. 8.
    Miller, K., Joldes, G., Lance, D., Wittek, A.: Total lagrangian explicit dynamics finite element algorithm for computing soft tissue deformation. Communications in Numerical Methods in Engineering 23, 121–134 (2007)zbMATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Bathe, K.J.: Finite Element Procedures. Prentice-Hall, Inc., Englewood Cliffs (1996)Google Scholar
  10. 10.
    Yushkevich, P.A., Piven, J., Cody Hazlett, H., Gimpel Smith, R., Ho, S., Gee, J.C., Gerig, G.: User-guided 3D active contour segmentation of anatomical structures. Neuroimage 31(3), 1116–1128 (2006)CrossRefGoogle Scholar
  11. 11.
    Geuzaine, C., Remacle, J.F.: Gmsh: a three-dimensional finite element mesh generator with built-in pre- and post-processing facilities. International Journal for Numerical Methods in Engineering 79(11), 1309–1331 (2009)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Rueckert, D., Sonoda, L.I., Hayes, C., Hill, D.L.G., Leach, M.O., Hawkes, D.J.: Non-rigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging 18(8), 712–721 (1999)CrossRefGoogle Scholar
  13. 13.
    Schnabel, J.A., Rueckert, D., Quist, M., Blackall, J.M., Castellano-Smith, A.D., Hartkens, T., Penney, G.P., Hall, W.A., Liu, H., Truwit, C.L., Gerritsen, F.A., Hill, D.L.G., Hawkes, D.J.: A generic framework for non-rigid registration based on non-uniform multi-level free-form deformations. In: Niessen, W.J., Viergever, M.A. (eds.) MICCAI 2001. LNCS, vol. 2208, pp. 573–581. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  14. 14.
    Pratt, P.: Image guidance and surgery simulation using inverse nonlinear finite element methods. In: Bello, F., Edwards, E. (eds.) ISBMS 2008. LNCS, vol. 5104, pp. 185–190. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  15. 15.
    Lerotic, M., Chung, A.J., Mylonas, G., Yang, G.Z.: pq-space based non-photorealistic rendering for augmented reality. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 102–109. Springer, Heidelberg (2007)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Philip Pratt
    • 1
  • Danail Stoyanov
    • 1
  • Marco Visentini-Scarzanella
    • 1
  • Guang-Zhong Yang
    • 1
  1. 1.Institute of Biomedical Engineering Imperial College of Science, Technology and MedicineLondonUK

Personalised recommendations