Abstract
This paper presents an endoscopic vision framework for model-based 3D guidance of surgical instruments used in robotized laparoscopic surgery. In order to develop such a system, a variety of challenging segmentation, tracking and reconstruction problems must be solved. With this minimally invasive surgical technique, every single instrument has to pass through an insertion point in the abdominal wall and is mounted on the end-effector of a surgical robot which can be controlled by automatic visual feedback. The motion of any laparoscopic instrument is then constrained and the goal of the automated task is to safety bring instruments at desired locations while avoiding undesirable contact with internal organs. For this ”eye-to-hands” configuration with a stationary camera, most control strategies require the knowledge of the out-of-field of view insertion points location and we demonstrate it can be achieved in vivo thanks to a sequence of (instrument) motions without markers and without the need of an external measurement device. In so doing, we firstly present a real-time region-based color segmentation which integrates this motion constraint to initiate the search for region seeds. Secondly, a novel pose algorithm for the wide class of cylindrical-shaped instruments is developed which can handle partial occlusions as it is often the case in the abdominal cavity. The foreseen application is a good training ground to evaluate the robustness of segmentation algorithms and positioning techniques since main difficulties came from the scene understanding and its dynamical variations. Experiments in the lab and in real surgical conditions have been conducted. The experimental validation is demonstrated through the 3D positioning of instruments’ axes (4 DOFs) which must lead to motionless insertion points disturbed by the breathing motion.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wei, G.Q., Arbter, K., Hirzinger, G.: Real-time visual servoing for laparoscopic surgery. IEEE Engineering in Medicine and Biology 16, 40–45 (1997)
Burschka, D., Corso, J.J., Dewan, M., Hager, G.D., Lau, W., Li, M., Lin, H., Marayong, P., Ramey, N.: Navigating inner space: 3-d assistance for minimally invasive surgery. In: Workshop Advances in Robot Vision, IEEE/RSJ Int’l Conf. on Intelligent Robots and Systems, Sendai, Japan, pp. 67–78. IEEE, Los Alamitos (2004)
Krupa, A., Gangloff, J., Doignon, C., de Mathelin, M., Morel, G., Leroy, J., Soler, L., Marescaux, J.: Autonomous 3-d positioning of surgical instruments in robotized laparoscopic surgery using visual servoing. IEEE Trans. on Robotics and Automation 10, 842–853 (2003)
Nageotte, F., Zanne, P., de Mathelin, M., Doignon, C.: A circular needle path planning method for suturing in laparoscopic surgery. In: Proceedings of the IEEE Int’l Conf. on Robotics and Automation, Barcelone, Spain, pp. 516–521. IEEE Computer Society Press, Los Alamitos (2005)
Casals, A., Amat, J., Prats, D., Laporte, E.: Vision guided robotic system for laparoscopic surgery. In: Proc. of the IFAC Int. Congress on Advanced Robotics, Barcelona, Spain, pp. 33–36 (1995)
Wang, Y.F., Uecker, D.R., Wang, Y.: A new framework for vision-enabled and robotically assisted minimally invasive surgery. Journal of Computerized Medical Imaging and Graphics 22, 429–437 (1998)
Hayashibe, M., Nakamura, Y.: Laser-pointing endoscope system for intra-operative geometric registration. In: Proc. of the 2001 IEEE International Conference on Robotics and Automation, Seoul, South Korea, IEEE Computer Society Press, Los Alamitos (2001)
Ortmaier, T., Hirzinger, G.: Cartesian control issues for minimally invasive robot surgery. In: Proceedings of the IEEE/RSJ Int’l Conf. on Intelligent Robots and Systems, Takamatsu, Japan, IEEE, Los Alamitos (2000)
Voros, S., Orvain, E., Cinquin, P., Long, J.A.: Automatic detection of instruments in laparoscopic images: a first step towards high level command of robotized endoscopic holders. In: IEEE Conf. on Biomed. Robotics and Biomechatronics, IEEE Computer Society Press, Los Alamitos (2006)
Doignon, C., Nageotte, F., de Mathelin, M.: Detection of grey regions in color images: application to the segmentation of a surgical instrument in robotized laparoscopy. In: Proceedings of the IEEE/RSJ Int’l Conference on Intelligent Robots and Systems, Sendai, Japan, IEEE, Los Alamitos (2004)
van de Weijer, J., Gevers, T., Geusebroek, J.M.: Color edge detection by photometric quasi-invariants. In: Proc. of ICCV, Nice, France, pp. 1520–1526 (2003)
Perona, P., Shiota, T., Malik, J.: Anisotropic diffusion. In: Geometry-driven diffusion in Computer Vision, pp. 73–92. Kluwer Academic Publishers, Dordrecht (1994)
Doignon, C., Graebling, P., de Mathelin, M.: Real-time segmentation of surgical instruments inside the abdominal cavity using a joint hue saturation color feature. Real-Time Imaging 11, 429–442 (2005)
Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography. Int’l Journal of Computer Vision 9, 137–154 (1992)
Weinshall, D., Tomasi, C.: Linear an incremental acquisition of invariant shape models from image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 17, 512–517 (1995)
Sturm, P., Triggs, W.: A factorization based algorithm for multi-image projective structure and motion. In: Buxton, B.F., Cipolla, R. (eds.) ECCV 1996. LNCS, vol. 1065, pp. 709–720. Springer, Heidelberg (1996)
Ma, Y., Soatto, S., Košeká, J., Sastry, S.: Invitation to 3D Vision: From Images to Geometric Models. Springer, Heidelberg (2003)
Bartoli, A., Sturm, P.: The 3d line motion matrix and alignment of line reconstructions. In: Proceedings of CVPR, Hawaii, USA, pp. 287–292 (2001)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge Univ. Press, Cambridge (2000)
Wong, K.Y., Mendonça, P.R.S., Cipolla, R.: Reconstruction of surfaces of revolution from single uncalibrated views. Image and Vis. Computing 22, 829–836 (2004)
Espiau, B., Chaumette, F., Rives, P.: A new approach to visual servoing in robotics. IEEE Transactions on Robotics and Automation 8, 313–326 (1992)
Haralick, R.M., Shapiro, L.G.: Computer and Robot Vision, vol. 2. Addison-Wesley Publishing, Reading (1992)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Doignon, C., Nageotte, F., de Mathelin, M. (2007). Segmentation and Guidance of Multiple Rigid Objects for Intra-operative Endoscopic Vision. In: Vidal, R., Heyden, A., Ma, Y. (eds) Dynamical Vision. WDV WDV 2006 2005. Lecture Notes in Computer Science, vol 4358. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-70932-9_24
Download citation
DOI: https://doi.org/10.1007/978-3-540-70932-9_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-70931-2
Online ISBN: 978-3-540-70932-9
eBook Packages: Computer ScienceComputer Science (R0)