Abstract
Purpose
Recent research has revealed that incision planning in laser surgery deploying stylus and tablet outperforms micromanipulator control. However, vision-based adaption to dynamic surgical scenes has not been addressed so far. In this study, scene motion compensation for tablet-based planning by means of tissue deformation tracking is discussed.
Methods
A stereo-based method for motion tracking with piecewise affine deformation modeling is presented. Proposed parametrization relies on the epipolar constraint to enforce left–right consistency in the energy minimization problem. Furthermore, the method implements illumination-invariant tracking and appearance-based occlusion detection. Performance is assessed on laparoscopic and laryngeal in vivo data. In particular, tracking accuracy is measured under various conditions such as occlusions and simulated laser cuttings. Experimental validation is extended to a user study conducted on a tablet-based interface that integrates the tracking for image stabilization.
Results
Tracking accuracy measurement reveals a root-mean-square error of 2.45 mm for the laparoscopic and 0.41 mm for the laryngeal dataset. Results successfully demonstrate stereoscopic tracking under changes in illumination, translation, rotation and scale. In particular, proposed occlusion detection scheme can increase robustness against tracking failure. Moreover, assessed user performance indicates significantly increased path tracing accuracy and usability if proposed tracking is deployed to stabilize the view during free-hand path definition.
Conclusion
The presented algorithm successfully extends piecewise affine deformation tracking to stereo vision taking the epipolar constraint into account. Improved surgical performance as demonstrated for laser incision planning highlights the potential of presented method regarding further applications in computer-assisted surgery.
Similar content being viewed by others
References
Andreff N, Tamadazte B (2016) Laser steering using virtual trifocal visual servoing. Int J Robot Res 35(6):672–694. doi:10.1177/0278364915585585
Bouguet JY (2000) Pyramidal implementation of the lucas kanade feature tracker. Microprocessor Research Labs, Intel Corporation, Tech. rep
Brunet F, Gay-Bellile V, Bartoli A, Navab N, Malgouyres R (2011) Feature-driven direct non-rigid image registration. Int J Comput Vis 93(1):33–52. doi:10.1007/s11263-010-0407-x
Dagnino G, Mattos L, Caldwell D (2015) A vision-based system for fast and accurate laser scanning in robot-assisted phonomicrosurgery. Int J Comput Assist Radiol Surg 10(2):217–229. doi:10.1007/s11548-014-1078-9
Du X, Clancy N, Arya S, Hanna G, Kelly J, Elson D, Stoyanov D (2015) Robust surface tracking combining features, intensity and illumination compensation. Int J Comput Assist Radiol Surg 10(12):1915–1926. doi:10.1007/s11548-015-1243-9
Fichera L, Pardo D, Illiano P, Ortiz J, Caldwell DG, Mattos LS (2016) Online estimation of laser incision depth for transoral microsurgery: approach and preliminary evaluation. Int J Med Robot Comput Assist Surg 12(1):53–61. doi:10.1002/rcs.1656
Filzmoser P, Ruiz-Gazen A, Thomas-Agnan C (2013) Identification of local multivariate outliers. Stat Pap 55(1):29–47. doi:10.1007/s00362-013-0524-z
Giannarou S, Visentini-Scarzanella M, Yang GZ (2013) Probabilistic tracking of affine-invariant anisotropic regions. Pattern Anal Mach Intell IEEE Trans 35(1):130–143. doi:10.1109/TPAMI.2012.81
Kalal Z, Mikolajczyk K, Matas J (2010) Forward-backward error: automatic detection of tracking failures. In: Pattern recognition (ICPR), 2010 20th international conference on, pp 2756–2759. doi:10.1109/ICPR.2010.675
Lau W, Ramey N, Corso J, Thakor N, Hager G (2004) Stereo-based endoscopic tracking of cardiac surface deformation. In: Medical image computing and computer-assisted intervention, MICCAI 2004, vol 3217, pp 494–501. doi:10.1007/978-3-540-30136-3_61
Lewis JR (1991) Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. ACM SIGCHI Bull 23(1):78–81. doi:10.1145/122672.122692
Mattos LS, Deshpande N, Barresi G, Guastini L, Peretti G (2014) A novel computerized surgeon-machine interface for robot-assisted laser phonomicrosurgery. Laryngoscope 124(8):1887–1894. doi:10.1002/lary.24566
McGill R, Tukey JW, Larsen WA (1978) Variations of box plots. Am Stat 32(1):12–16
Mountney P, Stoyanov D, Yang GZ (2010) Three-dimensional tissue deformation recovery and tracking. Signal Process Mag IEEE 27(4):14–24. doi:10.1109/MSP.2010.936728
Mountney P, Yang GZ (2008) Soft tissue tracking for minimally invasive surgery: learning local deformation online. Med Image Comput Comput Assist Interv MICCAI 5242:364–372. doi:10.1007/978-3-540-85990-1_44
Niemz MH (2013) Laser-tissue interactions: fundamentals and applications. Springer Science & Business Media, Berlin
Ortmaier T, Gröger M, Boehm DH, Falk V, Hirzinger G (2005) Motion estimation in beating heart surgery. IEEE Trans Biomed Eng 52(10):1729–1740. doi:10.1109/TBME.2005.855716
Pilet J, Lepetit V, Fua P (2008) Fast non-rigid surface detection, registration and realistic augmentation. Int J Comput Vis 76(2):109–122. doi:10.1007/s11263-006-0017-9
Puerto-Souza G, Mariottini GL (2013) A fast and accurate feature-matching algorithm for minimally-invasive endoscopic images. Med Imaging IEEE Trans 32(7):1201–1214. doi:10.1109/TMI.2013.2239306
Richa R, Poignet P, Liu C (2010) Three-dimensional motion tracking for beating heart surgery using a thin-plate spline deformable model. Int J Rob Res 29(2–3):218–230. doi:10.1177/0278364909356600
Rubinstein M, Armstrong W (2011) Transoral laser microsurgery for laryngeal cancer: a primer and review of laser dosimetry. Lasers Med Sci 26(1):113–124. doi:10.1007/s10103-010-0834-5
Sauvée M, Poignet P, Triboulet J, Dombre E, Malis E, Demaria R (2006) 3d heart motion estimation using endoscopic monocular vision system. Model Control Biomed Syst 6:141–146. doi:10.3182/20060920-3-FR-2912.00029
Schoob A, Kundrat D, Kahrs L, Ortmaier T (2016) Comparative study on surface reconstruction accuracy of stereo imaging devices for microsurgery. Int J Comput Assist Radiol Surg 11(1):145–156. doi:10.1007/s11548-015-1240-z
Schoob A, Kundrat D, Kleingrothe L, Kahrs L, Andreff N, Ortmaier T (2015) Tissue surface information for intraoperative incision planning and focus adjustment in laser surgery. Int J Comput Assist Radiol Surg 10(2):171–181. doi:10.1007/s11548-014-1077-x
Schoob A, Lekon S, Kundrat D, Kahrs LA, Mattos LS, Ortmaier T (2015) Comparison of tablet-based strategies for incision planning in laser microsurgery. In: Proceedings of SPIE medical imaging: image-guided procedures, vol 9415. doi:10.1117/12.2081032
Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: a survey. Med Imaging IEEE Trans 32(7):1153–1190. doi:10.1109/TMI.2013.2265603
Stoyanov D, Darzi A, Yang GZ (2005) A practical approach towards accurate dense 3d depth recovery for robotic laparoscopic surgery. Comput Aided Surg 10(4):199–208. doi:10.3109/10929080500230379
Stoyanov D, Mylonas GP, Deligianni F, Darzi A, Yang GZ (2005) Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Medical image computing and computer-assisted intervention—MICCAI 2005, pp 139–146. Springer, Berlin. doi:10.1007/11566489_18
Stoyanov D, Yang GZ (2007) Stabilization of image motion for robotic assisted beating heart surgery. In: Medical image computing and computer-assisted intervention, MICCAI 2007, vol 4791, pp 417–424 . doi:10.1007/978-3-540-75757-3_51
Stoyanov D, Yang GZ (2009) Soft tissue deformation tracking for robotic assisted minimally invasive surgery. In: Engineering in medicine and biology society, 2009. EMBC 2009. Annual international conference of the IEEE, pp 254–257. doi:10.1109/IEMBS.2009.5334010
Tan DJ, Holzer S, Navab N, Ilic S (2014) Deformable template tracking in 1ms. In: Proceedings of the British machine vision conference. BMVA Press
Tang HW, Brussel HV, Sloten JV, Reynaerts D, De Win G, Cleynenbreugel BV, Koninckx PR (2006) Evaluation of an intuitive writing interface in robot-aided laser laparoscopic surgery. Comput Aided Surg 11(1):21–30. doi:10.3109/10929080500450886
Visentini-Scarzanella M, Merrifield R, Stoyanov D, Yang GZ (2010)Tracking of irregular graphical structures for tissue deformation recovery in minimally invasive surgery. In: Jiang T, Navab N, Pluim J, Viergever M (eds.) Medical image computing and computer-assisted intervention, MICCAI 2010, vol 6363. Springer, Heidelberg, pp 261–268 . doi:10.1007/978-3-642-15711-0_33
Wong WK, Yang B, Liu C, Poignet P (2013) A quasi-spherical triangle-based approach for efficient 3-d soft-tissue motion tracking. Mechatron IEEE/ASME Trans 18(5):1472–1484. doi:10.1109/TMECH.2012.2203919
Yang B, Wong WK, Liu C, Poignet P (2014) 3d soft-tissue tracking using spatial-color joint probability distribution and thin-plate spline model. Pattern Recognit 47(9):2962–2973. doi:10.1016/j.patcog.2014.03.020
Ye M, Giannarou S, Meining A, Yang GZ (2016) Online tracking and retargeting with applications to optical biopsy in gastrointestinal endoscopic examinations. Med Image Anal 30:144–157. doi:10.1016/j.media.2015.10.003
Yip M, Lowe D, Salcudean S, Rohling R, Nguan C (2012) Tissue tracking and registration for image-guided surgery. Med Imaging IEEE Trans 31(11):2169–2182. doi:10.1109/TMI.2012.2212718
Zabih R, Woodfill J (1994) Non-parametric local transforms for computing visual correspondence. In: Eklundh JO (ed) Computer vision ECCV 94, vol 801, pp 151–158. Springer, Berlin. doi:10.1007/BFb0028345
Zagorchev L, Goshtasby A (2006) A comparative study of transformation functions for nonrigid image registration. Image Process IEEE Trans 15(3):529–538. doi:10.1109/TIP.2005.863114
Zhang TY, Suen CY (1984) A fast parallel algorithm for thinning digital patterns. Commun ACM 27(3):236–239. doi:10.1145/357994.358023
Zhu J, Lyu M, Huang T (2009) A fast 2d shape recovery approach by fusing features and appearance. Pattern Anal Mach Intell IEEE Trans 31(7):1210–1224. doi:10.1109/TPAMI.2008.151
Acknowledgments
This research has received funding from the European Union Seventh Framework Programme FP7/2007-2013–Challenge 2—Cognitive Systems, Interaction, Robotics—under grant agreement \(\upmu \)RALP—no 288663. We thank Giorgio Peretti from the Department of Otorhinolaryngology, University of Genoa, Italy, providing in vivo laryngeal data. We also would like to thank Guang-Zhong Yang, Danail Stoyanov and Peter Mountney for the in vivo data provided by the Hamlyn Centre for Robotic Surgery, Imperial College London, UK.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors declare that they have no conflict of interest.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Schoob, A., Laves, MH., Kahrs, L.A. et al. Soft tissue motion tracking with application to tablet-based incision planning in laser surgery. Int J CARS 11, 2325–2337 (2016). https://doi.org/10.1007/s11548-016-1420-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-016-1420-5