In order to assist in the identification of renal vasculature and tumour boundaries in robot-assisted partial nephrectomy, robust ultrasound probe calibration and tracking methods are introduced. Contemporaneous image guidance during these crucial stages of the procedure should ultimately lead to improved safety and quality of outcome for the patient, through reduced positive margin rates, segmental clamping, shorter ischaemic times and nephron-sparing resection.
Small KeyDot markers with circular dot patterns are attached to a miniature pickup ultrasound probe. Generic probe calibration is superseded by a more robust scheme based on a sequence of physical transducer measurements. Motion prediction combined with a reduced region-of-interest in the endoscopic video feed facilitates real-time tracking and registration performance at full HD resolutions.
Quantitative analysis confirms that circular dot patterns result in an improved translational and rotational working envelope, in comparison with the previous chessboard pattern implementation. Furthermore, increased robustness is observed with respect to prevailing illumination levels and out-of-focus images due to relatively small endoscopic depths of field.
Circular dot patterns should be employed in this context as they result in improved performance and robustness. This facilitates clinical usage and interpretation of the combined video and ultrasound overlay. The efficacy of the overall system is demonstrated in the first human clinical case.
This is a preview of subscription content, log in to check access.
Buy single article
Instant unlimited access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Hughes-Hallett A, Mayer E, Marcus H, Cundy T, Pratt P, Darzi A, Vale J (2013) Augmented reality partial nephrectomy: examining the current status and future perspectives. Urology 83(2). Elsevier, 266–273
Hughes-Hallett A, Pratt P, Mayer E, Martin S, Darzi A, Vale J (2014) Image guidance for all—TilePro display of 3-dimensionally reconstructed images in robotic partial nephrectomy. Urology 84(1). Elsevier, 237–243
Sun M, Wagner A, San Francisco I, Brook A, Kavoussi L, Russo P, Steele G, Viterbo R, Pedrosa I (2012) Need for intraoperative ultrasound and surgical recommendation for partial nephrectomy: correlation with tumor imaging features and urologist practice patterns. Ultrasound Q 28(1):21–27
Leven J, Burschka D, Kumar R, Zhang G, Blumenkranz S, Xiangtian D, Awad M, Hager G, Marohn M, Choti M, Hasser C, Taylor R (2005) DaVinci canvas: a telerobotic surgical system with integrated, robot-assisted, laparoscopic ultrasound capability. In: MICCAI, LNCS 3749. Springer, Berlin, pp 811–818
Schneider C, Dachs G, Hasser C, Choti M, DiMaio S, Taylor R (2010) Robot-assisted laparoscopic ultrasound. In: Information Processing and in Computer-Assisted Interventions, LNCS 6135. Springer, Berlin, pp 67–80
Schneider C, Guerrero J, Nguan CY, Rohling R, Salcudean SE (2011) Intra-operative “Pick-Up” ultrasound for robot assisted surgery with vessel extraction and registration: a feasibility study. In: International conference on information processing in computer assisted interventions. Springer, Berlin, pp 122–132
Cheung CL, Wedlake C, Moore J, Pautler SE, Peters TM (2010) Fused video and ultrasound images for minimally invasive partial nephrectomy: a phantom study. In: MICCAI, Part III, LNCS 6363. Springer, Berlin, pp 408–415
Pratt P, Di Marco A, Payne C, Darzi A, Yang G-Z (2012) Intraoperative ultrasound guidance for transanal endoscopic microsurgery. In: MICCAI, Part I, LNCS 7510. Springer, Berlin, pp 463–470
Jayarathne U, McLeod AJ, Peters T, Chen E (2013) Robust intraoperative US probe tracking using a monocular endoscopic camera. In: MICCAI, Part III, LNCS 8151. Springer, Berlin, pp 363–370
Hughes-Hallett A, Pratt P, Mayer E, Di Marco A, Yang G-Z, Vale J, Darzi A (2013) Intraoperative ultrasound overlay in robot-assisted partial nephrectomy: first clinical experience. Eur Urol 65(3). Elsevier, 671–972
Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, Inc., Sebastopol
Suzuki S, Abe K (1985) Topological structural analysis of digitized binary images by border following. Comput Vis Graph Image Process 30(1):32–46
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334
Fuchs H, State A, Pisano E, Garrett W, Hirota G, Livingston M, Whitton M, Pizer S (1996) Towards performing ultrasound-guided needle biopsies from within a head-mounted display. In: Visualization in Biomedical Computing, LNCS 1131. Springer, pp 591–600
Pratt P, Bergeles C, Darzi A, Yang G-Z (2014) Practical intraoperative stereo camera calibration. In: Proceedings of the international conference on medical image computing and computer-assisted intervention. Part II, LNCS 8674. Springer, Berlin, pp 667–675
Yushkevich P, Piven J, Cody-Hazlett H, Gimpel-Smith R, Ho S, Gee J, Gerig G (2006) User-guided 3D active contour segmentation of anatomical structures. Neuroimage 31(3):1116–1128
Yip M, Adebar T, Rohling R, Salcudean S, Nguan C (2010) 3D Ultrasound to stereoscopic camera registration through an air-tissue boundary. In: MICCAI, LNCS 6362. Springer, Berlin, pp 626–634
Cheng A, Kang J, Taylor R, Boctor E (2012) Direct 3D Ultrasound to video registration using photoacoustic effect. In: MICCAI, LNCS 7511. Springer, Berlin, pp 552–559
The authors are grateful for support from The Hamlyn Centre and the NIHR Biomedical Research Centre funding scheme and would like to thank members of the operating theatre team at the Surgical Innovation Centre, St Mary’s Hospital, Paddington, London.
Conflict of interest
The authors declare that they have no conflict of interest.
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.
Informed consent was obtained from all individual participants included in the study.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Pratt, P., Jaeger, A., Hughes-Hallett, A. et al. Robust ultrasound probe tracking: initial clinical experiences during robot-assisted partial nephrectomy. Int J CARS 10, 1905–1913 (2015) doi:10.1007/s11548-015-1279-x
- Image guidance
- Partial nephrectomy