Abstract
Augmented-Reality (AR) displays increase surgeon’s visual awareness of high-risk surgical targets (e.g., the location of a tumor) by accurately overlaying pre-operative radiological 3-D model onto the intra-operative laparoscopic video. Existing AR systems are not robust to sudden camera motion or prolonged occlusions, which can cause the loss of those anchor points tracked along the video sequence, and thus the loss of the AR display. In this paper, we present a novel AR system, integrated with a novel feature-matching method, to automatically recover the lost augmentation by predicting the image locations of the AR anchor image-points after sudden image changes. Extensive experiments on challenging surgical video data are presented that show the accuracy, speed, and robustness of our designs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Richa, R., Bo, A.P.L., Poignet, P.: Towards robust 3d visual tracking for motion compensation in beating heart surgery. Medical Image Analysis (2010)
Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A., Yang, G.Z.: Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 139–146. Springer, Heidelberg (2005)
Mountney, P., Yang, G.-Z.: Motion compensated Slam for image guided surgery. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part II. LNCS, vol. 6362, pp. 496–504. Springer, Heidelberg (2010)
Giannarou, S., Visentini-Scarzanella, M., Yang, G.: Probabilistic tracking of affine-invariant anisotropic regions. IEEE T.PAMI (2012)
Yip, M.C., Lowe, D.G., Salcudean, S.E., Rohling, R.N., Nguan, C.Y.: Real-time methods for long-term tissue feature tracking in endoscopic scenes. In: Abolmaesumi, P., Joskowicz, L., Navab, N., Jannin, P. (eds.) IPCAI 2012. LNCS, vol. 7330, pp. 33–43. Springer, Heidelberg (2012)
Puerto-Souza, G.A., Mariottini, G.L.: Hierarchical multi-affine (HMA) algorithm for fast and accurate feature matching in minimally-invasive surgical images. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS (2012)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comp. Vis. 60(2), 91–110 (2004)
Cho, M., Lee, J., Lee, K.M.: Feature correspondence and deformable object matching via agglomerative correspondence clustering. In: Proc. 9th Int. Conf. Comp. Vis., pp. 1280–1287 (2009)
Puerto-Souza, G.A., Adibi, M., Cadeddu, J.A., Mariottini, G.L.: Adaptive multi-affine (AMA) feature-matching algorithm and its application to minimally-invasive surgery images. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2371–2376 (September 2011)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)
Morel, J.M., Yu, G.: ASIFT: A new framework for fully affine invariant image comparison. Journal on Imaging Sciences 2(2), 438–469 (2009)
Szeliski, R.: Computer vision: algorithms and applications. Springer-Verlag New York Inc. (2010)
Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. 60, 1615–1630 (2005)
Mountney, P., Lo, B., Thiemjarus, S., Stoyanov, D., Yang, G.-Z.: A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery. In: Ayache, N., Ourselin, S., Maeder, A. (eds.) MICCAI 2007, Part II. LNCS, vol. 4792, pp. 34–41. Springer, Heidelberg (2007)
Del Bimbo, A., Franco, F., Pernici, F.: Local shape estimation from a single keypoint. In: Proc. Comp. Vis. Patt. Rec. Workshops, pp. 23–28 (2010)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge Univ. Press (2000)
Hailin, J., Favaro, P., Soatto, S.: Real-time feature tracking and outlier rejection with changes in illumination. In: Proc. IEEE Intern. Conf. on Computer Vision, vol. 1, pp. 684–689 (2001)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part I. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006)
Urban, M., Matas, J., Chum, O., Pajdla, T.: Robust wide baseline stereo for maximally stable extremal regions. Image and Vision Computing 22(10), 761–767 (2004)
Siegwart, R., Nourbakhsh, I.R., Scaramuzza, D.: Introduction to Autonomous Mobile Robots. MIT Press (2011)
Rosten, E., Drummond, R.: Fusing points and lines for high performance tracking. In: Proceedings of the International Conference on Computer Vision, pp. 1508–1511 (2005)
Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference, Manchester, UK, vol. 15, p. 50 (1988)
Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: International Conference on Computer Vision Theory and Application, VISSAPP 2009, pp. 331–340. INSTICC Press (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Puerto-Souza, G.A., Castaño-Bardawil, A., Mariottini, GL. (2013). Real-Time Feature Matching for the Accurate Recovery of Augmented-Reality Display in Laparoscopic Videos. In: Linte, C.A., Chen, E.C.S., Berger, MO., Moore, J.T., Holmes, D.R. (eds) Augmented Environments for Computer-Assisted Interventions. AE-CAI 2012. Lecture Notes in Computer Science, vol 7815. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38085-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-642-38085-3_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-38084-6
Online ISBN: 978-3-642-38085-3
eBook Packages: Computer ScienceComputer Science (R0)