Abstract
We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., & Cohen, M. (2004a). Interactive digital photomontage. ACM Transactions on Graphics, 294–302.
Agarwala, A., Hertzmann, A., Salesin, D. H., & Seitz, S. M. (2004b). Keyframe-based tracking for rotoscoping and animation. ACM Transactions on Graphics, 23(3), 584–591.
Agarwala, A., Agrawala, M., Cohen, M., Salesin, D., & Szeliski, R. (2006). Photographing long scenes with multi-viewpoint panoramas. ACM Transactions on Graphics, 25(3), 853–861.
Baker, S., & Matthews, I. (2002). Lucas-Kanade 20 years on: A unifying framework: Part 1 (Tech. rep.). Robotics Institute, Carnegie Mellon University, Pittsburg, PA.
Bricault, I., Ferretti, G., & Cinquin, P. (1998). Registration of real and ct-derived virtual bronchoscopic images to assist transbronchial biopsy. IEEE Transactions on Medical Imaging, 17(5), 703–714.
Carrascosa, P., Capunay, C., Lopez, E. M., Ulla, M., Castiglioni, R., & Carrascosa, J. (2006). Multidetector ct colonoscopy: evaluation of the perspective-filet view virtual colon dissection technique for the detection of elevated lesions. Abdominal Imaging.
Carroll, R. E., & Seitz, S. M. (2007). Rectified surface mosaics. In Computer vision, 2007 ICCV 2007 IEEE 11th international conference (pp. 1–8).
Chen, S. E. (1995). Quicktime vr: an image-based approach to virtual environment navigation. In SIGGRAPH ’95 (pp. 29–38).
Fattal, R., Lischinski, D., & Werman, M. (2002). Gradient domain high dynamic range compression. ACM Transactions on Graphics, 21(3), 249–256.
Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press.
Helferty, J. P., Hoffman, E. A., McLennan, G., & Higgins, W. E. (2004). Ct-video registration accuracy for virtual guidance of bronchoscopy. In SPIE (Vol. 5369, pp. 150–164).
Konen, W., Breiderhoff, B., & Scholz, M. (2007). Real-time image mosaic for endoscopic video sequences. In Bildverarbeitung für die Medizin 2007 (pp. 298–302).
Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proceedings of the 7th international joint conference on artificial intelligence (pp. 674–679).
Miranda-Luna, R., Daul, C., Blondel, W., Hernandez-Mier, Y., Wolf, D., & Guillemin, F. (2008). Mosaicing of bladder endoscopic image sequences: Distortion calibration and registration algorithm. IEEE Transactions on Biomedical Engineering, 55(2), 541–553.
Mori, K., Suenaga, Y., Toriwaki, J., Hasegawa, J., Kataa, K., Takabatake, H., & Natori, H. (2000). A method for tracking camera motion of real endoscope by using virtual endoscopy system. In SPIE (Vol. 3978, pp. 122–133).
Pérez, P., Gangnet, M., & Blake, A. (2003). Poisson image editing. ACM Transactions on Graphics, 22(3), 313–318.
Rai, L., & Higgins, W. E. (2006). Image-based rendering method for mapping endoscopic video onto ct-based endoluminal views. In SPIE (Vol. 6141, pp. 1–12).
Reeff, M., Gerhard, F., Cattin, P., & Szkely, G. (2006). Mosaicing of endoscopic placenta images. In C. Hochberger & R. Liskowsky (Eds.), Lecture notes in infomatics: Vol. P-93. Informatik 2006. Informatik für menschen (pp. 467–474).
Rousso, B., Peleg, S., Finci, I., & Rav-Acha, A. (1998). Universal mosaicing using pipe projection. In Intl. conf. on comp. vision (pp. 945–952).
Seibel, E. J., Johnston, R. S., & Mellville, C. D. (2006). A full-color scanning fiber endoscope. In Proc. SPIE, opt. fibers sens. for med. diagnost. treatment appl. VI (Vol. 6083, pp. 9–16).
Seitz, S. M., & Kim, J. (2002). The space of all stereo images. International Journal of Computer Vision, 48(1), 21–38.
Seshamani, S., Lau, W., & Hager, G. D. (2006). Real-time endoscopic mosaicking. In MICCAI (pp. 355–363).
Szeliski, R. (1996). Video mosaics for virtual environments. IEEE Computer Graphics and Applications, 16(2), 22–30.
Szeliski, R., & Shum, H. Y. (1997). Creating full view panoramic image mosaics and environment maps. In: SIGGRAPH ’97 (pp. 251–258).
Truong, T., Kitasaka, T., Mori, K., & Suenaga, Y. (2006). Fast and accurate tract unfolding based on stable volumetric image deformation. In SPIE (Vol. 6143, pp. 412–423).
Vercauteren, T., Perchant, A., Malandain, G., Pennec, X., & Ayache, N. (2006). Robust mosaicing with correction of motion distortions and tissue deformations for in vivo fibered microscopy. Medical Image Analysis, 10(5), 673–692.
Warmath, J. R., Cao, Z., Bao, P., Herline, A. J., & Galloway, R. L. Jr. (2005). Semi-automatic staging system for rectal cancer using spatially oriented unwrapped endorectal ultrasound. In SPIE (Vol. 5744, pp. 425–434).
Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer, C. E., & Salesin, D. H. (1997). Multiperspective panoramas for cel animation. In SIGGRAPH ’97 (pp. 243–250).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Open Access This is an open access article distributed under the terms of the Creative Commons Attribution Noncommercial License (https://creativecommons.org/licenses/by-nc/2.0), which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Carroll, R.E., Seitz, S.M. Rectified Surface Mosaics. Int J Comput Vis 85, 307–315 (2009). https://doi.org/10.1007/s11263-009-0264-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0264-7