Abstract
We address the problem of depth and ego-motion estimation from omnidirectional images. We propose a correspondence-free structure-from-motion problem for sequences of images mapped on the 2-sphere. A novel graph-based variational framework is first proposed for depth estimation between pairs of images. The estimation is cast as a TV-L1 optimization problem that is solved by a fast graph-based algorithm. The ego-motion is then estimated directly from the depth information without explicit computation of the optical flow. Both problems are finally addressed together in an iterative algorithm that alternates between depth and ego-motion estimation for fast computation of 3D information from motion in image sequences. Experimental results demonstrate the effective performance of the proposed algorithm for 3D reconstruction from synthetic and natural omnidirectional images.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agrawal, A., Chellappa, R.: Robust ego-motion estimation and 3d model refinement using depth based parallax model. In: Proceedings of IEEE International Conference on Image Processing, vol. 4, pp. 2483–2486 (2004). doi:10.1109/ICIP.2004.1421606
Aujol, J., Gilboa, G., Chan, T., Osher, S.: Structure-texture image decomposition—modeling, algorithms, and parameter selection. Int. J. Comput. Vis. 67(1), 111–136 (2006). doi:10.1007/s11263-006-4331-z
Bagnato, L., Frossard, P., Vandergheynst, P.: Optical flow and depth from motion for omnidirectional images using a tv-l1 variational framework on graphs. In: Proceedings of IEEE International Conference on Image Processing, pp. 1469–1472 (2009). doi:10.1109/ICIP.2009.5414552
Baker, S., Nayar, S.K.: A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vis. 35, 175–196 (1999). doi:10.1023/A:1008128724364
Beauchemin, S., Barron, J.: The computation of optical flow. ACM Comput. Surv. 27(3), 433–466 (1995)
Bruss, A., Horn, B.: Passive navigation. Comput. Vis. Graph. 21(1), 3–20 (1983)
Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20(1–2), 89–97 (2004)
Daniilidis, K., Makadia, A., Bulow, T.: Image processing in catadioptric planes: spatiotemporal derivatives and optical flow computation. In: Proceedings of the Third Workshop on Omnidirectional Vision, pp. 3–10 (2002)
Faugeras, O., Luong, Q.T., Papadopoulo, T.: The Geometry of Multiple Images. MIT Press, New York (2001)
Gilboa, G., Osher, S.: Nonlocal operators with applications to image processing. Multiscale Model. Simul. 7(3), 1005–1028 (2008). doi:10.1137/070698592
Gluckman, J., Nayar, S.: Ego-motion and omnidirectional cameras. In: Proceedings of Sixth International Conference on Computer Vision, pp. 999–1005 (1998)
Hanna, K.: Direct multi-resolution estimation of ego-motion and structure from motion. In: Proceedings of the IEEE Workshop on Visual Motion, pp. 156–162 (1991)
Heeger, D., Jepson, A.: Subspace methods for recovering rigid motion. 1. algorithm and implementation. Int. J. Comput. Vis. 7(2), 95–117 (1992)
Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17(1–3), 185–203 (1981)
Horn, B., Weldon, E.: Direct methods for recovering motion. Int. J. Comput. Vis. 2(1), 51–76 (1988)
Jepson, A., Heeger, D.: A fast subspace algorithm for recovering rigid motion. In: Proceedings of the IEEE Workshop on Visual Motion, pp. 124–131 (1991). doi:10.1109/WVM.1991.212779
Lucas, B., Kanade, T.: An iterative image registration technique with an application to stereo vision. Int. Joint Conf. Artif. Intell. 81, 674–679 (1981)
Makadia, A., Geyer, C., Daniilidis, K.: Correspondence-free structure from motion. Int. J. Comput. Vis. 75(3) (2007)
Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20, 99–120 (2004)
Peyré, G., Bougleux, S., Cohen, L.: Non-local regularization of inverse problems. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) Computer Vision, ECCV 2008. Lecture Notes in Computer Science, vol. 5304, pp. 57–68. Springer, Berlin/Heidelberg (2008)
Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal. Physica D 60, 259–268 (1992)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47, 7–42 (2002)
Sinclair, D., Blake, A., Murray, D.: Robust estimation of egomotion from normal flow. Int. J. Comput. Vis. 13(1), 57–69 (1994)
Tian, T., Tomasi, C., Heeger, D.: Comparison of approaches to egomotion computation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 315–320 (1996)
Tosic, I., Bogdanova, I., Frossard, P., Vandergheynst, P.: Multiresolution motion estimation for omnidirectional images. In: Proceedings of EUSIPCO (2005)
Zach, C., Pock, T., Bischof, H.: A duality based approach for realtime tv-l1 optical flow. In: Hamprecht, F., Schnörr, C., Jähne, B. (eds.) Pattern Recognition. Lecture Notes in Computer Science, vol. 4713, pp. 214–223. Springer, Berlin/Heidelberg (2007)
Zhou, D., Scholkopf, B.: A regularization framework for learning from graph data. In: ICML Workshop on Statistical Relational Learning, pp. 132–137 (2004)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been partially supported by the Swiss National Science Foundation under Grant 200021-125651.
Rights and permissions
About this article
Cite this article
Bagnato, L., Frossard, P. & Vandergheynst, P. A Variational Framework for Structure from Motion in Omnidirectional Image Sequences. J Math Imaging Vis 41, 182–193 (2011). https://doi.org/10.1007/s10851-011-0267-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10851-011-0267-1