Abstract
Factorization using Singular Value Decomposition (SVD) is often used for recovering 3D shape and motion from feature correspondences across multiple views. SVD is powerful at finding the global solution to the associated least-square-error minimization problem. However, this is the correct error to minimize only when the x and y positional errors in the features are uncorrelated and identically distributed. But this is rarely the case in real data. Uncertainty in feature position depends on the underlying spatial intensity structure in the image, which has strong directionality to it. Hence, the proper measure to minimize is covariance-weighted squared-error (or the Mahalanobis distance). In this paper, we describe a new approach to covariance-weighted factorization, which can factor noisy feature correspondences with high degree of directional uncertainty into structure and motion. Our approach is based on transforming the raw-data into a covariance-weighted data space, where the components of noise in the different directions are uncorrelated and identically distributed. Applying SVD to the transformed data now minimizes a meaningful objective function in this new data space. This is followed by a linear but suboptimal second step to recover the shape and motion in the original data space. We empirically show that our algorithm gives very good results for varying degrees of directional uncertainty. In particular, we show that unlike other SVD-based factorization algorithms, our method does not degrade with increase in directionality of uncertainty, even in the extreme when only normal-flow data is available. It thus provides a unified approach for treating corner-like points together with points along linear structures in the image.
Similar content being viewed by others
References
Aguiar, P.M.Q. and Moura, J.M.F. 1999. Factorization as a rank 1 problem. IEEE Computer Vision and Pattern Recognition Conference 9, A:178–184
Anandan, P. 1989. A computational framework and an algorithm for the measurement of visual motion. International Journal of Computer Vision, 2:283–310.
Anandan, P. and Avidan, S. 2000. Integrating local affine into global perspective images in the joint image space. In European Conference on Computer Vision, Dublin, pp. 907–921.
Ben-Ezra, M., Peleg, S., and Werman, M. 2000. Real-time motion analysis with linear programming. International Journal of Computer Vision, 78:32–52.
Criminisi, A., Reid, I., and Zisserman, A. 1998. Duality, rigidity and planar parallax. In European Conference on Computer Vision, Freiburg.
Hanna, K. and Okamoto, N.E. 1993. Combining stereo and motion for direct estimation of scene structure. In International Conference on Computer Vision, Berlin, Germany, pp. 357–365.
Irani, M. 2002. Multi-frame correspondence estimation using sub-space constraints. International Journal of Computer Vision, 48(3):173–194 (shorter version appeared in International Conference on Computer Vision, 1999, pp. 626-633).
Irani, M. and Anandan, P. 1996. Parallax geometry of pairs of points for 3d scene analysis. In European Conference on Computer Vision, Cambridge, UK, pp. 17–30.
Irani, M. and Anandan, P. 2000. Factorization with uncertainty. In European Conference on Computer Vision, Dublin, pp. 539–553.
Irani, M., Anandan, P., and Cohen, M. 1999. Direct recovery of planar-parallax from multiple frames. In Vision Algorithms: Theory and Practice Workshop, Corfu.
Irani, M., Anandan, P., and Weinshall, D. 1998. From reference frames to reference planes: Multi-view parallax geometry and applications. In European Conference on Computer Vision, Freiburg.
Irani, M., Rousso, B., and Peleg, S. 1997. Recovery of ego-motion using region alignment. IEEE Trans. on Pattern Analysis and Machine Intelligence, 19(3):268–272.
Kanatani, K. 1996. Statistical Optimization for Geometric Computation: Theory and Practice. North-Holland: Amsterdam, The Netherlands.
Kumar, R., Anandan, P., and Hanna, K. 1994. Direct recovery of shape from multiple views: A parallax based approach. In Proc. 12th International Conference on Pattern Recognition, Elsevier Science: Amsterdam, The Netherlands, pp. 685–688.
Leedan, Y. and Meer, P. 2000. Heteroscedastic regression in computer vision: Problems with bilinear constraint. International Journal on Computer Vision, 37(2):127–150.
Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Image Understanding Workshop, pp. 121–130.
Matei, B. and Meer, P. 2000. A general method for errors-in-variables problems in computer vision. IEEE Computer Vision and Pattern Recognition Conference, 2:18–25.
Morris, D. and Kanade, T. 1998. A unified factorization algorithm for points, line segments and planes with uncertain models. International Conference on Computer Vision, pp. 696–702.
Morris, D., Kanatani, K., and Kanade, T. 1999. Uncertainty modeling for optimal structure from motion. In Vision Algorithms: Theory and Practice Workshop, Corfu, pp. 33–40.
Oliensis, J. 1999. A multi-frame structure-from-motion algorithm under perspective projection. International Journal of Computer Vision, 34(2/3):163–192.
Oliensis, J. and Genc, Y. 2001. Fast and accurate algorithms for projective multi-image structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6):546–559.
Poelman, C.J. and Kanade, T. 1997. A paraperspective factorization method for shape and motion recovery. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19:206–218.
Quan, L. and Kanade, T. 1996. A factorization method for affine structure from line correspondences. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 803–808.
Sawhney, H. 1994. 3D geometry from planar parallax. In IEEE Conference on Computer Vision and Pattern Recognition.
Shapiro, L.S. 1995. Affine Analysis of Image Sequences. Cambridge University Press: Cambridge, UK.
Shashua, A. and Navab, N. 1994. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, pp. 483–489.
Stein, G.P. and Shashua, A. 2000. Model-based brightness constraints: On direct estimation of structure and motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(9):992–1015.
Sturm, P. and Triggs, B. 1996. A factorization based algorithm for multi-image projective structure and motion. European Conference on Computer Vision, 2:709–720.
Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision, 9:137–154.
Triggs, W. 2000. Plane + parallax, tensors, and factorization. In European Conference on Computer Vision, Dublin, pp. 522–538.
Van Huffel, S. and Vandewalle, J. 1991. The Total Least Squares Problem. SIAM: Philadelphia, PA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Anandan, P., Irani, M. Factorization with Uncertainty. International Journal of Computer Vision 49, 101–116 (2002). https://doi.org/10.1023/A:1020137420717
Issue Date:
DOI: https://doi.org/10.1023/A:1020137420717