Incorporating Constraints and Prior Knowledge into Factorization Algorithms – An Application to 3D Recovery

  • Amit Gruber
  • Yair Weiss
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3940)


Matrix factorization is a fundamental building block in many computer vision and machine learning algorithms. In this work we focus on the problem of ”structure from motion” in which one wishes to recover the camera motion and the 3D coordinates of certain points given their 2D locations. This problem may be reduced to a low rank factorization problem. When all the 2D locations are known, singular value decomposition yields a least squares factorization of the measurements matrix. In realistic scenarios this assumption does not hold: some of the data is missing, the measurements have correlated noise, and the scene may contain multiple objects. Under these conditions, most existing factorization algorithms fail while human perception is relatively unchanged. In this work we present an EM algorithm for matrix factorization that takes advantage of prior information and imposes strict constraints on the resulting matrix factors. We present results on challenging sequences.


Measurement Matrix Neural Information Processing System Temporal Coherence Factorization Algorithm Structure From Motion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Andersen, R.A., Bradley, D.C.: Perception of three-dimensional structure from motion. Trends in Cognitive Sciences 2, 222–228 (1998)CrossRefGoogle Scholar
  2. 2.
    Brand, M.E.: Incremental singular value decomposition of uncertain data with missing values. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 707–720. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  3. 3.
    Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 1071–1076 (1995)Google Scholar
  4. 4.
    Deerwester, S.C., Dumais, S.T., Landauer, T.K., Furnas, G.W., Harshman, R.A.: Indexing by latent semantic analysis. Journal of the American Society of Information Science 41(6), 391–407 (1990)CrossRefGoogle Scholar
  5. 5.
    Gear, C.W.: Multibody grouping from motion images. Inernational Journal of Computer Vision (IJCV), 133–150 (1998)Google Scholar
  6. 6.
    Gelb, A. (ed.): Applied Optimal Estimation. MIT Press, Cambridge (1974)Google Scholar
  7. 7.
    Gruber, A., Weiss, Y.: Factorization with uncertainty and missing data: Exploiting temporal coherence. In: Proceedings of Neural Information Processing Systems (NIPS) (2003)Google Scholar
  8. 8.
    Gruber, A., Weiss, Y.: Multibody factorization with uncertainty and missing data using the EM algorithm. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
  9. 9.
    Irani, M., Anandan, P.: Factorization with uncertainty. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 539–553. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  10. 10.
    Jacobs, D.: Linear fitting with missing data: Applications to structure-from-motion and to characterizing intensity images. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 206–212 (1997)Google Scholar
  11. 11.
    Morris, D.D., Kanade, T.: A unified factorization algorithm for points, line segments and planes with uncertain models. In: Proceedings of the International Conference on Computer Vision (ICCV), January 1999, pp. 696–702 (1999)Google Scholar
  12. 12.
    Roweis, S.: EM algorithms for PCA and SPCA. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 431–437 (1997)Google Scholar
  13. 13.
    Rubin, D., Thayer, D.: EM algorithms for ML factor analysis. Psychometrika 47(1), 69–76 (1982)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Shum, H.Y., Ikeuchi, K., Reddy, R.: Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intellignece (PAMI), 854–867 (September 1995)Google Scholar
  15. 15.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization method. International Journal of Computer Vision (IJCV) 9(2), 137–154 (1992)CrossRefGoogle Scholar
  16. 16.
    Ullman, S.: The interpertation of visual motion. MIT Press, Cambridge (1979)Google Scholar
  17. 17.
    Zelnik-Manor, L., Machline, M., Irani, M.: Multi-body segmentation: Revisiting motion consistency. In: Workshop on Vision and Modeling of Dynamic Scenes, with ECCV (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Amit Gruber
    • 1
  • Yair Weiss
    • 1
  1. 1.School of Computer Science and EngineeringThe Hebrew University of JerusalemJerusalemIsrael

Personalised recommendations