Incorporating Constraints and Prior Knowledge into Factorization Algorithms – An Application to 3D Recovery
Matrix factorization is a fundamental building block in many computer vision and machine learning algorithms. In this work we focus on the problem of ”structure from motion” in which one wishes to recover the camera motion and the 3D coordinates of certain points given their 2D locations. This problem may be reduced to a low rank factorization problem. When all the 2D locations are known, singular value decomposition yields a least squares factorization of the measurements matrix. In realistic scenarios this assumption does not hold: some of the data is missing, the measurements have correlated noise, and the scene may contain multiple objects. Under these conditions, most existing factorization algorithms fail while human perception is relatively unchanged. In this work we present an EM algorithm for matrix factorization that takes advantage of prior information and imposes strict constraints on the resulting matrix factors. We present results on challenging sequences.
KeywordsMeasurement Matrix Neural Information Processing System Temporal Coherence Factorization Algorithm Structure From Motion
Unable to display preview. Download preview PDF.
- 3.Costeira, J., Kanade, T.: A multi-body factorization method for motion analysis. In: Proceedings of the International Conference on Computer Vision (ICCV), pp. 1071–1076 (1995)Google Scholar
- 5.Gear, C.W.: Multibody grouping from motion images. Inernational Journal of Computer Vision (IJCV), 133–150 (1998)Google Scholar
- 6.Gelb, A. (ed.): Applied Optimal Estimation. MIT Press, Cambridge (1974)Google Scholar
- 7.Gruber, A., Weiss, Y.: Factorization with uncertainty and missing data: Exploiting temporal coherence. In: Proceedings of Neural Information Processing Systems (NIPS) (2003)Google Scholar
- 8.Gruber, A., Weiss, Y.: Multibody factorization with uncertainty and missing data using the EM algorithm. In: Proceedings of Computer Vision and Pattern Recognition (CVPR) (2004)Google Scholar
- 10.Jacobs, D.: Linear fitting with missing data: Applications to structure-from-motion and to characterizing intensity images. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), pp. 206–212 (1997)Google Scholar
- 11.Morris, D.D., Kanade, T.: A unified factorization algorithm for points, line segments and planes with uncertain models. In: Proceedings of the International Conference on Computer Vision (ICCV), January 1999, pp. 696–702 (1999)Google Scholar
- 12.Roweis, S.: EM algorithms for PCA and SPCA. In: Proceedings of Neural Information Processing Systems (NIPS), pp. 431–437 (1997)Google Scholar
- 14.Shum, H.Y., Ikeuchi, K., Reddy, R.: Principal component analysis with missing data and its application to polyhedral object modeling. IEEE Transactions on Pattern Analysis and Machine Intellignece (PAMI), 854–867 (September 1995)Google Scholar
- 16.Ullman, S.: The interpertation of visual motion. MIT Press, Cambridge (1979)Google Scholar
- 17.Zelnik-Manor, L., Machline, M., Irani, M.: Multi-body segmentation: Revisiting motion consistency. In: Workshop on Vision and Modeling of Dynamic Scenes, with ECCV (2002)Google Scholar