Abstract
We present an information theoretic approach to define the problem of structure from motion (SfM) as a blind source separation one. Given that for almost all practical joint densities of shape points, the marginal densities are non-Gaussian, we show how higher-order statistics can be used to provide improvements in shape estimates over the methods of factorization via Singular Value Decomposition (SVD), bundle adjustment and Bayesian approaches. Previous techniques have either explicitly or implicitly used only second-order statistics in models of shape or noise. A further advantage of viewing SfM as a blind source problem is that it easily allows for the inclusion of noise and shape models, resulting in Maximum Likelihood (ML) or Maximum a Posteriori (MAP) shape and motion estimates. A key result is that the blind source separation approach has the ability to recover the motion and shape matrices without the need to explicitly know the motion or shape pdf. We demonstrate that it suffices to know whether the pdf is sub- or super-Gaussian (i.e., semi-parametric estimation) and derive a simple formulation to determine this from the data. We provide extensive experimental results on synthetic and real tracked points in order to quantify the improvement obtained from this technique.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belouchrani, A., Abed-Meraim, K., Cardoso, J.-F., & Moulines, E. (1997). A blind source separation technique using second-order statistics. IEEE Transactions on Signal Processing, 45(2), 434–444.
Brand, M. (2005). A direct method for 3D factorization of nonrigid motion observed in 2D. In Proc. IEEE conference on computer vision and pattern recognition, San Diego (CA) (Vol. 2, pp. 122–128). Los Alamitos: IEEE Comput. Soc.
Buchanan, A. M., & Fitzgibbon, A. W. (2005). Damped Newton algorithms for matrix factorization with missing data. In Proc. IEEE computer society conference on computer vision and pattern recognition (Vol. 2, pp. 316–322). Los Alamitos: IEEE Comput. Soc.
Chen, P., & Suter, D. (2004). Recovering the missing components in a large noisy low-rank matrix: application to SFM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1051–1063.
Cichocki, A., Douglas, S., & Amari, S. (1998). Robust techniques for independent component analysis (ICA) with noisy data. Neurocomputing, 22, 113–129.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal Royal Statistical Society, 30(1), 1–38.
Ding, L., & Martinez, A. M. (2008). Precise detailed detection of faces and facial features. In Proc. IEEE conference on computer vision and patter recognition, Anchorage (AK). Los Alamitos: IEEE Comput. Soc.
Eriksson, J., & Koivunen, V. (2004). Identifiability, separability, and uniqueness of linear ICA models. IEEE Signal Processing Letters, 11(7), 601–604.
Faugeras, O. (2001). Three-dimensional computer vision. Cambridge: MIT Press.
Fitzgibbon, A., & Zisserman, A. (1998). Automatic 3D model acquisition and generation of new images from video sequences. In Proc. European signal processing conference (pp. 1261–1269), Rhodes, Greece.
Forsyth, D., Ioffe, S., & Haddon, J. (1999). Bayesian structure from motion. In Proc. IEEE international conference on computer vision (ICCV’99), Corfu, Greece (pp. 660–665). Los Alamitos: IEEE Comput. Soc.
Fortuna, J., & Martinez, A. M. (2006). A blind source separation approach to structure from motion. In Proc. third international symposium on 3D data processing, visualization and transmission (pp. 145–152), Chapel Hill (NC).
Fujiki, J., Akaho, S., & Murata, N. (2004). Nonlinear PCA/ICA for the structure from motion problem. Lecture Notes in Computer Science, 3195, 150–757.
Haykin, S. (ed.) (2000). Unsupervised adaptive filtering, vol. 1: blind source separation. New York: Wiley.
Hyvarinen, A. (1998). Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood. Neurocomputing, 22, 49–67.
Hyvarinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley.
Irani, M., & Anandan, P. (2002). Factorization with uncertainty. International Journal of Computer Vision, 49(2), 101–116.
Jia, H., & Martinez, A. M. (2010). Low-rank matrix fitting based on subspace perturbation analysis with applications to structure from motion. IEEE Transactions an Pattern Analysis and Machine Intelligence, 31(5), 841–854.
Kahl, F., & Heyden, A. (1999). Affine structure and motion from points, lines, and conics. International Journal of Computer Vision, 33(3), 163–180.
Kahl, F., & Heyden, A. (2001). Auto-calibration and Euclidean reconstruction from continuous motion. In Proc. IEEE international conference on computer vision, Vancouver, Canada (pp. 572–577). Los Alamitos: IEEE Comput. Soc.
Le cam, L. (1986). The central limit theorem around 1935. Statistical Science, 1(1), 78–96.
Lee, T., Girolami, M., & Sejnowski, T. (1999a). Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation, 11, 417–441.
Lee, T., Lewicki, M., Girolami, M., & Sejnowski, T. (1999b). Blind source separation of more sources than mixtures using overcomplete representations. IEEE Signal Processing Letters, 6(4), 87–90.
Morris, D., & Kanade, T. (1998). A unified factorization algorithm for points, line segments and planes with uncertainty models. In Proceedings IEEE international conference on computer vision, Bombay, India (pp. 696–702). Los Alamitos: IEEE Comput. Soc.
Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9(2), 137–154.
Triggs, B., McLauchlan, P., Hartley, R., & Fitzgibbon, A. (2000). Bundle adjustment—a modern synthesis. In Vision algorithms: theory and practice (pp. 298–375). Berlin: Springer.
Zhang, L., Curless, B., Hertzmann, A., & Seitz, S. (2003). Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multi-view stereo. In Proc. 9th IEEE international conference on computer vision, Nice, France (pp. 618–625). Los Alamitos: IEEE Comput. Soc.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fortuna, J., Martinez, A.M. Rigid Structure from Motion from a Blind Source Separation Perspective. Int J Comput Vis 88, 404–424 (2010). https://doi.org/10.1007/s11263-009-0313-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-009-0313-2