International Journal of Computer Vision

, Volume 88, Issue 3, pp 404–424 | Cite as

Rigid Structure from Motion from a Blind Source Separation Perspective

  • Jeff Fortuna
  • Aleix M. Martinez


We present an information theoretic approach to define the problem of structure from motion (SfM) as a blind source separation one. Given that for almost all practical joint densities of shape points, the marginal densities are non-Gaussian, we show how higher-order statistics can be used to provide improvements in shape estimates over the methods of factorization via Singular Value Decomposition (SVD), bundle adjustment and Bayesian approaches. Previous techniques have either explicitly or implicitly used only second-order statistics in models of shape or noise. A further advantage of viewing SfM as a blind source problem is that it easily allows for the inclusion of noise and shape models, resulting in Maximum Likelihood (ML) or Maximum a Posteriori (MAP) shape and motion estimates. A key result is that the blind source separation approach has the ability to recover the motion and shape matrices without the need to explicitly know the motion or shape pdf. We demonstrate that it suffices to know whether the pdf is sub- or super-Gaussian (i.e., semi-parametric estimation) and derive a simple formulation to determine this from the data. We provide extensive experimental results on synthetic and real tracked points in order to quantify the improvement obtained from this technique.

Structure from motion Bundle adjustment Blind source separation Subspace analysis Bayesian analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Belouchrani, A., Abed-Meraim, K., Cardoso, J.-F., & Moulines, E. (1997). A blind source separation technique using second-order statistics. IEEE Transactions on Signal Processing, 45(2), 434–444. CrossRefGoogle Scholar
  2. Brand, M. (2005). A direct method for 3D factorization of nonrigid motion observed in 2D. In Proc. IEEE conference on computer vision and pattern recognition, San Diego (CA) (Vol. 2, pp. 122–128). Los Alamitos: IEEE Comput. Soc. Google Scholar
  3. Buchanan, A. M., & Fitzgibbon, A. W. (2005). Damped Newton algorithms for matrix factorization with missing data. In Proc. IEEE computer society conference on computer vision and pattern recognition (Vol. 2, pp. 316–322). Los Alamitos: IEEE Comput. Soc. Google Scholar
  4. Chen, P., & Suter, D. (2004). Recovering the missing components in a large noisy low-rank matrix: application to SFM. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(8), 1051–1063. CrossRefGoogle Scholar
  5. Cichocki, A., Douglas, S., & Amari, S. (1998). Robust techniques for independent component analysis (ICA) with noisy data. Neurocomputing, 22, 113–129. zbMATHCrossRefGoogle Scholar
  6. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal Royal Statistical Society, 30(1), 1–38. MathSciNetGoogle Scholar
  7. Ding, L., & Martinez, A. M. (2008). Precise detailed detection of faces and facial features. In Proc. IEEE conference on computer vision and patter recognition, Anchorage (AK). Los Alamitos: IEEE Comput. Soc. Google Scholar
  8. Eriksson, J., & Koivunen, V. (2004). Identifiability, separability, and uniqueness of linear ICA models. IEEE Signal Processing Letters, 11(7), 601–604. CrossRefGoogle Scholar
  9. Faugeras, O. (2001). Three-dimensional computer vision. Cambridge: MIT Press. Google Scholar
  10. Fitzgibbon, A., & Zisserman, A. (1998). Automatic 3D model acquisition and generation of new images from video sequences. In Proc. European signal processing conference (pp. 1261–1269), Rhodes, Greece. Google Scholar
  11. Forsyth, D., Ioffe, S., & Haddon, J. (1999). Bayesian structure from motion. In Proc. IEEE international conference on computer vision (ICCV’99), Corfu, Greece (pp. 660–665). Los Alamitos: IEEE Comput. Soc. Google Scholar
  12. Fortuna, J., & Martinez, A. M. (2006). A blind source separation approach to structure from motion. In Proc. third international symposium on 3D data processing, visualization and transmission (pp. 145–152), Chapel Hill (NC). Google Scholar
  13. Fujiki, J., Akaho, S., & Murata, N. (2004). Nonlinear PCA/ICA for the structure from motion problem. Lecture Notes in Computer Science, 3195, 150–757. Google Scholar
  14. Haykin, S. (ed.) (2000). Unsupervised adaptive filtering, vol. 1: blind source separation. New York: Wiley. Google Scholar
  15. Hyvarinen, A. (1998). Independent component analysis in the presence of Gaussian noise by maximizing joint likelihood. Neurocomputing, 22, 49–67. CrossRefGoogle Scholar
  16. Hyvarinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley. CrossRefGoogle Scholar
  17. Irani, M., & Anandan, P. (2002). Factorization with uncertainty. International Journal of Computer Vision, 49(2), 101–116. zbMATHCrossRefGoogle Scholar
  18. Jia, H., & Martinez, A. M. (2010). Low-rank matrix fitting based on subspace perturbation analysis with applications to structure from motion. IEEE Transactions an Pattern Analysis and Machine Intelligence, 31(5), 841–854. Google Scholar
  19. Kahl, F., & Heyden, A. (1999). Affine structure and motion from points, lines, and conics. International Journal of Computer Vision, 33(3), 163–180. CrossRefGoogle Scholar
  20. Kahl, F., & Heyden, A. (2001). Auto-calibration and Euclidean reconstruction from continuous motion. In Proc. IEEE international conference on computer vision, Vancouver, Canada (pp. 572–577). Los Alamitos: IEEE Comput. Soc. Google Scholar
  21. Le cam, L. (1986). The central limit theorem around 1935. Statistical Science, 1(1), 78–96. zbMATHCrossRefMathSciNetGoogle Scholar
  22. Lee, T., Girolami, M., & Sejnowski, T. (1999a). Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources. Neural Computation, 11, 417–441. CrossRefGoogle Scholar
  23. Lee, T., Lewicki, M., Girolami, M., & Sejnowski, T. (1999b). Blind source separation of more sources than mixtures using overcomplete representations. IEEE Signal Processing Letters, 6(4), 87–90. CrossRefGoogle Scholar
  24. Morris, D., & Kanade, T. (1998). A unified factorization algorithm for points, line segments and planes with uncertainty models. In Proceedings IEEE international conference on computer vision, Bombay, India (pp. 696–702). Los Alamitos: IEEE Comput. Soc. Google Scholar
  25. Tomasi, C., & Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9(2), 137–154. CrossRefGoogle Scholar
  26. Triggs, B., McLauchlan, P., Hartley, R., & Fitzgibbon, A. (2000). Bundle adjustment—a modern synthesis. In Vision algorithms: theory and practice (pp. 298–375). Berlin: Springer. CrossRefGoogle Scholar
  27. Zhang, L., Curless, B., Hertzmann, A., & Seitz, S. (2003). Shape and motion under varying illumination: unifying structure from motion, photometric stereo, and multi-view stereo. In Proc. 9th IEEE international conference on computer vision, Nice, France (pp. 618–625). Los Alamitos: IEEE Comput. Soc. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Dept. Electrical and Computer EngineeringThe Ohio State UniversityColumbusUSA

Personalised recommendations