Bilinear Factorization via Augmented Lagrange Multipliers

  • Alessio Del Bue
  • João Xavier
  • Lourdes Agapito
  • Marco Paladini
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6314)


This paper presents a unified approach to solve different bilinear factorization problems in Computer Vision in the presence of missing data in the measurements. The problem is formulated as a constrained optimization problem where one of the factors is constrained to lie on a specific manifold. To achieve this, we introduce an equivalent reformulation of the bilinear factorization problem. This reformulation decouples the core bilinear aspect from the manifold specificity. We then tackle the resulting constrained optimization problem with Bilinear factorization via Augmented Lagrange Multipliers (BALM). The mechanics of our algorithm are such that only a projector onto the manifold constraint is needed. That is the strength and the novelty of our approach: it can handle seamlessly different Computer Vision problems. We present experiments and results for two popular factorization problems: Non-rigid Structure from Motion and Photometric Stereo.


Constrain Optimization Problem Structure From Motion Photometric Stereo Augmented Lagrange Multiplier Manifold Constraint 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-642-15561-1_21_MOESM1_ESM.wmv (11.7 mb)
Electronic Supplementary Material (11,991 KB)


  1. 1.
    Bascle, B., Blake, A.: Separability of pose and expression in facial tracking and animation. In: Proc. 6th International Conference on Computer Vision, Bombay, India, pp. 323–328 (1998)Google Scholar
  2. 2.
    Basri, R., Jacobs, D., Kemelmacher, I.: Photometric stereo with general, unknown lighting. International Journal of Computer Vision 72(3), 239–257 (2007)CrossRefGoogle Scholar
  3. 3.
    Bertsekas, D.: Constrained optimization and Lagrange multiplier methods. Academic Press, New York (1982)zbMATHGoogle Scholar
  4. 4.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head, South Carolina, June 2000, pp. 690–696 (2000)Google Scholar
  5. 5.
    Buchanan, A.M., Fitzgibbon, A.: Damped newton algorithms for matrix factorization with missing data. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Diego, California, vol. 2, pp. 316–322 (2005)Google Scholar
  6. 6.
    Chandraker, M., Kriegman, D.: Globally optimal bilinear programming for computer vision applications. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–8 (2008)Google Scholar
  7. 7.
    Conn, A., Gould, N., Sartenaer, A., Toint, P.: Convergence properties of an augmented Lagrangian algorithm for optimization with a combination of general equality and linear constraints. SIAM Journal on Optimization 6(3), 674–703 (1996)zbMATHCrossRefMathSciNetGoogle Scholar
  8. 8.
    Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. on Image Processing 13(9), 1200–1212 (2004)CrossRefGoogle Scholar
  9. 9.
    Hestenes, M.: Multiplier and gradient methods. Journal of Optimization Theory and Applications 4(5), 303–320 (1969)zbMATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Lin, Z., Chen, M., Wu, L., Ma, Y.: The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices. UIUC Technical Report UILU-ENG-09-2215 (2009)Google Scholar
  11. 11.
    Mai, F., Hung, Y.S.: Augmented lagrangian-based algorithm for projective reconstruction from multiple views with minimization of 2d reprojection error. Journal of Signal Processing Systems (2009)Google Scholar
  12. 12.
    Marques, M., Costeira, J.P.: Estimating 3D shape from degenerate sequences with missing data. In: Computer Vision and Image Understanding (2008)Google Scholar
  13. 13.
    Paladini, M., Del Bue, A., Stosic, M., Dodig, M., Xavier, J., Agapito, L.: Factorization for Non-Rigid and Articulated Structure using Metric Projections. In: Proc. IEEE Conference on Computer Vision and Pattern Recognition, Miami, Florida, pp. 2898–2905 (2009)Google Scholar
  14. 14.
    Shaji, A., Chandran, S., Suter, D.: Manifold Optimisation for Motion Factorisation. In: 19th International Conference on Pattern Recognition (ICPR 2008), pp. 1–4 (2008)Google Scholar
  15. 15.
    Tenenbaum, J., Freeman, W.: Separating style and content with bilinear models. Neural Computation 12(6), 1247–1283 (2000)CrossRefGoogle Scholar
  16. 16.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: A factorization approach. International Journal of Computer Vision 9(2) (1992)Google Scholar
  17. 17.
    Torresani, L., Hertzmann, A., Bregler., C.: Non-rigid structure-from-motion: Estimating shape and motion with hierarchical priors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 878–892 (2008)Google Scholar
  18. 18.
    Wiberg, T.: Computation of principal components when data are missing. In: COMPSTAT 1976, Proc. Comput. Stat., 2nd Symp., Berlin, pp. 229–236 (1976)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Alessio Del Bue
    • 1
  • João Xavier
    • 2
  • Lourdes Agapito
    • 3
  • Marco Paladini
    • 3
  1. 1.Istituto Italiano di TecnologiaGenovaItaly
  2. 2.ISR - Istituto Superior TécnicoLisbonPortugal
  3. 3.Queen Mary University of LondonLondonUK

Personalised recommendations