Optimal Multi-frame Correspondence with Assignment Tensors

  • R. Oliveira
  • R. Ferreira
  • J. P. Costeira
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3953)


Establishing correspondence between features of a set of images has been a long-standing issue amongst the computer vision community. We propose a method that solves the multi-frame correspondence problem by imposing a rank constraint on the observed scene, i.e. rigidity is assumed. Since our algorithm is based solely on a geometrical (global) criterion, it does not suffer from issues usually associated to local methods, such as the aperture problem.

We model feature matching by introducing the assignment tensor, which allows simultaneous feature alignment for all images, thus providing a coherent solution to the calibrated multi-frame correspondence problem in a single step of linear complexity. Also, an iterative method is presented that is able to cope with the non-calibrated case. Moreover, our method is able to seamlessly reject a large number of outliers in every image, thus also handling occlusion in an integrated manner.


Cost Function Linear Complexity Measurement Matrix Correspondence Problem Matching Candidate 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  2. 2.
    Nemhauser, G., Wolsey, L.: Integer and Cobinatorial Optimization. John Wiley & Sons, Chichester (1999)Google Scholar
  3. 3.
    Dellaert, F., et al.: Structure From Motion Without Correspondence. In: Proc. CVPR, South Carolina, USA (June 2000)Google Scholar
  4. 4.
    Fazel, M., Hindi, H., Boyd, S.: A Rank Minimization Heuristic with Application to Minimum Order System Approximation. In: Proc. ACC (June 2001)Google Scholar
  5. 5.
    Ferrari, V., Tuytelaars, T., van Gool, L.: Wide-Baseline Multiple-View Correspondences. In: Proc. ICCV (October 2003)Google Scholar
  6. 6.
    Heyden, A., Berthilsson, R., Sparr, G.: An iterative factorization method for projective structure and motion from image sequences. Image and Vision Computing(17) 13(1), 981–991 (1999)CrossRefGoogle Scholar
  7. 7.
    Irani, M., Anandan, P.: Factorization with Uncertaint. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 539–553. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  8. 8.
    Kolmogorov, V., Zabih, R.: Multi-camera Scene Reconstruction via Graph Cuts. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2352, pp. 82–96. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Maciel, J., Costeira, J.: A Global Solution to Sparse Correspondence Problems. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(2) (February 2003)Google Scholar
  10. 10.
    Martinec, D., Pajdla, T.: 3D Reconstruction by Fitting Low-Rank Matrices with Data. In: Proc. CVPR (June 2005)Google Scholar
  11. 11.
    Oliveira, R., Costeira, J., Xavier, J.: Optimal Point Correspondence through the Use of Rank Constraints. In: Proc. CVPR, San Diego, USA (June 2005)Google Scholar
  12. 12.
    Roy, S., Cox, I.: A Maximum-Flow Formulation of the N-Camera Stereo Correspondence Problem. In: Proc. ICCV (January 1998)Google Scholar
  13. 13.
    Shafique, K., Shah, M.: A Non-Iterative Greedy Algorithm for Multi-frame Point Correspondence. In: Proc. ICCV (October 2003)Google Scholar
  14. 14.
    Sturm, P., Triggs, B.: A factorization based algorithm for multi-image projective structure and motion. In: Proc. ECCV, April 1996, pp. 709–720 (1996)Google Scholar
  15. 15.
    Tomasi, C., Kanade, T.: Shape from motion from image sreams under orthography: a factorization method. IJCV 9(2), 137–154 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • R. Oliveira
    • 1
  • R. Ferreira
    • 1
  • J. P. Costeira
    • 1
  1. 1.Instituto de Sistemas e RobóticaInstituto Superior Técnico 

Personalised recommendations