View-Invariant Recognition Using Corresponding Object Fragments

  • Evgeniy Bart
  • Evgeny Byvatov
  • Shimon Ullman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3022)


We develop a novel approach to view-invariant recognition and apply it to the task of recognizing face images under widely separated viewing directions. Our main contribution is a novel object representation scheme using ‘extended fragments’ that enables us to achieve a high level of recognition performance and generalization across a wide range of viewing conditions. Extended fragments are equivalence classes of image fragments that represent informative object parts under different viewing conditions. They are extracted automatically from short video sequences during learning. Using this representation, the scheme is unique in its ability to generalize from a single view of a novel object and compensate for a significant change in viewing direction without using 3D information. As a result, novel objects can be recognized from viewing directions from which they were not seen in the past. Experiments demonstrate that the scheme achieves significantly better generalization and recognition performance than previously used methods.


Mutual Information Face Recognition Recognition Performance Object Part Active Appearance Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Mundy, J., Zisserman, A.: Geometric Invariance in Computer Vision. The MIT press, Cambridge (1992)Google Scholar
  2. 2.
    Tuytelaars, T., Gool, L.V.: Wide baseline stereo matching based on local, affinely invariant regions. In: British Machine Vision Conference, pp. 412–425 (2000)Google Scholar
  3. 3.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 128–142. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  4. 4.
    Wallraven, C., Bülthoff, H.H.: Automatic acquisition of exemplar-based representations for recognition from image sequences. In: CVPR 2001 - Workshop on Models vs. Exemplars (2001)Google Scholar
  5. 5.
    Beymer, D.J.: Face recognition under varying pose. Technical Report AIM-1461, MIT Artificial Intelligence Lab (1993)Google Scholar
  6. 6.
    Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: Rockwood, A. (ed.) Siggraph 1999, Computer Graphics Proceedings, Los Angeles, pp. 187–194. Addison Wesley Longman, Amsterdam (1999)Google Scholar
  7. 7.
    Nagashima, Y., Agawa, H., Kishino, F.: 3D face model reproduction method using multi view images. In: Proceedings of the SPIE, Visual Communications and Image Processing 1991, vol. 1606, pp. 566–573 (1991)Google Scholar
  8. 8.
    Lowe, D.G.: Three-dimensional object recognition from single two-dimensional images. Artificial Intelligence 31, 355–395 (1987)CrossRefGoogle Scholar
  9. 9.
    Wiskott, L., Fellous, J.M., Krüger, N., von der Malsburg, C.: Face recognition by elastic bunch graph matching. In: Jain, L.C., Halici, U., Hayashi, I., Lee, S.B. (eds.) Intelligent Biometric Techniques in Fingerprint and Face Recognition, pp. 355–396. CRC Press, Boca Raton (1999)Google Scholar
  10. 10.
    Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. In: Burkhardt, H., Neumann, B. (eds.) ECCV 1998. LNCS, vol. 1407, pp. 484–498. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  11. 11.
    Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 71–86 (1991)CrossRefGoogle Scholar
  12. 12.
    Murase, H., Nayar, S.: Visual learning and recognition of 3-d objects from appearance. International Journal of Computer Vision 14, 5–24 (1995)CrossRefGoogle Scholar
  13. 13.
    Pentland, A., Moghaddam, B., Starner, T.: View-based and modular eigenspaces for face recognition. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA (1994)Google Scholar
  14. 14.
    Sali, E., Ullman, S.: Combining class-specific fragments for object recognition. In: British Machine Vision Conference, pp. 203–213 (1999)Google Scholar
  15. 15.
    Ullman, S., Vidal-Naquet, M., Sali, E.: Visual features of intermediate complexity and their use in classification. Nature Neuroscience 5, 682–687 (2002)Google Scholar
  16. 16.
    Agarwal, S., Roth, D.: Learning a sparse representation for object detection. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2353, pp. 113–127. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  17. 17.
    Phillips, P.J., Wechsler, H., Huang, J., Rauss, P.: The FERET database and evaluation procedure for face recognition algorithms. Image and Vision Computing 16, 295–306 (1998)CrossRefGoogle Scholar
  18. 18.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Technical Report CMU-CS-91-132, Carnegie Mellon University (1991)Google Scholar
  19. 19.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Chichester (1991)zbMATHCrossRefGoogle Scholar
  20. 20.
    Amit, Y., Geman, D.: Shape quantization and recognition with randomized trees. Neural Computation 9, 1545–1588 (1997)CrossRefGoogle Scholar
  21. 21.
    Mel, B.W.: SEEMORE: Combining color, shape and texture histogramming in a neurally-inspired approach to visual object recognition. Neural Computation 9, 777–804 (1997)CrossRefGoogle Scholar
  22. 22.
    Weber, M., Welling, M., Perona, P.: Towards automatic discovery of object categories. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 101–109 (2002)Google Scholar
  23. 23.
    Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 218–233 (2003)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  • Evgeniy Bart
    • 1
  • Evgeny Byvatov
    • 1
  • Shimon Ullman
    • 1
  1. 1.Department of Computer Science And Applied MathematicsWeizmann Institute of ScienceRehovotIsrael

Personalised recommendations