Multiple View Feature Descriptors from Image Sequences via Kernel Principal Component Analysis

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3021)


We present a method for learning feature descriptors using multiple images, motivated by the problems of mobile robot navigation and localization. The technique uses the relative simplicity of small baseline tracking in image sequences to develop descriptors suitable for the more challenging task of wide baseline matching across significant viewpoint changes. The variations in the appearance of each feature are learned using kernel principal component analysis (KPCA) over the course of image sequences. An approximate version of KPCA is applied to reduce the computational complexity of the algorithms and yield a compact representation. Our experiments demonstrate robustness to wide appearance variations on non-planar surfaces, including changes in illumination, viewpoint, scale, and geometry of the scene.


Feature Descriptor Image Patch Training Sequence Scale Invariant Feature Transform Robot Navigation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Baumberg, A.: Reliable feature matching across widely separated views. In: Proc. CVPR (2000)Google Scholar
  2. 2.
    Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces vs. Fisherfaces: recognition using class specific linear projection. In: ECCV 1996. LNCS, vol. 1065, Springer, Heidelberg (1996)Google Scholar
  3. 3.
    Bishop, C.: Neural Networks for Pattern Recognition. Oxford U. Press (1995)Google Scholar
  4. 4.
    Bunch, J., Nielsen, C.: Updating the singular value decomposition. Numerische Mathematik 31, 111–129 (1978)zbMATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Carneiro, G., Jepson, A.: Phase-based local features. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 282–296. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  6. 6.
    Cootes, T., Wheeler, G., Walker, K., Taylor, C.: View-Based Active Appearance Models. Image and Vision Computing 20, 657–664 (2002)CrossRefGoogle Scholar
  7. 7.
    Dufournaud, Y., Schmid, C., Horaud, R.: Matching images with different resolutions. In: Proc. CVPR (2000)Google Scholar
  8. 8.
    Edelman, S., Intrator, N., Poggio, T.: Complex cells and object recognition (1997) (unpublished manuscript)Google Scholar
  9. 9.
    Ferrari, V., Tuytelaars, T., Van Gool, L.: Wide-baseline muliple-view correspondences. In: Proc. CVPR, Madison, USA (June 2003)Google Scholar
  10. 10.
    Fitzgibbon, A., Zisserman, A.: Joint manifold distance: a new approach to appearance based clustering. In: Proc. CVPR (2003)Google Scholar
  11. 11.
    Frey, B., Jojic, N.: Transformed Component Analysis: Joint Estimation of Spatial Transformations and Image Components. In: Proc. ICCV (1999)Google Scholar
  12. 12.
    Harris, C., Stephens, M.: A combined corner and edge detector. In: Alvey Vision Conference (1988)Google Scholar
  13. 13.
    Jin, H., Soatto, S., Yezzi, A.: Multi-view stereo beyond Lambert. In: CVPR (2003)Google Scholar
  14. 14.
    Johnson, A., Herbert, M.: Object recognition by matching oriented points. In: CVPR (1997)Google Scholar
  15. 15.
    Koenderink, J., van Doorn, A.: Generic neighborhood operators. IEEE Trans. Pattern Analysis and Machine Intell. 14, 597–605 (1992)CrossRefGoogle Scholar
  16. 16.
    Kwok, J., Tsang, I.: The Pre-Image Problem in Kernel Methods. In: Proc. 20th Int. Conf. on Machine Learning (2003)Google Scholar
  17. 17.
    Lanckriet, G., Cristianini, N., Bartlett, P., El Ghaoui, L., Jordan, M.: Learning the kernel matrix with semi-definite programming. In: Proc. 19th Int. Conf. on Machine Learning, Sydney, Australia (2002)Google Scholar
  18. 18.
    Levy, A., Lindenbaum, M.: Sequential Karhunen-Loeve Basis Extraction and its Application to Images. IEEE Trans. on Image Processing (August 2000)Google Scholar
  19. 19.
    Lindeberg, T.: Scale-Space Theory in Computer Vision. Kluwer Academic Publishers, Boston (1994)Google Scholar
  20. 20.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Preprint, submitted to IJCV (version date: June 2003)Google Scholar
  21. 21.
    Lowe, D.: Object recognition from local scale-invariant features. In: Proc. ICCV, Corfu, Greece (September 1999)Google Scholar
  22. 22.
    Mikolajczyk, K., Schmid, C.: Indexing based on scale invariant interest points. In: Proc. 8th ICCV (2001)Google Scholar
  23. 23.
    Mikolajczyk, K., Schmid, C.: An affine invariant interest point detector. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 128–142. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  24. 24.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. In: Proc. CVPR (June 2003)Google Scholar
  25. 25.
    Pavlov, D., Chudova, D., Smyth, P.: Towards scalable support vector machines using squashing. In: Proc. Int. Conf. on Knowledge Discovery in Databases (2000)Google Scholar
  26. 26.
    Pritchett, P., Zisserman, A.: Wide baseline stereo matching. In: 6th ICCV (1998)Google Scholar
  27. 27.
    Rothganger, F., et al.: 3D object modeling and recognition using affine-invariant patches and multi-view spatial constraints. In: Proc. CVPR (June 2003)Google Scholar
  28. 28.
    Schaffalitzky, F., Zisserman, A.: Viewpoint invariant texture matching and wide baseline stereo. In: Proc. ICCV (July 2001)Google Scholar
  29. 29.
    Schaffalitzky, F., Zisserman, A.: Multi-view matching for unordered image sets, or ‘How do I organize my holiday snaps?’ ”. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 414–431. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  30. 30.
    Schölkopf, B., Smola, A.: Learning with Kernels. The MIT Press, Cambridge (2002)Google Scholar
  31. 31.
    Schölkopf, B., Smola, A., Müller, K.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation 10, 1299–1319 (1998)CrossRefGoogle Scholar
  32. 32.
    Schölkopf, B., et al.: Input Space vs. Feature Space in Kernel-Based Methods. IEEE Transactions on Neural Networks (1999)Google Scholar
  33. 33.
    Schölkopf, B., Knirsch, P., Smola, A., Burges, C.: Fast Approximation of Support Vector Kernel Expansions, and an Interpretation of Clustering as Approximation in Feature Spaces. In: DAGM Symposium Mustererkennung. LNCS, Springer, Heidelberg (1998)Google Scholar
  34. 34.
    Schmid, C., Mohr, R.: Local Greyvalue Invariants for Image Retrieval. Pattern Analysis and Machine Intelligence (1997)Google Scholar
  35. 35.
    Shi, J., Tomasi, C.: Good Features to Track. In: IEEE CVPR (1994)Google Scholar
  36. 36.
    Tell, D., Carlsson, S.: Wide Baseline Point Matching Using Affine Invariants Computed from Intensity Profiles. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 814–828. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  37. 37.
    Tomasi, C., Kanade, T.: Detection and tracking of point features. Tech. Rept. CMU-CS-91132. Pittsburgh: Carnegie Mellon U. School of Computer Science (1991)Google Scholar
  38. 38.
    Tuytelaars, T., Van Gool, L.: Wide Baseline Stereo based on Local, Affinely invariant Regions. In: British Machine Vision Conference, pp. 412–422 (2000)Google Scholar
  39. 39.
    Tuytelaars, T., Van Gool, L.: Matching Widely Separated Views based on Affine Invariant Regions. To appear in Int. J. on Computer Vision (2004)Google Scholar
  40. 40.
    Zabih, R., Woodfill, J.: Non-Parametric Local Transforms for Computing Visual Correspondence. In: Eklundh, J.-O. (ed.) ECCV 1994. LNCS, vol. 801, Springer, Heidelberg (1994)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2004

Authors and Affiliations

  1. 1.University of CaliforniaLos AngelesUSA
  2. 2.Honda Research InstituteMountain ViewUSA

Personalised recommendations