Dense, Scale-Less Descriptors

Abstract

Establishing correspondences between two images requires matching similar image regions. To do this effectively, local representations must be designed to allow for meaningful comparisons. As we discuss in previous chapters, one such representation is the SIFT descriptor used by SIFT flow. The scale selection required to make SIFT scale invariant, however, is only known to be possible at sparse interest points, where local image information varies sufficiently. SIFT flow and similar methods consequently settle for descriptors extracted using manually determined scales, kept constant for all image pixels. In this chapter we discuss alternative representations designed to capture multiscale appearance, even in image regions where existing methods for scale selection are not effective. We show that (1) SIFTs extracted from different scales, even in low contrast areas, vary in their values and so single scale selection often results in poor matches when images show content at different scales. (2) We propose representing pixel appearances with sets of SIFTs extracted at multiple scales. Finally, (3) low-dimensional, linear subspaces are shown to accurately represent such SIFT sets. By mapping these subspaces to points we obtain a novel representation, the Scale-Less SIFT (SLS), which can be used in a dense manner, throughout the image, to represent multiscale image appearances. We demonstrate how the use of the SLS descriptor can act as an alternative to existing, single scale representations, allowing for accurate dense correspondences between images with scale-varying content.

References

  1. 1.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2001)CrossRefGoogle Scholar
  2. 2.
    Basri, R., Hassner, T., Zelnik-Manor, L.: Approximate nearest subspace search with applications to pattern recognition. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (2007)CrossRefGoogle Scholar
  3. 3.
    Basri, R., Hassner, T., Zelnik-Manor, L.: A general framework for approximate nearest subspace search. In: Proceedings of the International Conference on Computer Vision Workshop, pp. 109–116. IEEE, New York (2009)Google Scholar
  4. 4.
    Basri, R., Hassner, T., Zelnik-Manor, L.: Approximate nearest subspace search. Trans. Pattern Anal. Mach. Intell. 33(2), 266–278 (2010)CrossRefGoogle Scholar
  5. 5.
    Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008)CrossRefGoogle Scholar
  6. 6.
    Brox, T., Bregler, C., Malik, J.: Large displacement optical flow. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 41–48 (2009)Google Scholar
  7. 7.
    Bruhn, A., Weickert, J., Schnörr, C.: Lucas/kanade meets horn/schunck: combining local and global optic flow methods. Int. J. Comput. Vis. 61(3), 211–231 (2005)CrossRefGoogle Scholar
  8. 8.
    Edelman, A., Arias, T., Smith, S.: The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20, 303–353 (1998)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004) [ISBN: 0521540518]CrossRefMATHGoogle Scholar
  10. 10.
    Hassner, T., Mayzels, V., Zelnik-Manor, L.: On SIFTs and their scales. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1522–1528. IEEE, New York. http://www.openu.ac.il/home/hassner/projects/siftscales (2012)
  11. 11.
    Kokkinos, I., Yuille, A.: Scale invariance without scale selection. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1–8. Available: vision.mas.ecp.fr/Personnel/iasonas/code/distribution.zip (2008)
  12. 12.
    Lindeberg, T.: Scale-space theory: A basic tool for analysing structures at different scales. J. App. Stat. 21(2), 225–270 (1994)CrossRefGoogle Scholar
  13. 13.
    Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 79–116 (1998)CrossRefGoogle Scholar
  14. 14.
    Lindeberg, T.: Principles for automatic scale selection. Handb. Comput. Vis. Appl. 2, 239–274 (1999)Google Scholar
  15. 15.
    Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.: SIFT flow: dense correspondence across different scenes. In: European Conference on Computer Vision, pp. 28–42. people.csail.mit.edu/celiu/ECCV2008/ (2008)
  16. 16.
    Liu, C., Yuen, J., Torralba, A.: SIFT flow: dense correspondence across scenes and its applications. Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)CrossRefGoogle Scholar
  17. 17.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  18. 18.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)Google Scholar
  19. 19.
    Mikolajczyk, K.: Detection of Local Features Invariant to Affine Transformations. Ph.D. thesis, Institut National Polytechnique de Grenoble, Grenoble (2002)Google Scholar
  20. 20.
    Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)CrossRefGoogle Scholar
  21. 21.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  22. 22.
    Morel, J., Yu, G.: Is sift scale invariant? Inverse Problems Imaging (IPI) 5(1), 115–136 (2011)CrossRefMATHGoogle Scholar
  23. 23.
    Nowak, E., Jurie, F., Triggs, B.: Sampling strategies for bag-of-features image classification. In: European Conference on Computer Vision, pp. 490–503 (2006)Google Scholar
  24. 24.
    Qiu, W., Wang, X., Bai, X., Yuille, A., Tu, Z.: Scale-space sift flow. In: Proceedings of the Winter Conference on Applications of Computer Vision. IEEE, New York (2014)Google Scholar
  25. 25.
    Simon, I., Seitz, S.: A probabilistic model for object recognition, segmentation, and non-rigid correspondence. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1–7 (2007)Google Scholar
  26. 26.
    Strecha, C., Tuytelaars, T., Gool, L.: Dense matching of multiple wide-baseline views. In: Proceedings of the International Conference on Computer Vision (2003)CrossRefGoogle Scholar
  27. 27.
    Tau, M., Hassner, T.: Dense correspondences across scenes and scales. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2015)Google Scholar
  28. 28.
    Tola, E., Lepetit, V., Fua, P.: Daisy: an efficient dense descriptor applied to wide-baseline stereo. Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2009)CrossRefGoogle Scholar
  29. 29.
    Varma, M., Garg, R.: Locally invariant fractal features for statistical texture classification. In: Proceedings of the International Conference on Computer Vision, pp. 1–8 (2007)Google Scholar
  30. 30.
    Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms. In: Proceedings of the International Conference on Multimeadia, pp. 1469–1472. Available: www.vlfeat.org/ (2010)
  31. 31.
    Wolf, L., Hassner, T., Maoz, I.: Face recognition in unconstrained videos with matched background similarity. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 529–534 (2011)Google Scholar
  32. 32.
    Yang, H., Lin, W.Y., Lu, J.: Daisy filter flow: A generalized discrete approach to dense correspondences. In: Proceedings of the Conference on Computer Vision and Pattern Recognition. IEEE, New York (2014)Google Scholar
  33. 33.
    Yao, J., Cham, W.: 3D modeling and rendering from multiple wide-baseline images by match propagation. Signal Process. Image Commun. 21(6), 506–518 (2006)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Department of Mathematics and Computer ScienceThe Open University of IsraelRaananaIsrael
  2. 2.Technion Israel Institute of TechnologyHaifaIsrael

Personalised recommendations