Scale-Space SIFT Flow

  • Weichao Qiu
  • Xinggang Wang
  • Xiang Bai
  • Alan Yuille
  • Zhuowen Tu

Abstract

The SIFT flow algorithm has been widely used for the image matching/ registration task and it is particularly effective in handling image pairs from similar scenes but with different object configurations. The way in which the dense SIFT features are computed at a fixed scale in the SIFT flow method might however limit its capability of dealing with scenes having great scale changes. In this work, we propose a simple, intuitive, and effective approach, Scale-Space SIFT flow, to deal with the large object scale differences. We introduce a scale field to the SIFT flow function to automatically explore the scale changes. Our approach achieves a similar performance as the SIFT flow method for natural scenes but obtains significant improvement for the images with large scale differences. Compared with a recent method that addresses a similar problem, our approach shows its advantage being more effective and efficient.

Keywords

Scale Field Scale Space Image Match Angular Error Smoothness Term 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

This work is supported by the National Natural Science Foundation of China (grant No. 61222308) and the NSF awards IIS-1216528 (IIS-1360566), IIS-0844566 (IIS-1360568), and IIS-0917141.

References

  1. 1.
    Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. Int. J. Comput. Vis. 92(1), 1–31 (2011)CrossRefGoogle Scholar
  2. 2.
    Belongie, S., Malik, J., Puzicha, J.: Shape matching and object recognition using shape contexts. Trans. Pattern Anal. Mach. Intell. 24(4), 509–522 (2002)CrossRefGoogle Scholar
  3. 3.
    Crowley, J.: A Representation for Visual Information. Ph.D. thesis, Carnegie Mellon University (1981)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 886–893. IEEE, New York (2005)Google Scholar
  5. 5.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70(1), 41–54 (2006)CrossRefGoogle Scholar
  6. 6.
    Hassner, T., Mayzels, V., Zelnik-Manor, L.: On sifts and their scales. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1522–1528. IEEE, New York (2012)Google Scholar
  7. 7.
    Kim, J., Liu, C., Sha, F., Grauman, K.: Deformable spatial pyramid matching for fast dense correspondences. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 2307–2314. IEEE, New York (2013)Google Scholar
  8. 8.
    Kokkinos, I., Yuille, A.: Scale invariance without scale selection. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE, New York (2008)Google Scholar
  9. 9.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 2169–2178. IEEE, New York (2006)Google Scholar
  10. 10.
    Lindeberg, T.: Scale-Space Theory in Computer Vision. Springer Science & Business Media, New York (1993)MATHGoogle Scholar
  11. 11.
    Lindeberg, T.: Feature detection with automatic scale selection. Int. J. Comput. Vis. 30(2), 79–116 (1998)CrossRefGoogle Scholar
  12. 12.
    Liu, H., Yan, S.: Common visual pattern discovery via spatially coherent correspondences. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1609–1616. IEEE, New York (2010)Google Scholar
  13. 13.
    Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.T.: Sift flow: Dense correspondence across different scenes. In: European Conference on Computer Vision, pp. 28–42. Springer, New York (2008)Google Scholar
  14. 14.
    Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing: label transfer via dense scene alignment. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 1972–1979. IEEE, New York (2009)Google Scholar
  15. 15.
    Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing via label transfer. Trans. Pattern Anal. Mach. Intell. 33(12), 2368–2382 (2011)CrossRefGoogle Scholar
  16. 16.
    Liu, C., Yuen, J., Torralba, A.: Sift flow: dense correspondence across scenes and its applications. Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)CrossRefGoogle Scholar
  17. 17.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  18. 18.
    Mikolajczyk, K., Schmid, C.: Scale & affine invariant interest point detectors. Int. J. Comput. Vis. 60(1), 63–86 (2004)CrossRefGoogle Scholar
  19. 19.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. Trans. Pattern Anal. Mach. Intell. 12(7), 629–639 (1990)CrossRefGoogle Scholar
  20. 20.
    Rubinstein, M., Liu, C., Freeman, W.T.: Annotation propagation in large image databases via dense image correspondence. In: European Conference on Computer Vision, pp. 85–99. Springer (2012)Google Scholar
  21. 21.
    Tola, E., Lepetit, V., Fua, P.: Daisy: An efficient dense descriptor applied to wide-baseline stereo. Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2010)CrossRefGoogle Scholar
  22. 22.
    Witkin, A.: Scale space filtering. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1019–1022 (1983)Google Scholar
  23. 23.
    Yuille, A.L., Poggio, T.A.: Scaling theorems for zero crossings. Trans. Pattern Anal. Mach. Intell. 15(1), 15–25 (1986)CrossRefGoogle Scholar
  24. 24.
    Zhao, J., Ma, J., Tian, J., Ma, J., Zhang, D.: A robust method for vector field learning with application to mismatch removing. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 2977–2984. IEEE, New York (2011)Google Scholar
  25. 25.
    Zhou, F., De la Torre, F.: Factorized graph matching. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 127–134. IEEE, New York (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Weichao Qiu
    • 1
  • Xinggang Wang
    • 1
  • Xiang Bai
    • 1
  • Alan Yuille
    • 2
  • Zhuowen Tu
    • 3
  1. 1.Department of Electronics and Information EngineeringHuazhong University of Science and TechnologyHubeiChina
  2. 2.University of California, Los AngelesLos AngelesUSA
  3. 3.University of California, San DiegoLa JollaUSA

Personalised recommendations