Skip to main content
SpringerLink
Log in

An Analysis of the Factors Affecting Keypoint Stability in Scale-Space

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

The most popular image matching algorithm SIFT, introduced by D. Lowe a decade ago, has proven to be sufficiently scale invariant to be used in numerous applications. In practice, however, scale invariance may be weakened by various sources of error inherent to the SIFT implementation affecting the stability and accuracy of keypoint detection. The density of the sampling of the Gaussian scale-space and the level of blur in the input image are two of these sources. This article presents a numerical analysis of their impact on the extracted keypoints stability. Such an analysis has both methodological and practical implications, on how to compare feature detectors and on how to improve SIFT. We show that even with a significantly oversampled scale-space numerical errors prevent from achieving perfect stability. Usual strategies to filter out unstable detections (e.g., poorly contrasted extrema) are shown to be inefficient. We also prove that the effect of the error in the assumption on the initial blur is asymmetric and that the method is strongly degraded in the presence of aliasing or without a correct assumption on the camera blur. This analysis leads to a series of practical recommendations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. Indeed, thanks to a change of variable, we have

    $$\begin{aligned} G_c H_\lambda u_0 (\mathbf{x})&=\int _{\mathbb {R}^2} G_c(\mathbf{x}') u_0(\lambda \mathbf{x}- \lambda \mathbf{x}')d\mathbf{x}' \\&=\int _{\mathbb {R}^2} G_{\lambda c}(\mathbf{x}'') u_0(\lambda \mathbf{x}- \mathbf{x}'')d\mathbf{x}'' = H_\lambda G_{\lambda c} u_0 (\mathbf{x}). \end{aligned}$$

References

  1. Agrawal, M., Konolige, K., Blas, M.: CenSurE: center surround extremas for realtime feature detection and matching. In: ECCV, pp. 102–115. Springer, Heidelberg (2008)

  2. Ancuti, C., Bekaert, P.: SIFT-CCH: increasing the SIFT distinctness by color co-occurrence histograms. In: ISPA. 5th International Symposium on IEEE, pp. 130–135. (2007)

  3. Bay, H., Tuytelaars, T., van Gool, L.: SURF: Speeded up robust features. In: ECCV (2006)

  4. Brown, M., Lowe, D.: Automatic panoramic image stitching using invariant features. IJCV 74(1), 59–73 (2007)

    Article  Google Scholar 

  5. Brown, M., Szeliski, R., Winder, S.: Multi-image matching using multi-scale oriented patches. In: CVPR (2005)

  6. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust independent elementary features. In: ECCV. pp. 778–792. Springer, Heidelberg (2010)

  7. Chen, J., Shan, S., He, C., Zhao, G., Pietikainen, M., Chen, X., Gao, W.: WLD: a robust local image descriptor. PAMI 32(9), 1705–1720 (2010)

    Article  Google Scholar 

  8. Cordes, K., Muller, O., Rosenhahn, B., Ostermann, J.: HALF-SIFT: High-accurate Localized Features for SIFT. In: CVPR Workshops. pp. 31–38 (2009)

  9. Delbracio, M., Musé, P., Almansa, A.: Non-parametric sub-pixel local point spread function estimation. IPOL (2012)

  10. Dickscheid, T., Schindler, F., Förstner, W.: Coding images with local features. IJCV 94(2), 154–174 (2011)

    Article  MATH  Google Scholar 

  11. Florack, L.: A spatio-frequency trade-off scale for scale-space filtering. IEEE Trans. Pattern Anal. Mach. Intell. 22(9), 1050–1055 (2000)

    Article  Google Scholar 

  12. Förstner, W., Dickscheid, T., Schindler, F.: Detecting interpretable and accurate scale-invariant keypoints. In: ICCV (2009)

  13. Grabner, M., Grabner, H., Bischof, H.: Fast approximated SIFT. In: ACCV, pp. 918–927. Springer, Heidelberg (2006)

  14. Ke, Y., Sukthankar, R.: PCA-SIFT: a more distinctive representation for local image descriptors. In: CVPR (2004)

  15. Leutenegger, S., Chli, M., Siegwart, R.: BRISK: binary robust invariant scalable keypoints. In: ICCV (2011)

  16. Lindeberg, T.: Scale-Space Theory in Computer Vision. Springer, New York (1993)

    MATH  Google Scholar 

  17. Liu, C., Yuen, J., Torralba, A., Sivic, J., Freeman, W.: SIFT Flow: Dense correspondence across different scenes. In: ECCV, pp. 28–42. Springer, Heidelberg (2008)

  18. Loncomilla, P., Ruiz-del Solar, J.: Improving sift-based object recognition for robot applications. In: Image Analysis and Processing ICIAP 2005, vol. 3617, pp. 1084–1092. Springer, Berlin (2005). http://dx.doi.org/10.1007/11553595_133

  19. Lowe, D.: Object recognition from local scale-invariant features. In: ICCV (1999)

  20. Lowe, D.: Distinctive image features from scale-invariant keypoints. IJCV 60, 91–110 (2004)

    Article  Google Scholar 

  21. Mainali, P., Lafruit, G., Yang, Q., Geelen, B., Van Gool, L., Lauwereins, R.: SIFER: scale-invariant feature detector with error resilience. IJCV 104(2), 172–197 (2013)

    Article  MATH  Google Scholar 

  22. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. PAMI 27(10), 1615–1630 (2005)

    Article  Google Scholar 

  23. Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. IJCV 65(1–2), 43–72 (2005)

    Article  Google Scholar 

  24. Morel, J.M., Yu, G.: Is SIFT scale invariant? Inverse Probl. Imaging 5(1), 115–136 (2011)

    Article  MATH  Google Scholar 

  25. Moreno, P., Bernardino, A., Santos-Victor, J.: Improving the SIFT descriptor with smooth derivative filters. Pattern Recognit. Lett. 30(1), 18–26 (2009)

    Article  Google Scholar 

  26. Pele, O., Werman, M.: A linear time histogram metric for improved SIFT matching. In: ECCV. pp. 495–508. Springer, Heidelberg (2008)

  27. Rabin, J., Delon, J., Gousseau, Y.: A statistical approach to the matching of local features. SIAM J. Imaging Sci. 2(3), 931–958 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  28. Rey-Otero, I., Delbracio, M.: Anatomy of the SIFT method. Image Process. Line 4, 370–396 (2014)

    Article  Google Scholar 

  29. Rey-Otero, I., Delbracio, M.: Computing an exact gaussian scale-space. Image Process. Line 6, 8–26 (2016)

    Article  Google Scholar 

  30. Rey-Otero, I., Morel, J.M., Delbracio, M.: An analysis of scale-space sampling in SIFT. In: Image processing (ICIP), 2014 IEEE international conference. pp. 4847–4851. (2014)

  31. Riggi, F., Toews, M., Arbel, T.: Fundamental matrix estimation via TIP-transfer of invariant parameters. In: ICPR (2006)

  32. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: ICCV (2011)

  33. Sadek, R.: Some problems on temporally consistent video editing and object recognition. Ph.D. thesis, Universitat Pompeu Fabra (2012)

  34. Sadek, R., Constantinopoulos, C., Meinhardt, E., Ballester, C., Caselles, V.: On affine invariant descriptors related to SIFT. SIAM 5(2), 652–687 (2012)

    MathSciNet  MATH  Google Scholar 

  35. Strecha, C., von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: CVPR (2008)

  36. Tola, E., Lepetit, V., Fua, P.: A fast local descriptor for dense matching. In: CVPR (2008)

  37. Tola, E., Lepetit, V., Fua, P.: DAISY: an efficient dense descriptor applied to wide-baseline stereo. PAMI 32(5), 815–830 (2010)

    Article  Google Scholar 

  38. Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. 3(3), 177–280 (2008)

    Article  Google Scholar 

  39. Van De Sande, K., Gevers, T., Snoek, C.: Evaluating color descriptors for object and scene recognition. PAMI 32(9), 1582–1596 (2010)

    Article  Google Scholar 

  40. Vedaldi, A., Fulkerson, B.: VLFeat: An open and portable library of computer vision algorithms. In: Proceedings of the 18th ACM international conference on multimed (2010)

  41. Weickert, J., Ishikawa, S., Imiya, A.: Linear scale-space has first been proposed in Japan. J. Math. Imaging Vis. 10(3), 237–252 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  42. Winder, S., Brown, M.: Learning local image descriptors. In: CVPR (2007)

  43. Winder, S., Hua, G., Brown, M.: Picking the best DAISY. In: CVPR (2009)

  44. Zeisl, B., Georgel, P.F., Schweiger, F., Steinbach, E.G., Navab, N., Munich, G.: Estimation of location uncertainty for scale invariant features points. In: BMVC. pp. 1–12 (2009)

Download references

Author information

Authors and Affiliations

  1. CMLA, ENS-Cachan, Cachan, France

    Ives Rey-Otero & Jean-Michel Morel

  2. ECE, Duke University, Durham, USA

    Mauricio Delbracio

Authors
  1. Ives Rey-Otero
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Michel Morel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mauricio Delbracio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ives Rey-Otero.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rey-Otero, I., Morel, JM. & Delbracio, M. An Analysis of the Factors Affecting Keypoint Stability in Scale-Space. J Math Imaging Vis 56, 554–572 (2016). https://doi.org/10.1007/s10851-016-0657-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-016-0657-5

Keywords

Search

Navigation