Skip to main content
View expanded cover

European Conference on Computer Vision

ECCV 2018: Computer Vision – ECCV 2018 Workshops pp 401–413Cite as

SConE: Siamese Constellation Embedding Descriptor for Image Matching

Part of the Lecture Notes in Computer Science book series (LNIP,volume 11129)

Abstract

Numerous computer vision applications rely on local feature descriptors, such as SIFT, SURF or FREAK, for image matching. Although their local character makes image matching processes more robust to occlusions, it often leads to geometrically inconsistent keypoint matches that need to be filtered out, e.g. using RANSAC. In this paper we propose a novel, more discriminative, descriptor that includes not only local feature representation, but also information about the geometric layout of neighbouring keypoints. To that end, we use a Siamese architecture that learns a low-dimensional feature embedding of keypoint constellation by maximizing the distances between non-corresponding pairs of matched image patches, while minimizing it for correct matches. The 48-dimensional floating point descriptor that we train is built on top of the state-of-the-art FREAK descriptor achieves significant performance improvement over the competitors on a challenging TUM dataset.

Keywords

  • Feature descriptor
  • Image matching
  • Siamese networks

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-11009-3_24
  • Chapter length: 13 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   89.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-11009-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   119.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.

References

  1. Agarwal, S., et al.: Building Rome in a day. Commun. ACM 54(10), 105–112 (2011)

    CrossRef  Google Scholar 

  2. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. IJCV 74(1), 59–73 (2007)

    CrossRef  Google Scholar 

  3. Lynen, S., Sattler, T., Bosse, M., Hesch, J.A., Pollefeys, M., Siegwart, R.: Get out of my lab: large-scale, real-time visual-inertial localization. In: Robotics: Science and Systems (2015)

    Google Scholar 

  4. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)

    MathSciNet  CrossRef  Google Scholar 

  5. Alahi, A., Ortiz, R., Vandergheynst, P.: FREAK: fast retina keypoint. In: CVPR (2012)

    Google Scholar 

  6. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32

    CrossRef  Google Scholar 

  7. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to sift or surf. In: ICCV (2011)

    Google Scholar 

  8. Trzcinski, T., Christoudias, M., Lepetit, V., Fua, P.: Boosting binary keypoint descriptors. In: CVPR (2013)

    Google Scholar 

  9. Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Moreno-Noguer, F.: Discriminative learning of deep convolutional feature point descriptors. In: ICCV (2015)

    Google Scholar 

  10. Yi, K.M., Trulls, E., Lepetit, V., Fua, P.: LIFT: learned invariant feature transform. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 467–483. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_28

    CrossRef  Google Scholar 

  11. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: IROS (2012)

    Google Scholar 

  12. Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: MatchNet: unifying feature and metric learning for patch-based matching. In: CVPR (2015)

    Google Scholar 

  13. Loquercio, A., Dymczyk, M., Zeisl, B., Lynen, S., Gilitschenski, I., Siegwart, R.: Efficient descriptor learning for large scale localization. In: ICRA (2017)

    Google Scholar 

  14. Niepert, M., Ahmed, M., Kutzkov, K.: Learning convolutional neural networks for graphs. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, ICML 2016, vol. 48, pp. 2014–2023. JMLR.org (2016)

    Google Scholar 

  15. Defferrard, M., Bresson, X., Vandergheynst, P.: Convolutional neural networks on graphs with fast localized spectral filtering. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, NIPS 2016, USA, pp. 3844–3852. Curran Associates, Inc. (2016)

    Google Scholar 

  16. Muja, M., Lowe, D.G.: Fast matching of binary features. In: Computer and Robot Vision (CRV) (2012)

    Google Scholar 

  17. Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. TPAMI 36(11), 2227–2240 (2014)

    CrossRef  Google Scholar 

  18. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    MathSciNet  CrossRef  Google Scholar 

  19. Raguram, R., Chum, O., Pollefeys, M., Matas, J., Frahm, J.M.: USAC: a universal framework for random sample consensus. TPAMI 35(8), 2022–2038 (2013)

    CrossRef  Google Scholar 

  20. Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: GMS: grid-based motion statistics for fast, ultra-robust feature correspondence. In: CVPR (2017)

    Google Scholar 

  21. Bromley, J., Guyon, I., LeCun, Y., Säckinger, E., Shah, R.: Signature verification using a “siamese” time delay neural network. In: NIPS (1993)

    Google Scholar 

  22. Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: CVPR (2006)

    Google Scholar 

  23. Klambauer, G., Unterthiner, T., Mayr, A., Hochreiter, S.: Self-normalizing neural networks. In: Guyon, I., et al. (eds.) Advances in Neural Information Processing Systems 30, pp. 971–980. Curran Associates, Inc. (2017)

    Google Scholar 

  24. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    CrossRef  Google Scholar 

  25. https://opencv.org/

  26. Rosten, E., Drummond, T.: Machine learning for high-speed corner detection. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 430–443. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_34

    CrossRef  Google Scholar 

  27. Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: CVPR (2012)

    Google Scholar 

  28. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: DeepFlow: large displacement optical flow with deep matching. In: ICCV (2013)

    Google Scholar 

  29. Lin, W.-Y.D., Cheng, M.-M., Lu, J., Yang, H., Do, M.N., Torr, P.: Bilateral functions for global motion modeling. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8692, pp. 341–356. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10593-2_23

    CrossRef  Google Scholar 

  30. Lipman, Y., Yagev, S., Poranne, R., Jacobs, D.W., Basri, R.: Feature matching with bounded distortion. ACM Trans. Graph. 33(3), 26:1–26:14 (2014)

    CrossRef  Google Scholar 

Download references

Acknowledgement

This research was supported by Google Sponsor Research Agreement under the project “Efficient visual localization on mobile devices”.

The Titan X Pascal used for this research was donated by the NVIDIA Corporation.

Author information

Authors and Affiliations

  1. Warsaw University of Technology, Warsaw, Poland

    Tomasz Trzcinski, Jacek Komorowski, Lukasz Dabala, Konrad Czarnota & Grzegorz Kurzejamski

  2. Tooploox, Wrocław, Poland

    Tomasz Trzcinski

  3. Google, Mountain View, USA

    Simon Lynen

Authors
  1. Tomasz Trzcinski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacek Komorowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lukasz Dabala
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konrad Czarnota
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Grzegorz Kurzejamski
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Simon Lynen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacek Komorowski .

Editor information

Editors and Affiliations

  1. Technical University of Munich, Garching, Germany

    Laura Leal-Taixé

  2. Technische Universität Darmstadt, Darmstadt, Germany

    Stefan Roth

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Trzcinski, T., Komorowski, J., Dabala, L., Czarnota, K., Kurzejamski, G., Lynen, S. (2019). SConE: Siamese Constellation Embedding Descriptor for Image Matching. In: Leal-Taixé, L., Roth, S. (eds) Computer Vision – ECCV 2018 Workshops. ECCV 2018. Lecture Notes in Computer Science(), vol 11129. Springer, Cham. https://doi.org/10.1007/978-3-030-11009-3_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-11009-3_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-11008-6

  • Online ISBN: 978-3-030-11009-3

  • eBook Packages: Computer ScienceComputer Science (R0)