Skip to main content
View expanded cover

International Conference on Advanced Concepts for Intelligent Vision Systems

ACIVS 2020: Advanced Concepts for Intelligent Vision Systems pp 492–503Cite as

SuperNCN: Neighbourhood Consensus Network for Robust Outdoor Scenes Matching

  • 973 Accesses

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12002)

Abstract

In this paper, we present a framework for computing dense keypoint correspondences between images under strong scene appearance changes. Traditional methods, based on nearest neighbour search in the feature descriptor space, perform poorly when environmental conditions vary, e.g. when images are taken at different times of the day or seasons. Our method improves finding keypoint correspondences in such difficult conditions. First, we use Neighbourhood Consensus Networks to build spatially consistent matching grid between two images at a coarse scale. Then, we apply Superpoint-like corner detector to achieve pixel-level accuracy. Both parts use features learned with domain adaptation to increase robustness against strong scene appearance variations. The framework has been tested on a RobotCar Seasons dataset, proving large improvement on pose estimation task under challenging environmental conditions.

Keywords

  • Feature matching
  • Pose estimation
  • Domain adaptation

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-40605-9_42
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   79.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-40605-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   99.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.

References

  1. Alcantarilla, P.F., Solutions, T.: Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Trans. Patt. Anal. Mach. Intell 34(7), 1281–1298 (2011)

    Google Scholar 

  2. Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: GMS: grid-based motion statistics for fast, ultra-robust feature correspondence. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4181–4190 (2017)

    Google Scholar 

  3. Chopra, S., Hadsell, R., LeCun, Y., et al.: Learning a similarity metric discriminatively, with application to face verification. In: CVPR (1), pp. 539–546 (2005)

    Google Scholar 

  4. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2018)

    Google Scholar 

  5. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    MathSciNet  CrossRef  Google Scholar 

  6. Ganin, Y., Lempitsky, V.: Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, vol. 37, pp. 1180–1189. JMLR.org (2015)

    Google Scholar 

  7. Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2030–2096 (2016)

    MathSciNet  Google Scholar 

  8. Kramer, S.: A bayesian perspective on why the EKF fails in passive tracking. In: Proceedings of the IEEE 1996 National Aerospace and Electronics Conference NAECON 1996, vol. 1, pp. 98–101. IEEE (1996)

    Google Scholar 

  9. Lowe, D.G.: Object recognition from local scale-invariant features. In: Proceedings of the International Conference on Computer Vision, ICCV 1999, vol. 2, p. 1150. IEEE Computer Society, Washington, DC (1999)

    Google Scholar 

  10. Maddern, W., Pascoe, G., Linegar, C., Newman, P.: 1 Year, 1000km: the Oxford RobotCar dataset. Int. J. Robot. Res. (IJRR) 36(1), 3–15 (2017)

    CrossRef  Google Scholar 

  11. Melekhov, I., Tiulpin, A., Sattler, T., Pollefeys, M., Rahtu, E., Kannala, J.: DGC-net: dense geometric correspondence network. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1034–1042. IEEE (2019)

    Google Scholar 

  12. Nistér, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 0756–777 (2004)

    CrossRef  Google Scholar 

  13. Ranjan, A., Black, M.J.: Optical flow estimation using a spatial pyramid network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4161–4170 (2017)

    Google Scholar 

  14. Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: Deepmatching: hierarchical deformable dense matching. Int. J. Comput. Vision 120(3), 300–323 (2016)

    MathSciNet  CrossRef  Google Scholar 

  15. Rocco, I., Cimpoi, M., Arandjelović, R., Torii, A., Pajdla, T., Sivic, J.: Neighbourhood consensus networks. In: Proceedings of the 32nd Conference on Neural Information Processing Systems (2018)

    Google Scholar 

  16. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: Orb: an efficient alternative to sift or surf. In: Proceedings of the 2011 International Conference on Computer Vision, ICCV 2011, pp. 2564–2571. IEEE Computer Society, Washington, DC (2011)

    Google Scholar 

  17. Sattler, T., et al.: Benchmarking 6dof outdoor visual localization in changing conditions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8601–8610 (2018)

    Google Scholar 

  18. Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)

    Google Scholar 

  19. Trzcinski, T., Komorowski, J., Dabala, L., Czarnota, K., Kurzejamski, G., Lynen, S.: Scone: siamese constellation embedding descriptor for image matching. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)

    Google Scholar 

  20. Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1385–1392 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computer Science, Warsaw University of Technology, Warsaw, Poland

    Grzegorz Kurzejamski, Jacek Komorowski, Lukasz Dabala, Konrad Czarnota & Tomasz Trzcinski

  2. Google, Zurich, Switzerland

    Simon Lynen

Authors
  1. Grzegorz Kurzejamski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacek Komorowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lukasz Dabala
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Konrad Czarnota
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Simon Lynen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tomasz Trzcinski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jacek Komorowski .

Editor information

Editors and Affiliations

  1. DGA, Paris, France

    Jacques Blanc-Talon

  2. University of Auckland, Auckland, New Zealand

    Patrice Delmas

  3. Ghent University, Ghent, Belgium

    Prof. Wilfried Philips

  4. CSIRO, Canberra, Australia

    Dan Popescu

  5. University of Antwerp, Wilrijk, Belgium

    Paul Scheunders

Rights and permissions

Reprints and Permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Kurzejamski, G., Komorowski, J., Dabala, L., Czarnota, K., Lynen, S., Trzcinski, T. (2020). SuperNCN: Neighbourhood Consensus Network for Robust Outdoor Scenes Matching. In: Blanc-Talon, J., Delmas, P., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2020. Lecture Notes in Computer Science(), vol 12002. Springer, Cham. https://doi.org/10.1007/978-3-030-40605-9_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-40605-9_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-40604-2

  • Online ISBN: 978-3-030-40605-9

  • eBook Packages: Computer ScienceComputer Science (R0)