Skip to main content
SpringerLink
Log in

Multi-condition Place Generator for Robust Place Recognition

  • Conference paper
  • First Online:
MultiMedia Modeling (MMM 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11961))

Included in the following conference series:

  • 2830 Accesses

Abstract

As an image retrieval task, visual place recognition (VPR) encounters two technical challenges: appearance variations resulted from external environment changes and the lack of cross-domain paired training data. To overcome these challenges, multi-condition place generator (MPG) is introduced for data generation. The objective of MPG is two-fold, (1) synthesizing realistic place samples corresponding to multiple conditions; (2) preserving the place identity information during the generation procedure. While MPG smooths the appearance disparities under various conditions, it also suffers image distortion. For this reason, we propose the relative quality based triplet (RQT) loss by reshaping the standard triplet loss such that it down-weights the loss assigned to low-quality images. By taking advantage of the innovations mentioned above, a condition-invariant VPR model is trained without the labeled training data. Comprehensive experiments show that our method outperforms state-of-the-art algorithms by a large margin on several challenging benchmarks.

This work was supported by the National Natural Science Foundation of China (No. 81373555), Special Fund of the Ministry of Education of China (No. 2018A11005) and Jihua Lab under Grant No.Y80311W180.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307 (2016)

    Google Scholar 

  2. Chen, Z., et al.: Deep learning features at scale for visual place recognition. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 3223–3230. IEEE (2017)

    Google Scholar 

  3. Choi, Y., Choi, M., Kim, M., Ha, J.W., Kim, S., Choo, J.: StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797 (2018)

    Google Scholar 

  4. Deng, W., Zheng, L., Ye, Q., Kang, G., Yang, Y., Jiao, J.: Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 994–1003 (2018)

    Google Scholar 

  5. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  6. Glover, A.J., Maddern, W.P., Milford, M.J., Wyeth, G.F.: FAB-MAP+ RatSLAM: appearance-based SLAM for multiple times of day. In: 2010 IEEE International Conference on Robotics and Automation, pp. 3507–3512. IEEE (2010)

    Google Scholar 

  7. Gomez-Ojeda, R., Lopez-Antequera, M., Petkov, N., Jiménez, J.G.: Training a convolutional neural network for appearance-invariant place recognition. CoRR abs/1505.07428 (2015). http://arxiv.org/abs/1505.07428

  8. Hu, Y., Wu, X., Yu, B., He, R., Sun, Z.: Pose-guided photorealistic face rotation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8398–8406 (2018)

    Google Scholar 

  9. Jiang, S., Min, W., Mei, S.: Hierarchy-dependent cross-platform multi-view feature learning for venue category prediction. IEEE Trans. Multimedia 21(6), 1609–1619 (2018)

    Article  Google Scholar 

  10. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2015)

    Google Scholar 

  11. Latif, Y., Garg, R., Milford, M., Reid, I.: Addressing challenging place recognition tasks using generative adversarial networks. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 2349–2355. IEEE (2018)

    Google Scholar 

  12. Liu, Y., Zhang, H.: Visual loop closure detection with a compact image descriptor. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1051–1056. IEEE (2012)

    Google Scholar 

  13. McManus, C., Churchill, W., Maddern, W., Stewart, A.D., Newman, P.: Shady dealings: robust, long-term visual localisation using illumination invariance. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 901–906. IEEE (2014)

    Google Scholar 

  14. Merrill, N., Huang, G.: Lightweight unsupervised deep loop closure. In: Robotics: Science and Systems XIV, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, 26–30 June 2018 (2018). http://www.roboticsproceedings.org/rss14/p32.html

  15. Milford, M., et al.: Sequence searching with deep-learnt depth for condition-and viewpoint-invariant route-based place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 18–25 (2015)

    Google Scholar 

  16. Milford, M.J., Wyeth, G.F.: SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: 2012 IEEE International Conference on Robotics and Automation, pp. 1643–1649. IEEE (2012)

    Google Scholar 

  17. Naseer, T., Burgard, W., Stachniss, C.: Robust visual localization across seasons. IEEE Trans. Rob. 34(2), 289–302 (2018)

    Article  Google Scholar 

  18. Olid, D., Fácil, J.M., Civera, J.: Single-view place recognition under seasonal changes. arXiv preprint arXiv:1808.06516 (2018)

  19. Porav, H., Maddern, W., Newman, P.: Adversarial training for adverse conditions: robust metric localisation using appearance transfer. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 1011–1018. IEEE (2018)

    Google Scholar 

  20. Sattler, T., et al.: Benchmarking 6DOF outdoor visual localization in changing conditions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8601–8610 (2018)

    Google Scholar 

  21. Singh, G., Kosecka, J.: Visual loop closing using gist descriptors in manhattan world. In: ICRA Omnidirectional Vision Workshop (2010)

    Google Scholar 

  22. Sünderhauf, N., Protzel, P.: BRIEF-Gist-closing the loop by simple means. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1234–1241. IEEE (2011)

    Google Scholar 

  23. Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11(Dec), 3371–3408 (2010)

    MathSciNet  MATH  Google Scholar 

  24. Windrim, L., Melkumyan, A., Murphy, R., Chlingaryan, A., Nieto, J.: Unsupervised feature learning for illumination robustness. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 4453–4457. IEEE (2016)

    Google Scholar 

  25. Windrim, L., Ramakrishnan, R., Melkumyan, A., Murphy, R.J.: A physics-based deep learning approach to shadow invariant representations of hyperspectral images. IEEE Trans. Image Process. 27(2), 665–677 (2018)

    Article  MathSciNet  Google Scholar 

  26. Wohlhart, P., Lepetit, V.: Learning descriptors for object recognition and 3D pose estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3109–3118 (2015)

    Google Scholar 

  27. Xiang, G., Tao, Z.: Unsupervised learning to detect loops using deep neural networks for visual SLAM system. Auton. Robots 41(1), 1–18 (2017)

    Article  Google Scholar 

  28. Yin, P., et al.: A multi-domain feature learning method for visual place recognition. In: 2019 IEEE International Conference on Robotics and Automation (ICRA) (2019)

    Google Scholar 

  29. Zhong, Z., Zheng, L., Luo, Z., Li, S., Yang, Y.: Invariance matters: exemplar memory for domain adaptive person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 598–607 (2019)

    Google Scholar 

  30. Zhong, Z., Zheng, L., Zheng, Z., Li, S., Yang, Y.: Camera style adaptation for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5157–5166 (2018)

    Google Scholar 

  31. Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., Torralba, A.: Places: a 10 million image database for scene recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1452–1464 (2017)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shanghai Key Laboratory of Intelligent Information Processing, School of Computer Science, Fudan University, Shanghai, China

    Yiting Cheng & Wenqiang Zhang

  2. Academy for Engineering and Technology, Fudan University, Shanghai, China

    Yankai Wang & Lizhe Qi

Authors
  1. Yiting Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yankai Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lizhe Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wenqiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wenqiang Zhang .

Editor information

Editors and Affiliations

  1. Korea Advanced Institute of Science and, Daejeon, Korea (Republic of)

    Yong Man Ro

  2. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  3. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Junmo Kim

  4. National Cheng Kung University, Tainan City, Taiwan

    Wei-Ta Chu

  5. Tsinghua University, Beijing, China

    Peng Cui

  6. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Jung-Woo Choi

  7. National Tsing Hua University, Hsinchu, Taiwan

    Min-Chun Hu

  8. Ghent University, Ghent, Belgium

    Wesley De Neve

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cheng, Y., Wang, Y., Qi, L., Zhang, W. (2020). Multi-condition Place Generator for Robust Place Recognition. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11961. Springer, Cham. https://doi.org/10.1007/978-3-030-37731-1_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37731-1_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37730-4

  • Online ISBN: 978-3-030-37731-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation