Skip to main content
SpringerLink
Log in

Image-Based Geo-Localization Using Satellite Imagery

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

The problem of localization on a geo-referenced satellite map given a query ground view image is useful yet remains challenging due to the drastic change in viewpoint. To this end, in this paper we work on the extension of our earlier work on the Cross-View Matching Network (CVM-Net) (Hu et al. in IEEE conference on computer vision and pattern recognition (CVPR), 2018) for the ground-to-aerial image matching task since the traditional image descriptors fail due to the drastic viewpoint change. In particular, we show more extensive experimental results and analyses of the network architecture on our CVM-Net. Furthermore, we propose a Markov localization framework that enforces the temporal consistency between image frames to enhance the geo-localization results in the case where a video stream of ground view images is available. Experimental results show that our proposed Markov localization framework can continuously localize the vehicle within a small error on our Singapore dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  • Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., et al. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. CoRR, arXiv:1603.04467.

  • Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2016). Netvlad: CNN architecture for weakly supervised place recognition. In IEEE conference on computer vision and pattern recognition.

  • Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., & Sivic, J. (2017). Netvlad: CNN architecture for weakly supervised place recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, PP(99).

  • Bansal, M., Daniilidis, K., & Sawhney, H. (2012). Ultra-wide baseline facade matching for geo-localization. In European conference on computer vision.

  • Bansal, M., Sawhney, H. S., Cheng, H., & Daniilidis, K. (2011). Geo-localization of street views with aerial image databases. In ACM international conference on multimedia.

  • Bay, H., Tuytelaars, T., & Van Gool, L. (2006). Surf: Speeded up robust features. In European conference on computer vision.

  • Chen, W., Chen, X., Zhang, J., & Huang, K. (2017). Beyond triplet loss: A deep quadruplet network for person re-identification. In IEEE conference on computer vision and pattern recognition.

  • Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In The IEEE conference on computer vision and pattern recognition (CVPR).

  • Chopra, S., Hadsell, R., LeCun, Y. (2005). Learning a similarity metric discriminatively, with application to face verification. In IEEE conference on computer vision and pattern recognition.

  • Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., & Li, F. F. (2009). Imagenet: A large-scale hierarchical image database. In IEEE conference on computer vision and pattern recognition.

  • Han, X., Leung, T., Jia, Y., Sukthankar, R., & Berg, A. C. (2015). Matchnet: Unifying feature and metric learning for patch-based matching. In IEEE conference on computer vision and pattern recognition.

  • Hays, J., & Efros, A. A. (2008) Im2gps: Estimating geographic information from a single image. In IEEE conference on computer vision and pattern recognition.

  • He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In IEEE conference on computer vision and pattern recognition.

  • Hermans, A., Beyer, L., & Leibe, B. (2017). In defense of the triplet loss for person re-identification. CoRR, arXiv:1703.07737.

  • Hu, S., Feng, M., Nguyen, R. M. H., & Hee Lee, G. (2018). CVM-net: Cross-view matching network for image-based ground-to-aerial geo-localization. In IEEE conference on computer vision and pattern recognition (CVPR).

  • Huang, G., Liu, Z., van der Maaten, L., Weinberger, K. Q. (2017). Densely connected convolutional networks. In The IEEE conference on computer vision and pattern recognition (CVPR).

  • Jegou, H., Douze, M., Schmid, C., & Perez, P. (2010). Aggregating local descriptors into a compact image representation. In IEEE conference on computer vision and pattern recognition.

  • Kim, D. K., & Walter, M. R. (2017). Satellite image-based localization via learned embeddings. In IEEE international conference on robotics and automation.

  • Kingma, D., & Ba, J. (2014). Adam: A method for stochastic optimization. CoRR, arXiv:1412.6980.

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems.

  • Lin, T. Y., Belongie, S., & Hays, J. (2013). Cross-view image geolocalization. In IEEE conference on computer vision and pattern recognition.

  • Lin, T. Y., Cui, Y., Belongie, S., & Hays, J. (2015) Learning deep representations for ground-to-aerial geolocalization. In IEEE conference on computer vision and pattern recognition.

  • Liu, P., Geppert, M., Heng, L., Sattler, T., Geiger, A., & Pollefeys, M. (2018). Towards robust visual odometry with a multi-camera system. In IEEE/RSJ international conference on intelligent robots and systems.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • McManus, C., Churchill, W., Maddern, W., Stewart, A. D., & Newman, P. (2014). Shady dealings: Robust, long-term visual localisation using illumination invariance.

  • Middelberg, S., Sattler, T., Untzelmann, O., & Kobbelt, L. (2014). Scalable 6-dof localization on mobile devices. In European conference on computer vision.

  • Nister, D., & Stewenius, H. (2006). Scalable recognition with a vocabulary tree. In IEEE conference on computer vision and pattern recognition.

  • Noda, M., Takahashi, T., Deguchi, D., Ide, I., Murase, H., Kojima, Y., & Naito, T. (2010). Vehicle ego-localization by matching in-vehicle camera images to an aerial image. In Asian conference on computer vision workshops.

  • Oh Song, H., Xiang, Y., Jegelka, S., & Savarese, S. (2016). Deep metric learning via lifted structured feature embedding. In IEEE conference on computer vision and pattern recognition.

  • Sattler, T., Havlena, M., Schindler, K., & Pollefeys, M. (2016). Large-scale location recognition and the geometric burstiness problem. In Conference on computer vision and pattern recognition.

  • Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In IEEE conference on computer vision and pattern recognition.

  • Senlet, T., & Elgammal, A. (2011). A framework for global vehicle localization using stereo images and satellite and road maps.

  • Senlet, T., & Elgammal, A. (2012). Satellite image based precise robot localization on sidewalks.

  • Shan, Q., Wu, C., Curless, B., Furukawa, Y., Hernandez, C., & Seitz, S. M. (2014). Accurate geo-registration by ground-to-aerial image matching. In International conference on 3D vision.

  • Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. CoRR, arXiv:1409.1556.

  • Sivic, J., & Zisserman, A. (2003). Video google: A text retrieval approach to object matching in videos. In IEEE international conference on computer vision.

  • Stumm, E., Mei, C., Lacroix, S., Nieto, J., Hutter, M., & Siegwart, R. (2016). Robust visual place recognition with graph kernels. In IEEE conference on computer vision and pattern recognition.

  • Thrun, S. (2002). Particle filters in robotics. In Proceedings of the eighteenth conference on uncertainty in artificial intelligence.

  • Thrun, S., Fox, D., Burgard, W., & Dallaert, F. (2001). Robust Monte Carlo localization for mobile robots. Artificial Intelligence, 128, 99–141.

    Article  Google Scholar 

  • Tian, Y., Chen, C., & Shah, M. (2017). Cross-view image matching for geo-localization in urban environments. In IEEE conference on computer vision and pattern recognition.

  • Viswanathan, A., Pires, B. R., Huber, D. (2014). Vision based robot localization by ground to satellite matching in GPS-denied situations. In IEEE/RSJ international conference on intelligent robots and systems.

  • Vo, N., Jacobs, N., & Hays, J. (2017). Revisiting im2gps in the deep learning era.

  • Vo, N. N., & Hays, J. (2016). Localizing and orienting street views using overhead imagery. In European conference on computer vision.

  • Wang, J., Zhou, F., Wen, S., Liu, X., & Lin, Y. (2017). Deep metric learning with angular loss.

  • Workman, S., & Jacobs, N. (2015). On the location dependence of convolutional neural network features. In IEEE conference on computer vision and pattern recognition workshops.

  • Workman, S., Souvenir, R., & Jacobs, N. (2015). Wide-area image geolocalization with aerial reference imagery.

  • Zagoruyko, S., & Komodakis, N. (2015). Learning to compare image patches via convolutional neural networks. In IEEE conference on computer vision and pattern recognition.

  • Zamir, A. R., & Shah, M. (2014). Image geo-localization based on multiple nearest neighbor feature matching using generalized graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(8), 1546–1558.

    Article  Google Scholar 

  • Zhai, M., Bessinger, Z., Workman, S., & Jacobs, N. (2017). Predicting ground-level scene layout from aerial imagery. In IEEE conference on computer vision and pattern recognition (2017).

  • Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., & Oliva, A. (2014). Learning deep features for scene recognition using places database. In Advances in Neural Information Processing Systems.

Download references

Author information

Authors and Affiliations

  1. National University of Singapore, Computing 1, 13 Computing Drive, Singapore, Singapore

    Sixing Hu & Gim Hee Lee

Authors
  1. Sixing Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gim Hee Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sixing Hu.

Additional information

Communicated by Anelia Angelova, Gustavo Carneiro, Niko Sünderhauf, Jürgen Leitner.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, S., Lee, G.H. Image-Based Geo-Localization Using Satellite Imagery. Int J Comput Vis 128, 1205–1219 (2020). https://doi.org/10.1007/s11263-019-01186-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-019-01186-0

Keywords

Search

Navigation