Autonomous Robots

, Volume 39, Issue 3, pp 363–387 | Cite as

Learning place-dependant features for long-term vision-based localisation

Article

Abstract

This paper presents an alternative approach to the problem of outdoor, persistent visual localisation against a known map. Instead of blindly applying a feature detector/descriptor combination over all images of all places, we leverage prior experiences of a place to learn place-dependent feature detectors (i.e., features that are unique to each place in our map and used for localisation). Furthermore, as these features do not represent low-level structure, like edges or corners, but are in fact mid-level patches representing distinctive visual elements (e.g., windows, buildings, or silhouettes), we are able to localise across extreme appearance changes. Note that there is no requirement that the features posses semantic meaning, only that they are optimal for the task of localisation. This work is an extension on previous work (McManus et al. in Proceedings of robotics science and systems, 2014b) in the following ways: (i) we have included a landmark refinement and outlier rejection step during the learning phase, (ii) we have implemented an asynchronous pipeline design, (iii) we have tested on data collected in an urban environment, and (iv) we have implemented a purely monocular system. Using over 100 km worth of data for training, we present localisation results from Begbroke Science Park and central Oxford.

Keywords

Feature learning Appearance changes Cross seasonal Visual localisation Long-term autonomy Outdoor localisation 

References

  1. Anati, R., Scaramuzza, D., Derpanis, K., & Daniilidis, K. (2012). Robot localization using soft object detection. In Proceedings of the IEEE international conference on robotics and automation (ICRA), St. Paul.Google Scholar
  2. Atanasov, N., Zhu, M., Daniilidis, K., & Pappas, G. J. (2014). Semantic localisation via the matrix permanent. In Proceedings of robotics science and systems (RSS), Berkeley.Google Scholar
  3. Bao, S. Y., & Savarese, S. (2011). Semantic structure from motion. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp. 2025–2032.Google Scholar
  4. Bay, H., Ess, A., Tuytelaars, T., & Gool, L. (2008). Surf: Speeded up robust features. Computer Vision and Image Understanding (CVIU), 110(3), 346–359.CrossRefGoogle Scholar
  5. Castle, R. O., Gawley, D. J., Klein, G., & Murray, D. W. (2007). Towards simultaneous recognition, localization and mapping for hand-held and wearable cameras. In Proceedings of the IEEE international conference on in robotics and automation (ICRA), Rome.Google Scholar
  6. Churchill, W., & Newman, P. (2012). Practice makes perfect? Managing and leveraging visual experiences for lifelong navigation. In Proceedings of the international conference on robotics and automation, Saint Paul.Google Scholar
  7. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Proceedings of the conference on computer vision and pattern recognition (pp. 886–893), San Diego.Google Scholar
  8. Davison, A., & Murray, D. (2002). Simultaneous localization and map-building using active vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7), 865–880.CrossRefGoogle Scholar
  9. Davison, A., Reid, I., Motlon, N., & Stasse, O. (2007). Monoslam: Real-time single camera slam. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(6), 1052–1067.CrossRefGoogle Scholar
  10. Doersch, C., Singh, S., Gupta, A., Sivic, J., & Efros, A. (2012). What makes paris look like Paris? ACM Transactions on Graphics, 31(4), 101.CrossRefGoogle Scholar
  11. Furgale, P., & Barfoot, T. (2001). Visual teach and repeat for long-range rover autonomy. Journal of Field Robotics, Special Issue on “Visual Mapping and Navigation Outdoors”, 27(5), 534–560.Google Scholar
  12. Hartley, R., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd ed.). Cambridge: Cambridge University Press, ISBN: 0521540518.Google Scholar
  13. Johns, E., & Yang, G.-Z. (2013). Feature co-occurrence maps: Appearance-based localisation throughout the day. In Proceedings of the international conference on robotics and automation.Google Scholar
  14. Kaess, M., Johannson, H., Roberts, R., Ila, V., Leonard, J., & Dellaert, F. (2012). isam2: Incremental smoothing and mapping using the bayes tree. Internatioanl Journal of Robotics Research, 31(2), 216–235.CrossRefGoogle Scholar
  15. Ko, D. W., Yi, C., & Suh, I. H. (2013). Semantic mapping and navigation: A bayesian approach. In Proceedings of the IEEE/RSJ international conference on intelligent robotics and systems (IROS), pp. 2630–2636.Google Scholar
  16. Konolige, K., Bowman, J., Chen, J., Mihelich, P., Calonder, M., Lepetit, V., et al. (2010). View-based maps. The International Journal of Robotics Research, 29(8), 941–957.CrossRefGoogle Scholar
  17. Lategahn, H., Beck, J., Kitt, B., & Stiller, C. (2013). How to learn an illumination robust image feature for place recognition. IEEE intelligent vehicles symposium, Gold Coast.Google Scholar
  18. Levenberg, K. (1944). A method for the solution of certain non-linear problems in least squares. The Quarterly of Applied Mathematics, 2, 164–168.MATHMathSciNetGoogle Scholar
  19. Linegar, C., Churchill, W., & Newman, P. (2015). Work smart, not hard: Recalling relevant experiences for vast-scale but time-constrained localisation. In IEEE international conference on robotics and automation (ICRA), Seattle.Google Scholar
  20. Lowe, D. (2004). Distinctive image features from scale-invariant key points. International Journal of Computer Vision, 60(2), 91–110.CrossRefGoogle Scholar
  21. Maddern, W., Stewart, A., McManus, C., Upcroft, B., Churchill, W., & Newman, P. (2014). Illumination invariant imaging: Applications in robust vision-based localisation, mapping and classification for autonomous vehicles. In Proceedings of the visual place recognition in changing environments workshop, IEEE international conference on robotics and automation, Hong Kong.Google Scholar
  22. McKinnon, D., Smith, R., & Upcroft, B. (2012). A semi-local method for iterative depth-map refinement. In Proceedings of the IEEE international conference on in robotics and automation (ICRA).Google Scholar
  23. McManus, C. (2010). The unscented kalman filter for state estimation. Presented at the simultaneous localization and mapping (SLAM) workshop, 7th Canadian conference on computer vision (CRV). Google Scholar
  24. McManus, C., Churchill, W., Maddern, W., Stewart, A., & Newman, P. (2014a). Shady dealings: Robust, long-term visual localisation using illumination invariance. In Proceedings of the IEEE international conference on robotics and automation (ICRA), Hong Kong.Google Scholar
  25. McManus, C., Churchill, W., Napier, A., Davis, B., & Newman, P. (2013). Distraction suppression for vision-based pose estimation at city scales. In Proceedings of the IEEE international conference on robotics and automation, Karlsruhe.Google Scholar
  26. McManus, C., Upcroft, B., & Newman, P. (2014b). Scene signatures: Localised and point-less features for localisation. In Proceedings of robotics science and systems (RSS), Berkley.Google Scholar
  27. Milford, M. (2013). Vision-based place recognition: How low can you go? The International Journal of Robotics Research, 32(7), 766–789.CrossRefGoogle Scholar
  28. Milford, M. & Wyeth, G. (2012). Seqslam: Visual route-based navigation for sunny summer days and stormy winter nights. In Proceedings of the IEEE international conference on robotics and automation (ICRA), Saint Paul.Google Scholar
  29. Naseer, T., Spinello, L., Burgard, W., & Stachniss, C. (2014). Robust visual robot localization across seasons using network flows. In AAAI conference on artifical intelligence (AAAI), Quebec.Google Scholar
  30. Neubert, P., Sunderhauf, N., & Protzel, P. (2013). Appearance change prediction for long-term navigation across seasons. In European Conference on mobile robotics (ECMR).Google Scholar
  31. Piniés, P., Paz, L. M., Gálvez-López, D., & Tardós, J. D. (2010). Ci-graph simultaneous localisation and mappin for three-dimensional reconstruction of large and complex environments using a multicamera system. Journal of Field Robotics, 27(5), 561–586.CrossRefGoogle Scholar
  32. Ranaganathan, A., Matsumoto, S., & Ilstrup, D. (2013). Towards illumination invariance for visual localization. Proceedings of the IEEE international conference on in robotics and automation (ICRA) (pp. 3791–3798), Karlsruhe.Google Scholar
  33. Renato F. Salas-Moreno, Richard A. Newcombe, H. S. P. H. J. K. & Davison, A. J. (2013). Slam++: Simultaneous localisation and mapping at the level of object. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR).Google Scholar
  34. Richardson, A. & Olson, E. (2013). Learning convolutional filters for interest point detection. In Proceedings of the IEEE international conference on robotics and automation (ICRA).Google Scholar
  35. Sibley, G., Mei, C., Reid, I., & Newman, P. (2010). Vast-scale outdoor navigation using adaptive relative bundle adjustment. The International Journal of Robotics Research, 29(8), 958–980.CrossRefGoogle Scholar
  36. Singh, S., Gupta, A., & Efros, A. A. (2012). Unsupervised discovery of mid-level discriminative patches. In Proceedings of the European conference on computer vision (ECCV).Google Scholar
  37. Stewart, A. & Newman, P. (2012). Laps - localisation using appearance of prior structure: 6-dof monocular camera localisation using prior pointclouds. In Proceedings of the international conference on robotics and automation, Saint Paul.Google Scholar
  38. Stewart, A. D. (2015). Localisation using the appearance of prior structure. PhD thesis, University of Oxford.Google Scholar
  39. Valgren, C., & Lilienthal, A. (2007). Sift, surf & seasons: Long-term outdoor localization using local features. In Proceedings of the 3rd European conference on mobile robotics (ECMR).Google Scholar
  40. Valgren, C., & Lilienthal, A. (2010). Sift, surf and seasons: Appearance-based long-term localization in outdoor environments. Robotics and Autonomous Systems, 58(2), 149–156.CrossRefGoogle Scholar
  41. Yi, C., Suh, I. H., Lim, G. H., & Choi, B.-U. (2009). Active-semantic localization with a single consumer-grade camera. In Proceedings of the IEEE international conference on systems, man and cybernetics (SMC), pp. 2161–2166.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Mobile Robotics GroupUniversity of OxfordOxfordUK
  2. 2.CyPhy LabQueensland University of TechnologyBrisbaneAustralia

Personalised recommendations