Skip to main content

Advertisement

SpringerLink
Log in

Leveraging variable sensor spatial acuity with a homogeneous, multi-scale place recognition framework

  • Original Article
  • Published:
Biological Cybernetics Aims and scope Submit manuscript

Abstract

Most robot navigation systems perform place recognition using a single-sensor modality and one, or at most two heterogeneous map scales. In contrast, mammals perform navigation by combining sensing from a wide variety of modalities including vision, auditory, olfactory and tactile senses with a multi-scale, homogeneous neural map of the environment. In this paper, we develop a multi-scale, multi-sensor system for mapping and place recognition that combines spatial localization hypotheses at different spatial scales from multiple different sensors to calculate an overall place recognition estimate. We evaluate the system’s performance over three repeated 1.5-km day and night journeys across a university campus spanning outdoor and multi-level indoor environments, incorporating camera, WiFi and barometric sensory information. The system outperforms a conventional camera-only localization system, with the results demonstrating not only how combining multiple sensing modalities together improves performance, but also how combining these sensing modalities over multiple scales further improves performance over a single-scale approach. The multi-scale mapping framework enables us to analyze the naturally varying spatial acuity of different sensing modalities, revealing how the multi-scale approach captures each sensing modality at its optimal operation point where a single-scale approach does not, and enables us to then weight sensor contributions at different scales based on their utility for place recognition at that scale.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  • Andreasson H, Duckett T, Lilienthal A (2008) A minimalistic approach to appearance-based visual SLAM. IEEE Trans Robot 24(5):1–11

    Article  Google Scholar 

  • Arras KO, Tomatis N, Jensen BT, Siegwart R (2001) Multisensor on-the-fly localization:: precision and reliability for applications. Robot Auton Syst 34(23):131–143. https://doi.org/10.1016/S0921-8890(00)00117-2. ISSN 0921-8890. European Workshop on Advanced Mobile Robots

  • Barry C, Burgess N (2014) Neural mechanisms of self-location. Curr Biol 24(8):R330–R339

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Bay H, Tuytelaars T, Van Gool L (2006) SURF: speeded up robust features. In: Computer vision ECCV 2006

  • Berkvens R, Jacobson A, Milford M, Peremans H, Weyn M (2014) Biologically inspired SLAM using Wi-Fi. In: IEEE intelligent robots and systems

  • Bonardi F, Ainouz S, Boutteau R, Dupuis Y, Savatier X, Vasseur P (2017) PHROG: a multimodal feature for place recognition. Sensors 17(5):1167

    Article  PubMed Central  Google Scholar 

  • Bosse M, Newman P, Leonard J, Soika M, Feiten W, Teller S (2003) An atlas framework for scalable mapping. In: International conference on robotics and automation, vol 2. IEEE, Taipei, pp 1899–1906. ISBN 0780377362

  • Burgess N, O’Keefe J (1996) Neuronal computations underlying the firing of place cells and their role in navigation. Hippocampus 6(6):749–762

    Article  PubMed  CAS  Google Scholar 

  • Chen Z, Jacobson A, Erdem UM, Hasselmo ME, Milford M (2014) Multi-scale bio-inspired place recognition. In: 2014 IEEE international conference on robotics and automation (ICRA)

  • Cummins M, Newman P (2011) Appearance-only SLAM at large scale with FAB-MAP 2.0. Int J Robot Res 30:1100–1123

    Article  Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Computer vision and pattern recognition. CVPR 2005. IEEE computer society conference on, vol 1. IEEE, pp 886–893

  • Davison AJ, Reid ID, Molton ND, Stasse O (2007) MonoSLAM: real-time single camera SLAM. IEEE Trans Pattern Anal Mach Intell 29(6):1052–1067

    Article  PubMed  Google Scholar 

  • Faragher RM, Sarno C, Newman M (2012) Opportunistic radio SLAM for indoor navigation using smartphone sensors. In: Position location and navigation symposium (PLANS), 2012 IEEE/ION. IEEE, Myrtle Beach, pp 120–128. https://doi.org/10.1109/PLANS.2012.6236873. ISBN 978-1-4673-0387-3

  • Fenton AA, Kao H-Y, Neymotin SA, Olypher A, Vayntrub Y, Lytton WW, Ludvig N (2008) Unmasking the CA1 ensemble place code by exposures to small and large environments: more place cells and multiple, irregularly arranged, and expanded place fields in the larger space. J Neurosci 28(44):11250–11262

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Ferris B, Fox D, Lawrence N (2007) WiFi-SLAM using Gaussian process latent variable models. In: Proceedings of the 20th international joint conference on artificial intelligence, pp 2480–2485

  • Frost BJ, Mouritsen H (2006) The neural mechanisms of long distance animal navigation. Curr Opin Neurobiol 16(4):481–488

    Article  PubMed  CAS  Google Scholar 

  • Geva-Sagiv M, Las L, Yovel Y, Ulanovsky N (2015) Spatial cognition in bats and rats: from sensory acquisition to multiscale maps and navigation. Nat Rev Neurosci 16(2):94–108

    Article  PubMed  CAS  Google Scholar 

  • Glover A, Maddern W, Warren M, Reid S, Milford M, Wyeth G (2012) OpenFABMAP: an open source toolbox for appearance-based loop closure detection. In: Robotics and automation (ICRA), 2012 IEEE international conference on, pp 4730–4735. https://doi.org/10.1109/ICRA.2012.6224843. ISBN 1050-4729

  • Hargreaves EL, Rao G, Lee I, Knierim JJ (2005) Major dissociation between medial and lateral entorhinal input to dorsal hippocampus. Science 308(5729):1792

    Article  PubMed  CAS  Google Scholar 

  • Huang J, Millman D, Quigley M, Stavens D, Thrun S, Aggarwal A (2011) Efficient, generalized indoor WiFi GraphSLAM. In: International conference on robotics and automation. https://doi.org/10.1109/ICRA.2011.5979643. ISBN 978-1-61284-386-5

  • Jacobson A, Chen Z, Milford M (2015a) Autonomous multisensor calibration and closed-loop fusion for SLAM. J Field Robot 32(1):85–122

  • Jacobson A, Chen Z, Milford M (2015b) Online place recognition calibration for out-of-the-box SLAM. In: IEEE/RSJ international conference on intelligent robots and systems

  • Jung MW, Wiener SI, McNaughton BL (1994) Comparison of spatial firing characteristics of units in dorsal and ventral hippocampus of the rat. J Neurosci 14(12):7347–7356

    Article  PubMed  CAS  Google Scholar 

  • Kawewong A, Tongprasit N, Tangruamsub S, Hasegawa O (2010) Online and incremental appearance-based SLAM in highly dynamic environments. Int J Robot Res 30(1):33–55

  • Keefe JO, Burgess N (1996) Geometric determinants of the place fields of hippocampal neurons. Nature 381(6581):425

    Article  Google Scholar 

  • Konolige K, Agrawal M (2008) FrameSLAM: from bundle adjustment to real-time visual mapping. IEEE Trans Robot 24:1066–1077

    Article  Google Scholar 

  • Kuipers B, Byun YT (1991) A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Robot Auton Syst 8(1):47–63

    Article  Google Scholar 

  • Kuipers B, Modayil J, Beeson P, MacMahon M, Savelli F (2004) Local metrical and global topological maps in the hybrid spatial semantic hierarchy. In: International conference on robotics and automation

  • Lategahn H, Beck J, Kitt B, Stiller C (2013) How to learn an illumination robust image feature for place recognition. In: Intelligent vehicles symposium (IV), 2013 IEEE. IEEE, pp 285–291

  • Li B, Harvey B, Gallagher T (2013) Using barometers to determine the height for indoor positioning. In: Indoor positioning and indoor navigation (IPIN), 2013 international conference on. IEEE, pp 1–7

  • Lowe DG (1999) Object recognition from local scale-invariant features. In: International conference on computer vision, Kerkyra

  • Maaswinkel H, Whishaw IQ (1999) Homing with locale, taxon, and dead reckoning strategies by foraging rats: sensory hierarchy in spatial navigation. Behav Brain Res 99(2):143–152. https://doi.org/10.1016/S0166-4328(98)00100-4. ISSN 0166-4328

  • McNaughton B, Barnes C, Gerrard J, Gothard K, Jung M, Knierim J, Kudrimoti H, Qin Y, Skaggs W, Suster M et al (1996) Deciphering the hippocampal polyglot: the hippocampus as a path integration system. J Exp Biol 199(1):173–185

    PubMed  CAS  Google Scholar 

  • Milford M, Wyeth G (2012) SeqSLAM: visual route-based navigation for sunny summer days and stormy winter nights. In: IEEE international conference on robotics and automation, St Paul

  • Muller RU, Kubie JL, Ranck JB (1987) Spatial firing patterns of hippocampal complex-spike cells in a fixed environment. J Neurosci 7(7):1935–1950

    Article  PubMed  CAS  Google Scholar 

  • Nakazawa K, McHugh TJ, Wilson MA, Tonegawa S (2004) NMDA receptors, place cells and hippocampal spatial memory. Nat Rev Neurosci 5(5):361–372

    Article  PubMed  CAS  Google Scholar 

  • Olivia A (2005) Gist of the scene. In: Itti L, Rees G, Tsotsos JK (eds) Neurobiology of attention. Elsevier, Amsterdam

    Google Scholar 

  • Paz LM, Pinies P, Tardos JD, Neira J (2008) Large-scale 6-DOF SLAM with stereo-in-hand. IEEE Trans Robot 24:946–957

    Article  Google Scholar 

  • Ravassard P, Kees A, Willers B, Ho D, Aharoni D, Cushman J, Aghajan ZM, Mehta MR (2013) Multisensory control of hippocampal spatiotemporal selectivity. Science 340(6138):1342–1346

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  • Rossier J, Haeberli C, Schenk F (2000) Auditory cues support place navigation in rats when associated with a visual cue. Behav Brain Res 117(1–2):209–214. https://doi.org/10.1016/S0166-4328(00)00293-X. ISSN 01664328

  • Sheppard JP, Raposo D, Churchland AK (2013) Dynamic weighting of multisensory stimuli shapes decision-making in rats and humans. J Vis 13(6):4

    Article  PubMed  PubMed Central  Google Scholar 

  • Skaggs WE, McNaughton BL (1996) Theta phase precession in hippocampal. Hippocampus 6:149–172

    Article  PubMed  CAS  Google Scholar 

  • Solstad T, Moser EI, Einevoll GT (2006) From grid cells to place cells: a mathematical model. Hippocampus 16:1026–1031

    Article  PubMed  Google Scholar 

  • Spiers HJ, Barry C (2015) Neural systems supporting navigation. Curr Opin Behav Sci 1:47–55

    Article  Google Scholar 

  • Vanderelst D, Steckel J, Boen A, Peremans H, Holderied MW (2016) Place recognition using batlike sonar. eLife 5:e14188

    Article  PubMed  PubMed Central  Google Scholar 

  • Weyn M (2011) Opportunistic seamless localization, Ph.D. thesis. Universiteit Antwerpen

  • Wiener SI, Berthoz A, Zugaro MB (2002) Multisensory processing in the elaboration of place and head direction responses by limbic system neurons. Cogn Brain Res 14(1):75–90

    Article  Google Scholar 

  • Wikenheiser AM, Redish AD (2015) Hippocampal theta sequences reflect current goals. Nat Neurosci 18(2):289–294

    Article  PubMed  PubMed Central  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical Engineering and Computer Science, Queensland University of Technology, Brisbane, Australia

    Adam Jacobson, Zetao Chen & Michael Milford

Authors
  1. Adam Jacobson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zetao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Milford
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adam Jacobson.

Additional information

Communicated by J. Leo van Hemmen.

This work was supported by a funding from the Australian Research Council Centre of Excellence CE140100016 in Robotic Vision and a Future Fellowship FT140101229 to MM.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jacobson, A., Chen, Z. & Milford, M. Leveraging variable sensor spatial acuity with a homogeneous, multi-scale place recognition framework. Biol Cybern 112, 209–225 (2018). https://doi.org/10.1007/s00422-017-0745-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00422-017-0745-7

Keywords

Search

Navigation