An IPM Approach to Multi-robot Cooperative Localization: Pepper Humanoid and Wheeled Robots in a Shared Space

  • M. Hassan TanveerEmail author
  • Antonio Sgorbissa
  • Antony Thomas
Conference paper
Part of the Lecture Notes in Electrical Engineering book series (LNEE, volume 613)


In this work we investigate the problem of multi-robot cooperative localization in dynamic environments. Specifically, we propose an approach where wheeled robots are localized using the monocular camera embedded in the head of a Pepper humanoid robot, to the end of minimizing deviations from their paths and avoiding each other during navigation tasks. Indeed, position estimation requires obtaining a linear relationship between points in the image and points in the world frame: to this end, an Inverse Perspective mapping (IPM) approach has been adopted to transform the acquired image into a bird eye view of the environment. The scenario is made more complex by the fact that Pepper’s head is moving dynamically while tracking the wheeled robots, which requires to consider a different IPM transformation matrix whenever the attitude (Pitch and Yaw) of the camera changes. Finally, the IPM position estimate returned by Pepper is merged with the estimate returned by the odometry of the wheeled robots through an Extened Kalman Filter. Experiments are shown with multiple robots moving along different paths in a shared space, by avoiding each other without onboard sensors, i.e., by relying only on mutual positioning information.


Multi-robot cooperative localization Wheeled robots Humanoid robots Inverse perspective mapping 



This work has been partially funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 737858 (CARESSES (


  1. 1.
    Boyle DP, Gupta HV, Sorooshian S (2000) Toward improved calibration of hydrologic models: Combining the strengths of manual and automatic methods. Water Resour Res 36(12):3663–3674CrossRefGoogle Scholar
  2. 2.
    Bruno B, Chong N, Kamide H, Kanoria S, Lee J, Lim Y, Pandey A, Papadopoulos C, Papadopoulos I, Pecora F, Saffiotti A, Sgorbissa A (2017) Paving the way for culturally competent robots: a position paper. In: RO-MAN 2017 - 26th IEEE international symposium on robot and human interactive communication, vol 2017-January, pp 553–560Google Scholar
  3. 3.
    Civera J, Davison AJ, Montiel JM (2008) Inverse depth parametrization for monocular slam. IEEE Trans Rob 24(5):932–945CrossRefGoogle Scholar
  4. 4.
    Guo C, Meguro JI, Kojima Y, Naito T (2014) Automatic lane-level map generation for advanced driver assistance systems using low-cost sensors. In: 2014 IEEE international conference on robotics and automation (ICRA), pp 3975–3982. IEEEGoogle Scholar
  5. 5.
    Laganiere R (2000) Compositing a bird’s eye view mosaic. Image 10:3Google Scholar
  6. 6.
    Lemaignan S, Warnier M, Sisbot EA, Clodic A, Alami R (2017) Artificial cognition for social human-robot interaction: an implementation. Artif Intell 247:45–69MathSciNetCrossRefGoogle Scholar
  7. 7.
    Lin CC, Wang MS (2012) A vision based top-view transformation model for a vehicle parking assistant. Sensors 12(4):4431–4446CrossRefGoogle Scholar
  8. 8.
    Ma L, Yang X, Tao D (2014) Person re-identification over camera networks using multi-task distance metric learning. IEEE Trans Image Process 23(8):3656–3670MathSciNetCrossRefGoogle Scholar
  9. 9.
    Mallot HA, Bülthoff HH, Little J, Bohrer S (1991) Inverse perspective mapping simplifies optical flow computation and obstacle detection. Biol Cybern 64(3):177–185CrossRefGoogle Scholar
  10. 10.
    Mastrogiovanni F, Sgorbissa A, Zaccaria R (2009) Context assessment strategies for ubiquitous robots. In: Proceedings IEEE international conference on robotics and automation (ICRA 2009), pp 2717–2722Google Scholar
  11. 11.
    Mastrogiovanni F, Sgorbissa A, Zaccaria R (2009) Robust navigation in an unknown environment with minimal sensing and representation. IEEE Trans Syst Man Cybern B Cybern 39(1):212–229CrossRefGoogle Scholar
  12. 12.
    Maurino DE, Reason J, Johnston N, Lee RB (2017) Beyond aviation human factors: safety in high technology systems. RoutledgeGoogle Scholar
  13. 13.
    Miraldo P, Araujo H (2013) Calibration of smooth camera models. IEEE Trans Pattern Anal Mach Intell 35(9):2091–2103CrossRefGoogle Scholar
  14. 14.
    Moreno D, Taubin G (2012) Simple, accurate, and robust projector-camera calibration. In: 2012 second international conference on 3D imaging, modeling, processing, visualization and transmission (3DIMPVT), pp 464–471. IEEE (2012)Google Scholar
  15. 15.
    Morro A, Sgorbissa A, Zaccaria R (2011) Path following for unicycle robots with an arbitrary path curvature. IEEE Trans Rob 27(5):1016–1023CrossRefGoogle Scholar
  16. 16.
    Mukhtar A, Xia L, Tang TB (2015) Vehicle detection techniques for collision avoidance systems: a review. IEEE Trans Intell Transp Syst 16(5):2318–2338CrossRefGoogle Scholar
  17. 17.
    Munaro M, Basso F, Menegatti E (2016) OpenPTrack: open source multi-camera calibration and people tracking for RGB-D camera networks. Rob Auton Syst 75:525–538CrossRefGoogle Scholar
  18. 18.
    Muscolo G, Recchiuto C (2017) Flexible structure and wheeled feet to simplify biped locomotion of humanoid robots. Int J Hum Rob 14(1)CrossRefGoogle Scholar
  19. 19.
    Oliveira M, Santos V, Sappa AD (2015) Multimodal inverse perspective mapping. Inf Fusion 24:108–121CrossRefGoogle Scholar
  20. 20.
    Pandey AK, Gelin R (2018) A mass-produced sociable humanoid robot: pepper: the first machine of its kind. IEEE Rob Autom Mag 25(3):40–48CrossRefGoogle Scholar
  21. 21.
    Parmiggiani A, Fiorio L, Scalzo A, Sureshbabu A, Randazzo M, Maggiali M, Pattacini U, Lehmann H, Tikhanoff V, Domenichelli D, Cardellino A, Congiu P, Pagnin A, Cingolani R, Natale L, Metta G (2017) The design and validation of the R1 personal humanoid. In: IEEE international conference on intelligent robots and systems, vol 2017-September, pp 674–680Google Scholar
  22. 22.
    Rubenstein M, Cornejo A, Nagpal R (2014) Programmable self-assembly in a thousand-robot swarm. Science 345(6198):795–799CrossRefGoogle Scholar
  23. 23.
    Saeedi S, Trentini M, Seto M, Li H (2016) Multiple-robot simultaneous localization and mapping: a review. J Field Rob 33(1):3–46CrossRefGoogle Scholar
  24. 24.
    Saffiotti A, Broxvall M, Gritti M, LeBlanc K, Lundh R, Rashid J, Seo B, Cho Y (2008) The PEIS-ecology project: vision and results. In: 2008 IEEE/RSJ international conference on intelligent robots and systems, IROS, pp 2329–2335Google Scholar
  25. 25.
    Siciliano B, Khatib O (2016) Springer handbook of robotics. Springer, HeidelbergCrossRefGoogle Scholar
  26. 26.
    Stein G, Dagan E, Mano O, Shashua, A (2017) Collision warning system. US Patent 9,656,607Google Scholar
  27. 27.
    Tanveer MH, Recchiuto CT, Sgorbissa A (2018) Analysis of path following and obstacle avoidance for multiple wheeled robots in a shared workspace. Robotica 37(1):80–108CrossRefGoogle Scholar
  28. 28.
    Tanveer MH, Recchiuto CT, Sgorbissa A (2018) Coordinated behaviour with a Pepper Humanoid robot to estimate the distance of other robot using inverse perspective mapping. In: IEEE international conference on automation and robotics (ICAROB)Google Scholar
  29. 29.
    Tanveer MH, Sgorbissa A (2018) An inverse perspective mapping approach using monocular camera of pepper humanoid robot to determine the position of other moving robot in plane. In: Proceedings of the 15th international conference on informatics in control, automation and robotics, vol 2. ICINCO, pp 219–225. INSTICC, SciTePressGoogle Scholar
  30. 30.
    Tuohy S, O’Cualain D, Jones E, Glavin M (2010) Distance determination for an automobile environment using inverse perspective mapping in OpenCV. In: IET irish signals and systems conference (ISSC)Google Scholar
  31. 31.
    Van der Walt S, Schönberger JL, Nunez-Iglesias J, Boulogne F, Warner JD, Yager N, Gouillart E, Yu T (2014) scikit-image: image processing in Python. PeerJ 2:e453CrossRefGoogle Scholar
  32. 32.
    Wang X (2013) Intelligent multi-camera video surveillance: a review. Pattern Recogn Lett 34(1):3–19CrossRefGoogle Scholar
  33. 33.
    Yenikaya S, Yenikaya G, Düven E (2013) Keeping the vehicle on the road: a survey on on-road lane detection systems. ACM Comput Surv (CSUR) 46(1):2CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • M. Hassan Tanveer
    • 1
    Email author
  • Antonio Sgorbissa
    • 1
  • Antony Thomas
    • 1
  1. 1.Department of Bio-Robotics and Intelligent Systems (DIBRIS)University of GenovaGenovaItaly

Personalised recommendations