Skip to main content

Autonomous Navigation of Multiple Robots with Sensing and Communication Constraints Based on Mixed Reality

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.


This paper presents a robotic navigation system that uses mixed reality concepts to develop sensing and communication virtual devices, based on the visual localization of the robot in the environment. The main objective of the navigation system is to provide conditions for the use of very simple robots with severe limitations on the mentioned peripheral devices for simulation, analysis and test of multi-robot applications. In an experiment with real robots, each one receives its virtual navigation skills in an independent way from the tool that emulates the function of such peripherals. Thus, the behavior of a group of robots, independently commanded, is implemented in the virtual environment and accomplished in the real world. An experiment composed by real multiple Sphero robots executing an exploratory task within an unknown dynamic environment is carried out to validate the proposed navigation system. The use of mixed reality concepts allows an easy implementation of cooperation mechanisms based on indirect communication skill and fuzzy controllers for the robots’ movement. The results confirm the feasibility of the proposed autonomous navigation system.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12










  • Almeida, J. P. L. S., Nakashima, R. T., Neves-Jr, F., & de Arruda, L. V. R. (2019). Bio-inspired on-line path planner for cooperative exploration of unknown environment by a multi-robot system. Robotics and Autonomous Systems, 112, 32–48.

    Article  Google Scholar 

  • Antoun, A., Valentini, G., Hocquard, E., Wiandt, B., Trianni, V., & Dorigo, M. (2016). Kilogrid: A modular virtualization environment for the kilobot robot. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 3809–3814).

  • Arvin, F., Krajník, T., Turgut, A. E., & Yue, S. (2015). Cos: Artificial pheromone system for robotic swarms research. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 407–412).

  • da Silva Guerra, R., Boedecker, J., Mayer, N., Yanagimachi, S., Hirosawa, Y., Yoshikawa, K., Namekawa, M., Asada, M. (2006). Citizen eco-be! league: Bringing new flexibility for research and education to robocup. In Proceedings of the meeting of special interest group on AI challenges (vol. 23, pp. 13–18).

  • da Silva Guerra, R., Boedecker, J., Mayer, N., Yanagimachi, S., Ishiguro, H., & Asada, M. (2007a). A new minirobotics system for teaching and researching agent-based programming. In Computers and advanced technology in education (pp. 39–44).

  • da Silva Guerra, R., Boedecker, J., Yamauchi, K., Maekawa, T., Asada, M., Hirosawa, Y., Namekawa, M., Yoshikawa, K., Yanagimachi, S., Masubuchi, S., & Nishimura, K. (2007b). Citizen eco-be! and the robocup physical visualization league.

  • Dorigo, M., Birattari, M., & Stützle, T. (2006). Ant colony optimization—Artificial ants as a computational intelligence technique. IEEE Computational Intelligence Magazine, 1, 28–39.

    Article  Google Scholar 

  • Dorigo, M., Maniezzo, V., & Colorni, A. (1996). Ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 26(1), 29–41.

    Article  Google Scholar 

  • Fiala, M. (2005). Artag, a fiducial marker system using digital techniques. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05) (Vol. 2, pp. 590–596 ).

  • Gerndt, R., Schridde, C., & da Silva Guerra, R. (2008). On the aspects of simulation in the robocup mixed reality soccer systems. In Workshop at international conference on simulation, modeling and programming for autonomous robots (pp. 159–166).

  • Hönig, W., Milanes, C., Scaria, L., Phan, T., Bolas, M., & Ayanian, N. (2015). Mixed reality for robotics. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 5382–5387).

  • Millard, A. G., Redpath, R., Jewers, A. M., Arndt, C., Joyce, R., Hilder, J. A., et al. (2018). Ardebug: An augmented reality tool for analysing and debugging swarm robotic systems. Frontiers in Robotics and AI, 5, 87.

    Article  Google Scholar 

  • Passino, K. M., & Yurkovich, S. (1998). Fuzzy control. Boston: Addison-Wesley.

    MATH  Google Scholar 

  • Quigley, M., Conley, K., Gerkey, B. P., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009). ROS: An open-source robot operating system. In ICRA workshop on open source software.

  • Reina, A., Cope, A. J., Nikolaidis, E., Marshall, J. A. R., & Sabo, C. (2017). Ark: Augmented reality for kilobots. IEEE Robotics and Automation Letters, 2(3), 1755–1761.

    Article  Google Scholar 

  • Reina, A., Salvaro, M., Francesca, G., Garattoni, L., Pinciroli, C., Dorigo, M., & Birattari, M. (2015). Augmented reality for robots: Virtual sensing technology applied to a swarm of e-pucks. In 2015 NASA/ESA conference on adaptive hardware and systems (AHS) (pp. 1–6).

  • Sabattini, L., Secchi, C., & Fantuzzi, C. (2017). Multi-robot systems implementing complex behaviors under time-varying topologies. European Journal of Control, 38, 73–87.

    Article  MathSciNet  MATH  Google Scholar 

  • Santos, H. B., Teixeira, M. A. S., de Oliveira, A. S., de Arruda, L. V. R., & Neves, F. (2017). Control of mobile robots using actionlib. In A. Koubaa (Ed.), Robot operating system (ROS): The complete reference (volume 2) (pp. 161–189). Cham: Springer.

    Chapter  Google Scholar 

  • Simões, M. A. C., de Souza, J. R., de Assis Moura Pimentel, F., & Frias, D. (2011). Mr-simulator: A simulator for the mixed reality competition of robocup. In: J. Ruiz-del Solar, E. Chown, & P. G. Plöger (eds.) RoboCup 2010: Robot soccer world cup XIV (pp. 82–96). Berlin, Heidelberg: Springer.

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to João Paulo Lima Silva de Almeida.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The authors acknowledge financial support from Brazilian Research Council (CNPq): Grants 305816/2014-4 and 309119/2015-4.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 7260 KB)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Almeida, J.P.L.S., Nakashima, R.T., Neves-Jr, F. et al. Autonomous Navigation of Multiple Robots with Sensing and Communication Constraints Based on Mixed Reality. J Control Autom Electr Syst 31, 1165–1176 (2020).

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: