A Comparison of Various Approaches to Reinforcement Learning Algorithms for Multi-robot Box Pushing

Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 63)


In this paper, a comparison of reinforcement learning algorithms and their performance on a robot box pushing task is provided. The robot box pushing problem is structured as both a single agent problem and also a multi-agent problem. A Q-learning algorithm is applied to the single-agent box pushing problem, and three different Q-learning algorithms are applied to the multi-agent box pushing problem. Both sets of algorithms are applied on a dynamic environment that is comprised of static objects, a static goal location, a dynamic box location, and dynamic agent positions. A simulation environment is developed to test the four algorithms, and their performance is compared through graphical explanations of test results. The comparison shows that the newly applied reinforcement algorithm out-performs the previously applied algorithms on the robot box pushing problem in a dynamic environment.


Reinforcement learning Q-learning Multi-robot Box-pushing 


  1. 1.
    Chakraborty, J., Konar, A., Nagar, A., Das, S.: Rotation and translation selective pareto optimal solution to the box-pushing problem by mobile robots using NSGA-II. In: 2009 IEEE Congress on Evolutionary Computation, pp. 2120–2126, May 2009.
  2. 2.
    Hwang, K.S., Ling, J.L., Wang, W.H.: Adaptive reinforcement learning in box-pushing robots. In: 2014 IEEE International Conference on Automation Science and Engineering (CASE), pp. 1182–1187, August 2014.
  3. 3.
    La, H.M., Lim, R., Sheng, W.: Multirobot cooperative learning for predator avoidance. IEEE Trans. Control Syst. Technol. 23(1), 52–63 (2015). Scholar
  4. 4.
    La, H.M., Lim, R.S., Sheng, W., Chen, J.: Cooperative flocking and learning in multi-robot systems for predator avoidance. In: 2013 IEEE International Conference on Cyber Technology in Automation, Control and Intelligent Systems, pp. 337–342, May 2013.
  5. 5.
    Parra-Gonzalez, E.F., Ramirez-Torres, J.G., Toscano-Pulido, G.: A new object path planner for the box pushing problem. In: 2009 Electronics, Robotics and Automotive Mechanics Conference (CERMA), pp. 119–124, September 2009.
  6. 6.
    Rakshit, P., Konar, A., Nagar, A.K.: Multi-robot box-pushing in presence of measurement noise. In: 2016 IEEE Congress on Evolutionary Computation (CEC), pp. 4926–4933, July 2016.
  7. 7.
    Wang, Y., Silva, C.W.D.: Multi-robot box-pushing: Single-agent q-learning vs. team q-learning. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3694–3699, October 2006.
  8. 8.
    Wang, Y., de Silva, C.W.: An object transportation system with multiple robots and machine learning. In: Proceedings of the 2005, American Control Conference, vol. 2, pp. 1371–1376, June 2005.
  9. 9.
    Yasuda, T., Ohkura, K., Yamada, K.: Multi-robot cooperation based on continuous reinforcement learning with two state space representations. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics, p. 4475, October 2013.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Electrical and Biomedical EngineeringUniversity of Nevada-RenoRenoUSA
  2. 2.Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and EngineeringUniversity of Nevada-RenoRenoUSA

Personalised recommendations