Skip to main content
SpringerLink
Log in

Neural Network Controller against Environment: A Coevolutive approach to Generalize Robot Navigation Behavior

  • Published:
Journal of Intelligent and Robotic Systems Aims and scope Submit manuscript

Abstract

In this paper, a new coevolutive method, called Uniform Coevolution, is introduced to learn weights of a neural network controller in autonomous robots. An evolutionary strategy is used to learn high-performance reactive behavior for navigation and collisions avoidance. The introduction of coevolutive over evolutionary strategies allows evolving the environment, to learn a general behavior able to solve the problem in different environments. Using a traditional evolutionary strategy method, without coevolution, the learning process obtains a specialized behavior. All the behaviors obtained, with/without coevolution have been tested in a set of environments and the capability of generalization is shown for each learned behavior. A simulator based on a mini-robot Khepera has been used to learn each behavior. The results show that Uniform Coevolution obtains better generalized solutions to examples-based problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Axelrod, R.: The Evolution of Cooperation, Basic Books, New York, 1984.

    Google Scholar 

  2. Axelrod, R.: Evolution of strategies in the iterated prisioner's dilemma, in: L. Davis (ed.), Genetic Algorithms and Simulated Annealing, Morgan Kaufman, 1989.

  3. Baluja, S.: Evolution of an artificial neural network based autonomus plan vehicle controller, IEEE Trans. Systems Man Cybernet. 26(3) (1996), 450–463.

    Google Scholar 

  4. Berlanga, A., Isasi, P., Molina, J.M., and Sanchis, A.: Competitive evolution to find generalized solutions: the arms race perspective, in: Proc. of Intelligent Engineering Systems INES, Austria, 1998, pp. 61–65.

  5. Berlanga, A., Isasi, P., Sanchis, A., and Molina, J. M.: Distance modulation competitive coevolution method to find initial configuration independent cellular automata rules, in: IEEE Internat. Conf. on Systems, Man and Cybernetics, Japan, 1999, pp. 607–612.

  6. Berlanga, A., Sanchis, A., Isasi, P., and Molina, J. M.: Neural networks robot controller trained with evolution strategies, in: Proc. of 1999 Congress on Evolutionary Computation, CEC99, 1999.

  7. Braitenberg, V.: Vehicles: Experiments on Synthetic Psychology, MIT Press Cambridge, MA, 1984.

    Google Scholar 

  8. Brooks, R. A.: Intelligence without representation, Artificial Intelligence 47 (1991), 139–159.

    Google Scholar 

  9. Brooks, R. A.: Artificial life and real robots, in: Toward a Practice of Autonomous Systems: Proc. of the 1st European Conf. on Artificial Life, MIT Press, Cambridge, MA, 1992.

    Google Scholar 

  10. Cliff, D. T., Husband, P., and Harvey, I.: Explorations in evolutionary robotics, Adaptive Behaviour (1993), 73–110.

  11. Cliff, D. and Miller, G. F.: Tracking the red queen: Measurements of adaptive progress in coevolutionary simulations, in: F. Morán, A. Moreno, J. J. Merelo and P. Chacón (eds), Proc. of the 3rd European Conf. on Artificial Life, Springer, Berlin, 1995, pp. 200–218.

    Google Scholar 

  12. Dorigo, M. and Snepf, U.: Genetics based machine learning and behavior based robotics: A new sysnthesis, IEEE Trans. Systems Man Cybernet. 23 (1993), 141–153.

    Google Scholar 

  13. Dorigo, M. and Colombetti, M.: Robot shaping: Developing autonomous agents through learning, Artificial Intelligence 71(2) (1994), 321–370.

    Google Scholar 

  14. Dorigo, M.: Alecsys and the autonomouse: Learning to control a real robot by distributed classifier systems, Machine Learning 19 (1995), 209–240.

    Google Scholar 

  15. Ficici, Sevan, G. and Pollack, J. B.: Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states, in: Adami, Belew, Kitano and Talor (eds), Proc. of the 6th Internat. Conf. on Artificial Life, MIT Press, Cambridge, MA, 1998.

    Google Scholar 

  16. Floreano, D. and Mondada, F.: Evolution of homming navigation in a real mobile robot, Proc. IEEE Trans. Systems Man Cybernet. 26(3) (1996), 396–407.

    Google Scholar 

  17. Floreano, D. and Nolfi, S.: Adaptive behavior in competing co-evolving species, in: Proc. of the 4th European Workshop on Evolutionary Robotics, Springer, Berlin, 1997.

    Google Scholar 

  18. Franceschini, N., Pichon, J. M., and Blanes, C.: Real time visuomotor control: from flies to robots, in: Proc. of the 5th Internat. Conf. on Advanced Robotics, 1991, pp. 91–95.

  19. García-Martínez, R. and Borrajo, D.: An integrated approach of learning, planning, and execution, J. Intelligent Robotic Systems (2000), accepted for publication.

  20. Gaudiano, P., Zalama, E., and Lopez, J.: An unsupervised neural network for low level control of a wheeled mobile robot: noise resistance stability and hardware implementation, IEEE Trans. Systems Man Cybernet. 26(3) (June 1996), 485–495.

    Google Scholar 

  21. Goldberg, D.: Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, New York, 1989.

    Google Scholar 

  22. Graf, D. H. and LaLoncle, W. R.: A neural controller for collision-free movement of general robot manipulators, in: Proc. of the IEEE 2nd Internat. Conf. on Neural Networks, Vol. I, 1988, pp. 77-84.

    Google Scholar 

  23. Handley, S. G.: The genetic planner: The automatic generation of plans for a mobile robot via genetic programming, in: Proc. of IEEE Internat. Symp. on Intelligent Control, Chicago, 1994, pp. 190–195.

  24. Hillis, W. D.: Co-evolving parasites improve simulated evolution as an optimization procedure, in: C. G. Langton (ed.), Artificial Life II, Santa Fe Institute, Addison-Wesley, Readin, MA, 1991, pp. 313–324.

    Google Scholar 

  25. Isasi, P., Berlanga, A., Molina, J. M., and Sanchis, A.: Robot controller against environment, a competitive evolution, in: Special Session on Evolution Computation, 15th IMACS World Congress 1997 on Scientific Computation, Modelling and Applied Mathematics, Germany, 1997.

  26. Ishikawa, S.: A method of autonomous mobile robot navigation by using fuzzy control, Adv. Robotics 9(1) (1995), 29–52.

    Google Scholar 

  27. Koza, J.: Genetic Programming, MIT Press, Cambridge, MA, 1992.

    Google Scholar 

  28. Koza, J.: Evolution of subsumption architecture that perform a wall following task for an autonomous mobile robot via genetic programming, in: Computational Learning Theory and Natural Learning Systems, Vol. 2, MIT Press, Cambridge, MA, 1994, pp. 321–346.

    Google Scholar 

  29. Kwok, D. P., Leung, T. P., and Feng, S.: Genetic algorithms for the optimal dynamic control of robot arms, in: Proc. of the 19th Annual Conf. of IEEE Industrial Electronic Society, Vol. 1, Maui, 1993, pp. 381–385.

    Google Scholar 

  30. Lindergren, K. and Nordahl, M. G.: Artificial food webs, in: Artificial Life III, Addison-Wesley, Reading, MA, 1994, pp. 73–103.

    Google Scholar 

  31. Maes, P. and Brooks, R.: Learning to coordinate behaviors, in: Proc. of the 8th National Conf. on Arfificial Intelligence, Morgan Kaufmann, San Mateo, CA, 1990, pp. 796-802.

    Google Scholar 

  32. Mahavedan, S. and Connell, J.: Automatic programming of behavior-based robots using reinforcement learning, Artificial Intelligence 55(1992), 311–365.

    Google Scholar 

  33. Mataric, J. and Cliff, D.: Challenges in evolving controllers for physical robots, J. Robotics and Autonomous Systems 19(1) (1996), 67–83.

    Google Scholar 

  34. Matellán, V., Fernández, C., and Molina, J. M.: Genetic learning of fuzzy reactive controllers, Robotics Autonom. Systems 25(1/2) (1998), 33–41.

    Google Scholar 

  35. McKerrow, P. J.: Introduction to Robotics, Addison-Wesley, Reading, MA, 1991.

    Google Scholar 

  36. Miglino, O., Hautop, H., and Nolfi, S.: Evolving mobile robots in simulated and real environment, Artificial Life 2 (1995), 417–434.

    Google Scholar 

  37. Millán, J. R.: Rapid, safe, an incremental learning of navigation strategies, IEEE Trans. Systems Man Cybernet. 26(3) (June 1996), 408–420.

    Google Scholar 

  38. Molina, J. M., Sanchis, A., Berlanga, A., and Isasi, P.: An enhanced classifier system for autonomous robot navigation in dynamic environments, in: Intelligent Automation and Soft Computing, Autosoft Press, 1998, in press.

  39. Mondada, F. and Franzi, P. I.: Mobile robot miniaturization: A tool for investigation in control algorithms, in: Proc. of the 2nd Internat. Conf. on Fuzzy Systems, San Francisco, USA, 1993.

  40. Nagata, S., Sekiguchi, M., and Asakawa, K.: Mobile robot control by a structures hierarchical neural network, IEEE Control Systems Mag. (April 1990), 69–76.

  41. Paredis, J.: Coevolutionary computation, Artificial Life 2 (1996), 355–375.

    Google Scholar 

  42. Pomerleau, D.: Neural Network Perception for Mobile Robot Guidance, Kluwer Academic, Dordrecht, 1993.

    Google Scholar 

  43. Pollack, J. B., Blair, A. D., and Land, M.: Coevolution of a backgammon player, in: V. C. G. Langton (ed.), Proc. of Artificial Life, MIT Press, Reading, MA, 1996.

    Google Scholar 

  44. Rechenberg, I.: Evolutionsstrategie: Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution, Frommann-Holzboog, Stuttgart, 1973.

    Google Scholar 

  45. Rechenberg, I.: Evolution strategy: Nature's way of optimization, in: Optimization: Methods and Applications, Possibilities and Limitations, Springer, Berlin, 1989, pp. 106–126.

    Google Scholar 

  46. Rosin, C. D. and Belew, R. K.: New methods for competitive coevolution, Evolutionary Computation 5 (1997), 1–29.

    Google Scholar 

  47. Sanchis, A., Molina, J. M., Isasi, P., and Segovia, J.: RTCS: A reactive with tags classifier system, J. Intelligent Robotic Systems (1999), in press.

  48. Molina, J. M., Sanchis, A., Berlanga, A., and Isasi, P.: An enhanced classifier system for autonomous robot navigation in dynamic environments, Intelligent Automat. Soft Comput. 6(2) (2000), 113–124.

    Google Scholar 

  49. Shakey, N.: Shakey the robot, Technical Report, SRI A.I., 1984.

  50. Solano, J. and Jones, D. I.: Generation of collision-free paths, a genetic approach, in: Proc. of the IEEE Colloquium on Genetic Algorithms for Control and Systems Engineering, London, 1993.

  51. Sommaruga, L., Merino, I., Matellán, V., and Molina, J.: A distributed simulator for intelligent autonomous robots, in: 4th Internat. Symp. on Intelligent Robotic Systems-SIRS96, Lisboa, Portugal, 1996.

  52. Schwefel, H. P.: Numerical Optimization of Computer Models, Wiley, New York, 1981.

    Google Scholar 

  53. Stone, P. and Veloso, M.: A layered approach to learning client behaviors in the robocup soccer server, Appl. Artificial Intelligence J. 12 (1998), 165–188.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ScaLab, Universidad Carlos III, Avda Universidad, 30, 28911, Madrid, Spain

    A. Berlanga, A. Sanchis, P. Isasi & J. M. Molina

Authors
  1. A. Berlanga
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Sanchis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. Isasi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. M. Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Berlanga, A., Sanchis, A., Isasi, P. et al. Neural Network Controller against Environment: A Coevolutive approach to Generalize Robot Navigation Behavior. Journal of Intelligent and Robotic Systems 33, 139–166 (2002). https://doi.org/10.1023/A:1014643811186

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1014643811186

Search

Navigation