Skip to main content

GP for Continuous Control: Teacher or Learner? The Case of Simulated Modular Soft Robots

  • Chapter
  • First Online:
Genetic Programming Theory and Practice XX

Part of the book series: Genetic and Evolutionary Computation ((GEVO))

Abstract

We consider the problem of optimizing a controller for agents whose observation and action spaces are continuous, i.e., where the controller is a multivariate real function \(f: \mathbb {R}^n \rightarrow \mathbb {R}^m\). We use genetic programming (GP) for solving this optimization problem. Namely, we employ a multi-tree-based GP variant, where a candidate solution is an array of m trees, each encoding a univariate function of the agent observation. We compare this form of optimization against the more common one where the controller is a multi-layer perceptron, with a predefined topology, whose weights are optimized through (neuro)evolution (NE). Moreover, we consider an evolutionary algorithm, GraphEA, that directly evolves graphs, each having n input nodes and m output nodes. We apply these three approaches to the case of simulated modular soft robots , where a robot is an aggregation of identical soft modules, each employing a controller that processes the local observation and produces the local action. We find that, in our scenario, multi-tree-based GP is competitive with NE and tends to produce different behaviors. We then experimentally investigate the possibility of optimizing a controller using another, pre-optimized one, as teacher, i.e., we realize a form of offline imitation learning . We consider all the teacher-learner pairs resulting from the three evolutionary algorithms and find that NE is a better learner than GP and GraphEA. However, controllers obtained through offline imitation learning are far less effective than those obtained through direct evolution. We hypothesize that this gap in effectiveness may be explained by the possibility, given by direct evolution, of exploring during the simulations a larger portion of the observation-action space.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bacardit, J., Brownlee, A.E., Cagnoni, S., Iacca, G., McCall, J., Walker, D.: The intersection of evolutionary computation and explainable AI. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 1757–1762 (2022)

    Google Scholar 

  2. Bartoli, A., De Lorenzo, A., Medvet, E., Squillero, G.: Multi-level diversity promotion strategies for grammar-guided genetic programming. Appl. Soft Comput. 83, 105599 (2019)

    Article  Google Scholar 

  3. Ferigo, A., Iacca, G., Medvet, E., Pigozzi, F.: Evolving Hebbian learning rules in voxel-based soft robots. IEEE Trans. Cogn. Dev. Syst. (2022)

    Google Scholar 

  4. Floreano, D., Dürr, P., Mattiussi, C.: Neuroevolution: from architectures to learning. Evol. Intell. 1, 47–62 (2008)

    Article  Google Scholar 

  5. Françoso Dal Piccol Sotto, L., Kaufmann, P., Atkinson, T., Kalkreuth, R., Porto Basgalupp, M.: Graph representations in genetic programming. Genet. Program. Evolvable Mach. 22(4), 607–636 (2021)

    Google Scholar 

  6. Harding, S., Miller, J.F.: Evolution of robot controller using cartesian genetic programming. In: Genetic Programming: 8th European Conference, EuroGP 2005, Lausanne, Switzerland, March 30-April 1, 2005. Proceedings 8, pp. 62–73. Springer (2005)

    Google Scholar 

  7. Hiller, J., Lipson, H.: Automatic design and manufacture of soft robots. IEEE Trans. Robot. 28(2), 457–466 (2012)

    Article  Google Scholar 

  8. Jin, L., Li, S., Yu, J., He, J.: Robot manipulator control using neural networks: a survey. Neurocomputing 285, 23–34 (2018)

    Article  Google Scholar 

  9. Kadlic, B., Sekaj, I., Perneckỳ, D.: Design of continuous-time controllers using cartesian genetic programming. IFAC Proc. Vol. 47(3), 6982–6987 (2014)

    Article  Google Scholar 

  10. Koza, J.R., Rice, J.P.: Automatic programming of robots using genetic programming. In: AAAI, vol. 92, pp. 194–207 (1992)

    Google Scholar 

  11. La Cava, W., Orzechowski, P., Burlacu, B., de França, F.O., Virgolin, M., Jin, Y., Kommenda, M., Moore, J.H.: Contemporary symbolic regression methods and their relative performance (2021). arXiv:2107.14351

  12. Legrand, J., Terryn, S., Roels, E., Vanderborght, B.: Reconfigurable, multi-material, voxel-based soft robots. IEEE Robot. Autom. Lett. (2023)

    Google Scholar 

  13. Lewis, M.A., Fagg, A.H., Solidum, A., et al.: Genetic programming approach to the construction of a neural network for control of a walking robot. In: ICRA, pp. 2618–2623. Citeseer (1992)

    Google Scholar 

  14. Lobov, S.A., Zharinov, A.I., Makarov, V.A., Kazantsev, V.B.: Spatial memory in a spiking neural network with robot embodiment. Sens. 21(8), 2678 (2021)

    Article  Google Scholar 

  15. Medvet, E., Bartoli, A.: Evolutionary optimization of graphs with graphea. In: International Conference of the Italian Association for Artificial Intelligence, pp. 83–98. Springer (2021)

    Google Scholar 

  16. Medvet, E., Bartoli, A., De Lorenzo, A., Fidel, G.: Evolution of distributed neural controllers for voxel-based soft robots. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pp. 112–120 (2020a)

    Google Scholar 

  17. Medvet, E., Bartoli, A., De Lorenzo, A., Seriani, S.: 2D-VSR-Sim: a simulation tool for the optimization of 2-D voxel-based soft robots. SoftwareX 12, 100573 (2020)

    Article  Google Scholar 

  18. Medvet, E., Nadizar, G., Manzoni, L.: JGEA: a modular java framework for experimenting with evolutionary computation. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 2009–2018 (2022a)

    Google Scholar 

  19. Medvet, E., Nadizar, G., Pigozzi, F.: On the impact of body material properties on neuroevolution for embodied agents: the case of voxel-based soft robots. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 2122–2130 (2022b)

    Google Scholar 

  20. Medvet, E., Pozzi, S., Manzoni, L.: A general purpose representation and adaptive EA for evolving graphs. In: Proceedings of the Genetic and Evolutionary Computation Conference (2023)

    Google Scholar 

  21. Mei, Y., Chen, Q., Lensen, A., Xue, B., Zhang, M.: Explainable artificial intelligence by genetic programming: a survey. IEEE Trans. Evol. Comput. (2022)

    Google Scholar 

  22. Miller, J.F., Harding, S.L.: Cartesian genetic programming. In: Proceedings of the 10th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 2701–2726 (2008)

    Google Scholar 

  23. Nadizar, G., Medvet, E., Miras, K.: On the schedule for morphological development of evolved modular soft robots. In: European Conference on Genetic Programming (Part of EvoStar), pp. 146–161. Springer (2022a)

    Google Scholar 

  24. Nadizar, G., Medvet, E., Nichele, S., Pontes-Filho, S.: An experimental comparison of evolved neural network models for controlling simulated modular soft robots. Appl. Soft Comput. 110610 (2023a)

    Google Scholar 

  25. Nadizar, G., Medvet, E., Ramstad, H.H., Nichele, S., Pellegrino, F.A., Zullich, M.: Merging pruning and neuroevolution: towards robust and efficient controllers for modular soft robots. Knowl. Eng. Rev. 37 (2022b)

    Google Scholar 

  26. Nadizar, G., Medvet, E., Walker, K., Risi, S.: A fully-distributed shape-aware neural controller for modular robots. In: Proceedings of the Genetic and Evolutionary Computation Conference (2023b)

    Google Scholar 

  27. Nolfi, S.: Behavioral and Cognitive Robotics: an Adaptive Perspective. Stefano Nolfi (2021)

    Google Scholar 

  28. Nordin, P., Banzhaf, W.: Genetic programming controlling a miniature robot. In: Working Notes for the AAAI Symposium on Genetic Programming, vol. 61, p. 67. MIT, Cambridge, MA, USA, AAAI (1995)

    Google Scholar 

  29. Pfeifer, R., Bongard, J.: How the body shapes the way we think: a new view of intelligence. MIT press (2006)

    Google Scholar 

  30. Pigozzi, F., Tang, Y., Medvet, E., Ha, D.: Evolving modular soft robots without explicit inter-module communication using local self-attention. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 148–157 (2022)

    Google Scholar 

  31. Poli, R., Langdon, W.B., McPhee, N.F.: A field guide to genetic programming (2008)

    Google Scholar 

  32. Salvato, E., Fenu, G., Medvet, E., Pellegrino, F.A.: Crossing the reality gap: a survey on sim-to-real transferability of robot controllers in reinforcement learning. IEEE Access (2021)

    Google Scholar 

  33. Seo, K., Hyun, S.: Toward automatic gait generation for quadruped robots using cartesian genetic programming. In: Applications of Evolutionary Computation: 16th European Conference, EvoApplications 2013, Vienna, Austria, April 3–5, 2013. Proceedings 16, pp. 599–605. Springer (2013)

    Google Scholar 

  34. Squillero, G., Tonda, A.: Divergence of character and premature convergence: a survey of methodologies for promoting diversity in evolutionary optimization. Inf. Sci. 329, 782–799 (2016)

    Article  Google Scholar 

  35. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  36. Sui, X., Cai, H., Bie, D., Zhang, Y., Zhao, J., Zhu, Y.: Automatic generation of locomotion patterns for soft modular reconfigurable robots. Appl. Sci. 10(1), 294 (2020)

    Article  Google Scholar 

  37. Talamini, J., Medvet, E., Bartoli, A., De Lorenzo, A.: Evolutionary synthesis of sensing controllers for voxel-based soft robots. In: ALIFE 2019: The 2019 Conference on Artificial Life, pp. 574–581. MIT Press (2019)

    Google Scholar 

  38. Turner, A.J., Miller, J.F.: Recurrent cartesian genetic programming. In: Parallel Problem Solving from Nature–PPSN XIII: 13th International Conference, Ljubljana, Slovenia, September 13–17, 2014. Proceedings 13, pp. 476–486. Springer (2014)

    Google Scholar 

  39. Virgolin, M., Alderliesten, T., Witteveen, C., Bosman, P.A.: Improving model-based genetic programming for symbolic regression of small expressions. Evol. Comput. 29(2), 211–237 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Eric Medvet .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Medvet, E., Nadizar, G. (2024). GP for Continuous Control: Teacher or Learner? The Case of Simulated Modular Soft Robots. In: Winkler, S., Trujillo, L., Ofria, C., Hu, T. (eds) Genetic Programming Theory and Practice XX. Genetic and Evolutionary Computation. Springer, Singapore. https://doi.org/10.1007/978-981-99-8413-8_11

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8413-8_11

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8412-1

  • Online ISBN: 978-981-99-8413-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics