Skip to main content

Testing Autonomous Robot Control Software Using Procedural Content Generation

  • Conference paper
Computer Safety, Reliability, and Security (SAFECOMP 2013)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 8153))

Included in the following conference series:

Abstract

We present a novel approach for reducing manual effort when testing autonomous robot control algorithms. We use procedural content generation, as developed for the film and video game industries, to create a diverse range of test situations. We execute these in the Player/Stage robot simulator and automatically rate them for their safety significance using an event-based scoring system. Situations exhibiting dangerous behaviour will score highly, and are thus flagged for the attention of a safety engineer. This process removes the time-consuming tasks of hand-crafting and monitoring situations while testing an autonomous robot control algorithm. We present a case study of the proposed approach – we generated 500 randomised situations, and our prototype tool simulated and rated them. We have analysed the three highest rated situations in depth, and this analysis revealed weaknesses in the smoothed nearness-diagram control algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Fletcher, L., Teller, S., Olson, E., Moore, D., Kuwata, Y., How, J., Leonard, J., Miller, I., Campbell, M., Huttenlocher, D., Nathan, A., Kline, F.-R.: The MIT – Cornell Collision and Why it Happened. Journal of Field Robotics 25, 775–807 (2008)

    Article  Google Scholar 

  2. Martin, G.A., Hughes, C.E.: A Scenario Generation Framework for Automating Instructional Support in Scenario-based Training. In: Proceedings of the Spring Simulation Multiconference (2010)

    Google Scholar 

  3. Knight, J.C., Leveson, N.G.: A Large Scale Experiment In N-Version Programming. In: Proceedings of the Fifteenth International Symposium on Fault-Tolerant Computing, Ann Arbor, MI, pp. 135–139 (1985)

    Google Scholar 

  4. Togelius, J., Yannakakis, G.N., Stanley, K.O., Browne, C.: Search-based Procedural Content Generation: A Taxonomy and Survey. IEEE Transactions on Computational Intelligence and AI in Games (2011)

    Google Scholar 

  5. Togelius, J., Preuss, M., Yannakakis, G.N.: Towards multiobjective procedural map generation. In: Proceedings of the Workshop on Procedural Content Generation in Games (PGGames 2010), Monterey, CA (2010)

    Google Scholar 

  6. Roden, T., Parberry, I.: From Artistry to Automation: A Structured Methodology for Procedural Content Creation. In: Rauterberg, M. (ed.) ICEC 2004. LNCS, vol. 3166, pp. 151–156. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  7. Togelius, J., Yannakakis, G.N., Stanley, K.O., Browne, C.: Search-Based Procedural Content Generation Applications of Evolutionary Computation (2010)

    Google Scholar 

  8. Nguyen, C.D., Perini, A., Tonella, P.: Ontology-based test generation for multiagent systems. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1315–1320 (2008)

    Google Scholar 

  9. Archer, J.: Indicators for traffic safety assessment and prediction and their application in micro-simulation modelling. PhD thesis, KTH, Stockholm (2005)

    Google Scholar 

  10. Gerkey, B.P., Vaughan, R.T., Howard, A.: The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems. In: Proceedings of the Intl. Conf. on Advanced Robotics (ICAR), Coimbra, Portugal, pp. 317–323 (2003)

    Google Scholar 

  11. Perlin, K.: An Image Synthesizer. SIGGRAPH Comput. Graph. 19, 287–296 (1985)

    Article  Google Scholar 

  12. Hart, P.E.: A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics 4, 100–107 (1968)

    Article  Google Scholar 

  13. Durham, J., Bullo, F.: Smooth Nearness-Diagram Navigation. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2008), pp. 690–695. IEEE (2008)

    Google Scholar 

  14. Arnold, J.: Robot Hazard Analysis using Procedural Content Generation. MEng thesis, University of York (2012)

    Google Scholar 

  15. Parish, Y.I.H., Müller, P.: Procedural Modeling of Cities. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, New York, USA, pp. 301–308 (2001)

    Google Scholar 

  16. Ashlock, D.A., Manikas, T.W., Ashenayi, K.: Evolving A Diverse Collection of Robot Path Planning Problems. In: IEEE Congress on Evolutionary Computation, Vancouver, BC, Canada (2006)

    Google Scholar 

  17. Nguyen, C.D., Perini, A., Tonella, P., Miles, S., Harman, M., Luck, M.: Evolutionary testing of autonomous software agents. In: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2009 (2009)

    Google Scholar 

  18. Wardziński, A.: The role of situation awareness in assuring safety of autonomous vehicles. In: Górski, J. (ed.) SAFECOMP 2006. LNCS, vol. 4166, pp. 205–218. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  19. Nguyen, C.D., Perini, A., Tonella, P.: Constraint-based Evolutionary Testing of Autonomous Distributed. In: Proceedings of the Software Testing Verification and Validation Workshop, Lillehammer, pp. 221–230 (2008)

    Google Scholar 

  20. Fiorini, P., Shiller, Z.: Motion Planning in Dynamic Environments Using the Relative Velocity Paradigm. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 560–565 (1993)

    Google Scholar 

  21. Large, F., Laugier, C., Shiller, Z.: Navigation among moving obstacles using the NLVO: Principles and applications to intelligent vehicles. Autonomous Robots 19, 159–171 (2005)

    Article  Google Scholar 

  22. Fulgenzi, C., Spalanzani, A., Laugier, C.: Dynamic Obstacle Avoidance in uncertain environment combining PVOs and Occupancy Grid. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1610–1616 (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Arnold, J., Alexander, R. (2013). Testing Autonomous Robot Control Software Using Procedural Content Generation. In: Bitsch, F., Guiochet, J., Kaâniche, M. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2013. Lecture Notes in Computer Science, vol 8153. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40793-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-40793-2_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-40792-5

  • Online ISBN: 978-3-642-40793-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics