Abstract
We present a novel approach for reducing manual effort when testing autonomous robot control algorithms. We use procedural content generation, as developed for the film and video game industries, to create a diverse range of test situations. We execute these in the Player/Stage robot simulator and automatically rate them for their safety significance using an event-based scoring system. Situations exhibiting dangerous behaviour will score highly, and are thus flagged for the attention of a safety engineer. This process removes the time-consuming tasks of hand-crafting and monitoring situations while testing an autonomous robot control algorithm. We present a case study of the proposed approach – we generated 500 randomised situations, and our prototype tool simulated and rated them. We have analysed the three highest rated situations in depth, and this analysis revealed weaknesses in the smoothed nearness-diagram control algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Fletcher, L., Teller, S., Olson, E., Moore, D., Kuwata, Y., How, J., Leonard, J., Miller, I., Campbell, M., Huttenlocher, D., Nathan, A., Kline, F.-R.: The MIT – Cornell Collision and Why it Happened. Journal of Field Robotics 25, 775–807 (2008)
Martin, G.A., Hughes, C.E.: A Scenario Generation Framework for Automating Instructional Support in Scenario-based Training. In: Proceedings of the Spring Simulation Multiconference (2010)
Knight, J.C., Leveson, N.G.: A Large Scale Experiment In N-Version Programming. In: Proceedings of the Fifteenth International Symposium on Fault-Tolerant Computing, Ann Arbor, MI, pp. 135–139 (1985)
Togelius, J., Yannakakis, G.N., Stanley, K.O., Browne, C.: Search-based Procedural Content Generation: A Taxonomy and Survey. IEEE Transactions on Computational Intelligence and AI in Games (2011)
Togelius, J., Preuss, M., Yannakakis, G.N.: Towards multiobjective procedural map generation. In: Proceedings of the Workshop on Procedural Content Generation in Games (PGGames 2010), Monterey, CA (2010)
Roden, T., Parberry, I.: From Artistry to Automation: A Structured Methodology for Procedural Content Creation. In: Rauterberg, M. (ed.) ICEC 2004. LNCS, vol. 3166, pp. 151–156. Springer, Heidelberg (2004)
Togelius, J., Yannakakis, G.N., Stanley, K.O., Browne, C.: Search-Based Procedural Content Generation Applications of Evolutionary Computation (2010)
Nguyen, C.D., Perini, A., Tonella, P.: Ontology-based test generation for multiagent systems. In: Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 1315–1320 (2008)
Archer, J.: Indicators for traffic safety assessment and prediction and their application in micro-simulation modelling. PhD thesis, KTH, Stockholm (2005)
Gerkey, B.P., Vaughan, R.T., Howard, A.: The Player/Stage Project: Tools for Multi-Robot and Distributed Sensor Systems. In: Proceedings of the Intl. Conf. on Advanced Robotics (ICAR), Coimbra, Portugal, pp. 317–323 (2003)
Perlin, K.: An Image Synthesizer. SIGGRAPH Comput. Graph. 19, 287–296 (1985)
Hart, P.E.: A Formal Basis for the Heuristic Determination of Minimum Cost Paths. IEEE Transactions on Systems Science and Cybernetics 4, 100–107 (1968)
Durham, J., Bullo, F.: Smooth Nearness-Diagram Navigation. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2008), pp. 690–695. IEEE (2008)
Arnold, J.: Robot Hazard Analysis using Procedural Content Generation. MEng thesis, University of York (2012)
Parish, Y.I.H., Müller, P.: Procedural Modeling of Cities. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, New York, USA, pp. 301–308 (2001)
Ashlock, D.A., Manikas, T.W., Ashenayi, K.: Evolving A Diverse Collection of Robot Path Planning Problems. In: IEEE Congress on Evolutionary Computation, Vancouver, BC, Canada (2006)
Nguyen, C.D., Perini, A., Tonella, P., Miles, S., Harman, M., Luck, M.: Evolutionary testing of autonomous software agents. In: Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2009 (2009)
Wardziński, A.: The role of situation awareness in assuring safety of autonomous vehicles. In: Górski, J. (ed.) SAFECOMP 2006. LNCS, vol. 4166, pp. 205–218. Springer, Heidelberg (2006)
Nguyen, C.D., Perini, A., Tonella, P.: Constraint-based Evolutionary Testing of Autonomous Distributed. In: Proceedings of the Software Testing Verification and Validation Workshop, Lillehammer, pp. 221–230 (2008)
Fiorini, P., Shiller, Z.: Motion Planning in Dynamic Environments Using the Relative Velocity Paradigm. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 560–565 (1993)
Large, F., Laugier, C., Shiller, Z.: Navigation among moving obstacles using the NLVO: Principles and applications to intelligent vehicles. Autonomous Robots 19, 159–171 (2005)
Fulgenzi, C., Spalanzani, A., Laugier, C.: Dynamic Obstacle Avoidance in uncertain environment combining PVOs and Occupancy Grid. In: Proceedings of the IEEE International Conference on Robotics and Automation, pp. 1610–1616 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Arnold, J., Alexander, R. (2013). Testing Autonomous Robot Control Software Using Procedural Content Generation. In: Bitsch, F., Guiochet, J., Kaâniche, M. (eds) Computer Safety, Reliability, and Security. SAFECOMP 2013. Lecture Notes in Computer Science, vol 8153. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40793-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-642-40793-2_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40792-5
Online ISBN: 978-3-642-40793-2
eBook Packages: Computer ScienceComputer Science (R0)