Abstract
Industries such as flexible manufacturing and home care will be transformed by the presence of robotic assistants. Assurance of safety and functional soundness for these robotic systems will require rigorous verification and validation. We propose testing in simulation using Coverage-Driven Verification (CDV) to guide the testing process in an automatic and systematic way. We use a two-tiered test generation approach, where abstract test sequences are computed first and then concretized (e.g., data and variables are instantiated), to reduce the complexity of the test generation problem. To demonstrate the effectiveness of our approach, we developed a testbench for robotic code, running in ROS-Gazebo, that implements an object handover as part of a human-robot interaction (HRI) task. Tests are generated to stimulate the robot’s code in a realistic manner, through stimulating the human, environment, sensors, and actuators in simulation. We compare the merits of unconstrained, constrained and model-based test generation in achieving thorough exploration of the code under test, and interesting combinations of human-robot interactions. Our results show that CDV combined with systematic test generation achieves a very high degree of automation in simulation-based verification of control code for robots in HRI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Standards ISO 13482:2014 for robotic assistants and ISO 10218 (parts I and II) for industrial robotics.
- 2.
- 3.
- 4.
Available at: https://github.com/robosafe/testbench-v3.
- 5.
- 6.
References
Alexander, R., Hawkins, H., Rae, D.: Situation coverage - a coverage criterion for testing autonomous robots. Technical report, Department of Computer Science, University of York (2015)
Araiza-Illan, D., Western, D., Pipe, A., Eder, K.: Coverage-driven verification — an approach to verify code for robots that directly interact with humans. In: Piterman, N., et al. (eds.) HVC 2015. LNCS, vol. 9434, pp. 69–84. Springer, Heidelberg (2015). doi:10.1007/978-3-319-26287-1_5
Bird, D., Munoz, C.: Automatic generation of random self-checking test cases. IBM Syst. J. 22(3), 229–245 (1983)
Boren, J., Cousins, S.: The SMACH high-level executive. IEEE Robot. Autom. Mag. 17(4), 18–20 (2010)
Clarke, E.M., Grumberg, O., Peled, D.A.: Model Checking. MIT Press, Cambridge (1999)
Eder, K., Harper, C., Leonards, U.: Towards the safety of human-in-the-loop robotics: challenges and opportunities for safety assurance of robotic co-workers. In: Proceedings of ROMAN, pp. 660–665 (2014)
Gaudel, M.-C.: Counting for random testing. In: Wolff, B., Zaïdi, F. (eds.) ICTSS 2011. LNCS, vol. 7019, pp. 1–8. Springer, Heidelberg (2011)
Grigore, E.C., Eder, K., Lenz, A., Skachek, S., Pipe, A.G., Melhuish, C.: Towards safe human-robot interaction. In: Groß, R., Alboul, L., Melhuish, C., Witkowski, M., Prescott, T.J., Penders, J. (eds.) TAROS 2011. LNCS, vol. 6856, pp. 323–335. Springer, Heidelberg (2011)
Haedicke, F., Le, H., Grosse, D., Drechsler, R.: CRAVE: an advanced constrained random verification environment for SystemC. In: Proceedings of SoC, pp. 1–7 (2012)
Hartmanns, A., Hermanns, H.: A modest approach to checking probabilistic timed automata. In: Proceedings of QEST, pp. 187–196 (2009)
Julius, A.A., Fainekos, G.E., Anand, M., Lee, I., Pappas, G.J.: Robust test generation and coverage for hybrid systems. In: Bemporad, A., Bicchi, A., Buttazzo, G. (eds.) HSCC 2007. LNCS, vol. 4416, pp. 329–342. Springer, Heidelberg (2007)
Kim, J., Esposito, J.M., Kumar, R.: Sampling-based algorithm for testing and validating robot controllers. Int. J. Robot. Res. 25(12), 1257–1272 (2006)
Lackner, H., Schlingloff, B.: Modeling for automated test generation a comparison. In: Proceedings of MBEES Workshop (2012)
Lakhotia, K., McMinn, P., Harman, M.: Automated test data generation for coverage: havent we solved this problem yet? In: Proceedings of TAIC (2009)
Lenz, A., Skachek, S., Hamann, K., Steinwender, J., Pipe, A., Melhuish, C.: The BERT2 infrastructure: an integrated system for the study of human-robot interaction. In: Proceedings of IEEE-RAS Humanoids, pp. 346–351 (2010)
Mossige, M., Gotlieb, A., Meling, H.: Testing robot controllers using constraint programming and continuous integration. Inf. Softw. Technol. 57, 169–185 (2014)
Nielsen, B., Skou, A.: Automated test generation from timed automata. Int. J. Softw. Tools Technol. Transfer. 5, 59–77 (2003)
Nielsen, B.: Towards a method for combined model-based testing and analysis. In: Proceedings of MODELSWARD, pp. 609–618 (2014)
Petters, S., Thomas, D., Friedmann, M., von Stryk, O.: Multilevel testing of control software for teams of autonomous mobile robots. In: Carpin, S., Noda, I., Pagello, E., Reggiani, M., von Stryk, O. (eds.) SIMPAR 2008. LNCS (LNAI), vol. 5325, pp. 183–194. Springer, Heidelberg (2008)
Pinho, T., Moreira, A.P., Boaventura-Cunha, J.: Framework using ROS and SimTwo simulator for realistic test of mobile robot controllers. In: Proceedings of CONTROLO, pp. 751–759 (2014)
Piziali, A.: Functional verification coverage measurement and analysis. Kluwer Academic (2004)
Sankaranarayanan, S., Fainekos, G.E.: Falsification of temporal properties of hybrid systems using the cross-entropy method. In: Proceedings of HSCC, pp. 125–134 (2012)
Stocker, R., Dennis, L., Dixon, C., Fisher, M.: Verifying brahms human-robot teamwork models. In: del Cerro, L.F., Herzig, A., Mengin, J. (eds.) JELIA 2012. LNCS, vol. 7519, pp. 385–397. Springer, Heidelberg (2012)
Trojanek, P., Eder, K.: Verification and testing of mobile robot navigation algorithms: a case study in SPARK. In: Proceedings of IROS, pp. 1489–1494 (2014)
Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Testi., Verification Reliab. 22, 297–312 (2012)
Webster, M., Dixon, C., Fisher, M., Salem, M., Saunders, J., Koay, K.L., Dautenhahn, K.: Formal verification of an autonomous personal robotic assistant. In: Proceedings of AAAI FVHMS, pp. 74–79 (2014)
Acknowledgement
This work is part of the EPSRC-funded project “Trustworthy Robotic Assistants” (refs. EP/K006320/1 and EP/K006223/1).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Araiza-Illan, D., Western, D., Pipe, A.G., Eder, K. (2016). Systematic and Realistic Testing in Simulation of Control Code for Robots in Collaborative Human-Robot Interactions. In: Alboul, L., Damian, D., Aitken, J. (eds) Towards Autonomous Robotic Systems. TAROS 2016. Lecture Notes in Computer Science(), vol 9716. Springer, Cham. https://doi.org/10.1007/978-3-319-40379-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-319-40379-3_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-40378-6
Online ISBN: 978-3-319-40379-3
eBook Packages: Computer ScienceComputer Science (R0)