Skip to main content
SpringerLink
Log in

Testing Autonomous and Highly Configurable Systems: Challenges and Feasible Solutions

  • Chapter
  • First Online:
Automated Driving

Abstract

Proving techniques and methods for safety critical systems in order to ensure a certain behavior as well as their corresponding safety requirements has still been a challenge for many years. Although the current situation in many areas like the automotive industry has improved a lot, new challenges are in sight especially when considering autonomous and adaptive systems approaching. Such systems have to reason about the current state and stimuli from their environment without humans in the loop or are allowed to change their behavior over time. Such systems induce new requirements for quality assurance and in particular testing. Here the focus has to be on providing guarantees of a wanted behavior before deployment of the systems even in case of changes or failures that might arise at runtime. In this paper, we discuss the underlying challenges and potential feasible solutions. In addition, we highlight similarities and differences with the current situation of testing safety critical systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. ISO 26262:2011: Road vehicles functional safety (International Organization for Standardization, Geneva, 2011)

    Google Scholar 

  2. N.G. Leveson, Engineering a Safer World (MIT Press, Cambridge, MA, 2011)

    Google Scholar 

  3. H. Altinger, F. Wotawa, M. Schurius, Testing Methods Used in the Automotive Industry: Results from a Survey. Proceedings JAMAICA’14, San Jose, CA, 21 July 2014

    Google Scholar 

  4. P. Ammann, J. Offutt, Introduction to Software Testing (Cambridge University Press, Cambridge, MA, 2008)

    Book  Google Scholar 

  5. J. Glenford, Myers, The Art of Software Testing (Wiley, New York, 1979)

    Google Scholar 

  6. I. Schieferdecker, Model-based testing. IEEE Software 29(1), 14–18 (2012)

    Article  Google Scholar 

  7. M. Utting, B. Legeard, Practical Model-Based Testing – A Tools Approach (Morgan Kaufmann, Burlington, MA, 2007)

    Google Scholar 

  8. J. Cámara, R. de Lemos, C. Ghezzi, A. Lopes (eds.), Assurances for Self-Adaptive Systems – Principles, Models, and Techniques. LNCS 7740 (Springer, Berlin, 2013)

    Google Scholar 

  9. F. Nafz, J. Steghöfer, H. Seebach, W. Reif, Formal Modeling and Verification of Self-Systems Based on Observer/Controller-Architectures, in LNCS 7740 [8] (Springer, Berlin, 2013), pp. 80–111.

    Google Scholar 

  10. G. Steinbauer, F. Wotawa, Model-Based Reasoning for Self-Adaptive Systems – Theory and Practice, in LNCS 7740 [8] (Springer, Berlin, 2013), pp. 187–213

    Google Scholar 

  11. M. Leucker, C. Schallhart, A brief account of runtime verification. J. Logic Algebr. Program. 78(5), 293–303 (2009)

    Article  Google Scholar 

  12. C. Artho, H. Barringer, A. Goldberg, K. Havelund, S. Khurshid, M. Lowry, C. Pasareanu, G. Rosu, K. Sen, W. Visser, R. Washington, Combining test case generation and runtime verification. Theor. Comput. Sci. 336(2–3), 209–234 (2005)

    Article  MathSciNet  Google Scholar 

  13. D. Kuhn, R. Kacker, Y. Lei, Introduction to Combinatorial Testing. Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series (Taylor & Francis, London, 2013)

    Google Scholar 

  14. D.R. Kuhn, R.N. Kacker, Y. Lei, J. Hunter, Combinatorial software testing. Computer August, 94–96 (2009)

    Article  Google Scholar 

  15. L. Yu, Y. Lei, R. Kacker, D. Kuhn, Acts: A combinatorial test generation tool, in IEEE Sixth International Conference on Software Testing, Verification and Validation (ICST), 2013, pp. 370–375

    Google Scholar 

  16. L. Yu, K. Raghu, D. Richard Kuhn, V. Okun, J. Lawrence, IPOG/IPOG-D: Efficient test generation for multi-way combinatorial testing. Software Test. Verif. Reliab. 18, 125–148 (2008)

    Article  Google Scholar 

  17. F. Wotawa, I. Pill, Testing Configuration Knowledge-Bases, in 16th International Configuration Workshop, 2014. http://ceur-ws.org/Vol-1220/06_confws2014_submission_13.pdf, pp. 39–46

  18. T.Y. Chen, S.C. Cheung, S.M. Yiu, Metamorphic Testing: A New Approach for Generating Next Test Cases, Technical Report HKUST-CS98-01 (Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong, 1998)

    Google Scholar 

  19. T.Y. Chen, J. Feng, T.H. Tse, Metamorphic Testing of Programs on Partial Differential Equations: A Case Study, in Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC ’02), Los Alamitos, CA (IEEE Computer Society, 2002), pp. 327–333

    Google Scholar 

Download references

Acknowledgment

The research work has been carried out as part of the 3CCar project co-funded by the Electronic Component Systems for European Leadership Joint Undertaking (ECSEL JU) grant agreement number 662192-3Ccar-ECSEL-2014-1 and the FFG grant agreement number 848715.

Author information

Authors and Affiliations

  1. Institute for Software Technology, Technische Universität Graz, Graz, Austria

    Franz Wotawa

Authors
  1. Franz Wotawa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Franz Wotawa .

Editor information

Editors and Affiliations

  1. Institute of Electrical Measurement and Measurement Signal Processing, Virtual Vehicle Research Center and Graz University of Technology, Graz, Austria

    Daniel Watzenig

  2. Institute of Automation and Control, Graz University of Technology, Graz, Austria

    Martin Horn

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Wotawa, F. (2017). Testing Autonomous and Highly Configurable Systems: Challenges and Feasible Solutions. In: Watzenig, D., Horn, M. (eds) Automated Driving. Springer, Cham. https://doi.org/10.1007/978-3-319-31895-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-31895-0_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-31893-6

  • Online ISBN: 978-3-319-31895-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation