Abstract
Design, execution and evaluation of tests for safety-critical systems require considerable effort and skill and consume a large part of today's development costs. Due to the growing complexity of control systems, it has to be expected that their trustworthy test will become unmanageable in the future, if only conventional techniques, requiring a high degree of human interaction during the test process, are applied. In this article, we will focus on test automation for reactive real-time systems, with emphasis on Hardware-in-the-Loop tests analyzing the behaviour of combined software and hardware components. To illustrate possible approaches for this test problem, we describe a concept based on specifications written in Real-Time CSP. For the implementation of test generation and evaluation algorithms transition system representations are used, as can be obtained by Formal Systems' FDR tool. An industrial application of the method is presented and used for the evaluation of the benefits of formal methods-based testing in comparison with conventional techniques. Furthermore, we will indicate research topics in this field which are likely to become important for further improvements of the test process. Specifically, the benefits arising from an approach combining formal verification and testing will be discussed. Our presentation aims less at “promoting” a specific solution, but tries to illustrate the basic problems to be tackled with any formal method, when trying to develop test automation concepts to be applied in the context of reactive systems.
Chapter PDF
Similar content being viewed by others
References
K. R. Apt and E.-R. Olderog. Verification of Sequential and Concurrent Programs. Springer-Verlag, Berlin Heidelberg New York (1991).
E. Brinksma: A theory for the derivation of tests. In P. H. J. van Eijk-C. A. Vissers and M. Diaz (Eds.): The Formal Description Technique LOTOS. Elsevire Science Publishers B. V. (North-Holland), (1989), 235–247.
RTCA DO178B: Development considerations in airborne computer systems. (1993).
J. Davies: Specification and Proof in Real-Time CSP. Cambridge University Press (1993).
ELPRO LET GmbH: Programmablaufplan — Bahnübergang. ELPRO LET GmbH (1994).
Formal Systems Ltd.: Failures Divergence Refinement. User Manual and Tutorial Version 1.4. Formal Systems (Europe) Ltd (1994).
M.-C. Gaudel: Testing can be formal, too. In P. D. Mosses, M. Nielsen and M. I. Schwartzbach (Eds.): Proceedings of TAPSOFT '95: Theory and Practice of Software Development. Aarhus, Denmark, May 1995, Springer (1995).
M. C. Hennessy: Algebraic Theory of Processes. MIT Press (1988).
C.A.R. Hoare. Communicating sequential processes. Prentice-Hall International, Englewood Cliffs NJ (1985).
H. M. Hörcher and J. Peleska: The Role of Formal Specifications in Software Test. Tutorial, held at the FME '94.
H. M. Hörcher: Improving Software Tests using Z Specifications. To appear in J. P. Bowen and M. G. Hinchey (Eds.): ZUM '95: 9th International Conference of Z Users, LNCS, Springer (1995).
E. Mikk: Compilation of Z Specifications into C for Automatic Test Result Evaluation. To appear in J. P. Bowen and M. G. Hinchey (Eds.): ZUM '95: 9th International Conference of Z Users, LNCS, Springer (1995).
R. Milner: Communication and Concurrency. Prentice-Hall International, Englewood Cliffs NJ (1989).
M. Müllerburg: Systematic Testing: a Means for Validating Reactive Systems. In EuroSTAR'94: Proceedings of the 2nd European Intern. Conf. on Software Testing, Analysis & Review. British Computer Society, (1994).
G. J. Myers: The Art of Software-Testing. John Wiley & Sons, New York (1979).
J. Peleska: Trustworthy Tests for Reactive Systems — Automation of Real-Time Testing. In preparation, JP Software-Consulting (1996).
J. Peleska and M. Siegel: From Testing Theory to Test Driver Implementation. To appear in Proceedings of the Formal Methods Europe Conference FME '96, LNCS, Springer-Verlag, (1996).
J. Peleska: Testautomatisierung für diskrete Steuerungen, Anwendung: Bahnübergangssteuerung — Abschlußbericht Phase 1. Technical Report 06/95, JP Software-Consulting (1995).
J. Peleska: Formal Methods and the Development of Dependable Systems. Technical Report 07/95, JP Software-Consulting (1995).
D. J. Richardson, T. O. O'Malley and C. Tittle Moore: Approaches to Specification-Based Testing. In ACM SIGSoft 89: Third Symposium on Software Testing, Analysis and Verification, December (1989).
D. J. Richardson, S. Leif Aha and T. O. O'Malley: Specification-based Test Oracles for Reactive Systems. In Proceedings of the 14th International Conference on Software Engineering, Melborne, Australia, May (1992).
E. Weyuker, T. Goradia and A. Singh: Automatically Generating Test Data from a Boolean Specification. IEEE Transactions on Software Engineering Vol. 20, N0. 5, (1994).
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1996 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Peleska, J. (1996). Test automation for safety-critical systems: Industrial application and future developments. In: Gaudel, MC., Woodcock, J. (eds) FME'96: Industrial Benefit and Advances in Formal Methods. FME 1996. Lecture Notes in Computer Science, vol 1051. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-60973-3_79
Download citation
DOI: https://doi.org/10.1007/3-540-60973-3_79
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-60973-5
Online ISBN: 978-3-540-49749-3
eBook Packages: Springer Book Archive