Skip to main content
SpringerLink
Log in

A Search-Based Automated Test-Data Generation Framework for Safety-Critical Systems

  • Chapter
Systems Engineering for Business Process Change: New Directions

Abstract

This paper presents the results of a three year research program to develop an automated test-data generation framework to support the testing of safety-critical software systems. The generality of the framework comes from the exploitation of domain independent search techniques, allowing new test criteria to be addressed by constructing functions that quantify the suitability of test-data against the test-criteria. The paper presents four applications of the framework — specification falsification testing, structural testing, exception condition testing and worst-case execution time testing. The results of three industrial scale case-studies are also presented to show that the framework offers useful support in the development safety-critical software systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D.J. Andrews (ed.). Information technology — Programming languages, their environments and system software interfaces — Vienna Development Method — Specification Language — Part 1: Base Language. ISO/IEC 13817-1, December 1996.

    Google Scholar 

  2. ANSI/MIL-STD 1815A. Reference manual for the Ada programming language, 1983.

    Google Scholar 

  3. J. Barnes. High Integrity Ada: The SPARK Approach. Addison-Wesley, 1997.

    Google Scholar 

  4. B. Beizer, Software Testing Techniques. Thomson Computer Press, 2nd edition, 1990.

    Google Scholar 

  5. W. Burkhardt. Generating programs from syntax. Computing, 2(1):83–94, 1967.

    Article  Google Scholar 

  6. A. Burns and A. Wellings. Real-Time Systems and Their Programming Languages. International Computer Science Series. Addison Wesley, 1990.

    Google Scholar 

  7. R.A. Caruana and J.D. Schaffer. Representation and hidden bias: Gray vs. binary coding for genetic algorithms. In Proceedings of the Fifth International Conference on Machine Learning. Morgan Kaufmann, 1988.

    Google Scholar 

  8. T. Chow. Testing software design modeled by finite-state machines. IEEE Transactions on Software Engineering, 4:178–187, May 1978.

    Article  MATH  Google Scholar 

  9. L. Clarke. A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering, SE-2(3):215–222, September 1976.

    Article  Google Scholar 

  10. L.D. Davis, editor. Handbook of Genetic Algorithms. Van Nostrand Reihold, 1991.

    Google Scholar 

  11. R. Demillo and A. Offutt. Constraint-based automatic test data generation. IEEE Transactions on Software Engineering, 17(9):900–910, 1991.

    Article  Google Scholar 

  12. R. Ferguson and B. Korel. The chaining approach for software test data generation. ACM Transactions on Software Engineering and Methodology, 5(1):63–86, 1996.

    Article  Google Scholar 

  13. J.H. Holland. Adaptation in Natural and Artificial Systems. University of Michigan Press, 1975.

    Google Scholar 

  14. WE. Howden. Weak mutation testing and completeness of test sets. IEEE Transactions on Software Engineering, SE-8(4):371–379, July 1982.

    Article  Google Scholar 

  15. A.C.T. Inc. The gnat ada-95 compiler, 1997. http://www.gnat.com/.

  16. ISO/IEC 8652:1995. Ada 95: Language Reference Manual, 1995.

    Google Scholar 

  17. C.Z. Janikow and Z. Michalewiz. An experimental comparison of binary and floating point representations in genetic algorithms. In R.K. Belew and L.B. Booker, (eds), Proceedings of the Fourth International Conference on Genetic Algorithms. Morgan Kaufmann, 1991.

    Google Scholar 

  18. B. Jones, H. Sthamer and D. Eyres. The automatic generation of software test data sets using adaptive search techniques. In Proceedings of 3rd International Conference on Software Quality Management, volume 2, pp. 435–444, 1995.

    Google Scholar 

  19. B. Jones, H. Sthamer and D. Eyres. Generating test-data for Ada procedures using genetic algorithms. In Genetic Algorithms in Engineering Systems: Innovations and Applications, pages 65–70. IEEE, September 1995.

    Google Scholar 

  20. B. Jones, H. Sthamer and D. Eyres. Automatic structural testing using genetic algorithms. Software Engineering Journal 11(5):299–306, 1996.

    Article  Google Scholar 

  21. S. King, J. Hammond, R. Chapman and A. Pryor. The value of verification: Positive experience of industrial proof. In Formal Methods 1999 Technical Symposium, 1999.

    Google Scholar 

  22. S. Kirkpatrick, J.C.D. Gelatt and M.P. Vecchi. Optimization by simulated annealing. Science, 220(4598):671–680, May 1983.

    Article  MathSciNet  MATH  Google Scholar 

  23. B. Korel. Automated software test data generation. IEEE Transactions on Software Engineering, 16(8):870–879, 1990.

    Article  Google Scholar 

  24. B. Korel. Automated test data generation for programs with procedures. In International Symposium on Software Testing and Analysis, pp. 209–215. ACM/SIGSOFT, 1996.

    Google Scholar 

  25. N.G. Leveson. Safeware: System Safety and Computers. Addison Wesley, 1995.

    Google Scholar 

  26. N.G. Leveson and P.R. Harvey. Analyzing software safety. IEEE Transactions on Software Engineering, SE-9(5):569–579, 1983.

    Article  Google Scholar 

  27. J.L. Lions. Ariane 5: Flight 501 failure report. Technical report, ESA/CNES, July 1996.

    Google Scholar 

  28. Z. Michalewicz. Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, 1996.

    Google Scholar 

  29. W. Miller and D. Spooner. Automatic generation of floating-point test data. IEEE Transactions on Software Engineering, SE-2(3): 223–226, September 1976.

    Article  MathSciNet  Google Scholar 

  30. A.J. Offutt and J. Pan. The dynamic domain reduction procedure for test data generation. — http://www.isse.gmu.edu/faculty/ofut/rsrch/atdg.html, 1996.

  31. M. Ould. Testing — a challenge to method and tool developers. Software Engineering Journal, 6(2): 59–64, March 1991.

    Article  Google Scholar 

  32. R. Pargas, MJ. Harrold and R. Peck. Test-data generation using genetic algorithms. Journal of Software Testing, Verification and Reliability, 9(4): 263–282, 1999.

    Article  Google Scholar 

  33. Praxis Critical Systems. Spark-Ada Documentation 2.0, 1995.

    Google Scholar 

  34. C.R. Reeves. Modern Heuristic Search Methods, Chapter 1 — Modern Heuristic Techniques, pages 1–25. Wiley, 1996.

    Google Scholar 

  35. P.C.S. Rod Chapman. Private communication, 1997.

    Google Scholar 

  36. N. Tracey, J. Clark and K. Mander. Automated program flaw finding using simulated annealing. In International Symposium on Software Testing and Analysis, pages 73–81. ACM/SIGSOFT, 1998.

    Google Scholar 

  37. N. Tracey, J. Clark, and K. Mander. The way forward for unifying dynamic test case generation: The optimisation-based approach. In International Workshop on Dependable Computing and Its Applications, pp. 169–180. IFIP, January 1998.

    Google Scholar 

  38. N. Tracey, J. Clark, K. Mander and J. McDermid. An automated framework for structural test-data generation. In Proceedings of the International Conference on Automated Software Engineering. IEEE, October 1998.

    Google Scholar 

  39. N. Tracey, J. Clark, J. McDermid and K. Mander. Integrating safety analysis with automatic test-data generation for software safety verification. In Proceedings of the 17th International Conference on System Safety, pp. 128–137. System Safety Society, August 1999.

    Google Scholar 

  40. N. Tracey, A. Stephenson, J. Clark, and J. McDermid. A sefet change oriented process for safetycritical systems. In: In the Proceedings of Software Change and Evolution Workshop. International Conference on Software Engineering. IEEE, May 1999.

    Google Scholar 

  41. N.J. Tracey, J. Clark, K. Mander and J. McDermid. Automated test-data generation for exception conditions. Software Practice and Experience, 30(1):61–79, January 2000.

    Article  Google Scholar 

  42. A.L. Watkins. The automatic generation of test data using genetic algorithms. Proceedings of the 4th Software Quality Conference, 2:3000–3309, 1995.

    Google Scholar 

  43. J. Wegener, H.H. Sthamer, B.F. Jones and D.E. Eyres. Testing real-time systems using genetic algorithms. Software Quality Journal, 6(2):127–135, 1997.

    Article  Google Scholar 

  44. S. Xanthakis, C. Ellis, C. SkoURLas, A.L. Gall, S. Katsikas and K. Karapoulios. Application des algorithmes genetiques au test des logiciels. In: Proceedings of 5th International Conference on Software Engineering, pp. 625–638, 1992.

    Google Scholar 

  45. X. Yang. The automatic generation of software test data from Z specifications. Technical report, Department of Computer Studies, University of Glamorgan, 1995.

    Google Scholar 

Download references

Authors
  1. Nigel Tracey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Clark
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John McDermid
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Keith Mander
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Electronics and Computer Science, University of Southampton, Southampton, SO17 1BJ, UK

    Peter Henderson MSc, PhD, MBCS

Rights and permissions

Reprints and permissions

Copyright information

© 2002 Springer-Verlag London

About this chapter

Cite this chapter

Tracey, N., Clark, J., McDermid, J., Mander, K. (2002). A Search-Based Automated Test-Data Generation Framework for Safety-Critical Systems. In: Henderson, P. (eds) Systems Engineering for Business Process Change: New Directions. Springer, London. https://doi.org/10.1007/978-1-4471-0135-2_12

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0135-2_12

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-1084-2

  • Online ISBN: 978-1-4471-0135-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation