Advertisement

Automatic Test Data Generation by Multi-objective Optimisation

  • Norbert Oster
  • Francesca Saglietti
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4166)

Abstract

This paper presents a technique for automated test data generation applicable to both procedural and object-oriented programs. During the generation, the test cases are optimised such as to maximise structural code coverage by minimising at the same time the number of test cases required. To cope with these two inherently conflicting goals, hybrid self-adaptive and multi-objective evolutionary algorithms are applied. Our approach is based on a preliminary activity that provides support for the automatic instrumentation of source code in order to record the relevant data flow information at runtime. By exclusively utilising the insight gained hereby, test data sets are successively enhanced towards the goals mentioned above. Finally, the efficiency of the test set generated is evaluated in terms of its fault detection capability by means of mutation testing. In addition, the actual coverage percentage achieved is determined by taking into account the results of a static data flow analysis of the system under test. Thanks to the dramatic decrease of effort required for generating and verifying test cases, the technique presented here allows to substantially improve the V&V-phase of complex, safety-relevant software. Preliminary experimental results gained so far are reported in the paper.

Keywords

testing data flow evolutionary algorithms automated test data generation object-oriented software mutation testing 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Liggesmeyer, P.: Software-Qualität – Testen, Analysieren und Verifizieren von Software. Spektrum - Akademischer Verlag, Heidelberg (2002)zbMATHGoogle Scholar
  2. 2.
    O’Sullivan, M., Vössner, S., Wegener, J.: Testing temporal correctness of real-time systems - a new approach using genetic algorithms and cluster analysis. In: EuroSTAR 1998 Software Testing Analysis & Review. EuroSTAR, Munich Park Hilton, vol. 6, pp. 397–418 (1998)Google Scholar
  3. 3.
    Hutchins, M., Foster, H., Goradia, T., Ostrand, T.: Experiments on the effectiveness of dataflow- and controlflow-based test adequacy criteria. In: Proceedings of the 16th International Conference on Software Engineering. ICSE, vol. 16, pp. 191–200. IEEE, Los Alamitos (1994)CrossRefGoogle Scholar
  4. 4.
    Michael, C.C., McGraw, G.: Automated software test data generation for complex programs. In: Automated Software Engineering. Thirteenth IEEE Conference on Automated Software Engineering, pp. 136–146. IEEE, Los Alamitos (1998)Google Scholar
  5. 5.
    Baresel, A.: Automatisierung von Strukturtests mit evolutionären Algorithmen. Diplomarbeit, Lehr- und Forschungsgebiet Softwaretechnik, Humboldt-Universität Berlin, Berlin (2000)Google Scholar
  6. 6.
    Wegener, J., Buhr, K., Pohlheim, H.: Automatic test data generation for structural testing of embedded software systems by evolutionary testing. In: Langdon, W.B., Cantú-Paz, E. (eds.) GECCO 2002. Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1233–1240. Morgan Kaufmann, New York (2002)Google Scholar
  7. 7.
    Baresel, A., Sthamer, H., Wegener, J.: Applying evolutionary testing to search for critical defects. In: Deb, K., Poli, R., et al. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 1427–1428. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  8. 8.
    Jee, E., Yoo, J., Cha, S.: Control and data flow testing on function block diagramms. In: Winther, R., Gran, B.A., Dahll, G. (eds.) SAFECOMP 2005. LNCS, vol. 3688, pp. 67–80. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  9. 9.
    Rapps, S., Weyuker, E.J.: Selecting software test data using data flow information. IEEE Transactions on Software Engineering SE-11(4), 367–375 (1985)CrossRefGoogle Scholar
  10. 10.
    Ntafos, S.C.: On testing with required elements. In: Proceedings of COMPSAC 1981, Chicago, pp. 132–139. IEEE Computer Society, Los Alamitos (1981)Google Scholar
  11. 11.
    Laski, J.W., Korel, B.: A data flow oriented program testing strategy. IEEE Transactions on Software Engineering 9(3), 347–354 (1983)CrossRefGoogle Scholar
  12. 12.
    Ostrand, T.J., Weyuker, E.J.: Data flow-based test adequacy analysis for languages with pointers. In: Proceedings of the Symposium on Testing, Analysis, and Verification, pp. 74–86. ACM Press, New York (1991)CrossRefGoogle Scholar
  13. 13.
    Chatterjee, R., Ryder, B.G., Landi, W.A.: Relevant context inference. In: Proceedings of the 26th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 133–146. ACM Press, New York (1999)CrossRefGoogle Scholar
  14. 14.
    Chatterjee, R., Ryder, B.G.: Data-flow-based testing of object-oriented libraries. Technical Report DCS-TR-433, Department of Computer Science, Rutgers University, New Jersey (2001)Google Scholar
  15. 15.
    Oster, N., Dorn, R.D.: A data flow approach to testing object-oriented java-programs. In: Spitzer, C., Schmocker, U., Dang, V.N. (eds.) Probabilistic Safety Assessment and Management (PSAM7/ESREL 2004), vol. 2, pp. 1114–1119. Springer, London (2004)Google Scholar
  16. 16.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  17. 17.
    Wilke, P., Gröbner, M., Oster, N.: A hybrid genetic algorithm for school timetabling. In: McKay, B., Slaney, J.K. (eds.) Canadian AI 2002. LNCS (LNAI), vol. 2557, pp. 455–464. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  18. 18.
    Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evolutionary Computation 2(3), 221–248 (1994)CrossRefGoogle Scholar
  19. 19.
    Zitzler, E., Thiele, L.: Multiobjective optimization using evolutionary algorithms – a comparative case study. Technical report, Swiss Federal Institute of Technology Zurich, Computer Engineering and Communication Networks Laboratory (TIK), Gloriastrasse 35, CH-8092 Zurich, Switzerland (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Norbert Oster
    • 1
  • Francesca Saglietti
    • 1
  1. 1.Department of Software EngineeringUniversity of Erlangen-NurembergGermany

Personalised recommendations