Advertisement

eCrash: a Genetic Programming-Based Testing Tool for Object-Oriented Software

  • José Carlos Bregieiro Ribeiro
  • Ana Filipa NogueiraEmail author
  • Francisco Fernández de Vega
  • Mário Alberto Zenha-Rela

Abstract

This paper describes the methodology, architecture and features of the eCrash framework, a Java-based tool which employs Strongly-Typed Genetic Programming to automate the generation of test data for the structural unit testing of Object-Oriented programs. The application of Evolutionary Algorithms to Test Data generation is often referred to as Evolutionary Testing. eCrash implements an Evolutionary Testing strategy developed with three major purposes: improving the level of performance and automation of the Software Testing process; minimising the interference of the tool’s users on the Test Object analysis to a minimum; and mitigating the impact of users decisions in the Test Data generation process.

Keywords

Evolutionary Testing Object-Orientation Search-Based Software Engineering Unit Testing Genetic Programming 

References

  1. Baresi, L., Lanzi, P.L., Miraz, M.: Testful: An evolutionary test approach for java. In: Proceedings of the 2010 Third International Conference on Software Testing, Verification and Validation, ICST ’10, pp. 185–194. IEEE Computer Society, Washington, DC, USA (2010). DOI 10.1109/ICST.2010.54. URL http://dx.doi.org/10.1109/ICST.2010.54
  2. Baresi, L., Miraz, M.: Testful: automatic unit-test generation for java classes. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2, ICSE ’10, pp. 281–284. ACM, New York, NY, USA (2010). DOI 10.1145/1810295.1810353. URL http://doi.acm.org/10.1145/1810295.1810353
  3. Fraser, G., Arcuri, A.: Evolutionary generation of whole test suites. In: Proceedings of the 2011 11th International Conference on Quality Software, QSIC ’11, pp. 31–40. IEEE Computer Society, Washington, DC, USA (2011). DOI 10.1109/QSIC.2011.19. URL http://dx.doi.org/10.1109/QSIC.2011.19
  4. Fraser, G., Arcuri, A.: Evosuite: automatic test suite generation for object-oriented software. In: Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering, ESEC/FSE ’11, pp. 416–419. ACM, New York, NY, USA (2011). DOI 10.1145/2025113.2025179. URL http://doi.acm.org/10.1145/2025113.2025179
  5. Fraser, G., Arcuri, A.: Sound empirical evidence in software testing. In: 34th International Conference on Software Engineering, ICSE 2012, June 2–9, 2012, Zurich, Switzerland, pp. 178–188. IEEE (2012)Google Scholar
  6. Fraser, G., Arcuri, A.: Whole test suite generation. IEEE Trans. Softw. Eng. 39(2), 276–291 (2013). DOI 10.1109/TSE.2012.14. URL http://dx.doi.org/10.1109/TSE.2012.14
  7. Fraser, G., Arcuri, A., McMinn, P.: Test suite generation with memetic algorithms. In: Proceeding of the fifteenth annual conference on Genetic and evolutionary computation conference, GECCO ’13, pp. 1437–1444. ACM, New York, NY, USA (2013). DOI 10.1145/2463372.2463548. URL http://doi.acm.org/10.1145/2463372.2463548
  8. Harman, M.: Automated test data generation using search based software engineering. In: AST ’07: Proceedings of the Second International Workshop on Automation of Software Test, p. 2. IEEE Computer Society, Washington, DC, USA (2007). DOI http://dx.doi.org/10.1109/AST.2007.4
  9. Harman, M., Mansouri, S.A., Zhang, Y.: Search based software engineering: A comprehensive analysis and review of trends techniques and applications. Tech. Rep. TR-09-03, Department of Computer Science, King’s College London (2009). URL http://www.dcs.kcl.ac.uk/technical-reports/papers/TR-09-03.pdf
  10. Kinneer, A., Dwyer, M., Rothermel, G.: Sofya: A flexible framework for development of dynamic program analysis for java software. Tech. Rep. TR-UNL-CSE-2006-0006, University of Nebraska (2006). URL http://sofya.unl.edu/
  11. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems). The MIT Press (1992). URL http://www.amazon.ca/exec/obidos/redirect?tag=citeulike04-20&path=ASIN/0262111705
  12. Kropp, N.P., Jr., P.J.K., Siewiorek, D.P.: Automated robustness testing of off-the-shelf software components. In: Symposium on Fault-Tolerant Computing, pp. 230–239 (1998). URL citeseer.ist.psu.edu/kropp98automated.html
  13. Luke, S.: ECJ 21: A Java evolutionary computation library. http://cs.gmu.edu/~eclab/projects/ecj/ (2013)
  14. McMinn, P.: Search-based software test data generation: A survey. Software Testing, Verification and Reliability 14(2), 105–156 (2004). URL citeseer.ist.psu.edu/mcminn04searchbased.html
  15. McMinn, P., Holcombe, M.: The state problem for evolutionary testing (2003). URL citeseer.ist.psu.edu/mcminn03state.html
  16. Miraz, M., Lanzi, P.L., Baresi, L.: Testful: using a hybrid evolutionary algorithm for testing stateful systems. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, GECCO ’09, pp. 1947–1948. ACM, New York, NY, USA (2009). DOI 10.1145/1569901.1570252. URL http://doi.acm.org/10.1145/1569901.1570252
  17. Montana, D.J.: Strongly typed genetic programming. Evolutionary Computation 3(2), 199–230 (1995)CrossRefGoogle Scholar
  18. Naik, S., Tripathy, P.: Software Testing and Quality Assurance: Theory and Practice. Wiley (2008)Google Scholar
  19. Nogueira, A.F., Ribeiro, J.C.B., de Vega, F.F., Zenha-Rela, M.A.: ecrash: An empirical study on the apache ant project. In: Proceedings of the 5th International Symposium on Search Based Software Engineering (SSBSE ’13), vol. 8084. Springer, St. Petersburg, Russia (2013)Google Scholar
  20. Pacheco, C., Ernst, M.D.: Randoop: feedback-directed random testing for java. In: OOPSLA ’07: Companion to the 22nd ACM SIGPLAN conference on Object-oriented programming systems and applications companion, pp. 815–816. ACM, New York, NY, USA (2007). DOI http://doi.acm.org/10.1145/1297846.1297902
  21. Pavlov, Y., Fraser, G.: Semi-automatic search-based test generation. In: Proceedings of the 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, ICST ’12, pp. 777–784. IEEE Computer Society, Washington, DC, USA (2012). DOI 10.1109/ICST.2012.176. URL http://dx.doi.org/10.1109/ICST.2012.176
  22. Ribeiro, J.C.B.: Search-based test case generation for object-oriented java software using strongly-typed genetic programming. In: GECCO ’08: Proceedings of the 2008 GECCO Conference Companion on Genetic and Evolutionary Computation, pp. 1819–1822. ACM, New York, NY, USA (2008). DOI http://doi.acm.org/10.1145/1388969.1388979
  23. Ribeiro, J.C.B.: Contributions for improving genetic programming-based approaches to the evolutionary testing of object-oriented software. Ph.D. thesis, Universidad de Extremadura, Espaã (2010)Google Scholar
  24. Ribeiro, J.C.B., de Vega, F.F., Zenha-Rela, M.: Using dynamic analysis of java bytecode for evolutionary object-oriented unit testing. In: SBRC WTF 2007: Proceedings of the 8th Workshop on Testing and Fault Tolerance at the 25th Brazilian Symposium on Computer Networks and Distributed Systems, pp. 143–156. Brazilian Computer Society (SBC) (2007)Google Scholar
  25. Ribeiro, J.C.B., Zenha-Rela, M., de Vega, F.F.: ecrash: a framework for performing evolutionary testing on third-party java components. In: CEDI JAEM’07: Proceedings of the I Jornadas sobre Algoritmos Evolutivos y Metaheuristicas at the II Congreso Español de Informática, pp. 137–144 (2007)Google Scholar
  26. Ribeiro, J.C.B., Zenha-Rela, M., de Vega, F.F.: A strategy for evaluating feasible and unfeasible test cases for the evolutionary testing of object-oriented software. In: AST ’08: Proceedings of the 3rd International Workshop on Automation of Software Test, pp. 85–92. ACM, New York, NY, USA (2008). DOI http://doi.acm.org/10.1145/1370042.1370061
  27. Ribeiro, J.C.B., Zenha-Rela, M.A., Fernández de Vega, F.: Test case evaluation and input domain reduction strategies for the evolutionary testing of object-oriented software. Inf. Softw. Technol. 51(11), 1534–1548 (2009). DOI http://dx.doi.org/10.1016/j.infsof.2009.06.009
  28. Ribeiro, J.C.B., Zenha-Rela, M.A., de Vega, F.F.: Strongly-typed genetic programming and purity analysis: input domain reduction for evolutionary testing problems. In: GECCO ’08: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, pp. 1783–1784. ACM, New York, NY, USA (2008). DOI http://doi.acm.org/10.1145/1389095.1389439
  29. Ribeiro, J.C.B., Zenha-Rela, M.A., de Vega, F.F.: Adaptive evolutionary testing: an adaptive approach to search-based test case generation for object-oriented software. In: NICSO 2010 - International Workshop on Nature Inspired Cooperative Strategies for Optimization, Studies in Computational Intelligence. Springer (2010)Google Scholar
  30. Ribeiro, J.C.B., Zenha-Rela, M.A., de Vega, F.F.: Enabling object reuse on genetic programming-based approaches to object-oriented evolutionary testing. In: EuroGP 2010 - 13th European Conference on Genetic Programming (to appear), Lecture Notes in Computer Science. Springer (2010)Google Scholar
  31. Tassey, G.: The economic impacts of inadequate infrastructure for software testing. Tech. rep., National Institute of Standards and Technology (2002)Google Scholar
  32. Tonella, P.: Evolutionary testing of classes. In: ISSTA ’04: Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, pp. 119–128. ACM Press, New York, NY, USA (2004). DOI http://doi.acm.org/10.1145/1007512.1007528
  33. Vincenzi, A.M.R., Delamaro, M.E., Maldonado, J.C., Wong, W.E.: Establishing structural testing criteria for java bytecode. Softw. Pract. Exper. 36(14), 1513–1541 (2006). DOI http://dx.doi.org/10.1002/spe.v36:14
  34. Wappler, S., Wegener, J.: Evolutionary unit testing of object-oriented software using a hybrid evolutionary algorithm. In: CEC’06: Proceedings of the 2006 IEEE Congress on Evolutionary Computation, pp. 851–858. IEEE (2006)Google Scholar
  35. Wappler, S., Wegener, J.: Evolutionary unit testing of object-oriented software using strongly-typed genetic programming. In: GECCO ’06: Proceedings of the 8th annual conference on Genetic and evolutionary computation, pp. 1925–1932. ACM Press, New York, NY, USA (2006). DOI http://doi.acm.org/10.1145/1143997.1144317
  36. Zakhour, S., Hommel, S., Royal, J., Rabinovitch, I., Risser, T., Hoeber, M.: The Java Tutorial: A Short Course on the Basics, 4th Edition (Java Series), 4th edn. Prentice Hall PTR (2006). URL http://www.worldcat.org/isbn/0321334205

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • José Carlos Bregieiro Ribeiro
    • 1
  • Ana Filipa Nogueira
    • 2
    Email author
  • Francisco Fernández de Vega
    • 3
  • Mário Alberto Zenha-Rela
    • 2
  1. 1.Polytechnic Institute of Leiria, Morro do Lena, Alto do VieiroLeiriaPortugal
  2. 2.University of Coimbra, CISUC, DEICoimbraPortugal
  3. 3.University of ExtremaduraMéridaSpain

Personalised recommendations