Combining Search-Based and Adaptive Random Testing Strategies for Environment Model-Based Testing of Real-Time Embedded Systems

  • Muhammad Zohaib Iqbal
  • Andrea Arcuri
  • Lionel Briand
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7515)

Abstract

Effective system testing of real-time embedded systems (RTES) requires a fully automated approach. One such black-box system testing approach is to use environment models to automatically generate test cases and test oracles along with an environment simulator to enable early testing of RTES. In this paper, we propose a hybrid strategy, which combines (1+1) Evolutionary Algorithm (EA) and Adaptive Random Testing (ART), to improve the overall performance of system testing that is obtained when using each single strategy in isolation. An empirical study is carried out on a number of artificial problems and one industrial case study. The novel strategy shows significant overall improvement in terms of fault detection compared to individual performances of both (1+1) EA and ART.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Adjir, N., De Saqui-Sannes, P., Rahmouni, K.M.: Testing Real-Time Systems Using TINA. In: Núñez, M., Baker, P., Merayo, M.G. (eds.) TESTCOM 2009. LNCS, vol. 5826, pp. 1–15. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Ali, S., Iqbal, M.Z., Arcuri, A., Briand, L.: A Search-based OCL Constraint Solver for Model-based Test Data Generation. In: 11th International Conference on Quality Software, pp. 41–50. IEEE (2011)Google Scholar
  3. 3.
    Andrews, J., Briand, L., Labiche, Y., Namin, A.: Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Transactions on Software Engineering 32, 608–624 (2006)CrossRefGoogle Scholar
  4. 4.
    Andrews, J.H., Menzies, T., Li, F.C.H.: Genetic algorithms for randomized unit testing. IEEE Transactions on Software Engineering 37, 80–94 (2011)CrossRefGoogle Scholar
  5. 5.
    Arcuri, A., Briand, L.: A Practical Guide for Using Statistical Tests to Assess Randomized Algorithms in Software Engineering. In: 33rd International Conference on Software Engineering (ICSE), pp. 1–10 (2011)Google Scholar
  6. 6.
    Arcuri, A., Iqbal, M.Z., Briand, L.: Black-Box System Testing of Real-Time Embedded Systems Using Random and Search-Based Testing. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 95–110. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Arcuri, A., Iqbal, M.Z., Briand, L.: Random Testing: Theoretical Results and Practical Implications. IEEE Transactions on Software Engineering 38, 258–277 (2012)CrossRefGoogle Scholar
  8. 8.
    Auguston, M., Michael, J.B., Shing, M.: Environment behavior models for automation of testing and assessment of system safety. Information and Software Technology 48, 971–980 (2006)CrossRefGoogle Scholar
  9. 9.
    Broekman, B.M., Notenboom, E.: Testing Embedded Software. Addison-Wesley Co., Inc. (2003)Google Scholar
  10. 10.
    Heisel, M., Hatebur, D., Santen, T., Seifert, D.: Testing Against Requirements Using UML Environment Models. In: Fachgruppentreffen Requirements Engineering und Test, Analyse & Verifikation, pp. 28–31. GI (2008)Google Scholar
  11. 11.
    Iqbal, M.Z., Arcuri, A., Briand, L.: Automated System Testing of Real-Time Embedded Systems Based on Environment Models. Simula Research Laboratory, Technical Report (2011-19) (2011)Google Scholar
  12. 12.
    Iqbal, M.Z., Arcuri, A., Briand, L.: Code Generation from UML/MARTE/OCL Environment Models to Support Automated System Testing of Real-Time Embedded Software. Simula Research Laboratory, Technical Report (2011-04) (2011)Google Scholar
  13. 13.
    Iqbal, M.Z., Arcuri, A., Briand, L.: Empirical Investigation of Search Algorithms for Environment Model-Based Testing of Real-Time Embedded Software. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), pp. 199–209. ACM (2012)Google Scholar
  14. 14.
    Iqbal, M.Z., Arcuri, A., Briand, L.: Environment Modeling with UML/MARTE to Support Black-Box System Testing for Real-Time Embedded Systems: Methodology and Industrial Case Studies. In: Petriu, D.C., Rouquette, N., Haugen, Ø. (eds.) MODELS 2010, Part I. LNCS, vol. 6394, pp. 286–300. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  15. 15.
    Grabowski, J., Nielsen, B. (eds.): FATES 2004. LNCS, vol. 3395. Springer, Heidelberg (2005)MATHGoogle Scholar
  16. 16.
    Lefticaru, R., Ipate, F.: Functional search-based testing from state machines. In: Proceedings of the International Conference on Software Testing, Verification, and Validation, pp. 525–528. IEEE Computer Society (2008)Google Scholar
  17. 17.
    Mühlenbein, H.: How genetic algorithms really work: I. mutation and hillclimbing. In: Parallel Problem Solving from Nature, vol. 2, pp. 15–25 (1992)Google Scholar
  18. 18.
    OMG: Modeling and Analysis of Real-time and Embedded systems (MARTE), Version 1.0 (2009), http://www.omg.org/spec/MARTE/1.0/
  19. 19.
    OMG: Unified Modeling Language Superstructure, Version 2.3 (2010), http://www.omg.org/spec/UML/2.3/
  20. 20.
    Schneckenburger, C., Schweiggert, F.: Investigating the dimensionality problem of Adaptive Random Testing incorporating a local search technique. In: IEEE International Conference on Software Testing Verification and Validation Workshop (ICSTW 2008), pp. 241–250 (2008)Google Scholar
  21. 21.
    Tappenden, A.F., Miller, J.: A novel evolutionary approach for adaptive random testing. IEEE Transactions on Reliability 58, 619–633 (2009)CrossRefGoogle Scholar
  22. 22.
    Zheng, M., Alagar, V., Ormandjieva, O.: Automated generation of test suites from formal specifications of real-time reactive systems. The Journal of Systems & Software 81, 286–304 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Muhammad Zohaib Iqbal
    • 1
    • 2
  • Andrea Arcuri
    • 1
  • Lionel Briand
    • 1
    • 3
  1. 1.Simula Research LaboratoryCertus Center for V & VLysakerNorway
  2. 2.Department of InformaticsUniversity of OsloNorway
  3. 3.SnT CenterUniversity of LuxembourgLuxembourg

Personalised recommendations