Preference-Based Many-Objective Evolutionary Testing Generates Harder Test Cases for Autonomous Agents

  • Sabrine Kalboussi
  • Slim Bechikh
  • Marouane Kessentini
  • Lamjed Ben Said
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8084)


Despite the high number of existing works in software testing within the SBSE community, there are very few ones that address the problematic of agent testing. The most prominent work in this direction is by Nguyen et al. [13], which formulates this problem as a bi-objective optimization problem to search for hard test cases from a robustness viewpoint. In this paper, we extend this work by: (1) proposing a new seven-objective formulation of this problem and (2) solving it by means of a preference-based many-objective evolutionary method. The obtained results show that our approach generates harder test cases than Nguyen et al. method ones. Moreover, Nguyen et al. method becomes a special case of our method since the user can incorporate his/her preferences within the search process by emphasizing some testing aspects over others.


Agent testing many-objective optimization user’s preferences 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adra, S.F., Griffin, I., Fleming, P.J.: A Comparative Study of Progressive Preference Articulation Techniques for Multiobjective Optimisation. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 908–921. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  2. 2.
    Bechikh, S., Ben Said, L., Ghédira, K.: Searching for Knee Regions of the Pareto Front using Mobile Reference Points. Soft Computing 15(9), 1807–1823 (2011)CrossRefGoogle Scholar
  3. 3.
    Ben Said, L., Bechikh, S., Ghédira, K.: The r-Dominance: A New Dominance Relation for Interactive Evolutionary Multicriteria Decision Making. IEEE Trans. on Evolutionary Computation 14(5), 801–818 (2010)CrossRefGoogle Scholar
  4. 4.
    Coelho, R., Kulesza, U., Staa, A., Lucena, C.: Unit Testing in Multi-agent Systems using Mock Agents and Aspects. In: International Workshop on Software Engineering for Large-Scale Multi-agent Systems, pp. 83–90 (2006)Google Scholar
  5. 5.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II. IEEE Trans. on Evolutionary Computation 6(2), 182–197 (2002)CrossRefGoogle Scholar
  6. 6.
    Harman, M., Ph, U., Jones, B.F.: Search-Based Software Engineering. Information and Software Technology 43, 833–839 (2001)CrossRefGoogle Scholar
  7. 7.
    Harman, M., Mansouri, S.A., Zhang, Y.: Search-Based Software Engineering: Trends, Techniques and Applications. ACM Computing Surveys 45(1), 11 (2012)CrossRefGoogle Scholar
  8. 8.
    Hughes, E.J.: Evolutionary Many-objective Optimization: Many Once or One Many? In: IEEE Congress on Evolutionary Computation, pp. 222–227 (2005)Google Scholar
  9. 9.
    McMinn, P.: Search-Based Software Testing: Past, Present and Future. In: 4th International Workshop on Search-Based Software Testing, pp. 153–163 (2011)Google Scholar
  10. 10.
    McMinn, P.: Search-based software test data generation: A survey. Software Testing, Verification and Reliability 14(2), 105–156 (2004)CrossRefGoogle Scholar
  11. 11.
    McMinn, P., Harman, M., Lakhotia, K., Hassoun, Y., Wegener, J.: Input Domain Reduction through Irrelevant Variable Removal and Its Effect on Local, Global, and Hybrid Search-Based Structural Test Data Generation. IEEE Trans. on Software Engineering 38(2), 453–477 (2012)CrossRefGoogle Scholar
  12. 12.
  13. 13.
    Nguyen, C.D., Miles, S., Perini, A., Tonella, P., Harman, M., Luck, M.: Evolutionary Testing of Autonomous Software Agents. Autonomous Agents and Multi-Agent Systems 25(2), 260–283 (2012)CrossRefGoogle Scholar
  14. 14.
    Nunez, M., Rodriguez, I., Rubio, F.: Specification and Testing of Autonomous Agents in E-Commerce Systems. Software Testing, Verification and Reliability 15(4), 211–233 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sabrine Kalboussi
    • 1
  • Slim Bechikh
    • 1
  • Marouane Kessentini
    • 2
  • Lamjed Ben Said
    • 1
  1. 1.SOIE Lab, ISG-TunisUniversity of TunisTunisia
  2. 2.CSMissouri University of Science and TechnologyMissouriUSA

Personalised recommendations