Preference-Based Many-Objective Evolutionary Testing Generates Harder Test Cases for Autonomous Agents
Despite the high number of existing works in software testing within the SBSE community, there are very few ones that address the problematic of agent testing. The most prominent work in this direction is by Nguyen et al. , which formulates this problem as a bi-objective optimization problem to search for hard test cases from a robustness viewpoint. In this paper, we extend this work by: (1) proposing a new seven-objective formulation of this problem and (2) solving it by means of a preference-based many-objective evolutionary method. The obtained results show that our approach generates harder test cases than Nguyen et al. method ones. Moreover, Nguyen et al. method becomes a special case of our method since the user can incorporate his/her preferences within the search process by emphasizing some testing aspects over others.
KeywordsAgent testing many-objective optimization user’s preferences
Unable to display preview. Download preview PDF.
- 1.Adra, S.F., Griffin, I., Fleming, P.J.: A Comparative Study of Progressive Preference Articulation Techniques for Multiobjective Optimisation. In: Obayashi, S., Deb, K., Poloni, C., Hiroyasu, T., Murata, T. (eds.) EMO 2007. LNCS, vol. 4403, pp. 908–921. Springer, Heidelberg (2007)CrossRefGoogle Scholar
- 4.Coelho, R., Kulesza, U., Staa, A., Lucena, C.: Unit Testing in Multi-agent Systems using Mock Agents and Aspects. In: International Workshop on Software Engineering for Large-Scale Multi-agent Systems, pp. 83–90 (2006)Google Scholar
- 8.Hughes, E.J.: Evolutionary Many-objective Optimization: Many Once or One Many? In: IEEE Congress on Evolutionary Computation, pp. 222–227 (2005)Google Scholar
- 9.McMinn, P.: Search-Based Software Testing: Past, Present and Future. In: 4th International Workshop on Search-Based Software Testing, pp. 153–163 (2011)Google Scholar
- 12.Nguyen, C.D.: Web page, tools, http://selab.fbk.eu/dnguyen/public/cleaner-agent.tgz