On Quality Indicators for Black-Box Level Set Approximation

  • Michael T. M. Emmerich
  • André H. Deutz
  • Johannes W. Kruisselbrink
Part of the Studies in Computational Intelligence book series (SCI, volume 447)


This chapter reviews indicators that can be used to compute the quality of approximations to level sets for black-box functions. Such problems occur, for instance, when finding sets of solutions to optimization problems or in solving nonlinear equation systems. After defining and motivating level set problems from a decision theoretic perspective, we discuss quality indicators that could be used to measure how well a set of points approximates a level set. We review simple indicators based on distance, indicators from biodiversity, and propose novel indicators based on the concept of Hausdorff distance. We study properties of these indicators with respect to continuity, spread, and monotonicity and also discuss computational complexity. Moreover, we study the use of these indicators in a simple indicatorbased evolutionary algorithm for level set approximation.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Allgower, E.L., Georg, K.: Introduction to Numerical Continuation Methods. SIAM (2003)Google Scholar
  2. 2.
    Atkinson, A.B.: On the Measurement of Inequality. Journal of Economy 2, 244–263 (1970)Google Scholar
  3. 3.
    Branke, J.: Reducing the Sampling Variance when Searching for Robust Solutions. In: GECCO, pp. 235–242. Morgan Kaufmann (2001)Google Scholar
  4. 4.
    de Berg, M., van Kreveld, M., Overmars, M., Schwarzkopf, O.: Computational Geometry: Algorithms and Applications. Springer (2000)Google Scholar
  5. 5.
    Harada, K., Sakuma, J., Kobayashi, S., Ono, I.: Uniform Sampling of Local Pareto-Optimal Solution Curves by Pareto Path Following and its Applications in Multi-Objective GA. In: Lipson, H. (ed.) GECCO, pp. 813–820. ACM (2007)Google Scholar
  6. 6.
    Hausdorff, F.: Grundzüge der Mengenlehre, 1st edn., Berlin (1914)Google Scholar
  7. 7.
    Gielis, J.: A Generic Transformation that Unifies a Wide Range of Natural and Abstract Shapes. American Journal of Botany 90(3), 333–338 (2003)CrossRefGoogle Scholar
  8. 8.
    Hillermeier, C.: Generalized Homotopy Approach to Multiobjective Optimization. JOTA 110(3), 557–583 (2001)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Pompeiu, D.: Sur la Continuité des Fonctions de Variables Complexes. Annales de la Faculté des Sciences de Toulouse Sér. 2 7(3), 265–315 (1905)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Rényi, A.: Wahrscheinlichkeitsrechnung. VEB Verlag (1971)Google Scholar
  11. 11.
    Schütze, O., Dell’Aere, A., Dellnitz, M.: On Continuation Methods for the Numerical Treatment of Multi-Objective Optimization Problems. In: Branke, et al. (eds.) Practical Approaches to Multi-Objective Optimization. Dagstuhl Seminar Proc. 04461. IBFI, Germany (2005)Google Scholar
  12. 12.
    Schütze, O., Coello Coello, C.A., Mostaghim, S., Talbi, E.-G., Dellnitz, M.: Hybridizing Evolutionary Strategies with Continuation Methods for Solving Multi-Objective Problems. Engineering Optimization 40(5), 383–402 (2008)MathSciNetCrossRefGoogle Scholar
  13. 13.
    Schütze, O., Esquivel, X., Lara, A., Coello Coello, C.A.: Some Comments on GD and IGD and Relations to the Hausdorff Distance. In: Pelikan, M., Branke, J. (eds.) GECCO (Companion), pp. 1971–1974. ACM (2010)Google Scholar
  14. 14.
    Schütze, O., Esquivel, X., Lara, A., Coello Coello, C.A.: Using the Averaged Hausdorff Distance as a Performance Measure in Evolutionary Multi-Objective Optimization. IEEE, Transact. EC (2010) (to appear)Google Scholar
  15. 15.
    Shir, O.M., Preuss, M., Naujoks, B., Emmerich, M.: Enhancing Decision Space Diversity in Evolutionary Multiobjective Algorithms. In: Ehrgott, M., Fonseca, C.M., Gandibleux, X., Hao, J.-K., Sevaux, M. (eds.) EMO 2009. LNCS, vol. 5467, pp. 95–109. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  16. 16.
    Solow, A., Polasky, S.: Measuring Biological Diversity. Environmental and Ecological Statistics 1, 95–103 (1994)CrossRefGoogle Scholar
  17. 17.
    Ulrich, T., Bader, J., Thiele, L.: Defining and Optimizing Indicator-Based Diversity Measures in Multiobjective Search. In: Schaefer, R., Cotta, C., Kołodziej, J., Rudolph, G. (eds.) PPSN XI. LNCS, vol. 6238, pp. 707–717. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  18. 18.
    Ulrich, T., Thiele, L.: Maximizing Population Diversity in Single-Objective Optimization. In: Krasnogor, N. (ed.) GECCO, pp. 641–648. ACM (2011)Google Scholar
  19. 19.
    Weitzman, M.L.: On Diversity. The Quarterly Journal of Economics 107(2), 363–405 (1992)zbMATHCrossRefGoogle Scholar
  20. 20.
    Yuval, G.: Finding Nearest Neighbors. In: Proc. of Inf. Process. Lett., pp. 63–65 (1976)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2013

Authors and Affiliations

  • Michael T. M. Emmerich
    • 1
  • André H. Deutz
    • 1
  • Johannes W. Kruisselbrink
    • 1
  1. 1.LIACSLeidenThe Netherlands

Personalised recommendations