An Empirical Assessment of the Properties of Inverted Generational Distance on Multi- and Many-Objective Optimization

  • Leonardo C. T. Bezerra
  • Manuel López-Ibáñez
  • Thomas Stützle
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10173)

Abstract

The inverted generational distance (IGD) is a metric for assessing the quality of approximations to the Pareto front obtained by multi-objective optimization algorithms. The IGD has become the most commonly used metric in the context of many-objective problems, i.e., those with more than three objectives. The averaged Hausdorff distance and \(\textit{IGD}^+\) are variants of the IGD proposed in order to overcome its major drawbacks. In particular, the IGD is not Pareto compliant and its conclusions may strongly change depending on the size of the reference front. It is also well-known that different metrics assign more importance to various desired features of approximation fronts, and thus, they may disagree when ranking them. However, the precise behavior of the IGD variants is not well-understood yet. In particular, \(\textit{IGD}^+\), the only IGD variant that is weakly Pareto-compliant, has received significantly less attention. This paper presents an empirical analysis of the IGD variants. Our experiments evaluate how these metrics are affected by the most important factors that intuitively describe the quality of approximation fronts, namely, spread, distribution and convergence. The results presented here already reveal interesting insights. For example, we conclude that, in order to achieve small IGD or \(\textit{IGD}^+\) values, the approximation front size should match the reference front size.

Keywords

Multi-objective optimization Performance assessment Inverted generational distance 

Notes

Acknowledgments

The research presented in this paper has received funding from the COMEX project (P7/36) within the IAP Programme of BelSPO. T. Stützle acknowledges support from the Belgian F.R.S.-FNRS, of which he is a senior research associate.

References

  1. 1.
    Beume, N., Fonseca, C.M., López-Ibáñez, M., Paquete, L., Vahrenhold, J.: On the complexity of computing the hypervolume indicator. IEEE Trans. Evol. Comput. 13(5), 1075–1082 (2009)CrossRefGoogle Scholar
  2. 2.
    Bezerra, L.C.T.: A component-wise approach to multi-objective evolutionary algorithms: from flexible frameworks to automatic design. Ph.D. thesis, IRIDIA, École polytechnique, Université Libre de Bruxelles, Belgium (2016)Google Scholar
  3. 3.
    Bezerra, L.C.T., López-Ibáñez, M., Stützle, T.: An empirical assessment of the properties of inverted generational distance indicators on multi- and many-objective optimization: supplementary material (2016). http://iridia.ulb.ac.be/supp/IridiaSupp. 2016-006/
  4. 4.
    Coello Coello, C.A., Reyes Sierra, M.: A study of the parallelization of a coevolutionary multi-objective evolutionary algorithm. In: Monroy, R., Arroyo-Figueroa, G., Sucar, L.E., Sossa, H. (eds.) MICAI 2004. LNCS (LNAI), vol. 2972, pp. 688–697. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24694-7_71 CrossRefGoogle Scholar
  5. 5.
    Deb, K., Jain, S.: An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: Solving problems with box constraints. IEEE Trans. Evol. Comput. 18(4), 577–601 (2014)CrossRefGoogle Scholar
  6. 6.
    Dubois-Lacoste, J., López-Ibáñez, M., Stützle, T.: Improving the anytime behavior of two-phase local search. Ann. Math. Artif. Intell. 61(2), 125–154 (2011)MathSciNetCrossRefMATHGoogle Scholar
  7. 7.
    Ishibuchi, H., Akedo, N., Nojima, Y.: Behavior of multiobjective evolutionary algorithms on many-objective knapsack problems. IEEE Trans. Evol. Comput. 19(2), 264–283 (2015)CrossRefGoogle Scholar
  8. 8.
    Ishibuchi, H., Masuda, H., Nojima, Y.: A study on performance evaluation ability of a modified inverted generational distance indicator. In: Silva, S. et al. (ed.) GECCO, pp. 695–702. ACM Press (2015)Google Scholar
  9. 9.
    Ishibuchi, H., Masuda, H., Tanigaki, Y., Nojima, Y.: Modified distance calculation in generational distance and inverted generational distance. In: Gaspar-Cunha, A., Henggeler Antunes, C., Coello, C.C. (eds.) EMO 2015. LNCS, vol. 9019, pp. 110–125. Springer, Heidelberg (2015). doi:10.1007/978-3-319-15892-1_8 Google Scholar
  10. 10.
    Jiang, S., Ong, Y.S., Zhang, J., Feng, L.: Consistencies and contradictions of performance metrics in multiobjective optimization. IEEE Trans. Cybern. 44(12), 2391–2404 (2014)CrossRefGoogle Scholar
  11. 11.
    Schütze, O., Esquivel, X., Lara, A., Coello, C.A.C.: Using the averaged Hausdorff distance as a performance measure in evolutionary multiobjective optimization. IEEE Trans. Evol. Comput. 16(4), 504–522 (2012)CrossRefGoogle Scholar
  12. 12.
    Van Veldhuizen, D.A., Lamont, G.B.: Multiobjective evolutionary algorithms: analyzing the state-of-the-art. Evol. Comput. 8(2), 125–147 (2000)CrossRefGoogle Scholar
  13. 13.
    Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto evolutionary algorithm. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999)CrossRefGoogle Scholar
  14. 14.
    Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE Trans. Evol. Comput. 7(2), 117–132 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Leonardo C. T. Bezerra
    • 1
  • Manuel López-Ibáñez
    • 2
  • Thomas Stützle
    • 3
  1. 1.DCC-CI, Universidade Federal da Paraíba (UFPB)João PessoaBrazil
  2. 2.Alliance Manchester Business SchoolUniversity of ManchesterManchesterUK
  3. 3.IRIDIA, Université Libre de Bruxelles (ULB)BrusselsBelgium

Personalised recommendations