A new stopping criterion for multi-objective evolutionary algorithms: application in the calibration of a hydrologic model

  • Juan Carlos Ticona GutierrezEmail author
  • Daniela Santini Adamatti
  • Juan Martin Bravo
Review Paper


Multi-objective genetic algorithms have been successfully applied in a wide variety of problems. Although widely used, there are few theoretical guidelines for determining when to stop the search. Many users commonly use rules like stopping when there is no significant improvement during the last generations or when a certain number of generations are reached. In this paper, we propose a new stopping criterion approach and evaluate its performance with three widely used evolutionary algorithms in the calibration of a hydrologic model. The stopping criterion is based on the minimum number of generations required to achieve a determined number of non-dominated solutions in Pareto Front. The new stopping criterion was tested in the lumped hydrologic model IPH-II calibration, using the genetic algorithms NSGA-II, NSGA-III, and SPEA-II and two objective functions. The generational distance, spacing, and maximum spread metrics were used to assess the performance of the proposed stopping criterion in comparison to the standard criterion. Results show no significant loss in goodness of fit associated with the proposed stopping criterion, both in calibration and validation periods. Performance metrics have shown similar values when the standard and the proposed stopping criteria were compared. However, the average computational time to complete the optimization process was reduced up to 38.2% when the proposed stopping criterion was used. Thus, it can be concluded that the new stopping criterion reduces the iteration workload without compromising the accuracy of solution sets.


Multi-objective evolutionary algorithm Lumped hydrologic model Stopping criterion NSGA-II NSGA-III SPEA-II 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



We thank the anonymous reviewers and editor whose comments/suggestions helped to improve and clarify this manuscript.

Funding information

The Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq: of Brazil gave the scholarships awarded to JCTG and DSA, process numbers 141181/2015-0 and 141448/2015-6. This work was also supported by CNPq’s project number 443834/2014-8.


  1. 1.
    Aytug, H., Koehler, G.J.: Stopping criteria for finite length genetic algorithms. ORSA J. Comput. 8(2), 183–191 (1996)CrossRefGoogle Scholar
  2. 2.
    Aytug, H., Koehler, G.J.: New stopping criterion for genetic algorithms. Eur. J. Oper. Res. 126(3), 662–674 (2000)CrossRefGoogle Scholar
  3. 3.
    Bekele, E.G., Nicklow, J.W.: Multi-objective automatic calibration of SWAT using NSGA-II. J. Hydrol. 341(3–4), 165–176 (2007)CrossRefGoogle Scholar
  4. 4.
    Bravo, J. M., Allasia, D., Collischonn, W., Tassi, R., Meller, A., Tucci, C.E.M.: Avaliação visual e numérica da calibração do modelo hidrológico IPH II com fins educacionais. In: XVII Simpósio Brasileiro de Recursos Hídricos, 2007, São Paulo. Anais do XVII Simpósio Brasileiro de Recursos Hídricos. Porto Alegre: Associação Brasileira de Recursos Hídricos, v. 1 (2007)Google Scholar
  5. 5.
    Chugh, T., Sindhya, K., Hakanen, J., Miettinen, K.: A survey on handling computationally expensive multiobjective optimization problems with evolutionary algorithms. Soft. Comput. 23(9), 3137–3166 (2019)CrossRefGoogle Scholar
  6. 6.
    Coello, C.A.C., Lamont, G.B., Van Veldhuizen, D.A.: Evolutionary Algorithms for Solving Multi-Objective Problems. Springer, New York (2007)Google Scholar
  7. 7.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.A.M.T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002)CrossRefGoogle Scholar
  8. 8.
    Deb, K., Jain, H.: An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: solving problems with box constraints. IEEE Trans. Evol. Comput. 18(4), 602–622 (2014). CrossRefGoogle Scholar
  9. 9.
    Ercan, M.B., Goodall, J.L.: Design and implementation of a general software library for using NSGA-II with SWAT for multi-objective model calibration. Environ. Model. Softw. 84, 112–120 (2016)CrossRefGoogle Scholar
  10. 10.
    Fernández, F.V., Esteban-Muller, J., Roca, E., Castro-López, R.: Stopping criteria in evolutionary algorithms for multi-objective performance optimization of integrated inductors. In: IEEE Congress on Evolutionary Computation. pp. 1–8. (2010)Google Scholar
  11. 11.
    Garcia, F., Folton, N., Oudin, L.: Which objective function to calibrate rainfall–runoff models for low-flow index simulations? Hydrol. Sci. J. 62(7), 1149–1166 (2017)CrossRefGoogle Scholar
  12. 12.
    Germano, A., Tucci, C.E.M., Silveira, A.L.L.d.: Estimativa dos parâmetros do Modelo IPH II para algumas bacias urbanas brasileiras. Rev. Bras. Recur. Hidr. 3(4), 103–120 (1998)Google Scholar
  13. 13.
    Guo, J., Zhou, J., Zou, Q., Liu, Y., Song, L.: A novel multi-objective shuffled complex differential evolution algorithm with application to hydrological model parameter optimization. Water Resour. Manag. 27(8), 2923–2946 (2013)CrossRefGoogle Scholar
  14. 14.
    Guo, J., Zhou, J., Lu, J., Zou, Q., Zhang, H., Bi, S.: Multi-objective optimization of empirical hydrological model for streamflow prediction. J. Hydrol. 511, 242–253 (2014)CrossRefGoogle Scholar
  15. 15.
    Ishibuchi, H., Masuda, H., Tanigaki, Y., Nojima, Y.: Difficulties in specifying reference points to calculate the inverted generational distance for many-objective optimization problems. In: 2014 IEEE Symposium on Computational Intelligence in Multi-Criteria Decision-Making (MCDM), pp. 170–177 (2014)Google Scholar
  16. 16.
    Krause, P., Boyle, D.P., Bäse, F.: Comparison of different efficiency criteria for hydrological model assessment. Adv. Geosci. 5, 89–97 (2005)CrossRefGoogle Scholar
  17. 17.
    Legates, D.R., McCabe Jr., G.J.: Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation. Water Resour. Res. 35(1), 233–241 (1999)CrossRefGoogle Scholar
  18. 18.
    Li, Z., Harman, M., Hierons, R.M.: Search algorithms for regression test case prioritization. IEEE Trans. Softw. Eng. 33(4), 225–237 (2007)CrossRefGoogle Scholar
  19. 19.
    Lin, F., Chen, X., Yao, H.: Evaluating the use of Nash-Sutcliffe efficiency coefficient in goodness-of-fit measures for daily runoff simulation with SWAT. J. Hydrol. Eng. 22(11), 05017023 (2017)CrossRefGoogle Scholar
  20. 20.
    Madsen, H.: Automatic calibration of a conceptual rainfall–runoff model using multiple objectives. J. Hydrol. 235(3–4), 276–288 (2000)CrossRefGoogle Scholar
  21. 21.
    Martí, L., García, J., Berlanga, A., Molina, J.M.: A stopping criterion for multi-objective optimization evolutionary algorithms. Inf. Sci. 367, 700–718 (2016)CrossRefGoogle Scholar
  22. 22.
    Martinek, P., Maršík, J.: Optimized Design of Analogue Circuits Using DE Algorithms. In: 2005 IMAPS CS International Conference Proceedings. pp. 385–389 (2005)Google Scholar
  23. 23.
    Moriasi, D.N., Arnold, J.G., Van Liew, M.W., Binger, R.L., Harmel, R.D., Veith, T.L.: Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans. ASABE. 50(3), 885–900 (2007)CrossRefGoogle Scholar
  24. 24.
    Mostafaie, A., Forootan, E., Safari, A., Schumacher, M.: Comparing multi-objective optimization techniques to calibrate a conceptual hydrological model using in situ runoff and daily GRACE data. Comput. Geosci. 1–26 (2017)Google Scholar
  25. 25.
    Moussa, R., Chahinian, N.: Comparison of different multi-objective calibration criteria using a conceptual rainfall-runoff model of flood events. Hydrol. Earth Syst. Sci. 13(4), 519–535 (2008)CrossRefGoogle Scholar
  26. 26.
    Nazemi, Alireza, Yao, Xin, Chan, Andrew H.: Extracting a set of robust Pareto-optimal parameters for hydrologic models using NSGA-II and SCEM. In: 2006 IEEE International Conference on Evolutionary Computation. 1901–1908 (2006)Google Scholar
  27. 27.
    Peng, T., Zhou, J., Zhang, C., Sun, N.: Modeling and combined application of orthogonal Cashaotic NSGA-II and improved TOPSIS to optimize a conceptual hydrological model. Water Resour. Manag. 32(11), 3781–3799 (2018)CrossRefGoogle Scholar
  28. 28.
    Ramesh, S., Kannan, S., Baskar, S.: Application of modified NSGA-II algorithm to multi-objective reactive power planning. Appl. Soft Comput. 12(2), 741–753 (2012)CrossRefGoogle Scholar
  29. 29.
    Rangaiah, G.P., Sharma, S., Lin, H.W.: Evaluation of two termination criteria in evolutionary algorithms for multi-objective optimization of complex chemical processes. Chem. Eng. Res. Des. 124, 58–65 (2017)CrossRefGoogle Scholar
  30. 30.
    Pushpalatha, R., Perrin, C., Le Moine, N., Andréassian, V.: A review of efficiency criteria suitable for evaluating low-flow simulations. J. Hydrol. 420, 171–182 (2012)CrossRefGoogle Scholar
  31. 31.
    Reynolds, J.E., Halldin, S., Xu, C.Y., Seibert, J., Kauffeldt, A.: Sub-daily runoff predictions using parameters calibrated on the basis of data with a daily temporal resolution. J. Hydrol. 550, 399–411 (2017)CrossRefGoogle Scholar
  32. 32.
    Rudenko, O., Schoenauer, M.: A steady performance stopping criterion for Pareto-based evolutionary algorithms. In 6th International Multi-Objective Programming and Goal Programming Conference. (2004)Google Scholar
  33. 33.
    Savic, D.: Single-objective vs. multiobjective optimisation for integrated decision support. International Congress on Environmental Modelling and Software. (2002)Google Scholar
  34. 34.
    Shafii, M., De Smedt, F.: Multi-objective calibration of a distributed hydrological model (WetSpa) using a genetic algorithm. Hydrol. Earth Syst. Sci. 13(11), 2137–2149 (2009)CrossRefGoogle Scholar
  35. 35.
    Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. J. Evol. Comput. 2(3), 221–248 (1995)CrossRefGoogle Scholar
  36. 36.
    Tucci, C.E.M., Clarke, R.T.: Adaptative forecasting with a conceptual rainfall-runoff model. In: Hydrological Forecasting Proceedings of the Oxford Symposium IAHS. 129, 425–454 (1980)Google Scholar
  37. 37.
    Yang, J., Castelli, F., Chen, Y.: Multiobjective sensitivity analysis and optimization of distributed hydrologic model MOBIDIC. Hydrol. Earth Syst. Sci. 18(10), 4101–4112 (2014)CrossRefGoogle Scholar
  38. 38.
    Yapo, P.O., Gupta, H.V., Sorooshian, S.: Multiobjective global optimization for hydrologic models. J. Hydrol. 204, 83–97 (1998)CrossRefGoogle Scholar
  39. 39.
    Zielinski, K., Laur, R.: Stopping criteria for differential evolution in constrained single-objective optimization. In: Chakraborty, U.K. (ed.) Advances in Differential Evolution, pp. 111–138. Springer, Berlin, Heidelberg (2008)CrossRefGoogle Scholar
  40. 40.
    Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–270 (1999)CrossRefGoogle Scholar
  41. 41.
    Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the Strength Pareto Evolutionary Algorithm. TIK-report. 103 (2001)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Instituto de Pesquisas HidráulicasUniversidade Federal do Rio Grande do SulPorto AlegreBrazil

Personalised recommendations