Abstract
We propose a new iterative search procedure for the numerical treatment of unconstrained multi-objective optimization problems (MOPs) which steers the search along a predefined direction given in objective space. Based on this idea we will present two methods: directed search (DS) descent which seeks for improvements of the given model, and a novel continuation method (DS continuation) which allows to search along the Pareto set of a given MOP. One advantage of both methods is that they can be realized with and without gradient information, and if neighborhood information is available the computation of the search direction comes even for free. The latter makes our algorithms interesting candidates for local search engines within memetic strategies. Further, the approach can be used to gain some interesting insights into the nature of multi-objective stochastic local search which may explain one facet of the success of multi-objective evolutionary algorithms (MOEAs). Finally, we demonstrate the strength of the method both as standalone algorithm and as local search engine within a MOEA.
Similar content being viewed by others
Notes
If the rank of \(J:= J(x_0)\) is k (i.e., maximal) the pseudo inverse is given by \(J^+ = J^T(JJ^T)^{-1}\).
We note that the original idea of NBI is not to maximize the distance from \(F(x_0)\) for a given point \(x_0\), but this is a straightforward adaption to the current context.
References
Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, Chichester (2001)
Beume, N., Naujoks, B., Emmerich, M.: SMS-EMOA: multiobjective selection based on dominated hypervolume. Eur. J. Oper. Res. 181(3), 1653–1669 (2007)
Zhang, Q., Li, H.: MOEA/D: a multi-objective evolutionary algorithm based on decomposition. IEEE Trans. Evol. Comput. 11(6), 712–731 (2007)
Knowles, J., Corne, D.: M-PAES: a memetic algorithm for multiobjective optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation, pp. 325–332. Piscataway, NJ (2000)
Ishibuchi, H., Yoshida, T., Murata, T.: Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. Evol. Comput. 7(2), 204–223 (2003)
Lara, A., Sanchez, G., Coello, C.A.C., Schütze, O.: HCS: a new local search strategy for memetic multiobjective evolutionary algorithms. IEEE Trans. Evol. Comput. 14(1), 112–132 (2010)
Vasile, M., Zuiani, F.: Multi-agent collaborative search: an agent-based memetic multi-objective optimization algorithm applied to space trajectory design. Proc. Inst. Mech. Eng. G 225(11), 1211–1227 (2011)
Zuiani, F., Vasile, M.: Multi agent collaborative search based on tchebycheff decomposition. Comput. Optim. Appl. 56(1), 189–208 (2013)
Roy, B.: Problems and methods with multiple objective functions. Math. Program. 1, 239–266 (1971)
Gembicki, F.W., Haimes, Y.Y.: Approach to performance and multiobjective sensivity optimization: the goal attainment method. IEEE Trans. Autom. Control 20, 769–771 (1975)
Das, I., Dennis, J.: Normal-boundary intersection: a new method for generating the Pareto surface in nonlinear multicriteria optimization problems. SIAM J. Optim. 8, 631–657 (1998)
Potschka, A., Logist, F., Impe, J.F.V., Bock, H.G.: Tracing the Pareto frontier in bi-objective optimization problems by ODE techniques. Num. Algorithms 57(2), 217–233 (2011)
Bosman, P.A.N.: On gradients and hybrid evolutionary algorithms for real-valued multiobjective optimization. IEEE Trans. Evol. Comput. 16(1), 51–69 (2012)
Schütze, O., Lara, A., Coello, C.A.C.: The directed search method for unconstrained multi-objective optimization problems. In: Proceedings of the EVOLVE—a bridge between probability, set oriented numerics, and evolutionary computation, pp. 1–4 (2011)
Mejia, E., Schütze, O.: A predictor corrector method for the computation of boundary points of a multi-objective optimization problem. In: CCE 2010, pp. 395–399 (2010)
Lara, A., Alvarado, S., Salomon, S., Avigad, G., Coello, C.A.C., Schütze, O.: The gradient free directed search method as local search within multi-objective evolutionary algorithms. In: EVOLVE II, pp. 153–168 (2013)
Pareto, V.: Manual of Political Economy. The MacMillan Press, London (1971). Original edition in French in 1927
Hillermeier, C.: Nonlinear Multiobjective Optimization—A Generalized Homotopy Approach. Birkhäuser, Basel (2001)
Kuhn, H., Tucker, A.: Nonlinear programming. In: Neumann, J. (ed.) Proceeding of the 2nd Berkeley symposium on mathematical statistics and probability, pp. 481–492 (1951)
Nocedal, J., Wright, S.: Numerical Optimization. Springer Series in Operations Research and Financial Engineering. Springer, New York (2006)
Deuflhard, P., Borneman, F.: Scientific Computing with Ordinary Differential Equations. Texts in Applied Mathematics 42. Springer, New York (2002)
Allgower, E.L., Georg, K.: Numerical Continuation Methods. Springer, New York (1990)
Harada, K., Sakuma, J., Kobayashi, S., Ono, I.: Uniform sampling of local Pareto-optimal solution curves by Pareto path following and its applications in multi-objective GA. In: GECCO (2007)
Wang, H.: Zigzag search for continuous multiobjective optimization. Inf. J. Comput. 25(4), 654–665 (2013)
Schäffler, S., Schultz, R., Weinzierl, K.: A stochastic method for the solution of unconstrained vector optimization problems. J. Optim. Theory Appl. 114(1), 209–222 (2002)
Henderson, M.E.: Multiple parameter continuation: computing implicitly defined k-manifolds, I. J. Bifurc. Chaos 12(3), 451–476 (2003)
Schütze, O., Dell’Aere, A., Dellnitz, M.: On continuation methods for the numerical treatment of multi-objective optimization problems. In: Branke, J. et al. (ed.) Practical Approaches to Multi-objective Optimization, no. 04461 in Dagstuhl Seminar Proceedings (2005)
Boissonnat, J.-D., Ghosh, A.: Triangulating smooth submanifolds with light scaffolding. Math. Comput. Sci. 4(4), 431–461 (2010)
Brown, M., Smith, R.E.: Directed multi-objective optimisation. Int. J. Comput. Syst. Signals 6(1), 3–17 (2005)
Köppen, M., Yoshida, K.: Many-objective particle swarm optimization by gradual leader selection. In: ICANNGA 2007, pp. 323–331. Springer, Berlin (2007)
Schütze, O., Laumanns, M., Tantar, E., Coello, C.A.C., Talbi, E.-G.: Convergence of stochastic search algorithms to gap-free Pareto front approximations. In: GECCO-2007, pp. 892–901 (2007)
Ishibuchi, H., Murata, T.: Multi-objective genetic local search algorithm. In: Proceedings of 3rd IEEE International Conference on Evolutionary Computation, Nagoya, Japan, pp. 119–124 (1996)
Domínguez, I.S., Aguirre, A.H., Valdez, S.I.: A new EDA by a gradient-driven density. In: Parallel Problem Solving from Nature—PPSN XIII—13th International Conference, pp. 352–361 (2014)
Alvarado, S.: El punto a punto de las técnicas de búsquedas local para algoritmos de optimización multiobjetivo. MSc Thesis, Cinvestav-IPN (2012)
Deb, K., Thiele, L., Laumanns, M., Zitzler, E.: Scalable test problems for evolutionary multiobjective optimization. In: Abraham, A., Jain, L., Goldberg, R. (eds.) Evolutionary Multiobjective Optimization. Theoretical Advances and Applications, pp. 105–145. Springer, London (2005)
Motta, R., Silvana, S.A.M., Lyra, P.: A modified NBI and NC method for the solution of n-multiobjective optimization problems. Struct. Multidiscip. Optim. 46(2), 239–259 (2012)
Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms: empirical results. Evol. Comput. 8(2), 173–195 (2000)
Caponio, A., Neri, F.: Integrating Cross-Dominance Adaptation in Multi-objective Memetic Algorithms. Springer, Berlin (2009)
Lin, J.-Y., Chen, Y.-P.: Analysis on the collaboration between global search and local search in memetic computation. IEEE Trans. Evol. Comput. 15(5), 608–623 (2011)
Knowles, J., Corne, D.: Memetic algorithms for multiobjective optimization: issues, methods and prospects. Recent Advances in Memetic Algorithms, pp. 313–352. Springer, Berlin (2005)
Shukla, P.: On gradient based local search methods in unconstrained evolutionary multi-objective mptimization. In: Obayashi, S., et al. (eds.) EMO, pp. 96–110. Springer, Heidelberg (2007)
Zhang, Q., Zhou, A., Zhao, S., Suganthan, P. N., Liu, W., Tiwari, S.: Multi-objective optimisation test instances for the CEC 2009 special session and competition, Working Report CES-887, School of Computer Science and Electrical Engineering, University of Essex, revised on 20/04/2009 (2008)
Zhang, Q., Liu, W., Li, H.: The performance of a new version of MOEA/D on CEC09 unconstrained MOP test instances. In: IEEE Congress on Evolutionary Computation, 2009 (CEC’09), pp. 203–208. IEEE (2009)
Schütze, O., Esquivel, X., Lara, A., Coello, C.A.C.: Using the averaged Hausdorff distance as a performance measure in evolutionary multi-objective optimization. IEEE Trans. Evol. Comput. 16(4), 504–522 (2012)
Zitzler, E.: Evolutionary Algorithms for Multiobjective Optimization: Methods and Applications. Ph.D. Thesis, ETH Zurich, Switzerland (1999)
Acknowledgments
The first author acknowledges support from Conacyt Project No. 128554.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Schütze, O., Martín, A., Lara, A. et al. The directed search method for multi-objective memetic algorithms. Comput Optim Appl 63, 305–332 (2016). https://doi.org/10.1007/s10589-015-9774-0
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10589-015-9774-0