Advertisement

Some Single- and Multiobjective Optimization Techniques

  • Sanghamitra Bandyopadhyay
  • Sriparna Saha

Abstract

Several metaheuristic techniques optimizing both single and multiple objectives are described in detail in this chapter. Mathematical formulations of the single and multiobjective optimization problems are provided. Different concepts related to multiobjective optimization are described in detail. Two popular metaheuristics, namely genetic algorithms and simulated annealing, are elaborately discussed. Several existing multiobjective evolutionary techniques (MOEAs) are described in brief. Apart from MOEAs there exist several multiobjective simulated annealing (MOSA)-based techniques. These are also described in this chapter. Finally a detailed description of a multiobjective simulated annealing-based technique, AMOSA, is provided, along with an analysis of its time complexity. Comparative results with some existing MOEA and MOSA techniques are presented for several benchmark test problems.

Keywords

Simulated Annealing Pareto Front Multiobjective Optimization Multiobjective Optimization Problem Nondominated Solution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 3.
    Multiobjective simulated annealing. http://www.dcs.ex.ac.uk/people/kismith/mosa/results/tec/
  2. 22.
    Bandyopadhyay, S., Maulik, U., Pakhira, M.K.: Clustering using simulated annealing with probabilistic redistribution. Int. J. Pattern Recognit. Artif. Intell. 15(2), 269–285 (2001) CrossRefGoogle Scholar
  3. 24.
    Bandyopadhyay, S., Pal, S.K.: Classification and Learning Using Genetic Algorithms Applications in Bioinformatics and Web Intelligence. Springer, Heidelberg (2007) zbMATHGoogle Scholar
  4. 25.
    Bandyopadhyay, S., Pal, S.K., Aruna, B.: Multi-objective GAs, quantitative indices and pattern classification. IEEE Trans. Syst. Man Cybern., Part B, Cybern. 34(5), 2088–2099 (2004) CrossRefGoogle Scholar
  5. 26.
    Bandyopadhyay, S., Pal, S.K., Murthy, C.A.: Simulated annealing based pattern classification. Inf. Sci. 109(1–4), 165–184 (1998) MathSciNetCrossRefGoogle Scholar
  6. 29.
    Bandyopadhyay, S., Saha, S., Maulik, U., Deb, K.: A simulated annealing based multi-objective optimization algorithm: AMOSA. IEEE Trans. Evol. Comput. 12(3), 269–283 (2008) CrossRefGoogle Scholar
  7. 40.
    Bhandarkar, S.M., Zhang, H.: Image segmentation using evolutionary computation. IEEE Trans. Evol. Comput. 3(1), 1–21 (1999) CrossRefGoogle Scholar
  8. 52.
    Caves, R., Quegan, S., White, R.: Quantitative comparison of the performance of SAR segmentation algorithms. IEEE Trans. Image Process. 7(11), 1534–1546 (1998) CrossRefGoogle Scholar
  9. 56.
    Chipperfield, A., Whidborne, J., Fleming, P.: Evolutionary algorithms and simulated annealing for MCDM. In: Multicriteria Decision Making – Advances in MCDM Models, Algorithms, Theory and Applications, pp. 16.1–16.32. Kluwer Academic, Boston (1999) Google Scholar
  10. 62.
    Coello Coello, C.A.: A comprehensive survey of evolutionary-based multiobjective optimization techniques. Knowl. Inf. Syst. 1(3), 129–156 (1999) Google Scholar
  11. 63.
    Coello Coello, C.A., Veldhuizen, D.V., Lamont, G.: Evolutionary Algorithms for Solving Multi-Objective Problems. Kluwer Academic, Boston (2002) zbMATHGoogle Scholar
  12. 65.
    Corne, D.W., Jerram, N.R., Knowles, J.D., Oates, M.J.: PESA-II: Region-based selection in evolutionary multiobjective optimization. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 283–290. Morgan Kaufmann, San Francisco (2001). citeseer.ist.psu.edu/corne01pesaii.html Google Scholar
  13. 66.
    Corne, D.W., Knowles, J.D., Oates, M.J.: The Pareto-envelope based selection algorithm for multiobjective optimisation. In: Proceedings of the Parallel Problem Solving from Nature – PPSN VI, Springer Lecture Notes in Computer Science, pp. 869–878 (2000) Google Scholar
  14. 68.
    Czyzak, P., Jaszkiewicz, A.: Pareto simulated annealing – A metaheuristic technique for multiple-objective combinatorial optimization. J. Multi-Criteria Decis. Anal. 7(1), 34–47 (1998) zbMATHCrossRefGoogle Scholar
  15. 69.
    Das, I., Dennis, J.: A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems. Struct. Optim. 14(1), 63–69 (1997) CrossRefGoogle Scholar
  16. 74.
    Davis, L. (ed.): Genetic Algorithms and Simulated Annealing. Morgan Kaufmann, Los Altos (1987) zbMATHGoogle Scholar
  17. 75.
    Davis, L. (ed.): Handbook of Genetic Algorithms. Van Nostrand-Reinhold, New York (1991) Google Scholar
  18. 76.
    Deb, K.: Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evol. Comput. 7(3), 205–230 (1999) CrossRefGoogle Scholar
  19. 77.
    Deb, K.: Multi-objective Optimization Using Evolutionary Algorithms. Wiley, England (2001) zbMATHGoogle Scholar
  20. 78.
    Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 182–197 (2002) CrossRefGoogle Scholar
  21. 79.
    DeJong, K.: Learning with genetic algorithms: An overview. Mach. Learn. 3(2-3), 121–138 (1988) CrossRefGoogle Scholar
  22. 90.
    Engrand, P.: A multi-objective approach based on simulated annealing and its application to nuclear fuel management. In: 5th International Conference on Nuclear Engineering, Nice, France, pp. 416–423 (1997) Google Scholar
  23. 91.
    Erickson, M., Mayer, A., Horn, J.: Multi-objective optimal design of groundwater remediation systems: Application of the niched Pareto genetic algorithm (NPGA). Adv. Water Resour. 25(1), 51–65 (2002) CrossRefGoogle Scholar
  24. 99.
    Fieldsend, J., Everson, R., Singh, S.: Using unconstrained elite archives for multi-objective optimisation. IEEE Trans. Evol. Comput. 7(3), 305–323 (2003) CrossRefGoogle Scholar
  25. 101.
    Fonseca, C.M., Fleming, P.J.: An overview of evolutionary algorithms in multiobjective optimization. Evol. Comput. 3(1), 1–16 (1995) CrossRefGoogle Scholar
  26. 111.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984) zbMATHCrossRefGoogle Scholar
  27. 112.
    Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley, New York (1989) zbMATHGoogle Scholar
  28. 116.
    Grefenstette, J.J.: Optimization of control parameters for genetic algorithms. IEEE Trans. Syst. Man Cybern. 16, 122–128 (1986) CrossRefGoogle Scholar
  29. 123.
    Hapke, M., Jaszkiewicz, A., Slowinski, R.: Pareto simulated annealing for fuzzy multi-objective combinatorial optimization. J. Heuristics 6(3), 329–345 (2000) zbMATHCrossRefGoogle Scholar
  30. 137.
    Hughes, E.J.: Evolutionary many-objective optimization: Many once or one many. In: Proceedings of 2005 Congress on Evolutionary Computation, Edinburgh, Scotland, UK, September 2–5, 2005, pp. 222–227 (2005) CrossRefGoogle Scholar
  31. 138.
    Ingber, L.: Very fast simulated re-annealing. Math. Comput. Model. 12(8), 967–973 (1989) MathSciNetzbMATHCrossRefGoogle Scholar
  32. 139.
    Ishibuchi, H., Doi, T., Nojima, Y.: Incorporation of scalarizing fitness functions into evolutionary multiobjective optimization algorithms. In: Parallel Problem Solving from Nature IX (PPSN-IX), vol. 4193, pp. 493–502 (2006) CrossRefGoogle Scholar
  33. 140.
    Ishibuchi, H., Murata, T.: A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 28(3), 392–403 (1998) CrossRefGoogle Scholar
  34. 141.
    Ishibuchi, H., Yoshida, T., Murata, T.: Balance between genetic search and local search in memetic algorithms for multiobjective permutation flowshop scheduling. IEEE Trans. Evol. Comput. 6(6), 721–741 (1984) Google Scholar
  35. 143.
    Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice Hall, Englewood Cliffs (1988) zbMATHGoogle Scholar
  36. 147.
    Jaszkiewicz, A.: Comparison of local search-based metaheuristics on the multiple objective knapsack problem. Found. Comput. Dec. Sci. 26(1), 99–120 (2001) MathSciNetGoogle Scholar
  37. 158.
    Kirkpatrick, S.: Optimization by simulated annealing: Quantitative studies. J. Stat. Phys. 34(5/6), 975–986 (1984) MathSciNetCrossRefGoogle Scholar
  38. 159.
    Kirkpatrick, S., Gelatt, C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220, 671–680 (1983) MathSciNetzbMATHCrossRefGoogle Scholar
  39. 160.
    Kirpatrick, S., Vecchi, M.P.: Global wiring by simulated annealing. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. CAD-2(4), 215–222 (1983) Google Scholar
  40. 161.
    Knowles, J.D., Corne, D.W.: Approximating the nondominated front using the Pareto archived evolution strategy. Evol. Comput. 8(2), 149–172 (2000) CrossRefGoogle Scholar
  41. 164.
    Konak, A., Coit, D., Smith, A.: Multi-objective optimization using genetic algorithms: A tutorial. Reliab. Eng. Syst. Saf. 91(9), 992–1007 (2006). http://linkinghub.elsevier.com/retrieve/pii/S0951832005002012 CrossRefGoogle Scholar
  42. 171.
    Kwanghoon, S., Jung, K.H., Alexander, W.E.: A mean field annealing approach to robust corner detection. IEEE Trans. Syst. Man Cybern., Part B, Cybern. 28(1), 82–90 (1998) CrossRefGoogle Scholar
  43. 191.
    Maulik, U., Bandyopadhyay, S., Trinder, J.: SAFE: An efficient feature extraction technique. J. Knowl. Inf. Syst. 3(3), 374–387 (2001) zbMATHCrossRefGoogle Scholar
  44. 193.
    Maulik, U., Bandyopadhyay, S., Mukhopadhyay, A.: Multiobjective Genetic Algorithms for Clustering – Applications in Data Mining and Bioinformatics. Springer, Heidelberg (2011) zbMATHCrossRefGoogle Scholar
  45. 195.
    Metropolis, N., Rosenbluth, A.W., Rosenbloth, M.N., Teller, A.H., Teller, E.: Equation of state calculation by fast computing machines. J. Chem. Phys. 21(6), 1087–1092 (1953) CrossRefGoogle Scholar
  46. 197.
    Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, New York (1992) zbMATHGoogle Scholar
  47. 206.
    Nam, D., Park, C.H.: Pareto-based cost simulated annealing for multiobjective optimization. In: Proceedings of the 4th Asia-Pacific Conference on Simulated Evolution and Learning (SEAL’02), vol. 2, pp. 522–526. Nanyang Technical University, Orchid Country Club, Singapore (2002) Google Scholar
  48. 207.
    Nam, D.K., Park, C.H.: Multiobjective simulated annealing: A comparative study to evolutionary algorithms. Int. J. Fuzzy Syst. 2(2), 87–97 (2000) Google Scholar
  49. 210.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, Heidelberg (2007) Google Scholar
  50. 236.
    Rudolph, G.: Convergence analysis of canonical genetic algorithms. IEEE Trans. Neural Netw. 5(1), 96–101 (1994) CrossRefGoogle Scholar
  51. 249.
    Schaffer, J.: Multiple objective optimization with vector evaluated genetic algorithms. In: Genetic Algorithms and Their Applications: Proceedings of the First International Conference on Genetic Algorithms, pp. 93–100 (1985) Google Scholar
  52. 250.
    Schott, J.R.: Fault tolerant design using single and multi-criteria genetic algorithms. Master’s thesis, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Boston, MA (1995) Google Scholar
  53. 252.
    Serafini, P.: Simulated annealing for multiple objective optimization problems. In: Proceedings of the Tenth International Conference on Multiple Criteria Decision Making: Expand and Enrich the Domains of Thinking and Application, vol. 1, pp. 283–292. Springer, Berlin (1994) Google Scholar
  54. 257.
    Smith, K.I., Everson, R.M., Fieldsend, J.E.: Dominance measures for multi-objective simulated annealing. In: Proceedings of the 2004 IEEE Congress on Evolutionary Computation (CEC’04), pp. 23–30 (2004) Google Scholar
  55. 258.
    Smith, K.I., Everson, R.M., Fieldsend, J.E., Murphy, C., Misra, R.: Dominance-based multi-objective simulated annealing. IEEE Trans. Evol. Comput. 12(3), 323–342 (2008) CrossRefGoogle Scholar
  56. 259.
    Sontag, E., Sussman, H.: Image restoration and segmentation using the annealing algorithm. IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. CAD-2(4), 215–222 (1983) Google Scholar
  57. 261.
    Srinivas, N., Deb, K.: Multiobjective optimization using nondominated sorting in genetic algorithms. Evol. Comput. 2(3), 221–248 (1994) CrossRefGoogle Scholar
  58. 270.
    Suman, B.: Study of self-stopping PDMOSA and performance measure in multiobjective optimization. Comput. Chem. Eng. 29(5), 1131–1147 (2005) CrossRefGoogle Scholar
  59. 271.
    Suman, B.: Multiobjective simulated annealing – A metaheuristic technique for multiobjective optimization of a constrained problem. Found. Comput. Dec. Sci. 27(3), 171–191 (2002) Google Scholar
  60. 272.
    Suman, B.: Simulated annealing based multiobjective algorithm and their application for system reliability. Eng. Optim. 35(4), 391–416 (2003) CrossRefGoogle Scholar
  61. 273.
    Suman, B.: Study of simulated annealing based multiobjective algorithm for multiobjective optimization of a constrained problem. Comput. Chem. Eng. 28(9), 1849–1871 (2004) CrossRefGoogle Scholar
  62. 274.
    Suman, B., Kumar, P.: A survey of simulated annealing as a tool for single and multiobjective optimization. J. Oper. Res. Soc. 57(10), 1143–1160 (2006) zbMATHCrossRefGoogle Scholar
  63. 275.
    Suppapitnarm, A., Seffen, K., Parks, G., Clarkson, P.: A simulated annealing algorithm for multiobjective optimization. Eng. Optim. 33(1), 59–85 (2000) CrossRefGoogle Scholar
  64. 276.
    Szu, H.H., Hartley, R.L.: Fast simulated annealing. Phys. Lett. A 122(3–4), 157–162 (1987) CrossRefGoogle Scholar
  65. 283.
    Toussaint, G.T.: Pattern recognition and geometrical complexity. In: Proc. Fifth International Conf. on Pattern Recognition, Miami Beach, December 1980, pp. 1324–1347 (1980) Google Scholar
  66. 285.
    Tuyttens, D., Teghem, J., El-Sherbeny, N.: A particular multiobjective vehicle routing problem solved by simulated annealing. In: Metaheuristics for Multiobjective Optimization, vol. 535, 133–152 (2003) CrossRefGoogle Scholar
  67. 286.
    Ulungu, E.L., Teghaem, J., Fortemps, P., Tuyttens, D.: MOSA method: A tool for solving multiobjective combinatorial decision problems. J. Multi-Criteria Decis. Anal. 8(4), 221–236 (1999) zbMATHCrossRefGoogle Scholar
  68. 298.
    Yao, X.: A new simulated annealing algorithm. Int. J. Comput. Math. 56, 161–168 (1995) zbMATHCrossRefGoogle Scholar
  69. 307.
    Zitzler, E., Deb, K., Thiele, L.: Comparison of multiobjective evolutionary algorithms: Empirical results. Evol. Comput. 8(2), 173–195 (2000) CrossRefGoogle Scholar
  70. 308.
    Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: Improving the strength pareto evolutionary algorithm. Tech. Rep. 103, Gloriastrasse 35, CH-8092 Zurich, Switzerland (2001) Google Scholar
  71. 309.
    Zitzler, E., Thiele, L.: Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach. IEEE Trans. Evol. Comput. 3(4), 257–271 (1999) CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sanghamitra Bandyopadhyay
    • 1
  • Sriparna Saha
    • 2
  1. 1.Machine Intelligence UnitIndian Statistical InstituteKolkataIndia
  2. 2.Dept. of Computer ScienceIndian Institute of TechnologyPatnaIndia

Personalised recommendations