Abstract
We propose a global optimization algorithm called GOSAM (Global Optimization using Support vector regression based Adaptive Multistart) that applies statistical machine learning techniques, viz. Support Vector Regression (SVR) to adaptively direct iterative search in large-scale global optimization. At each iteration, GOSAM builds a training set of the objective function’s local minima discovered till the current iteration, and applies SVR to construct a regressor that learns the structure of the local minima. In the next iteration the search for the local minimum is started from the minimum of this regressor. The idea is that the regressor for local minima will generalize well to the local minima not obtained so far in the search, and hence its minimum would be a ‘crude approximation’ to the global minimum. This approximation improves over time, leading the search towards regions that yield better local minima and eventually the global minimum. Simulation results on well known benchmark problems show that GOSAM requires significantly fewer function evaluations to reach the global optimum, in comparison with methods like Particle Swarm optimization and Genetic Algorithms. GOSAM proves to be relatively more efficient as the number of design variables (dimension) increases. GOSAM does not require explicit knowledge of the objective function, and also does not assume any specific properties. We also discuss some real world applications of GOSAM involving constrained and design optimization problems.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
http://www.cadence.com/products/custom_ic/spectre/index.aspx
Agakov, F., Bonilla, E., Cavazos, J., Franke, B., Fursin, G., O’Boyle, M., Thomson, J., Toussaint, M., Williams, C.: Using machine learning to focus iterative optimisation. In: Proceedings of the 4th Annual International Symposium on Code Generation and Optimization (CGO), New York, NY, USA, pp. 295–305 (2006)
Baluja, S., Barto, A., Boese, K., Boyan, J., Buntine, W., Carson, T., Caruana, R., Davies, S., Dean, T., Dietterich, T., Hazlehurst, S., Impagliazzo, R., Jagota, A., Kim, K., McGovern, A., Moll, R., Moss, E., Perkins, T., Sanchis, L., Su, L., Wang, X., Wolpert, D.: Statistical machine learning for large-scale optimization. Neural Computing Surveys 3, 1–58 (2000)
Black, F., Litterman, R.: Global portfolio optimization. Financial Analysts Journal 48(5), 28–43 (1992)
Boese, K., Kahng, A.B., Muddu, S.: A new adaptive multi-start technique for combinatorial global optimizations. Operations Research Letters 16(2), 101–113 (1994)
Bordes, A., Bottou, L.: The huller: A simple and efficient online SVM. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) ECML 2005. LNCS (LNAI), vol. 3720, pp. 505–512. Springer, Heidelberg (2005), http://leon.bottou.org/papers/bordes-bottou-2005
Boyan, J.: Learning evaluation functions for global optimization. Phd dissertation, CMU (1998)
Boyan, J., Moore, A.: Learning evaluation functions for global optimization and boolean satisfiability. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, vol. 15, pp. 3–10. John Wiley and Sons Ltd., Chichester (1998), http://www.cs.cmu.edu/~jab/cv/pubs/boyan.stage2.ps.gz
Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B 26(1), 29–41 (1996), http://iridia.ulb.ac.be/~mdorigo/ACO/publications.html
GEATbx: Genetic and evolutionary algorithm toolbox (1994), http://www.geatbx.com/docu/fcnindex.html
Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison-Wesley Longman Publishing Co., Inc., Boston (1989)
Grossmann, I.E. (ed.): Global optimization in engineering design. Kluwer Academic Publishers, Dordrecht (1996)
Gunn, S.: Support vector machines for classification and regression. Technical report, Image Speech and Intelligent Systems Research Group, University of Southampton, UK (1998), http://www.isis.ecs.soton.ac.uk/resources/svminfo/
Horst, R., Tuy, H.: Global optimization:deterministic approaches. Springer, Berlin (1993)
Houck, C., Joines, J., Kay, M.: A genetic algorithm for function optimization: A matlab implementation. NCSU-IE TR 95-09 (1995), http://www.ie.ncsu.edu/mirage/GAToolBox/gaot/
Jayadeva, Dutta Roy, S.C., Chaudhary, A.: Compact analogue neural network: A new paradigm for neural based combinatorial optimisation. IEE Proc-Circuits Devices Syst. 146(3) (1999)
Jaydeva, Shah, S., Chandra, S.: Learning to optimize vlsi design problems. In: INDICON, pp. 1–5. IEEE, New Delhi (2006)
Johnson, D., McGeoch, L.: The travelling salesman problem: A case study in local optimisation. In: Local Search in Combinatorial Optimisation, pp. 215–310. John Wiley and Sons, London (1997)
Kazerounian, K., Wang, Z.: Global versus local optimization in redundancy resolution of robotic manipulators. The International Journal of Robotics Research 7(5), 2–12 (1988)
Kennedy, J., Eberhart, R.: Particle swarm optimization. In: Proceedings of the IEEE International Conference on Neural Networks, Perth, Australia, vol. 4, pp. 1942–1948 (1995)
Kirkpatrick, S., Gelatt, C., Vecchi, M.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)
LINDO SYSTEMS Inc.: LINDO API User’s Manual (2002)
Madsen, K.: Test problems for global optimization, http://www2.imm.dtu.dk/~km/Test_ex_forms/test_ex.html
Mangasarian, O.: Nonlinear Programming. SIAM, Philadelphia (1994)
Moles, C., Mendes, P., Banga, J.: Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Research 13, 2467–2474 (2003)
Neumaier, A.: Global optimization, http://www.mat.univie.ac.at/~neum/glopt/applications.html
Platt, J.: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In: Advances in Kernel Methods - Support Vector Learning, pp. 185–208. MIT Press, Cambridge (1999)
Russel, S., Norvig, P.: Artificial intelligence: a modern approach. Prentice Hall, Englewood Cliffs (1995)
Singh, J.: PSO algorithm toolbox (2003), http://psotoolbox.sourceforge.net/
Wang, M., Yang, X., Sarrafzadeh, M.: Congestion minimization during placement. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 19(10), 1140–1148 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Jayadeva, Shah, S., Chandra, S. (2010). Learning Global Optimization Through a Support Vector Machine Based Adaptive Multistart Strategy. In: Tenne, Y., Goh, CK. (eds) Computational Intelligence in Optimization. Adaptation, Learning, and Optimization, vol 7. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12775-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-12775-5_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12774-8
Online ISBN: 978-3-642-12775-5
eBook Packages: EngineeringEngineering (R0)