Abstract
In this chapter, an efficient optimization algorithm is presented for the problems with hard to evaluate objective functions. It uses the reinforcement learning principle to determine the particle move in search for the optimum process. A model of successful actions is build and future actions are based on past experience. The step increment combines exploitation of the known search path and exploration for the improved search direction. The algorithm does not require any prior knowledge of the objective function, nor does it require any characteristics of such function. It is simple, intuitive and easy to implement and tune. The optimization algorithm was tested using several multi-variable functions and compared with other widely used random search optimization algorithms. Furthermore, the training of a multi-layer perceptron, to find a set of optimized weights, is treated as an optimization problem. The optimized multi-layer perceptron was applied to Iris database classification. Finally, the algorithm is used in image recognition to find a familiar object with retina sampling and micro-saccades.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Arfken, G.: Lagrange Multipliers, 3rd edn. §17.6 in Mathematical Methods for Physicists, pp. 945–950. Academic Press, Orlando (1985)
Belur, S.: A random search method for the optimization of a function of n variables. MATLAB central file exchange, http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=100
Cassin, B., Solomon, S.: Dictionary of Eye Terminology. Triad Publishing Company, Gainsville (1990)
Detecting a Cell Using Image Segmentation. Image Processing Toolbox, the Mathworks, http://www.mathworks.com/products/image/demos.html
Dixon, L.C.W., Szego, G.P.: The optimization problem: An introduction. Towards Global Optimization II. North Holland, New York (1978)
Eelder, J.A., Mead, R.: A simplex method for function minimization. The Computer Journal 7, 308–313 (1965)
Facegen Modeller. Singular Inversions, http://www.facegen.com/products.htm
del Toro Garcia, X., Neri, F., Cascella, G.L., Salvatore, N.: A surrogate associated Hooke-Jeeves algorithm to optimize the control system of a PMSM drive. IEEE ISIE, 347–352 (July 2006)
Hill, D.L.G., Batchelor, P.: Registration methodology: concepts and algorithms. In: Hajnal, J.V., Hill, D.L.G., Hawkes, D.J. (eds.) Medical Image Registration. Medical Image Registration. CRC, Boca Raton (2001)
Hooke, R., Jeeves, T.A.: Direct search solution of numerical and statistical problems. Journal of the Association for Computing Machinery 8, 212–229 (1961)
Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proc. IEEE Int. Conf. Neural Networks, Perth, Australia, December 1995, vol. 4, pp. 1942–1948 (1995)
Kim, H., Pang, S., Je, H.: Support vector machine ensemble with bagging. In: Lee, S.-W., Verri, A. (eds.) SVM 2002. LNCS, vol. 2388, p. 397. Springer, Heidelberg (2002)
Kirkpatrick, S., Gelatt Jr., C.D., Vecchi, M.P.: Optimization by simulated annealing. Science 220(4598), 671–680 (1983)
Leontitsis, A.: Hybrid Particle Swarm Optimization, MATLAB central file exchange, http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do? (objectId=6497)
Lewis, R.M., Torczon, V., Trosset, M.W.: Direct search methods: Then and now. Journal of Computational and Applied Mathematics 124(1), 191–207 (2000)
Li, Y.: Active Vision through Invariant Representations and Saccade Movements. Master thesis, School of Electrical Engineering and Computer Science, Ohio University (2006)
Liu, Y., Starzyk, J.A., Zhu, Z.: Optimized Approximation Algorithm in Neural Networks without overfitting. IEEE Trans. on Neural Networks 19(4), 983–995 (2008)
Lustig, I.J., Marsten, R.E., Shanno, D.F.: Computational Experience with a Primal-Dual Interior Point Method for Linear Programming. Linear Algebra and its Application 152, 191–222 (1991)
Martinez-Conde, S., Macknik, S.L., Hubel, D.H.: The role of fixational eye movements in visual perception. Nature Reviews Neuroscience 5(3), 229–240 (2004)
Ong, Y.-S.: Max-min surrogate-assisted evolutionary algorithm for robust design. IEEE Trans. on Evolutionary Computation 10(4), 392–404 (2006)
Powell, M.J.D.: An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal 7, 155–162 (1964)
Fisher, R.A.: Iris Plants Database (July 1988), http://faculty.cs.byu.edu/~cgc/Teaching/CS_478/iris.arff
Rosenbrock, H.H.: An automatic method for finding the greatest or least value of a function. The Computer Journal 3, 175–184 (1960)
Sheela, B.V.: An optimized step-size random search. Computer Methods in Applied Mechanics and Engineering 19(1), 99–106 (1979)
Snyman, J.A.: Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms. Springer, Heidelberg (2005)
Starzyk, J.A.: Motivation in Embodied Intelligence. In: Frontiers in Robotics, Automation and Control, October 2008, pp. 83–110. I-Tech Education and Publishing (2008), http://www.intechweb.org/book.php?%20id=78&content=subject&sid=11
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Torczon, V.: On the Convergence of Pattern Search Algorithms. SIAM Journal on Optimization 17(1), 1–25 (1997)
Vandekerckhove, J.: General simulated annealing algorithm, MATLAB central file exchange, http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=10548
Ypma, T.J.: Historical development of the Newton-Raphson method. SIAM Review 37(4), 531–551 (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Starzyk, J.A., Liu, Y., Batog, S. (2010). A Novel Optimization Algorithm Based on Reinforcement Learning. In: Tenne, Y., Goh, CK. (eds) Computational Intelligence in Optimization. Adaptation, Learning, and Optimization, vol 7. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12775-5_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-12775-5_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-12774-8
Online ISBN: 978-3-642-12775-5
eBook Packages: EngineeringEngineering (R0)