Abstract
Deep learning-based approaches have been paramount in the last years, mainly due to their outstanding results in several application domains, that range from face and object recognition to handwritten digits identification. Convolutional Neural Networks (CNN) have attracted a considerable attention since they model the intrinsic and complex brain working mechanism. However, the huge amount of parameters to be set up may turn such approaches more prone to configuration errors when using a manual tuning of the parameters. Since only a few works have addressed such shortcoming by means of meta-heuristic-based optimization, in this paper we introduce the Harmony Search algorithm and some of its variants for CNN optimization, being the proposed approach validated in the context of fingerprint and handwritten digit recognition, as well as image classification.
Chapter PDF
Similar content being viewed by others
Keywords
- Harmony Search
- Convolutional Neural Network
- Harmony Search Algorithm
- Harmony Memory
- Harmony Memory Consider Rate
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Alia, O., Mandava, R.: The variants of the harmony search algorithm: an overview. Artificial Intelligence Review 36, 49–68 (2011)
Arel, I., Rose, D., Karnowski, T.: Deep machine learning - a new frontier in artificial intelligence research [research frontier]. IEEE Computational Intelligence Magazine 5(4), 13–18 (2010)
Bishop, C.: Neural networks for pattern recognition. Oxford University Press (1995)
Eberhart, R.C., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micromachine and Human Science, pp. 39–43 (1995)
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 1915–1929 (2013)
Fedorovici, L.O., Precup, R.E., Dragan, F., Purcaru, C.: Evolutionary optimization-based training of convolutional neural networks for ocr applications. In: 2013 17th International Conference on System Theory, Control and Computing (ICSTCC), pp. 207–212, October 2013
Geem, Z.W.: Music-Inspired Harmony Search Algorithm: Theory and Applications, 1st edn. Springer Publishing Company, Incorporated (2009)
Kennedy, J., Eberhart, R.C.: Particle swarm optimization. In: Proceedings of the IEEE International Joint Conference on Neural Networks, pp. 1942–1948. IEEE Press (1995)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11), 2278–2324 (1998)
LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of 2010 IEEE International Symposium on Circuits and Systems, pp. 253–256 (2010)
Mahdavi, M., Fesanghary, M., Damangir, E.: An improved harmony search algorithm for solving optimization problems. Applied Mathematics and Computation 188(2), 1567–1579 (2007)
Omran, M.G., Mahdavi, M.: Global-best harmony search. Applied Mathematics and Computation 198(2), 643–656 (2008)
Pan, Q.K., Suganthan, P., Tasgetiren, M.F., Liang, J.: A self-adaptive global best harmony search algorithm for continuous optimization problems. Applied Mathematics and Computation 216(3), 830–848 (2010)
Wan, L., Zeiler, M., Zhang, S., Cun, Y.L., Fergus, R.: Regularization of neural networks using dropconnect. In: Dasgupta, S., Mcallester, D. (eds.) Proceedings of the 30th International Conference on Machine Learning. vol. 28, pp. 1058–1066. JMLR Workshop and Conference Proceedings (2013), http://jmlr.org/proceedings/papers/v28/wan13.pdf
Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bulletin 1(6), 80–83 (1945)
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding (2014). arXiv preprint arXiv:1408.5093
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Rosa, G., Papa, J., Marana, A., Scheirer, W., Cox, D. (2015). Fine-Tuning Convolutional Neural Networks Using Harmony Search. In: Pardo, A., Kittler, J. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2015. Lecture Notes in Computer Science(), vol 9423. Springer, Cham. https://doi.org/10.1007/978-3-319-25751-8_82
Download citation
DOI: https://doi.org/10.1007/978-3-319-25751-8_82
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25750-1
Online ISBN: 978-3-319-25751-8
eBook Packages: Computer ScienceComputer Science (R0)