H-PSO-LSTM: Hybrid LSTM Trained by PSO for Online Handwriter Identification
The automatic writer’s recognition from his manuscript is a topical issue handling online writing. Recurrent neural networks (RNNs) are an effective means of solving such problem. More specifically, RNN networks with Long and Short Term Memory (LSTM) represent an ideal mean for writer’s recognition. Intuitively, LSTM networks are based on the gradient method for their learning processes. In addition, an LSTM node presents a complex data processing machine.
Our hybrid approach combining LSTM and PSO (H-PSO-LSTM) presents the purpose of this paper and increases the performance of the network.
Experiments were carried out on a Biometrics Ideal Test (BIT) bilingual database (Chinese and English). The BIT deals with a large number of writers (between 130 and 188). With H-PSO-LSTM, we were able to improve the learning performance accuracy to 91.9% instead of 81.2%.
KeywordsRecurrent Neural Network Long-Short Term Memory Particle Swarm Optimization Biometrics Ideal Test Random Hybrid Strokes
The research leading to these results has received funding from the Ministry of Higher Education and Scientific Research of Tunisia under the grant agreement number LR11ES48.
- 3.Dhahri, H., Alimi, M.A.: The modified differential evolution and the RBF (MDE-RBF) neural network for time series prediction. In: IJCNN 2006, International Joint Conference on Neural Network, pp. 2938–2943. IEEE, Vancouver (2006)Google Scholar
- 4.Boubaker, H., Kherallah, M., Alimi, M.A.: New algorithm of straight or curved baseline detection for short Arabic handwritten writing. In: ICDAR 2009, 10th International Conference on Document Analysis and Recognition, pp. 778–782. IEEE, Barcelona (2009)Google Scholar
- 7.Bezine, H., Alimi, M.A., Derbel, N.: Handwriting trajectory movements controlled by a beta-elliptic model. In: ICDAR 2003, Proceedings of the International Conference on Document Analysis and Recognition, Scotland, pp. 1228–1232 (2003)Google Scholar
- 10.Chen, K., Yan, Z., Huo, Q.: A context-sensitive-chunk BPTT approach to training deep LSTM/BLSTM recurrent neural networks for offline handwriting recognition. In: ICDAR 2015, 13th International Conference on Document Analysis and Recognition, France, pp. 411–415. IEEE (2015)Google Scholar
- 13.Elbaati, A., Boubaker, H., Kherallah, M., Ennaji, A., El Abed, H., Alimi, M.A.: Arabic handwriting recognition using restored stroke chronology. In: ICDAR 2009, 10th International Conference on Document Analysis and Recognition, Beijing, China, pp. 411–415. IEEE (2009)Google Scholar
- 14.Huang, T.Y., Li, C.J., Hsu, T.W.: Structure and parameter learning algorithm of Jordan type recurrent neural networks. In: IJCNN 2007, International Joint Conference Neural Networks, pp. 1819–1824. IEEE, Barcelona (2007)Google Scholar
- 15.Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: ICML 2013, International Conference on Machine Learning, Atlanta, pp. 1310–1318 (2013)Google Scholar
- 19.Elloumi, W., Alimi, M.A.: A more efficient MOPSO for optimization. In: AICCSA 2010, ACS/IEEE International Conference on Computer System and Applications, Tunisia, pp. 1–7 (2010)Google Scholar
- 20.Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. (2016)Google Scholar
- 24.Bali, O., Elloumi, W., Abraham, A., Alimi, M.A.: GPU PSO and ACO applied to TSP for vehicle security tracking. J. Inf. Assur. Secur. 11(6), 369–384 (2016)Google Scholar
- 25.Elloumi, W., Alimi, M.A.: Combinatory optimization of ACO and PSO. In: META 2008, Second International Conference on Metaheuristics and Nature Inspired Computing, Tunisia, pp. 1–8 (2008)Google Scholar
- 26.Feng, M., Pan, H.: A modified PSO algorithm based on cache replacement algorithm. In: CIS 2014, Computational Intelligence and Security, China, pp. 558–562 (2014)Google Scholar
- 29.Sanjeevi, S.G., Nikhila, A.N., Khan, T., Sumathi, G.: Hybrid PSO-SA algorithm for training a neural network for classification. Int. J. Comput. Sci. Eng. Appl. (IJCSEA 2011) 1(6), 73–83 (2011)Google Scholar
- 30.Dhahri, H., Alimi, M.A.: The modified differential evolution and the RBF (MDE-RBF) neural network for time series prediction. In: IJCNN 2006, International Joint Conference on Neural Networks, pp. 2938–2943. IEEE, Vancouver (2006)Google Scholar