Timeliness online regularized extreme learning machine
To improve the learning performance, a novel online sequential extreme learning machine (ELM) algorithm for single-hidden layer feedforward networks is proposed with regularization mechanism in a unified framework. The proposed algorithm is called timeliness online regularized extreme learning machine (TORELM). Like the timeliness managing extreme learning machine which improves online sequential extreme learning machine by incorporating timeliness management scheme into ELM approach for the incremental training samples, TORELM also analyzes the training data one-by-one or chunk-by-chunk (a block of data) with fixed or varied chunk size under the similar framework. Meanwhile, the newly incremental training data could be prior to the historical data by maximizing the contribution of the newly increasing training data, since in some cases it may be more feasible that the incremental data can contribute reasonable weights to represent the current system situation in accordance with the practical analysis. Furthermore, in consideration of the disproportion between empirical risk and structural risk in some traditional learning methods, we add regularization technique to the timeliness scheme of TORELM through the use of a weight factor to balance them to achieve better generalization performance. Hence, TORELM has its unique feature of higher generalization capability in most cases with a small testing error while implementing online sequential learning. In addition, this algorithm is still competitive in training time compared with other schemes. Finally, the simulation results show that TORELM can achieve higher learning accuracy and better stability than other ELM-based machine learning methods.
KeywordsIncremental learning Extreme learning machine Timeliness online regularized extreme learning machine Regularization
This research is funded by the National Natural Science Foundation of China under Grants 61174103, 61272357 and 61300074, the National Key Technologies R&D Program of China under Grant 2015BAK38B01, the Aerospace Science Foundation of China under Grant 2014ZA74001, and the Fundamental Research Funds for the Central Universities under Grant 06500025.
- 8.Cambria E, Huang GB, Kasun LLC, Zhou H, Vong CM, Lin J, Yin J, Cai Z, Liu Q, Li K, Leung VCM, Feng L, Ong YS, Lim MH, Akusok A, Lendasse A, Corona F, Nian R, Miche Y, Gastaldo P, Zunino R, Decherchi S, Yang X, Mao K, Oh BS, Jeon J, Toh KA, Teoh ABJ, Kim J, Yu H, Chen Y, Liu J (2013) Extreme learning machines. IEEE Intell Syst 28(6):30–59CrossRefGoogle Scholar
- 19.Yuan L, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13):3391–3395Google Scholar
- 22.Deng WY, Zheng QH, Chen L (2009) Regularized extreme learning machine. In: Proceedings of the IEEE symposium on computational intelligence and data mining, pp 389–395Google Scholar
- 26.Glenn F, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of the 7th ACM SIGKDD international conference on knowledge discovery and data mining, pp 77–86Google Scholar
- 31.Stock Data (2016) YAHOO Finance API. http://ichart.yahoo.com/table.csv?s=600005.SS&a=00&b=01&c=2003&d=04&e=40&f=2012&g=d. Accessed 5 May 2016
- 32.Graham L, Oppacher F (2008) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/Hill-Valley. Accessed 5 May 2016
- 33.Dias DB, Peres SM, Bíscaro HH (2009) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/Libras+Movement. Accessed 5 May 2016
- 34.Roesler O (2013) UCI machine learning repository, University of California, Irvine, School of Information and Computer Sciences. http://archive.ics.uci.edu/ml/datasets/EEG+Eye+State. Accessed 5 May 2016