Applying Least Angle Regression to ELM
Basic extreme learning machines apply least square solution to calculate the neural network’s output weights. In the presence of outliers and multi-collinearity, the least square solution becomes invalid. In order to fix this problem, a new kind of extreme learning machine is proposed. An outlier detection technique is introduced to locate outliers and avoid their interference. The least square solution is replaced by regularization for output weights calculation during which the number of hidden nodes is also automatically chosen. Simulation results show that the proposed model has good prediction performance on both normal datasets and datasets contaminated by outliers.
KeywordsMachine learning Extreme learning machine Least angle regression Outlier detection Robustness
Unable to display preview. Download preview PDF.
- 1.Asuncion, A., Newman, D.: UCI machine learning repository (2007), http://archive.ics.uci.edu/ml/
- 2.Chang, C., Lin, C.: Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 1–27 (2011)Google Scholar
- 5.Frénay, B., Verleysen, M.: Using svms with randomised feature spaces: an extreme learning approach. In: European Symposium on Artificial Neural Networks (ESANN), pp. 315–320 (2010)Google Scholar
- 7.Hoerl, A., Kennard, R.: Ridge regression: biased estimation for nonorthogonal problems. Technometrics 42, 80–86 (2000)Google Scholar
- 12.Jaeger, H.: The ”echo state” approach to analysing and training recurrent neural networks-with an erratum note. GMD Report 148, German National Research Center for Information Technology (2001)Google Scholar
- 13.Lendasse, A., Sorjamaa, A., Miche, Y.: Op-elm toolbox (2008), http://www.cis.hut.fi/projects/tsp/index.php?page=OPELM
- 15.Mitra, K., Veeraraghavan, A., Chellappa, R.: Robust regression using sparse learning for high dimensional parameter estimation problems. In: IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 3846–3849 (2010)Google Scholar
- 16.Skoglund, K.: Regression and pca software (2011), http://www2.imm.dtu.dk/~ksjo/kas/software/spca
- 17.Zhu, Q., Huang, G.B.: Extreme learning machine toolbox (2004), http://www.ntu.edu.sg/eee/icis/cv/egbhuang.htm