An Enhanced Extreme Learning Machine Based on Liu Regression

Abstract

Extreme learning machine (ELM) is one of the most remarkable machine learning algorithm in consequence of superior properties particularly its speed. ELM algorithm tends to have some drawbacks like instability and poor generalization performance in the presence of perturbation and multicollinearity. This paper introduces a novel algorithm based on Liu regression estimator (L-ELM) to handle these drawbacks. Different selection approaches have been used to determine the appropriate Liu biasing parameter. The new algorithm is tested against the basic ELM, RR-ELM, AUR-ELM and OP-ELM on nine well-known benchmark data sets. Statistical significance tests have been carried out. Experimental results show that L-ELM for at least one Liu biasing parameter generally outperforms basic ELM, RR-ELM, AUR-ELM and OP-ELM in terms of stability and generalization performance with a little lost of speed. Conversely, the training time of L-ELM is generally much slower than RR-ELM, AUR-ELM and OP-ELM. Consequently, the proposed algorithm can be considered a powerful alternative to avoid the loss of performance in regression studies