Kernel online sequential ELM algorithm with sliding window subject to time-varying environments

Abstract

Extreme learning machine (ELM) is an emerging machine learning algorithm with single-hidden-layer feedforward neural networks (SLFNs). The key strength of ELM algorithm is the significantly fast training speed and good generalization performance since the learning parameters of hidden nodes are generated randomly. The kernel online sequential ELM (KOS-ELM) is a straightforward extension of the well-known recursive least-squares method to the ELM framework. KOS-ELM is a good choice for the online learning of stationary applications. However, there are lots of situations and applications which are time varying quickly. It is unreasonable and inaccurate to pay equal emphasis on both old and new observations. In this paper, we proposed a modified KOS-ELM algorithm with forgetting mechanism (KOS-ELMF) to deal with the time-sensitive applications. A sliding window is applied to limit the active training data in order to ’forget’ the old observations. The size of the sliding window can change based on the forecast error automatically. The automatic determination of model parameters can avoid human interference and save training time. Empirical study of KOS-ELMF on several benchmark applications shows that the proposed approach achieves more satisfied and robust performance, compared with other ELM-related algorithms.