Exact Incremental Learning for a Single Non-linear Neuron Based on Taylor Expansion and Greville Formula

Abstract

Traditional machine learning algorithms are focused on batch learning from a static data set or from a well-known distribution. However, these algorithms take a considerable amount of time to learn a large amount of training data and besides many of them are not able to deal with nonstationary distributions. Recent machine learning challenges require the capability of online learning in nonstationary environments. Thus, in this work we propose a new learning method, for single-layer neural networks, that introduces a forgetting function in an incremental learning algorithm. The algorithm employs a recursive formula in order to obtain the solution of a weighted least squares problem. The performance of the method is experimentally checked over different data sets. The proposed algorithm has demonstrated high adaptation to changes while maintaining a low consumption of computational resources.