A Regularized Minimum Cross-Entropy Algorithm on Mixtures of Experts for Time Series Prediction
The well-known mixtures of experts(ME) model is usually trained by expectation maximization(EM) algorithm for maximum likelihood learning. However, we have to first determine the number of experts, which is often hardly known. Derived from regularization theory, a regularized minimum cross-entropy(RMCE) algorithm is proposed to train ME model, which can automatically make model selection. When time series is modeled by ME, it is demonstrated by some climate prediction experiments that RMCE algorithm outperforms EM algorithm. We also compare RMCE algorithm with other regression methods such as back-propagation(BP) algorithm and normalized radial basis function(NRBF) network, and find that RMCE algorithm still shows promising results.
KeywordsExpectation Maximization Expectation Maximization Algorithm Normalize Mean Square Error Time Series Prediction Expert Network
Unable to display preview. Download preview PDF.