A Fast and Numerically Robust Neural Network Training Algorithm
A fast and numerically robust algorithm for training feedforward neural networks (FNNs) using a recursive prediction error (RPE) method, along with an adaptively-adjustable time-varying forgetting factor technique, is presented first. Then a U-D factorization-based RPE (UD-RPE) algorithm is proposed to further improve the training rate and accuracy of the FNNs. In comparison with the backpropagation (BP) and existing RPE based training algorithms, the proposed UD-RPE can provide substantially better results in fewer iterations with fewer hidden nodes and improve significantly convergence rate and numerical stability. In addition, it is less sensitive to start-up parameters, such as initial weights and covariance matrix. It has also good model prediction ability and need less training time. The effectiveness of the proposed algorithm is demonstrated via two nonlinear systems modeling and identification examples.
KeywordsRoot Mean Square Error Hide Node Training Algorithm Multiple Input Multiple Output Multiple Input Multiple Output
Unable to display preview. Download preview PDF.
- 8.Singhal, S., Wu, L.: Training feed-forward networks with the extended Kalman algorithm. In: Proc. Int. Conf. on ASSP, pp. 1187–1190 (1989)Google Scholar