Competitive Associative Nets and Cross-Validation for Estimating Predictive Uncertainty on Regression Problems
This article describes the competitive associative net called CAN2 and cross-validation which we have used for making prediction and estimating predictive uncertainty on the regression problems at the Evaluating Predictive Uncertainty Challenge. The CAN2 with an efficient batch learning method for reducing empirical (training) error is combined with cross-validation for making prediction (generalization) error small and estimating predictive distribution accurately. From an analogy of Bayesian learning, a stochastic analysis is derived to indicate a validity of our method.
KeywordsTraining Dataset Regression Problem Predictive Distribution Multivariate Adaptive Regression Spline Piecewise Linear Approximation
Unable to display preview. Download preview PDF.
- 3.Rumelhart, D.E., Zipser, D.: A feature discovery by competitive learning. In: Rumelhart, D.E., McClelland, J.L., The PDP Research Group (eds.) Parallel Distributed Processing, vol. 1, pp. 151–193. The MIT Press, Cambridge (1986)Google Scholar
- 4.Mueller, P., Insua, D.R.: Issues in Bayesian analysis of neural network models. Neural Computation 10, 571–592 (1995)Google Scholar
- 7.Kurogi, S., Tou, M., Terada, S.: Rainfall estimation using competitive associative net. In: Proc. of 2001 IEICE General Conference, vol. SD-1, pp. 260–261 (2001) (in Japanese)Google Scholar
- 8.Kurogi, S.: Asymptotic optimality of competitive associative nets for their learning in function approximation. In: Proc. of the 9th International Conference on Neural Information Processing, vol. 1, pp. 507–511 (2002)Google Scholar
- 9.Kurogi, S.: Asymptotic optimality of competitive associative nets and its application to incremental learning of nonlinear functions. Trans. of IEICE D-II J86-D-II(2), 184–194 (2003) (in Japanese)Google Scholar
- 10.Kurogi, S., Ueno, T., Sawa, M.: A batch learning method for competitive associative net and its application to function approximation. In: Proc. of SCI 2004, vol. V, pp. 24–28 (2004)Google Scholar
- 11.Kurogi, S., Ueno, T., Sawa, M.: Batch learning competitive associative net and its application to time series prediction. In: Proc. of IJCNN 2004, International Joint Conference on Neural Networks, Budapest, Hungary, July 25-29 (2004), CD-ROMGoogle Scholar
- 13.Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proc. of the Fourteenth International Conference 18 on Artificial Intelligence (IJCAI), pp. 1137–1143. Morgan Kaufmann, San Mateo (1995)Google Scholar
- 15.Elisseeff, A., Pontil, M.: Leave-one-out error and stability of learning algorithms with applications. In: Advances in Learning Theory: Methods, Models and Applications. NATO Advanced Study Institute on Learning Theory and Practice, pp. 111–130 (2002)Google Scholar