Advertisement

Competitive Associative Nets and Cross-Validation for Estimating Predictive Uncertainty on Regression Problems

  • Shuichi Kurogi
  • Miho Sawa
  • Shinya Tanaka
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3944)

Abstract

This article describes the competitive associative net called CAN2 and cross-validation which we have used for making prediction and estimating predictive uncertainty on the regression problems at the Evaluating Predictive Uncertainty Challenge. The CAN2 with an efficient batch learning method for reducing empirical (training) error is combined with cross-validation for making prediction (generalization) error small and estimating predictive distribution accurately. From an analogy of Bayesian learning, a stochastic analysis is derived to indicate a validity of our method.

Keywords

Training Dataset Regression Problem Predictive Distribution Multivariate Adaptive Regression Spline Piecewise Linear Approximation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
  2. 2.
    Kohonen, T.: Associative Memory. Springer, Heidelberg (1977)CrossRefMATHGoogle Scholar
  3. 3.
    Rumelhart, D.E., Zipser, D.: A feature discovery by competitive learning. In: Rumelhart, D.E., McClelland, J.L., The PDP Research Group (eds.) Parallel Distributed Processing, vol. 1, pp. 151–193. The MIT Press, Cambridge (1986)Google Scholar
  4. 4.
    Mueller, P., Insua, D.R.: Issues in Bayesian analysis of neural network models. Neural Computation 10, 571–592 (1995)Google Scholar
  5. 5.
    Bernardo, J.M., Smith, A.D.M.: Bayesian Theory. John Wiley, New York (1994)CrossRefMATHGoogle Scholar
  6. 6.
  7. 7.
    Kurogi, S., Tou, M., Terada, S.: Rainfall estimation using competitive associative net. In: Proc. of 2001 IEICE General Conference, vol. SD-1, pp. 260–261 (2001) (in Japanese)Google Scholar
  8. 8.
    Kurogi, S.: Asymptotic optimality of competitive associative nets for their learning in function approximation. In: Proc. of the 9th International Conference on Neural Information Processing, vol. 1, pp. 507–511 (2002)Google Scholar
  9. 9.
    Kurogi, S.: Asymptotic optimality of competitive associative nets and its application to incremental learning of nonlinear functions. Trans. of IEICE D-II J86-D-II(2), 184–194 (2003) (in Japanese)Google Scholar
  10. 10.
    Kurogi, S., Ueno, T., Sawa, M.: A batch learning method for competitive associative net and its application to function approximation. In: Proc. of SCI 2004, vol. V, pp. 24–28 (2004)Google Scholar
  11. 11.
    Kurogi, S., Ueno, T., Sawa, M.: Batch learning competitive associative net and its application to time series prediction. In: Proc. of IJCNN 2004, International Joint Conference on Neural Networks, Budapest, Hungary, July 25-29 (2004), CD-ROMGoogle Scholar
  12. 12.
    Efron, B.: Estimating the error rate of a prediction rule: improvement on crossvalidation. Journal of the American Statistical Association 78(382), 316–331 (1983)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proc. of the Fourteenth International Conference 18 on Artificial Intelligence (IJCAI), pp. 1137–1143. Morgan Kaufmann, San Mateo (1995)Google Scholar
  14. 14.
    Efron, B., Tbshirani, R.: Improvements on cross-validation: the.632+ bootstrap method. Journal of the American Statistical Association 92, 548–560 (1997)MathSciNetMATHGoogle Scholar
  15. 15.
    Elisseeff, A., Pontil, M.: Leave-one-out error and stability of learning algorithms with applications. In: Advances in Learning Theory: Methods, Models and Applications. NATO Advanced Study Institute on Learning Theory and Practice, pp. 111–130 (2002)Google Scholar
  16. 16.
    Farmer, J.D., Sidorowich, J.J.: Predicting chaotic time series. Phys. Rev.Lett. 59, 845–848 (1987)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Friedman, J.H.: Multivariate adaptive regression splines. Ann. Stat. 19, 1–50 (1991)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Jordan, M.I., Jacobs, R.A.: Hierarchical mixtures of experts and the EM algorithm. Neural Computation 6, 181–214 (1994)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Shuichi Kurogi
    • 1
  • Miho Sawa
    • 1
  • Shinya Tanaka
    • 1
  1. 1.Kyushu Institute of TechnologyKitakyushu FukuokaJapan

Personalised recommendations