Algebraic Analysis for Singular Statistical Estimation

  • Sumio Watanabe
Conference paper

DOI: 10.1007/3-540-46769-6_4

Part of the Lecture Notes in Computer Science book series (LNCS, volume 1720)
Cite this paper as:
Watanabe S. (1999) Algebraic Analysis for Singular Statistical Estimation. In: Watanabe O., Yokomori T. (eds) Algorithmic Learning Theory. ALT 1999. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence), vol 1720. Springer, Berlin, Heidelberg

Abstract

This paper clarifies learning efficiency of a non-regular parametric model such as a neural network whose true parameter set is an analytic variety with singular points. By using Sato’s b-function we rigorously prove that the free energy or the Bayesian stochastic complexity is asymptotically equal to λ1 log n − (m1 − 1) log log n+constant, where λ1 is a rational number, m1 is a natural number, and n is the number of training samples. Also we show an algorithm to calculate λ1 and m1 based on the resolution of singularity. In regular models, 2λ1 is equal to the number of parameters and m1 = 1, whereas in non-regular models such as neural networks, 2λ1 is smaller than the number of parameters and m1 ≥ 1.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Copyright information

© Springer-Verlag Berlin Heidelberg 1999

Authors and Affiliations

  • Sumio Watanabe
    • 1
  1. 1.P&I Lab.Tokyo Institute of TechnologyYokohamaJapan

Personalised recommendations