Journal of Intelligent Information Systems

, Volume 25, Issue 3, pp 275–291

A Fixed-Distribution PAC Learning Theory for Neural FIR Models

Article

Abstract

The PAC learning theory creates a framework to assess the learning properties of static models. This theory has been extended to include learning of modeling tasks with m-dependent data given that the data are distributed according to a uniform distribution. The extended theory can be applied for learning of nonlinear FIR models with the restriction that the data are unformly distributed.

In this paper, The PAC learning scheme is extended to deal with any FIR model regardless of the distribution of the data. This fixed-distribution m-dependent extension of the PAC learning theory is then applied to the learning of FIR three-layer feedforward sigmoid neural networks.

PAC learning nonlinear FIR model multi-layer feedforward neural networks m-dependency 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bartlett, P.(1996). The Sample Complexity of Pattern Classification with Neural Networks: The Size of the Weights is More Important Than the Size of the Network. Amer. Statistical Assoc. Math. Soc. Transactions, 17, 277–364.Google Scholar
  2. Barron, A.R. (1994). Approximation and Estimation Bounds for Artificial Neural Networks. Machine Learning, 14, 115–133.MATHGoogle Scholar
  3. Barron, A.R. (1994). Approximation and Estimation Bounds for Artificial Neural Networks. Machine Learning, 14, 115–133.MATHGoogle Scholar
  4. Benedek, G.M. and Itai, A. (1991a). Dominating Distributions and Learnability. In Proc. Fifth Workshop on Computational Learning Theory, ACM (pp. 253–264).Google Scholar
  5. Benedek, G.M. and Itai, A. (1991a). Dominating Distributions and Learnability. In Proc. Fifth Workshop on Computational Learning Theory, ACM (pp. 253–264).Google Scholar
  6. Benedek, G.M. and Itai, A. (1991b). Learnability with Respect to Fixed Distributions. Theoretical Computer Sys., 86(2), 377–390.MathSciNetGoogle Scholar
  7. Benedek, G.M. and Itai, A. (1991b). Learnability with Respect to Fixed Distributions. Theoretical Computer Sys., 86(2), 377–390.MathSciNetGoogle Scholar
  8. Campi, M.C. and Kumar P.R. (1996). Learning Dynamical Systems in a Stationary Environment. Proc. 31th IEEE Conf. Decision and Control, 16(2), 2308–2311.Google Scholar
  9. Campi, M.C. and Kumar P.R. (1996). Learning Dynamical Systems in a Stationary Environment. Proc. 31th IEEE Conf. Decision and Control, 16(2), 2308–2311.Google Scholar
  10. Hoeffding, W. (1961). Probability Inequalities for Sum of Bounded Random Variables. Amer. Statistical Assoc. Math. Soc. Transactions, 17, 277–364.Google Scholar
  11. Hoeffding, W. (1961). Probability Inequalities for Sum of Bounded Random Variables. Amer. Statistical Assoc. Math. Soc. Transactions, 17, 277–364.Google Scholar
  12. Iosifesco, M. and Theodorescu, R. (1969). Random Processes and Learning. Springer-Verlog.Google Scholar
  13. Iosifesco, M. and Theodorescu, R. (1969). Random Processes and Learning. Springer-Verlog.Google Scholar
  14. Najarian, K. (2001a). On Learning of Sigmoid Neural Networks. Complexity, 6(4), 39–45.CrossRefMathSciNetGoogle Scholar
  15. Najarian, K. (2001a). On Learning of Sigmoid Neural Networks. Complexity, 6(4), 39–45.CrossRefMathSciNetGoogle Scholar
  16. Najarian, K. (2001b). On Learning and Computational Complexity of FIR Radial Basis Function Networks Part I: Learning. In Proceedings of The 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'2001).Google Scholar
  17. Najarian, K. (2001c). On Learning and Computational Complexity of FIR Radial Basis Function Networks Part II: Complexity Measures. In Proceedings of The 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'2001). Google Scholar
  18. Najarian, K. (2002a). On Stochastic Stability of Dynamic Neural Networks. Discrete and Continuous Dynamical Systems, Sup., 656–663.Google Scholar
  19. Najarian, K. (2002b). Learning of FIR and ARX Neural Networks With Empirical Risk Minimization Algorithm. In Proceedings of the 15th IFAC World Conference on Automatic Control (IFAC2002), Barcelona, Spain.Google Scholar
  20. Najarian, K. (2002c). Learning-Based Complexity Evaluation of Radial Basis Function Networks. Neural Processing Letters, 16(2), 137–150.CrossRefGoogle Scholar
  21. Najarian, K. (2002d).FIR Volterra Kernel Neural Models and PAC Learning. Complexity, 7(6), 48–55.CrossRefMathSciNetGoogle Scholar
  22. Najarian, K. (2002d).FIR Volterra Kernel Neural Models and PAC Learning. Complexity, 7(6), 48–55.CrossRefMathSciNetGoogle Scholar
  23. Najarian, K., Dumont, G.A., and Davies, M.S. (1999a). A Learning-Theory-Based Training Algorithm for Variable-Structure Dynamic Neural Modeling. In Proc. Inter. Joint Conf. Neural Networks (IJCNN99). Google Scholar
  24. Najarian, K., Dumont, G.A., and Davis, M.S. (2001a). Modeling of Neuromuscular Blockade System Using Neural Networks. In Proc. of IEEE-EMBS'2001.Google Scholar
  25. Najarian, K., Dumont, G.A., and Davies, M.S. (2001b). PAC Learning in Nonlinear FIR Models. International Journal of Adaptive Control and Signal Processing, 15(1), 37–52.CrossRefGoogle Scholar
  26. Najarian, K., Dumont, G.A., and Davies, M.S. (2001b). PAC Learning in Nonlinear FIR Models. International Journal of Adaptive Control and Signal Processing, 15(1), 37–52.CrossRefGoogle Scholar
  27. Najarian, K., Dumont, G.A., Davies, M.S., and Heckman, N.E. (1999b) Learning of FIR Models Under Uniform Distribution. In Proc. of The American Control Conference, San Dieo, U.S.A. (ACC1999) (pp. 864–869).Google Scholar
  28. Najarian, K., Dumont, G.A., Davies, M.S., and Heckman, N.E. (1999b) Learning of FIR Models Under Uniform Distribution. In Proc. of The American Control Conference, San Dieo, U.S.A. (ACC1999) (pp. 864–869).Google Scholar
  29. Valiant, L.G. (1984). A Theory of Learnable. Comm. ACM, (pp. 1134–1142).Google Scholar
  30. Valiant, L.G. (1984). A Theory of Learnable. Comm. ACM, (pp. 1134–1142).Google Scholar
  31. Vapnik, V.N. (1996). Statistical Learning Theory. New York: Wiley.Google Scholar
  32. Vidyasagar, M. (1997). A Theory of Learning and Generalization. Springer.Google Scholar
  33. Vidyasagar, M. (1997). A Theory of Learning and Generalization. Springer.Google Scholar
  34. Warren, D.S. and Najarian, K. (2002). Learning Theory Applied to Sigmoid Network Classification of Protein Biological Function Using Primary Structure. Discrete and Continuous Dynamical Systems, Sup., 898–904.Google Scholar
  35. Weyer, E., Williamson, R.C., and Mareels, I.M.Y. (1996). Sample Complexity of Least Squares Identification of FIR Models. 13th IFAC Triennial World Conggrss (pp. 239–243).Google Scholar

Copyright information

© Springer Science+Business Media, Inc. 2005

Authors and Affiliations

  1. 1.Computer Science DepartmentUniversity of North Carolina at CharlotteCharlotteUSA

Personalised recommendations