New Generation Computing

, Volume 15, Issue 1, pp 3–25 | Cite as

Towards realistic theories of learning

  • Naoki Abe
Special Issue


In computational learning theory continuous efforts are made to formulate models of machine learning that are more realistic than previously available models. Two of the most popular models that have been recently proposed, Valiant’s PAC learning model and Angluin’s query learning model, can be thought of as refinements of preceding models such as Gold’s classic paradigm of identification in the limit, in which the question ofhow fast the learning can take place is emphasized. A considerable amount of results have been obtained within these two frameworks, resolving the learnability questions of many important classes of functions and languages. These two particular learning models are by no means comprehensive, and many important aspects of learning are not directly addressed in these models. Aiming towards more realistic theories of learning, many new models and extensions of existing learning models that attempt to formalize such aspects have been developed recently. In this paper, we will review some of these new extensions and models in computational learning theory, concentrating in particular on those proposed and studied by researchers at Theory NEC Laboratory RWCP, and their colleagues at other institutions.


Computational Learning Theory Machine Learning Genetic Information Processing Probabilistic PAC Learning On-line Learning of Rational Choice Population Learning Query Learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1).
    Abe, N., “Feasible learnability of Formal Grammars and the Theory of Natural Language Acquisition,” inProceedings of COLING-88, August 1988.Google Scholar
  2. 2).
    Angluin, D. and Kharitonov, M., “When Wont’ Membership Queries Help?” inProc. of the 23rd Symposium on Theory of Computing, ACM Press, New York, NY, pp. 444–454, 1991.Google Scholar
  3. 3).
    Abe, N. and Mamitsuka, H., “A New Method for Predicting Protein Secondary Structures Based on Stochastic Tree Grammars,” inProceedings of the Eleventh International Conference on Machine Learning, 1994.Google Scholar
  4. 4).
    Abe, N. and Takeuchi, J., “The ‘Lob-Pass’ Problem and an On-Line Learning Model of Rational Choice,” inProceedings of the Sixth Annual ACM Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1993.Google Scholar
  5. 5).
    Abe, N. and Warmuth, M. K., “On the Computational Complexity of Approximating Probability Distributions by Probabilistic Automata,”Machine Learning, 9, 2/3, pp. 205–260, 1992.zbMATHCrossRefGoogle Scholar
  6. 6).
    Brown, M., Hughey, R., Krogh, A., Mian, I. S., Sjolander, K., and Haussler, D., “Using Dirichlet Mixture Priors to Derive Hidden Markov Models for Protein Families,” inProceedings of the First International Conference on Intelligent Systems for Molecular Biology, pp. 47–55, 1993.Google Scholar
  7. 7).
    Goldman, S., Kearns, M., and Schapire, R., “On the Sample Complexity of Weak Learning,” inProceedings of the 1990 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1990.Google Scholar
  8. 8).
    Haussler, D., “Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications,”Information and Computation,100,1, September 1992.Google Scholar
  9. 9).
    Herrnstein, R., “Rational Choice Theory,”American Psychologist, 45, 3, pp. 356–367, 1990.CrossRefGoogle Scholar
  10. 10).
    Kearns, M. and Seung, S., “Learning from a Population of Hypotheses,” inProceedings of the Sixth Annual ACM Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1993.Google Scholar
  11. 11).
    Kearns, M. and Schapire, R., “Efficient Distribution-Free Learning of Probabilistic Concepts,”Journal of Computer and System Sciences,48,3, June 1994. A special issue on 31st IEEE Conference on Foundations of Computer Science.Google Scholar
  12. 12).
    Laird, P. D., “Efficient Unsupervised Learning,” inProceedings of the 1988 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1988.Google Scholar
  13. 13).
    Levinson, S. E., Rabiner, L. R., and Sondhi, M. M., “An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition,”The Bell System Technical Journal,62,4, April 1983.Google Scholar
  14. 14).
    Nakamura, A. and Abe, N., “Exact Learning of Linear Combinations of Monotone Terms from Function Value Queries,” inTheoretical Computer Science, 137, Elsevier, 1995.Google Scholar
  15. 15).
    Nakamura, A., Abe, N., and Takeuchi, J., “Efficient Distribution-Free Population Learning of Simple Concepts,” inProceedings of the Fifth International Workshop on Algorithmic Learning Theory, Springer-Verlag, October 1994.Google Scholar
  16. 16).
    Osherson, D., Stob, M., and Weinstein, S.,Systems that Learn: An Introduction for Cognitive and Computer Scientists, MIT Press, 1986.Google Scholar
  17. 17).
    Paz, A.,Introduction to Probabilistic Automata, Academic Press, 1971.Google Scholar
  18. 18).
    Pollard, D.,Convergence of Stochastic Processes, Springer-Verlag, 1984.Google Scholar
  19. 19).
    Rissanen, J., “Stochastic Complexity and Modeling,”The Annals of Statistics, 14, 3, pp. 1080–1100, 1986.zbMATHCrossRefMathSciNetGoogle Scholar
  20. 20).
    Rost, B and Sander, C., “Prediction of Protein Secondary Structure at Better Than 70% Accuracy,”J. Mol. Biol., 232, pp. 584–599, 1993.CrossRefGoogle Scholar
  21. 21).
    Sakakibara, Y., Brown, M., Underwood, R. C., Mian, I. S., and Haussler, D., “Stochastic Context-Free Grammars for Modeling RNA,” inProceedings of the 27th Hawaii International Conference on System Sciences, volume V, pp. 284–293, 1994.Google Scholar
  22. 22).
    Schabes, Y., “Stochastic Lexicalized Tree Adjoining Grammars,” inProceedings of COLING-92, pp. 426–432, 1992.Google Scholar
  23. 23).
    Sloan, R. H., “Computational Learning Theory: New Models and Algorithms,”Ph.D thesis, MIT, 1989. Issued as MIT/LCS/TR-448.Google Scholar
  24. 24).
    Sander, C. and Schneider, R., “Database of Homology-Derived Structures and the Structural Meaning of Sequence Alignment,”Proteins: Struct. Funct. Genet., 9, pp. 56–68, 1991.CrossRefGoogle Scholar
  25. 25).
    Vitter, J. and Lin, J., “Learning in Parallel,”Information Computing, pp. 179–202, 1992.Google Scholar
  26. 26).
    Vijay-Shanker, K. and Joshi, A. K., “Some Computational Properties of Tree Adjoining Grammars,” in23rd Meeting of A. C. L., 1985.Google Scholar
  27. 27).
    Yamanishi, K., “A Learning Criterion for Stochastic Rules,”Machine Learning, 9, 2/3, a special issue for COLT’90, 1992.Google Scholar
  28. 28).
    Yamanishi, K. and Konagaya, A., “Learning Stochastic Motifs from Genetic Sequences,” inthe Eighth International Workshop on Machine Learning, 1991.Google Scholar

Copyright information

© Ohmsha, Ltd. and Springer 1997

Authors and Affiliations

  • Naoki Abe
    • 1
  1. 1.Theory NEC Laboratory, RWCP c/o C&C Research LaboratoriesNEC CorporationKawasakiJapan

Personalised recommendations