Skip to main content
SpringerLink
Log in

Towards realistic theories of learning

  • Special Issue
  • Published:
New Generation Computing Aims and scope Submit manuscript

Abstract

In computational learning theory continuous efforts are made to formulate models of machine learning that are more realistic than previously available models. Two of the most popular models that have been recently proposed, Valiant’s PAC learning model and Angluin’s query learning model, can be thought of as refinements of preceding models such as Gold’s classic paradigm of identification in the limit, in which the question ofhow fast the learning can take place is emphasized. A considerable amount of results have been obtained within these two frameworks, resolving the learnability questions of many important classes of functions and languages. These two particular learning models are by no means comprehensive, and many important aspects of learning are not directly addressed in these models. Aiming towards more realistic theories of learning, many new models and extensions of existing learning models that attempt to formalize such aspects have been developed recently. In this paper, we will review some of these new extensions and models in computational learning theory, concentrating in particular on those proposed and studied by researchers at Theory NEC Laboratory RWCP, and their colleagues at other institutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abe, N., “Feasible learnability of Formal Grammars and the Theory of Natural Language Acquisition,” inProceedings of COLING-88, August 1988.

  2. Angluin, D. and Kharitonov, M., “When Wont’ Membership Queries Help?” inProc. of the 23rd Symposium on Theory of Computing, ACM Press, New York, NY, pp. 444–454, 1991.

    Google Scholar 

  3. Abe, N. and Mamitsuka, H., “A New Method for Predicting Protein Secondary Structures Based on Stochastic Tree Grammars,” inProceedings of the Eleventh International Conference on Machine Learning, 1994.

  4. Abe, N. and Takeuchi, J., “The ‘Lob-Pass’ Problem and an On-Line Learning Model of Rational Choice,” inProceedings of the Sixth Annual ACM Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1993.

    Google Scholar 

  5. Abe, N. and Warmuth, M. K., “On the Computational Complexity of Approximating Probability Distributions by Probabilistic Automata,”Machine Learning, 9, 2/3, pp. 205–260, 1992.

    Article  MATH  Google Scholar 

  6. Brown, M., Hughey, R., Krogh, A., Mian, I. S., Sjolander, K., and Haussler, D., “Using Dirichlet Mixture Priors to Derive Hidden Markov Models for Protein Families,” inProceedings of the First International Conference on Intelligent Systems for Molecular Biology, pp. 47–55, 1993.

  7. Goldman, S., Kearns, M., and Schapire, R., “On the Sample Complexity of Weak Learning,” inProceedings of the 1990 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1990.

    Google Scholar 

  8. Haussler, D., “Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications,”Information and Computation,100,1, September 1992.

  9. Herrnstein, R., “Rational Choice Theory,”American Psychologist, 45, 3, pp. 356–367, 1990.

    Article  Google Scholar 

  10. Kearns, M. and Seung, S., “Learning from a Population of Hypotheses,” inProceedings of the Sixth Annual ACM Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1993.

    Google Scholar 

  11. Kearns, M. and Schapire, R., “Efficient Distribution-Free Learning of Probabilistic Concepts,”Journal of Computer and System Sciences,48,3, June 1994. A special issue on 31st IEEE Conference on Foundations of Computer Science.

  12. Laird, P. D., “Efficient Unsupervised Learning,” inProceedings of the 1988 Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, California, August 1988.

    Google Scholar 

  13. Levinson, S. E., Rabiner, L. R., and Sondhi, M. M., “An Introduction to the Application of the Theory of Probabilistic Functions of a Markov Process to Automatic Speech Recognition,”The Bell System Technical Journal,62,4, April 1983.

  14. Nakamura, A. and Abe, N., “Exact Learning of Linear Combinations of Monotone Terms from Function Value Queries,” inTheoretical Computer Science, 137, Elsevier, 1995.

  15. Nakamura, A., Abe, N., and Takeuchi, J., “Efficient Distribution-Free Population Learning of Simple Concepts,” inProceedings of the Fifth International Workshop on Algorithmic Learning Theory, Springer-Verlag, October 1994.

  16. Osherson, D., Stob, M., and Weinstein, S.,Systems that Learn: An Introduction for Cognitive and Computer Scientists, MIT Press, 1986.

  17. Paz, A.,Introduction to Probabilistic Automata, Academic Press, 1971.

  18. Pollard, D.,Convergence of Stochastic Processes, Springer-Verlag, 1984.

  19. Rissanen, J., “Stochastic Complexity and Modeling,”The Annals of Statistics, 14, 3, pp. 1080–1100, 1986.

    Article  MATH  MathSciNet  Google Scholar 

  20. Rost, B and Sander, C., “Prediction of Protein Secondary Structure at Better Than 70% Accuracy,”J. Mol. Biol., 232, pp. 584–599, 1993.

    Article  Google Scholar 

  21. Sakakibara, Y., Brown, M., Underwood, R. C., Mian, I. S., and Haussler, D., “Stochastic Context-Free Grammars for Modeling RNA,” inProceedings of the 27th Hawaii International Conference on System Sciences, volume V, pp. 284–293, 1994.

  22. Schabes, Y., “Stochastic Lexicalized Tree Adjoining Grammars,” inProceedings of COLING-92, pp. 426–432, 1992.

  23. Sloan, R. H., “Computational Learning Theory: New Models and Algorithms,”Ph.D thesis, MIT, 1989. Issued as MIT/LCS/TR-448.

  24. Sander, C. and Schneider, R., “Database of Homology-Derived Structures and the Structural Meaning of Sequence Alignment,”Proteins: Struct. Funct. Genet., 9, pp. 56–68, 1991.

    Article  Google Scholar 

  25. Vitter, J. and Lin, J., “Learning in Parallel,”Information Computing, pp. 179–202, 1992.

  26. Vijay-Shanker, K. and Joshi, A. K., “Some Computational Properties of Tree Adjoining Grammars,” in23rd Meeting of A. C. L., 1985.

  27. Yamanishi, K., “A Learning Criterion for Stochastic Rules,”Machine Learning, 9, 2/3, a special issue for COLT’90, 1992.

  28. Yamanishi, K. and Konagaya, A., “Learning Stochastic Motifs from Genetic Sequences,” inthe Eighth International Workshop on Machine Learning, 1991.

Download references

Author information

Authors and Affiliations

  1. Theory NEC Laboratory, RWCP c/o C&C Research Laboratories, NEC Corporation, 4-1-1 Miyazaki, Miyamae-ku, 216, Kawasaki, Japan

    Naoki Abe

Authors
  1. Naoki Abe
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Real World Computing Partnership

Naoki Abe: He received the B. S. and M. S. degrees in Computer Science from Massachusetts Institute of Technology in 1984, and obtained the Ph. D. degree in Computer and Information Sciences from the University of Pennsylvania in 1989. He was a post doctoral researcher at the University of California, Santa Cruz from 1989 to 1990, and conducted research in computational learning theory. He joined NEC Corporation in 1990, where he is currently a principal researcher in the C & C Research Laboratories. His research interests include computational learning theory and its application to genetic information processing and natural language processing.

About this article

Cite this article

Abe, N. Towards realistic theories of learning. New Gener Comput 15, 3–25 (1997). https://doi.org/10.1007/BF03037558

Download citation

  • Received:

  • Revised:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03037558

Keywords

Search

Navigation