# Learning Conjunctive Concepts in Structural Domains

- 557 Downloads
- 1 Citations

## Abstract

We study the problem of learning conjunctive concepts from examples on structural domains like the blocks world. This class of concepts is formally defined, and it is shown that even for samples in which each example (positive or negative) is a two-object scene, it is NP-complete to determine if there is any concept in this class that is consistent with the sample. We demonstrate how this result affects the feasibility of Mitchell's version of space approach and how it shows that it is unlikely that this class of concepts is polynomially learnable from random examples alone in the PAC framework of Valiant. On the other hand, we show that for any fixed bound on the number of objects per scene, this class is polynomially learnable if, in addition to providing random examples, we allow the learning algorithm to make subset queries. In establishing this result, we calculate the capacity of the hypothesis space of conjunctive concepts in a structural domain and use a general theorem of Vapnik and Chervonenkis. This latter result can also be used to estimate a sample size sufficient for heuristic learning techniques that do not use queries.

## References

- Angluin, D. (1988). Queries and concept learning.
*Machine Learning, 2,*319-342.Google Scholar - Baum, E. & Haussler, D. (1989). What size net gives valid generalization?
*Neural Computation, 1,*151-160.Google Scholar - Buntine, W.
*Induction of Horn clauses:Methods and the plausible generalization algorithm*. (Technical Report). New South Wales, Australia: New South Wales Institute of Technology, Department of Computer Science.Google Scholar - Blumer, A., Ehrenfeucht, A., Haussler, D. & Warmuth, M. (1989). Learnability and the Vapnik-Chervonenkis dimension.
*J*.*ACM, 36*.Google Scholar - Boriga, A., Mitchell, T. & Williamson, K. Learning improved integrity constraints and schemas from exceptions in data and knowledge bases. In M. Brodie and J. Mylopoulos, (Eds. ),
*On knowledge base management systems,*New York: Springer-Verlag.Google Scholar - Bundy, A., Silver, B. & Plummer, D. (1985). An analytical comparison of some rule-learning programs.
*Artificial Intelligence, 27,*137-181.Google Scholar - Cohen, P. & Feigenbaum, E. (1982).
*Handbook of Artificial Intelligence*(Vol. 3). William Kaufmann.Google Scholar - Dietterich, T. G. & Michalski, R. S. (1983). A comparitive review of selected methods for learning from examples. In
*Machine learning:An artificial intelligence approach*. Palo Alto, CA: Tioga Press.Google Scholar - Duda, R. & Hart, P. (1973).
*Pattern classification and scene analysis*. John P. Wiley & Sons.Google Scholar - Ehrenfeucht, A., Haussler, D., Kearns, M. & Valiant, L. (1989). A general lower bound on the number of examples needed for learning.
*Information and Computation, 82*.Google Scholar - Garey, M. & Johnson, D. (1979).
*Computers and intractability:A guide to the theory of NP-completeness*. San Francisco, CA: W. H. Freeman.Google Scholar - Gill, J. (1977). Probabilistic Turing machines.
*SIAM J Comput, 6,*675-695.Google Scholar - Haussler, D. (1987).
*Learning conjunctive concepts in structural domains*. (Technical Report UCSC-CRL-87-01). Santa Cruz, CA: University of California.Google Scholar - Haussler, D. (1988). Quantifying inductive bias:AI learning algorithms and Valiant's learning frame-work.
*Artificial Intelligence, 36,*177-221.Google Scholar - Haussler, D., Kearns, M., Littlestone, N. & Warmuth, M. (1988).
*Equivalence of models for polynomial leamability*. (Technical Report UCSC-CRL-88-06). Santa Cruz, CA: University of California.Google Scholar - Haussler, D. & Welzl, E. (1987). Epsilon nets and simplex range queries.
*Discrete and Comp*.*Geometry, 2,*127-151.Google Scholar - Hayes-Roth, F. & McDermott, J. (1978). An interference matching technique for inducing abstractions.
*CACM, 21,*401-410.Google Scholar - Kearns, M., Li, M., Pitt, L. & Valiant, L. (1987). Recent results in Boolean concept learning.
*Pro-ceedings of the Fourth International Workshop on Machine Learning*(pp. 337-352). Irvine, CA.Google Scholar - Kodratoff, Y. & Ganascia, J. (1986). Improving the generalization step in learning. In R. Michalski, J. Carbonell & T. Mitchell (Eds. ),
*Machine learning 11*. Los Altos, CA: Morgan Kaufmann.Google Scholar - Knapman, J. (1978). A critical review of Winston's learning structural descriptions from examples.
*AISB Quarterly, 31,*319-320.Google Scholar - Michalski, R. S. (1983). A theory and methodology of inductive learning. In
*Machine learning:An artificial intelligence approach*. Palo Alto, CA: Tioga Press.Google Scholar - Mitchell, T. M. (1980).
*The need for biases in learning generalizations*. (Technical Report CBM-TR-117). New Brunswick, NJ: Rutgers University, Department of Computer Science.Google Scholar - Mitchell, T. M. (1982). Generalization as search.
*Artificial Intelligence, 18,*203-226.Google Scholar - Mitchell, T. M., Keller, R. M. & Kedar-Cabelli, S. T. (1988). Explanation-based generalization:A unify-ing view.
*Machine Learning, 1,*47-80.Google Scholar - Muggleton, S. & Buntine, W. (1988). Machine invention of first-order predicates by inverting resolution.
*Proceedings of the Fifth International Conference on Machine Learning*(pp. 339-352). Ann Arbor, MI.Google Scholar - Natarajan, B. K. (1989). On learning sets and functions.
*Machine Learning, 4,*67-97.Google Scholar - Pearl, J. (1978). On the connection between the complexity and credibility of inferred models.
*Int*.*J*.*Gen*.*Sys*.,*4,*255-264..Google Scholar - Pitt, L. & Valiant, L. G. (1988). Computational limitations on learning from examples.
*Journal of the ACM, 35,*965-984.Google Scholar - Sammut, C. & Banerji, R. (1986). Learning concepts by asking questions. In R. Michalski, J. Carbonell & T. Mitchell, (Eds. ),
*Machine learning II*. Los Altos, CA: Morgan Kaufmann.Google Scholar - Stepp, R. (1987). Machine learning from structured objects.
*Proceedings of the Fourth International Workshop on Machine Learning*(pp. 353-363). Irvine, CA.Google Scholar - Subramanian, D. & Feigenbaum, J. (1986). Factorization in experiment generation.
*Proceedings of the AAAl-86*(pp. 518-522). Philadelphia, PA.Google Scholar - Utgoff, P. (1986). Shift of bias for inductive concept learning. In R. Michalski, J. Carbonell & T. Mitchell, (Eds. ),
*Machine learning II,*Los Altos, CA: Morgan Kaufmann.Google Scholar - Valiant, L. G. (1984). A theory of the learnable.
*CACM, 27,*1134-1142.Google Scholar - Valiant, L. G. (1985). Learning disjunctions of conjunctions.
*Proceedings of the Ninth International Joint Conference on Artificial Intelligence*(pp. 560-566). Los Angeles, CA.Google Scholar - Vapnik, V. N. (1982).
*Estimation of dependences based on empirical data*. New York: Springer-Verlag.Google Scholar - Vapnik, V. N. & Chervonenkis, A. (1971). On the uniform convergence of relative frequencies of events to their probabilities.
*Th*.*Prob*.*and its Appl*.,*16,*264-280.Google Scholar - Vere, S. A. (1975). Induction of concepts in the predicate calculus.
*Proceedings of the Fourth Inter-national Joint Conference on Artificial Intelligence*(pp. 281-287). Tbilisi, USSR.Google Scholar - Winston, P. (1975). Learning structural descriptions from examples. In P. H. Winston, (Ed. ),
*The psychology of computer vision*. New York: McGraw-Hill.Google Scholar - Winston, P. (1984).
*Artificial intelligence*. Addison-Wesley.Google Scholar