Abstract
We consider two issues in polynomial-time exact learning of concepts using membership and equivalence queries: (1) errors or omissions in answers to membership queries, and (2) learning finite variants of concepts drawn from a learnable class.
To study (1), we introduce two new kinds of membership queries: limited membership queries and malicious membership queries. Each is allowed to give incorrect responses on a maliciously chosen set of strings in the domain. Instead of answering correctly about a string, a limited membership query may give a special “I don't know” answer, while a malicious membership query may give the wrong answer. A new parameter Lis used to bound the length of an encoding of the set of strings that receive such incorrect answers. Equivalence queries are answered correctly, and learning algorithms are allowed time polynomial in the usual parameters and L. Any class of concepts learnable in polynomial time using equivalence and malicious membership queries is learnable in polynomial time using equivalence and limited membership queries; the converse is an open problem. For the classes of monotone monomials and monotone k-term DNF formulas, we present polynomial-time learning algorithms using limited membership queries alone. We present polynomial-time learning algorithms for the class of monotone DNF formulas using equivalence and limited membership queries, and using equivalence and malicious membership queries.
To study (2), we consider classes of concepts that are polynomially closed under finite exceptions and a natural operation to add exception tables to a class of concepts. Applying this operation, we obtain the class of monotone DNF formulas with finite exceptions. We give a polynomial-time algorithm to learn the class of monotone DNF formulas with finite exceptions using equivalence and membership queries. We also give a general transformation showing that any class of concepts that is polynomially closed under finite exceptions and is learnable in polynomial time using standard membership and equivalence queries is also polynomial-time learnable using malicious membership and equivalence queries. Corollaries include the polynomial-time learnability of the following classes using malicious membership and equivalence queries: deterministic finite acceptors, boolean decision trees, and monotone DNF formulas with finite exceptions.
Article PDF
Similar content being viewed by others
References
Anderson, J. R. (1980). Cognitive Psychology and Its Implications. W. H. Freeman and Company.
Angluin, D. (1987). Learning regular sets from queries and counter examples. Inform. Comput., 75(2):87–106.
Angluin, D. (1988). Queries and concept learning. Machine Learning, 2(4): 319–342.
Angluin, D. (1994). Exact learning of μ-DNF formulas with malicious membership queries. Technical Report YALEU/DCS/TR-1020, Yale University Department of Computer Science.
Angluin, D., & Kriķis, M. (1994a). Malicious membership queries and exceptions. Technical Report YALEU/-DCS/TR-1019, Yale University Department of Computer Science.
Angluin, D., & Kriķis, M. (1994b). Learning with malicious membership queries and exceptions. In Proc. 7th Annu. ACM Workshop on Comput. Learning Theory, pages 57–66. ACM Press, New York, NY.
Angluin, D., & Slonim, D. K. (1994). Randomly fallible teachers: learning monotone DNF with an incomplete membership oracle. Machine Learning, 14(1): 7–26.
Auer, P., & Long, P. M. (1994). Simulating access to hidden information while learning. In Proc. of the 26th Annual ACM Symposium on Theory of Computing, pages 263–272. ACM Press, New York, NY.
Baum, E. (1991). Neural net algorithms that learn in polynomial time from examples and queries. IEEE Transactions on Neural Networks, 2: 5–19.
Board, R., & Pitt, L. (1992). On the necessity of Occam algorithms. Theoret. Comput. Sci., 100: 157–184.
Bshouty, N. H., (1993). Exact learning via the monotone theory. In Proceedings of the 34th Annual Symposium on Foundations of Computer Science, pages 302–311. IEEE Computer Society Press, Los Alamitos, CA.
Bultman, W. J. (1991). Topics in the Theory of Machine Learning and Neural Computing. PhD thesis, University of Illinois at Chicago Mathematics Department.
Dean, T., Angluin, D., Basye, K., Engelson, S., Kaelbling, L., Kokkevis, E., & Maron, O. (1995). Learning finite automata with stochastic output functions and an application to map learning. Machine Learning, 18(1): 81–108.
Frazier, M., Goldman, S., Mishra, N., & Pitt, L. (1994). Learning from a consistently ignorant teacher. In Proc. 7th Annu. ACM Workshop on Comput. Learning Theory, pages 328–339. ACM Press, New York, NY.
Goldman, S. A., Kearns, M. J., & Schapire, R. E. (1993). Exact identification of read-once formulas using fixed points of amplification functions. SIAM J. Comput., 22(4): 705–726.
Goldman, S. A., & Mathias, H. D. (1992). Learning k-term DNF formulas with an incomplete membership oracle. In Proc. 5th Annu. Workshop on Comput. Learning Theory, pages 77–84. ACM Press, New York, NY.
Kearns, M. (1993). Efficient noise-tolerant learning from statistical queries. In Proc. 25th Annu. ACM Sympos. Theory Comput., pages 392–401. ACM Press, New York, NY.
Kushilevitz, E., & Mansour, Y. (1993). Learning decision trees using the Fourier spectrum. SIAM J. Comput., 22(6): 1331–1348. Earlier version appeared in STOC 1991.
Lang, K. J., & Baum, E. B. (1992). Query learning can work poorly when a human oracle is used. In International Joint Conference on Neural Networks, Beijing.
Maass, W., & Turán, G. (1992). Lower bound methods and separation results for on-line learning models. Machine Learning, 9: 107–145.
Ron, D., & Rubinfeld, R. (1995). Learning fallible deterministic finite automata. Machine Learning, 18(2/3):149–185.
Sakakibara, K. (1991). On learning from queries and counterexamples in the presence of noise. Inform. Proc. Lett., 37(5):279–284.
Sloan, R. H., & Turán G. (1994). Learning with queries but incomplete information. In Proc. 7th Annu. ACM Workshop on Comput. Learning Theory, pages 237–245. ACM Press, New York, NY.
Valiant, L. G. (1985). Learning disjunctions of conjunctions. In Proceedings of the 9th International Joint Conference on Artificial Intelligence, vol. 1, pages 560–566, Los Angeles, California. International Joint Committee for Artificial Intelligence.
Zhuravlev, Y., & Kogan, Y. (1985). Realization of boolean functions with a small number of zeros by disjunctive normal forms, and related problems. Soviet Math. Doklady, 32:771–775.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Angluin, D., Kriķis, M., Sloan, R.H. et al. Malicious Omissions and Errors in Answers to Membership Queries. Machine Learning 28, 211–255 (1997). https://doi.org/10.1023/A:1007311411259
Issue Date:
DOI: https://doi.org/10.1023/A:1007311411259