Algorithmica

, Volume 22, Issue 1, pp 53–75

PAC Learning Intersections of Halfspaces with Membership Queries

  • S. Kwek
  • L. Pitt

DOI: 10.1007/PL00013834

Cite this article as:
Kwek, S. & Pitt, L. Algorithmica (1998) 22: 53. doi:10.1007/PL00013834

Abstract.

A randomized learning algorithm {POLLY} is presented that efficiently learns intersections of s halfspaces in n dimensions, in time polynomial in both s and n . The learning protocol is the PAC (probably approximately correct) model of Valiant, augmented with membership queries. In particular, {POLLY} receives a set S of m = poly(n,s,1/ε,1/δ) randomly generated points from an arbitrary distribution over the unit hypercube, and is told exactly which points are contained in, and which points are not contained in, the convex polyhedron P defined by the halfspaces. {POLLY} may also obtain the same information about points of its own choosing. It is shown that after poly(n , s , 1/ε , 1/δ , log(1/d) ) time, the probability that {POLLY} fails to output a collection of s halfspaces with classification error at most ε , is at most δ . Here, d is the minimum distance between the boundary of the target and those examples in S that are not lying on the boundary. The parameter log(1/d) can be bounded by the number of bits needed to encode the coefficients of the bounding hyperplanes and the coordinates of the sampled examples S . Moreover, {POLLY} can be extended to learn unions of k disjoint polyhedra with each polyhedron having at most s facets, in time poly(n , k , s , 1/ε , 1/δ , log(1/d) , 1/γ ) where γ is the minimum distance between any two distinct polyhedra.

Key words. PAC learning, Membership queries, Intersections of halfspaces, Unions of polyhedra, Occam algorithm.

Copyright information

© 1998 Springer-Verlag New York Inc.

Authors and Affiliations

  • S. Kwek
    • 1
  • L. Pitt
    • 2
  1. 1.Department of Computer Science, Washington University, St. Louis, MO 63130, USA. kwek@cs.wustl.edu.US
  2. 2.Computer Science Department, University of Illinois, Urbana, IL 61801, USA. pitt@cs.uiuc.edu.US