# Learning sparse linear combinations of basis functions over a finite domain

## Abstract

We study the problem of identifying a *GF* (2)-valued ({0,1}-valued) or real-valued function over a finite domain by querying the values of the target function at points of the learner's choice. We analyze the (*function value query*) *learning complexity* of subclasses of such functions, namely, the number of queries needed to identify an arbitrary function in a given subclass. Since the whole class of *GF* (2)-valued functions, and the class of real-valued functions, over a domain with cardinality *N* is each an *N*-dimensional vector space, an arbitrary function in each of these classes can be written as a linear combination of *N* functions from an arbitrary basis. *The size of function f with respect to basis B* is defined to be the number of non-zero coefficients in the unique representation of *f* as a linear combination of functions in *B*. We show upper and lower bounds on the learning complexity of the size-*k* subclasses for various basis, including the basis that enjoys the minimum learning complexity (among all bases). We also consider subclasses of real-valued functions representable by a linear combination of basis functions *with non-negative coefficients*. We analyze the learning complexity of such subclasses for two bases, and show that neither of them attains the lower bound we obtained for the basis with the minimum learning complexity.

## Preview

Unable to display preview. Download preview PDF.

## References

- [HW]L. Hellerstein and M. Warmuth, “Interpolating GF[2] polynomials.” Unpublished manuscript.Google Scholar
- [MS77]F. J. MacWilliams and N. J. A. Sloane.
*The Theory of Error-Correcting Codes*,*Noth-Holland*,*Amsterdam*, 1977.Google Scholar - [NA93]A. Nakamura and N. Abe, “Exact Learning of Linear Combination of Monotone Terms from Function Value Queries.” Proceedings of 4th International Workshop, ALT'93 (
*Lecture Notes in Artificial intelligence*744), pp.300–313, 1993.Google Scholar - [NA95]A. Nakamura and N. Abe, “Exact Learning of Linear Combination of Monotone Terms from Function Value Queries.”
*Theoretical Computer Science*137, pp.159–176, 1995. (Journal version of [NA93])Google Scholar - [RB91]R. M. Roth and G. M. Benedek. “Interpolation and approximation of sparse multivariate polynomials over GF[2].”
*SIAM J. Comput.*20(2), pp.291–314, 1991.Google Scholar