Abstract
Usually, local search methods are considered to be slow. In our paper, we present a simulated annealing-based local search algorithm for the approximation of Boolean functions with a proven time complexity that behaves relatively fast on randomly generated functions. The functions are represented by disjunctive normal forms (DNFs). Given a set of m uniformly distributed positive and negative examples of length n generated by a target function F(x 1,...,x n), where the DNF consists of conjunctions with at most ℓ literals only, the algorithm computes with high probability a hypothesis H of length m · ℓ such that the error is zero on all examples. Our algorithm can be implemented easily and we obtained a relatively high percentage of correct classifications on test examples that were not presented in the learning phase. For example, for randomly generated F with n = 64 variables and a training set of m = 16384 examples, the error on the same number of test examples was about 19% on positive and 29% on negative examples, respectively. The proven complexity bound provides the basis for further studies on the average case complexity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
E.H.L. Aarts and J.H.M. Korst, Simulated Annealing and Boltzmann Machines: A Stochastic Approach, Wiley & Sons: New York, 1989.
E.H.L. Aarts, Local Search in Combinatorial Optimization, Wiley & Sons: New York, 1998.
H. Aizenstein and L. Pitt, “On the learnability of disjunctive normal form formulas,” Machine Learning, vol. 19, pp. 183–208, 1995.
A. Albrecht, S.K. Cheung, K.S. Leung, and C.K.Wong, “Stochastic simulations of two-dimensional composite packings,” J. of Computational Physics, vol. 136, pp. 559–579, 1997.
D. Angluin, “Queries and concept learning,” Machine Learning, vol. 2, pp. 319–342, 1988.
A. Bachem, W. Hochstättler, B. Steckemetz, and A. Volmer, “Computational experience with general equilibrium problems,” Computational Optimization and Applications, vol. 6, pp. 213–225, 1996.
E.J. Bredensteiner and K.P. Bennett, “Feature minimization within decision trees,” Computational Optimization and Applications, vol. 10, pp. 111–126, 1998.
E.J. Bredensteiner and K.P. Bennett, “Multicategory classification by support vector machines,” Computational Optimization and Applications, vol. 12, pp. 53–79, 1999.
O. Catoni, “Rough large deviation estimates for simulated annealing: Applications to exponential schedules,” The Annals of Probability, vol. 20, pp. 1109–1146, 1992.
O. Catoni, “Metropolis, simulated annealing, and iterated energy transformation algorithms: Theory and experiments,” J. of Complexity, vol. 12, pp. 595–623, 1996.
V. Černy, “A thermodynamical approach to the travelling salesman problem: An efficient simulation algorithm,” J. Optim. Theory Appl., vol. 45, pp. 41–51, 1985.
P. Clark and T. Niblett, “The CN2 induction algorithm,” Machine Learning, vol. 3, pp. 261–283, 1989.
S.D. Flåm, “Learning equilibrium play: A myopic approach,” Computational Optimization and Applications, vol. 14, pp. 87–102, 1999.
B. Hajek, “Cooling schedules for optimal annealing,” Mathem. Oper. Res., vol. 13, pp. 311–329, 1988.
J. Jackson, “An efficient membership-quey algorithm for learning DNF with respect to the uniform distribution,” In Proc. of the 35th Annual Symposium on Foundations of Computer Science, 1994, pp. 42–53.
M. Kearns, M. Li, L. Pitt, and L.G. Valiant, “Recent results on Boolean concept learning,” in Proc. 4th Int. Workshop on Machine Learning, 1987, pp. 337–352.
S. Kirkpatrick, C.D. Gelatt, Jr., and M.P. Vecchi, “Optimization by simulated annealing,” Science, vol. 220, pp. 671–680, 1983.
M.H. Lim, Y. Yuan, and S. Omatu, “Efficient genetic algorithms using simple genes exchange local search policy for the quadratic assignment problem,” Computational Optimization and Applications, vol. 15, pp. 249–268, 2000.
Y. Mansour, “An n O(log log n)learning algorithm for DNF under the uniform distribution,” J. of Computer and Systems Sciences, vol. 50, pp. 543–550, 1995.
R.J. Mooney, “Encouraging experimental results on learning CNF,” Machine Learning, vol. 19, pp. 79–92, 1995.
G. Righini, “Annealing algorithms for multisource absolute location problems on graphs,” Computational Optimization and Applications, vol. 7, pp. 325–337, 1997.
F. Romeo and A. Sangiovanni-Vincentelli, “A theoretical framework for simulated annealing,” Algorithmica, vol. 6, pp. 302–345, 1991.
H. Shvaytser, “Learnable and nonlearnable visual concepts,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, pp. 459–466, 1990.
L.G. Valiant, “A theory of the learnable,” Comm. ACM, vol. 27, pp. 1134–1142, 1984.
K. Verbeurgt, “Learning DNF under the uniform distribution in quasi-polynomial time,” in Proc. of the 3rd Annual Workshop on Computational Learning Theory, 1990, pp. 314–326.
D.F. Wong and C.L. Liu, “Floorplan design of VLSI circuits,” Algorithmica, vol. 4, pp. 263–291, 1989.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Albrecht, A., Wong, CK. Approximation of Boolean Functions by Local Search. Computational Optimization and Applications 27, 53–82 (2004). https://doi.org/10.1023/B:COAP.0000004980.80957.05
Issue Date:
DOI: https://doi.org/10.1023/B:COAP.0000004980.80957.05