Abstract
Many learning situations involve separation of labeled training instances by hyperplanes. Consistent separation is of theoretical interest, but the real goal is rather to minimize the number of errors using a bounded number of hyperplanes. Exact minimization of empirical error in a high-dimensional grid induced into the feature space by axis-parallel hyperplanes is NP-hard. We develop two approximation schemes with performance guarantees, a greedy set covering scheme for producing a consistently labeled grid, and integer programming rounding scheme for finding the minimum error grid with bounded number of hyperplanes.
Chapter PDF
Similar content being viewed by others
Keywords
- Approximation Algorithm
- Separation Condition
- Numerical Attribute
- Linear Program Relaxation
- Empirical Error
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Vapnik, V.N.: Estimation of Dependencies Based on Empirical Data. Springer, New York (1982)
Kearns, M.J., Schapire, R.E., Sellie, L.M.: Toward efficient agnostic learning. Machine Learn. 17, 115–141 (1994)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learn. 29, 131–163 (1997)
Călinescu, G., Dumitrescu, A., Wan, P.J.: Separating points by axis-parallel lines. In: Proc. Sixteenth Canadian Conference on Computational Geometry, pp. 7–10 (2004)
Elomaa, T., Rousu, J.: Efficient multisplitting revisited: Optima-preserving elimination of partition candidates. Data Mining and Knowl. Discovery 8, 97–126 (2004)
Elomaa, T., Rousu, J.: On decision boundaries of naïve Bayes in continuous domains. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 144–155. Springer, Heidelberg (2003)
Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)
Hyafil, L., Rivest, R.R.: Constructing optimal binary decision trees is NP-complete. Inf. Process. Lett. 5, 15–17 (1976)
Megiddo, N.: On the complexity of polyhedral separability. Discrete Comput. Geom. 3, 325–337 (1988)
Blum, A., Rivest, R.R.: Training a 3-node neural net is NP-complete. Neural Networks 5, 117–127 (1992)
Grigni, M., Mirelli, V., Papadimitriou, C.H.: On the difficulty of designing good classifiers. SIAM J. Comput. 30, 318–323 (2000)
Das, G., Goodrich, M.: On the complexity of optimization problems for 3-dimensional convex polyhedra and decision trees. Comput. Geom. 8, 123–137 (1997)
Auer, P., Holte, R.C., Maass, W.: Theory and application of agnostic PAC-learning with small decision trees. In: Proc. Twelfth International Conference on Machine Learning, pp. 21–29. Morgan Kaufmann, San Francisco (1995)
Dobkin, D., Gunopulos, D.: Geometric problems in machine learning. In: Lin, M.C., Manocha, D. (eds.) FCRC-WS 1996 and WACG 1996. LNCS, vol. 1148, pp. 121–132. Springer, Heidelberg (1996)
Dobkin, D., Gunopulos, D.: Concept learning with geometric hypotheses. In: Proc. Eighth Annual Conference on Computational Learning Theory, pp. 329–336. ACM Press, New York (1995)
Dobkin, D., Gunopulos, D., Maass, W.: Computing the maximum bichromatic discrepancy, with applications in computer graphics and machine learning. J. Comput. Syst. Sci. 52, 453–470 (1996)
Chlebus, B.S., Nguyen, S.H.: On finding optimal discretizations for two attributes. In: Polkowski, L., Skowron, A. (eds.) RSCTC 1998. LNCS (LNAI), vol. 1424, pp. 537–544. Springer, Heidelberg (1998)
Vazirani, V.V.: Approximation Algorithms. Springer, Heidelberg (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Elomaa, T., Kujala, J., Rousu, J. (2005). Approximation Algorithms for Minimizing Empirical Error by Axis-Parallel Hyperplanes. In: Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds) Machine Learning: ECML 2005. ECML 2005. Lecture Notes in Computer Science(), vol 3720. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11564096_53
Download citation
DOI: https://doi.org/10.1007/11564096_53
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29243-2
Online ISBN: 978-3-540-31692-3
eBook Packages: Computer ScienceComputer Science (R0)