Approximation Algorithms for Minimizing Empirical Error by Axis-Parallel Hyperplanes

  • Tapio Elomaa
  • Jussi Kujala
  • Juho Rousu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3720)


Many learning situations involve separation of labeled training instances by hyperplanes. Consistent separation is of theoretical interest, but the real goal is rather to minimize the number of errors using a bounded number of hyperplanes. Exact minimization of empirical error in a high-dimensional grid induced into the feature space by axis-parallel hyperplanes is NP-hard. We develop two approximation schemes with performance guarantees, a greedy set covering scheme for producing a consistently labeled grid, and integer programming rounding scheme for finding the minimum error grid with bounded number of hyperplanes.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vapnik, V.N.: Estimation of Dependencies Based on Empirical Data. Springer, New York (1982)Google Scholar
  2. 2.
    Kearns, M.J., Schapire, R.E., Sellie, L.M.: Toward efficient agnostic learning. Machine Learn. 17, 115–141 (1994)MATHGoogle Scholar
  3. 3.
    Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Machine Learn. 29, 131–163 (1997)MATHCrossRefGoogle Scholar
  4. 4.
    Călinescu, G., Dumitrescu, A., Wan, P.J.: Separating points by axis-parallel lines. In: Proc. Sixteenth Canadian Conference on Computational Geometry, pp. 7–10 (2004)Google Scholar
  5. 5.
    Elomaa, T., Rousu, J.: Efficient multisplitting revisited: Optima-preserving elimination of partition candidates. Data Mining and Knowl. Discovery 8, 97–126 (2004)CrossRefMathSciNetGoogle Scholar
  6. 6.
    Elomaa, T., Rousu, J.: On decision boundaries of naïve Bayes in continuous domains. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 144–155. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  7. 7.
    Shawe-Taylor, J., Cristianini, N.: Kernel Methods for Pattern Analysis. Cambridge University Press, Cambridge (2004)Google Scholar
  8. 8.
    Hyafil, L., Rivest, R.R.: Constructing optimal binary decision trees is NP-complete. Inf. Process. Lett. 5, 15–17 (1976)MATHCrossRefMathSciNetGoogle Scholar
  9. 9.
    Megiddo, N.: On the complexity of polyhedral separability. Discrete Comput. Geom. 3, 325–337 (1988)MATHCrossRefMathSciNetGoogle Scholar
  10. 10.
    Blum, A., Rivest, R.R.: Training a 3-node neural net is NP-complete. Neural Networks 5, 117–127 (1992)CrossRefGoogle Scholar
  11. 11.
    Grigni, M., Mirelli, V., Papadimitriou, C.H.: On the difficulty of designing good classifiers. SIAM J. Comput. 30, 318–323 (2000)MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Das, G., Goodrich, M.: On the complexity of optimization problems for 3-dimensional convex polyhedra and decision trees. Comput. Geom. 8, 123–137 (1997)MATHCrossRefMathSciNetGoogle Scholar
  13. 13.
    Auer, P., Holte, R.C., Maass, W.: Theory and application of agnostic PAC-learning with small decision trees. In: Proc. Twelfth International Conference on Machine Learning, pp. 21–29. Morgan Kaufmann, San Francisco (1995)Google Scholar
  14. 14.
    Dobkin, D., Gunopulos, D.: Geometric problems in machine learning. In: Lin, M.C., Manocha, D. (eds.) FCRC-WS 1996 and WACG 1996. LNCS, vol. 1148, pp. 121–132. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  15. 15.
    Dobkin, D., Gunopulos, D.: Concept learning with geometric hypotheses. In: Proc. Eighth Annual Conference on Computational Learning Theory, pp. 329–336. ACM Press, New York (1995)CrossRefGoogle Scholar
  16. 16.
    Dobkin, D., Gunopulos, D., Maass, W.: Computing the maximum bichromatic discrepancy, with applications in computer graphics and machine learning. J. Comput. Syst. Sci. 52, 453–470 (1996)MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Chlebus, B.S., Nguyen, S.H.: On finding optimal discretizations for two attributes. In: Polkowski, L., Skowron, A. (eds.) RSCTC 1998. LNCS (LNAI), vol. 1424, pp. 537–544. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  18. 18.
    Vazirani, V.V.: Approximation Algorithms. Springer, Heidelberg (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Tapio Elomaa
    • 1
  • Jussi Kujala
    • 1
  • Juho Rousu
    • 2
  1. 1.Institute of Software SystemsTampere University of Technology 
  2. 2.Department of Computer ScienceRoyal Holloway University of London 

Personalised recommendations