Annals of Operations Research

, Volume 174, Issue 1, pp 121–134 | Cite as

Alternating local search based VNS for linear classification

  • Frank Plastria
  • Steven De Bruyne
  • Emilio Carrizosa
Article

Abstract

We consider the linear classification method consisting of separating two sets of points in d-space by a hyperplane. We wish to determine the hyperplane which minimises the sum of distances from all misclassified points to the hyperplane. To this end two local descent methods are developed, one grid-based and one optimisation-theory based, and are embedded into a VNS metaheuristic scheme. Computational results show these approaches to be complementary, leading to a single hybrid VNS strategy which combines both approaches to exploit the strong points of each. Extensive computational tests show that the resulting method can always be expected to approach the global optimum close enough that any deviations from the global optimum are irrelevant with respect to the classification power.

Keywords

Data mining Classification Linear classification Heuristic minimisation Norm-distance Variable neighbourhood search Local search Grid search Cell search 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Audet, C., Hansen, P., Karam, A., Ng, C. D., & Perron, S. (2008). Exact L 2-norm plane separation. Optimization Letters, 2, 483–495. CrossRefGoogle Scholar
  2. Carrizosa, E., & Plastria, F. (2008). Optimal expected distance separating halfspace. Mathematics of Operations Research, 33, 662–677. CrossRefGoogle Scholar
  3. Hansen, P., & Mladenović, N. (2003). Variable neighborhood search. In F. Glover & G. Kochenberger (Eds.), Handbook of metaheuristics (pp. 145–184). Dordrecht: Kluwer Academic. Google Scholar
  4. Horst, R., & Tuy, H. (1990). Global optimization: deterministic approaches. Berlin: Springer. Google Scholar
  5. Karam, A. (2005). Essays on linear discrimination. Ph.D. thesis at HEC Montréal, Option Méthodes Quantitatives de Gestion. Google Scholar
  6. Karam, A., Caporossi, G., & Hansen, P. (2007). Arbitrary-norm hyperplane separation by variable neighbourhood search. IMA Journal of Management Mathematics, 18, 173–189. CrossRefGoogle Scholar
  7. Mangasarian, O. L. (1994). Misclassification minimization. Journal of Global Optimization, 5, 309–323. CrossRefGoogle Scholar
  8. Mangasarian, O. L. (1999). Arbitrary-norm separating plane. Operations Research Letters, 24, 15–23. CrossRefGoogle Scholar
  9. Mladenović, N., & Hansen, P. (1997). Variable neighborhood search. Computers and Operations Research, 24, 1097–1100. CrossRefGoogle Scholar
  10. Musicant, D. (1998). NDC Normally distributed clustered datasets. http//www.mathcs.carleton.edu/faculty/dmusican/ndc/.
  11. Newman, D. J., Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI Repository of machine learning databases. http//www.ics.uci.edu/~mlearn/MLRepository.html, Irvine, CA: University of California, Department of Information and Computer Science.
  12. Plastria, F., & Carrizosa, E. (2001). Gauge-distances and median hyperplanes. Journal of Optimization Theory and Applications, 110, 173–182. CrossRefGoogle Scholar
  13. Plastria, F., & Carrizosa, E. (2002). Optimal distance separating halfspace (Working Paper BEIF/124). Vrije Universiteit Brussel, http//www.optimization-online.org/DB_HTML/2004/10/970.html.
  14. Plastria, F., De Bruyne, S., & Carrizosa, E. (2008). Dimensionality reduction for classification: comparison of techniques and dimension choice. In C. Tang, C. X. Ling, X. Zhou, N. J. Cercone, & X. Li (Eds.), Notes in artificial intelligence : Vol. 5139. Advanced data mining and applications (pp. 411–418). Berlin: Springer. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Frank Plastria
    • 1
  • Steven De Bruyne
    • 1
  • Emilio Carrizosa
    • 2
  1. 1.MOSIVrije Universiteit BrusselBrusselsBelgium
  2. 2.Universidad de SevillaSevilleSpain

Personalised recommendations