Alternating local search based VNS for linear classification
We consider the linear classification method consisting of separating two sets of points in d-space by a hyperplane. We wish to determine the hyperplane which minimises the sum of distances from all misclassified points to the hyperplane. To this end two local descent methods are developed, one grid-based and one optimisation-theory based, and are embedded into a VNS metaheuristic scheme. Computational results show these approaches to be complementary, leading to a single hybrid VNS strategy which combines both approaches to exploit the strong points of each. Extensive computational tests show that the resulting method can always be expected to approach the global optimum close enough that any deviations from the global optimum are irrelevant with respect to the classification power.
KeywordsData mining Classification Linear classification Heuristic minimisation Norm-distance Variable neighbourhood search Local search Grid search Cell search
Unable to display preview. Download preview PDF.
- Hansen, P., & Mladenović, N. (2003). Variable neighborhood search. In F. Glover & G. Kochenberger (Eds.), Handbook of metaheuristics (pp. 145–184). Dordrecht: Kluwer Academic. Google Scholar
- Horst, R., & Tuy, H. (1990). Global optimization: deterministic approaches. Berlin: Springer. Google Scholar
- Karam, A. (2005). Essays on linear discrimination. Ph.D. thesis at HEC Montréal, Option Méthodes Quantitatives de Gestion. Google Scholar
- Musicant, D. (1998). NDC Normally distributed clustered datasets. http//www.mathcs.carleton.edu/faculty/dmusican/ndc/.
- Newman, D. J., Hettich, S., Blake, C. L., & Merz, C. J. (1998). UCI Repository of machine learning databases. http//www.ics.uci.edu/~mlearn/MLRepository.html, Irvine, CA: University of California, Department of Information and Computer Science.
- Plastria, F., & Carrizosa, E. (2002). Optimal distance separating halfspace (Working Paper BEIF/124). Vrije Universiteit Brussel, http//www.optimization-online.org/DB_HTML/2004/10/970.html.
- Plastria, F., De Bruyne, S., & Carrizosa, E. (2008). Dimensionality reduction for classification: comparison of techniques and dimension choice. In C. Tang, C. X. Ling, X. Zhou, N. J. Cercone, & X. Li (Eds.), Notes in artificial intelligence : Vol. 5139. Advanced data mining and applications (pp. 411–418). Berlin: Springer. CrossRefGoogle Scholar