Classification through incremental max–min separability
- 182 Downloads
Piecewise linear functions can be used to approximate non-linear decision boundaries between pattern classes. Piecewise linear boundaries are known to provide efficient real-time classifiers. However, they require a long training time. Finding piecewise linear boundaries between sets is a difficult optimization problem. Most approaches use heuristics to avoid solving this problem, which may lead to suboptimal piecewise linear boundaries. In this paper, we propose an algorithm for globally training hyperplanes using an incremental approach. Such an approach allows one to find a near global minimizer of the classification error function and to compute as few hyperplanes as needed for separating sets. We apply this algorithm for solving supervised data classification problems and report the results of numerical experiments on real-world data sets. These results demonstrate that the new algorithm requires a reasonable training time and its test set accuracy is consistently good on most data sets compared with mainstream classifiers.
KeywordsClassification Data mining Data analysis Supervised learning Piecewise linear classifier
Dr. Adil Bagirov is the recipient of an Australian Research Council Australian Research Fellowship (Project number: DP 0666061). Dr. Adil Bagirov and Prof. Bülent Karasözen are thankful for the support of TUBITAK (Turkish Scientific and Technical Research Council) and the Australian Mathematical Sciences Institute which initiated this current work by their supporting mutual visits. We would like to thank two anonymous referees for their useful suggestions that improved the quality of the paper.
- 4.Palm HC (1990) A new piecewise linear classifier. Pattern recognition. In: Proceedings 10th international conference on 16–21 June, vol 1, pp 742–744Google Scholar
- 7.Bagirov AM, Ugon J (2005) Supervised data classification via max–min separability. In: Jeyakumar V, Rubinov AM (eds). Continuous optimisation: current trends and modern applications. Springer, Berlin, pp 175–208Google Scholar
- 13.Kuncheva L (2000) Clustering and selection model of classifier combination. In: Proceedings of knowledge-based engineering systems and allied technologies. Brighton, UKGoogle Scholar
- 14.Jackowski K, Wozniak M (2009) Algorithm of designing compound recognition system on the basis of combining classifiers with simultaneous splitting feature space into competence areas. Pattern Anal Appl. doi: 10.1007/s10044-008-0137-7
- 20.Schulmeister B, Wysotzki F (1994) The piecewise linear classifier DIPOL92. In: Bergadano F, De Raedt L (eds) Proceedings of the European conference on machine learning on machine learning (Catania, Italy). Springer, New York, pp 411–414Google Scholar
- 21.Michie D, Spiegelhalter DJ, Taylor CC (eds) (1994) Machine learning, neural and statistical classification. Ellis Horwood, LondonGoogle Scholar
- 22.Bagirov AM (1999) Minimization methods for one class of nonsmooth functions and calculation of semi-equilibrium prices. In: Eberhard A et al (eds) Progress in optimization: contribution from Australasia. Kluwer, Dordrecht, pp 147–175Google Scholar
- 24.Asuncion A, Newman DJ (2007) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine, CA. http://www.ics.uci.edu/mlearn/MLRepository.html