Abstract
Particle swarm optimization (PSO) is used in several combinatorial optimization problems. In this work, particle swarms are used to solve quadratic programming problems with quadratic constraints. The central idea is to use PSO to move in the direction towards optimal solution rather than searching the entire feasible region. Binary classification is posed as a quadratically constrained quadratic problem and solved using the proposed method. Each class in the binary classification problem is modeled as a multidimensional ellipsoid to form a quadratic constraint in the problem. Particle swarms help in determining the optimal hyperplane or classification boundary for a data set. Our results on the Iris, Pima, Wine, Thyroid, Balance, Bupa, Haberman, and TAE datasets show that the proposed method works better than a neural network and the performance is close to that of a support vector machine.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Dantzig G B 1963 Linear programming. Princeton, NJ : University Press
Khachiyan L G 1979 A polynomial algorithm in linear programming. Doklady Akademia Nauk SSSR 244:S 1093– 1096
Karmarkar N 1984 A new polynomial-time algorithm for linear programming. Combinatorica 4: 373–395
Bishop C M 2006 Pattern recognition and machine learning. Springer, Berlin
Duda R O, Hart P E and Stork D G 2001 Pattern classification. Wiley
Derrac J, García S and Herrera F 2014 Fuzzy nearest neighbor algorithms: Taxonomy, experimental analysis and prospects. Inform. Sci. 260: 98–119
Bishop C M 1995 Neural networks for pattern recognition. Oxford university press
Cortes C and Vapnik V 1995 Support-vector networks. Mach. Learn. 20 (3): 273–297
Platt J 1998 Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B, Burges C, Smola A (eds) Advances in Kernel methods – support vector learning
Fernández A, López V, Galar M, Jesus M J and Herrera F 2013 Analysing the classification of imbalanced data-sets with multiple classes: Binarization techniques and ad-hoc approaches. Knowledge-based syst.: 97–110
Galar M, Fernández A, Barrenechea E and Herrera F 2013 EUSBoost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling. Pattern Recognit. 46 (12): 3460–3471
López V, Fernández A, García S, Palade V and Herrera F 2013 An insight into classification with imbalanced data. Empirical results and current trends on using data intrinsic characteristics. Information sciences
Gonzalez-Abril L, Velasco F, Angulo C and Ortega J A 2013 A study on output normalization in multiclass SVMs. Pattern Recognit. Lett.: 344–348
Boyd S and Vandenberghe L 2004 Convex optimization: Cambridge University Press
Bomze I M 1998 On standard quadratic optimization problems. J. Global Optimiz. 13: 369–387
Bomze I M and Schachinger W 2010 Multi-standard quadratic optimization: interior point methods and cone programming reformulation. Comput. Optimiz. Appl. 45: 237– 256
Kennedy J and Eberhart R C 1995 Particle swarm optimization. In: Proceedings of IEEE International Conference on Neural Networks. IV, pp 1942–1948
Spadoni M and Stefanini L 2012 A differential evolution algorithm to deal with box, linear and quadratic-convex constraints for boundary optimization. J. Global Optimiz. 52 (1): 171–192
Zhan Z -H, Zhang J, Li Y and Chung H S -H 2009 Adaptive particle swarm optimization. IEEE Trans. Syst. Man Cybern. - B: Cybern. 39 (6): 1362–1381
Kennedy J, Eberhart R C and Shi Y 2001 Swarm intelligence. Morgan Kaufmann Publishers, San Francisco
Zanella F, Varagnolo D, Cenedese A, Pillonetto G, Schenato L 2012 Multidimensional Newton-Raphson consensus for distributed convex optimization. In: The 2012 American Control Conference (ACC), pp 1079–1084
Matei I, Baras J S 2012 A performance comparison between two consensus-based distributed optimization algorithms. In: 3rd IFAC Workshop on Distributed Estimation and Control in Networked Systems, pp 168–173
Nedi A and Ozdaglar A 2009 Distributed subgradient methods for multi-agent optimization. IEEE Trans. Autom. Control 54 (1): 48–61
Nedi A, Ozdaglar A and Parrilo P A 2010 Constrained consensus and optimization in multi-agent networks. IEEE Trans. Autom. Control 55 (4): 922–938
Boyd S, Parikh N, Chu E, Peleato B and Eckstein J 2010 Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations Trends Mach. Learn. 3 (1): 1–122
Forero P A, Cano A and Giannakis G B 2010 Consensus-based distributed support vector machines. J. Mach. Learn. Res. 11: 1663–1707
Kennedy J, Eberhart R C 1997 A discrete binary version of the particle swarm algorithm. In: Proceedings of the World Multiconference on Systems, Cybernetics and Informatics, pp 4104–4109
Cervantes A, Galvan I M, Isasi P 2005 A comparison between the Pittsburgh and Michigan approaches for the binary PSO algorithm. In: Congress on evolutionary computation, pp 290–297
UC Irvine Machine Learning Repository 2014 http://archive.ics.uci.edu/ml/
Gonzalez-Abril L, Nuñez H, Angulo C and Velasco F 2014 GSVM : An SVM for handling imbalanced accuracy between classes in bi-classification problems. Appl. Soft Comput. 17: 23–31
Kumar D and Ramakrishnan A G 2014 Quadratically constrained quadratic programming for classification using particle swarms and applications. CoRR. arXiv:1407.6315
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
KUMAR, D., RAMAKRISHNAN, A.G. Binary classification posed as a quadratically constrained quadratic programming and solved using particle swarm optimization. Sādhanā 41, 289–298 (2016). https://doi.org/10.1007/s12046-016-0466-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12046-016-0466-y