A Novel Binary Particle Swarm Optimization Algorithm and Its Applications on Knapsack and Feature Selection Problems

Conference paper
Part of the Proceedings in Adaptation, Learning and Optimization book series (PALO, volume 8)


Particle swarm optimisation (PSO) is a well-known evolutionary computation technique, which has been applied to solve many optimisation problems. There are two main types of PSO, which are continuous PSO (CPSO) and binary PSO (BPSO). Since PSO is originally proposed to address continuous problems, CPSO has been studied extensively while there are only a few studies about BPSO. In a standard PSO algorithm, momentum is an important component, which preserves the swarm’s diversity. However, since movements in binary search spaces and continuous search spaces are different, it is not appropriate to apply directly the momentum concept of CPSO to BPSO. This paper introduces a new momentum concept to BPSO, which leads to a novel BPSO algorithm, named SBPSO. SBPSO is compared with a recent BPSO algorithm, named PBPSO, in two well-known binary problems: knapsack and feature selection. The experimental results on knapsack datasets show that SBPSO can find better solutions than PBPSO. In feature selection problems, SBPSO can select a smaller number of features and still achieve similar or better accuracies than PBPSO and using all the original features in a comparative computation time.


Particle swarm optimisation Feature selection Knapsack Binary optimisation 


  1. 1.
    Asuncion, A., Newman, D.: UCI machine learning repository (2007)Google Scholar
  2. 2.
    Bagheri, A., Peyhani, H.M., Akbari, M.: Financial forecasting using anfis networks with quantum-behaved particle swarm optimization. Expert Systems with Applications 41(14), 6235–6250 (2014)CrossRefGoogle Scholar
  3. 3.
    Blum, C., Li, X.: Swarm intelligence in optimization. In: Swarm Intelligence, pp. 43–85. Springer (2008)Google Scholar
  4. 4.
    Drake, J.H., Özcan, E., Burke, E.K.: A case study of controlling crossover in a selection hyper-heuristic framework using the multidimensional knapsack problem. Evolutionary computation 24(1), 113–141 (2016)CrossRefGoogle Scholar
  5. 5.
    Foulds, L.R.: Optimization techniques: an introduction. Springer Science & Business Media (2012)Google Scholar
  6. 6.
    Ganesh, M.R., Krishna, R., Manikantan, K., Ramachandran, S.: Entropy based binary particle swarm optimization and classification for ear detection. Engineering Applications of Artificial Intelligence 27, 115–128 (2014)CrossRefGoogle Scholar
  7. 7.
    Gholizadeh, S., Moghadas, R.: Performance-based optimum design of steel frames by an improved quantum particle swarm optimization. Advances in Structural Engineering 17(2), 143–156 (2014)CrossRefGoogle Scholar
  8. 8.
    Jordehi, A.R., Jasni, J.: Particle swarm optimisation for discrete optimisation problems: a review. Artificial Intelligence Review 43(2), 243–258 (2015)CrossRefGoogle Scholar
  9. 9.
    Kennedy, J., Eberhart, R., et al.: Particle swarm optimization. In: Proceedings of IEEE international conference on neural networks. vol. 4, pp. 1942–1948. Perth, Australia (1995)Google Scholar
  10. 10.
    Kennedy, J., Eberhart, R.C.: A discrete binary version of the particle swarm algorithm. In: Systems, Man, and Cybernetics. Computational Cybernetics and Simulation. IEEE International Conference on. vol. 5, pp. 4104–4108 (1997)Google Scholar
  11. 11.
    Khanesar, M.A., Teshnehlab, M., Shoorehdeli, M.A.: A novel binary particle swarm optimization. In: Control Automation, 2007. MED ’07. Mediterranean Conference on. pp. 1–6 (June 2007)Google Scholar
  12. 12.
    Liu, J., Mei, Y., Li, X.: An analysis of the inertia weight parameter for binary particle swarm optimization. IEEE Transactions on Evolutionary Computation PP(99), 1–1. doi: 10.1109/TEVC.2015.2503422 (2015)
  13. 13.
    Neshatian, K., Zhang, M.: Genetic programming for feature subset ranking in binary classification problems. In: Genetic programming, pp. 121–132. Springer (2009)Google Scholar
  14. 14.
    Nguyen, H., Xue, B., Liu, I., Zhang, M.: Filter based backward elimination in wrapper based pso for feature selection in classification. In: Evolutionary Computation (CEC), 2014 IEEE Congress on. pp. 3111–3118 (July 2014)Google Scholar
  15. 15.
    Nguyen, H., Xue, B., Liu, I., Zhang, M.: Pso and statistical clustering for feature selection: A new representation. In: Simulated Evolution and Learning, Lecture Notes in Computer Science, vol. 8886, pp. 569–581. Springer International Publishing (2014)Google Scholar
  16. 16.
    Pampara, G., Franken, N., Engelbrecht, A.P.: Combining particle swarm optimisation with angle modulation to solve binary problems. In: 2005 IEEE Congress on Evolutionary Computation. vol. 1, pp. 89–96 Vol. 1 (Sept 2005)Google Scholar
  17. 17.
    Pearl, J.: Heuristics: intelligent search strategies for computer problem solving (1984)Google Scholar
  18. 18.
    Unler, A., Murat, A.: A discrete particle swarm optimization method for feature selection in binary classification problems. European Journal of Operational Research 206(3), 528–539 (2010)CrossRefzbMATHGoogle Scholar
  19. 19.
    Wang, L., long Zheng, X., yao Wang, S.: A novel binary fruit fly optimization algorithm for solving the multidimensional knapsack problem. Knowledge-Based Systems 48, 17–23 (2013)Google Scholar
  20. 20.
    Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Transactions on Evolutionary Computation 20(4), 606–626 (Aug 2016)CrossRefGoogle Scholar
  21. 21.
    Xue, B., Cervante, L., Shang, L., Browne, W.N., Zhang, M.: A multi-objective particle swarm optimisation for filter-based feature selection in classification problems. Connection Science (2-3), 91–116 (2012)CrossRefGoogle Scholar
  22. 22.
    Xue, B., Nguyen, S., Zhang, M.: A new binary particle swarm optimisation algorithm for feature selection. In: European Conference on the Applications of Evolutionary Computation. pp. 501–513. Springer (2014)Google Scholar
  23. 23.
    Yuan, H., Tseng, S.S., Gangshan, W., Fuyan, Z.: A two-phase feature selection method using both filter and wrapper. In: Systems, Man, and Cybernetics. IEEE International Conference on. vol. 2, pp. 132–136. IEEE (1999)Google Scholar
  24. 24.
    Zambrano-Bigiarini, M., Clerc, M., Rojas, R.: Standard particle swarm optimisation 2011: A baseline for future pso improvements. In: 2013 IEEE Congress on Evolutionary Computation. pp. 2337–2344 (2013)Google Scholar
  25. 25.
    Zhang, Y., Wang, S., Phillips, P., Ji, G.: Binary pso with mutation operator for feature selection using decision tree applied to spam detection. Knowledge-Based Systems 64, 22–31 (2014)CrossRefGoogle Scholar
  26. 26.
    Zhang, Y., Wu, L., Wang, S.: Ucav path planning by fitness-scaling adaptive chaotic particle swarm optimization. Mathematical Problems in Engineering 2013, 1–8 (2013)MathSciNetGoogle Scholar
  27. 27.
    Zhao, H., Sinha, A.P., Ge, W.: Effects of feature construction on classification performance: An empirical study in bank failure prediction. Expert Systems with Applications 36(2), 2633–2644 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand

Personalised recommendations