Skip to main content
SpringerLink
Log in

Enhancing a machine learning binarization framework by perturbation operators: analysis on the multidimensional knapsack problem

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Solving combinatorial optimization problems is of great interest in the areas of computer science and operations research. Optimization algorithms and particularly metaheuristics are constantly improved in order to reduce execution times, increase the quality of solutions and address larger instances. In this work, an improvement of the binarization framework which uses the K-means technique is developed. To achieve this, a perturbation operator based on the K-nearest neighbor technique is incorporated into the framework with the aim of generating more robust binarized algorithms. The technique of K-nearest neighbors is used for improving the properties of diversification and intensification of metaheuristics in its binary version. The contribution of the K-nearest neighbors perturbation operator to the final results is systematically analyzed. Particle Swarm Optimization and Cuckoo Search are used as metaheuristic techniques. To verify the results, the well-known multidimensional knapsack problem is tackled. A computational comparison is made with the state-of-the-art of metaheuristic techniques that use general mechanisms of binarization. The results show that our improved framework produces consistently better results. In this sense, the contribution of the operator which uses the K-nearest neighbors technique is investigated finding that this operator contributes significantly to the quality of the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. OR-Library: http://www.brunel.ac.uk/mastjjb/jeb/orlib/mknapinfo.html.

  2. OR-Library: http://www.brunel.ac.uk/mastjjb/jeb/orlib/mknapinfo.html.

  3. Best values within our comparison are indicated in bold. This also holds for the next table.

References

  1. Ahmad SR, Bakar AA, Yaakub MR (2015) Metaheuristic algorithms for feature selection in sentiment analysis. In: Science and Information Conference (SAI), pp 222–226. IEEE

  2. Al-Madi N, Faris H, Mirjalili S (2019) Binary multi-verse optimization algorithm for global optimization and discrete problems. Int J Mach Learn Cybern 1–21

  3. Asta S, Özcan E, Curtois T (2016) A tensor based hyper-heuristic for nurse rostering. Knowl-Based Syst 98:185–199

    Google Scholar 

  4. Cadenas JM, Garrido MC, Muñoz E (2009) Using machine learning in a cooperative hybrid parallel strategy of metaheuristics. Inf Sci 179(19):3255–3267

    Google Scholar 

  5. Calvet L, de Armas J, Masip D, Juan AA (2017) Learnheuristics: hybridizing metaheuristics with machine learning for optimization with dynamic inputs. Open Math 15(1):261–280

    MathSciNet  MATH  Google Scholar 

  6. Caserta M, Voß S (2010) Metaheuristics: intelligent problem solving. Springer, Boston, pp 1–38

    Google Scholar 

  7. Chen E, Li J, Liu X (2011) In search of the essential binary discrete particle swarm. Appl Soft Comput 11(3):3260–3269

    Google Scholar 

  8. Chou J-S, Ngo N-T, Pham A-D (2015) Shear strength prediction in reinforced concrete deep beams using nature-inspired metaheuristic support vector regression. J Comput Civ Eng 30(1):04015002

    Google Scholar 

  9. Chou J-S, Nguyen T-K (2018) Forward forecast of stock price using sliding-window metaheuristic-optimized machine-learning regression. IEEE Trans Industr Inf 14(7):3132–3142

    Google Scholar 

  10. Chou J-S, Pham A-D (2017) Nature-inspired metaheuristic optimization in least squares support vector regression for obtaining bridge scour information. Inf Sci 399:64–80

    Google Scholar 

  11. Chou J-S, Thedja JPP (2016) Metaheuristic optimization within machine learning-based classification system for early warnings related to geotechnical problems. Autom Constr 68:65–80

    Google Scholar 

  12. Crawford B, Soto R, Astorga G, García J, Castro C, Paredes F (2017) Putting continuous metaheuristics to work in binary search spaces. Complexity, 2017:Article ID 8404231

  13. Damaševičius R, Woźniak M (2017) State flipping based hyper-heuristic for hybridization of nature inspired algorithms. In: International Conference on Artificial Intelligence and Soft Computing, pp 337–346. Springer

  14. de Alvarenga Rosa R, Machado AM, Ribeiro GM, Mauri GR (2016) A mathematical model and a clustering search metaheuristic for planning the helicopter transportation of employees to the production platforms of oil and gas. Comput Ind Eng 101:303–312

    Google Scholar 

  15. De Jong K (2007) Parameter setting in EAs: a 30 year perspective. Parameter setting in evolutionary algorithms. Springer, Berlin, pp 1–18

    Google Scholar 

  16. de León AD, Lalla-Ruiz E, Melián-Batista B, Moreno-Vega JM (2017) A machine learning-based system for berth scheduling at bulk terminals. Expert Syst Appl 87:170–182

    Google Scholar 

  17. Fernandes S, Setoue K, Adeli H, Papa J (2017) Fine-tuning enhanced probabilistic neural networks using metaheuristic-driven optimization. In: Bio-Inspired Computation and Applications in Image Processing, pp 25–45. Elsevier

  18. Fong S, Wong R, Vasilakos AV (2016) Accelerated PSO swarm search feature selection for data stream mining big data. IEEE Trans Serv Comput 9(1):33–45

    Google Scholar 

  19. Fréville A (2004) The multidimensional 0–1 knapsack problem: an overview. Eur J Oper Res 155(1):1–21

    MathSciNet  MATH  Google Scholar 

  20. García J, Altimiras F, Peña A, Astorga G, Peredo O (2018a) A binary cuckoo search big data algorithm applied to large-scale crew scheduling problems. Complexity, 2018:Article ID 8395193

  21. García, J, Crawford B, Soto R, Astorga G (2017) A percentile transition ranking algorithm applied to knapsack problem. In: Proceedings of the Computational Methods in Systems and Software, pp 126–138. Springer

  22. García J, Crawford B, Soto R, Astorga G (2018b) A percentile transition ranking algorithm applied to binarization of continuous swarm intelligence metaheuristics. In: International Conference on Soft Computing and Data Mining, pp 3–13. Springer

  23. García J, Crawford B, Soto R, Astorga G (2019a) A clustering algorithm applied to the binarization of swarm intelligence continuous metaheuristics. Swarm Evol Comput 44:646–664

    Google Scholar 

  24. García J, Crawford B, Soto R, Castro C, Paredes F (2018c) A k-means binarization framework applied to multidimensional knapsack problem. Appl Intell 48(2):357–380

    Google Scholar 

  25. García J, Moraga P, Valenzuela M, Crawford B, Soto R, Pinto H, Peña A, Altimiras F, Astorga G (2019b) A db-scan binarization algorithm applied to matrix covering problems. Comput Intell Neurosci, 2019

  26. Garey M, Johnson D (1979) A guide to the theory of NP-completeness. Comput Intractability

  27. Guo H, Liu B, Cai D, Lu T (2018) Predicting protein-protein interaction sites using modified support vector machine. Int J Mach Learn Cybernet 9(3):393–398

    Google Scholar 

  28. Iooss B, Lemaître P (2015) A review on global sensitivity analysis methods. Uncertainty management in simulation-optimization of complex systems. Springer, Berlin, pp 101–122

    Google Scholar 

  29. Jin Y, Qu R, Atkin J (2014) A population-based incremental learning method for constrained portfolio optimisation. In: Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), 2014 16th International Symposium on, pp 212–219. IEEE

  30. Juan AA, Faulin J, Grasman SE, Rabe M, Figueira G (2015) A review of simheuristics: extending metaheuristics to deal with stochastic combinatorial optimization problems. Oper Res Perspect 2:62–72

    MathSciNet  Google Scholar 

  31. Kennedy J, Eberhart RC (1997) A discrete binary version of the particle swarm algorithm. In: Systems, Man, and Cybernetics, 1997. Computational Cybernetics and Simulation., 1997 IEEE International Conference on, vol 5, pp 4104–4108. IEEE

  32. Korkmaz S, Babalik A, Kiran MS (2018) An artificial algae algorithm for solving binary optimization problems. Int J Mach Learn Cybernet 9(7):1233–1247

    Google Scholar 

  33. Kuo R, Kuo P, Chen YR, Zulvia FE (2016a) Application of metaheuristics-based clustering algorithm to item assignment in a synchronized zone order picking system. Appl Soft Comput 46:143–150

    Google Scholar 

  34. Kuo R-J, Mei C, Zulvia FE, Tsai C (2016b) An application of a metaheuristic algorithm-based clustering ensemble method to APP customer segmentation. Neurocomputing 205:116–129

    Google Scholar 

  35. Leonard BJ, Engelbrecht AP, Cleghorn CW (2015) Critical considerations on angle modulated particle swarm optimisers. Swarm Intell 9(4):291–314

    Google Scholar 

  36. Li Z-q, Zhang H-l, Zheng J-h, Dong M-j, Xie Y-f, Tian Z-j (2011) Heuristic evolutionary approach for weighted circles layout. Information and automation. Springer, Berlin, pp 324–331

    Google Scholar 

  37. Liu J, Wu C, Cao J, Wang X, Teo KL (2016) A binary differential search algorithm for the 0–1 multidimensional knapsack problem. Appl Math Model 40(23–24):9788–9805

    MathSciNet  MATH  Google Scholar 

  38. Liu W, Liu L, Cartes D (2007) Angle modulated particle swarm optimization based defensive islanding of large scale power systems. IEEE Power Engineering Society Conference and Exposition in Africa 1–8

  39. López-Ibáñez M, Dubois-Lacoste J, Cáceres LP, Birattari M, Stützle T (2016) The irace package: Iterated racing for automatic algorithm configuration. Oper Res Perspect 3:43–58

    MathSciNet  Google Scholar 

  40. Mann PS, Singh S (2017) Energy efficient clustering protocol based on improved metaheuristic in wireless sensor networks. J Netw Comput Appl 83:40–52

    Google Scholar 

  41. Martin S, Ouelhadj D, Beullens P, Ozcan E, Juan AA, Burke EK (2016) A multi-agent based cooperative approach to scheduling and routing. Eur J Oper Res 254(1):169–178

    MathSciNet  MATH  Google Scholar 

  42. Mirghasemi S, Andreae P, Zhang M (2019) Domain-independent severely noisy image segmentation via adaptive wavelet shrinkage using particle swarm optimization and fuzzy c-means. Expert Syst Appl 133:126–150

    Google Scholar 

  43. Moiz DZ, AE, Mezioud C, Draa A (2015) Binary bat algorithm: On the efficiency of mapping functions when handling binary problems using continuous-variable-based metaheuristics. In: Computer Science and Its Applications - 5th IFIP TC 5 International Conference, CIIA 2015, Saida, Algeria, May 20-21, 2015, Proceedings, pp 3–14

  44. Pampara G (2012) Angle modulated population based algorithms to solve binary problems. PhD thesis, University of Pretoria, Pretoria

  45. Ries J, Beullens P (2015) A semi-automated design of instance-based fuzzy parameter tuning for metaheuristics based on decision tree induction. J Oper Res Soc 66(5):782–793

    Google Scholar 

  46. Santos HG, Ochi LS, Marinho EH, Drummond LMDA (2006) Combining an evolutionary algorithm with data mining to solve a single-vehicle routing problem. Neurocomputing 70(1–3):70–77

    Google Scholar 

  47. Saremi S, Mirjalili S, Lewis A (2015) How important is a transfer function in discrete heuristic algorithms. Neural Comput Appl 26(3):625–640

    Google Scholar 

  48. Smith-Miles K, Baatar D, Wreford B, Lewis R (2014) Towards objective measures of algorithm performance across instance space. Comput Oper Res 45:12–24

    MathSciNet  MATH  Google Scholar 

  49. Streichert F, Stein G, Ulmer H, Zell A (2003) A clustering based niching method for evolutionary algorithms. Genetic and evolutionary computation conference. Springer, Berlin, pp 644–645

    MATH  Google Scholar 

  50. Swagatam D, Rohan M, Rupam K (2013) Multi-user detection in multi-carrier cdma wireless broadband system using a binary adaptive differential evolution algorithm. Proceedings of the 15th annual conference on Genetic and evolutionary computation, GECCO, pp 1245–1252

  51. Talbi E-G (2016) Combining metaheuristics with mathematical programming, constraint programming and machine learning. Ann Oper Res 240(1):171–215

    MathSciNet  MATH  Google Scholar 

  52. Tsai C-W, Chiang M-C, Ksentini A, Chen M (2016) Metaheuristic algorithms for healthcare: open issues and challenges. Compu Electr Eng 53:421–434

    Google Scholar 

  53. Tyasnurita R, Özcan E, Shahriar A, John R (2015) Improving performance of a hyper-heuristic using a multilayer perceptron for vehicle routing. In: 15th UK Workshop on Computational Intelligence, UK

  54. Vasquez M, Hao J-K (2001) A logic-constrained knapsack formulation and a tabu algorithm for the daily photograph scheduling of an earth observation satellite. Comput Optim Appl 20(2):137–157

    MathSciNet  MATH  Google Scholar 

  55. Veček N, Mernik M, Filipič B, Črepinšek M (2016) Parameter tuning with chess rating system (CRS-tuning) for meta-heuristic algorithms. Inf Sci 372:446–469

    Google Scholar 

  56. Xue Y, Xue B, Zhang M (2019) Self-adaptive particle swarm optimization for large-scale feature selection in classification. ACM Trans Knowl Discov Data (TKDD) 13(5):50

    Google Scholar 

  57. Yalcinoz T, Altun H (2001) Power economic dispatch using a hybrid genetic algorithm. IEEE Power Eng Rev 21(3):59–60

    Google Scholar 

  58. Yang M-H (2001) An efficient algorithm to allocate shelf space. Eur J Oper Res 131(1):107–118

    MATH  Google Scholar 

  59. Yang Y, Mao Y, Yang P, Jiang Y (2013) The unit commitment problem based on an improved firefly and particle swarm optimization hybrid algorithm. In: Chinese Automation Congress (CAC), 2013, pp 718–722. IEEE

  60. Zhang G (2011) Quantum-inspired evolutionary algorithms: a survey and empirical study. J Heuristics 17(3):303–351

    MATH  Google Scholar 

  61. Zhang X, Wu C, Li J, Wang X, Yang Z, Lee J-M, Jung K-H (2016) Binary artificial algae algorithm for multidimensional knapsack problems. Appl Soft Comput 43:583–595

    Google Scholar 

  62. Zheng B, Zhang J, Yoon SW, Lam SS, Khasawneh M, Poranki S (2015) Predictive modeling of hospital readmissions using metaheuristics and data mining. Expert Syst Appl 42(20):7110–7120

    Google Scholar 

Download references

Acknowledgements

José García was supported by the Grant CONICYT/FONDECYT/INICIACION/11180056.

Author information

Authors and Affiliations

  1. Pontificia Universidad Católica de Valparaíso, 2362807, Valparaíso, Chile

    José García

  2. Department of Industrial Engineering and Business Information Systems, Universiteit Twente, Enschede, The Netherlands

    Eduardo Lalla-Ruiz

  3. Institute of Information Systems, University of Hamburg, Hamburg, Germany

    Stefan Voß

  4. Department of Mechanical Engineering, University of Chile, Beauchef 850, Santiago, Chile

    Enrique López Droguett

Authors
  1. José García
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eduardo Lalla-Ruiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stefan Voß
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Enrique López Droguett
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José García.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

García, J., Lalla-Ruiz, E., Voß, S. et al. Enhancing a machine learning binarization framework by perturbation operators: analysis on the multidimensional knapsack problem. Int. J. Mach. Learn. & Cyber. 11, 1951–1970 (2020). https://doi.org/10.1007/s13042-020-01085-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01085-8

Keywords

Search

Navigation