A novel hybrid GA–PSO framework for mining quantitative association rules

Abstract

Discovering association rules is a useful and common technique for data mining in which dependencies among datasets are shown. Discovering the rules from continuous numeric datasets is one of the common challenges in data mining. Furthermore, another restriction imposed by algorithms in this area is the need to determine the minimum threshold for the criteria of support and confidence. By drawing on two heuristic optimization techniques, to wit, the genetic algorithm (GA) and particle swarm optimization (PSO) algorithm, a hybrid algorithm for extracting quantitative association rules was developed in this research. Accurate and interpretable rules result from the integration of the multiple objectives GA with the multiple objective PSO algorithms, which redresses the balance in the exploitation and exploration tasks. The useful and appropriate rules and the most suitable numerical intervals are discovered by proposing a multi-criteria method in which there is no need to discretize numerical values and to determine threshold values of minimum support and confidence. Different criteria are used to determine appropriate rules. In this algorithm, the selected rules are extracted based on confidence, interestingness and comprehensibility. The results gained over five real-world datasets evidence the effectiveness of the proposed method. By hybridization of the GA and the PSO algorithm, the proposed approach has achieved considerable improvements compared with the basic algorithms in the criteria of the number of extracted rules from dataset, high confidence measure and support percentage.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

References

  1. Abdel-Kader RF (2011) Hybrid discrete PSO with GA operators for efficient QoS-multicast routing. Ain Shams Eng J 2(1):21–31

    Google Scholar 

  2. Agarwal A, Nanavati N (2016, December) Association rule mining using hybrid GA–PSO for multi-objective optimisation. In: 2016 IEEE international conference on computational intelligence and computing research (ICCIC). IEEE, pp 1–7

  3. Alatas B, Akin E (2008) Rough particle swarm optimization and its applications in data mining. Soft Comput 12(12):1205–1218

    MATH  Google Scholar 

  4. Alatas B, Akin E (2009) Multi-objective rule mining using a chaotic particle swarm optimization algorithm. Knowl Based Syst 22(6):455–460

    Google Scholar 

  5. Alataş B, Akin E (2006) An efficient genetic algorithm for automated mining of both positive and negative quantitative association rules. Soft Comput 10(3):230–237

    Google Scholar 

  6. Alatas B, Akin E, Karci A (2008) MODENAR: multi-objective differential evolution algorithm for mining numeric association rules. Appl Soft Comput 8(1):646–656

    Google Scholar 

  7. Alcala-Fdez J, Flugy-Pape N, Bonarini A, Herrera F (2010) Analysis of the effectiveness of the genetic algorithms based on extraction of association rules. Fundam Inform 98(1):1–14

    MathSciNet  Google Scholar 

  8. Álvarez VP, Vázquez JM (2012) An evolutionary algorithm to discover quantitative association rules from huge databases without the need for an a priori discretization. Expert Syst Appl 39(1):585–593

    Google Scholar 

  9. Angeline PJ (1998, March) Evolutionary optimization versus particle swarm optimization: philosophy and performance differences. In: International conference on evolutionary programming. Springer, Berlin, pp 601–610

    Google Scholar 

  10. Beiranvand V, Mobasher-Kashani M, Bakar AA (2014) Multi-objective PSO algorithm for mining numerical association rules without a priori discretization. Expert Syst Appl 41(9):4259–4273

    Google Scholar 

  11. Can U, Alatas B (2017) Automatic mining of quantitative association rules with gravitational search algorithm. Int J Softw Eng Knowl Eng 27(03):343–372

    Google Scholar 

  12. Cervante L, Xue B, Zhang M, Shang L (2012, June) Binary particle swarm optimisation for feature selection: a filter based approach. In: 2012 IEEE congress on evolutionary computation (CEC). IEEE, pp 1–8

  13. Chen MY (2013) A hybrid ANFIS model for business failure prediction utilizing particle swarm optimization and subtractive clustering. Inf Sci 220:180–195

    Google Scholar 

  14. Coello C, Lamont GB, Van Veldhuizen DA (2007) Evolutionary algorithms for solving multi-objective problems, vol 5. Springer, New York, pp 79–104

    MATH  Google Scholar 

  15. De Jong K (1975) Analysis of the behavior of a class of genetic adaptive systems. Ph.D. Thesis, University of Michigan, Ann Arbor, MI

  16. Deb K (2001) Multi-objective optimization using evolutionary algorithms, vol 16. Wiley, Hoboken

    MATH  Google Scholar 

  17. Djenouri Y, Drias H, Habbas Z, Mosteghanemi H (2012, December) Bees swarm optimization for web association rule mining. In: 2012 IEEE/WIC/ACM international conferences on web intelligence and intelligent agent technology (WI-IAT), vol 3. IEEE, pp 142–146

  18. Du S, Li W, Cao K (2006, June) A learning algorithm of artificial neural network based on GA–PSO. In: The sixth world congress on intelligent control and automation, 2006. WCICA 2006, vol 1. IEEE, pp 3633–3637

  19. Eberhart RC, Shi Y (1998, March) Comparison between genetic algorithms and particle swarm optimization. In: International conference on evolutionary programming. Springer, Berlin, pp 611–616

    Google Scholar 

  20. Efendigil T, Önüt S, Kahraman C (2009) A decision support system for demand forecasting with artificial neural networks and neuro-fuzzy models: a comparative analysis. Expert Syst Appl 36(3):6697–6707

    Google Scholar 

  21. Ehrgott M (2005) Multicriteria optimization, vol 491. Springer, Berlin

    MATH  Google Scholar 

  22. Ghosh A, Nath B (2004) Multi-objective rule mining using genetic algorithms. Inf Sci 163(1–3):123–133

    MathSciNet  Google Scholar 

  23. Goldberg DE (1989) Genetic algorithms in search optimization and machine learning, vol 41. Addison Wesley, Boston

    MATH  Google Scholar 

  24. Gupta M (2012) Application of weighted particle swarm optimization in association rule mining. International Journal of Computer Science and Informatics 1:2231–5292

    Google Scholar 

  25. Guvenir HA, Uysal I (2000) Bilkent university function approximation repository. http://funapp.cs.bilkent.edu.tr/DataSets/. Accessed 7 May 2018

  26. Haeri A, Tavakkoli-Moghaddam R (2012) Developing a hybrid data mining approach based on multi-objective particle swarm optimization for solving a traveling salesman problem. J Bus Econ Manag 13(5):951–967

    Google Scholar 

  27. Han J, Kambe M (2006) Data mining: concepts and techniques, 2nd edn. Morgan Kaufmann, Burlington

    Google Scholar 

  28. Hsieh Y, Lee P, You P (2018) Immune based evolutionary algorithm for determining the optimal sequence of multiple disinfection operations. Sci Iran 26:959–974. https://doi.org/10.24200/sci.2018.20324

    Article  Google Scholar 

  29. Huang SJ (2000) An immune-based optimization method to capacitor placement in a radial distribution system. IEEE Trans Power Deliv 15(2):744–749

    Google Scholar 

  30. Agrawal R, Imieliński, T, Swami A (1993) Mining association rules between sets of items in large databases. In: ACM sigmod record, vol 22, no 2. ACM, pp 207–216

  31. Indira K, Kanmani S (2015) Mining association rules using hybrid genetic algorithm and particle swarm optimisation algorithm. Int J Data Anal Tech Strateg 7(1):59–76

    Google Scholar 

  32. Juang CF (2004) A hybrid of genetic algorithm and particle swarm optimization for recurrent network design. IEEE Trans Syst Man Cybern Part B (Cybern) 34(2):997–1006

    Google Scholar 

  33. Kennedy J, Eberhart R (1995, November) Particle swarm optimization. In: IEEE international conference on neural networks, 1995. Proceedings, vol 4. IEEE, pp 1942–1948

  34. Kiziloluk S, Alatas B (2015) Automatic mining of numerical classification rules with parliamentary optimization algorithm. Adv Electr Comput Eng 15(4):17–24

    Google Scholar 

  35. Kokoç M, Ersöz S, Aktepe A, Türker AK (2016) Improvement of facility layout by using data mining algorithms and an application. Int J Intell Syst Appl Eng 4(Special Issue):92–100

    Google Scholar 

  36. Kou Z, Xi L (2018) Binary particle swarm optimization-based association rule mining for discovering relationships between machine capabilities and product features. Math Probl Eng 2018:1–16

    Google Scholar 

  37. Kumar DT, Soleimani H, Kannan G (2014) Forecasting return products in an integrated forward/reverse supply chain utilizing an ANFIS. Int J Appl Math Comput Sci 24(3):669–682

    MathSciNet  MATH  Google Scholar 

  38. Kuo RJ, Chao CM, Chiu YT (2011) Application of particle swarm optimization to association rule mining. Appl Soft Comput 11(1):326–336

    Google Scholar 

  39. Lukovac V, Pamučar D, Popović M, Đorović B (2017) Portfolio model for analyzing human resources: an approach based on neuro-fuzzy modeling and the simulated annealing algorithm. Expert Syst Appl 90:318–331

    Google Scholar 

  40. Luna JM, Romero JR, Ventura S (2013) Grammar-based multi-objective algorithms for mining association rules. Data Knowl Eng 86:19–37

    Google Scholar 

  41. Martín D, Rosete A, Alcalá-Fdez J, Herrera F (2014) QAR-CIP-NSGA-II: a new multi-objective evolutionary algorithm to mine quantitative association rules. Inf Sci 258:1–28

    MathSciNet  Google Scholar 

  42. Martín D, Alcalá-Fdez J, Rosete A, Herrera F (2016) NICGAR: A Niching Genetic Algorithm to mine a diverse set of interesting quantitative association rules. Inf Sci 355:208–228

    Google Scholar 

  43. Martín D, Martínez-Ballesteros M, García-Gil D, Alcalá-Fdez J, Herrera F, Riquelme-Santos JC (2018) MRQAR: a generic MapReduce framework to discover quantitative association rules in big data problems. Knowl Based Syst 153:176–192

    Google Scholar 

  44. Martínez-Ballesteros M, Martínez-Álvarez F, Troncoso A, Riquelme JC (2011) An evolutionary algorithm to discover quantitative association rules in multidimensional time series. Soft Comput 15(10):2065

    Google Scholar 

  45. Martínez-Ballesteros M, Bacardit J, Troncoso A, Riquelme JC (2015) Enhancing the scalability of a genetic algorithm to discover quantitative association rules in large-scale datasets. Integr Comput Aided Eng 22(1):21–39

    Google Scholar 

  46. Mata J, Alvarez JL, Riquelme JC (2001) Mining numeric association rules with genetic algorithms. In: Artificial neural nets and genetic algorithms. Springer, Vienna, pp 264–267

    Google Scholar 

  47. Miettinen K (1999) Nonlinear multiobjective optimization. International series in operations research and management science, vol 12. Kluwer Academic Publishers, Dordrecht

    MATH  Google Scholar 

  48. Miller RJ, Yang Y (1997) Association rules over interval data. ACM SIGMOD Rec 26(2):452–461

    Google Scholar 

  49. Minaei-Bidgoli B, Barmaki R, Nasiri M (2013) Mining numerical association rules via multi-objective genetic algorithms. Inf Sci 233:15–24

    Google Scholar 

  50. Moslehi F, Haeri A (2019) A genetic algorithm based framework for mining quantitative association rules without specifying minimum support and minimum confidence. Sci Iranica. https://doi.org/10.24200/SCI.2019.51110.2010

    Article  Google Scholar 

  51. Moslehi P, Bidgoli BM, Nasiri M, Salajegheh A (2011) Multi-objective numeric association rules mining via ant colony optimization for continuous domains without specifying minimum support and minimum confidence. Int J Comput Sci Issues (IJCSI) 8(5):34–41

    Google Scholar 

  52. Nasiri M, Sadat TL, Minaee B (2011) Numeric multi-objective rule mining using simulated annealing algorithm. Int J Appl Oper Res 1(2):37–48

    Google Scholar 

  53. Olafsson S, Li X, Wu S (2008) Operations research and data mining. Eur J Oper Res 187(3):1429–1448

    MathSciNet  MATH  Google Scholar 

  54. Padillo F, Luna JM, Herrera F, Ventura S (2018) Mining association rules on big data through MapReduce genetic programming. Integr Comput Aided Eng 25(1):31–48

    Google Scholar 

  55. Pamucar D, Ćirović G (2018) Vehicle route selection with an adaptive neuro fuzzy inference system in uncertainty conditions. Decis Mak Appl Manag Eng 1(1):13–37

    Google Scholar 

  56. Robinson J, Sinton S, Rahmat-Samii Y (2002) Particle swarm, genetic algorithm, and their hybrids: optimization of a profiled corrugated horn antenna. In: Antennas and propagation society international symposium, vol 1. IEEE, pp 314–317

  57. Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, Malaysia

    MATH  Google Scholar 

  58. Sadeghi H, Zolfaghari M, Heydarizade M (2011) Estimation of electricity demand in residential sector using genetic algorithm approach. J Ind Eng Prod Res 22(1):43–50

    Google Scholar 

  59. Santana-Quintero LV, Hernández-Díaz AG, Molina J, Coello CAC, Caballero R (2010) DEMORS: a hybrid multi-objective optimization algorithm using differential evolution and rough set theory for constrained problems. Comput Oper Res 37(3):470–480

    MathSciNet  MATH  Google Scholar 

  60. Shekarian E, Olugu EU, Abdul-Rashid SH, Kazemi N (2016) Analyzing optimization techniques in inventory models: the case of fuzzy economic order quantity problems. In: Proceedings of the 2016 international conference on industrial engineering and operations management, pp 1229–1240

  61. Shi Y, Eberhart RC (1999) Empirical study of particle swarm optimization. In: Proceedings of the 1999 congress on evolutionary computation, 1999. CEC 99, vol 3. IEEE, pp 1945–1950

  62. Smyth P, Goodman RM (1992) An information theoretic approach to rule induction from databases. IEEE Trans Knowl Data Eng 4(4):301–316

    Google Scholar 

  63. Sremac S, Tanackov I, Kopic M, Radovic D (2018) ANFIS model for determining the economic order quantity. Decis Mak Appl Manag Eng 1:1–12

    Google Scholar 

  64. Srikant R, Agrawal R (1996, June) Mining quantitative association rules in large relational tables. In: ACM sigmod record, vol 25, no 2. ACM, pp 1–12

  65. Tahyudin I, Nambo H (2017, July) The rules determination of numerical association rule mining optimization by using combination of PSO and cauchy distribution. In: International conference on management science and engineering management. Springer, Cham, pp 151–165

    Google Scholar 

  66. Yan X, Zhang C, Zhang S (2009) Genetic algorithm-based strategy for identifying association rules without specifying actual minimum support. Expert Syst Appl 36(2):3066–3076

    Google Scholar 

  67. Li C, Liu Y, Zhou, A, Kang, L, & Wang H (2007, September) A fast particle swarm optimization algorithm with cauchy mutation and natural selection strategy. In: International symposium on intelligence computation and applications. Springer, Berlin, pp 334–343

Download references

Acknowledgements

The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the manuscript.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Abdorrahman Haeri.

Ethics declarations

Conflict of interest

Fateme Moslehi declares that she has no conflict of interest. Abdorrahman Haeri declares that he has no conflict of interest. Francisco Martínez-Álvarez declares that he has no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Communicated by V. Loia.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Moslehi, F., Haeri, A. & Martínez-Álvarez, F. A novel hybrid GA–PSO framework for mining quantitative association rules. Soft Comput 24, 4645–4666 (2020). https://doi.org/10.1007/s00500-019-04226-6

Download citation

Keywords

  • Quantitative association rule mining
  • Multi-objective optimization
  • Hybridization
  • Genetic algorithm
  • Particle swarm optimization