Skip to main content
SpringerLink
Log in

MoMAC: Multi-objective optimization to combine multiple association rules into an interpretable classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

A crucial characteristic of machine learning models in various domains (such as medical diagnosis, financial analysis, or real-time process monitoring) is the interpretability. The interpretation supports humans in understanding the meaning behind every single prediction made by the machine, and enables the user to assess trustworthiness before acting on the predictions. This article presents our work in building an interpretable classification model based on association rule mining and multi-objective optimization. The classification model itself is a rule list, making a single prediction based on multiple rules. The rule list consists of If ... THEN statements that are understandable to humans. We choose these rules from a large set of pre-mined rules according to an interestingness measure which is formulated as a function of basic probabilities related to the rules. We learned the interestingness measure through multi-objective optimization, concentrating on two objectives: the classifier’s size in terms of number of rules and prediction accuracy. The model is called MoMAC, “Multi-Objective optimization to combine Multiple Association rules into an interpretable Classification”. The experimental results on benchmark datasets demonstrate that MoMAC outperforms other existing rule-based classification methods in terms of classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Code Availability

https://github.com/banhdzui/MoMAC-v1.git

References

  1. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. Proceedings of the Interntational Conference on Very Large Databases: 487–499

  2. Angelino E, Larus-Stone N, Alabi D, Seltzer M, Rudin C (2017) Learning certifiably optimal rule lists for categorical data. J Mach Learn Res 18:8753–8830

    MathSciNet  MATH  Google Scholar 

  3. Bertsimas D, Dunn J (2017) Optimal classification trees. Mach Learn 106:1039–1082

    Article  MathSciNet  Google Scholar 

  4. Chen C, Rudin C (2018) An optimization approach to learning falling rule lists. AISTATS: 604–612

  5. Dash P, Fiore-Gartland AJ, Hertz T, Wang GC, Sharma S, Souquette A, Crawford JC, Clemens EB, Nguyen TH, Kedzierska K, et al. (2017) Quantifiable predictive features define epitope-specific t cell receptor repertoires. Nature 547:89–93

    Article  Google Scholar 

  6. Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6:182–197

    Article  Google Scholar 

  7. Dembczyński K., Kotłowski W, Słowiński R (2008) Maximum likelihood rule ensembles. Proceedings of the International Conference on Machine Learning: 224–231

  8. Dembczyński K., Kotłowski W, Słowiński R (2010) Ender: a statistical framework for boosting decision rules. Data Mining and Knowledge Discovery 21:52–90

    Article  MathSciNet  Google Scholar 

  9. Djenouri Y, Belhadi A, Fournier-Viger P, Fujita H (2018) Mining diversified association rules in big datasets: A cluster/GPU/genetic approach. Inform Sci 459:117–134

    Article  MathSciNet  Google Scholar 

  10. Djenouri Y, Lin JCW, Nørvåg K, Ramampiaro H (2019) Highly efficient pattern mining based on transaction decomposition. In: 2019 IEEE 35th International Conference on Data Engineering (ICDE). IEEE, pp 1646–1649

  11. Dong G, Li J (2009) Emerging pattern based classification. Encyclopedia of Database Systems: 985

  12. Dong G, Zhang X, Wong L, Li J (1999) CAEP: Classification by aggregating emerging patterns. International Conference on Discovery Science: 30–42

  13. Dua D, Graff C (2019) UCI machine learning repository. http://archive.ics.uci.edu/ml1646–1649

  14. Fan H, Ramamohanarao K (2006) Fast discovery and the generalization of strong jumping emerging patterns for building compact and accurate classifiers. IEEE Trans Knowl Data Eng 18:721–737

    Article  Google Scholar 

  15. Friedman JH, Popescu BE (2008) Predictive learning via rule ensembles. Annals Appl Stat 2:916–954

    Article  MathSciNet  Google Scholar 

  16. Geng L, Hamilton HJ (2006) Interestingness measures for data mining: A survey. ACM Computing Surveys (CSUR) 38:9–es

    Article  Google Scholar 

  17. Lakkaraju H, Bach SH, Leskovec J (2016) Interpretable decision sets: A joint framework for description and prediction. Proceedings of the International Conference on Knowledge Discovery and Data Mining: 1675–1684

  18. Letham B, Rudin C, McCormick TH, Madigan D (2015) Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. Annals Appl Stat 9:1350–1371

    Article  MathSciNet  Google Scholar 

  19. Li J, Dong G, Ramamohanarao K (2000) Instance-based classification by emerging patterns. European Conference on Principles of Data Mining and Knowledge Discovery: 191– 200

  20. Li W, Han J, Pei J (2001) CMAR: accurate and efficient classification based on multiple class-association rules. IEEE International Conference on Data Mining: 369–376

  21. Liu B, Hsu W, Ma Y (1998) Integrating classification and association rule mining. KDD: 80–86

  22. Luna JM, Fournier-Viger P, Ventura S (2019) Frequent itemset mining: A 25 years review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 9(6)

  23. Mattiev J, Kavšek B (2020) A compact and understandable associative classifier based on overall coverage. Procedia Computer Science 170:1161–1167

    Article  Google Scholar 

  24. Mita G, Papotti P, Filippone M, Michiardi P (2020) LIBRE: Learning interpretable boolean rule ensembles. In: International conference on artificial intelligence and statistics. PMLR, pp 245–255

  25. Molnar C (2018) Interpretable machine learning: a guide for making black box models explainable. Leanpub

  26. New York Civil Liberties Union (2014) Stop and frisk data http://www.nyclu.org/content/stop-and-frisk-data

  27. Paul Y, Kumar N (2020) A comparative study of famous classification techniques and data mining tools. In: Proceedings of ICRIC. Springer, pp 627–644

  28. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: Machine learning in python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  29. Quinlan JR (2014) C4.5: Programs for machine learning. Elsevier

  30. Rajab KD (2019) New associative classification method based on rule pruning for classification of datasets. IEEE Access 7 :157783–157795

    Article  Google Scholar 

  31. Rijnbeek PR, Kors JA (2010) Finding a short and accurate decision rule in disjunctive normal form by exhaustive search. Mach Learn 80:33–62

    Article  MathSciNet  Google Scholar 

  32. Rudin C (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat Mach Intell 1:206–215

    Article  Google Scholar 

  33. Sharma R, Kaushik M, Peious SA, Yahia SB, Draheim D (2020) Expected vs. unexpected: selecting right measures of interestingness. In: International conference on big data analytics and knowledge discovery. Springer, pp 38–47

  34. Song K, Lee K (2017) Predictability-based collective class association rule mining. Expert Syst Appl: 1–7

  35. Sood N, Zaiane O (2020) Building a competitive associative classifier. In: International conference on big data analytics and knowledge discovery. Springer, pp 223–234

  36. Venturini L, Baralis E, Garza P (2017) Scaling associative classification for very large datasets. Journal of Big Data 4(1):1–24

    Article  Google Scholar 

  37. Wang F, Rudin C (2015) Falling rule lists. Artif Intell Stat: 1013–1022

  38. Wei D, Dash S, Gao T, Günlük O (2019) Generalized linear rule models. In: Proceedings of the International Conference on Machine Learning, pp 6687–6696

  39. Yang G, Mabu SM, Shimada K, Gong Y, Hirasawa K (2009) Ranking association rules for classification based on genetic network programming. Proceedings of the annual conference on genetic and evolutionary computation: 1917–1918

  40. Yang G, Shimada K, Mabu S, Hirasawa K (2008) A nonlinear model to rank association rules based on semantic similarity and genetic network programming. IEEE Trans Electrical Electronic Eng: 1–9

  41. Yang H, Rudin C, Seltzer M (2017) Scalable bayesian rule lists. Proceedings of the International Conference on Machine Learning 70:3921–3930

    Google Scholar 

  42. Yin X, Han J (2003) CPAR: Classification based on predictive association rules. In: Proceedings of the SIAM international conference on data mining, pp 331–335

Download references

Funding

The work was supported by Universiteit Antwerpen under BOF docpro grant to the first author.

Author information

Authors and Affiliations

  1. Adrem Data Lab, Department of Computer Science, University of Antwerp, Antwerp, Belgium

    Danh Bui-Thi, Pieter Meysman & Kris Laukens

Authors
  1. Danh Bui-Thi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pieter Meysman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kris Laukens
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Danh Bui-Thi or Kris Laukens.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bui-Thi, D., Meysman, P. & Laukens, K. MoMAC: Multi-objective optimization to combine multiple association rules into an interpretable classification. Appl Intell 52, 3090–3102 (2022). https://doi.org/10.1007/s10489-021-02595-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02595-w

Keywords

Search

Navigation