Abstract
Mining big data poses computational and memory challenges because of the astonishing rate of data generation when addressed by traditional mining methods. To deal with such problems we can take advantage of parallel programming such as MapReduce which permits parallel processing in massively distributed environment. In this paper, we address the issue of mining association rules from big datasets in such environments. For this, we introduce two contributions. The first one consists on exploiting irreducible paradigm for attributes reduction. The second one is to introduce a new generic parallel algorithm called DGARM for mining generic association rules from big data. We carried out exhaustive experiments over real world datasets to illustrate the efficiency of DGARM for large real world datasets.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Kryszkiewicz, J.: Concise representations of association rules. In: Proceedings of Exploratory Workshop on Pattern Detection and Discovery in Data Mining (ESF), LNAI, vol. 2447, pp. 92–109. Springer, London, UK (2002)
Han, J., Pei, J., Yin, Y.: Mining frequent patterns without candidate generation. In: ACM SIGMOD International Conference on Management of Data, no. 29, pp. 1–12 (2000)
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 1(51), 107–113 (2008)
Riondato, M., DeBrabant, J.A., Fonseca, R., Upfal, E.: PARMA: “a parallel randomized algorithm for approximate association rules mining in MapReduce. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 85–94 (2012)
Lin, K., Chung, S.-H.: A fast and resource efficient mining algorithm for discovering frequent patterns in distributed computing environments. Future Gener. Comput. Syst. 52, 49–58 (2015)
Asha, P., Srinivasan, S.: Distributed association rule mining with load balancing in grid environment. J. Comput. Theor. Nanosci. 13(1), 33–42 (2016)
Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The hadoop distributed file system. In: the IEEE 26th Symposium on Mass Storage, pp. 1–10 (2010)
Moens, S., Aksehirli, E., Goethals, B.: Frequent itemset mining for big data. In: IEEE International Conference on in Big Data, pp. 111–118 (2013)
Kovacs, F., Illes, J.: Frequent itemset mining on hadoop. In: Proceedings of IEEE 9th International Conference on Computational Cybernetics (ICCC), pp. 241–245 (2013)
Riondato, M., DeBrabant, J.A., Fonseca, R., Upfal, E.: PARMA: a parallel randomized algorithm for approximate association rules mining in MapReduce. In: Proceedings of the 21st ACM International Conference on Information and Knowledge Management, pp. 85–94 (2012)
Gasmi, G., BenYahia, S., Nguifo, E.M., Slimani, Y.: IGB: a new informative generic base of association rules. In: Proceedings of the Intl. Ninth Pacific-Asia Conference on Knowledge Data Discovery (PAKDD 2005). LNAI, vol. 3518, pp. 81–90. Spring, Hanoi, Vietnam (2005)
Bastide, Y., Pasquier, N., Taouil, R., Lakhal, L., Stumme, G.: Mining minimal non-redundant association rules using frequent closed itemsets. In: Proceedings of the International Conference DOOD 2000, LNAI, vol. 1, no. 861, pp. 972–986. Springer, London (2000)
Wang, S.-Q., Yang, Y.-B., Gao, Y., Chen, G.-P., Zhang, Y.: Mapreduce based closed frequent itemset mining with efficient redundancy filtering. In: ICDM Workshop, pp. 449–453 (2012)
Li, H., Wang, Y., Zhang, D., Zhang, M., Chang, E.Y.: PFP: parallel FP growth for query recommendation. In: ACM Conference on Recommender Systems (RecSys), pp. 107–114 (2008)
Zitouni, M., Akbarinia, R., Yahia, S.B., Masseglia, F.: A prime number based approach for closed frequent itemset mining in big data. In: the 26th International conference on database and expert systems applications (DEXA’2015), vol. 9261, pp. 509–516 (2015)
Ines, B., Samir, E.: Integrated generic association rule based classifier. In: DEXA Workshops, pp. 514–515 (2007)
http://www.philippe-fournier-viger.com/spmf/index.php?link=datasets.php
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Bouraoui, M., Bouzouita, I., Touzi, A.G. (2020). Map-Reduce Based Generic Basis of Association Rules Mining from Big Bata. In: Liu, Y., Wang, L., Zhao, L., Yu, Z. (eds) Advances in Natural Computation, Fuzzy Systems and Knowledge Discovery. ICNC-FSKD 2019. Advances in Intelligent Systems and Computing, vol 1075. Springer, Cham. https://doi.org/10.1007/978-3-030-32591-6_69
Download citation
DOI: https://doi.org/10.1007/978-3-030-32591-6_69
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32590-9
Online ISBN: 978-3-030-32591-6
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)