Skip to main content
SpringerLink
Log in

Fuzzy-Based Kernelized Clustering Algorithms for Handling Big Data Using Apache Spark

  • Conference paper
  • First Online:
Proceedings of 6th International Conference on Harmony Search, Soft Computing and Applications (ICHSA 2020)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 1275))

Included in the following conference series:

Abstract

In this paper, we propose a novel Kernelized Scalable Random Sampling with Iterative Optimization Fuzzy c-Means (KSRSIO-FCM) and a Kernelized Scalable Literal Fuzzy c-Means (KSLFCM) clustering algorithms for big data framework. The evolution of kernelized clustering algorithms led us to deal with the nonlinear separable problems by applying kernel Radial Basis Functions (RBF) which map the input data space nonlinearly into a high-dimensional feature space. The experimental result shows that the KSRSIO-FCM algorithm achieves significant improvement in terms of F-score, Adjusted Rand Index (ARI), and Normalized Mutual Information (NMI) for Big Data. Experimentation is performed on well-known IRIS datasets to show the effectiveness of proposed KSRSIO-FCM in comparison with KSLFCM. The KSRSIO-FCM implemented on Apache Spark shows better potential for Big Data clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. A. Saxena, M. Prasad, A. Gupta, N. Bharill, O.P. Patel, A. Tiwari, M.J. Er, W. Ding, C.T. Lin, A review of clustering techniques and developments. Neuro-Comput. 267, 664–681 (2017)

    Google Scholar 

  2. K. Hirota, W. Pedrycz, Fuzzy computing for data mining. Proc. IEEE 87(9), 1575–1600 (1999)

    Article  Google Scholar 

  3. N.S. Iyer, A. Kandel, M. Schneider, Feature-based fuzzy classification for interpretation of mammograms. Fuzzy Sets Syst. 114(2), 271–280 (2000)

    Article  Google Scholar 

  4. W. Phillips II., R. Velthuizen, S. Phuphanich, L. Hall, L. Clarke, M. Silbiger, Application of fuzzy c-means segmentation technique for tissue differentiation in mr images of a hemorrhagic glioblastoma multiforme. Magn. Reson. Imaging 13(2), 277–290 (1995)

    Article  Google Scholar 

  5. R.O. Duda, P.E. Hart, et al., Pattern Classification and Scene Analysis, vol. 3 (Wiley, New York, 1973)

    Google Scholar 

  6. Y. Leung, J.S. Zhang, Z.B. Xu, Clustering by scalespace filtering. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1396–1410 (2000)

    Article  Google Scholar 

  7. J.C. Bezdek, Pattern Recognition with Fuzzy Objective Function Algorithms (Springer Science & Business Media, 2013)

    Google Scholar 

  8. Z. Zhang, T.C. Havens, Scalable approximation of kernel fuzzy c-means, in 2013 IEEE International Conference on Big Data (IEEE, 2013), pp. 161–168

    Google Scholar 

  9. P. Hore, L.O. Hall, D.B. Goldgof, Single pass fuzzy c-means, in 2007 IEEE International Fuzzy Systems Conference (IEEE, 2007), pp. 1–7

    Google Scholar 

  10. J.K. Parker, L.O. Hall, Accelerating fuzzy-c means using an estimated subsample size. IEEE Trans. Fuzzy Syst. 22(5), 1229–1244 (2013)

    Article  Google Scholar 

  11. J. Gantz, D. Reinsel, The digital universe in 2020: big data, bigger digital shadows, and biggest growth in the far east. IDC iView: IDC Analyze the Future 2007 (2012), pp. 1–16

    Google Scholar 

  12. G. Zhao, C. Ling, D. Sun, Sparksw: scalable distributed computing system for large-scale biological sequence alignment, in 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (IEEE, 2015), pp. 845–852

    Google Scholar 

  13. T.C. Havens, J.C. Bezdek, C. Leckie, L.O. Hall, M. Palaniswami, Fuzzy c-means algorithms for very large data. IEEE Trans. Fuzzy Syst. 20(6), 1130–1146 (2012)

    Article  Google Scholar 

  14. N. Bharill, A. Tiwari, Handling big data with fuzzy based classification approach, in Advance Trends in Soft Computing (Springer, 2014), pp. 219–227

    Google Scholar 

  15. N. Bharill, A. Tiwari, A. Malviya, Fuzzy based scalable clustering algorithms for handling big data using apache spark. IEEE Trans. Big Data 2(4), 339–352 (2016)

    Article  Google Scholar 

  16. J. Dean, S. Ghemawat, Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  17. N. Bharill, A. Tiwari, A. Malviya Fuzzy based clustering algorithms to handle big data with implementation on apache spark, in 2016 IEEE Second International Conference on Big Data Computing Service and Applications (BigDataService) (IEEE, 2016), pp. 95–104

    Google Scholar 

  18. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M.J. Franklin, S. Shenker, I. Stoica, Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing, in Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation (USENIX Association, 2012), pp. 2–2

    Google Scholar 

  19. D. Borthakur, et al., HDFS architecture guide. Hadoop Apache Project 53, 1–13 (2008)

    Google Scholar 

  20. S. Tang, B. He, C. Yu, Y. Li, K. Li, A survey on spark ecosystem for big data processing (2018), arXiv preprint arXiv:181108834

  21. J.F. Kolen, T. Hutcheson, Reducing the time complexity of the fuzzy c- means algorithm. IEEE Trans. Fuzzy Syst. 10(2), 263–267 (2002)

    Article  Google Scholar 

  22. V.K. Vavilapalli, A.C. Murthy, C. Douglas, S. Agarwal, M. Konar, R. Evans, T. Graves, J. Lowe, H. Shah, S. Seth, et al., Apache hadoop yarn: yet another resource negotiator, in Proceedings of the 4th Annual Symposium on Cloud Computing (ACM, 2013), p. 5

    Google Scholar 

  23. V. Schwämmle, O.N. Jensen, A simple and fast method to determine the parameters for fuzzy c–means cluster analysis. Bioinformatics 26(22), 2841–2848 (2010)

    Google Scholar 

  24. Y. Wang, L. Chen, J.P. Mei, Incremental fuzzy clustering with multiple medoids for large data. IEEE Trans. Fuzzy Syst. 22(6), 1557–1568 (2014)

    Article  Google Scholar 

  25. K.Y. Yeung, W.L. Ruzzo, Details of the adjusted rand index and clustering algorithms, supplement to the paper an empirical study on principal component analysis for clustering gene expression data. Bioinformatics 17(9), 763–774 (2001)

    Article  Google Scholar 

  26. A. Strehl, J. Ghosh, Cluster ensembles—a knowledge reuse frame-work for combining multiple partitions. J. Mach. Learn. Res. 3(Dec), 583–617 (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology Indore, Indore, India

    Preeti Jha, Aruna Tiwari, Neha Nagendra & Mukkamalla Mounika

  2. Department of Computer Science and Engineering, Mahindra University, Ecole Centrale School of Engineering, Hyderabad, India

    Neha Bharill

  3. Biotechnology, ICAR-Indian Institute of Soybean Research Indore, Indore, India

    Milind Ratnaparkhe

Authors
  1. Preeti Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aruna Tiwari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Neha Bharill
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Milind Ratnaparkhe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Neha Nagendra
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mukkamalla Mounika
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Preeti Jha .

Editor information

Editors and Affiliations

  1. Istanbul University – Cerrahpaşa, Istanbul, Turkey

    Sinan Melih Nigdeli

  2. Korea University, Seoul, Korea (Republic of)

    Joong Hoon Kim

  3. Istanbul University - Cerrahpaşa, Istanbul, Turkey

    Gebrail Bekdaş

  4. Dr. BR Ambedkar National Institute of Technology, Jalandhar, India

    Anupam Yadav

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Jha, P., Tiwari, A., Bharill, N., Ratnaparkhe, M., Nagendra, N., Mounika, M. (2021). Fuzzy-Based Kernelized Clustering Algorithms for Handling Big Data Using Apache Spark. In: Nigdeli, S.M., Kim, J.H., Bekdaş, G., Yadav, A. (eds) Proceedings of 6th International Conference on Harmony Search, Soft Computing and Applications. ICHSA 2020. Advances in Intelligent Systems and Computing, vol 1275. Springer, Singapore. https://doi.org/10.1007/978-981-15-8603-3_37

Download citation

Publish with us

Policies and ethics

Search

Navigation