Advertisement

Research on a New Density Clustering Algorithm Based on MapReduce

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 849)

Abstract

The empirical solution parameters for the Density-Based Spatial Clustering of Applications with Noise(DBSCAN) resulted in poor Clustering effect and low execution efficiency, An adaptive DBSCAN algorithm based on genetic algorithm and MapReduce programming framework is proposed. The genetic algorithm (minPts) and scanning radius size (Eps) optimized intensive interval threshold, at the same time, combined with the similarities and differences of data sets using the Hadoop cluster parallel computing ability of two specifications, the data is reasonable of serialization, finally realizes the adaptive parallel clustering efficiently. Experimental results show that the improved algorithm (GA) - DBSCANMR when dealing with the data set of magnitude 3 times execution efficiency is improved DBSCAN algorithm, clustering quality improved by 10%, and this trend increases as the amount of data, provides a more precise threshold DBSCAN algorithm to determine the implementation of the method.

Keywords

DBSCAN MinPts Eps Genetic algorithm MapReduce 

References

  1. 1.
    Chen, G., Liu, B.Q., Wu, Y.: An adaptive DBSCAN algorithm based on gaussian distribution. Microelectron. Comput. 30(03), 27–30 (2013)Google Scholar
  2. 2.
    Feng, Z., Qian, X., Zhao, N.: Greedy DBSCAN: a DBSCAN improvement algorithm for multi-density clustering. Comput. Appl. Res. (9) (2016)Google Scholar
  3. 3.
    Ester, M., Kriegel, H.P., Sander, J., et al.: A Density-based algorithm for discovering clusters in large spatial databases with noise. pp. 226–231 (2008)Google Scholar
  4. 4.
    Wang, W.T., Wu, Y.L., Tang, C.Y., et al.: Adaptive density-based spatial clustering of applications with noise (DBSCAN) according to data. In: International Conference on Machine Learning and Cybernetics, IEEE (2015)Google Scholar
  5. 5.
    Uncu, O., Gruver, W.A., Kotak, D.B., et al.: GRIDBSCAN: grid density-based spatial clustering of applications with noise. vol. 4, pp. 2976–2981 (2006)Google Scholar
  6. 6.
    Shufen, L., Dongxue, M., Xiaoyan, W.: Study on DBSCAN Algorithm Based on Grid Element. J. Jilin Univ. Eng. Sci. 44(4), 1135–1139 (2014)Google Scholar
  7. 7.
    Luo, Q.: DBSCAN algorithm based on cloud computing. Wuhan University of Technology (2013)Google Scholar
  8. 8.
    Yang, Y.: Study on adaptive density clustering algorithm based on MapReduce. Tianjin University (2013)Google Scholar
  9. 9.
    He, Y., Tan, H., Luo, W., et al.: MR-DBSCAN: an efficient parallel density-based clustering algorithm using MapReduce. In: IEEE 17th International Conference on Parallel and Distributed Systems, IEEE Computer Society 2011, pp. 473–480 (2011)Google Scholar
  10. 10.
    Lei, L., Nie, R., Wang, J., et al.: Improved DBSCAN algorithm based on MapReduce. Comput. Sci. (s2) 396–399 (2015)Google Scholar
  11. 11.
    Li, L., Xi, Y.: Research on clustering algorithm and its parallelization strategy. In: International Conference on Computational and Information Sciences, pp. 325–328 (2011)Google Scholar
  12. 12.
    Design, application and application of data clustering algorithm based on MapReduce. J. East China Inst. TechnolGoogle Scholar
  13. 13.
    Song, J., Xu, S., Guo, C., et al.: A task distribution algorithm for optimizing energy consumption of MapReduce system. J. Comput. 39(2), 323–338 (2016)MathSciNetGoogle Scholar
  14. 14.
    Wang, Z., Chen, Q., Li, Z., et al.: MapReduce data equalization method based on incremental partition strategy. J. Comput. (1) 19–35 (2016)Google Scholar
  15. 15.
    Yaling, X., Jifu, Z., Xiao, Q.: Data placement strategy in MapReduce cluster environment. J. Softw. (8), 2056–2073 (2015)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.BaoTou Teacher’s CollegeBaotouChina

Personalised recommendations