Big Data Performance Analysis on a Hadoop Distributed File System Based on Modified Partitional Clustering Algorithm
This paper proposes a Big Data Performance Analysis based on a modified Partitional Clustering Algorithm (PCA) on a Hadoop Distributed File System (HDFS) which is commonly used in various business applications. This paper has utilized an improved K-means clustering algorithm, which selects the initial clustering centers based on the density parameters. After calculating the density parameter, the data with largest density parameter is selected as the first initial clustering center point, all the left data in the field is deleted from the dataset. By repeating the above phases, K initial clustering centers are found. A new method to improve the precision and packing effect of the K-means computation is needed as there is a poor assurance of finding an initial centers. The proposed approach does not select the initial clustering algorithm randomly, so the stable K value can be obtained by calculating Variance based Cluster Validity Index (VCVI). The performance of the proposed method is evaluated with the parameters Precision, Clustering time and Recall. The experimental result shows that the proposed approach reduces the complexity along with various parameters are compared with existing methods.
KeywordsHDFS PCA VCVI K-means cluster Density parameter Clustering time Recall
- 5.Anwar, M., Oakes, M.: Data mining of audiology patient records: factors influencing the choice of hearing aid type. In: BMC Medical Informatics and Decision Making, vol. 12, no. 1 (2012)Google Scholar
- 6.Eluri, V., Ramesh, M., Al-Jabri, A., Jane, M.: A comparative study of various clustering techniques on big data sets using Apache Mahout. In: 2016 3rd MEC International Conference on Big Data and Smart City (ICBDSC) (2016)Google Scholar
- 9.Borthakur, D.: The hadoop distributed file system: architecture and design. Hadoop Project Website, pp. 1–14 (2007)Google Scholar