Big Data Performance Analysis on a Hadoop Distributed File System Based on Modified Partitional Clustering Algorithm

Abstract

This paper proposes a Big Data Performance Analysis based on a modified Partitional Clustering Algorithm (PCA) on a Hadoop Distributed File System (HDFS) which is commonly used in various business applications. This paper has utilized an improved K-means clustering algorithm, which selects the initial clustering centers based on the density parameters. After calculating the density parameter, the data with largest density parameter is selected as the first initial clustering center point, all the left data in the field is deleted from the dataset. By repeating the above phases, K initial clustering centers are found. A new method to improve the precision and packing effect of the K-means computation is needed as there is a poor assurance of finding an initial centers. The proposed approach does not select the initial clustering algorithm randomly, so the stable K value can be obtained by calculating Variance based Cluster Validity Index (VCVI). The performance of the proposed method is evaluated with the parameters Precision, Clustering time and Recall. The experimental result shows that the proposed approach reduces the complexity along with various parameters are compared with existing methods.