Charismatic Document Clustering Through Novel K-Means Non-negative Matrix Factorization (KNMF) Algorithm Using Key Phrase Extraction
- 9 Downloads
The tedious challenging of Big Data is to store and retrieve of required data from the search engines. Problem Defined There is an obligation for the quick and efficient retrieval of useful information for the many organizations. The elementary idea is to arrange these computing files of organization into individual folders in an hierarchical order of folders. Manually, to order these files into folders, there is an ardent need to know about the file contents and name of the files to give impression of files, so that it provides an alignment of certain set of files as a bunch. Problem Statement Manual grouping of files has its own complications, for example when these files are in numerous amounts and also their contents cannot be illustrious by their labels. Therefore, it’s an intense requirement for Document clustering with data processing machines for enthusiastic results. Existing System A couple of analyzers are impending with dynamic algorithms and comprehensive analogy of extant algorithms, but, yet, these have been restricted to organizations and colleges. After recent updated rules of NMF their raised a self interest in document clustering. These rules gave trust in its performances with better results when compared to Latent Semantic Indexing with Singular Value Decomposition. Proposed System A new working miniature called Novel K-means Non-Negative Matrix Factorization (KNMF) is implemented using renovated guidelines of NMF which has been diagnosed for clustering documents consequently. A new data set called Newsgroup20 is considered for the exploratory purpose. Removal of common clutter/stop words using keywords from Key Phrase Extraction Algorithm and a new proposed Iterated Lovin stemming will be utilized in preprocessing step inassisting to KNMF. Compared to the Porter stemmer and Lovins stemmer algorithms, Iterative Lovins algorithm is providing 5% more reduction. 60% of the document terms are been minimized to root as remaining terms are already root words. Eventually, an appeal to these processes named “Progressive Text mining radical” is developed inlateral exertion of K-Means algorithm from the defined Apache Mahout Project which is used to analyze the performance of the MapReduce framework in Hadoop.
KeywordsDocument clustering K-means non-negative matrix factorization (KNMF) Iterated lovin stemmers Key phrase extraction Stopwords MapReduce Hadoop Term frequency-inverse term frequency (Tf-IDF)
This work is financially supported by the Department of Science and Technology (DST), Science and Engineering Research Board (SERB) under the scheme of ECR. We thank DST_SERB for the financial support to carry the research work.
- 3.Baeza Yates, R., Ribeiro Neto, B., et al.: Modern Information Retrieval. ACMPress, New York (1999)Google Scholar
- 5.Guduru, N.: Text Mining with Support Vector Machines (SVM) and Non-Negative Matrix Factorization (NMF) Algorithm. Master’s Thesis, University of Rhode Island, CS Department (2006)Google Scholar
- 9.Lee, D.D., Seung, H.: Algorithm for non-negative matrix factorization. In: Dietterich, T.G., Tresp, V. (eds.) Advances in Neural Information Processing Systems, Volume 13, Proceedings of the Conference: 556562. The MIT PressGoogle Scholar
- 10.Ding, C., He, X., Simon, H.D.: On the equivalence of nonnegative matrix factorization (NMF) and spectral clustering. In: Proceedings of the 2005 SIAM International Conference on Data Mining, pp. 606–610. Society for Industrial and Applied Mathematics (2005)Google Scholar
- 11.Xu, W., Liu, X., Gong, Y.: Document clustering based on NON-negative matrix factorization. In: Proceedings in ACM SIGIR, pp. 267–273 (2003)Google Scholar
- 12.Yang, C.F., Ye, M., Zhao, J.: Document clustering based on non-negative sparse matrix factorization. In: International Conference on Advances in Natural Computation, pp. 557–563 (2005)Google Scholar
- 13.Kanjani, K.: Parallel Non Negative Matrix Factorization for Document Clustering. CPSC-659 (Parallel and Distributed Numerical Algorithms) Course. Texas A&M University, Tech. Rep. (2007)Google Scholar
- 15.Lovins, J.B.: Development of a stemming algorithm. Mech. Translat. Comp. Linguistics 11(1–2), 22–31 (1968)Google Scholar
- 16.Laxmi, H.V.T.E.V., Somasundaram, K.: 2HARS: heterogeneity-aware resource scheduling in grid environment using K-centroids clustering and PSO techniques. Int. J. Appl. Eng. Res. 10(7), 18047–18062 (2015)Google Scholar
- 17.Laxmi Lydia, E., Ben Swarup, M., Narsimham, C.: A disparateness–aware scheduling using K-centroids clustering and PSO techniques in hadoop cluster. Int. J. Adv. Found. Res. Comput. 2(12) (2015)Google Scholar
- 18.Laxmi Lydia, E.: Text mining with lucene and hadoop: document clustering with updated rules of NMF non-negative matrix factorization. Int. J. Pure Appl. Math. 118, 191–198 (2018)Google Scholar