Some Novelties in Map Reducing Techniques to Retrieve and Analyze Big Data for Effective Processing

Abstract

As we are living in the era of digital data, it has become necessary to make use of it in intelligent ways. The data generation is not only increasing, but the rate with which data generates is also increasing. Huge data obtained from many sources can be processed and analyzed so as to get useful information. But, the problem is with volume of data, velocity with which data increases, and also different variety and complex structure of data. Storing such large amount of data and the process of retrieving huge data when required are time consuming. One of the solutions for effective processing of Big Data is parallel processing. The software solutions like Hadoop provides way to store and also to implement parallel processing of Big Data. In most of the situations, Big Data cannot be stored in a single system. Distributed File System that can run on different clusters can be used to process Big Data. By using MapReduce model, large dataset can be computed on commodity hardware clusters. This paper presents a novel work of implementing MapReduce technique to analyze and retrieve data. An attempt is made to retrieve data by adopting MapReduce technique. A task is divided into number of sub-tasks, and these sub-tasks can be processed simultaneously by different processors or number of commodity hardware. A novel and effective way of implementing MapReduce is represented in this paper. In other words, this work examines the method and the outcome of MapReduce technique, which is a solution to the problem of processing huge amount of data.