Abstract
Big data analytics has simplified processing complexity of large dataset in a distributed environment. Many state-of-the-art platforms i.e. smart grid has adopted the processing structure of big data and manages a large volume of data through MapReduce paradigm at distribution ends. Thus, whenever a wireless IoT edge node bundles a sensor dataset into storage media, MapReduce agent performs analytics and generates output into the grid repository. This practice has efficiently reduced the consumption of resources in such a giant network and strengthens other components of the smart grid to perform data analytics through aggregate programming. However, it consumes an operational latency of accessing large dataset from a central repository. As we know that, smart grid processes I/O operations of multi-homing networks, therefore, it accesses large datasets for processing MapReduce jobs at wireless IoT edge nodes. As a result, aggregate MapReduce at wireless IoT edge node produces a network congestion and operational latency problem. To overcome this issue, we propose Wireless IoT Edge-enabled Block Replica Strategy (WIEBRS), that stores in-place, partition-based and multi-homing block replica to respective edge nodes. This reduces the delay latency of accessing datasets for aggregate MapReduce and increases the performance of the job in the smart grid. The simulation results show that WIEBRS effective decreases operational latency with an increment of aggregate MapReduce job performance in the smart grid.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
LaValle, S., et al. (2011). Big data, analytics and the path from insights to value. MIT Sloan Management Review, 52(2), 21.
Cloudera. (2016). The modern platform for data management and analytics. Cloudera. http://www.cloudera.com/. Accessed April 27, 2016.
M. Technologies. (2016). Featured customers. https://www.mapr.com/. Accessed April 27, 2017.
Welcome to Apache Hadoop!. (2014). http://hadoop.apache.org/. Accessed April 27, 2017.
Apache Hadoop 2.7.2 Apache Hadoop YARN. (2016). https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/YARN.html. Accessed April 27, 2017.
Apache Hadoop 2.7.2 HDFS users guide. (2016). https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html. Accessed April 27, 2017.
Apache Hadoop 2.7.2 MapReduce Tutorial. (2016). https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html. Accessed April 27, 2017.
Karun, A. K., & Chitharanjan, K. (2013). A review on hadoopHDFS infrastructure extensions. In 2013 IEEE conference on information and communication technologies (ICT). IEEE.
Tsuruoka, Y. (2016). Cloud computing-current status and future directions. Journal of Information Processing, 24(2), 183–194.
Tuballa, M. L., & Abundo, M. L. (2016). A review of the development of Smart Grid technologies. Renewable and Sustainable Energy Reviews, 59, 710–725.
Gungor, V. C., et al. (2011). Smart grid technologies: Communication technologies and standards. IEEE transactions on Industrial Informatics, 7(4), 529–539.
Bera, S., Misra, S., & Rodrigues, J. J. P. C. (2015). Cloud computing applications for smart grid: A survey. IEEE Transactions on Parallel and Distributed Systems, 26(5), 1477–1494.
Spivak, A., & Nasonov, D. (2016). Data preloading and data placement for MapReduce performance improving. Procedia Computer Science, 101, 379–387.
Maheshwari, N., Nanduri, R., & Varma, V. (2012). Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Generation Computer Systems, 28(1), 119–127.
Li, H., Ota, K., & Dong, M. (2018). Learning IoT in edge: Deep learning for the internet of things with edge computing. IEEE Network, 32(1), 96–101.
Chowdhery, A., et al. (2018). Urban IoT edge analytics. In A. Rahmani, P. Liljeberg, J.-S. Preden & A. Jantsch (Eds.), Fog computing in the internet of things (pp. 101–120). Cham: Springer.
Zhang, D., Shou, Y., & Xu, J. (2018). A mapreduce-based approach for shortest path problem in road networks. Journal of Ambient Intelligence and Humanized Computing, 9(46), 1–9.
Lu, Z., et al. (2017). IoTDeM: An IoT Big Data-oriented MapReduce performance prediction extended model in multiple edge clouds. Journal of Parallel and Distributed Computing, 118, 316–327.
Sharma, S. K., & Wang, X. (2017). Live data analytics with collaborative edge and cloud processing in wireless IoT networks. IEEE Access, 5, 4621–4635.
José, M., & Hutchison, D. (2018). Game theory for multi-access edge computing: Survey, use cases, and future trends. In IEEE Communications Surveys & Tutorials.
Qureshi, N. M. F., Shin, D. R., Siddiqui, I. F., & Chowdhry, B. S. (2017). Storage-tag-aware scheduler for hadoop cluster. IEEE Access, 5, 13742–13755.
(2016) RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment. KSII Transactions on Internet and Information Systems 10(9)
Abbas, A., Farah Siddiqui, I., Lee, S. U., Kashif Bashir, A., Ejaz, W., & Qureshi, N. M. F. (2018). Multi-objective optimum solutions for IoT-based feature models of software product line. IEEE Access, 6, 12228–12239.
Acknowledgements
This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1A6A3A11932892).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Qureshi, N.M.F., Siddiqui, I.F., Unar, M.A. et al. An Aggregate MapReduce Data Block Placement Strategy for Wireless IoT Edge Nodes in Smart Grid. Wireless Pers Commun 106, 2225–2236 (2019). https://doi.org/10.1007/s11277-018-5936-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11277-018-5936-6