Skip to main content

An Aggregate MapReduce Data Block Placement Strategy for Wireless IoT Edge Nodes in Smart Grid

Abstract

Big data analytics has simplified processing complexity of large dataset in a distributed environment. Many state-of-the-art platforms i.e. smart grid has adopted the processing structure of big data and manages a large volume of data through MapReduce paradigm at distribution ends. Thus, whenever a wireless IoT edge node bundles a sensor dataset into storage media, MapReduce agent performs analytics and generates output into the grid repository. This practice has efficiently reduced the consumption of resources in such a giant network and strengthens other components of the smart grid to perform data analytics through aggregate programming. However, it consumes an operational latency of accessing large dataset from a central repository. As we know that, smart grid processes I/O operations of multi-homing networks, therefore, it accesses large datasets for processing MapReduce jobs at wireless IoT edge nodes. As a result, aggregate MapReduce at wireless IoT edge node produces a network congestion and operational latency problem. To overcome this issue, we propose Wireless IoT Edge-enabled Block Replica Strategy (WIEBRS), that stores in-place, partition-based and multi-homing block replica to respective edge nodes. This reduces the delay latency of accessing datasets for aggregate MapReduce and increases the performance of the job in the smart grid. The simulation results show that WIEBRS effective decreases operational latency with an increment of aggregate MapReduce job performance in the smart grid.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  1. LaValle, S., et al. (2011). Big data, analytics and the path from insights to value. MIT Sloan Management Review, 52(2), 21.

    Google Scholar 

  2. Cloudera. (2016). The modern platform for data management and analytics. Cloudera. http://www.cloudera.com/. Accessed April 27, 2016.

  3. M. Technologies. (2016). Featured customers. https://www.mapr.com/. Accessed April 27, 2017.

  4. Welcome to Apache Hadoop!. (2014). http://hadoop.apache.org/. Accessed April 27, 2017.

  5. Apache Hadoop 2.7.2 Apache Hadoop YARN. (2016). https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/YARN.html. Accessed April 27, 2017.

  6. Apache Hadoop 2.7.2 HDFS users guide. (2016). https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html. Accessed April 27, 2017.

  7. Apache Hadoop 2.7.2 MapReduce Tutorial. (2016). https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html. Accessed April 27, 2017.

  8. Karun, A. K., & Chitharanjan, K. (2013). A review on hadoopHDFS infrastructure extensions. In 2013 IEEE conference on information and communication technologies (ICT). IEEE.

  9. Tsuruoka, Y. (2016). Cloud computing-current status and future directions. Journal of Information Processing, 24(2), 183–194.

    Article  Google Scholar 

  10. Tuballa, M. L., & Abundo, M. L. (2016). A review of the development of Smart Grid technologies. Renewable and Sustainable Energy Reviews, 59, 710–725.

    Article  Google Scholar 

  11. Gungor, V. C., et al. (2011). Smart grid technologies: Communication technologies and standards. IEEE transactions on Industrial Informatics, 7(4), 529–539.

    Article  Google Scholar 

  12. Bera, S., Misra, S., & Rodrigues, J. J. P. C. (2015). Cloud computing applications for smart grid: A survey. IEEE Transactions on Parallel and Distributed Systems, 26(5), 1477–1494.

    Article  Google Scholar 

  13. Spivak, A., & Nasonov, D. (2016). Data preloading and data placement for MapReduce performance improving. Procedia Computer Science, 101, 379–387.

    Article  Google Scholar 

  14. Maheshwari, N., Nanduri, R., & Varma, V. (2012). Dynamic energy efficient data placement and cluster reconfiguration algorithm for mapreduce framework. Future Generation Computer Systems, 28(1), 119–127.

    Article  Google Scholar 

  15. Li, H., Ota, K., & Dong, M. (2018). Learning IoT in edge: Deep learning for the internet of things with edge computing. IEEE Network, 32(1), 96–101.

    Article  Google Scholar 

  16. Chowdhery, A., et al. (2018). Urban IoT edge analytics. In A. Rahmani, P. Liljeberg, J.-S. Preden & A. Jantsch (Eds.), Fog computing in the internet of things (pp. 101–120). Cham: Springer.

  17. Zhang, D., Shou, Y., & Xu, J. (2018). A mapreduce-based approach for shortest path problem in road networks. Journal of Ambient Intelligence and Humanized Computing, 9(46), 1–9.

    Google Scholar 

  18. Lu, Z., et al. (2017). IoTDeM: An IoT Big Data-oriented MapReduce performance prediction extended model in multiple edge clouds. Journal of Parallel and Distributed Computing, 118, 316–327.

    Article  Google Scholar 

  19. Sharma, S. K., & Wang, X. (2017). Live data analytics with collaborative edge and cloud processing in wireless IoT networks. IEEE Access, 5, 4621–4635.

    Article  Google Scholar 

  20. José, M., & Hutchison, D. (2018). Game theory for multi-access edge computing: Survey, use cases, and future trends. In IEEE Communications Surveys & Tutorials.

  21. Qureshi, N. M. F., Shin, D. R., Siddiqui, I. F., & Chowdhry, B. S. (2017). Storage-tag-aware scheduler for hadoop cluster. IEEE Access, 5, 13742–13755.

    Article  Google Scholar 

  22. (2016) RDP: A storage-tier-aware Robust Data Placement strategy for Hadoop in a Cloud-based Heterogeneous Environment. KSII Transactions on Internet and Information Systems 10(9)

  23. Abbas, A., Farah Siddiqui, I., Lee, S. U., Kashif Bashir, A., Ejaz, W., & Qureshi, N. M. F. (2018). Multi-objective optimum solutions for IoT-based feature models of software product line. IEEE Access, 6, 12228–12239.

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1A6A3A11932892).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Nawab Muhammad Faseeh Qureshi.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qureshi, N.M.F., Siddiqui, I.F., Unar, M.A. et al. An Aggregate MapReduce Data Block Placement Strategy for Wireless IoT Edge Nodes in Smart Grid. Wireless Pers Commun 106, 2225–2236 (2019). https://doi.org/10.1007/s11277-018-5936-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11277-018-5936-6

Keywords

  • Wireless IoT edge node
  • HDFS
  • Smart grid
  • Hadoop
  • Aggregate MapReduce block placement