Advertisement

A real-time distributed cluster storage optimization for massive data in internet of multimedia things

  • Yanning Zhang
  • Shuai Liu
Article
  • 1 Downloads

Abstract

With the rapid growth of massive data in the Internet of Multimedia Things, there are some problems of insufficient storage space and unbalanced load in the current methods. For the problem of massive real-time data storage, a distributed cluster storage optimization method is proposed. Considering the impact of replica cost and the generation of intermediate data on the replica layout, a replica generation and storage strategy is given with consideration of cost and storage space. In the data center, the data sensitivity and data access frequency is used as migration factors to achieve massive data migration. The improved collaborative evolution method is used to code the task scheduling particle swarm in massive data storage to obtain the optimal solution, and achieve massive real-time data distributed cluster storage for the Internet of things. The experimental results showed that the cost of data management by this method was only between 10 and 15, which showed that this method can effectively improve data access speed, reduce storage space, lower cost and better load balancing.

Keywords

Internet of multimedia things Massive data Real time Distributed optimization Cluster storage 

Notes

Acknowledgements

This work is supported by Programs of National Natural Science Foundation of China (No: 61502254), Program for Yong Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region (Grant No.NJYT-18-B10), Open Funds of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education (Grant No.93K172018K07. Research on mass real time data storage system of Internet of things (No. CJGX2016-KYYZK003).

We want to thank Prof Li from Jilin University for his carefully checking to this paper.

References

  1. 1.
    Chen B, Kai MT, Washio T (2015) Half-space mass: a maximally robust and efficient data depth method. Mach Learn 100(2-3):677–699MathSciNetCrossRefGoogle Scholar
  2. 2.
    Cherubini G, Jelitto J, Venkatesan V (2016) Cognitive storage for big data. Computer 49(4):43–51CrossRefGoogle Scholar
  3. 3.
    El-Rabiaey MA, Areed NFF, Obayya SSA (2016) Novel plasmonic data storage based on nematic liquid crystal layers. J Lightwave Technol 34(16):3726–3732CrossRefGoogle Scholar
  4. 4.
    Guo HY, Fang J, Li D (2017) A multi-source streaming data real-time storage system based on load balance. Computer Engineering and Science 39(4):641–647Google Scholar
  5. 5.
    Ishii T, Shimada K, Hoshizawa T (2015) Analysis of vibration effects on holographic data storage system. Jpn J Appl Phys 54(9S):09MA04CrossRefGoogle Scholar
  6. 6.
    Kaneko K, Kawamoto Y, Nishiyama H (2015) An efficient utilization of intermittent surface–satellite optical links by using mass storage device embedded in satellites. Perform Eval 87(C):37–46CrossRefGoogle Scholar
  7. 7.
    Kang LL, Wang CY (2018) Research on storage and sharing strategy of massive heterogeneous data in internet of things. The digital world, no. 2, pp 135Google Scholar
  8. 8.
    Li YL, Chang ZQ (2017) Mobile cloud data storage based on Gibbs sampling and probability distribution estimation. Comput Eng 43(1):13–19Google Scholar
  9. 9.
    Lin ZG, Zhang XH, Liu YP et al (2016) Snake-like slot time data storage algorithm based on storage threshold. Comput Eng 42(12):32–38Google Scholar
  10. 10.
    Liu S, Fu W, Deng H (2013a) Distributional fractal creating algorithm in parallel environment. Int J Distrib Sens NetwGoogle Scholar
  11. 11.
    Liu S, Fu W, Zhao W (2013b) A novel fusion method by static and moving facial capture. Math Probl EngGoogle Scholar
  12. 12.
    Ma L, Yang HX, Liu JP (2016) User privacy data storage method under big data environment. Computer Simulation 33(2):465–468Google Scholar
  13. 13.
    Pan Z, Liu S, W F (2017) A review of visual moving target tracking. Multimedia Tools and Applications 76(16):16989–17018CrossRefGoogle Scholar
  14. 14.
    Wijetunge CD, Saeed I, Boughton BA (2015) Exims an improved data analysis pipeline based on a new peak picking method for exploring imaging mass spectrometry data. Bioinformatics 31(19):3198–3206CrossRefGoogle Scholar
  15. 15.
    Xu Z, Schrama E, Wal WVD (2015) Optimization of regional constraints for estimating the Greenland mass balance with grace level-2 data. Geophys J Int 202(1):381–393CrossRefGoogle Scholar
  16. 16.
    Yang G, Liu S (2014) Distributed cooperative algorithm for k-M set with negative integer k by fractal symmetrical property. International Journal of Distributed Sensor NetworksGoogle Scholar
  17. 17.
    Yang CT, Shih WC, Huang CL (2016) On construction of a distributed data storage system in cloud. Computing 98(1-2):93–118MathSciNetCrossRefGoogle Scholar
  18. 18.
    Yu C, Wang J, Ling WQ (2016) Design and implementation of a hybrid storage and query system based on Hadoop for massive traffic data. Information Technology and Informatization (z1):82–86Google Scholar
  19. 19.
    Zhang Y, Bhamber R, Riba-Garcia I (2015) Streaming visualisation of quantitative mass spectrometry data based on a novel raw signal decomposition method. Proteomics 15(8):1419–1427CrossRefGoogle Scholar
  20. 20.
    Zhao D, Katsouras I, Asadi K (2016) Retention of intermediate polarization states in ferroelectric materials enabling memories for multi-bit data storage. Appl Phys Lett 108(23):1040–1090Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Telecommunication Engineering InstituteBeijing PolytechnicBeijingChina
  2. 2.College of Computer ScienceInner Mongolia UniversityHohhotChina
  3. 3.Inner Mongolia Key Laboratory of Social Computing and Data ProcessingHohhotChina
  4. 4.Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of EducationChangchunChina

Personalised recommendations