A real-time distributed cluster storage optimization for massive data in internet of multimedia things
- 19 Downloads
With the rapid growth of massive data in the Internet of Multimedia Things, there are some problems of insufficient storage space and unbalanced load in the current methods. For the problem of massive real-time data storage, a distributed cluster storage optimization method is proposed. Considering the impact of replica cost and the generation of intermediate data on the replica layout, a replica generation and storage strategy is given with consideration of cost and storage space. In the data center, the data sensitivity and data access frequency is used as migration factors to achieve massive data migration. The improved collaborative evolution method is used to code the task scheduling particle swarm in massive data storage to obtain the optimal solution, and achieve massive real-time data distributed cluster storage for the Internet of things. The experimental results showed that the cost of data management by this method was only between 10 and 15, which showed that this method can effectively improve data access speed, reduce storage space, lower cost and better load balancing.
KeywordsInternet of multimedia things Massive data Real time Distributed optimization Cluster storage
This work is supported by Programs of National Natural Science Foundation of China (No: 61502254), Program for Yong Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region (Grant No.NJYT-18-B10), Open Funds of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education (Grant No.93K172018K07. Research on mass real time data storage system of Internet of things (No. CJGX2016-KYYZK003).
We want to thank Prof Li from Jilin University for his carefully checking to this paper.
- 4.Guo HY, Fang J, Li D (2017) A multi-source streaming data real-time storage system based on load balance. Computer Engineering and Science 39(4):641–647Google Scholar
- 7.Kang LL, Wang CY (2018) Research on storage and sharing strategy of massive heterogeneous data in internet of things. The digital world, no. 2, pp 135Google Scholar
- 8.Li YL, Chang ZQ (2017) Mobile cloud data storage based on Gibbs sampling and probability distribution estimation. Comput Eng 43(1):13–19Google Scholar
- 9.Lin ZG, Zhang XH, Liu YP et al (2016) Snake-like slot time data storage algorithm based on storage threshold. Comput Eng 42(12):32–38Google Scholar
- 10.Liu S, Fu W, Deng H (2013a) Distributional fractal creating algorithm in parallel environment. Int J Distrib Sens NetwGoogle Scholar
- 11.Liu S, Fu W, Zhao W (2013b) A novel fusion method by static and moving facial capture. Math Probl EngGoogle Scholar
- 12.Ma L, Yang HX, Liu JP (2016) User privacy data storage method under big data environment. Computer Simulation 33(2):465–468Google Scholar
- 16.Yang G, Liu S (2014) Distributed cooperative algorithm for k-M set with negative integer k by fractal symmetrical property. International Journal of Distributed Sensor NetworksGoogle Scholar
- 18.Yu C, Wang J, Ling WQ (2016) Design and implementation of a hybrid storage and query system based on Hadoop for massive traffic data. Information Technology and Informatization (z1):82–86Google Scholar
- 20.Zhao D, Katsouras I, Asadi K (2016) Retention of intermediate polarization states in ferroelectric materials enabling memories for multi-bit data storage. Appl Phys Lett 108(23):1040–1090Google Scholar