Abstract
With the rapid growth of massive data in the Internet of Multimedia Things, there are some problems of insufficient storage space and unbalanced load in the current methods. For the problem of massive real-time data storage, a distributed cluster storage optimization method is proposed. Considering the impact of replica cost and the generation of intermediate data on the replica layout, a replica generation and storage strategy is given with consideration of cost and storage space. In the data center, the data sensitivity and data access frequency is used as migration factors to achieve massive data migration. The improved collaborative evolution method is used to code the task scheduling particle swarm in massive data storage to obtain the optimal solution, and achieve massive real-time data distributed cluster storage for the Internet of things. The experimental results showed that the cost of data management by this method was only between 10 and 15, which showed that this method can effectively improve data access speed, reduce storage space, lower cost and better load balancing.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Chen B, Kai MT, Washio T (2015) Half-space mass: a maximally robust and efficient data depth method. Mach Learn 100(2-3):677–699
Cherubini G, Jelitto J, Venkatesan V (2016) Cognitive storage for big data. Computer 49(4):43–51
El-Rabiaey MA, Areed NFF, Obayya SSA (2016) Novel plasmonic data storage based on nematic liquid crystal layers. J Lightwave Technol 34(16):3726–3732
Guo HY, Fang J, Li D (2017) A multi-source streaming data real-time storage system based on load balance. Computer Engineering and Science 39(4):641–647
Ishii T, Shimada K, Hoshizawa T (2015) Analysis of vibration effects on holographic data storage system. Jpn J Appl Phys 54(9S):09MA04
Kaneko K, Kawamoto Y, Nishiyama H (2015) An efficient utilization of intermittent surface–satellite optical links by using mass storage device embedded in satellites. Perform Eval 87(C):37–46
Kang LL, Wang CY (2018) Research on storage and sharing strategy of massive heterogeneous data in internet of things. The digital world, no. 2, pp 135
Li YL, Chang ZQ (2017) Mobile cloud data storage based on Gibbs sampling and probability distribution estimation. Comput Eng 43(1):13–19
Lin ZG, Zhang XH, Liu YP et al (2016) Snake-like slot time data storage algorithm based on storage threshold. Comput Eng 42(12):32–38
Liu S, Fu W, Deng H (2013a) Distributional fractal creating algorithm in parallel environment. Int J Distrib Sens Netw
Liu S, Fu W, Zhao W (2013b) A novel fusion method by static and moving facial capture. Math Probl Eng
Ma L, Yang HX, Liu JP (2016) User privacy data storage method under big data environment. Computer Simulation 33(2):465–468
Pan Z, Liu S, W F (2017) A review of visual moving target tracking. Multimedia Tools and Applications 76(16):16989–17018
Wijetunge CD, Saeed I, Boughton BA (2015) Exims an improved data analysis pipeline based on a new peak picking method for exploring imaging mass spectrometry data. Bioinformatics 31(19):3198–3206
Xu Z, Schrama E, Wal WVD (2015) Optimization of regional constraints for estimating the Greenland mass balance with grace level-2 data. Geophys J Int 202(1):381–393
Yang G, Liu S (2014) Distributed cooperative algorithm for k-M set with negative integer k by fractal symmetrical property. International Journal of Distributed Sensor Networks
Yang CT, Shih WC, Huang CL (2016) On construction of a distributed data storage system in cloud. Computing 98(1-2):93–118
Yu C, Wang J, Ling WQ (2016) Design and implementation of a hybrid storage and query system based on Hadoop for massive traffic data. Information Technology and Informatization (z1):82–86
Zhang Y, Bhamber R, Riba-Garcia I (2015) Streaming visualisation of quantitative mass spectrometry data based on a novel raw signal decomposition method. Proteomics 15(8):1419–1427
Zhao D, Katsouras I, Asadi K (2016) Retention of intermediate polarization states in ferroelectric materials enabling memories for multi-bit data storage. Appl Phys Lett 108(23):1040–1090
Acknowledgements
This work is supported by Programs of National Natural Science Foundation of China (No: 61502254), Program for Yong Talents of Science and Technology in Universities of Inner Mongolia Autonomous Region (Grant No.NJYT-18-B10), Open Funds of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education (Grant No.93K172018K07. Research on mass real time data storage system of Internet of things (No. CJGX2016-KYYZK003).
We want to thank Prof Li from Jilin University for his carefully checking to this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Zhang, Y., Liu, S. A real-time distributed cluster storage optimization for massive data in internet of multimedia things. Multimed Tools Appl 78, 5479–5492 (2019). https://doi.org/10.1007/s11042-018-7006-1
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-7006-1