Abstract
Recent decades have seen the rapid development of cloud computing, resulting in a huge breakthrough for people to handle the data produced every second and everywhere. Meanwhile, data compression is becoming increasingly important, due to its great potential in benefiting both the network transportation and the storage. Based on the urgent demand in high-efficient compression method with balanced performance in both merits of compression time and ratio, this paper presents PLZMA, a parallel design of LZMA. Process-level and thread-level parallelisms are implemented according to the algorithm of LZMA, which have gained great improvement in compression time, while ensuring a fair compression ratio. Experimental results on real-world application showed that PLZMA is able to achieve more balanced performance over other famous methods. The parallel design is able to achieve a performance speedup of 8\(\times \) over the serial baseline, using 12 threads.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dean, J., Ghemawat, S.: Mapreduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Motley, C.F.: Telecommunication data compression apparatus and method, April 13 2004. US Patent 6,721,282
Yan, C., Zhang, Y., Dai, F., Li, L.: Highly parallel framework for HEVC motion estimation on many-core platform. In: Data Compression Conference (DCC), pp. 63–72. IEEE (2013)
Gan, L., Haohuan, F., Luk, W., Yang, C., Xue, W., Yang, G.: Solving mesoscale atmospheric dynamics using a reconfigurable dataflow architecture. IEEE Micro 37(4), 40–50 (2017)
Gan, L., Fu, H., Mencer, O., Luk, W., Yang, G.: Data flow computing in geoscience applications. Adv. Comput. 104, 125–158 (2017)
Burrows, M., Wheeler, D.J.: A block-sorting lossless data compression algorithm (1994)
Deutsch, P.L.: Deflate compressed data format specification version 1, 3 (1996)
Ziv, J., Lempel, A.: A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory 23(3), 337–343 (1977)
Gristwood, T., Fineran, P.C., Everson, L., Salmond, G.P.C.: PigZ, a TetR/AcrR family repressor, modulates secondary metabolism via the expression of a putative four-component resistance-nodulation-cell-division efflux pump, zrpadbc, in serratia sp. atcc 39006. Mol. Microbiol. 69(2), 418–435 (2008)
Adiego, J., Fuente, P.D.L.: Merging prediction by partial matching with structural contexts model, p. 522 (2004)
Berners-Lee, T., Fielding, R., Frystyk, H.: Hypertext transfer protocol-http/1.0. Technical report (1996)
Woods, J.: PPP deflate protocol (1996)
Boutell, T.: PNG (portable network graphics) specification version 1.0. (1997)
Deutsch, P., Gailly, J.-L.: Zlib compressed data format specification version 3.3. Technical report (1996)
Zhu, W., Xu, J., Ding, W., Shi, Y.: Adaptive LZMA-based coding for screen content. In: Picture Coding Symposium, pp. 373–376 (2013)
Kärkkäinen, J.: Fast BWT in small space by blockwise suffix sorting. Elsevier Science Publishers Ltd. (2007)
Culler, M., Dunfield, N.M., Weeks, J.R.: Snappy, a computer program for studying the geometry and topology of 3-manifolds (2017)
Pavlov, I.: Lzma sdk (software development kit) (2007)
Reinhold, L.M.: Quicklz website
Oberhumer, M.F.X.J.: Lzo-a real-time data compression library (2008). http://www.oberhumer.com/opensource/lzo/
Varsaki, A., Afendra, A.S., Vartholomatos, G., Tegos, G., Drainas, C.: Production of ice nuclei from two recombinant zymomonas mobilis strains employing the inaZ gene of pseudomonas syringae. Biotechnol. Lett. 20(7), 647–651 (1998)
Lembayung, W.: Comparative analysis on the izarc compression process and 7-zip (2011)
Acknowledgement
L. Gan, and J. Xu are supported by the National Natural Science Foundation of China (grant no. 61702297); and the China Postdoctoral Science Foundation (grant no. 2016M601031).
H. Fu, and X. Wang are supported by the National Key Research & Development Plan of China (grant no. 2017YFA0604500), the National Natural Science Foundation of China (grant no. 91530323, 41661134014, 41504040 and 61361120098); and the Tsinghua University Initiative Scientific Research Program (grant no. 20131089356).
G. Yang, and J. Yang are supported by the National Key Research & Development Plan of China (grant no. 2016YFA0602200).
X. Huang is supported by a grant from the State’s Key Project of Research and Development Plan (2016YFB0201100) and the National Natural Science Foundation of China (41375102).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, X. et al. (2018). PLZMA: A Parallel Data Compression Method for Cloud Computing. In: Vaidya, J., Li, J. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2018. Lecture Notes in Computer Science(), vol 11336. Springer, Cham. https://doi.org/10.1007/978-3-030-05057-3_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-05057-3_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-05056-6
Online ISBN: 978-3-030-05057-3
eBook Packages: Computer ScienceComputer Science (R0)