Abstract
Many results from scientific calculations are large-scale sparse matrices. The results of simulating volcanic ash diffusion are also a sparse matrix, and the values are clustered because of the characteristics of ash diffusion. The cost to store or transmit scientific data is usually high because such data are large scale. In this paper, we suggest a new storage format that is more efficient for storing clustered sparse matrix. Coordinate values are compressed more in the proposed format by saving only the first key value of consecutive non-zero elements and its length. The performance of the new format is the best among existing similar formats on ash diffusion simulation data, and the compressed size of the resulting file is comparable to a ZIP file. Because the new format can be applied partially to the data part of Network Common Data Form (NetCDF) files only, its header information is still readable directly from the compressed file, unlike zipped files.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Folch, A., Costa, A., Macedonio, G.: FALL3D: a computational model for transport and deposition of volcanic ash. J. Comput. Geosci. 35(6), 1334–1342 (2009)
Unidata Network Common Data Form (NetCDF). http://www.unidata.ucar.edu/software/netcdf/
Rew, R., Davis, G.: NetCDF: an interface for scientific data access. J. Comput. Graphics Appl. 10(4), 76–82 (1990)
Saad, Y.: Iterative methods for sparse linear systems. Siam, 92–94 (2003)
Pountain, D.: Run-length encoding. J. Byte 12(6), 317–319 (1987)
Williams, S., Oliker, L., Vuduc, R., Shalf, J., Yelick, K., Demmel, J.: Optimization of sparse matrix-vector multiplication on emerging multicore platforms. J. Parallel Comput. 35(3), 178–194 (2009)
Bell, N., Garland, M.: Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, pp. 18. ACM (2009)
Su, B.Y., Keutzer, K.: clSpMV: A cross-platform OpenCL SpMV framework on GPUs. In: Proceedings of the 26th ACM international conference on Supercomputing, pp. 353–364. ACM (2012)
Feng, X., Jin, H., Zheng, R., Hu, K., Zeng, J., Shao, Z.: Optimization of sparse matrix-vector multiplication with variant CSR on GPUs. In: 17th International Conference on Parallel and Distributed Systems (ICPADS), pp. 165–172. IEEE (2011)
Koza, Z., Matyka, M., Szkoda, S., Miroslaw, L.: Compressed multiple-row storage format. CoRR, (2012)
Ortega, G., Vazquez, F., GarcÃa, I., Garzon, E.M.: Fastspmm: an efficient library for sparse matrix matrix product on gpus. J. Comput. 57(7), 968–979 (2014)
Vazquez, F., Ortega, G., Fernandez, J.J., GarcÃa, I., Garzon, E.M.: Fast sparse matrix matrix product based on ELLR-T and gpu computing. In: 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications (ISPA), pp. 669–674. IEEE, (2012)
U. of Florida. Sparse matrix collection, http://www.cise.ufl.edu/research/sparse/matrices/
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hwang, S., Choi, G., Heo, D. (2016). Compressing Method of NetCDF Files Containing Clustered Sparse Matrix. In: Park, J., Chao, HC., Arabnia, H., Yen, N. (eds) Advanced Multimedia and Ubiquitous Engineering. Lecture Notes in Electrical Engineering, vol 354. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-47895-0_12
Download citation
DOI: https://doi.org/10.1007/978-3-662-47895-0_12
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-47894-3
Online ISBN: 978-3-662-47895-0
eBook Packages: EngineeringEngineering (R0)