Abstract
Layout methods for dense and sparse data are often seen as two separate problems with their own particular techniques. However, they are based on the same basic concepts. This paper studies how to integrate automatic data-layout and partition techniques for both dense and sparse data structures. In particular, we show how to include support for sparse matrices or graphs in Hitmap, a library for hierarchical tiling and automatic mapping of arrays. The paper shows that it is possible to offer a unique interface to work with both dense and sparse data structures. Thus, the programmer can use a single and homogeneous programming style, reducing the development effort and simplifying the use of sparse data structures in parallel computations. Our experimental evaluation shows that this integration of techniques can be effectively done without compromising performance.
Similar content being viewed by others
References
Kennedy K, Koelbel C, Zima H (2007) The rise and fall of high performance Fortran. In: Proceedings of the third ACM SIGPLAN conference on history of programming languages (HOPL III). ACM Press, New York, pp 7.1–7.22 (New York, NY, USA)
Chandra R, Dagum L, Kohr D, Maydan D, McDonald J, Menon R (2001) Parallel programming in OpenMP, 1st edn. Kaufmann Los Altos
Carlson WW, Draper JM, Culler DE, Yelick K, Brooks E, Warren K (1999) Introduction to UPC and language specification
Karypis G, Kumar V (1998) MeTiS—a software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices—version 4.0
Pellegrini F (2010) PT-Scotch and libScotch 5.1 user’s guide
Walshaw C (2002) The serial JOSTLE library user guide: version 3.0
Chamberlain BL, Deitz SJ, Iten D, Choi S-E (2010) User-defined distributions and layouts in chapel: philosophy and framework. In: Proceedings of the 2nd USENIX conference on Hot topics in parallelism (HotPar’10), p 12. USENIX Association
González-Escribano A, van Gemund AJ, Cardeñoso Payo V, Portales-Fernández R, Caminero-Granja JA (2005) A preliminary nested-parallel framework to efficiently implement scientific applications. In: High performance computing for computational science (VECPAR 2004), pp 541–555
González-Escribano A, Llanos DR (2009) Trasgo: a nested-parallel programming system. J Supercomput 58(2):226–234
Trasgo Group (2011) Hitmap repository. http://trasgo.dcs.fi.uva.es/hitmap
de Blas Cartón C, González-Escribano A, Llanos DR (2010) Effortless and efficient distributed data-partitioning in linear algebra. In: IEEE HPCC 2010, Sept. IEEE Press, Los Alamitos, pp 89–97
Fresno J, González-Escribano A, Llanos DR (2011) Automatic data partitioning applied to multigrid PDE solvers. In: 2011 19th international Euromicro conference on parallel, distributed and network-based processing. IEEE Press, Los Alamitos, pp 239–246
Barrett R, Berry M, Chan TF, Demmel J, Donato JM, Dongarra J, Eijkhout V, Pozo R, Romine C, Van der Vorst H (1995) Templates for the solution of linear systems: building blocks for iterative methods, 2nd edn. Mathematics of computation, vol 64
Wilkinson B, Allen M (2004) Parallel programming: techniques and applications using networked workstations and parallel computers. Prentice Hall, New York
Davis TA, Hu Y (2011) The University of Florida sparse matrix collection. ACM Trans Math Softw 38(1):1–25
Bikshandi G, Guo J, Hoeflinger D, Almasi G, Fraguela BB, Garzarán MJ, Padua D, von Praun C (2006) Programming for parallelism and locality with hierarchically tiled arrays. In: PPoPP 2006. ACM Press, Los Alamitos, pp 48–57
Chamberlain BL, Choi S-E, Deitz SJ, Iten D, Litvinov V (2011) Authoring user-defined domain maps in Chapel. In: Cray user group
Acknowledgements
This research is partly supported by the Ministerio de Industria, Spain (CENIT MARTA, CENIT OASIS, CENIT OCEANLIDER), Ministerio de Ciencia y Tecnología, Spain (CAPAP-H3 network, TIN2010-12011-E, TIN2011-25639), and the HPC-EUROPA2 project (project number: 228398) with the support of the European Commission—Capacities Area—Research Infrastructures Initiative.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fresno, J., Gonzalez-Escribano, A. & Llanos, D.R. Extending a hierarchical tiling arrays library to support sparse data partitioning. J Supercomput 64, 59–68 (2013). https://doi.org/10.1007/s11227-012-0757-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-012-0757-y