Skip to main content
SpringerLink
Log in

Extending a hierarchical tiling arrays library to support sparse data partitioning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Layout methods for dense and sparse data are often seen as two separate problems with their own particular techniques. However, they are based on the same basic concepts. This paper studies how to integrate automatic data-layout and partition techniques for both dense and sparse data structures. In particular, we show how to include support for sparse matrices or graphs in Hitmap, a library for hierarchical tiling and automatic mapping of arrays. The paper shows that it is possible to offer a unique interface to work with both dense and sparse data structures. Thus, the programmer can use a single and homogeneous programming style, reducing the development effort and simplifying the use of sparse data structures in parallel computations. Our experimental evaluation shows that this integration of techniques can be effectively done without compromising performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Kennedy K, Koelbel C, Zima H (2007) The rise and fall of high performance Fortran. In: Proceedings of the third ACM SIGPLAN conference on history of programming languages (HOPL III). ACM Press, New York, pp 7.1–7.22 (New York, NY, USA)

    Google Scholar 

  2. Chandra R, Dagum L, Kohr D, Maydan D, McDonald J, Menon R (2001) Parallel programming in OpenMP, 1st edn. Kaufmann Los Altos

    Google Scholar 

  3. Carlson WW, Draper JM, Culler DE, Yelick K, Brooks E, Warren K (1999) Introduction to UPC and language specification

  4. Karypis G, Kumar V (1998) MeTiS—a software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices—version 4.0

  5. Pellegrini F (2010) PT-Scotch and libScotch 5.1 user’s guide

  6. Walshaw C (2002) The serial JOSTLE library user guide: version 3.0

  7. Chamberlain BL, Deitz SJ, Iten D, Choi S-E (2010) User-defined distributions and layouts in chapel: philosophy and framework. In: Proceedings of the 2nd USENIX conference on Hot topics in parallelism (HotPar’10), p 12. USENIX Association

    Google Scholar 

  8. González-Escribano A, van Gemund AJ, Cardeñoso Payo V, Portales-Fernández R, Caminero-Granja JA (2005) A preliminary nested-parallel framework to efficiently implement scientific applications. In: High performance computing for computational science (VECPAR 2004), pp 541–555

    Chapter  Google Scholar 

  9. González-Escribano A, Llanos DR (2009) Trasgo: a nested-parallel programming system. J Supercomput 58(2):226–234

    Article  Google Scholar 

  10. Trasgo Group (2011) Hitmap repository. http://trasgo.dcs.fi.uva.es/hitmap

  11. de Blas Cartón C, González-Escribano A, Llanos DR (2010) Effortless and efficient distributed data-partitioning in linear algebra. In: IEEE HPCC 2010, Sept. IEEE Press, Los Alamitos, pp 89–97

    Google Scholar 

  12. Fresno J, González-Escribano A, Llanos DR (2011) Automatic data partitioning applied to multigrid PDE solvers. In: 2011 19th international Euromicro conference on parallel, distributed and network-based processing. IEEE Press, Los Alamitos, pp 239–246

    Chapter  Google Scholar 

  13. Barrett R, Berry M, Chan TF, Demmel J, Donato JM, Dongarra J, Eijkhout V, Pozo R, Romine C, Van der Vorst H (1995) Templates for the solution of linear systems: building blocks for iterative methods, 2nd edn. Mathematics of computation, vol 64

    Google Scholar 

  14. Wilkinson B, Allen M (2004) Parallel programming: techniques and applications using networked workstations and parallel computers. Prentice Hall, New York

    Google Scholar 

  15. Davis TA, Hu Y (2011) The University of Florida sparse matrix collection. ACM Trans Math Softw 38(1):1–25

    MathSciNet  Google Scholar 

  16. Bikshandi G, Guo J, Hoeflinger D, Almasi G, Fraguela BB, Garzarán MJ, Padua D, von Praun C (2006) Programming for parallelism and locality with hierarchically tiled arrays. In: PPoPP 2006. ACM Press, Los Alamitos, pp 48–57

    Google Scholar 

  17. Chamberlain BL, Choi S-E, Deitz SJ, Iten D, Litvinov V (2011) Authoring user-defined domain maps in Chapel. In: Cray user group

    Google Scholar 

Download references

Acknowledgements

This research is partly supported by the Ministerio de Industria, Spain (CENIT MARTA, CENIT OASIS, CENIT OCEANLIDER), Ministerio de Ciencia y Tecnología, Spain (CAPAP-H3 network, TIN2010-12011-E, TIN2011-25639), and the HPC-EUROPA2 project (project number: 228398) with the support of the European Commission—Capacities Area—Research Infrastructures Initiative.

Author information

Authors and Affiliations

  1. Dpto. Informática, Universidad de Valladolid, Valladolid, Spain

    Javier Fresno, Arturo Gonzalez-Escribano & Diego R. Llanos

Authors
  1. Javier Fresno
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arturo Gonzalez-Escribano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diego R. Llanos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arturo Gonzalez-Escribano.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fresno, J., Gonzalez-Escribano, A. & Llanos, D.R. Extending a hierarchical tiling arrays library to support sparse data partitioning. J Supercomput 64, 59–68 (2013). https://doi.org/10.1007/s11227-012-0757-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-012-0757-y

Keywords

Search

Navigation