The Journal of Supercomputing

, Volume 64, Issue 1, pp 59–68 | Cite as

Extending a hierarchical tiling arrays library to support sparse data partitioning

  • Javier Fresno
  • Arturo Gonzalez-Escribano
  • Diego R. Llanos


Layout methods for dense and sparse data are often seen as two separate problems with their own particular techniques. However, they are based on the same basic concepts. This paper studies how to integrate automatic data-layout and partition techniques for both dense and sparse data structures. In particular, we show how to include support for sparse matrices or graphs in Hitmap, a library for hierarchical tiling and automatic mapping of arrays. The paper shows that it is possible to offer a unique interface to work with both dense and sparse data structures. Thus, the programmer can use a single and homogeneous programming style, reducing the development effort and simplifying the use of sparse data structures in parallel computations. Our experimental evaluation shows that this integration of techniques can be effectively done without compromising performance.


Data partition Layouts Distributed computing Sparse data 


  1. 1.
    Kennedy K, Koelbel C, Zima H (2007) The rise and fall of high performance Fortran. In: Proceedings of the third ACM SIGPLAN conference on history of programming languages (HOPL III). ACM Press, New York, pp 7.1–7.22 (New York, NY, USA) Google Scholar
  2. 2.
    Chandra R, Dagum L, Kohr D, Maydan D, McDonald J, Menon R (2001) Parallel programming in OpenMP, 1st edn. Kaufmann Los Altos Google Scholar
  3. 3.
    Carlson WW, Draper JM, Culler DE, Yelick K, Brooks E, Warren K (1999) Introduction to UPC and language specification Google Scholar
  4. 4.
    Karypis G, Kumar V (1998) MeTiS—a software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices—version 4.0 Google Scholar
  5. 5.
    Pellegrini F (2010) PT-Scotch and libScotch 5.1 user’s guide Google Scholar
  6. 6.
    Walshaw C (2002) The serial JOSTLE library user guide: version 3.0 Google Scholar
  7. 7.
    Chamberlain BL, Deitz SJ, Iten D, Choi S-E (2010) User-defined distributions and layouts in chapel: philosophy and framework. In: Proceedings of the 2nd USENIX conference on Hot topics in parallelism (HotPar’10), p 12. USENIX Association Google Scholar
  8. 8.
    González-Escribano A, van Gemund AJ, Cardeñoso Payo V, Portales-Fernández R, Caminero-Granja JA (2005) A preliminary nested-parallel framework to efficiently implement scientific applications. In: High performance computing for computational science (VECPAR 2004), pp 541–555 CrossRefGoogle Scholar
  9. 9.
    González-Escribano A, Llanos DR (2009) Trasgo: a nested-parallel programming system. J Supercomput 58(2):226–234 CrossRefGoogle Scholar
  10. 10.
    Trasgo Group (2011) Hitmap repository.
  11. 11.
    de Blas Cartón C, González-Escribano A, Llanos DR (2010) Effortless and efficient distributed data-partitioning in linear algebra. In: IEEE HPCC 2010, Sept. IEEE Press, Los Alamitos, pp 89–97 Google Scholar
  12. 12.
    Fresno J, González-Escribano A, Llanos DR (2011) Automatic data partitioning applied to multigrid PDE solvers. In: 2011 19th international Euromicro conference on parallel, distributed and network-based processing. IEEE Press, Los Alamitos, pp 239–246 CrossRefGoogle Scholar
  13. 13.
    Barrett R, Berry M, Chan TF, Demmel J, Donato JM, Dongarra J, Eijkhout V, Pozo R, Romine C, Van der Vorst H (1995) Templates for the solution of linear systems: building blocks for iterative methods, 2nd edn. Mathematics of computation, vol 64 Google Scholar
  14. 14.
    Wilkinson B, Allen M (2004) Parallel programming: techniques and applications using networked workstations and parallel computers. Prentice Hall, New York Google Scholar
  15. 15.
    Davis TA, Hu Y (2011) The University of Florida sparse matrix collection. ACM Trans Math Softw 38(1):1–25 MathSciNetGoogle Scholar
  16. 16.
    Bikshandi G, Guo J, Hoeflinger D, Almasi G, Fraguela BB, Garzarán MJ, Padua D, von Praun C (2006) Programming for parallelism and locality with hierarchically tiled arrays. In: PPoPP 2006. ACM Press, Los Alamitos, pp 48–57 Google Scholar
  17. 17.
    Chamberlain BL, Choi S-E, Deitz SJ, Iten D, Litvinov V (2011) Authoring user-defined domain maps in Chapel. In: Cray user group Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Javier Fresno
    • 1
  • Arturo Gonzalez-Escribano
    • 1
  • Diego R. Llanos
    • 1
  1. 1.Dpto. InformáticaUniversidad de ValladolidValladolidSpain

Personalised recommendations