Large-Scale Scientific Irregular Computing on Clusters and Grids

  • Peter Brezany
  • Marian Bubak
  • Maciej Malawski
  • Katarzyna Zajcac
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2329)


Data sets involved in many scientific applications are often too massive to fit into main memory of even the most powerful computers and therefore they must reside on disk, and thus communication between internal and external memory, and not actual computation time, becomes the bottleneck in the computation. The most challenging are scientific and engineering applications that involve irregular (unstructured) computing phases. This paper discusses an integrated approach, namely ideas, techniques, concepts and software architecture for implementing such data intensive applications on computational clusters and the computational Grid, an emerging computing infrastructure. The experimental performance results achieved on a cluster of PCs are included.


Grid clusters irregular problems out-of-core computing runtime library 


  1. 1.
    Bershad, B., Black, D., DeWitt, D., Gibson, G., Li, K., Peterson, L., and Snir, M.: Operating System Support for High-Performance Parallel I/O Systems. Technical Report CCSF-4, Scalable I/O Initiative (1994)Google Scholar
  2. 2.
    Brezany, P., et al.: Automatic Parallelization of the AVL FIRE Benchmark for a Distributed-Memory System. In: Dongarra, J. et al. (eds.): Proceedings of PARA95. LNCS Vol. 1041 (1996) 50–60Google Scholar
  3. 3.
    Brezany, P., Bubak, M., Malawski, M., and Zajcac, K.: Advanced Library Support for Irregular and Out-of-Core Parallel Computing. In: Hertzberger, B., Hoekstra, B., Williams, R. (eds.): Proc. Int. Conf. High Performance Computing and Networking, Amsterdam, June 25–27, 2001, LNCS Vol. 2110, Springer-Verlag (2001) 435–444Google Scholar
  4. 4.
    Bubak, M., Kurzyniec, D., Łuszczek, P.: A Versatile Support for Binding Native Code to Java. In: Bubak, M., Afsarmanesh, H., Williams, R., Hertzberger, B. (eds.): Proc. Int. Conf. HPCN Amsterdam, May 2000, LNCS Vol. 1823. Springer-Verlag (2000) 373–384Google Scholar
  5. 5.
    Bubak, M., Kurzyniec D., Luszczek, P.: Creating Java to Native Code Interfaces with Janet Extension. in: Proceedings of 1st SGI Users Conference, Cracow, Poland, October 2000, ACC Cyfronet UMM (2000) 283–294Google Scholar
  6. 6.
    Carns, P.H. et al.: PVFS: A Parallel File System for Linux Clusters. In: Proc. of the Externe Linux Track: 4th Annual Linux Showcase and Conference Oct. 2000Google Scholar
  7. 7.
    Buyya, R. (ed.): High Performance Cluster Computing, Vol. 1 and 2. Prentice Hall, New Jersey (2000)Google Scholar
  8. 8.
    Arnold, D., Agrawal, S., Blackford, S., Dongarra, J., Miller, M., Sagi, K., Shi, Z. and Vadhiyar, S.: Users’ Guide to NetSolve V1.4 Tech. Report CS-01-467 University of Tenessee, Knoxville (2001)Google Scholar
  9. 9.
  10. 10.
    Foster, I. and Kesselman, C.: Globus: A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications, 11(2) (1997) 115–128CrossRefGoogle Scholar
  11. 11.
    Chervenak, A., Foster, I., Kesselman, C., Salisbury, C., and Tuecke, S.: The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. To be published in the Journal of Network and Computer Applications, 2001Google Scholar
  12. 12.
    Foster, I., Kesselman, C., and Tuecke, S.: The Anatomy of the Grid: Enabling Scalable Virtual Organizations. Intl. J. Supercomputer Applications, 15(3) (2001)Google Scholar
  13. 13.
  14. 14.
    Java Grande Forum.
  15. 15.
  16. 16.
    Kurzyniec, D.: Creating Java to Native Code Interfaces with Janet Extension. M. Sc. Thesis, Department of Computer Science at University of Mining and Metallurgy. Cracow, Poland, (August 2000)Google Scholar
  17. 17.
    Messina, P.: Distributed Supercomputing Applications. In: I. Foster and C. Kesselman (eds.), The Grid. Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publ., (1999)Google Scholar
  18. 18.
    Oldfield, R.: Summary of Existing and Developing Data Grids. White paper for the Remote Data Access group of the Global Grid Forum 1, Amsterdam, (March 2001)Google Scholar
  19. 19.
    Oldfield, R. and Kotz, D.: Armada: A parallel File System for Computational Grids. In Proceedings of the IEEE/ACM International Symposium on Cluster Computing and the Grid, Brisbane, Australia, (May 2001) 194–201Google Scholar
  20. 20.
    Ou, C.-W., Ranka, S.: SPRINT: Scalable Partitioning, Refinement, and INcremen-tal partitioning Techniques.
  21. 21.
    Ponnusamy, R. et al.: CHAOS Runtime Library. Techn. Report, University of Maryland, (May 1994)Google Scholar
  22. 22.
    Saltz, J., Crowley, K., Mirchandaney, R. and Berryman, H.: Run-time Scheduling and Execution of Loops on Message Passing Machines. Journal of Parallel and Distributed Computing, 8(2) (1990) 303–312CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Peter Brezany
    • 1
  • Marian Bubak
    • 2
    • 3
  • Maciej Malawski
    • 2
  • Katarzyna Zajcac
    • 2
  1. 1.Institute for Software ScienceUniversity of ViennaViennaAustria
  2. 2.AGHInstitute of Computer ScienceKrakówPoland
  3. 3.CYFRONETAcademic Computer CentreKrakówPoland

Personalised recommendations