Performance optimization of combined variable-cost computations and I/O

  • Sorin G. Nastea
  • Tarek El-Ghazawi
  • Ophir Frieder
Systems and Applications
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1253)


For applications involving large data sets yielding variablecost computations, achieving both efficient I/O and load balancing may become particularly challenging though performance-critical tasks. In this work, we introduce a data scheduling approach that integrates several optimizing techniques, including dynamic allocation, prefetching, and asynchronous I/O and communications. We show that good scalability is obtained by both hiding the I/O latency and appropriately balancing the workloads. We use a statistical metric for data skewness to further improve the performance by adequately selecting among data-scheduling. We test our approach on sparse benchmark matrices for matrix-vector computations and show experimentally that our method can accurately predict the relative performance of different input/output schemes for a given data set and choose the best technique accordingly.


load balancing parallel I/O data distribution skewness sparse matrix computations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    Arunachalan, M., Choudhary, A., Rullman, B., Implementation and evaluation of prefetching in the Intel Paragon Parallel File System. Proceedings of the 10th Int'l. Parallel Processing Symposium, Honolulu. Hawaii, April 15–19, 1996.Google Scholar
  2. [2]
    Cheung, A. L., Reeves. A. P., Sparse data representation, Proceedings Scalable High Performance Computing Conference. 1992.Google Scholar
  3. [3]
    Duff, I. S., Sparse matrix test problems, ACM Transactions on Mathematical Software, Vol. 15, No. 1, 1–14, 1989.Google Scholar
  4. [4]
    Duff, I. S., Grimes, R.G., Lewis, J. G., User's guide for the Harwell-Boeing Sparse Matrix Collection, CERFACS Report TR/PA/92/86.1992.Google Scholar
  5. [5]
    Nastea, S. G., El-Ghazawi, T., Frieder, O., Parallel input/output impact on sparse matrix compression. Proceedings of the IEEE Data Compression Conference, Snowbird, Utah, 1996.Google Scholar
  6. [6]
    Nastea, S., El-Ghazawi, T., Frieder, O., A statisticallybased multi-algorithmic approach for load-balancing sparse matrix computations. Proceedings of the IEEE Frontiers of Massively Parallel Computations (Frontiers '96), Annapolis, Maryland, October 1996.Google Scholar
  7. [7]
    Nastea, S., Frieder, O., El-Ghazawi, T., “Load balancing in sparse matrix-vector multiplication”. Proceedings of the IEEE Symposium of Parallel and Distributed Processing, New Orleans, Louisiana, October 1996.Google Scholar
  8. [8]
    Paolini, G. V., Santangelo, P., A graphic tool for the structure of large sparse matrices, IBM Technical Report, ICE-0034 IBM ECSEC Rome (1989).Google Scholar
  9. [9]
    Park, S. C., Draayer, J. P., Zheng, S. Q., Fast sparse matrix multiplication, Computer Physics Communications, Vol. 70, 1992.Google Scholar
  10. [10]
    Peters, A., Sparse matrix vector multiplication technique on the IBM 3090 VP, Parallel Computing 17, 1991.Google Scholar
  11. [11]
    Petiton, S., Saad, Y., Wu, K., Ferng, W., Basic Sparse matrix computations on the CM-5, International Journal of Modern Physics C vol. 4, No. 1, 63–83, 1993.Google Scholar
  12. [12]
    Nitzberg, B., Fineberg, S. A., Parallel I/O on Highly Parallel Systems, Tutorial notes, Proceedings “Supercomputing '94”, Washington D.C., 1994.Google Scholar
  13. [13]
    Rothberg, E., Schreiber, R., Improved load balancing in parallel sparse Choleski factorization, “Supercomputing '94”, Washington D.C., 1994.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1997

Authors and Affiliations

  • Sorin G. Nastea
    • 1
  • Tarek El-Ghazawi
    • 1
  • Ophir Frieder
    • 2
  1. 1.Department of Electr. Eng. and Comp. ScienceThe George Washington UniversityWashington, D.C.
  2. 2.Department of Computer ScienceFlorida Institute of TechnologyMelbourne

Personalised recommendations