Efficient Multidimensional Data Redistribution for Resizable Parallel Computations

  • Rajesh Sudarsan
  • Calvin J. Ribbens
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4742)


Traditional parallel schedulers running on cluster supercomputers support only static scheduling, where the number of processors allocated to an application remains fixed throughout the execution of the job. This results in under-utilization of idle system resources thereby decreasing overall system throughput. In our research, we have developed a prototype framework called ReSHAPE, which supports dynamic resizing of parallel MPI applications executing on distributed memory platforms. The resizing library in ReSHAPE includes support for releasing and acquiring processors and efficiently redistributing application state to a new set of processors. In this paper, we derive an algorithm for redistributing two-dimensional block-cyclic arrays from P to Q processors, organized as 2-D processor grids. The algorithm ensures a contention-free communication schedule for data redistribution if P r  ≤ Q r and P c  ≤ Q c . In other cases, the algorithm implements circular row and column shifts on the communication schedule to minimize node contention.


Dynamic scheduling Dynamic resizing Data redistribution Dynamic resource management process remapping resizable applications 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Blackford, L.S., Choi, J., Cleary, A., D’Azevedo, E., Demmel, J., Dhillon, I., Dongarra, J., Hammerling, S., Henry, G., Petitet, A., Stanley, K., Walker, D., Whaley, R.C.: ScaLAPACK User’s Guide, SIAM, Philadelphia (1997)Google Scholar
  2. 2.
    Chung, Y.C., Hsu, C.H., Bai, S.W.: A Basic-Cycle Calculation Technique for Efficient Dynamic Data Redistribution. IEEE Trans. Parallel Distrib. Syst. 9, 359–377 (1998)CrossRefGoogle Scholar
  3. 3.
    Desprez, F., Dongarra, J., Petitet, A., Randriamaro, C., Robert, Y.: Scheduling Block-Cyclic Array Redistribution. In: Proceedings of the Conference ParCo 1997, vol. 12, pp. 227–234 (1998)Google Scholar
  4. 4.
    Guo, M., Pan, Y.: Improving communication scheduling for array redistribution. J. Parallel Distrib. Comput. 65, 553–563 (2005)zbMATHCrossRefGoogle Scholar
  5. 5.
    Hsu, C.H., Chung, Y.C., Yang, D.L., Dow, C.R.: A Generalized Processor Mapping Technique for Array Redistribution. IEEE Trans. Parallel Distrib. Syst. 12, 743–757 (2001)CrossRefGoogle Scholar
  6. 6.
    Kalns, E.T., Ni, L.M.: Processor Mapping Techniques Toward Efficient Data Redistribution. IEEE Trans. Parallel Distrib. Syst. 6, 1234–1247 (1995)CrossRefGoogle Scholar
  7. 7.
    Kaushik, S.D., Huang, C.H., Johnson, R.W., Sadayappan, P.: An approach to communication-efficient data redistribution. In: ICS 1994: Proceedings of the 8th international conference on Supercomputing, pp. 364–373 (1994)Google Scholar
  8. 8.
    Lim, Y.W., Bhat, P.B., Prasanna, V.K.: Efficient Algorithms for Block-Cyclic Redistribution of Arrays. In: SPDP 1996. Proceedings of the 8th IEEE Symposium on Parallel and Distributed Processing, p. 74. IEEE Computer Society Press, Los Alamitos (1996)Google Scholar
  9. 9.
    Ramaswamy, S., Simons, B., Banerjee, P.: Optimizations for efficient array redistribution on distributed memory multicomputers. Journal of Parallel Distributed Computing 38, 217–228 (1996)zbMATHCrossRefGoogle Scholar
  10. 10.
    Thakur, R., Choudhary, A., Fox, G.: Runtime Array Redistribution in HPF Programs. In: Scalable High Performance Computing Conference, Knoxville, Tenn., pp. 309–316 (1994)Google Scholar
  11. 11.
    Thakur, R., Choudhary, A., Ramanujam, J.: Efficient Algorithms for Array Redistribution. IEEE Trans. Parallel Distrib. Syst. 7, 587–594 (1996)CrossRefGoogle Scholar
  12. 12.
    Walker, D.W., Otto, S.W.: Redistribution of block-cyclic data distributions using MPI. Concurrency: Practice and Experience 8, 707–728 (1996)CrossRefGoogle Scholar
  13. 13.
    Hsu, C.H., Bai, S.W., Chung, Y.C., Yang, C.S.: A Generalized Basic-Cycle Calculation Method for Efficient Array Redistribution. IEEE Trans. Parallel Distrib. Syst. 11, 1201–1216 (2000)CrossRefGoogle Scholar
  14. 14.
    Prylli, L., Tourancheau, B.: Efficient Block-Cyclic Data Redistribution. In: Fraigniaud, P., Mignotte, A., Bougé, L., Robert, Y. (eds.) Euro-Par 1996. LNCS, vol. 1123, pp. 155–164. Springer, Heidelberg (1996)Google Scholar
  15. 15.
    Lim, Y.W., Park, N., Prasanna, V.K.: Efficient Algorithms for Multi-dimensional Block-Cyclic Redistribution of Arrays. In: ICPP 1997: Proceedings of the international Conference on Parallel Processing, pp. 234–241 (1997)Google Scholar
  16. 16.
    Park, N., Prasanna, V.K., Raghavendra, C.S.: Efficient Algorithms for Block-Cyclic Array Redistribution Between Processor Sets. IEEE Transactions on Parallel and Distributed Systems 10, 1217–1240 (1999)CrossRefGoogle Scholar
  17. 17.
    Sudarsan, R., Ribbens, C.J.: ReSHAPE: A Framework for Dynamic Resizing and Scheduling of Homogeneous Applications in a Parallel Environment. Technical Report cs.DC/0703137, Computing Research Repository(CoRR) (2007)Google Scholar
  18. 18.
    Dongarra, J., Whaley, R.C.: A user’s guide to the BLACS v1.1. Technical Report CS-95-281, Computer Science Department, University of Tennesse, Knoxville, TN (also LAPACK Working Note #94) (1997)Google Scholar
  19. 19.
    MPICH2 v1.03 (2005), available from:

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Rajesh Sudarsan
    • 1
  • Calvin J. Ribbens
    • 1
  1. 1.Department of Computer Science, Virginia Tech, Blacksburg, VA 24061-0106 

Personalised recommendations