Controlling Concurrency and Expressing Synchronization in Charm++ Programs

  • Laxmikant V. Kale
  • Jonathan Lifflander
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8665)


Charm++ is a parallel programming system that evolved over the past 20 years to become a well-established system for programming parallel science and engineering applications, in addition to the combinatorial search applications with which it started. At its earliest point, the precursor to Charm++, the Chare Kernel, was a purely reactive specification, similar to most actor languages. This paper describes the evolution of a series of concurrency control mechanisms that have been deployed in Charm++ to tame this unrestricted concurrency in order to improve code clarity and/or to improve performance.


Load Balancer Controlling Concurrency Runtime System Remote Data Method Invocation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agha, G.: Actors: A Model of Concurrent Computation in Distributed Systems. MIT Press (1986)Google Scholar
  2. 2.
    Agha, G.A., Kim, W.: Actors: a unifying model for parallel and distributed computing. J. Syst. Archit. 45(15), 1263–1277 (1999)CrossRefGoogle Scholar
  3. 3.
    Antoniu, G., Bouge, L., Namyst, R.: An efficient and transparent thread migration scheme in the PM2 runtime system. In: Rolim, J.D.P. (ed.) IPPS-WS 1999 and SPDP-WS 1999. LNCS, vol. 1586, pp. 496–510. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  4. 4.
    Athas, W.C., Seitz, C.L.: Multicomputers: Message passing concurrent computers. IEEE Computer (August 1988)Google Scholar
  5. 5.
    Bisset, K., Aji, A., Marathe, M., Chun Feng, W.: High-performance biocomputing for simulating the spread of contagion over large contact networks. In: 2011 IEEE 1st International Conference on Computational Advances in Bio and Medical Sciences (ICCABS), pp. 26–32 (February 2011)Google Scholar
  6. 6.
    Booth, J.A.: Balancing priorities and load for state space search on large parallel machines. Master’s thesis, University of Illinois at Urbana-Champaign (2003)Google Scholar
  7. 7.
    Brunner, R., Phillips, J., Kalé, L.V.: Scalable molecular dynamics for large biomolecular systems. In: Proceedings of SuperComputing 2000 (2000)Google Scholar
  8. 8.
    Brunner, R.K., Kalé, L.V.: Handling application-induced load imbalance using parallel objects. In: Parallel and Distributed Computing for Symbolic and Irregular Applications, pp. 167–181. World Scientific Publishing (2000)Google Scholar
  9. 9.
    Caromel, D.: Abstract Control Types for Concurrency (Position Statement for the panel: How could object-oriented concepts and parallelism cohabit). In: O’Conner, L. (ed.) International Conference on Computer Languages (IEEE ICCL 1994), pp. 205–214. IEEE Computer Society Press (August 1993)Google Scholar
  10. 10.
    Tomlinson, C., Singh, V.: Inheritance and synchronization with enabled-sets. In: ACM OOPSLA, pp. 103–112 (1989)Google Scholar
  11. 11.
    Dongarra, J., Luszczek, P.: Introduction to the HPC Challenge Benchmark Suite. Technical Report UT-CS-05-544, University of Tennessee, Dept. of Computer Science (2005)Google Scholar
  12. 12.
    Gursoy, A., Kale, L.: Tolerating latency with dagger. In: Proceedings of the Eigth International Symposium on Computer and Information Sciences, Istanbul, Turkey (November 1993)Google Scholar
  13. 13.
    Gursoy, A., Kalé, L.: Dagger: Combining the Benefits of Synchronous and Asynchron ous Communication Styles. In: Proceedings of the 8th International Parallel Processing Symposium (April 1994)Google Scholar
  14. 14.
    Gursoy, A., Kalé, L.: Dagger: Combining the Benefits of Synchronous and Asynchronous Communication Styles. In: Siegel, H.G. (ed.) Proceedings of the 8th International Parallel Processing Symposium, Cancun, Mexico, pp. 590–596 (April 1994)Google Scholar
  15. 15.
    Halstead, R.: Multilisp: A Language for Concurrent Symbolic Computation. ACM Transactions on Programming Languages and Systems (October 1985)Google Scholar
  16. 16.
    Heroux, M.A., Doerfler, D.W., Crozier, P.S., Willenbring, J.M., Edwards, H.C., Williams, A., Rajan, M., Keiter, E.R., Thornquist, H.K., Numrich, R.W.: Improving performance via mini-applications. Technical report, Sandia National Laboratories (September 2009)Google Scholar
  17. 17.
    Hewitt, C., Bishop, P., Steiger, R.: A universal ACTOR formalism for artificial intelligence. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 235–245. SIAM (1973)Google Scholar
  18. 18.
    Huang, C., Kale, L.V.: Charisma: Orchestrating migratable parallel objects. In: Proceedings of IEEE International Symposium on High Performance Distributed Computing, HPDC (July 2007)Google Scholar
  19. 19.
    Huang, C., Zheng, G., Kumar, S., Kalé, L.V.: Performance Evaluation of Adaptive MPI. In: Proceedings of ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (March 2006)Google Scholar
  20. 20.
    Jetley, P., Gioachin, F., Mendes, C., Kale, L.V., Quinn, T.R.: Massively parallel cosmological simulations with ChaNGa. In: Proceedings of IEEE International Parallel and Distributed Processing Symposium 2008 (2008)Google Scholar
  21. 21.
    Kale, L.: The Chare Kernel parallel programming language and system. In: Proceedings of the International Conference on Parallel Processing, vol. II, pp. 17–25 (August 1990)Google Scholar
  22. 22.
    Kale, L., Arya, A., Bhatele, A., Gupta, A., Jain, N., Jetley, P., Lifflander, J., Miller, P., Sun, Y., Venkataraman, R., Wesolowski, L., Zheng, G.: Charm++ for productivity and performance: A submission to the 2011 HPC class II challenge. Technical Report 11-49, Parallel Programming Laboratory (November 2011)Google Scholar
  23. 23.
    Kale, L., Arya, A., Jain, N., Langer, A., Lifflander, J., Menon, H., Ni, X., Sun, Y., Totoni, E., Venkataraman, R., Wesolowski, L.: Migratable objects + active messages + adaptive runtime = productivity + performance a submission to 2012 HPC class II challenge. Technical Report 12-47, Parallel Programming Laboratory (November 2012)Google Scholar
  24. 24.
    Kale, L., Ramkumar, B., Saletore, V., Sinha, A.B.: Prioritization in parallel symbolic computing. In: Halstead Jr., R.H., Ito, T. (eds.) US/Japan WS 1992. LNCS, vol. 748, pp. 11–41. Springer, Heidelberg (1993)CrossRefGoogle Scholar
  25. 25.
    Kalé, L.V.: Parallel architectures for problem solving. PhD thesis, State Univ. of New York, Stony Brook, USA (1985)Google Scholar
  26. 26.
    Kalé, L.V.: Parallel execution of logic programs: the REDUCE-OR process model. In: Proceedings of Fourth International Conference on Logic Programming, pp. 616–632 (May 1987)Google Scholar
  27. 27.
    Kale, L.V.: Programming Models at Exascale: Adaptive Runtime Systems, Incomplete Simple Languages, and Interoperability. The International Journal of High Performance Computing Applications 23(4), 344–346 (2009)CrossRefGoogle Scholar
  28. 28.
    Kale, L.V., Bhandarkar, M.: Structured Dagger: A Coordination Language for Message-Driven Programming. In: Fraigniaud, P., Mignotte, A., Bougé, L., Robert, Y. (eds.) Euro-Par 1996. LNCS, vol. 1123, pp. 646–653. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  29. 29.
    Kale, L.V., Bhandarkar, M.: Structured Dagger: A Coordination Language for Message-Driven Programming. In: Fraigniaud, P., Mignotte, A., Bougé, L., Robert, Y. (eds.) Euro-Par 1996. LNCS, vol. 1123, pp. 646–653. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  30. 30.
    Bhandarkar, M.A., Brunner, R.K., Kalé, L.V.: Run-time Support for Adaptive Load Balancing. In: Rolim, J.D.P. (ed.) IPDPS 2000 Workshops. LNCS, vol. 1800, pp. 1152–1159. Springer, Heidelberg (2000)Google Scholar
  31. 31.
    Kale, L.V., Richards, B.H., Allen, T.D.: Efficient parallel graph coloring with prioritization. In: Queinnec, C., Halstead Jr., R.H., Ito, T. (eds.) PSLS 1995. LNCS, vol. 1068, pp. 190–208. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  32. 32.
    Kalé, L.V., Shu, W.: The Chare Kernel base language: Preliminary performance results. In: Proceedings of the 1989 International Conference on Parallel Processing, St. Charles, IL, pp. 118–121 (August 1989)Google Scholar
  33. 33.
    Keller, R., Lin, F., Tanaka, J.: Rediflow Multiprocessing. In: Digest of Papers COMPCON, Spring 1984, pp. 410–417 (February 1984)Google Scholar
  34. 34.
    Kumar, S.: Optimizing Communication for Massively Parallel Processing. PhD thesis, University of Illinois at Urbana-Champaign (May 2005)Google Scholar
  35. 35.
    Kumar, S., Shi, Y., Bohm, E., Kale, L.V.: Scalable, fine grain, parallelization of the car-parrinello ab initio molecular dynamics method. Technical report, UIUC, Dept. of Computer Science (2005)Google Scholar
  36. 36.
    Langer, A., Lifflander, J., Miller, P., Pan, K.-C., Kale, L.V., Ricker, P.: Scalable Algorithms for Distributed-Memory Adaptive Mesh Refinement. In: Proceedings of the 24th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD 2012), New York, USA (October 2012) (to appear)Google Scholar
  37. 37.
    Lawlor, O., Kalé, L.V.: Supporting dynamic parallel object arrays. In: Proceedings of ACM 2001 Java Grande/ISCOPE Conference, Stanford, CA, pp. 21–29 (June 2001)Google Scholar
  38. 38.
    Lifflander, J., Miller, P., Venkataraman, R., Arya, A., Jones, T., Kale, L.: Mapping dense lu factorization on multicore supercomputer nodes. In: Proceedings of IEEE International Parallel and Distributed Processing Symposium (May 2012)Google Scholar
  39. 39.
    Matsuoka, S., Yonezawa, A.: Analysis of Inheritance Anomaly in Object-Oriented Concurrent Languages. In: Agha, G., Wegner, P., Yonezawa, A. (eds.) Research Directions in Object-Based Concurrency. MIT Press (1993)Google Scholar
  40. 40.
    Ni, X., Meneses, E., Kalé, L.V.: Hiding checkpoint overhead in hpc applications with a semi-blocking algorithm. In: IEEE Cluster 2012, Beijing, China (September 2012)Google Scholar
  41. 41.
    Ramkumar, B., Kalé, L.V.: Compiled execution of the Reduce-Or process model on multiprocessors. In: The North American Conference on Logic Programming, pp. 313–331 (October 1989)Google Scholar
  42. 42.
    Krishnan, S., Kale, L.V.: A parallel array abstraction for data-driven objects. In: Proceedings of Parallel Object-Oriented Methods and Applications Conference, Santa Fe, NM (February 1996)Google Scholar
  43. 43.
    Sarood, O., Kalé, L.V.: A ‘cool’ load balancer for parallel applications. In: Proceedings of the 2011 ACM/IEEE conference on Supercomputing, Seattle, WA (November 2011)Google Scholar
  44. 44.
    Sinha, A., Kalé, L.: Information Sharing Mechanisms in Parallel Programs. In: Siegel, H. (ed.) Proceedings of the 8th International Parallel Processing Symposium, Cancun, Mexico, pp. 461–468 (April 1994)Google Scholar
  45. 45.
    Sinha, A.B., Kale, L.V., Ramkumar, B.: A dynamic and adaptive quiescence detection algorithm. Technical Report 93-11, Parallel Programming Laboratory, Department of Computer Science, University of Illinois, Urbana-Champaign (1993)Google Scholar
  46. 46.
    Taura, K., Matsuoka, S., Yonezawa, A.: An efficient implementation scheme of concurrent object-oriented languages on stock multicomputers. In: Proceedings of the 5th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. ACM SIGPLAN Notices (June 1993)Google Scholar
  47. 47.
    Tomlinson, C., Kim, W., Scheevel, M., Singh, V., Will, B., Agha, G.: Rosette: An object-oriented concurrent systems architecture. In: Proceedings of the 1988 ACM SIGPLAN Workshop on Object-based Concurrent Programming, OOPSLA/ECOOP 1988, pp. 91–93. ACM, New York (1988)CrossRefGoogle Scholar
  48. 48.
    Yelon, J., Kale, L.V.: Agents: An undistorted representation of problem structure. In: Huang, C.-H., Sadayappan, P., Banerjee, U., Gelernter, D., Nicolau, A., Padua, D.A. (eds.) LCPC 1995. LNCS, vol. 1033, pp. 551–565. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  49. 49.
    Yonezawa, A.: ABCL: An Object Oriented Concurrent System. MIT Press (1990)Google Scholar
  50. 50.
    Yonezawa, A., Briot, J.-P., Shibayama, E.: Object-oriented concurrent programming in ABCL/1. In: ACM SIGPLAN Notices, Proceedings of the OOPSLA 1986, vol. 21(11), pp. 258–268 (1986)Google Scholar
  51. 51.
    Zheng, G., Lawlor, O.S., Kalé, L.V.: Multiple flows of control in migratable parallel programs. In: 2006 International Conference on Parallel Processing Workshops (ICPPW 2006), Columbus, Ohio, pp. 435–444. IEEE Computer Society (August 2006)Google Scholar
  52. 52.
    Zheng, G., Ni, X., Kale, L.V.: A Scalable Double In-memory Checkpoint and Restart Scheme towards Exascale. In: Proceedings of the 2nd Workshop on Fault-Tolerance for HPC at Extreme Scale (FTXS), Boston, USA (June 2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Laxmikant V. Kale
    • 1
  • Jonathan Lifflander
    • 1
  1. 1.University of Illinois at Urbana-ChampaignUSA

Personalised recommendations