Abstract
We address the problem of providing software support for simulating complex physical systems that require multiple physical models, potentially at multiple scales and resolutions and implemented using different programming languages and distinct parallel programming paradigms. The individual models must be coupled to allow them to exchange information either at boundaries where the models align in physical space or in areas where they overlap in space. Employing multiple physical models presents several difficult challenges, both in modeling the physics correctly and in efficiently coupling multiple simulation codes for a complete physical system. As a solution we describe InterComm, a framework that addresses three main parts of the problem: (1) providing comprehensive support for specifying at runtime what data is to be transferred between models, (2) flexibly specifying and efficiently determining when the data should be moved, and (3) effectively deploying multiple simulation codes in a high performance distributed computing environment (the grid).
Similar content being viewed by others
Notes
https://www.dmso.mil/public/transition/hla
References
Foster I, Kesselman C (2003) The Grid 2: blueprint for a new computing infrastructure, 2nd edn. Morgan Kaufman/Elsevier, San Francisco, Amsterdam
Brown DL, Geoffrey S. Chesshire, William D. Henshaw, Quinlan DJ (1997) Overture: an object oriented software system for solving partial differential equations in serial and parallel environments. In: Proceedings of the 8th SIAM conference on parallel processing for scientific computing, SIAM 1997
Wu JS, Sussman A (2004) Flexible control of data transfers between parallel programs. In: Proceedings of the 5th international workshop on grid computing—GRID 2004, November 2004, IEEE Computer Society Press, Los Alamitos, pp 226–234
Snir M, Otto S, Huss-Lederman S, Walker D, Dongarra J (1998) MPI—the complete reference, 2nd edn. Scientific and Engineering Computation Series. MIT Press, New York
Nieplocha J, Harrison RJ, Littlefield RJ (1994) Global arrays: a portable shared-memory programming model for distributed memory computers. In: Proceedings Supercomputing ’94, November 1994, IEEE Computer Society Press, Los Alamitos, pp 340–349
Kohn S, Kumfert G, Painter J, Ribbens C (2001) Divorcing language dependencies from a scientific software library. In: Proceedings of the 11th SIAM conference on parallel processing for scientific computing. SIAM, March 2001
Wolski R, Spring N, Hayes J (1999) The Network Weather Service: a distributed resource performance forecasting service for metacomputing. J Future Gener Comput Syst 15(5-6):757–768
Rajasekar A, Wan M, Moore R (2002) MySRB & SRB—components of a data grid. In Proceedings of the 11th international symposium on high performance distributed computing (HPDC-11), July 2002, IEEE Computer Society Press, Los Alamitos
Yoon IC, Sussman A, Porter A (2005) And away we go: understanding the complexity of launching complex HPC applications. In: Proceedings of 2nd international workshop on software engineering for high performance computing systems applications, May 2005. Available at http://csdl.ics.hawaii.edu/se-hpcs/papers/sehpcs-proceedings.pdf, pp 45–49
Keahey K, Fasel P, Mniszewski S (2001) PAWS: collective interactions and data transfers. In: Proceedings of the 10th IEEE international symposium on high performance distributed computing (HPDC 2001), August 2001
Larson JW, Jacob R, Foster I, Guo J (2001) The Model Coupling Toolkit. In: Alexandrov VN, Dongarra JJ, Juliano BA, Renner RS, Tan CJK (eds) Proceedings of international conference on computational science (ICCS). Lecture Notes in Computer Science, vol 2073. Springer, Berlin Heidelberg New York, pp 185–194
Geist GA, Kohl JA, Papadopoulos PM (1997) CUMULVS: providing fault-tolerance, visualization and steering of parallel applications. Int J High Perform Comput Appl 11(3):224–236
Jiao X, Campbell MT, Heath MT (2003) Roccom: an object-oriented, data-centric software integration framework for multiphysics simulations. In: Proceedings of the 2003 international conference on supercomputing, June 2003, ACM Press, New York, pp 358–368
Armstrong R, Gannon D, Geist A, Keahey K, Kohn SR, McInnes L, Parker SR, Smolinski BA (1999) Toward a Common Component Architecture for high-performance scientific computing. In: Proceedings of the 8th IEEE international symposium on high performance distributed computing (HPDC’99), August 1999, IEEE Computer Society Press, Los Alamitos
Edjlali G, Sussman A, Saltz J (1997) Interoperability of data parallel runtime libraries. In: Proceedings of the 11th international parallel processing symposium, April 1997, IEEE Computer Society Press, Los Alamitos
Lee JY, Sussman A (2005) High performance communication between parallel programs. In: Proceedings of 2005 joint workshop on high-performance grid computing and high-level parallel programming models (HIPS-HPGC 2005). IEEE Computer Society Press, Los Alamitos, April 2005. Appears with the proceedings of IPDPS 2005
Eugster PTh, Felber PA, Guerraoui R, Kermarrec AM (2003) The many faces of publish/subscribe. ACM Comput Surv 35(2):114–131
Gannon D, Bramley R , Stuckey T, Villacis J, Balasubramanian J, Akman E, Breg F, Diwan S, Govindaraju M (1998) Developing component architectures for distributed scientific problem solving. IEEE Comput Sci Eng 5(2):50–63
Object Management Group (1995) The Common Object Request Broker: architecture and specification, revision 2.0 edition
Agrawal G, Sussman A, Saltz J (1995) An integrated runtime and compile-time approach for parallelizing structured and block structured applications. IEEE Trans Parallel Distrib Syst 6(7):747–754
Hwang YS, Moon B, Sharma SD, Ponnusamy R, Das R , Saltz JH (1995) Runtime and language support for compiling adaptive irregular programs. Softw Pract Exp 25(6):597–621
Fink SJ, Kohn SR, Baden SB (1998) Efficient run-time support for irregular block-structured applications. J Parallel Distrib Comput 50(1):61–82
Lee JY, Sussman A (2004) Efficient communication between parallel programs with InterComm. Technical Report CS-TR-4557 and UMIACS-TR-2004-04, University of Maryland, Department of Computer Science and UMIACS, January 2004
Koelbel C, Loveman D, Schreiber R, Steele G Jr, Zosel M (1994) The high performance Fortran handbook. MIT Press, New York
Ponnusamy R, Saltz J, Choudhary A, Hwang YS, Fox G (1995) Runtime support and compilation methods for user-specified irregular data distributions. IEEE Trans Parallel Distrib Syst 6(8):815–831
Czajkowski K, Fitzgerald S, Foster I, Kesselman C (2001) Grid information services for distributed resource sharing. In: Proceedings of the 10th IEEE international symposium on high performance distributed computing (HPDC 2001), August 2001, IEEE Computer Society Press, Los Alamitos
Foster I, Kesselman C, Nick J, Tuecke S (2002) Grid services for distributed system integration. IEEE Comput 36(6):37–46
Geist A, Beguelin A, Dongarra J, Jiand W, Manchek R, Sunderam V (1994) PVM: parallel virtual machine: a user’s guide and tutorial for networked parallel computing. MIT Press, New York
Allan BA, Armstrong RC, Wolfe AP, Ray J, Bernholdt DE, Kohl JA (2002) The CCA core specification in a distributed memory SPMD framework. Concur Comput Pract Exp 15(5):323–345
Govindaraju M, Krishnan S, Chiu K, Slominski A, Gannon D, Bramley R (2003) Merging the CCA component model with the OGSI framework. In: Proceedings of CCGrid2003: IEEE/ACM international symposium on cluster computing and the grid, May 2003, IEEE Computer Society Press, Los Alamitos
Acknowledgments
This research was supported by the National Science Foundation under Grants #EIA-0121161 and #ACI-9619020 (UC Subcontract #10152408), and NASA under Grants #NAG5-11994 and #NAG5-12652.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sussman, A. Building complex coupled physical simulations on the grid with InterComm. Engineering with Computers 22, 311–323 (2006). https://doi.org/10.1007/s00366-006-0037-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00366-006-0037-7