Scalable Simulation of Electromagnetic Hybrid Codes

  • Kalyan Perumalla
  • Richard Fujimoto
  • Homa Karimabadi
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3992)


New discrete-event formulations of physics simulation models are emerging that can outperform models based on traditional time-stepped techniques. Detailed simulation of the Earth’s magnetosphere, for example, requires execution of sub-models that are at widely differing timescales. In contrast to time-stepped simulation which requires tightly coupled updates to entire system state at regular time intervals, the new discrete event simulation (DES) approaches help evolve the states of sub-models on relatively indepen-dent timescales. However, parallel execution of DES-based models raises challenges with respect to their scalability and performance. One of the key challenges is to improve the computation granularity to offset synchronization and communication overheads within and across processors. Our previous work was limited in scalability and runtime performance due to the parallelization challenges. Here we report on optimizations we performed on DES-based plasma simulation models to improve parallel performance. The mapping of model to simulation processes is optimized via aggregation techniques, and the parallel runtime engine is optimized for communication and memory efficiency. The net result is the capability to simulate hybrid particle-in-cell (PIC) models with over 2 billion ion particles using 512 processors on supercomputing platforms.


Priority Queue Discrete Event Simulation Parallel Execution Simulation Engine Scalable Simulation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Karimabadi, H., Driscoll, J., Omelchenko, Y., Omidi, N.: A New Asynchronous Methodology for Modeling of Physical Systems: Breaking the Curse of Courant Condition. Journal of Computational Physics 205(2) (2005)Google Scholar
  2. 2.
    Karimabadi, H., Driscoll, J., Omelchenko, Y., Perumalla, K.S., Fujimoto, R.M., Omidi, N.: Parallel Discrete Event Simulation of Grid-based Models: Asynchronous Electromagnetic Hybrid Code, Proceedings. LNCS, pp. 580–588. Springer, Heidelberg (2005)Google Scholar
  3. 3.
    Fujimoto, R.M.: Parallel and Distributed Simulation Systems. Wiley Interscience, Hoboken (2000)Google Scholar
  4. 4.
    Chandy, K., Misra, J.: Asynchronous distributed simulation via a sequence of parallel computations. Communications of the ACM 24 (1981)Google Scholar
  5. 5.
    Chandy, K.M., Misra, J.: Distributed Simulation: A Case Study in Design and Verification of Distributed Programs. IEEE Transactions on Software Engineering SE-5(5), 440–452 (1978)Google Scholar
  6. 6.
    Jefferson, D.: Virtual Time. ACM Transactions on Programming Languages and Systems 7(3), 404–425 (1985)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Perumalla, K.S.: μsik - A Micro-Kernel for Parallel/Distributed Simulation Systems. In: Workshop on Principles of Advanced and Distributed Simulation (2005)Google Scholar
  8. 8.
    Tang, Y., Perumalla, K.S., Fujimoto, R.M., Karimabadi, H., Driscoll, J., Omelchenko, Y.: Optimistic Parallel Discrete Event Simulations of Physical Systems using Reverse Computation. In: Workshop on Principles of Advanced and Distributed Simulation (2005)Google Scholar
  9. 9.
    Hontalas, P., et al.: Performance of the Colliding Pucks Simulation on the Time Warp Operating System, Distributed Simulation (1989)Google Scholar
  10. 10.
    Lubachevsky, B.D.: Efficient Distributed Event-Driven Simulations of Multiple-Loop Networks. Communications of the ACM 32(1), 111–123 (1989)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Lubachevsky, B.: Several Unsolved Problems in Large-Scale Discrete Event Simulations. In: Workshop on Parallel and Distributed Simulation (1993)Google Scholar
  12. 12.
    Zeigler, B.P., et al.: Theory of Modeling & Simulation, 2nd edn. Academic Press, London (2000)Google Scholar
  13. 13.
    Nutaro, J.: Parallel Discrete Event Simulation with Application to Continuous Systems, in Department of Electrical and Computer Engineering, vol. Ph.D. Tucson, AZ: University of Arizona, p. 182 (2003)Google Scholar
  14. 14.
    Karimabadi, H., Krauss-Varban, H.D., Huba, J., Vu, H.X.: On Magnetic Reconnection Regimes and Associated Three-Dimensional Asymmetries: Hybrid, Hall-less Hybrid and Hall-MHD Simulations. Journal of Geophysical Research 109, 1–21 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Kalyan Perumalla
    • 1
  • Richard Fujimoto
    • 2
  • Homa Karimabadi
    • 3
  1. 1.Oak Ridge National LaboratoryOak RidgeUSA
  2. 2.Georgia Institute of TechnologyAtlantaUSA
  3. 3.SciberQuest IncSolana BeachUSA

Personalised recommendations