Skip to main content
SpringerLink
Log in

Inshimtu – A Lightweight In Situ Visualization “Shim”

  • Conference paper
  • First Online:
High Performance Computing (ISC High Performance 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13999))

Included in the following conference series:

Abstract

In situ visualization and analysis is a valuable yet under utilized commodity for the simulation community. There is hesitance or even resistance to adopting new methodologies due to the uncertainties that in situ holds for new users. There is a perceived implementation cost, maintenance cost, risk to simulation fault tolerance, potential lack of scalability, a new resource cost for running in situ processes, and more. The list of reasons why in situ is overlooked is long. We are attempting to break down this barrier by introducing Inshimtu. Inshimtu is an in situ “shim” library that enables users to try in situ before they buy into a full implementation. It does this by working with existing simulation output files, requiring no changes to simulation code. The core visualization component of Inshimtu is ParaView Catalyst, allowing it to take advantage of both interactive and non-interactive visualization pipelines that scale. We envision Inshimtu as stepping stone to show users the value of in situ and motivate them to move to one of the many existing fully-featured in situ libraries available in the community. We demonstrate the functionality of Inshimtu with a scientific workflow on the Shaheen II supercomputer.

Inshimtu is available for download at: https://github.com/kaust-vislab/Inshimtu-basic.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adhinarayanan, V., Feng, W., Rogers, D., Ahrens, J., Pakin, S.: Characterizing and modeling power and energy for extreme-scale in-situ visualization. In: IEEE Parallel and Distributed Processing Symposium (IPDPS), pp. 978–987 (2017)

    Google Scholar 

  2. Ahern, S., et al.: Scientific discovery at the exascale. In: Report for the DOE ASCR Workshop on Exascale Data Management, Analysis, and Visualization (2011)

    Google Scholar 

  3. Ahrens, J., Geveci, B., Law, C.: ParaView: an end-user tool for large-data visualization. In: The Visualization Handbook, pp. 717–731 (2005)

    Google Scholar 

  4. Ayachit, U., et al.: The SENSEI generic in situ interface. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 40–44 (2016)

    Google Scholar 

  5. Ayachit, U., et al.: ParaView catalyst: enabling in situ data analysis and visualization. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 25–29 (2015)

    Google Scholar 

  6. Ayachit, U., et al.: Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures. In: ACM/IEEE Conference for High Performance Computing, Networking, Storage and Analysis (SC16) (2016)

    Google Scholar 

  7. Bauer, A.C., et al.: In situ methods, infrastructures, and applications on high performance computing platforms, a state-of-the-art (STAR) report. In: Computer Graphics Forum, Proceedings of Eurovis 2016, vol. 35, no. 3 (2016)

    Google Scholar 

  8. Bethel, E.W., et al.: The sensei generic in situ interface: tool and processing portability at scale. In: Childs, H., Bennett, J.C., Garth, C. (eds.) In Situ Visualization for Computational Science. Mathematics and Visualization, pp. 281–306. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-81627-8_13

    Chapter  Google Scholar 

  9. Bethel, E.W., et al.: Proximity portability and in transit, m-to-n data partitioning and movement in sensei. In: Childs, H., Bennett, J.C., Garth, C. (eds.) In Situ Visualization for Computational Science. Mathematics and Visualization, pp. 439–460. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-81627-8_20

    Chapter  Google Scholar 

  10. Childs, H., et al.: A terminology for in situ visualization and analysis systems. Int. J. High Perform. Comput. Appl. 34(6), 676–691 (2020)

    Article  Google Scholar 

  11. Childs, H., et al.: A contract-based system for large data visualization. In: 2005 Proceedings of IEEE Visualization, pp. 190–198 (2005)

    Google Scholar 

  12. Childs, H., et al.: Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl. 30(3), 22–31 (2010)

    Article  Google Scholar 

  13. Childs, H., et al.: Visualization at extreme scale concurrency. In: High Performance Visualization: Enabling Extreme-Scale Scientific Insight. CRC Press, Boca Raton (2012)

    Google Scholar 

  14. Dayal, J., et al.: Flexpath: type-based publish/subscribe system for large-scale science analytics. In: IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID) (2014)

    Google Scholar 

  15. Docan, C., et al.: Dataspaces: an interaction and coordination framework for coupled simulation workflows. Clust. Comput. 15(2), 163–181 (2012)

    Article  Google Scholar 

  16. Dorier, M., et al.: Damaris/viz: a nonintrusive, adaptable and user-friendly in situ visualization framework. In: IEEE Symposium on Large-Scale Data Analysis and Visualization (LDAV), pp. 67–75 (2013)

    Google Scholar 

  17. Esposito, A., Holst, G.: In situ visualization of WRF data using universal data junction. In: Jagode, H., Anzt, H., Ltaief, H., Luszczek, P. (eds.) ISC High Performance 2021. LNCS, vol. 12761, pp. 475–483. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90539-2_32

    Chapter  Google Scholar 

  18. Fogal, T., Proch, F., Schiewe, A., Hasemann, O., Kempf, A., Krüger, J.: FreeProcessing: transparent in situ visualization via data interception. In: Eurographics Symposium on Parallel Graphics and Visualization: EG PGV: [Proceedings]/Sponsored by Eurographics Association in Cooperation with ACM SIGGRAPH. Eurographics Symposium on Parallel Graphics and Visualization, vol. 2014, p. 49. NIH Public Access (2014)

    Google Scholar 

  19. Godoy, W.F., et al.: ADIOS 2: the adaptable input output system. A framework for high-performance data management. SoftwareX 12, 100561 (2020)

    Google Scholar 

  20. Kress, J., et al.: Loosely coupled in situ visualization: a perspective on why it’s here to stay. In: Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 1–6 (2015)

    Google Scholar 

  21. Kress, J., et al.: Comparing the efficiency of in situ visualization paradigms at scale. In: Weiland, M., Juckeland, G., Trinitis, C., Sadayappan, P. (eds.) ISC High Performance 2019. LNCS, vol. 11501, pp. 99–117. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-20656-7_6

    Chapter  Google Scholar 

  22. Kress, J., et al.: Opportunities for cost savings with in-transit visualization. In: Sadayappan, P., Chamberlain, B.L., Juckeland, G., Ltaief, H. (eds.) ISC High Performance 2020. LNCS, vol. 12151, pp. 146–165. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50743-5_8

    Chapter  Google Scholar 

  23. Kress, J.M.: In-line vs. in-transit in situ: which technique to use at scale? (2020)

    Google Scholar 

  24. Larsen, M., et al.: The ALPINE in situ infrastructure: ascending from the ashes of strawman. In: Workshop on In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization (ISAV), pp. 42–46 (2017)

    Google Scholar 

  25. Moreland, K., et al.: Examples of in transit visualization. In: Proceedings of the 2nd International Workshop on Petascale Data Analytics: Challenges and Opportunities, pp. 1–6. ACM (2011)

    Google Scholar 

  26. Pugmire, D., et al.: Visualization as a service for scientific data. In: Nichols, J., Verastegui, B., Maccabe, A.B., Hernandez, O., Parete-Koon, S., Ahearn, T. (eds.) SMC 2020. CCIS, vol. 1315, pp. 157–174. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63393-6_11

    Chapter  Google Scholar 

  27. Pugmire, D., et al.: Visualization and analysis for near-real-time decision making in distributed workflows. In: 2016 IEEE International Parallel and Distributed Processing Symposium Workshops, pp. 1007–1013. IEEE (2016)

    Google Scholar 

  28. Pugmire, D., et al.: Fides: a general purpose data model library for streaming data. In: Jagode, H., Anzt, H., Ltaief, H., Luszczek, P. (eds.) ISC High Performance 2021. LNCS, vol. 12761, pp. 495–507. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-90539-2_34

    Chapter  Google Scholar 

  29. Rivi, M., Calori, L., Muscianisi, G., Slavnic, V.: In-situ visualization: state-of-the-art and some use cases. PRACE White Paper; PRACE, Brussels, Belgium (2012)

    Google Scholar 

  30. Whitlock, B., Favre, J., Meredith, J.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, pp. 101–109 (2011)

    Google Scholar 

  31. WRF – The Weather Research and Forecasting Model, v. 3.7.1 (2015)

    Google Scholar 

  32. Zhang, F., et al.: In-memory staging and data-centric task placement for coupled scientific simulation workflows. Concurr. Comput.: Pract. Exp. 29(12), e4147 (2017). https://doi.org/10.1002/cpe.4147

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by King Abdullah University of Science and Technology (KAUST). This research made use of the resources of the Visualization and Supercomputing Laboratories at KAUST.

Author information

Authors and Affiliations

  1. King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia

    James Kress, Glendon Holst, Hari Prasad Dasari, Shehzad Afzal, Ibrahim Hoteit & Thomas Theußl

Authors
  1. James Kress
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Glendon Holst
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hari Prasad Dasari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shehzad Afzal
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ibrahim Hoteit
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Thomas Theußl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to James Kress .

Editor information

Editors and Affiliations

  1. University of New Mexico, Albuquerque, NM, USA

    Amanda Bienz

  2. University of Edinburgh, Edinburgh, UK

    Michèle Weiland

  3. Université Paris-Saclay, Gif sur Yvette, France

    Marc Baboulin

  4. CERFACS, Toulouse, France

    Carola Kruse

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kress, J., Holst, G., Dasari, H.P., Afzal, S., Hoteit, I., Theußl, T. (2023). Inshimtu – A Lightweight In Situ Visualization “Shim”. In: Bienz, A., Weiland, M., Baboulin, M., Kruse, C. (eds) High Performance Computing. ISC High Performance 2023. Lecture Notes in Computer Science, vol 13999. Springer, Cham. https://doi.org/10.1007/978-3-031-40843-4_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-40843-4_19

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-40842-7

  • Online ISBN: 978-3-031-40843-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation