Skip to main content
SpringerLink
Log in

State of the Practice for Lattice Boltzmann Method Software

  • Review article
  • Published:
Archives of Computational Methods in Engineering Aims and scope Submit manuscript

Abstract

We analyze the state of software development practice for Lattice Boltzmann solvers by quantitatively and qualitatively measuring and comparing 24 software packages for 10 software qualities (installability, correctness/verifiability, reliability, robustness, usability, maintainability, reusability, understandability, visibility/transparency and reproducibility). Our reproducible analysis method employs a measurement template (containing 108 measures that are manually and automatically extracted from the project repository) and developer interviews (consisting of 20 questions). After measuring, we rank the software using the Analytic Hierarchy Process. Our ranking is roughly consistent with GitHub stars ranking, suggesting at least a partial correlation between the use of best practices and popularity. We find the state of the practice to be healthy with 67% of the measured packages ranking in the top two for at least two qualities, the majority of LBM generated artifacts corresponding to general recommendations from research software developers, common use of version control (67% of packages) and the adoption of a quasi-agile development process. Areas of best practice to potentially improve include adoption of continuous integration, API documentation and enforcement of programming style guides. We interviewed four developers to gain insight into their current pain points. Identified challenges include lack of development time, lack of funding, and difficulty with ensuring correctness. Developers are addressing these pain points by designing for change, circumventing the oracle problem and prioritizing documentation and usability. For future improvements we suggest the following: employing linters, conducting rigorous peer reviews, writing and submitting more papers on software, growing the number of contributors by following current recommendations for open source projects, and augmenting the theory manuals to include more requirements specification relevant information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Chen S, Doolen GD (1998) Lattice Boltzmann method for fluid flows. Annu Rev Fluid Mech 30(1):329–364

    MathSciNet  Google Scholar 

  2. Bao YB, Meskas J (2011) Lattice Boltzmann method for fluid simulations. Department of Mathematics, Courant Institute of Mathematical Sciences, New York, p 44

    Google Scholar 

  3. Ganji DD, Kachapi SHH (2015) Application of nonlinear systems in nanomechanics and nanofluids: analytical methods and applications. William Andrew, Norwich

    Google Scholar 

  4. Sadeghi R, Khodaei S, Ganame J, Keshavarz-Motamed Z (2020) Towards non-invasive computational-mechanics and imaging-based diagnostic framework for personalized cardiology for coarctation. Sci Rep 10(1):9048. https://doi.org/10.1038/s41598-020-65576-y

    Article  Google Scholar 

  5. Zhang D, Cai Q, Sai G (2018) Three-dimensional lattice-Boltzmann model for liquid water transport and oxygen diffusion in cathode of polymer electrolyte membrane fuel cell with electrochemical reaction. Electrochim Acta 262(2018):282–296. https://doi.org/10.1016/j.electacta.2017.12.189

    Article  Google Scholar 

  6. Anwar S, Sukop MC (2009) Regional scale transient groundwater flow modeling using Lattice Boltzmann methods. Comput Math Appl 58(5):1015–1023. https://doi.org/10.1016/j.camwa.2009.02.025

    Article  MathSciNet  Google Scholar 

  7. Sadeghi R, Gasner N, Khodaei S, Garcia J, Keshavarz-Motamed Z (2022) Impact of mixed valvular disease on coarctation hemodynamics using patient-specific lumped parameter and Lattice Boltzmann modeling. Int J Mech Sci 217:107038. https://doi.org/10.1016/j.ijmecsci.2021.107038

    Article  Google Scholar 

  8. Sadeghi R, Tomka B, Khodaei S, Garcia J, Ganame J, Keshavarz-Motamed Z (2022) Reducing morbidity and mortality in patients with coarctation requires systematic differentiation of impacts of mixed valvular disease on coarctation hemodynamics. J Am Heart Assoc 11(2):26

    Google Scholar 

  9. Smith S, Michalski P (2022) Digging deeper into the state of the practice for domain specific research software. In: Proceedings of the international conference on computational science, ICCS. pp 1–15

  10. Smith WS, Carette J, Michalski Pr, Dong A, Owojaiye O (2021) Methodology for assessing the state of the practice for domain X. https://arxiv.org/abs/2110.11575

  11. Dong A (2021) Assessing the state of the practice for medical imaging software. Master’s thesis. McMaster University, Hamilton

  12. Jung R, Gundlach S, Hasselbring W (2022) Thematic domain analysis for ocean modeling. Environ Modell Softw 150:105323. https://doi.org/10.1016/j.envsoft.2022.105323

    Article  Google Scholar 

  13. Michalski P (2021) State of the practice for lattice Boltzmann method software. Master’s thesis. McMaster University, Hamilton

  14. Smith WS, Lazzarato A, Carette J (2018) State of the practice for GIS software. https://arxiv.org/abs/1802.03422

  15. Spencer Smith W, Lazzarato A, Carette J (2016) State of practice for mesh generation software. Adv Eng Softw 100:53–71

    Google Scholar 

  16. Smith S, Sun Y, Carette J (2015) State of the practice for developing oceanographic software. McMaster University, Department of Computing and Software

  17. Spencer Smith W, Zeng Z, Carette J (2018) Seismology software: state of the practice. J Seismol 22(3):755–788

    Google Scholar 

  18. Smith WS, Sun Y, Carette J (2018) Statistical software for psychology: comparing development practices between CRAN and other communities, p 33. https://arxiv.org/abs/1802.07362

  19. Ghezzi C, Jazayeri M, Mandrioli D (2003) Fundamentals of software engineering, 2nd edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  20. ISO, IEC (2011) Systems and software engineering- Systems and software Quality Requirements and Evaluation (SQuaRE)- System and software quality models. Standard, International Organization for Standardization

  21. Lenhard J, Harrer S, Wirtz G (2013) Measuring the installability of service orchestrations using the square method. In: 2013 IEEE 6th International Conference on Service-Oriented Computing and Applications. IEEE, pp 118–125

  22. IEEE (1998) Recommended practice for software requirements specifications. IEEE Std 830-1998, pp 1–40. https://doi.org/10.1109/IEEESTD.1998.88286

  23. Musa JD, Iannino A, Okumoto K (1987) Software reliability: prediction and application

  24. Boehm BW (2007) Software engineering: Barry W. Boehm’s lifetime contributions to software development, management, and research, vol 69. Wiley, Hoboken

    Google Scholar 

  25. IEEE (1991) IEEE standard glossary of software engineering terminology. IEEE Standard

  26. Wiegers (2003) Software requirements, 2e. Microsoft Press, Boston

    Google Scholar 

  27. Nielsen J (2012) Usability 101: introduction to usability. https://www.nngroup.com/articles/usability-101-introduction-to-usability/

  28. Kalagiakos P (2003) The non-technical factors of reusability. In: Proceedings of the 29th conference on EUROMICRO. IEEE Computer Society, 124

  29. ISO, IEC (2001) ISO/IEC 9126. Software engineering - Product quality, ISO/IEC

  30. Benureau F, Rougier N (2017) Re-run, repeat, reproduce, reuse, replicate: transforming code into scientific contributions. http://arxiv.org/abs/1708.08205

  31. Seaton MA, Anderson RL, Metz S, Smith W (2013) DL_MESO: highly scalable mesoscale simulations. Mol Simul 39(10):796–821

    Google Scholar 

  32. Weik F, Weeber R, Szuttor K, Breitsprecher K, de Graaf J, Kuron M, Landsgesell J, Menke H, Sean D, Holm C (2019) ESPResSo 4.0–an extensible software package for simulating soft matter systems. Eur Phys J Spec Top 227(14):1789–1816

    Google Scholar 

  33. Halverson JD, Brandes T, Lenz O, Arnold A, Bevc S, Starchenko V, Kremer K, Stuehn T, Reith D (2013) ESPResSo++: a modern multiscale simulation package for soft matter systems. Comput Phys Commun 184(4):1129–1149

    Google Scholar 

  34. Bauer M, Köstler H, Rüde U (2021) lbmpy: automatic code generation for efficient parallel lattice Boltzmann methods. J Comput Sci 49(2021):101269

    MathSciNet  Google Scholar 

  35. Bedrunka MC, Wilde D, Kliemank M, Reith D, Foysi H, Kramer A (2021) Lettuce: PyTorch-based lattice Boltzmann framework. In: International conference on high performance computing. Springer, pp 40–55

  36. Desplat J-C, Pagonabarraga I, Bladon P (2001) LUDWIG: a parallel Lattice-Boltzmann code for complex fluids. Comput Phys Commun 134(3):273–290

    Google Scholar 

  37. Harwood ARG, O’Connor J, Muñoz JS, Santasmasas MC, Revell AJ (2018) LUMA: a many-core, fluid-structure interaction solver based on the lattice-Boltzmann method. SoftwareX 7(2018):88–94

    Google Scholar 

  38. Galindo-Torres SA (2013) A coupled discrete element lattice Boltzmann method for the simulation of fluid-solid interaction with particles of general shapes. Comput Methods Appl Mech Eng 265(2013):107–119

    MathSciNet  Google Scholar 

  39. Hasert M, Masilamani K, Zimny S, Klimach H, Qi J, Bernsdorf J, Roller S (2014) Complex fluid simulations with the parallel tree-based lattice Boltzmann solver Musubi. J Comput Sci 5(5):784–794

    MathSciNet  Google Scholar 

  40. Heuveline V, Krause MJ (2010) OpenLB: towards an efficient parallel open source library for lattice Boltzmann fluid flow simulations. In: International workshop on state-of-the-art in scientific and parallel computing. PARA, vol 9, p 570

  41. Latt J, Malaspinas O, Kontaxakis D, Parmigiani A, Lagrava D, Brogi F, Belgacem MB, Thorimbert Y, Leclaire S, Li S et al (2021) Palabos: parallel lattice Boltzmann solver. Comput Math Appl 81(2021):334–350

    MathSciNet  Google Scholar 

  42. Januszewski M, Kostur M (2014) Sailfish: a flexible multi-GPU implementation of the lattice Boltzmann method. Comput Phys Commun 185(9):2350–2368

    Google Scholar 

  43. Rokicki J, Laniewski-Wollk L (2016) Adjoint lattice Boltzmann for topology optimization on multi-GPU architecture. Comput Math Appl 71(3):833–848

    MathSciNet  Google Scholar 

  44. Bauer M, Eibl S, Godenschwager C, Kohl N, Kuron M, Rettinger C, Schornbaum F, Schwarzmeier C, Thönnes D, Köstler H et al (2021) waLBerla: a block-structured high-performance framework for multiphysics simulations. Comput Math Appl 81(2021):478–501

    MathSciNet  Google Scholar 

  45. Heuveline V, Krause MJ, Latt J (2009) Towards a hybrid parallelization of lattice Boltzmann methods. Comput Math Appl 58(5):1071–1080

    MathSciNet  Google Scholar 

  46. Graille B, Gouarin L (2017) pylbm documentation

  47. Mazzeo MD, Coveney PV (2008) HemeLB: ahigh performance parallel lattice-Boltzmann code for large scale fluid flow in complex geometries. Comput Phys Commun 178(12):894–914

    Google Scholar 

  48. Levesque M, Duvail M, Pagonabarraga I, Frenkel D, Rotenberg B (2013) Accounting for adsorption and desorption in lattice Boltzmann simulations. Phys Rev E 88(1):013308

    Google Scholar 

  49. Schmieschek S, Shamardin L, Frijters S, Krüger T, Schiller UD, Harting J, Coveney PV (2017) LB3D: a parallel implementation of the Lattice-Boltzmann method for simulation of interacting amphiphilic fluids. Comput Phys Commun 217(2017):149–161

    MathSciNet  Google Scholar 

  50. Gewaltig M-O, Cannon R (2012) Quality and sustainability of software tools in neuroscience. Cornell Univ Lib 2012:1–20

    Google Scholar 

  51. Gieniusz T (2019) GitStats. https://github.com/tomgi/git_stats. Accessed 27 May 2021

  52. Boyter B (2021) Sloc cloc and code. https://github.com/boyter/scc. Accessed 27 May 2021

  53. Saaty TL (1980) The analytic hierarchy process: planning, priority setting, resource allocation. McGraw-Hill Publishing Company, New York

    Google Scholar 

  54. Parnas DL (1976) On the design and development of program families. IEEE Trans Software Eng 1(1976):1–9

    Google Scholar 

  55. Weiss DM (1997) Defining families: the commonality analysis. submitted to IEEE Transactions on Software Engineering. http://www.research.avayalabs.com/user/weiss/Publications.html

  56. Smith S (2022) Software quality grades for lattice Boltzmann solvers. https://data.mendeley.com/datasets/5dym63wn6z/1. https://doi.org/10.17632/5dym63wn6z.1

  57. ESA (1991) ESA Software Engineering Standards, PSS-05-0 Issue 2. Technical Report. European Space Agency

  58. Robertson S, Robertson J (1999) Mastering the requirements process. In: New York NY (ed) Chapter Volere requirements specification template. ACM Press/Addison-Wesley Publishing Co, New York, pp 353–391

    Google Scholar 

  59. Rutkowski M, Gryglas W, Szumbarski J, Leonardi C, Łaniewski-Wołłk Ł (2020) Open-loop optimal control of a flapping wing using an adjoint Lattice Boltzmann method. Comput Math Appl 79(12):3547–3569

    MathSciNet  Google Scholar 

  60. Gray A, Stratford K (2013) Ludwig: multiple GPUs for a complex fluid lattice Boltzmann application. Designing Scientific Applications on GPUs. Chapman & Hall/CRC Numerical Analysis and Scientific Computing Series, Taylor & Francis, Boca Raton

    Google Scholar 

  61. Szulik K (2017) Don’t judge a project by its GitHub stars alone. https://blog.tidelift.com/dont-judge-a-project-by-its-github-stars-alone

  62. USGS (2019) USGS (United States Geological Survey) Software Plannning Checklist. https://www.usgs.gov/media/files/usgs-software-planning-checklist

  63. Schlauch T, Meinel M, Haupt C (2018) DLR software engineering guidelines. https://doi.org/10.5281/zenodo.1344612

  64. Alys B, James C, Peter F, Ian H, John N, Richard R, Robert T (2021) Scottish Covid-19 Response Consortium. https://github.com/ScottishCovidResponse/modelling-software-checklist/blob/main/software-checklist.md

  65. Wilson G, Bryan J, Cranston K, Kitzes J, Nederbragt L, Teal TK (2016) Good enough practices in scientific computing. http://arxiv.org/abs/1609.00037

  66. Smith B, Bartlett R, xSDK Developers (2018) xSDK community package policies. https://doi.org/10.6084/m9.figshare.4495136.v6

  67. Heroux MA, Bieman JM, Heaphy RT (2008) Trilinos developers guide part II: ASC software quality engineering practices version 2.0. https://faculty.csbsju.edu/mheroux/fall2012_csci330/TrilinosDevGuide2.pdf

  68. Thiel C (2020) EURISE Network Technical Reference. https://technical-reference.readthedocs.io/en/latest/

  69. van Gompel M, Noordzij J, de Valk R, Scharnhorst A (2016) Guidelines for software quality, CLARIAH Task Force 54.100. https://github.com/CLARIAH/software-quality-guidelines/blob/master/softwareguidelines.pdf

  70. Orviz P, García ÁL, Duma DC, Donvito G, David M, Gomes J (2017) A set of common software quality assurance baseline criteria for research projects. https://doi.org/10.20350/digitalCSIC/12543

  71. Prana GAA, Treude C, Thung F, Atapattu T, Lo D (2018) Categorizing the content of GitHub README Files. http://arxiv.org/abs/1802.06997

  72. Fogel K (2005) Producing open source software: how to run a successful free software project. O’Reilly Media Inc, Sebastopol

    Google Scholar 

  73. Genota L (2018) Why generation Z learners prefer youtube lessons over printed books. Education week. https://www.edweek.org/teaching-learning/why-generation-z-learners-prefer-youtube-lessons-over-printed-books/2018/09

  74. Software Sustainability Institute (2022) Online sustainability evaluation. https://www.software.ac.uk/resources/online-sustainability-evaluation

  75. Zadka M (2018) How to open source your Python library. https://opensource.com/article/18/12/tips-open-sourcing-python-libraries

  76. Münch J, Trieflinger S, Lang D (2019) Product roadmap—From vision to reality: a systematic literature review. In: 2019 IEEE international conference on engineering, technology and innovation (ICE/ITMC). pp 1–8. https://doi.org/10.1109/ICE.2019.8792654

  77. Phaal R, Farrukh CJP, Probert DR (2005) Developing a technology roadmapping system. In A unifying discipline for melting the boundaries technology management, pp 99–111. https://doi.org/10.1109/PICMET.2005.1509680

  78. Vähäniitty J, Lassenius C, Rautiainen K (2002) An approach to product roadmapping in small software product businesses. University of Technologie, Software Business and Engineering Institute, Helsinki

    Google Scholar 

  79. Pichler R (2012) Working with an agile product roadmap. https://www.romanpichler.com/blog/agile-product-roadmap/

  80. Tourani P , Adams B, Serebrenik A (2017) Code of conduct in open source projects. In: 2017 IEEE 24th international conference on software analysis, evolution and reengineering (SANER). pp 24–33. https://doi.org/10.1109/SANER.2017.7884606

  81. Singh V, Bongiovanni B, Brandon W (2021) Codes of conduct in Open Source Software–for warm and fuzzy feelings or equality in community? Softw Qual J 2021:1–40. https://doi.org/10.1007/s11219-020-09543-w

    Article  Google Scholar 

  82. Joint Task Force on Software Engineering Ethics IEEE-CS/ACM and Professional Practices (1999) Code of Ethics, IEEE Computer Society. https://www.computer.org/education/code-of-ethics

  83. Professional Engineers Act (2021) Professional Engineers Act, RSO 1990, c P. 28. https://canlii.ca/t/5568z

  84. Carty D (2020) Follow Google’s lead with programming style guides. https://www.techtarget.com/searchsoftwarequality/feature/Follow-Googles-lead-with-programming-style-guides

  85. Titus B (2015) Notes from“How to grow a sustainable software development process (for scientific software)”. http://ivory.idyll.org/blog/2015-growing-sustainable-software-development-process.html

  86. Heroux MA, Bernholdt DE (2018) Better (small) scientific software teams, tutorial in argonne training program on extreme-scale computing (ATPESC). https://doi.org/articles/journal_contribution/ATPESC_Software_Productivity_03_Better_Small_Scientific_Software_Teams/6941438

  87. Meng M, Steinhardt S, Schubert A (2018) Application programming interface documentation: what do software developers want? J Tech Writ Commun 48(3):295–330. https://doi.org/10.1177/0047281617721853

    Article  Google Scholar 

  88. Kelly DF (2007) A software chasm: software engineering and scientific computing. IEEE Softw 24(6):120. https://doi.org/10.1109/MS.2007.155

    Article  Google Scholar 

  89. Owojaiye O, Smith WS, Carette J, Michalski P, Dong A (2021) State of sustainability for research software (poster). In: SIAM-CSE 2021 conference on computational science and engineering, minisymposterium: software productivity and sustainability for CSE. https://doi.org/10.6084/m9.figshare.14039888.v2

  90. Storer T (2017) Bridging the chasm: a survey of software engineering practice in scientific programming. ACM Comput Surv 50(4):32. https://doi.org/10.1145/3084225

    Article  Google Scholar 

  91. Johanson AN, Hasselbring W (2018) Software engineering for computational science: past, present, future. Comput Sci Eng Accepted 2018:1–31

    Google Scholar 

  92. Faulk S, Loh E, Vanter MLVD, Squires S, Votta LG (2009) Scientific computing’s productivity gridlock: how software engineering can help. Comput Sci Eng 11(6):30–39. https://doi.org/10.1109/MCSE.2009.205

    Article  Google Scholar 

  93. de Souza MR, Haines R, Vigo M, Jay C (2019) What makes research software sustainable? An interview study with research software engineers. http://arxiv.org/abs/1903.06039

  94. Spencer Smith W, Koothoor N (2016) A document-driven method for certifying scientific computing software for use in nuclear safety analysis. Nucl Eng Technol 48(2):404–418. https://doi.org/10.1016/j.net.2015.11.008

    Article  Google Scholar 

  95. Heaton D, Carver JC (2015) Claims about the use of software engineering practices in science. Inf Softw Technol 67(C):207–219. https://doi.org/10.1016/j.infsof.2015.07.011

    Article  Google Scholar 

  96. Sanders R, Kelly D (2008) Dealing with risk in scientific software development. IEEE Softw 4:21–28

    Google Scholar 

  97. Nguyen-Hoan L, Flint S, Sankaranarayana R (2010) A survey of scientific software development. In: Proceedings of the 2010 ACM-IEEE international symposium on empirical software engineering and measurement (Bolzano-Bozen, Italy) (ESEM ’10). ACM, New York, Article 12, p 10. https://doi.org/10.1145/1852786.1852802

  98. Wiese IS, Polato I, Pinto G (2019) Naming the pain in developing scientific software. IEEE Softw 2019:1–1. https://doi.org/10.1109/MS.2019.2899838

    Article  Google Scholar 

  99. Carver JC, Kendall RP, Squires SE, Post DE (2007) Software development environments for scientific and engineering software: a series of case studies. In: ICSE ’07: Proceedings of the 29th international conference on software engineering. IEEE Computer Society, Washington, DC, USA, pp 550–559. https://doi.org/10.1109/ICSE.2007.77

  100. Segal J, Morris C (2008) Developing scientific software. IEEE Softw 25(4):18–20

    Google Scholar 

  101. Spencer Smith W (2016) A rational document driven design process for scientific computing software, chapter examples of the application of traditional software engineering practices to science. In: Carver NC, Hong, Thiruvathukal G (eds) Software engineering for science. Jeffrey C, Chapman and Hall/CRC, Boca Raton, pp 33–63

    Google Scholar 

  102. Wilson GV (2006) Where’s the real bottleneck in scientific computing? Scientists would do well to pick some tools widely used in the software industry. Am Sci 94:1

    Google Scholar 

  103. AlNoamany Y, Borghi JA (2018) Towards computational reproducibility: researcher perspectives on the use and sharing of software. PeerJ Comput Sci 4(e163):1–25

    Google Scholar 

  104. Smith WS (2018) Beyond software carpentry. In: 2018 International workshop on software engineering for science (held in conjunction with ICSE’18), pp 1–8

  105. Hilton M, Tunnell T, Huang K, Marinov D, Dig D (2016) Usage, costs, and benefits of continuous integration in open-source projects. In: 2016 31st IEEE/ACM international conference on automated software engineering (ASE). pp 426–437

  106. Humble J, Farley DG (2010) Continuous delivery: reliable software releases through build, test, and deployment automation. Addison-Wesley, Upper Saddle River

    Google Scholar 

  107. Fowler M (2006) Continuous integration. https://martinfowler.com/articles/continuousIntegration.html

  108. Shahin M, Babar MA, Zhu L (2017) Continuous integration, delivery and deployment: a systematic review on approaches, tools. Challenges and practices. IEEE Access 5(2017):3909–3943. https://doi.org/10.1109/ACCESS.2017.2685629

    Article  Google Scholar 

  109. Slaughter A, Permann C, Miller J, Alger B, Novascone S (2021) Continuous integration, in-code documentation, and automation for nuclear quality assurance conformance. Nucl Technol 207:1–8. https://doi.org/10.1080/00295450.2020.1826804

    Article  Google Scholar 

  110. Givler R (2020) A checklist of basic software engineering practices for data analysts and data scientists. https://www.linkedin.com/pulse/checklist-basic-software-engineering-practices-data-analysts-givler/?articleId=6681691007303630849

  111. Ackroyd KS, Kinder SH, Mant GR, Miller MC, Ramsdale CA, Stephenson PC (2008) Scientific software development at a research facility. IEEE Softw 25(4):44–51

    Google Scholar 

  112. Easterbrook SM, Johns TC (2009) Engineering the software for understanding climate change. Comut Sci Eng 11(6):65–74. https://doi.org/10.1109/MCSE.2009.193

    Article  Google Scholar 

  113. Segal J (2005) When software engineers met research scientists: a case study. Emp Softw Eng 10(4):517–536. https://doi.org/10.1007/s10664-005-3865-y

    Article  Google Scholar 

  114. Kelly D (2013) Industrial scientific software: a set of interviews on software development. In: Proceedings of the 2013 conference of the center for advanced studies on collaborative research (Ontario, Canada) (CASCON ’13). IBM Corp., Riverton, pp 299–310. http://dl.acm.org/citation.cfm?id=2555523.2555555

  115. Kelly D (2015) Scientific software development viewed as knowledge acquisition: towards understanding the development of risk-averse scientific software. J Syst Softw 109(2015):50–61. https://doi.org/10.1016/j.jss.2015.07.027

    Article  Google Scholar 

  116. Kelly DF, Sanders R (2008) Assessing the quality of scientific software. In: Proceedings of the first international workshop on software engineering for computational science and engineering (SECSE 2008). In conjunction with the 30th international conference on software engineering (ICSE), Leipzig. http://www.cse.msstate.edu/~SECSE08/schedule.htm

  117. Parnas DL (1972) On the criteria to be used in decomposing systems into modules. Commun ACM 15(2):1053–1058

    Google Scholar 

  118. Parnas DL, Clement PC, Weiss DM (1984) The modular structure of complex systems. In: International conference on software engineering. pp 408–419

  119. Spencer Smith W, Yu W (2009) A document driven methodology for improving the quality of a parallel mesh generation toolbox. Adv Eng Softw 40(11):1155–1167. https://doi.org/10.1016/j.advengsoft.2009.05.003

    Article  Google Scholar 

  120. Gamma E, Helm R, Vlissides J, Johnson IR (1995) Design patterns, elements of reusable object-oriented software. Addison-Wesley Professional, Boston

    Google Scholar 

  121. Hoffman DM, Strooper PA (1995) Software design, automated testing, and maintenance: a practical approach. International Thomson Computer Press, New York

    Google Scholar 

  122. ElSheikh AH, Spencer Smith W, Chidiac SE (2004) Semi-formal design of reliable mesh generation systems. Adv Eng Softw 35(12):827–841

    Google Scholar 

  123. Pinto G, Wiese I, Dias LF (2018) How do scientists develop and use scientific software? An external replication. In: Proceedings of 25th IEEE international conference on software analysis, evolution and reengineering. pp 582–591. https://doi.org/10.1109/SANER.2018.8330263

  124. Pinto G, Steinmacher I, Gerosa MA (2016) More common than you think: an in-depth study of casual contributors. In: 2016 IEEE 23rd international conference on software analysis, evolution, and reengineering (SANER), vol 1, pp 112–123. https://doi.org/10.1109/SANER.2016.68

  125. Nangia U, Katz DS (2017) Track 1 paper: surveying the U.S. National Postdoctoral Association regarding software use and training in research. Zenodo, pp 1–6. https://doi.org/10.5281/zenodo.814220. This paper was submitted to WSSSPE5.1. http://wssspe.researchcomputing.org.uk/wssspe5-1/ The final accepted version is https://doi.org/10.6084/m9.figshare.5328442

  126. Hannay JE, MacLeod C, Singer J, Langtangen HP, Pfahl D, Wilson G (2009) How do scientists develop and use scientific software?. In: Proceedings of the 2009 ICSE workshop on software engineering for computational science and engineering (SECSE ’09). IEEE Computer Society, Washington, pp 1–8. https://doi.org/10.1109/SECSE.2009.5069155

  127. Goble C (2014) Better software, better research. IEEE Internet Comput 18(5):4–8. https://doi.org/10.1109/MIC.2014.88

    Article  Google Scholar 

  128. Katerbow M, Feulner G (2018) Recommendations on the development, use and provision of Research Software. https://doi.org/10.5281/zenodo.1172988

  129. Howison J, Bullard J (2016) Software in the scientific literature: problems with seeing, finding, and using software mentioned in the biology literature. J Assoc Inf Sci Technol 67(9):2137–2155. https://doi.org/10.1002/asi.23538

    Article  Google Scholar 

  130. Crouch S, Hong NC, Hettrick S, Jackson M, Pawlik A, Sufi S, Carr L, De Roure D, Goble C, Parsons M (2013) The software sustainability institute: changing research software attitudes and practices. Comput Sci Eng 15(6):74–80. https://doi.org/10.1109/MCSE.2013.133

    Article  Google Scholar 

  131. Wilson G (2016) Software carpentry: lessons learned [version 2; referees: 3 approved]. F1000Res 3(62):1–12

    Google Scholar 

  132. Wilson G, Lumsdaine A (2006) Software carpentry: getting scientists to write better code by making them more productive. Comput Sci Eng 8(6):66–69. https://doi.org/10.1109/MCSE.2006.122

    Article  Google Scholar 

  133. Kanewala U, Bieman JM (2013) Techniques for testing scientific programs without an oracle. In: 5th International workshop on software engineering for computational science and engineering (SE-CSE), pp 48–57. https://doi.org/10.1109/SECSE.2013.6615099

  134. Kelly DF, Spencer Smith W, Meng N (2011) Software engineering for scientists. Comput Sci Eng 13(5):7–11

    Google Scholar 

  135. Kruchten P, Nord RL, Ozkaya I (2012) Technical debt: from metaphor to theory and practice. IEEE Softw 29(6):18–21

    Google Scholar 

  136. Lethbridge TC, Singer J, Forward A (2003) How software engineers use documentation: the state of the practice. IEEE Softw 20(6):35–39. https://doi.org/10.1109/MS.2003.1241364

    Article  Google Scholar 

  137. Stewart G et al (2017) A roadmap for HEP software and computing R &D for the 2020s. http://arxiv.org/abs/1712.06982

  138. Greg Wilson DA, Aruliah CT, Brown NP, Hong C, Davis M, Guy RT, Haddock SHD, Huff KD, Mitchell IM, Plumbley MD, Waugh B, White EP, Wilson P (2014) Best practices for scientific computing. PLoS Biol 12(1):e1001745. https://doi.org/10.1371/journal.pbio.1001745

    Article  Google Scholar 

  139. Spencer Smith W, Lai L, Khedri R (2007) Requirements analysis for engineering computation: a systematic approach for improving software reliability. Reliab Comput Spec Issue Reliab Eng Comput 13(1):83–107. https://doi.org/10.1007/s11155-006-9020-7

    Article  Google Scholar 

  140. Wikipedia (2022) Lint (software). https://en.wikipedia.org/wiki/Lint_(software)

  141. SourceLevel (2022) What is a linter and why your team should use it? https://sourcelevel.io/blog/what-is-a-linter-and-why-your-team-should-use-it

  142. Jones C (2008) Measuring defect potentials and defect removal efficiency. Crosstalk J Def Softw Eng 21(6):11–13

    Google Scholar 

  143. Ebert C, Jones C (2009) Embedded software: facts, figures, and future. Computer 42(4):42–52. https://doi.org/10.1109/MC.2009.118

    Article  Google Scholar 

  144. Kelly D, Shepard T (2000) Task-directed software inspection technique: an experiment and case study. In: CASCON ’00: Proceedings of the 2000 conference of the Centre for Advanced Studies on Collaborative research (Mississauga, Ontario, Canada). IBM Press, 6. http://portal.acm.org/citation.cfm?id=782040#

  145. Smith AM, Niemeyer KE, Katz DS, Barba LA, Githinji G, Gymrek M, Huff KD, Madan CR, Mayes AC, Moerman KM, Prins P, Ram K, Rokem A, Teal TK, Guimera RV, Vanderplas JT (2018) Journal of open source software (JOSS): design and first-year review. PeerJ Comput Sci 4:e147. https://doi.org/10.7717/peerj-cs.147

    Article  Google Scholar 

  146. Smith AM, Katz DS, Niemeyer KE, FORCE11 Software Citation Working Group (2016) Software citation principles. PeerJ 4:e2169v4. https://doi.org/10.7287/peerj.preprints.2169v4

  147. Chue Hong NP, Allen A, Gonzalez-Beltran A, de Waard A, Smith AM, Robinson C, Jones C, Bouquin D, Katz DS, Kennedy D, Ryder G, Hausman J, Hwang L, Jones MB, Harrison M, Crosas Me, Wu M, Löwe P, Haines R, Edmunds S, Stall S, Swaminathan S, Druskat S, Crick T, Morrell T, Pollard T (2019) Software citation checklist for developers. https://doi.org/10.5281/zenodo.3482769

  148. Katz DS, Chue Hong NP, Clark T, Muench A, Stall S, Bouquin D, Cannon M, Edmunds S, Faez T, Feeney P, Fenner M, Friedman M, Grenier G, Harrison M, Heber J, Leary A, MacCallum C, Murray H, Pastrana E, Perry K, Schuster D, Stockhause M, Yeston J (2021) Recognizing the value of software: a software citation guide [version 2; peer review: 2 approved]. F1000Res 9:1257. https://doi.org/10.12688/f1000research.26932.2

    Article  Google Scholar 

  149. Smith A (2022) Enhanced support for citations on GitHub. https://github.blog/2021-08-19-enhanced-support-citations-github/

  150. Garcia J (2016) How do you get programmers to join your project? https://opensource.com/business/16/9/how-to-get-programmers

  151. Jalan S (2016) How to attract new contributors to your open source project. https://www.freecodecamp.org/news/how-to-attract-new-contributors-to-your-open-source-project-46f8b791d787/

  152. Brian Proffitt (2017) How to attract new contributors. https://www.redhat.com/en/blog/how-attract-new-contributors

  153. McQuaid M (2018) The open source contributor funnel (or: why people don’t contribute to your open source project). https://mikemcquaid.com/2018/08/14/the-open-source-contributor-funnel-why-people-dont-contribute-to-your-open-source-project/

  154. Kuchner MJ (2012) Marketing for scientists: how to shine in tough times. Island Press, Washington

    Google Scholar 

  155. Roache PJ (1998) Verification and validation in computational science and engineering. Hermosa Publishers, Albuquerque

    Google Scholar 

  156. Hickey T, Qun J, Van Emden MH (2001) Interval arithmetic: from principles to implementation. J ACM 48(5):1038–1068. https://doi.org/10.1145/502102.502106

    Article  MathSciNet  Google Scholar 

  157. Kanewala U, Lundgren A (2016) Automated metamorphic testing of scientific software. In: Hong CNC, and George T (eds) Software Engineering for Science. Chapter examples of the application of traditional software engineering practices to science. Jeffrey C. Taylor & Francis, pp 151–174

  158. Smith WS, Lai L (2005) A new requirements template for scientific computing. In: Proceedings of the first international workshop on situational requirements engineering processes – methods, techniques and tools to support situation-specific requirements engineering processes, SREP’05. In: Ralyté J, Agerfalk P, Kraiem N (eds) Conjunction with 13th IEEE International Requirements Engineering Conference, Paris, France, pp 107–121

  159. Piccolo SR, Frampton MB (2016) Tools and techniques for computational reproducibility. GigaScience 5(1):30. https://doi.org/10.1186/s13742-016-0135-4

    Article  Google Scholar 

  160. Crick T, Hall BA, Ishtiaq S (2014) “Can i implement your algorithm?”: a model for reproducible research software. http://arxiv.org/abs/1407.5981

  161. Ionescu C, Jansson P (2012) Dependently-typed programming in scientific computing—examples from economic modelling. In: Revised selected papers of the 24th international symposium on implementation and application of functional languages (Lecture notes in computer science, vol 8241). Springer International Publishing, Cham, pp 140–156. https://doi.org/10.1007/978-3-642-41582-1_9

  162. Luciv DV, Koznov DV, Chernishev GA, Terekhov AN, Romanovsky KY, Grigoriev DA (2018) Detecting near duplicates in software documentation. Program Comput Softw 44(5):335–343. https://doi.org/10.1134/S0361768818050079

    Article  MathSciNet  Google Scholar 

  163. Whaley RC, Petitet A, Dongarra JJ (2001) Automated empirical optimization of software and the ATLAS project. Parallel Comput 27(1–2):3–35

    Google Scholar 

  164. Veldhuizen TL (1998) Arrays in Blitz++. In: Proceedings of the 2nd international scientific computing in object-oriented parallel environments (ISCOPE’98), lecture notes in computer science. Springer-Verlag

  165. Püschel M, Singer B, Veloso M, Moura José MF (2001) Fast automatic generation of DSP algorithms. In: International conference on computational science (ICCS). (Lecture notes in computer science, vol 2073). Springer, pp 97–106

  166. Carette J, Kiselyov O (2011) Multi-stage programming with Functors and Monads: eliminating abstraction overhead from generic code. Sci Comput Program 76(5):349–375

    MathSciNet  Google Scholar 

  167. Logg A, Mardal K-A, Wells GN (eds). (2012) Automated solution of differential equations by the finite element method. Lecture notes in computational science and engineering, vol 84. Springer. https://doi.org/10.1007/978-3-642-23099-8

  168. Ober I, Palyart M, Bruel J-M, Lugato D (2018) On the use of models for high-performance scientific computing applications: an experience report. Softw Syst Model 17(1):319–342. https://doi.org/10.1007/s10270-016-0518-0

    Article  Google Scholar 

  169. Matkerim B, Akhmed-Zaki D, Barata M (2013) Development high performance scientific computing application using model-driven architecture. Appl Math Sci 7(100):4961–4974

    Google Scholar 

  170. Szymczak D, Smith WS, Carette J (2016) Position paper: a knowledge-based approach to scientific software development. In: Proceedings of SE4Science’16 in conjunction with the international conference on software engineering (ICSE). In conjunction with ICSE 2016, Austin, p 4

  171. Smith WS, Carette J (2021) Sustainable software via generation. In: Proceedings of the 1st annual booth resource and innovation cluster (BRIC) symposium. 21

  172. Carette J, Smith S, Balaci J, Hunt A, Wu T-Y, Crawford S, Chen D, Szymczak D, MacLachlan B, Scime D, Niazi M (2021) Drasil. https://github.com/JacquesCarette/Drasil/tree/v0.1-alpha

  173. Ampatzoglou A, Bibi S, Avgeriou P, Verbeek M, Chatzigeorgiou A (2019) Identifying, categorizing and mitigating threats to validity in software engineering secondary studies. Inf Softw Technol 106:201–230. https://doi.org/10.1016/j.infsof.2018.10.006

    Article  Google Scholar 

  174. Runeson P, Höst M (2009) Guidelines for conducting and reporting case study research in software engineering. Emp Softw Eng 14(2):131–164. https://doi.org/10.1007/s10664-008-9102-8

    Article  Google Scholar 

  175. van Vliet H (2000) Software engineering. Principles and practice, 2nd edn. Wiley, New York

    Google Scholar 

  176. Smith WS, Carette J, McCutchan J (2008) Commonality analysis of families of physical models for use in scientific computing. In: Proceedings of the first international workshop on software engineering for computational science and engineering (SECSE08)

  177. Weiss DM (1998) Commonality analysis: a systematic process for defining families. In: International workshop on architectural reasoning for embedded systems. springer, pp 214–222. https://citeseer.ist.psu.edu/13585.html

  178. Smith WS, Chen C-H (2004) Commonality and requirements analysis for mesh generating software. In: Maurer F, Ruhe G (eds) Proceedings of the sixteenth international conference on software engineering and knowledge engineering (SEKE 2004). Banff, Alberta, pp 384–387

  179. Smith WS, McCutchan J, Carette J (2017) Commonality analysis for a family of material models. Technical Report CAS-17-01-SS. McMaster University, Department of Computing and Software

Download references

Acknowledgements

We would like to thank Ao Dong, Oluwaseun Owojaiye, Mohammad Ali Daeian, and Reza Sadeghi for fruitful discussions on topics relevant to this paper.

Author information

Authors and Affiliations

  1. Computing and Software Department, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4K1, Canada

    Spencer Smith, Peter Michalski & Jacques Carette

  2. Mechanical Engineering, McMaster University, 1280 Main Street West, Hamilton, ON, L8S 4L7, Canada

    Zahra Keshavarz-Motamed

Authors
  1. Spencer Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Peter Michalski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jacques Carette
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zahra Keshavarz-Motamed
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Spencer Smith.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smith, S., Michalski, P., Carette, J. et al. State of the Practice for Lattice Boltzmann Method Software. Arch Computat Methods Eng 31, 313–350 (2024). https://doi.org/10.1007/s11831-023-09981-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11831-023-09981-2

Search

Navigation