Skip to main content

Accurate Simulation of Operating System Updates in Neuroimaging Using Monte-Carlo Arithmetic

  • 864 Accesses

Part of the Lecture Notes in Computer Science book series (LNIP,volume 12959)

Abstract

Operating system (OS) updates introduce numerical perturbations that impact the reproducibility of computational pipelines. In neuroimaging, this has important practical implications on the validity of computational results, particularly when obtained in systems such as high-performance computing clusters where the experimenter does not control software updates. We present a framework to reproduce the variability induced by OS updates in controlled conditions. We hypothesize that OS updates impact computational pipelines mainly through numerical perturbations originating in mathematical libraries, which we simulate using Monte-Carlo arithmetic in a framework called “fuzzy libmath” (FL). We applied this methodology to pre-processing pipelines of the Human Connectome Project, a flagship open-data project in neuroimaging. We found that FL-perturbed pipelines accurately reproduce the variability induced by OS updates and that this similarity is only mildly dependent on simulation parameters. Importantly, we also found between-subject differences were preserved in both cases, though the between-run variability was of comparable magnitude for both FL and OS perturbations. We found the numerical precision in the HCP pre-processed images to be relatively low, with less than 8 significant bits among the 24 available, which motivates further investigation of the numerical stability of components in the tested pipeline. Overall, our results establish that FL accurately simulates results variability due to OS updates, and is a practical framework to quantify numerical uncertainty in neuroimaging.

Keywords

  • Computational reproducibility
  • Neuroimaging pipelines
  • Monte-Carlo arithmetic

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-87735-4_2
  • Chapter length: 10 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   54.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-87735-4
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   69.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.

References

  1. Andersson, J.L., Jenkinson, M., Smith, S., et al.: Non-linear registration, aka spatial normalisation FMRIB. Technical report TR07JA2, FMRIB Analysis Group of the University of Oxford (2007)

    Google Scholar 

  2. Botvinik-Nezer, R., et al.: Variability in the analysis of a single neuroimaging dataset by many teams. Nature 582(7810), 84–88 (2020)

    CrossRef  Google Scholar 

  3. Bowring, A., Maumet, C., Nichols, T.E.: Exploring the impact of analysis software on task fMRI results. Hum. Brain Mapp. 40, 1–23 (2019)

    CrossRef  Google Scholar 

  4. Chatelain, Y., de Oliveira Castro, P., Petit, E., Defour, D., Bieder, J., Torrent, M.: VeriTracer: context-enriched tracer for floating-point arithmetic analysis. In: 2018 IEEE 25th Symposium on Computer Arithmetic (ARITH), pp. 61–68. IEEE (2018)

    Google Scholar 

  5. Denis, C., de Oliveira Castro, P., Petit, E.: Verificarlo: checking floating point accuracy through Monte Carlo arithmetic. In: 2016 IEEE 23nd Symposium on Computer Arithmetic (ARITH), pp. 55–62 (2016)

    Google Scholar 

  6. Esteban, O., et al.: fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat. Methods 16(1), 111–116 (2019)

    CrossRef  Google Scholar 

  7. Glasser, M.F., et al.: The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage 80, 105–124 (2013)

    CrossRef  Google Scholar 

  8. Glatard, T., et al.: Reproducibility of neuroimaging analyses across operating systems. Front. Neuroinform. 9, 12 (2015)

    CrossRef  Google Scholar 

  9. Gronenschild, E.H.B.M., et al.: The effects of FreeSurfer version, workstation type, and Macintosh operating system version on anatomical volume and cortical thickness measurements. PloS ONE 7(6), e38234 (2012)

    CrossRef  Google Scholar 

  10. Hanke, M., Halchenko, Y.O.: Neuroscience runs on GNU/Linux. Front. Neuroinform. 5, 8 (2011)

    CrossRef  Google Scholar 

  11. Jenkinson, M., Bannister, P., Brady, M., Smith, S.: Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17(2), 825–841 (2002)

    CrossRef  Google Scholar 

  12. Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, M.W., Smith, S.M.: FSL. NeuroImage 62(2), 782–790 (2012)

    CrossRef  Google Scholar 

  13. Jenkinson, M., Smith, S.: A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5(2), 143–156 (2001)

    CrossRef  Google Scholar 

  14. Kaur, B., Dugré, M., Hanna, A., Glatard, T.: An analysis of security vulnerabilities in container images for scientific data analysis. GigaScience 10(6), giab025 (2021)

    CrossRef  Google Scholar 

  15. Kiar, G., Chatelain, Y., Salari, A., Evans, A.C., Glatard, T.: Data augmentation through Monte Carlo arithmetic leads to more generalizable classification in connectomics. bioRxiv (2020)

    Google Scholar 

  16. Parker, D.S.: Monte Carlo arithmetic: exploiting randomness in floating-point arithmetic. Computer Science Department, University of California, Los Angeles (1997)

    Google Scholar 

  17. Perkel, J.M.: Challenge to scientists: does your ten-year-old code still run? Nature 584(7822), 656–658 (2020)

    CrossRef  Google Scholar 

  18. Salari, A., Kiar, G., Lewis, L., Evans, A.C., Glatard, T.: File-based localization of numerical perturbations in data analysis pipelines. GigaScience 9(12), giaa106 (2020)

    CrossRef  Google Scholar 

  19. Van Essen, D.C., et al.: The WU-Minn human connectome project: an overview. NeuroImage 80, 62–79 (2013)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ali Salari .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Salari, A., Chatelain, Y., Kiar, G., Glatard, T. (2021). Accurate Simulation of Operating System Updates in Neuroimaging Using Monte-Carlo Arithmetic. In: , et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis. UNSURE PIPPI 2021 2021. Lecture Notes in Computer Science(), vol 12959. Springer, Cham. https://doi.org/10.1007/978-3-030-87735-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-87735-4_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-87734-7

  • Online ISBN: 978-3-030-87735-4

  • eBook Packages: Computer ScienceComputer Science (R0)