Abstract
Operating system (OS) updates introduce numerical perturbations that impact the reproducibility of computational pipelines. In neuroimaging, this has important practical implications on the validity of computational results, particularly when obtained in systems such as high-performance computing clusters where the experimenter does not control software updates. We present a framework to reproduce the variability induced by OS updates in controlled conditions. We hypothesize that OS updates impact computational pipelines mainly through numerical perturbations originating in mathematical libraries, which we simulate using Monte-Carlo arithmetic in a framework called “fuzzy libmath” (FL). We applied this methodology to pre-processing pipelines of the Human Connectome Project, a flagship open-data project in neuroimaging. We found that FL-perturbed pipelines accurately reproduce the variability induced by OS updates and that this similarity is only mildly dependent on simulation parameters. Importantly, we also found between-subject differences were preserved in both cases, though the between-run variability was of comparable magnitude for both FL and OS perturbations. We found the numerical precision in the HCP pre-processed images to be relatively low, with less than 8 significant bits among the 24 available, which motivates further investigation of the numerical stability of components in the tested pipeline. Overall, our results establish that FL accurately simulates results variability due to OS updates, and is a practical framework to quantify numerical uncertainty in neuroimaging.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andersson, J.L., Jenkinson, M., Smith, S., et al.: Non-linear registration, aka spatial normalisation FMRIB. Technical report TR07JA2, FMRIB Analysis Group of the University of Oxford (2007)
Botvinik-Nezer, R., et al.: Variability in the analysis of a single neuroimaging dataset by many teams. Nature 582(7810), 84–88 (2020)
Bowring, A., Maumet, C., Nichols, T.E.: Exploring the impact of analysis software on task fMRI results. Hum. Brain Mapp. 40, 1–23 (2019)
Chatelain, Y., de Oliveira Castro, P., Petit, E., Defour, D., Bieder, J., Torrent, M.: VeriTracer: context-enriched tracer for floating-point arithmetic analysis. In: 2018 IEEE 25th Symposium on Computer Arithmetic (ARITH), pp. 61–68. IEEE (2018)
Denis, C., de Oliveira Castro, P., Petit, E.: Verificarlo: checking floating point accuracy through Monte Carlo arithmetic. In: 2016 IEEE 23nd Symposium on Computer Arithmetic (ARITH), pp. 55–62 (2016)
Esteban, O., et al.: fMRIPrep: a robust preprocessing pipeline for functional MRI. Nat. Methods 16(1), 111–116 (2019)
Glasser, M.F., et al.: The minimal preprocessing pipelines for the Human Connectome Project. NeuroImage 80, 105–124 (2013)
Glatard, T., et al.: Reproducibility of neuroimaging analyses across operating systems. Front. Neuroinform. 9, 12 (2015)
Gronenschild, E.H.B.M., et al.: The effects of FreeSurfer version, workstation type, and Macintosh operating system version on anatomical volume and cortical thickness measurements. PloS ONE 7(6), e38234 (2012)
Hanke, M., Halchenko, Y.O.: Neuroscience runs on GNU/Linux. Front. Neuroinform. 5, 8 (2011)
Jenkinson, M., Bannister, P., Brady, M., Smith, S.: Improved optimization for the robust and accurate linear registration and motion correction of brain images. NeuroImage 17(2), 825–841 (2002)
Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, M.W., Smith, S.M.: FSL. NeuroImage 62(2), 782–790 (2012)
Jenkinson, M., Smith, S.: A global optimisation method for robust affine registration of brain images. Med. Image Anal. 5(2), 143–156 (2001)
Kaur, B., Dugré, M., Hanna, A., Glatard, T.: An analysis of security vulnerabilities in container images for scientific data analysis. GigaScience 10(6), giab025 (2021)
Kiar, G., Chatelain, Y., Salari, A., Evans, A.C., Glatard, T.: Data augmentation through Monte Carlo arithmetic leads to more generalizable classification in connectomics. bioRxiv (2020)
Parker, D.S.: Monte Carlo arithmetic: exploiting randomness in floating-point arithmetic. Computer Science Department, University of California, Los Angeles (1997)
Perkel, J.M.: Challenge to scientists: does your ten-year-old code still run? Nature 584(7822), 656–658 (2020)
Salari, A., Kiar, G., Lewis, L., Evans, A.C., Glatard, T.: File-based localization of numerical perturbations in data analysis pipelines. GigaScience 9(12), giaa106 (2020)
Van Essen, D.C., et al.: The WU-Minn human connectome project: an overview. NeuroImage 80, 62–79 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Salari, A., Chatelain, Y., Kiar, G., Glatard, T. (2021). Accurate Simulation of Operating System Updates in Neuroimaging Using Monte-Carlo Arithmetic. In: Sudre, C.H., et al. Uncertainty for Safe Utilization of Machine Learning in Medical Imaging, and Perinatal Imaging, Placental and Preterm Image Analysis. UNSURE PIPPI 2021 2021. Lecture Notes in Computer Science(), vol 12959. Springer, Cham. https://doi.org/10.1007/978-3-030-87735-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-030-87735-4_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-87734-7
Online ISBN: 978-3-030-87735-4
eBook Packages: Computer ScienceComputer Science (R0)
