Ensemble of 3D CNN Regressors with Data Fusion for Fluid Intelligence Prediction

  • Marina Pominova
  • Anna Kuzina
  • Ekaterina KondratevaEmail author
  • Svetlana Sushchinskaya
  • Evgeny Burnaev
  • Vyacheslav Yarkin
  • Maxim Sharaev
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11791)


In this work, we aimed at predicting children’s fluid intelligence scores based on structural T1-weighted MR images from the largest long-term study of brain development and child health. The target variable was regressed on a data collection site, sociodemographic variables, and brain volume, thus being independent to the potentially informative factors, which were not directly related to the brain functioning. We investigated both feature extraction and deep learning approaches as well as different deep CNN architectures and their ensembles. We proposed an advanced architecture of VoxCNNs ensemble, which yields MSE (92.838) on a blind test.


MRI analysis Fluid intelligence prediction Deep learning 3D convolutional neural networks VoxCNN ensemble 



The work was supported by the Russian Science Foundation under Grant 19-41-04109.

The considered problem was formulated in the scope of the Project “Machine Learning and Pattern Recognition for the development of diagnostic and clinical prognostic prediction tools in psychiatry, borderline mental disorders, and neurology”, granted by Skoltech Biomedical Initiative Program, Skolkovo Institute of Science and Technology, Moscow, Russia.


  1. 1.
    Carroll, J.B.: Human Cognitive Abilities: A Survey of Factor-Analytic Studies. Cambridge University Press, Cambridge (1993)CrossRefGoogle Scholar
  2. 2.
    RobertWCox: AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. In: Computers and Biomedical Research, vol. 29, no. 3, pp. 162–173 (1996)Google Scholar
  3. 3.
    Smith, S.M.: Fast robust automated brain extraction. Hum. Brain Mapp. 17(3), 143–155 (2002)CrossRefGoogle Scholar
  4. 4.
    Avants, B.B., Tustison, N., Song, G.: Advanced normalization tools (ANTS). Insight j 2, 1–35 (2009)Google Scholar
  5. 5.
    Rohlfing, T., et al.: The SRI24 multichannel atlas of normal adult human brain structure. Hum. Brain Mapp. 31(5), 798–819 (2010)CrossRefGoogle Scholar
  6. 6.
    Sadananthan, S.A., et al.: Skull stripping using graph cuts. NeuroImage 49(1), 225–239 (2010)CrossRefGoogle Scholar
  7. 7.
    Avants, B.B., et al.: An open source multivariate framework for n-tissue segmentation with evaluation on public data. Neuroinformatics 9(4), 381–400 (2011)CrossRefGoogle Scholar
  8. 8.
    Iglesias, J.E., et al.: Robust brain extraction across datasets and comparison with publicly available methods. IEEE Trans. Med. Imaging 30(9), 1617–1634 (2011)CrossRefGoogle Scholar
  9. 9.
    Burnaev, E.V., Prikhod’ko, P.V.: On a method for constructing ensembles of regression models. Autom. Remote Control 74(10), 1630–1644 (2013)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Burnaev, E., Vovk, V.: Efficiency of conformalized ridge regression. In: Balcan, M.F., Feldman, V., Szepesvari, C. (eds.) Proceedings of the 27th Conference on Learning Theory. Proceedings of Machine Learning Research, PMLR, Barcelona, Spain, 13–15 Jun 2014, vol. 35, pp. 605–622 (2014)Google Scholar
  11. 11.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  12. 12.
    Brown, S.A., et al.: The national consortium on alcohol and neurodevelopment in adolescence (NCANDA): a multisite study of adolescent development and substance use. J. Stud. Alcohol Drugs 76(6), 895–908 (2015)CrossRefGoogle Scholar
  13. 13.
    Burnaev, E., Zaytsev, A.: Surrogate modeling of multifidelity data for large samples. J. Commun. Technol. Electron. 60(12), 1348–1355 (2015)CrossRefGoogle Scholar
  14. 14.
    Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)Google Scholar
  15. 15.
    Tompson, J., et al.: Efficient object localization using convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 648–656 (2015)Google Scholar
  16. 16.
    Burnaev, E., Erofeev, P.: The influence of parameter initialization on the training time and accuracy of a nonlinear regression model. J. Commun. Technol. Electron. 61(6), 646–660 (2016). ISSN 1555-6557CrossRefGoogle Scholar
  17. 17.
    Burnaev, E., Nazarov, I.: Conformalized Kernel ridge regression. In: 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 45–52 (2016)Google Scholar
  18. 18.
    Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)
  19. 19.
    Goetz, M., et al.: DALSA: domain adaptation for supervised learning from sparsely annotated MR images. IEEE Trans. Med. Imaging 35(1), 184–196 (2016)CrossRefGoogle Scholar
  20. 20.
    He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp. 770–778 (2016)Google Scholar
  21. 21.
    Hosseini-Asl, E., Gimel’farb, G., El-Baz, A.: Alzheimer’s disease diagnostics by a deeply supervised adaptable 3D convolutional network. arXiv preprint arXiv:1607.00556 (2016)
  22. 22.
    Milletari, F., Navab, N., Ahmadi, S.-A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)Google Scholar
  23. 23.
    Paul, E.J., et al.: Dissociable brain biomarkers of UID intelligence. NeuroImage 137, 201–211 (2016)CrossRefGoogle Scholar
  24. 24.
    Szegedy, C., et al.: Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2818–2826 (2016)Google Scholar
  25. 25.
    Ghafoorian, M., et al.: Transfer learning for domain adaptation in MRI: application in brain lesion segmentation. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10435, pp. 516–524. Springer, Cham (2017). Scholar
  26. 26.
    Hunyadi, B., et al.: Tensor decompositions and data fusion in epileptic electroencephalography and functional magnetic resonance imaging data. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 7(1), e1197 (2017)Google Scholar
  27. 27.
    Korolev, S., et al.: Residual and plain convolutional neural networks for 3D brain MRI classification. In: IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017), pp. 835–838. IEEE (2017)Google Scholar
  28. 28.
    Lu, H., et al.: When unsupervised domain adaptation meets tensor representations. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 599–608 (2017)Google Scholar
  29. 29.
    Pfefferbaum, A., et al.: Altered brain developmental trajectories in adolescents after initiating drinking. Am. J. Psychiatry 175(4), 370–380 (2017)CrossRefGoogle Scholar
  30. 30.
    Zaytsev, A., Burnaev, E.: Large scale variable fidelity surrogate modeling. Ann. Math. Artif. Intell. 81(1), 167–186 (2017). ISSN 1573-7470MathSciNetCrossRefGoogle Scholar
  31. 31.
    Zaytsev, A., Burnaev, E.: Minimax approach to variable fidelity data interpolation. In: Singh, A., Zhu, J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Proceedings of Machine Learning Research, PMLR, Fort Lauderdale, FL, USA, 20–22 Apr 2017, vol. 54, pp. 652–661 (2017)Google Scholar
  32. 32.
    Chen, H., et al.: VoxResNet: deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage 170, 446–455 (2018)CrossRefGoogle Scholar
  33. 33.
    Hagler, D.J., et al.: Image processing and analysis methods for the adolescent brain cognitive development study. bioRxiv, p. 457739 (2018)Google Scholar
  34. 34.
    Ivanov, S., et al.: Learning connectivity patterns via graph kernels for fMRI-based Depression Diagnostics. In: Proceedings of IEEE International Conference on Data Mining Workshops (ICDMW), pp. 308–314 (2018)Google Scholar
  35. 35.
    Kuleshov, A., Bernstein, A., Burnaev, E.: Conformal prediction in manifold learning. In: Gammerman, A., et al. (eds.) Proceedings of the Seventh Workshop on Conformal and Probabilistic Prediction and Applications, Proceedings of Machine Learning Research, PMLR, vol. 91. pp. 234–253 (2018)Google Scholar
  36. 36.
    Notchenko, A., Kapushev, Y., Burnaev, E.: Large-scale shape retrieval with sparse 3D convolutional neural networks. In: van der Aalst, W.M.P., et al. (eds.) AIST 2017. LNCS, vol. 10716, pp. 245–254. Springer, Cham (2018). Scholar
  37. 37.
    Pominova, M., et al.: Voxelwise 3D convolutional and recurrent neural networks for epilepsy and depression diagnostics from structural and functional MRI Data. In: 2018 IEEE International Conference on Data Mining Workshops (ICDMW), pp. 299–307. IEEE (2018)Google Scholar
  38. 38.
    Sharaev, M., et al.: MRI-based diagnostics of depression concomitant with epilepsy: in search of the potential biomarkers. In: Proceedings of IEEE 5th International Conference on Data Science and Advanced Analytics, pp. 555–564 (2018)Google Scholar
  39. 39.
    Sharaev, M., et al.: Pattern recognition pipeline for neuroimaging data. In: Pancioni, L., Schwenker, F., Trentin, E. (eds.) ANNPR 2018. LNCS (LNAI), vol. 11081, pp. 306–319. Springer, Cham (2018). Scholar
  40. 40.
    Zhu, M., Liu, B., Li, J.: Prediction of general fluid intelligence using cortical measurements and underlying genetic mechanisms. In: IOP Conference Series: Materials Science and Engineering, vol. 381, no. 1, p. 012186. IOP Publishing (2018)Google Scholar
  41. 41.
    Eckle, K., Schmidt-Hieber, J.: A comparison of deep networks with ReLU activation function and linear spline-type methods. Neural Netw. 110, 232–242 (2019)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Marina Pominova
    • 1
  • Anna Kuzina
    • 1
  • Ekaterina Kondrateva
    • 1
    Email author
  • Svetlana Sushchinskaya
    • 1
  • Evgeny Burnaev
    • 1
  • Vyacheslav Yarkin
    • 1
  • Maxim Sharaev
    • 1
  1. 1.Skolkovo Institute of Science and TechnologyMoscowRussia

Personalised recommendations