Journal of Digital Imaging

, Volume 31, Issue 6, pp 895–903 | Cite as

Combination of rs-fMRI and sMRI Data to Discriminate Autism Spectrum Disorders in Young Children Using Deep Belief Network

  • Maryam Akhavan AghdamEmail author
  • Arash Sharifi
  • Mir Mohsen Pedram


In recent years, the use of advanced magnetic resonance (MR) imaging methods such as functional magnetic resonance imaging (fMRI) and structural magnetic resonance imaging (sMRI) has recorded a great increase in neuropsychiatric disorders. Deep learning is a branch of machine learning that is increasingly being used for applications of medical image analysis such as computer-aided diagnosis. In a bid to classify and represent learning tasks, this study utilized one of the most powerful deep learning algorithms (deep belief network (DBN)) for the combination of data from Autism Brain Imaging Data Exchange I and II (ABIDE I and ABIDE II) datasets. The DBN was employed so as to focus on the combination of resting-state fMRI (rs-fMRI), gray matter (GM), and white matter (WM) data. This was done based on the brain regions that were defined using the automated anatomical labeling (AAL), in order to classify autism spectrum disorders (ASDs) from typical controls (TCs). Since the diagnosis of ASD is much more effective at an early age, only 185 individuals (116 ASD and 69 TC) ranging in age from 5 to 10 years were included in this analysis. In contrast, the proposed method is used to exploit the latent or abstract high-level features inside rs-fMRI and sMRI data while the old methods consider only the simple low-level features extracted from neuroimages. Moreover, combining multiple data types and increasing the depth of DBN can improve classification accuracy. In this study, the best combination comprised rs-fMRI, GM, and WM for DBN of depth 3 with 65.56% accuracy (sensitivity = 84%, specificity = 32.96%, F1 score = 74.76%) obtained via 10-fold cross-validation. This result outperforms previously presented methods on ABIDE I dataset.


Autism spectrum disorder rs-fMRI sMRI Gray matter White matter Deep belief network 



The authors express gratitude to the Autism Brain Imaging Data Exchange (ABIDE), for generously sharing their data with the scientific community.


  1. 1.
    Rapin I, Tuchman RF: What is new in autism? Curr Opin Neurol. Apr 1 21(2):143–149, 2008CrossRefGoogle Scholar
  2. 2.
    Mueller S, Keeser D, Reiser MF, Teipel S, Meindl T: Functional and Structural MR Imaging in Neuropsychiatric Disorders, Part 2: Application in Schizophrenia and Autism. AJNR Am J Neuroradiol 33:2033–2037, 2012CrossRefGoogle Scholar
  3. 3.
    Office of Special Education Programs, United States Department Of Education, Twenty-Seventh Annual Report to Congress on the Implementation of the Individuals with Dis- abilities Education Act, 2005.Google Scholar
  4. 4.
    Levy SE, Mandell DS, Schultz RT: Autism. The Lancet 374(9701):1627–1638, 2009CrossRefGoogle Scholar
  5. 5.
    Coleman M, Gillberg C: The Autisms. Oxford; Oxford University Press, 2012Google Scholar
  6. 6.
    Waterhouse L: Rethinking Autism: Variation and Complexity. London: Academic Press, 2013Google Scholar
  7. 7.
    Fernell E, Eriksson MA, Gillberg C: Early diagnosis of autism and impact on prognosis: a narrative review. Clin. Epidemiol. 5:33–43, 2013CrossRefGoogle Scholar
  8. 8.
    Pennington ML, Cullinan D, Southern LB, Defining Autism: Variability in State Education Agency Definitions of and Evaluations for Autism Spectrum Disorders, 2014. Available at:,CrossRefGoogle Scholar
  9. 9.
    Saniano M, Pellegrino L, Casadio M, Summa S, Garbanio E, Rossi V, Dall’Agata D, Sanguineti V, Natural interface and virtual environments for the acquisition of street crossing and path following skills in adults with Autism Spectrum Disorders: a feasibility study. J Neuroeng Rehabil, 2015.Google Scholar
  10. 10.
    Yerys BE, Pennington BF: How do we establish a biological marker for a behaviorally defined disorder? Autism as a test case. Autism Res. 4(4):239–241, 2011CrossRefGoogle Scholar
  11. 11.
    Plitt M, Barnes KA, Martin A: Functional connectivity classification of autism identifies highly predictive brain features but falls short of biomarker standards. Neuroimage Clin. 7:359–366, 2015CrossRefGoogle Scholar
  12. 12.
    Mueller S, Keeser D, Reiser MF, Teipel S, Meindl T: Functional and Structural MR Imaging in Neuropsychiatric Disorders, Part 1: Imaging Techniques and Their Application in Mild Cognitive Impairment and Alzheimer Disease. AJNR Am J Neuroradiol 33:2033–2037, 2012CrossRefGoogle Scholar
  13. 13.
    Anderson JS, Nielsen JA, Froehlich AL, DuBray MB, Druzgal TJ, Cariello AN, Cooperrider JR, Zielinski BA, Ravichandran C, Fletcher PT, Alexander AL: Functional connectivity magnetic resonance imaging classification of autism. Brain. 134(12):3742–3754, 2011CrossRefGoogle Scholar
  14. 14.
    Uddin LQ, Supekar K, Lynch CJ, Khouzam A, Phillips J, Feinstein C, Ryali S, Menon V: Salience network-based classification and prediction of symptom severity in children with autism. JAMA Psychiatry 70(8):869–879, 2013CrossRefGoogle Scholar
  15. 15.
    Nielsen JA, Zielinski BA et al.: Multisite functional connectivity MRI classification of autism: ABIDE results. Front Hum Neurosci 7 (September:599, 2013CrossRefGoogle Scholar
  16. 16.
    Chen CP, Keown CL, Jahedi A, Nair A, Pflieger ME, Bailey BA, Müller RA: Diagnostic classification of intrinsic functional connectivity highlights somatosensory, default mode, and visual regions in autism. Neuroimage Clin. 8:238–245, 2015CrossRefGoogle Scholar
  17. 17.
    Ghiassian S, Greiner R, Jin P, Brown MRG: Using Functional or Structural Magnetic Resonance Images and Personal Characteristic Data to Identify ADHD and Autism. PLoS ONE. 11(12):e0166934, 2016CrossRefGoogle Scholar
  18. 18.
    Greimel E, Nehrkorn B, Schulte-Rüther M, Fink GR, Nickl-Jockschat T, Herpertz-Dahlmann B, Konrad K, Eickhoff SB: Changes in grey matter development in autism spectrum disorder. Brain Struct Funct. 218(4):929–942, 2013CrossRefGoogle Scholar
  19. 19.
    Wilkinson M, Wang R, van der Kouwe A, Takahashi E: White and gray matter fiber pathways in autism spectrum disorder revealed by ex vivo diffusion MR tractography. Brain Behav 6(7):e00483, 2016CrossRefGoogle Scholar
  20. 20.
    Bakhtiari R, Zürcher NR, Rogier O, Russo B, Hippolyte L, Granziera C, Araabi BN, Nili Ahmadabadi M, Hadjikhani N: Differences in white matter reflect atypical developmental trajectory in autism: A Tract-based Spatial Statistics study. Neuroimage Clin. 1(1):48–56, 2012CrossRefGoogle Scholar
  21. 21.
    McCarley RW, Nakamura M, Shenton ME, Salisbury DF: Combining ERP and structural MRI information in first episode schizophrenia and bipolar disorder. Clin EEG Neurosci 39(2):57–60, 2008CrossRefGoogle Scholar
  22. 22.
    Michael AM, Baum SA, White T, Demirci O, Andreasen NC, Segall JM, Jung RE, Pearlson G, Clark VP, Gollub RL, Schulz SC, Roffman JL, Lim KO, Ho BC, Bockholt HJ, Calhoun VD: Does function follow form? Methods to fuse structural and functional brain images show decreased linkage in schizophrenia. Neuroimage. 49(3):2626–2637, 2010CrossRefGoogle Scholar
  23. 23.
    Sui J, Pearlson G, Caprihan A, Adali T, Kiehl KA, Liu J, Yamamoto J, Calhoun VD: Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+joint ICA model. Neuroimage. 57(3):839–855, 2011CrossRefGoogle Scholar
  24. 24.
    Sui J, He H, Yu Q, Chen J, Rogers J, Pearlson G, Mayer A, Bustillo J, Canive J, Calhoun VD, Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by N-way MCCA + jICA. Fron Hum Neurosci, 7,2013.Google Scholar
  25. 25.
    Le Roux N, Bengio Y: Deep belief networks are compact universal approximators. Neural Comput. 22(8):2192–2207, 2010CrossRefGoogle Scholar
  26. 26.
    Plis SM, Hjelm D, Salakhutdinov R, Allen EA, Bockholt HJ, Long JD, Johnson HJ, Paulsen J, Turner JA, Calhoun VD: Deep learning for neuroimaging: a validation study. Front Neurosci, 8, 2014.Google Scholar
  27. 27.
    Suk HI, Lee SW, Shen D: Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. Neuroimage. 101:569–582, 2014 Available at: CrossRefGoogle Scholar
  28. 28.
    Suk HI, Lee SW, Shen D: Latent feature representation with stacked auto-encoder for AD/MCI diagnosis. Brain Struct Func 220(2):841–859, 2015CrossRefGoogle Scholar
  29. 29.
    Sarraf S, Tofighi G, Classification of Alzheimer’s Disease Using fMRI Data and Deep Learning Convolutional Neural Networks 2016. Available at:
  30. 30.
    Pang S, Yang X: Deep Convolutional Extreme Learning Machine and Its Application in Handwritten Digit Classification. Comput Intell Neurosci. 2016:1–10, 2016. Available at: CrossRefGoogle Scholar
  31. 31.
    Akkus Z, Galimzianova A, Hoogi A, Rubin DL, Erickson BJ: Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions. J Digit Imaging 30:449–459, 2017CrossRefGoogle Scholar
  32. 32.
    Olshausen BA: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381:607–609, 1996CrossRefGoogle Scholar
  33. 33.
    Hinton GE, Salakhutdinov RR: Reducing the dimensionality of data with neural networks. Science 313(5786):504–507, 2006CrossRefGoogle Scholar
  34. 34.
    Hinton GE, Osindero S, Teh YW: A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554, 2006CrossRefGoogle Scholar
  35. 35.
    Kuang D, Guo X, An X, Zhao Y, He L: Discrimination of ADHD based on fMRI data with Deep Belief Network. In: International Conference on Intelligent Computing, Aug 3.Springer, Cham, 2014, pp 225–232Google Scholar
  36. 36.
    Di Martino A, Yan CG, Li Q, Denio E, Castellanos FX et al.: The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 19(6):659–667, 2014 Available at:
  37. 37.
    Autism Brain Imaging Data Exchange,, accessed at 1/10/2017Google Scholar
  38. 38.
  39. 39.
    Jenkinson M, Smith SM: Pre-Processing of BOLD FMRI Data. Oxford University Centre for Functional MRI of the Brain (FMRIB), 2006.Google Scholar
  40. 40.
    Bowman FD, Guo Y, Derado G: Statistical Approaches to Functional Neuroimaging Data. Neuroimaging Clin 17(4, 2007):441–viii. CrossRefGoogle Scholar
  41. 41.
    Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M: Automated Anatomical Labeling of activations in SPM using a Macroscopic Anatomical Parcellation of the MNI MRI single-subject brain. NeuroImage. 15(1):273–289, 2002CrossRefGoogle Scholar
  42. 42.
    Available at: (LISA lab, University of Montreal, 2015).
  43. 43.
    Erickson BJ, Korfiatis P, Akkus Z, Kline TL: Machine Learning for Medical Imaging. RadioGraphics. Feb 17 37(2):505–515, 2017CrossRefGoogle Scholar
  44. 44.
  45. 45.
    Katuwal GJ, Cahill ND, Baum SA, Michael AM: The predictive power of structural MRI in Autism diagnosis. In 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); 2015, p 4270–4273Google Scholar
  46. 46.
    Cody H, Pelphrey K, Piven J: Structural and functional magnetic resonance imaging of autism. Int J Dev Neurosci 20(3–5):421–438, 2002CrossRefGoogle Scholar
  47. 47.
    Bennett MR, Lagopoulos J: Neurodevelopmental sequelae associated with gray and white matter changes and their cellular basis: A comparison between Autism Spectrum Disorder, ADHD and dyslexia. Int J Dev Neurosci 46:132–143, 2015CrossRefGoogle Scholar
  48. 48.
    Minshew NJ, Keller TA: The nature of brain dysfunction in autism: functional brain imaging studies. Curr Opin Neurol 23(2):124–130, 2010CrossRefGoogle Scholar
  49. 49.
    LeCun Y, Bengio Y, Hinton G: Deep learning. Nature 521:436–444, 2015CrossRefGoogle Scholar
  50. 50.
    Pinaya WH, Gadelha A, Doyle OM, Noto C, Zugman A, Cordeiro Q, Jackowski AP, Bressan RA, Sato JR: Using deep belief network modelling to characterize differences in brain morphometry in schizophrenia. Sci Rep Dec 12 6:38897, 2016CrossRefGoogle Scholar

Copyright information

© Society for Imaging Informatics in Medicine 2018

Authors and Affiliations

  1. 1.Department of Computer Engineering, Science and Research BranchIslamic Azad UniversityTehranIran
  2. 2.Department of Electrical and Computer EngineeringKharazmi UniversityTehranIran

Personalised recommendations