Skip to main content

Integrating Neural Networks and Dictionary Learning for Multidimensional Clinical Characterizations from Functional Connectomics Data

Part of the Lecture Notes in Computer Science book series (LNIP,volume 11766)


We propose a unified optimization framework that combines neural networks with dictionary learning to model complex interactions between resting state functional MRI and behavioral data. The dictionary learning objective decomposes patient correlation matrices into a collection of shared basis networks and subject-specific loadings. These subject-specific features are simultaneously input into a neural network that predicts multidimensional clinical information. Our novel optimization framework combines the gradient information from the neural network with that of a conventional matrix factorization objective. This procedure collectively estimates the basis networks, subject loadings, and neural network weights most informative of clinical severity. We evaluate our combined model on a multi-score prediction task using 52 patients diagnosed with Autism Spectrum Disorder (ASD). Our integrated framework outperforms state-of-the-art methods in a ten-fold cross validated setting to predict three different measures of clinical severity.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions


  1. Bishop, C.M.: Mixture density networks. Technical report. Citeseer (1994)

    Google Scholar 

  2. D’Souza, N.S., Nebel, M.B., Wymbs, N., Mostofsky, S., Venkataraman, A.: A generative-discriminative basis learning framework to predict clinical severity from resting state functional MRI data. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11072, pp. 163–171. Springer, Cham (2018).

    CrossRef  Google Scholar 

  3. Kawahara, J., et al.: BrainNetCNN: convolutional neural networks for brain networks; towards predicting neurodevelopment. NeuroImage 146, 1038–1049 (2017)

    CrossRef  Google Scholar 

  4. Nebel, M.B., et al.: Intrinsic visual-motor synchrony correlates with social deficits in autism. Bio. Psych. 79(8), 633–641 (2016)

    CrossRef  Google Scholar 

  5. Payakachat, N., et al.: Autism spectrum disorders: a review of measures for clinical, health services and cost-effectiveness applications. Expert Rev. Pharmacoeconomics Outcomes Res. 12(4), 485–503 (2012)

    CrossRef  Google Scholar 

  6. Rahim, M., et al.: Joint prediction of multiple scores captures better individual traits from brain images. NeuroImage 158, 145–154 (2017)

    CrossRef  Google Scholar 

  7. Sridharan, D., et al.: A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks. Proc. Nat. Acad. Sci. 105(34), 12569–12574 (2008)

    CrossRef  Google Scholar 

  8. Vieira, S., et al.: Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: methods and applications. Neurosci. Biobehav. Rev. 74, 58–75 (2017)

    CrossRef  Google Scholar 

  9. Wright, S., et al.: Numerical optimization. Springer Sci. 35(67–68), 7 (1999)

    Google Scholar 

Download references


This work was supported by the National Science Foundation CRCNS award 1822575, National Science Foundation CAREER award 1845430, National Institute of Mental Health (R01 MH085328-09, R01 MH078160-07, K01 MH109766 and R01 MH106564), National Institute of Neurological Disorders and Stroke (R01NS048527-08), and the Autism Speaks foundation.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Niharika Shimona D’Souza .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

D’Souza, N.S., Nebel, M.B., Wymbs, N., Mostofsky, S., Venkataraman, A. (2019). Integrating Neural Networks and Dictionary Learning for Multidimensional Clinical Characterizations from Functional Connectomics Data. In: Shen, D., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019. MICCAI 2019. Lecture Notes in Computer Science(), vol 11766. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-32247-2

  • Online ISBN: 978-3-030-32248-9

  • eBook Packages: Computer ScienceComputer Science (R0)