, Volume 17, Issue 2, pp 197–210 | Cite as

Multi-Objective Cognitive Model: a Supervised Approach for Multi-subject fMRI Analysis

  • Muhammad YousefnezhadEmail author
  • Daoqiang ZhangEmail author
Original Article


In order to decode human brain, Multivariate Pattern (MVP) classification generates cognitive models by using functional Magnetic Resonance Imaging (fMRI) datasets. As a standard pipeline in the MVP analysis, brain patterns in multi-subject fMRI dataset must be mapped to a shared space and then a classification model is generated by employing the mapped patterns. However, the MVP models may not provide stable performance on a new fMRI dataset because the standard pipeline uses disjoint steps for generating these models. Indeed, each step in the pipeline includes an objective function with independent optimization approach, where the best solution of each step may not be optimum for the next steps. For tackling the mentioned issue, this paper introduces Multi-Objective Cognitive Model (MOCM) that utilizes an integrated objective function for MVP analysis rather than just using those disjoint steps. For solving the integrated problem, we proposed a customized multi-objective optimization approach, where all possible solutions are firstly generated, and then our method ranks and selects the robust solutions as the final results. Empirical studies confirm that the proposed method can generate superior performance in comparison with other techniques.


Multi-objective cognitive model fMRI analysis Multivariate pattern Multi-objective optimization 



This work was supported in part by the National Natural Science Foundation of China (61422204 and 61473149), and NUAA Fundamental Research Funds (NE2013105).

Compliance with Ethical Standards

Conflict of interests

Muhammad Yousefnezhad and Daoqiang Zhang declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.


  1. Bennett, C.M., Baird, A., Miller, M.B., Wolfrod, G.L. (2009). Neural correlates of interspieces perspective taking in the post-mortem atlantic salmon: an argument for multiple comparisons correction. Human Brain Mapping, 1, 1995.Google Scholar
  2. Bradley, P.S., & Mangasarian, O.L. (1998). Feature selection via concave minimization and support vector machines. In 15th international conference on machine learning (ICML) (Vol. 98, pp. 82–90). Association for computing machinery (ACM). Madison, Wisconsin, USA.Google Scholar
  3. Cai, M.B., Schuck, N.W., Pillow, J.W., Niv, Y. (2016). A bayesian method for reducing bias in neural representational similarity analysis. In Advances in neural information processing systems (NIPS) (pp. 4951–4959).Google Scholar
  4. Carroll, M.K., Cecchi, G.A., Rish, I., Garg, R., Rao, A.R. (2009). Prediction and interpretation of distributed neural activity with sparse models. NeuroImage, 44(1), 112–122.CrossRefGoogle Scholar
  5. Chen, P.H., Chen, J., Yeshurun, Y., Hasson, U., Haxby, J., Ramadge, P.J. (2015). A reduced-dimension fmri shared response model. In 28th advances in neural information processing systems (NIPS-15) (pp. 460–468). Advances in neural information processing systems (NIPS). Montréal, Canada.Google Scholar
  6. Chen, P.H., Guntupalli, J.S., Haxby, J.V., Ramadge, P.J. (2014). Joint svd-hyperalignment for multi-subject fmri data alignment. In 24th international workshop on machine learning for signal processing (MLSP) (pp. 1–6). Reims, France: IEEE.Google Scholar
  7. Chen, P.H., Zhu, X., Zhang, H., Turek, J.S., Chen, J., Willke, T.L., Hasson, U., Ramadge, P.J. (2016). A convolutional autoencoder for multi-subject fmri data aggregation. In 29th workshop of representation learning in artificial and biological neural networks. Advances in neural information processing systems (NIPS). Barcelona, Spain.Google Scholar
  8. Conroy, B.R., Walz, J.M., Sajda, P. (2013). Fast bootstrapping and permutation testing for assessing reproducibility and interpretability of multivariate fmri decoding models. PLoS ONE, 8(11), 1–11.CrossRefGoogle Scholar
  9. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.Google Scholar
  10. Cox, D.D., & Savoy, R.L. (2003). Functional magnetic resonance imaging (fmri) “brain reading”: detecting and classifying distributed patterns of fmri activity in human visual cortex. NeuroImage, 19(2), 261–270.CrossRefGoogle Scholar
  11. Cox, R.W. (1996). Afni: software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical research, 29(3), 162–173.CrossRefGoogle Scholar
  12. Deb, K., Pratap, A., Agarwal, S., Meyarivan, T. (2002). A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Transactions on Evolutionary Computation, 6(2), 182–197.CrossRefGoogle Scholar
  13. Duncan, K.J., Pattamadilok, C., Knierim, I., Devlin, J.T. (2009). Consistency and variability in functional localisers. NeuroImage, 46(4), 1018–1026.CrossRefGoogle Scholar
  14. Eklund, A., Nichols, T.E., Knutsson, H. (2016). Cluster failure: why fMRI inferences for spatial extent have inflated false-positive rates. Proceedings of the National Academy of Sciences (PNAS), 113(28), 7900–7905.CrossRefGoogle Scholar
  15. Grosenick, L., Klingenberg, B., Katovich, K., Knutson, B., Taylor, J.E. (2013). Interpretable whole-brain prediction analysis with graphnet. NeuroImage, 72, 304–321.CrossRefGoogle Scholar
  16. Güçlü, U., & van Gerven, M.A. (2015). Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27), 10,005–10,014.CrossRefGoogle Scholar
  17. Guntupalli, J.S., Hanke, M., Halchenko, Y.O., Connolly, A.C., Ramadge, P.J., Haxby, J.V. (2016). A model of representational spaces in human cortex. Cerebral Cortex, 26(6), 2919–2934.CrossRefGoogle Scholar
  18. Hanke, M., Baumgartner, F.J., Ibe, P., Kaule, F.R., Pollmann, S., Speck, O., Zinke, W., Stadler, J. (2014). A high-resolution 7-tesla fmri dataset from complex natural stimulation with an audio movie. Scientific Data, 1, 1–8.CrossRefGoogle Scholar
  19. Haxby, J.V., Connolly, A.C., Guntupalli, J.S. (2014). Decoding neural representational spaces using multivariate pattern analysis. Annual Review of Neuroscience, 37, 435–456.CrossRefGoogle Scholar
  20. Haxby, J.V., Guntupalli, J.S., Connolly, A.C., Halchenko, Y.O., Conroy, B.R., Gobbini, M.I., Hanke, M., Ramadge, P.J. (2011). A common, high-dimensional model of the representational space in human ventral temporal cortex. Neuron, 72(2), 404–416.CrossRefGoogle Scholar
  21. Jenkinson, M., Beckmann, C.F., Behrens, T.E., Woolrich, M.W., Smith, S.M. (2012). Fsl. Neuroimage, 62(2), 782–790.CrossRefGoogle Scholar
  22. Kao, M.H. (2009). Multi-objective optimal experimental designs for er-fmri using matlab. Journal of Statistical Software, 30 (11), 1–13.CrossRefGoogle Scholar
  23. Kao, M.H., Mandal, A., Stufken, J. (2012). Constrained multiobjective designs for functional magnetic resonance imaging experiments via a modified non-dominated sorting genetic algorithm. Journal of the Royal Statistical Society. Series C: Applied Statistics, 61(4), 515–534.CrossRefGoogle Scholar
  24. Li, B., Tang, K., Li, J., Yao, X. (2016). Stochastic ranking algorithm for many-objective optimization based on multiple indicators. IEEE Transactions on Evolutionary Computation, 20(6), 924– 938.CrossRefGoogle Scholar
  25. Li, M., Yang, S., Liu, X. (2014). Shift-based density estimation for pareto-based algorithms in many-objective optimization. IEEE Transactions on Evolutionary Computation, 18(3), 348–365.CrossRefGoogle Scholar
  26. Lorbert, A., & Ramadge, P.J. (2012). Kernel hyperalignment. In 25th advances in neural information processing systems (NIPS-12). Advances in neural Information Processing Systems (NIPS) (pp. 1790–1798). Harveys, Lake Tahoe.Google Scholar
  27. Ma, X., Chou, C.A., Sayama, H., Chaovalitwongse, WA. (2016). Brain response pattern identification of fMRI data using a particle swarm optimization-based approach. Brain Informatics, 3(3), 181–192.CrossRefGoogle Scholar
  28. Mitchell, T.M., Shinkareva, S.V., Carlson, A., Chang, K.M., Malave, V.L., Mason, R.A., Just, M.A. (2008). Predicting human brain activity associated with the meanings of nouns. Science, 320(5880), 1191–1195.CrossRefGoogle Scholar
  29. Mohr, H., Wolfensteller, U., Frimmel, S., Ruge, H. (2015). Sparse regularization techniques provide novel insights into outcome integration processes. NeuroImage, 104, 163–176.CrossRefGoogle Scholar
  30. Norman, K.A., Polyn, S.M., Detre, G.J., Haxby, J.V. (2006). Beyond mind-reading: multi-voxel pattern analysis of fmri data. Trends in Cognitive Sciences, 10(9), 424–430.CrossRefGoogle Scholar
  31. Osher, D.E., Saxe, R.R., Koldewyn, K., Gabrieli, J.D., Kanwisher, N., Saygin, Z.M. (2015). Structural connectivity fingerprints predict cortical selectivity for multiple visual categories across cortex. Cerebral Cortex, 26(4), 1668–1683.CrossRefGoogle Scholar
  32. Pauli, R., Bowring, A., Reynolds, R., Chen, G., Nichols, T.E., Maumet, C. (2016). Exploring fmri results space: 31 variants of an fmri analysis in afni, fsl, and spm. Frontiers in Neuroinformatics, 10, 1–6.CrossRefGoogle Scholar
  33. Penny, W.D., Friston, K.J., Ashburner, J.T., Kiebel, S.J., Nichols, T.E. (2011). Statistical parametric mapping: the analysis of functional brain images. Academic Press. ISBN: 978-0-12-372560-8.Google Scholar
  34. Rademacher, J., Caviness, V.S., Steinmetz, H., Galaburda, A. (1993). Topographical variation of the human primary cortices: implications for neuroimaging, brain mapping, and neurobiology. Cerebral Cortex, 3(4), 313–329.CrossRefGoogle Scholar
  35. Tom, S.M., Fox, C.R., Trepel, C., Poldrack, R.A. (2007). The neural basis of loss aversion in decision-making under risk. Science, 315(5811), 515–518.CrossRefGoogle Scholar
  36. Wakeman, D.G., & Henson, R.N. (2015). A multi-subject, multi-modal human neuroimaging dataset. Scientific Data, 2, 1–10.CrossRefGoogle Scholar
  37. Walz, J.M., Goldman, R.I., Carapezza, M., Muraskin, J., Brown, T.R., Sajda, P. (2013). Simultaneous eeg-fmri reveals temporal evolution of coupling between supramodal cortical attention networks and the brainstem. Journal of Neuroscience, 33(49), 19,212–19,222.CrossRefGoogle Scholar
  38. Watson, J.D., Myers, R., Frackowiak, R.S.J., Hajnal, J.V., Woods, R.P., Mazziotta, J.C., Shipp, S., Zeki, S. (1993). Area v5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. Cerebral Cortex, 3(2), 79–94.CrossRefGoogle Scholar
  39. Xu, H., Lorbert, A., Ramadge, P.J., Guntupalli, J.S., Haxby, J.V. (2012). Regularized hyperalignment of multi-set fmri data. In Statistical signal processing workshop (SSP) (pp. 229–232). Ann Arbor, USA: IEEE.Google Scholar
  40. Yousefnezhad, M., & Zhang, D. (2016). Decoding visual stimuli in human brain by using anatomical pattern analysis on fmri images. In 8th international conference on brain inspired cognitive systems (BICS’16) (pp. 47–57). Beijing: Springer.Google Scholar
  41. Yousefnezhad, M., & Zhang, D. (2017). Local discriminant hyperalignment for multi-subject fmri data alignment. In 34th AAAI conference on artificial intelligence (AAAI-17). Association for the advancement of artificial intelligence (AAAI). San Francisco, California, USA.Google Scholar
  42. Yousefnezhad, M., & Zhang, D. (2017). Multi-region neural representation: a novel model for decoding visual stimuli in human brains. In 17th SIAM international conference on data mininig (SDM-17). Society for industrial and applied mathematics (SIAM). Houston, Texas, USA.Google Scholar
  43. Zitzler, E., & Künzli, S. (2004). Indicator-based selection in multiobjective search. In International conference on parallel problem solving from nature (pp. 832–842). Birmingham: Springer.Google Scholar
  44. Zou, H., & Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 67(2), 301–320.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.College of Computer Science and TechnologyNanjing University of Aeronautics and AstronauticsNanjingChina

Personalised recommendations