Induction in Neuroscience with Classification: Issues and Solutions

  • Emanuele Olivetti
  • Susanne Greiner
  • Paolo Avesani
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7263)

Abstract

Machine learning and pattern recognition techniques are increasingly adopted in neuroimaging-based neuroscience research. In many applications a classifier is trained on brain data in order to predict a variable of interest. Two leading examples are brain decoding and clinical diagnosis. Brain decoding consists of predicting stimuli or mental states from concurrent functional brain data. In clinical diagnosis it is the presence or absence of a given medical condition that is predicted from brain data. Observing accurate classification is considered to support the hypothesis of variable-related information within brain data. In this work we briefly review the literature on statistical tests for this kind of hypothesis testing problem. We claim that the current approaches to this hypothesis testing problem are suboptimal, do not cover all useful settings, and that they could lead to wrong conclusions. We present a more accurate statistical test and provide examples of its superiority.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Casella, G., Moreno, E.: Assessing Robustness of Intrinsic Tests of Independence in Two-Way Contingency Tables. Journal of the American Statistical Association 104(487), 1261–1271 (2009)MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Clémençon, S., Depecker, M., Vayatis, N.: AUC maximization and the two-sample problem. In: Proceedings of NIPS 2009, Advances in Neural Information Processing Systems 22, pp. 360–368. MIT Press (2009)Google Scholar
  3. 3.
    Cameron Craddock, R., Holtzheimer, P.E., Hu, X.P., Mayberg, H.S.: Disease state prediction from resting state functional connectivity. Magnetic Resonance in Medicine 62(6), 1619–1628 (2009)CrossRefGoogle Scholar
  4. 4.
    Dundar, M., Krishnapuram, B., Bi, J., Bharat Rao, R.: Learning classifiers when the training data is not IID. In: Proceedings of the 20th International Joint Conference on Artifical Intelligence, pp. 756–761. Morgan Kaufmann Publishers Inc., San Francisco (2007)Google Scholar
  5. 5.
    Gigerenzer, G.: Mindless statistics. Journal of Socio-Economics 33(5), 587–606 (2004)CrossRefGoogle Scholar
  6. 6.
    Golland, P., Fischl, B.: Permutation Tests for Classification: Towards Statistical Significance in Image-Based Studies. In: Taylor, C.J., Noble, J.A. (eds.) IPMI 2003. LNCS, vol. 2732, pp. 330–341. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  7. 7.
    Gretton, A., Borgwardt, K.M., Rasch, M., Schölkopf, B., Smola, A.J.: A Kernel Method for the Two-Sample-Problem. In: NIPS (2006)Google Scholar
  8. 8.
    Haxby, J.V., Ida Gobbini, M., Furey, M.L., Ishai, A., Schouten, J.L., Pietrini, P.: Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex. Science 293(5539), 2425–2430 (2001)CrossRefGoogle Scholar
  9. 9.
    Haynes, J.-D., Rees, G.: Decoding mental states from brain activity in humans. Nature Reviews Neuroscience 7(7), 523–534 (2006)CrossRefGoogle Scholar
  10. 10.
    Hubbard, R., Bayarri, M.J.: Confusion Over Measures of Evidence (p’s) Versus Errors (alpha’s) in Classical Statistical Testing. The American Statistician 57(3), 171–178 (2003)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Kass, R.E., Raftery, A.E.: Bayes Factors. Journal of the American Statistical Association 90(430), 773–795 (1995)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Klöppel, S., Stonnington, C.M., Chu, C., Draganski, B., Scahill, R.I., Rohrer, J.D., Fox, N.C., Jack, C.R., Ashburner, J., Frackowiak, R.S.J.: Automatic classification of MR scans in Alzheimer’s disease. Brain 131(3), 681–689 (2008)CrossRefGoogle Scholar
  13. 13.
    Kriegeskorte, N., Kyle Simmons, W., Bellgowan, P.S.F., Baker, C.I.: Circular analysis in systems neuroscience: the dangers of double dipping. Nature Neuroscience 12(5), 535–540 (2009)CrossRefGoogle Scholar
  14. 14.
    Lao, Z., Shen, D., Xue, Z., Karacali, B., Resnick, S.M., Davatzikos, C.: Morphological classification of brains via high-dimensional shape transformations and machine learning methods. NeuroImage 21(1), 46–57 (2004)CrossRefGoogle Scholar
  15. 15.
    Nigam, V.P.P., Graupe, D.: A neural-network-based detection of epilepsy. Neurological Research 26(1), 55–60 (2004)CrossRefGoogle Scholar
  16. 16.
    Olivetti, E., Mognon, A., Greiner, S., Avesani, P.: Brain Decoding: Biases in Error Estimation. In: 1st ICPR Workshop on Brain Decoding (August 2010)Google Scholar
  17. 17.
    Olivetti, E., Veeramachaneni, S., Avesani, P.: Testing for Information with Brain Decoding. In: IEEE International Workshop on Pattern Recognition in NeuroImaging (May 2011)Google Scholar
  18. 18.
    Olivetti, E., Veeramachaneni, S., Nowakowska, E.: Bayesian hypothesis testing for pattern discrimination in brain decoding. Pattern Recognition (May 2011)Google Scholar
  19. 19.
    Pereira, F., Mitchell, T., Botvinick, M.: Machine learning classifiers and fMRI: A tutorial overview. NeuroImage 45(1), 199–209 (2009)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Emanuele Olivetti
    • 1
  • Susanne Greiner
    • 1
  • Paolo Avesani
    • 1
  1. 1.NeuroInformatics Laboratory (NILab)Bruno Kessler Foundation and University of Trento (CIMeC)Italy

Personalised recommendations