Induction in Neuroscience with Classification: Issues and Solutions
Machine learning and pattern recognition techniques are increasingly adopted in neuroimaging-based neuroscience research. In many applications a classifier is trained on brain data in order to predict a variable of interest. Two leading examples are brain decoding and clinical diagnosis. Brain decoding consists of predicting stimuli or mental states from concurrent functional brain data. In clinical diagnosis it is the presence or absence of a given medical condition that is predicted from brain data. Observing accurate classification is considered to support the hypothesis of variable-related information within brain data. In this work we briefly review the literature on statistical tests for this kind of hypothesis testing problem. We claim that the current approaches to this hypothesis testing problem are suboptimal, do not cover all useful settings, and that they could lead to wrong conclusions. We present a more accurate statistical test and provide examples of its superiority.
Unable to display preview. Download preview PDF.
- 2.Clémençon, S., Depecker, M., Vayatis, N.: AUC maximization and the two-sample problem. In: Proceedings of NIPS 2009, Advances in Neural Information Processing Systems 22, pp. 360–368. MIT Press (2009)Google Scholar
- 4.Dundar, M., Krishnapuram, B., Bi, J., Bharat Rao, R.: Learning classifiers when the training data is not IID. In: Proceedings of the 20th International Joint Conference on Artifical Intelligence, pp. 756–761. Morgan Kaufmann Publishers Inc., San Francisco (2007)Google Scholar
- 7.Gretton, A., Borgwardt, K.M., Rasch, M., Schölkopf, B., Smola, A.J.: A Kernel Method for the Two-Sample-Problem. In: NIPS (2006)Google Scholar
- 16.Olivetti, E., Mognon, A., Greiner, S., Avesani, P.: Brain Decoding: Biases in Error Estimation. In: 1st ICPR Workshop on Brain Decoding (August 2010)Google Scholar
- 17.Olivetti, E., Veeramachaneni, S., Avesani, P.: Testing for Information with Brain Decoding. In: IEEE International Workshop on Pattern Recognition in NeuroImaging (May 2011)Google Scholar
- 18.Olivetti, E., Veeramachaneni, S., Nowakowska, E.: Bayesian hypothesis testing for pattern discrimination in brain decoding. Pattern Recognition (May 2011)Google Scholar