Abstract
Variance inflation is caused by a mismatch between linear projections of test and training data when projections are estimated on training sets smaller than the dimensionality of the feature space. We demonstrate that variance inflation can lead to an increased neuroimage decoding error rate for Support Vector Machines. However, good generalization may be recovered in part by a simple renormalization procedure. We show that with proper renormalization, cross-validation based parameter optimization leads to the acceptance of more non-linearity in neuroimage classifiers than would have been obtained without renormalization.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Abrahamsen, T.J., Hansen, L.K.: A cure for variance inflation in high dimensional kernel principal component analysis. Journal of Machine Learning Research, 2027–2044 (July 2011)
Akbani, R., Kwek, S., Japkowicz, N.: Applying Support Vector Machines to Imbalanced Datasets. In: Boulicaut, J.-F., Esposito, F., Giannotti, F., Pedreschi, D. (eds.) ECML 2004. LNCS (LNAI), vol. 3201, pp. 39–50. Springer, Heidelberg (2004)
Castro, C.L., Carvalho, M.A., Braga, A.P.: An Improved Algorithm for SVMs Classification of Imbalanced Data Sets. In: Palmer-Brown, D., Draganova, C., Pimenidis, E., Mouratidis, H. (eds.) EANN 2009. CCIS, vol. 43, pp. 108–118. Springer, Heidelberg (2009), http://dx.doi.org/10.1007/978-3-642-03969-0_11
Chang, C.-C., Lin, C.-J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:1–27:27 (2011), software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research 16, 321–357 (2002)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer Series in Statistics. Springer (2009), corr. 3rd printing 5th printing, edn.
Hull, J.J.: A database for handwritten text recognition research. IEEE Transactions on Pattern Analysis and Machine Intelligence 16(5), 550–554 (1994)
Kubat, M., Matwin, S.: Addressing the curse of imbalanced training sets: one-sided selection. In: Proc. 14th International Conference on Machine Learning, pp. 179–186. Morgan Kaufmann (1997)
Kwok, J.T.-Y.: Moderating the outputs of support vector machine classifiers. IEEE Transactions on Neural Networks 10, 1018–1031 (1999)
Lautrup, B., Hansen, L.K., Law, I., Svarer, C., Strother, S.C.: Massive weight sharing: A cure for extremely ill-posed problems. In: Supercomputing in Brain Research: From Tomography to Neural Networks. World Scientific Pub. Corp., pp. 137–148. World Scientific (1994)
Maratea, A., Petrosino, A.: Asymmetric Kernel Scaling for Imbalanced Data Classification. In: Fanelli, A.M., Pedrycz, W., Petrosino, A. (eds.) WILF 2011. LNCS, vol. 6857, pp. 196–203. Springer, Heidelberg (2011)
Osuna, E.E., Freund, R., Girosi, F.: Support vector machines: Training and applications. Tech. rep., MIT, Cambridge (1997)
Platt, J.C.: Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. In: Schlkopf, B., Burges, C.J.C., Smola, A.J. (eds.) Advances in Kernel Methods Support Vector Learning, pp. 61–74 (2000)
Raskutti, B., Kowalczyk, A.: Extreme re-balancing for SVMs: a case study. SIGKDD Explor. Newsl. 6(1), 60–69 (2004)
Rasmussen, P.M., Madsen, K.H., Lund, T.E., Hansen, L.K.: Visualization of nonlinear kernel models in neuroimaging by sensitivity maps. NeuroImage 55(3), 1120–1121 (2011)
Tang, Y., Zhang, Y.Q., Chawla, N.V., Krasser, S.: SVMs Modeling for Highly Imbalanced Classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) 39(1), 281–288 (2009)
Weiss, G.M.: Mining with rarity: a unifying framework. SIGKDD Explor. Newsl. 6, 7–19 (2004)
Wu, G., Chang, E.Y.: KBA: Kernel Boundary Alignment Considering Imbalanced Data Distribution. IEEE Transactions on Knowledge and Data Engineering 17(6), 786–795 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Abrahamsen, T.J., Hansen, L.K. (2012). Restoring the Generalizability of SVM Based Decoding in High Dimensional Neuroimage Data. In: Langs, G., Rish, I., Grosse-Wentrup, M., Murphy, B. (eds) Machine Learning and Interpretation in Neuroimaging. Lecture Notes in Computer Science(), vol 7263. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-34713-9_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-34713-9_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-34712-2
Online ISBN: 978-3-642-34713-9
eBook Packages: Computer ScienceComputer Science (R0)