Large Scale Indefinite Kernel Fisher Discriminant

  • Frank-Michael SchleifEmail author
  • Andrej Gisbrecht
  • Peter Tino
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9370)


Indefinite similarity measures can be frequently found in bio-informatics by means of alignment scores. Lacking an underlying vector space, the data are given as pairwise similarities only. Indefinite Kernel Fisher Discriminant (iKFD) is a very effective classifier for this type of data but has cubic complexity and does not scale to larger problems. Here we propose an extension of iKFD such that linear runtime and memory complexity is achieved for low rank indefinite kernels. Evaluation at several larger similarity data from various domains shows that the proposed method provides similar generalization capabilities while being substantially faster for large scale data.


Indefinite Kernels Kernel Fisher Discriminant Analysis Approximate Kernel Matrix Sparse Parameter Vector Underlying Data Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



A Marie Curie Intra-European Fellowship (IEF): FP7-PEOPLE-2012-IEF (FP7-327791-ProMoS) and support from the Cluster of Excellence 277 Cognitive Interaction Technology funded by the German Excellence Initiative is gratefully acknowledged. PT was supported by the EPSRC grant EP/L000296/1, “Personalized Health Care through Learning in the Model Space”. We would like to thank R. Duin, Delft University for various support with distools and prtools and Huanhuan Chen,University of Science and Technology of China, for providing support with the Probabilistic Classification Vector Machine.


  1. 1.
    Alabdulmohsin, I.M., Gao, X., Zhang, X.: Support vector machines with indefinite kernels. In: Phung, D., Li, H. (eds.) Proceedings of the Sixth Asian Conference on Machine Learning, ACML 2014. JMLR Proceedings, Nha Trang City, Vietnam, 26–28 November 2014, vol. 39 (2014).
  2. 2.
    Boeckmann, B., Bairoch, A., Apweiler, R., Blatter, M.-C., Estreicher, A., Gasteiger, E., Martin, M., Michoud, K., O’Donovan, C., Phan, I., Pilbout, S., Schneider, M.: The SWISS-PROT protein knowledgebase and its supplement TrEMBL in 2003. Nucleic Acids Res. 31, 365–370 (2003)CrossRefGoogle Scholar
  3. 3.
    Chen, H., Tino, P., Yao, X.: Probabilistic classification vector machines. IEEE Trans. Neural Netw. 20(6), 901–914 (2009)CrossRefGoogle Scholar
  4. 4.
    Chen, H., Tino, P., Yao, X.: Efficient probabilistic classification vector machine with incremental basis function selection. IEEE TNN-LS 25(2), 356–369 (2014)Google Scholar
  5. 5.
    Chen, Y., Garcia, E.K., Gupta, M.R., Rahimi, A., Cazzanti, L.: Similarity-based classification: concepts and algorithms. JMLR 10, 747–776 (2009)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Diethe, T., Hussain, Z., Hardoon, D.R., Shawe-Taylor, J.: Matching pursuit kernel fisher discriminant analysis. In: Dyk, D.A.V., Welling, M. (eds.) Proceedings of the Twelfth International Conference on Artificial Intelligence and Statistics, AISTATS 2009. JMLR Proceedings, 16–18 April 2009, Clearwater Beach, Florida, USA, vol. 5, pp. 121–128 (2009).
  7. 7.
    Dubuisson, M., Jain, A.: A modified hausdorff distance for object matching. In: Proceedings of the 12th IAPR International Conference on Pattern Recognition, 1994. Vol. 1 - Conference A: Computer Vision amp; Image Processing., vol. 1, pp. 566–568, October 1994Google Scholar
  8. 8.
    Duin, R.P.: prtools, March 2012Google Scholar
  9. 9.
    Gisbrecht, A., Schleif, F.-M.: Metric and non-metric proximity transformations at linear costs. Neurocomputing (2015, to appear)Google Scholar
  10. 10.
    Haasdonk, B.: Feature space interpretation of svms with indefinite kernels. IEEE Trans. Pattern Anal. Mach. Intell. 27(4), 482–492 (2005)CrossRefGoogle Scholar
  11. 11.
    Haasdonk, B., Pekalska, E.: Indefinite kernel fisher discriminant. In: 19th International Conference on Pattern Recognition (ICPR 2008), 8–11 December 2008, Tampa, Florida, USA, pp. 1–4. IEEE Computer Society (2008)Google Scholar
  12. 12.
    Ling, H., Jacobs, D.W.: Shape classification using the inner-distance. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 286–299 (2007)CrossRefGoogle Scholar
  13. 13.
    Mokbel, B., Hasenfuss, A., Hammer, B.: Graph-based representation of symbolic musical data. In: Torsello, A., Escolano, F., Brun, L. (eds.) GbRPR 2009. LNCS, vol. 5534, pp. 42–51. Springer, Heidelberg (2009) CrossRefGoogle Scholar
  14. 14.
    Neuhaus, M., Bunke, H.: Edit distance based kernel functions for structural pattern classification. Pattern Recogn. 39(10), 1852–1863 (2006)CrossRefzbMATHGoogle Scholar
  15. 15.
    Pekalska, E., Haasdonk, B.: Kernel discriminant analysis for positive definite and indefinite kernels. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1017–1031 (2009)CrossRefGoogle Scholar
  16. 16.
    Schleif, F.-M., Gisbrecht, A., Tino, P.: Probabilistic classification vector machine at large scale. In: Proceedings of ESANN 2015, pp. 555–560 (2015)Google Scholar
  17. 17.
    Schleif, F.-M., Gisbrecht, A.: Data analysis of (non-)metric proximities at linear costs. In: Hancock, E., Pelillo, M. (eds.) SIMBAD 2013. LNCS, vol. 7953, pp. 59–74. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  18. 18.
    Schleif, F.-M., Tino, P.: Indefinite proximity learning - a review. Neural Computation (2015, to appear)Google Scholar
  19. 19.
    Smith, T.F., Waterman, M.S.: Identification of common molecular subsequences. J. Mol. Biol. 147(1), 195–197 (1981)CrossRefGoogle Scholar
  20. 20.
    Williams, C.K.I., Seeger, M.: Using the nyström method to speed up kernel machines. In: NIPS 2000, pp. 682–688 (2000)Google Scholar
  21. 21.
    Yang, J., Fan, L.: A novel indefinite kernel dimensionality reduction algorithm: weighted generalized indefinite kernel discriminant analysis. Neural Process. Lett. 40(3), 301–313 (2014). doi: 10.1007/s11063-013-9330-9 CrossRefGoogle Scholar
  22. 22.
    Zhang, K., Kwok, J.T.: Clustered nyström method for large scale manifold learning and dimension reduction. IEEE Trans. Neural Netw. 21(10), 1576–1587 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Frank-Michael Schleif
    • 1
    Email author
  • Andrej Gisbrecht
    • 2
  • Peter Tino
    • 1
  1. 1.School of Computer ScienceUniversity of BirminghamBirminghamUK
  2. 2.CITEC Centre of ExcellenceBielefeld UniversityBielefeldGermany

Personalised recommendations