Non-Euclidean Dissimilarities: Causes, Embedding and Informativeness

  • Robert P. W. Duin
  • Elżbieta Pękalska
  • Marco Loog

Abstract

In many pattern recognition applications, object structure is essential for the discrimination purpose. In such cases, researchers often use recognition schemes based on template matching which lead to the design of non-Euclidean dissimilarity measures. A vector space derived from the embedding of the dissimilarities is desirable in order to use general classifiers. An isometric embedding of the symmetric non-Euclidean dissimilarities results in a pseudo-Euclidean space. More and better tools are available for the Euclidean spaces but they are not fully consistent with the given dissimilarities.

In this chapter, first a review is given of the various embedding procedures for the pairwise dissimilarity data. Next the causes are analyzed for the existence of non-Euclidean dissimilarity measures. Various ways are discussed in which the measures are converted into Euclidean ones. The purpose is to investigate whether the original non-Euclidean measures are informative or not. A positive conclusion is derived as examples can be constructed and found in real data for which the non-Euclidean characteristics of the data are essential for building good classifiers. (This chapter is based on previous publications by the authors, (Duin and Pękalska in Proc. SSPR & SPR 2010 (LNCS), pp. 324–333, 2010 and in CIARP (LNCS), pp. 1–24, 2011; Duin in ICEIS, pp. 15–28, 2010 and in ICPR, pp. 1–4, 2008; Duin et al. in SSPR/SPR, pp. 551–561, 2008; Pękalska and Duin in IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 38(6):729–744, 2008) and contains text, figures, equations, and experimental results taken from these papers.)

References

  1. 1.
    Anderson, J.A.: Logistic discrimination. In: Krishnaiah, P.R., Kanal, L.N. (eds.) Handbook of Statistics 2: Classification, Pattern Recognition and Reduction of Dimensionality, pp. 169–191. North-Holland, Amsterdam (1982) CrossRefGoogle Scholar
  2. 2.
    Andreu, G., Crespo, A., Valiente, J.M.: Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recognition. In: Proceedings of ICNN’97, International Conference on Neural Networks, vol. II, pp. 1341–1346. IEEE Service Center, Piscataway (1997) CrossRefGoogle Scholar
  3. 3.
    Bahlmann, C., Burkhardt, H.: The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 299–310 (2004) CrossRefGoogle Scholar
  4. 4.
    Bunke, H., Bühler, U.: Applications of approximate string matching to 2D shape recognition. Pattern Recognit. 26(12), 1797–1812 (1993) CrossRefGoogle Scholar
  5. 5.
    Bunke, H., Riesen, K.: Graph classification based on dissimilarity space embedding. In: Structural, Syntactic, and Statistical Pattern Recognition, pp. 996–1007 (2008) CrossRefGoogle Scholar
  6. 6.
    Bunke, H., Sanfeliu, A. (eds.): Syntactic and Structural Pattern Recognition Theory and Applications. World Scientific, Singapore (1990) MATHGoogle Scholar
  7. 7.
    Bunke, H., Shearer, K.: A graph distance metric based on the maximal common subgraph. Pattern Recognit. Lett. 19(3–4), 255–259 (1998) CrossRefMATHGoogle Scholar
  8. 8.
    Burghouts, G.J., Smeulders, A.W.M., Geusebroek, J.M.: The distribution family of similarity distances. In: Advances in Neural Information Processing Systems, vol. 20 (2007) Google Scholar
  9. 9.
    Plasencia Calana, Y., García Reyes, E.B., Orozco-Alzate, M., Duin, R.P.W.: Prototype selection for dissimilarity representation by a genetic algorithm. In: ICPR 2010, pp. 177–180 (2010) Google Scholar
  10. 10.
    Chang, C.-C., Lin, C.-J.: LIBSVM: A Library for Support Vector Machines (2001). Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm Google Scholar
  11. 11.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995) MATHGoogle Scholar
  12. 12.
    Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines. Cambridge University Press, Cambridge (2000) Google Scholar
  13. 13.
    Dietterich, T.G., Lathrop, R.H., Lozano-Perez, T.: Solving the multiple instance problem with axis-parallel rectangles. Artif. Intell. 89, 31–71 (1997) CrossRefMATHGoogle Scholar
  14. 14.
    Dubuisson, M.P., Jain, A.K.: Modified Hausdorff distance for object matching. In: Int. Conference on Pattern Recognition, vol. 1, pp. 566–568 (1994) Google Scholar
  15. 15.
    Duin, R.P.W., Pękalska, E.: Non-Euclidean dissimilarities: causes and informativeness. In: Hancock, E.R., et al. (eds.) Proc. SSPR & SPR 2010 (LNCS), vol. 6218, pp. 324–333. Springer, Berlin (2010) Google Scholar
  16. 16.
    Duin, R.P.W.: Non-Euclidean problems in pattern recognition related to human expert knowledge. In: Filipe, J., Cordeiro, J. (eds.) ICEIS. Lecture Notes in Business Information Processing, vol. 73, pp. 15–28. Springer, Berlin (2010) Google Scholar
  17. 17.
    Duin, R.P.W., Loog, M., Pękalska, E., Tax, D.M.J.: Feature-based dissimilarity space classification. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR Contests. Lecture Notes in Computer Science, vol. 6388, pp. 46–55. Springer, Berlin (2010) Google Scholar
  18. 18.
    Duin, R.P.W., Pękalska, E.: The dissimilarity representation for structural pattern recognition. In: CIARP (LNCS), vol. 7042, pp. 1–24. Springer, Berlin (2011) Google Scholar
  19. 19.
    Duin, R.P.W., de Ridder, D., Tax, D.M.J.: Experiments with object based discriminant functions; a featureless approach to pattern recognition. Pattern Recognit. Lett. 18(11–13), 1159–1166 (1997) CrossRefGoogle Scholar
  20. 20.
    Duin, R.P.W., Pękalska, E.: On refining dissimilarity matrices for an improved NN learning. In: ICPR, pp. 1–4 (2008) Google Scholar
  21. 21.
    Duin, R.P.W., Pękalska, E.: The dissimilarity space: between structural and statistical pattern recognition. Pattern Recognit. Lett. 33, 826–832 (2012) CrossRefGoogle Scholar
  22. 22.
    Duin, R.P.W., Pękalska, E., Harol, A., Lee, W.-J., Bunke, H.: On Euclidean corrections for non-Euclidean dissimilarities. In: SSPR/SPR, pp. 551–561 (2008) Google Scholar
  23. 23.
    Gold, S., Rangarajan, A.: A graduated assignment algorithm for graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 18(4), 377–388 (1996) CrossRefGoogle Scholar
  24. 24.
    Goldfarb, L.: A new approach to pattern recognition. In: Kanal, L.N., Rosenfeld, A. (eds.) Progress in Pattern Recognition, vol. 2, pp. 241–402. Elsevier, Amsterdam (1985) Google Scholar
  25. 25.
    Gower, J.C.: Metric and Euclidean Properties of Dissimilarity Coefficients. J. Classif. 3, 5–48 (1986) MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Graepel, T., Herbrich, R., Bollmann-Sdorra, P., Obermayer, K.: Classification on pairwise proximity data. In: Advances in Neural Information System Processing, vol. 11, pp. 438–444 (1999) Google Scholar
  27. 27.
    Graepel, T., Herbrich, R., Schölkopf, B., Smola, A., Bartlett, P., Müller, K.-R., Obermayer, K., Williamson, R.: Classification on proximity data with LP-machines. In: ICANN, pp. 304–309 (1999) Google Scholar
  28. 28.
    Haasdonk, B.: Feature space interpretation of SVMs with indefinite kernels. IEEE Trans. Pattern Anal. Mach. Intell. 25(5), 482–492 (2005) CrossRefGoogle Scholar
  29. 29.
    Haasdonk, B., Pękalska, E.: Indefinite kernel Fisher discriminant. In: ICPR, pp. 1–4 (2008) Google Scholar
  30. 30.
    Jacobs, D.W., Weinshall, D., Gdalyahu, Y.: Classification with non-metric distances: image retrieval and class representation. IEEE TPAMI 22(6), 583–600 (2000) CrossRefGoogle Scholar
  31. 31.
    Jain, A.K., Zongker, D.E.: Representation and recognition of handwritten digits using deformable templates. IEEE Trans. Pattern Anal. Mach. Intell. 19(12), 1386–1391 (1997) CrossRefGoogle Scholar
  32. 32.
    Joachims, T.: A probabilistic analysis of the Rocchio algorithm with TFIDF for text categorization. In: Proceedings of International Conference on Machine Learning, pp. 143–151 (1997) Google Scholar
  33. 33.
    Kondor, R.I., Jebara, T.: A kernel between sets of vectors. In: ICML, pp. 361–368 (2003) Google Scholar
  34. 34.
    Lee, W.J., Duin, R.P.W.: An inexact graph comparison approach in joint eigenspace. In: Structural, Syntactic, and Statistical Pattern Recognition, pp. 35–44 (2008) CrossRefGoogle Scholar
  35. 35.
    Lichtenauer, J.F., Hendriks, E.A., Reinders, M.J.T.: Sign language recognition by combining statistical DTW and independent classification. IEEE Trans. Pattern Anal. Mach. Intell. 30(11), 2040–2046 (2008) CrossRefGoogle Scholar
  36. 36.
    Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (COIL-100), Columbia University (1996) Google Scholar
  37. 37.
    Pękalska, E., Duin, R.P.W.: Dissimilarity representations allow for building good classifiers. Pattern Recognit. Lett. 23(8), 943–956 (2002) CrossRefMATHGoogle Scholar
  38. 38.
    Pękalska, E., Duin, R.P.W., Paclik, P.: Prototype selection for dissimilarity-based classifiers. Pattern Recognit. 39(2), 189–208 (2006) CrossRefMATHGoogle Scholar
  39. 39.
    Pękalska, E., Duin, R.P.W.: The Dissimilarity Representation for Pattern Recognition. Foundations and Applications. World Scientific, Singapore (2005) MATHGoogle Scholar
  40. 40.
    Pękalska, E., Duin, R.P.W.: Dissimilarity-based classification for vectorial representations. In: ICPR (3), pp. 137–140 (2006) Google Scholar
  41. 41.
    Pękalska, E., Duin, R.P.W.: Beyond traditional kernels: classification in two dissimilarity-based representation spaces. IEEE Trans. Syst. Man Cybern., Part C, Appl. Rev. 38(6), 729–744 (2008) CrossRefGoogle Scholar
  42. 42.
    Pękalska, E., Haasdonk, B.: Kernel discriminant analysis with positive definite and indefinite kernels. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1017–1032 (2009) CrossRefGoogle Scholar
  43. 43.
    Pękalska, E., Paclík, P., Duin, R.P.W.: A Generalized Kernel Approach to Dissimilarity Based Classification. J. Mach. Learn. Res. 2(2), 175–211 (2002) MathSciNetMATHGoogle Scholar
  44. 44.
    Porro-Muñoz, D., Duin, R.P.W., Talavera-Bustamante, I., Orozco-Alzate, M.: Classification of three-way data by the dissimilarity representation. Signal Process. 91(11), 2520–2529 (2011) CrossRefGoogle Scholar
  45. 45.
    Porro-Muñoz, D., Talavera, I., Duin, R.P.W., Hernández, N., Orozco-Alzate, M.: Dissimilarity representation on functional spectral data for classification. J. Chemom. 476–486 (2011) Google Scholar
  46. 46.
    Samsudin, N.A., Bradley, A.P.: Nearest neighbour group-based classification. Pattern Recognit. 43(10), 3458–3467 (2010) CrossRefMATHGoogle Scholar
  47. 47.
    Sebe, N., Lew, M.S., Huijsmans, D.P.: Toward improved ranking metrics. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1132–1143 (2000) CrossRefGoogle Scholar
  48. 48.
    Tax, D.M.J., Loog, M., Duin, R.P.W., Cheplygina, V., Lee, W.-J.: Bag dissimilarities for multiple instance learning. In: LNCS. Lecture Notes in Computer Science, vol. 7005, pp. 222–234. Springer, Berlin (2011) Google Scholar
  49. 49.
    Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998) MATHGoogle Scholar
  50. 50.
    Wilson, C.L., Garris, M.D.: Handprinted character database 3. Technical report, National Institute of Standards and Technology (1992) Google Scholar
  51. 51.
    Xiao, B., Hancock, E.R.: Geometric characterisation of graphs. In: CIAP, pp. 471–478 (2005) Google Scholar
  52. 52.
    Yang, J., Jiang, Y.-G., Hauptmann, A.G., Ngo, C.-W.: Evaluating bag-of-visual-words representations in scene classification. In: Ze Wang, J., Boujemaa, N., Del Bimbo, A., Li, J. (eds.) Multimedia Information Retrieval, pp. 197–206. ACM, New York (2007) Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Robert P. W. Duin
    • 1
  • Elżbieta Pękalska
    • 2
  • Marco Loog
    • 1
  1. 1.Pattern Recognition LaboratoryDelft University of TechnologyDelftThe Netherlands
  2. 2.ManchesterUK

Personalised recommendations