The Dissimilarity Representation for Structural Pattern Recognition

  • Robert P. W. Duin
  • Elżbieta Pȩkalska
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7042)

Abstract

The patterns in collections of real world objects are often not based on a limited set of isolated properties such as features. Instead, the totality of their appearance constitutes the basis of the human recognition of patterns. Structural pattern recognition aims to find explicit procedures that mimic the learning and classification made by human experts in well-defined and restricted areas of application. This is often done by defining dissimilarity measures between objects and measuring them between training examples and new objects to be recognized.

The dissimilarity representation offers the possibility to apply the tools developed in machine learning and statistical pattern recognition to learn from structural object representations such as graphs and strings. These procedures are also applicable to the recognition of histograms, spectra, images and time sequences taking into account the connectivity of samples (bins, wavelengths, pixels or time samples).

The topic of dissimilarity representation is related to the field of non-Mercer kernels in machine learning but it covers a wider set of classifiers and applications. Recently much progress has been made in this area and many interesting applications have been studied in medical diagnosis, seismic and hyperspectral imaging, chemometrics and computer vision. This review paper offers an introduction to this field and presents a number of real world applications.

Keywords

Covariance Schizophrenia Editing Cali 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Anderson, J.A.: Logistic discrimination. In: Krishnaiah, P.R., Kanal, L.N. (eds.) Handbook of Statistics 2: Classification, Pattern Recognition and Reduction of Dimensionality, pp. 169–191. North Holland, Amsterdam (1982)CrossRefGoogle Scholar
  2. 2.
    Andreu, G., Crespo, A., Valiente, J.M.: Selecting the toroidal self-organizing feature maps (TSOFM) best organized to object recognition. In: Proceedings of ICNN 1997, International Conference on Neural Networks, vol. II, pp. 1341–1346. IEEE Service Center, Piscataway (1997)Google Scholar
  3. 3.
    Arkedev, A.G., Braverman, E.M.: Computers and Pattern Recognition. Thompson, Washington, D.C (1966)Google Scholar
  4. 4.
    Bahlmann, C., Burkhardt, H.: The writer independent online handwriting recognition system frog on hand and cluster generative statistical dynamic time warping. IEEE Trans. Pattern Anal. Mach. Intell. 26(3), 299–310 (2004)CrossRefGoogle Scholar
  5. 5.
    Bicego, M., Cristani, M., Murino, V., Pękalska, E., Duin, R.P.W.: Clustering-based construction of hidden markov models for generative kernels. In: Cremers, D., Boykov, Y., Blake, A., Schmidt, F.R. (eds.) EMMCVPR 2009. LNCS, vol. 5681, pp. 466–479. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  6. 6.
    Bicego, M., Pękalska, E., Tax, D.M.J., Duin, R.P.W.: Component-based discriminative classification for hidden markov models. Pattern Recognition 42(11), 2637–2648 (2009)CrossRefMATHGoogle Scholar
  7. 7.
    Bishop, C.: Pattern Recognition and Machine Learning. Springer, Heidelberg (2006)MATHGoogle Scholar
  8. 8.
    Bognár, J.: Indefinite Inner Product Spaces. Springer, Heidelberg (1974)CrossRefMATHGoogle Scholar
  9. 9.
    Borg, I., Groenen, P.: Modern Multidimensional Scaling. Springer, New York (1997)CrossRefMATHGoogle Scholar
  10. 10.
    Bunke, H., Buhler, U.: Applications of approximate string matching to 2D shape recognition. Pattern Recognition 26(12), 1797–1812 (1993)CrossRefGoogle Scholar
  11. 11.
    Bunke, H., Riesen, K.: Graph classification on dissimilarity space embedding. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds.) S+SSPR 2008. LNCS, vol. 5342, p. 2. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    Burghouts, G.J., Smeulders, A.W.M., Geusebroek, J.M.: The distribution family of similarity distances. In: Advances in Neural Information Processing Systems, vol. 20 (2007)Google Scholar
  13. 13.
    Calana, Y.P., Reyes, E.B.G., Orozco-Alzate, M., Duin, R.P.W.: Prototype selection for dissimilarity representation by a genetic algorithm. In: ICPR 2010, pp. 177–180 (2010)Google Scholar
  14. 14.
    Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), software http://www.csie.ntu.edu.tw/~cjlin/libsvm
  15. 15.
    Cortes, C., Vapnik, V.: Support-vector networks. Machine Learning 20, 273–297 (1995)MATHGoogle Scholar
  16. 16.
    Devijver, P.A., Kittler, J.V.: Pattern Recognition: A Statistical Approach. Prentice-Hall, Englewood Cliffs (1982)MATHGoogle Scholar
  17. 17.
    Duda, R., Hart, P., Stork, D.: Pattern Classification, 2nd edn. John Wiley & Sons, Inc., Chichester (2001)MATHGoogle Scholar
  18. 18.
    Duin, R.P.W.: Compactness and complexity of pattern recognition problems. In: Perneel, C. (ed.) Proc. Int. Symposium on Pattern Recognition ’In Memoriam Pierre Devijver’, pp. 124–128. Royal Military Academy, Brussels (1999)Google Scholar
  19. 19.
    Duin, R.P.W., Pękalska, E.: Non-euclidean dissimilarities: Causes and informativeness. In: Hancock, E.R., Wilson, R.C., Windeatt, T., Ulusoy, I., Escolano, F. (eds.) SSPR&SPR 2010. LNCS, vol. 6218, pp. 324–333. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  20. 20.
    Duin, R.P.W., Pękalska, E.: The dissimilarity space: Bridging structural and statistical pattern recognition. Pattern Recognition Letters (in press, 2011)Google Scholar
  21. 21.
    Duin, R.P.W., Pękalska, E., Harol, A., Lee, W.-J., Bunke, H.: On euclidean corrections for non-euclidean dissimilarities. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds.) S+SSPR 2008. LNCS, vol. 5342, pp. 551–561. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  22. 22.
    Duin, R.P.W.: Non-euclidean problems in pattern recognition related to human expert knowledge. In: Filipe, J., Cordeiro, J. (eds.) ICEIS 2010. LNBIP, vol. 73, pp. 15–28. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  23. 23.
    Duin, R.P.W., Loog, M., Pękalska, E., Tax, D.M.J.: Feature-based dissimilarity space classification. In: Ünay, D., Çataltepe, Z., Aksoy, S. (eds.) ICPR 2010. LNCS, vol. 6388, pp. 46–55. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  24. 24.
    Duin, R., Pękalska, E.: On refining dissimilarity matrices for an improved nn learning. In: ICPR, pp. 1–4 (2008)Google Scholar
  25. 25.
    Edelman, S.: Representation and Recognition in Vision. MIT Press, Cambridge (1999)Google Scholar
  26. 26.
    Fu, K.: Syntactic Pattern Recognition and Applications. Prentice-Hall (1982)Google Scholar
  27. 27.
    Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press (1990)Google Scholar
  28. 28.
    Gold, S., Rangarajan, A.: A graduated assignment algorithm for graph matching. IEEE Trans. Pattern Anal. Mach. Intell. 18(4), 377–388 (1996)CrossRefGoogle Scholar
  29. 29.
    Goldfarb, L.: A new approach to pattern recognition. In: Kanal, L., Rosenfeld, A. (eds.) Progress in Pattern Recognition, vol. 2, pp. 241–402. Elsevier (1985)Google Scholar
  30. 30.
    Haasdonk, B.: Feature space interpretation of SVMs with indefinite kernels. IEEE TPAMI 25(5), 482–492 (2005)CrossRefGoogle Scholar
  31. 31.
    Haasdonk, B., Burkhardt, H.: Invariant kernel functions for pattern analysis and machine learning. Machine Learning 68(1), 35–61 (2007)CrossRefGoogle Scholar
  32. 32.
    Haasdonk, B., Pękalska, E.: Indefinite kernel fisher discriminant. In: ICPR, pp. 1–4 (2008)Google Scholar
  33. 33.
    Ibba, A., Duin, R.P.W., Lee, W.J.: A study on combining sets of differently measured dissimilarities. In: ICPR, pp. 3360–3363. IEEE (2010)Google Scholar
  34. 34.
    Kim, S.W., Duin, R.P.W.: On improving dissimilarity-based classifications using a statistical similarity measure. In: Bloch, I., Cesar Jr., R.M. (eds.) CIARP 2010. LNCS, vol. 6419, pp. 418–425. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  35. 35.
    Kruskal, J.: Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika 29, 1–27 (1964)MathSciNetCrossRefMATHGoogle Scholar
  36. 36.
    Laub, J., Roth, V., Buhmann, J.M., Müller, K.R.: On the information and representation of non-euclidean pairwise data. Pattern Recognition 39(10), 1815–1826 (2006)CrossRefMATHGoogle Scholar
  37. 37.
    Lee, W.J., Duin, R.P.W.: An inexact graph comparison approach in joint eigenspace. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds.) S+SSPR 2008. LNCS, vol. 5342, pp. 35–44. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  38. 38.
    Lichtenauer, J.F., Hendriks, E.A., Reinders, M.J.T.: Sign language recognition by combining statistical DTW and independent classification. IEEE Trans. Pattern Analysis and Machine Intelligence 30(11), 2040–2046 (2008)CrossRefGoogle Scholar
  39. 39.
    Lozano, M., Sotoca, J.M., Sánchez, J.S., Pla, F., Pękalska, E., Duin, R.P.W.: Experimental study on prototype optimisation algorithms for prototype-based classification in vector spaces. Pattern Recognition 39(10), 1827–1838 (2006)CrossRefMATHGoogle Scholar
  40. 40.
    Luo, B., Wilson, R.C., Hancock, E.R.: Spectral embedding of graphs. Pattern Recognition 36(10), 2213–2230 (2003)CrossRefMATHGoogle Scholar
  41. 41.
    Mottl, V., Seredin, O., Dvoenko, S., Kulikowski, C.A., Muchnik, I.B.: Featureless pattern recognition in an imaginary hilbert space. In: ICPR, vol. 2, pp. 88–91 (2002)Google Scholar
  42. 42.
    Mottl, V., Dvoenko, S., Seredin, O., Kulikowski, C., Muchnik, I.: Featureless pattern recognition in an imaginary Hilbert space and its application to protein fold classification. In: Perner, P. (ed.) MLDM 2001. LNCS (LNAI), vol. 2123, pp. 322–336. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  43. 43.
    Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (COIL-100), Columbia University (1996)Google Scholar
  44. 44.
    Neuhaus, M., Bunke, H.: Bridging the Gap Between Graph Edit Distance and Kernel Machines. World Scientific (2007)Google Scholar
  45. 45.
    Orozco-Alzate, M., Duin, R.P.W., Castellanos-Domínguez, G.: A generalization of dissimilarity representations using feature lines and feature planes. Pattern Recognition Letters 30(3), 242–254 (2009)CrossRefGoogle Scholar
  46. 46.
    Pękalska, E., Duin, R.P.W.: Dissimilarity representations allow for building good classifiers. Pattern Recognition Letters 23(8), 943–956 (2002)CrossRefMATHGoogle Scholar
  47. 47.
    Pękalska, E., Duin, R.P.W.: Beyond traditional kernels: Classification in two dissimilarity-based representation spaces. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 38(6), 729–744 (2008)CrossRefGoogle Scholar
  48. 48.
    Pękalska, E., Duin, R.P.W., Paclík, P.: Prototype selection for dissimilarity-based classifiers. Pattern Recognition 39(2), 189–208 (2006)CrossRefMATHGoogle Scholar
  49. 49.
    Pękalska, E., Duin, R.: The Dissimilarity Representation for Pattern Recognition. World Scientific, Singapore (2005)CrossRefMATHGoogle Scholar
  50. 50.
    Pekalska, E., Haasdonk, B.: Kernel discriminant analysis for positive definite and indefinite kernels. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1017–1032 (2009)CrossRefGoogle Scholar
  51. 51.
    Porro-Muñoz, D., Duin, R.P.W., Talavera-Bustamante, I., Orozco-Alzate, M.: Classification of three-way data by the dissimilarity representation. Signal Processing 91(11), 2520–2529 (2011)CrossRefGoogle Scholar
  52. 52.
    Porro-Muñoz, D., Talavera, I., Duin, R.P.W., Hernández, N., Orozco-Alzate, M.: Dissimilarity representation on functional spectral data for classification. Journal of Chemometrics, n/a–n/a (2011)Google Scholar
  53. 53.
    Ripley, B.D.: An introduction to statistical pattern recognition. Cambridge University Press, Cambridge (1996)MATHGoogle Scholar
  54. 54.
    Shepard, R.: The analysis of proximities: Multidimensional scaling with an unknown distance function. i. Psychometrika 27, 125–140 (1962)MathSciNetCrossRefMATHGoogle Scholar
  55. 55.
    Spillmann, B., Neuhaus, M., Bunke, H., Pękalska, E.z., Duin, R.P.W.: Transforming strings to vector spaces using prototype selection. In: Yeung, D.-Y., Kwok, J.T., Fred, A., Roli, F., de Ridder, D. (eds.) SSPR 2006 and SPR 2006. LNCS, vol. 4109, pp. 287–296. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  56. 56.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Academic Press (2008)Google Scholar
  57. 57.
    Ulas, A., Duin, R.P., Castellani, U., Loog, M., Mirtuono, P., Bicego, M., Murino, V., Bellani, M., Cerruti, S., Tansella, M., Brambilla, P.: Dissimilarity-based detection of schizophrenia. International Journal of Imaging Systems and Technology 21(2), 179–192 (2011)CrossRefGoogle Scholar
  58. 58.
    Vapnik, V.: Statistical Learning Theory. John Wiley & Sons, Inc. (1998)Google Scholar
  59. 59.
    Watanabe, S.: Pattern Recognition: Human and Mechanical. Wiley (1985)Google Scholar
  60. 60.
    Webb, A.: Statistical pattern recognition. Wiley (2002)Google Scholar
  61. 61.
    Weinberger, K.Q., Saul, L.K.: Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research 10, 207–244 (2009)MATHGoogle Scholar
  62. 62.
    Wilson, C., Garris, M.: Handprinted character database 3. Tech. rep., National Institute of Standards and Technology (February 1992)Google Scholar
  63. 63.
    Wilson, R., Luo, B., Hancock, E.: Pattern vectors from algebraic graph theory. IEEE Trans. on PAMI 27, 1112–1124 (2005)CrossRefGoogle Scholar
  64. 64.
    Wilson, R.C., Hancock, E.R.: Spherical embedding and classification. In: Hancock, E.R., Wilson, R.C., Windeatt, T., Ulusoy, I., Escolano, F. (eds.) SSPR&SPR 2010. LNCS, vol. 6218, pp. 589–599. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  65. 65.
    Wolpert, D.H. (ed.): The Mathematics of Generalization. Addison-Wesley, Reading (1995)MATHGoogle Scholar
  66. 66.
    Woznica, A., Kalousis, A., Hilario, M.: Learning to combine distances for complex representations. In: Ghahramani, Z. (ed.) ICML. ACM International Conference Proceeding Series, pp. 1031–1038. ACM (2007)Google Scholar
  67. 67.
    Xiao, B., Hancock, E.R.: Geometric characterisation of graphs. In: Roli, F., Vitulano, S. (eds.) ICIAP 2005. LNCS, vol. 3617, pp. 471–478. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  68. 68.
    Yang, L., Jin, R., Sukthankar, R., Liu, Y.: An efficient algorithm for local distance metric learning. In: AAAI. AAAI Press (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Robert P. W. Duin
    • 1
  • Elżbieta Pȩkalska
    • 2
  1. 1.Pattern Recognition LaboratoryDelft University of TechnologyThe Netherlands
  2. 2.School of Computer ScienceUniversity of ManchesterUnited Kingdom

Personalised recommendations