Advertisement

Pattern Analysis and Applications

, Volume 20, Issue 4, pp 1227–1243 | Cite as

Overlapping area hyperspheres for kernel-based similarity method

  • Alya Slimene
  • Ezzeddine Zagrouba
Short Paper

Abstract

Measuring similarity between sets of objects is a key step in a wide areas of machine learning. Popular examples include general classification framework and numerous applications in computer vision. In this paper, we propose a kernel-based similarity method which is inspired from an interesting biological behavior of trees and induced mathematically by formulating it as a quadratic optimization problem in a reproducing kernel Hilbert space (RKHS). The proposed method is compared to the maximum mean discrepancy, a recent and challenging kernel similarity method. We conduct and present several numerical experiments on synthetic data as well as real-word image data. The proposed method yields favorable performances in terms of classification performances in the context of supervised classification tasks on the challenging Caltech101 dataset and other datasets such as USPS and ETH80. Furthermore, the efficiency of the proposed method in the context of image segmentation through unsupervised clustering of superpixels has been also asserted.

Keywords

Similarity methods Kernel methods Image classification Overlapping hyperspheres 

References

  1. 1.
    Ackermann N (2005) A cauchy-schwarz type inequality for bilinear integrals on positive measures. Proc Am Math Soc 133(9):2647–2656CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Atallah MJ (1983) A linear time algorithm for the hausdorff distance between convex polygons. Inf Process Lett 17:207–209CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Bartha P (2010) By parallel reasoning. Oxford University Press, OxfordCrossRefGoogle Scholar
  4. 4.
    Bdoiu M, Clarkson KL (2008) Optimal core-sets for balls. Comput Geom 40(1):14–22CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24:509–522CrossRefGoogle Scholar
  6. 6.
    Berg AC, Malik J (2001) Geometric blur for template matching. In: IEEE conference on computer vision and pattern recognition, pp 607–614Google Scholar
  7. 7.
    Borgwardt KM, Gretton A, Rasch MJ, Kriegel H-P, Schölkopf B, Smola A (2007) A kernel method for the two sample problem. Adv Neural Inf Process Syst 19:513–520Google Scholar
  8. 8.
    Carli A, Castellani U, Bicego M, Murino V (2010) Dissimilarity-based representation for local parts. In: International workshop on cognitive information processing (CIP), pp 299–303Google Scholar
  9. 9.
    Cevikalp H (2010) Semi-supervised distance metric learning by quadratic programming. In: International conference on pattern recognition, pp 3352–3355Google Scholar
  10. 10.
    Chang C-C, Lin C-J (2011) LIBSVM: a library for support vector machines. ACM Trans Intell Syst Technol 2:1–27CrossRefGoogle Scholar
  11. 11.
    Chen P, Fan RE, Lin C (2005) Training support vector machines via smo-type decomposition methods. In: International conference on algorithmic learning theoryGoogle Scholar
  12. 12.
    Cheplygina V, Tax DMJ, Loog M (2015) On classification with bags, groups and sets. Pattern Recogn Lett 59(1):11–17CrossRefGoogle Scholar
  13. 13.
    Coen MH, Ansari M, Fillmore N (2011) Learning from spatial overlap. In: AAAI conference on artificial intelligence, 2011Google Scholar
  14. 14.
    Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619CrossRefGoogle Scholar
  15. 15.
    Crosland MP (1978) Gay-Lussac: scientist and bourgeois. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  16. 16.
    Delos V, Teissandier D (2015) Minkowski sum of polytopes defined by their vertices. J Appl Math Phys 3(1):62–67CrossRefGoogle Scholar
  17. 17.
    Diu M, Gangeh M, Kamel MS (2013) Unsupervised visual changepoint detection using maximum mean discrepancy. In: Image analysis and recognition, 2013Google Scholar
  18. 18.
    Diu M (2013) Image analysis applications of the maximum mean discrepancy distance measure, PhD thesis, University of Waterloo, 2013Google Scholar
  19. 19.
    Doménech JL, Gil-Pérez D, Gras-Martí A, Guisasola J, Martínez-Torregrosa J, Salinas J, Trumper R, Valdés P, Vilches A (2007) Teaching of energy issues: a debate proposal for a global reorientation. Sci Educ 16(1):43–64CrossRefGoogle Scholar
  20. 20.
    Dueck D, Frey BJ (2007) Non-metric affinity propagation for unsupervised image categorization. In: International conference on computer vision, 2007, pp 1–8Google Scholar
  21. 21.
    Farid H, Simoncelli EP (2004) Differentiation of discrete multidimensional signals. IEEE Trans Image Process 13(4):496–508CrossRefMathSciNetGoogle Scholar
  22. 22.
    Galego R, Ferreira R, Bernardino A, Grossmann E, Gaspar J (2013) Topological auto-calibration of central imaging sensors. In: Pattern recognition and image analysis, 2013, pp 476–483Google Scholar
  23. 23.
    Geng B, Tao D, Xu C (2011) Daml: Domain adaptation metric learning. IEEE Trans Image Process 20(10):2980–2989CrossRefzbMATHMathSciNetGoogle Scholar
  24. 24.
    Gönen M, Alpaydın E (2011) Multiple kernel learning algorithms. J Mach Learn Res 12:2211–2268zbMATHMathSciNetGoogle Scholar
  25. 25.
    Goodrich B, Albrecht D, Tischer P (2009) Algorithms for the computation of reduced Convex Hulls. In: Australasian joint conference on advances in artificial intelligence,Google Scholar
  26. 26.
    Grauman K, Darrell T (2007) The pyramid match kernel: efficient learning with sets of features. J Mach Learn Res 8:725–760zbMATHGoogle Scholar
  27. 27.
    Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13:723–773zbMATHMathSciNetGoogle Scholar
  28. 28.
    Hafiz AM, Bhat GM (2014) Handwritten digit recognition using slope detail features. Int J Comput Appl 93(5):14–19Google Scholar
  29. 29.
    Hans A (2009) In resonance with nature: holistic healing for plants and land. Floris Books, EdinburghGoogle Scholar
  30. 30.
    Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554CrossRefGoogle Scholar
  31. 31.
    Jacobs DW, Weinshall D, Gdalyahu Y (2000) Classification with nonmetric distances: image retrieval and class representation. IEEE Trans Pattern Anal Mach Intell 22(6):583–600CrossRefGoogle Scholar
  32. 32.
    Jones WP, Furnas GW (1987) Pictures of relevance: a geometric analysis of similarity measures. J Am Soc Inf Sci 38(6):420–442CrossRefGoogle Scholar
  33. 33.
    Kim B, Pineau J (2013) Maximum mean discrepancy imitation learning. Robot Sci SystGoogle Scholar
  34. 34.
    Kinnunen T, Li H (2010) An overview of text-independent speaker recognition: from features to supervectors. Speech Commun 52(1):12–40CrossRefGoogle Scholar
  35. 35.
    Klippel A, Weaver C (2008) Analyzing behavioral similarity measures in linguistic and non-linguistic conceptualization of spatial information and the question of individual differences. In: Workshop on information semantics and its implications for geographical analysisGoogle Scholar
  36. 36.
    Kondor R, Jebara T (2003) A kernel between sets of vectors. In: International conference on machine learningGoogle Scholar
  37. 37.
    Kumar P, Mitchell JSB, Yildirim EA (2003) Approximate minimum enclosing balls in high dimensions using core-sets. J Exp Alg 8:1zbMATHMathSciNetGoogle Scholar
  38. 38.
    Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  39. 39.
    Leibe, B, Schiele B (2003) Analyzing appearance and contour based methods for object categorization. In: IEEE conference on computer vision and pattern recognitionGoogle Scholar
  40. 40.
    Li S (2011) Concise formulas for the area and volume of a hyperspherical cap. Asian J Math Stat 4(1):66–70CrossRefMathSciNetGoogle Scholar
  41. 41.
    Lin YY, Liu TL (2011) Multiple kernel learning for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 33:1–14CrossRefGoogle Scholar
  42. 42.
    Linnett JW (1942) The relation between potential energy and interatomic distance in some diatomic molecules. Trans Faraday Soc 38:1–9CrossRefGoogle Scholar
  43. 43.
    Liu H, Ding X (2005) Handwritten character recognition using gradient feature and quadratic classifier with multiple discrimination schemes. In: International conference on document analysis and recognition, pp 19–23Google Scholar
  44. 44.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110CrossRefGoogle Scholar
  45. 45.
  46. 46.
    Martin D, Fowlkes C, Malik J Tal D (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: International conference on computer visionGoogle Scholar
  47. 47.
    Mira J, Sandoval F (1995) From natural to artificial neural computation. In: International workshop on artificial neural networksGoogle Scholar
  48. 48.
    Misra G, Golshan B, Terzi E (2012) A framework for evaluating the smoothness of data-mining results. In: Joint European conference on machine learning and knowledge discovery in databasesGoogle Scholar
  49. 49.
    Moon TK (1996) Similarity methods in signal processing. IEEE Trans Signal Process 44(4):827–833CrossRefGoogle Scholar
  50. 50.
    MOSEK A (2008) The MOSEK optimization toolbox for MATLAB manual, http://ww.mosek.com/
  51. 51.
    Muandet K, Fukumizu K, Dinuzzo F, Schölkopf B (2012) Learning from distributions via support measure machines. In: Advances in neural information processing systems, pp 10–18Google Scholar
  52. 52.
    Müller A (1997) Integral probability metrics and their generating classes of functions. Adv Appl Probab 29:429–443CrossRefzbMATHMathSciNetGoogle Scholar
  53. 53.
    Pekalska E, Duin RPW (2001) Automatic pattern recognition by similarity representations - a novel approach. Electron Lett 37:159–160CrossRefGoogle Scholar
  54. 54.
    Piciarelli C, Micheloni C, Foresti GL (2008) Trajectory-based anomalous event detection. IEEE Trans Circuits Syst Video Technol 18(11):1544–1554CrossRefGoogle Scholar
  55. 55.
    Plat JC (1998) Fast training of support vector machines using sequential minimal optimization. J Mach Learn Res 1889–1918Google Scholar
  56. 56.
    Rolle K (2015) Heat and mass transfer. Cengage Learning, Ohio, USAGoogle Scholar
  57. 57.
    Schneider R (2013) Convex bodies: The Brunn–Minkowski theory. Cambridge University Press, CambridgeCrossRefzbMATHGoogle Scholar
  58. 58.
    Schölkopf, B (2001) The kernel trick for distances. In Advances in neural information processing systemsGoogle Scholar
  59. 59.
    Sebe N, Tian Q, Lew MS, Huang TS (2008) Guest editorial: Similarity matching in computer vision and multimedia. Comput Vis Image Underst 110(3):309–311CrossRefGoogle Scholar
  60. 60.
    Seidenari L, Serra G, Bagdanov AD, Del Bimbo A (2014) Local pyramidal descriptors for image recognition. IEEE Trans Pattern Anal Mach Intell 36(5):1033–1040CrossRefGoogle Scholar
  61. 61.
    Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905CrossRefGoogle Scholar
  62. 62.
    Simonovits M (2003) How to compute the volume in high dimension? Math Progr 97(1):337–374CrossRefzbMATHMathSciNetGoogle Scholar
  63. 63.
    Slimene A, Zagrouba E (2013) Kernel maximum mean discrepancy for region merging approach. In: Computer analysis of images and patterns, pp 475–482Google Scholar
  64. 64.
    Stark M, Schiele B (2007) How good are local features for classes of geometric objects. In: IEEE international conference on computer visionGoogle Scholar
  65. 65.
    Sun S (2013) A survey of multi-view machine learning. Neural Comput Appl 23(7–8):2031–2038CrossRefGoogle Scholar
  66. 66.
    Tax DMJ, Duin RPW (2004) Support vector data description. Mach Learn 54(1):45–66CrossRefzbMATHGoogle Scholar
  67. 67.
    Torki M, Elgammal A (2010) Putting local features on a manifold. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 1743–1750Google Scholar
  68. 68.
    Wang B, Sung KK, Ng TK (2002) The localized consistency principle for image matching under non-uniform illumination variation and affine distortion. In: European Conference on Computer Vision, pp 205–219Google Scholar
  69. 69.
    Wang J, Sang N, Wang Z, Gao C (2016) Similarity learning with top-heavy ranking loss for person re-identification. IEEE Signal Process Lett 23(1):84–88CrossRefGoogle Scholar
  70. 70.
    Weibel C (2007) Minkowski sums of polytopes: combinatorics and computation, PhD thesis, Lcole polytechnique fdrale de Lausanne (EPFL)Google Scholar
  71. 71.
    Williams C, Seeger M (2001) Using the Nystrom method to speed up kernel machines. In: Advances in Neural Information Processing Systems, pp 682–688Google Scholar
  72. 72.
    Xiong H, Chen XW (2006) Kernel-based distance metric learning for microarray data classification. BMC Bioinformatics 7(1):299CrossRefMathSciNetGoogle Scholar
  73. 73.
    Z H, Berg AC, Maire M, Malik J (2006) SVM-KNN: discriminative nearest neighbor classification for visual category recognition. In: IEEE conference on computer vision and pattern recognition, pp 2126–2136Google Scholar
  74. 74.
    Zhu X, Suk HI, Shen D (2014) Matrix-similarity based loss function and feature selection for Alzheimer’s disease diagnosis. In: IEEE conference on computer vision and pattern recognition, pp 3089–3096Google Scholar

Copyright information

© Springer-Verlag London 2017

Authors and Affiliations

  1. 1.Laboratoire LIMTIC, LR16ES06, Institut Supérieur d’InformatiqueUniversité de Tunis El ManarArianaTunisia

Personalised recommendations