Beyond Standard Metrics – On the Selection and Combination of Distance Metrics for an Improved Classification of Hyperspectral Data

  • Uwe Knauer
  • Andreas Backhaus
  • Udo Seiffert
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 295)


Training and application of prototype based learning approaches such as Learning Vector Quantization, Radial Basis Function networks, and Supervised Neural Gas require the use of distance metrics to measure the similarities between feature vectors as well as class prototypes. While the Euclidean distance is used in many cases, the highly correlated features within the hyperspectral representation and the high dimensionality itself favor the use of more sophisticated distance metrics. In this paper we first investigate the role of different metrics for successful classification of hyperspectral data sets from real-world classification tasks. Second, it is shown that considerable performance gains can be achieved by a classification system that combines a number of prototype based models trained on differently parametrized divergence measures. Data sets are tested using a number of different combination strategies.


Divergence Metrics Hyperspectral SNG GLVQ RBF 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aggarwal, C.C., Hinneburg, A., Keim, D.A.: On the surprising behavior of distance metrics in high dimensional space. In: Van den Bussche, J., Vianu, V. (eds.) ICDT 2001. LNCS, vol. 1973, pp. 420–434. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  2. 2.
    Al-Ani, A., Deriche, M.: A new technique for combining multiple classifiers using the dempster-shafer theory of evidence. Journal of Artificial Intelligence Research 17, 333–361 (2002)MATHMathSciNetGoogle Scholar
  3. 3.
    Backhaus, A., Bollenbeck, F., Seiffert, U.: Robust classification of the nutrition state in crop plants by hyperspectral imaging and artificial neural networks. In: Proc. 3rd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing, Lisboa, Portugal (2011)Google Scholar
  4. 4.
    Bishop, C.M., Svensén, M.: Hierarchical Mixtures of Experts. In: 19th Conference on Uncertainty in Artificial Intelligence (2003)Google Scholar
  5. 5.
    Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)MATHCrossRefGoogle Scholar
  6. 6.
    Chen, X., Li, Y., Harrison, R., Zhang, Y.-Q.: Type-2 fuzzy logic-based classifier fusion for support vector machines. Applied Soft Computing 8(3), 1222–1231 (2008)CrossRefGoogle Scholar
  7. 7.
    Didaci, L., Fumera, G., Roli, F.: Diversity in Classifier Ensembles: Fertile Concept or Dead End? In: Zhou, Z.-H., Roli, F., Kittler, J. (eds.) MCS 2013. LNCS, vol. 7872, pp. 37–48. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  8. 8.
    Duin, R.P.W.: The combining classifier: to train or not to train. In: ICPR (2002)Google Scholar
  9. 9.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. In: European Conference on Computational Learning Theory, pp. 23–37 (1995)Google Scholar
  10. 10.
    Geweniger, T., Kästner, M., Villmann, T.: Optimization of parametrized divergences in fuzzy c-means. In: ESANN (2011)Google Scholar
  11. 11.
    Hammer, B., Strickert, M., Villmann, T.: Supervised Neural Gas with general similarity measure. Neural Processing Letters 21, 21–44 (2005)CrossRefGoogle Scholar
  12. 12.
    Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15, 1059–1068 (2002)CrossRefGoogle Scholar
  13. 13.
    Jordan, M.I., Jacobs, R.A.: Hierarchical Mixtures of experts and the EM-algorithm. Neural Computation 6(2), 181–214 (1994)CrossRefGoogle Scholar
  14. 14.
    Kang, S., Park, S.: A fusion neural network classifier for image classification. Pattern Recogn. Lett. 30(9), 789–793 (2009)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Kästner, M., Backhaus, A., Geweniger, T., Haase, S., Seiffert, U., Villmann, T.: Relevance learning in unsupervised vector quantization based on divergences. In: Laaksonen, J., Honkela, T. (eds.) WSOM 2011. LNCS, vol. 6731, pp. 90–100. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  16. 16.
    Khreich, W., Granger, E., Miri, A., Sabourin, R.: Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs. Pattern Recognition 43(8), 2732–2752 (2010)MATHCrossRefGoogle Scholar
  17. 17.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  18. 18.
    Knauer, U., Seiffert, U.: A Comparison of Late Fusion Methods for Object Detection. In: IEEE International Conference on Image Processing, pp. 1–8 (2013)Google Scholar
  19. 19.
    Knauer, U., Seiffert, U.: Cascaded Reduction and Growing of Result Sets for Combining Object Detectors. In: Zhou, Z.-H., Roli, F., Kittler, J. (eds.) MCS 2013. LNCS, vol. 7872, pp. 121–133. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  20. 20.
    Mendenhall, M.J., Merényi, E.: Relevance-based feature extraction for hyperspectral images. IEEE Transactions on Neural Networks 19(4), 658–672 (2008)CrossRefGoogle Scholar
  21. 21.
    Moody, J., Darken, C.J.: Fast learning in networks of locally tuned processing units. Neural Computation 1, 281–294 (1989)CrossRefGoogle Scholar
  22. 22.
    Mwebaze, E., Schneider, P., Schleif, F.-M., Haase, S., Villmann, T., Biehl, M.: Divergence based Learning Vector Quantization. In: Verleysen, M. (ed.) 18th European Symposium on Artificial Neural Networks (ESANN 2010), pp. 247–252. d-side publishing (2010)Google Scholar
  23. 23.
    Peltonen, J., Klami, A., Kaski, S.: Learning more accurate metrics for self-organizing maps. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 999–1004. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  24. 24.
    Płoński, P., Zaremba, K.: Improving performance of self-organising maps with distance metric learning method. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2012, Part I. LNCS, vol. 7267, pp. 169–177. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  25. 25.
    Schneider, P., Schleif, F.-M., Villmann, T., Biehl, M.: Generalized matrix learning vector quantizer for the analysis of spectral data. In: ESANN, pp. 451–456 (2008)Google Scholar
  26. 26.
    Villmann, T., Haase, S.: Divergence based vector quantization of spectral data. In: 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS), pp. 1–4 (2010)Google Scholar
  27. 27.
    Villmann, T., Haase, S.: Divergence-based vector quantization. Neural Comput. 23(5), 1343–1392 (2011)MATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Fraunhofer IFFMagdeburgGermany

Personalised recommendations