Advertisement

Combining Diverse One-Class Classifiers

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7209)

Abstract

Multiple Classifier Systems (MCSs) are the focus of intense research and a large variety of methods have been developed in order to exploit strengths of individual classifiers. In this paper we address the problem how to implement a multi-class classifier by an ensemble of one-class classifiers. To improve the performance of a compound classifier, different individual classifiers (which may e.g., differ in complexity, type, training algorithm or other) can be combined and that could increase its both performance, and robustness. The model of one-class classifiers is dedicated to recognize one class only, therefore it is a quite difficult to produce MCSs on the basis of it. One of the important problem is how to ensure diversity of classifier ensemble which consists of one-class classifiers. Well-known diversity measures have been developed for committees of multiclass classifiers. In this work we propose a novel diversity measure which can be applied to a set of one-class classifiers. Additionally we propose a classifier fusion model dedicated to one-class classifiers, which allows more than one classifier per class. We will try answer the question if increasing number of individual one-class classifier has an impact on quality of MCS. The proposed model was evaluated by computer experiments and their results prove that proposed model can outperform well known fusion methods.

Keywords

Pattern recognition machine learning one-class classification classifier ensemble diversity measure 
This is a preview of subscription content, log in to check access.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alpaydin, E.: Combined 5 x 2 cv f test for comparing supervised classification learning algorithms. Neural Computation 11(8), 1885–1892 (1999)CrossRefGoogle Scholar
  2. 2.
    Bishop, C.M.: Novelty detection and neural network validation. IEE Proceedings: Vision, Image and Signal Processing 141(4), 217–222 (1994)CrossRefGoogle Scholar
  3. 3.
    Burduk, R.: Costs-Sensitive Classification in Multistage Classifier with Fuzzy Observations of Object Features. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) HAIS 2011, Part II. LNCS (LNAI), vol. 6679, pp. 245–252. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  4. 4.
    Burduk, R., Kurzyñski, M.: Two-stage binary classifier with fuzzy-valued loss function. Pattern Analysis and Applications 9(4), 353–358 (2006)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Int. Res. 2, 263–286 (1995)zbMATHGoogle Scholar
  6. 6.
    Duan, K., Keerthi, S.S., Chu, W., Shevade, S.K., Poo, A.N.: Multi-Category Classification by Soft-Max Combination of Binary Classifiers. In: Windeatt, T., Roli, F. (eds.) MCS 2003. LNCS, vol. 2709, pp. 125–134. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  7. 7.
    Duda, R., Hart, P., Stork, D.: Pattern Classification. Wiley-Interscience (2001)Google Scholar
  8. 8.
    Duin, R.: The combining classifier: to train or not to train? In: Proceedings of 16th International Conference on Pattern Recognition, vol. 2, pp. 765–770 (2002)Google Scholar
  9. 9.
    van Erp, M., Vuurpijl, L., Schomaker, L.: An overview and comparison of voting methods for pattern recognition. In: Proceedings of Eighth International Workshop on Frontiers in Handwriting Recognition, pp. 195–200 (2002)Google Scholar
  10. 10.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
  11. 11.
    Giacinto, G., Perdisci, R., Del Rio, M., Roli, F.: Intrusion detection in computer networks by a modular ensemble of one-class classifiers. Inf. Fusion 9, 69–82 (2008)CrossRefGoogle Scholar
  12. 12.
    Hashem, S., Schmeiser, B., Yih, Y.: Optimal linear combinations of neural networks: an overview. In: 1994 IEEE International Conference on Neural Networks, IEEE World Congress on Computational Intelligence, June-2 July, vol. 3, pp. 1507–1512 (1994)Google Scholar
  13. 13.
    Ho, T.: The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)CrossRefGoogle Scholar
  14. 14.
    Hodge, V.J., Austin, J.: A survey of outlier detection methodologies. Artificial Intelligence Review 22(2), 85–126 (2004)zbMATHCrossRefGoogle Scholar
  15. 15.
    Hong, J.H., Min, J.K., Cho, U.K., Cho, S.B.: Fingerprint classification using one-vs-all support vector machines dynamically ordered with naïve bayes classifiers. Pattern Recogn. 41, 662–671 (2008)zbMATHCrossRefGoogle Scholar
  16. 16.
    Jain, A., Duin, R., Mao, J.: Statistical pattern recognition: a review. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(1), 4–37 (2000)CrossRefGoogle Scholar
  17. 17.
    Kittler, J., Alkoot, F.: Sum versus vote fusion in multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence 25(1), 110–115 (2003)CrossRefGoogle Scholar
  18. 18.
    Koch, M.W., Moya, M.M., Hostetler, L.D., Fogler, R.J.: Cueing, feature discovery, and one-class learning for synthetic aperture radar automatic target recognition. Neural Networks 8(7-8), 1081–1102 (1995)CrossRefGoogle Scholar
  19. 19.
    Krzanowski, W., Partrige, D.: Software Diversity: Practical Statistics for its Measurement and Exploatation. Department of Computer Science, University of Exeter. (1996)Google Scholar
  20. 20.
    Kuncheva, L.: Combining pattern classifiers: Methods and algorithms. Wiley-Interscience, New Jersey (2004)zbMATHCrossRefGoogle Scholar
  21. 21.
    Passerini, A., Pontil, M., Frasconi, P.: New results on error correcting output codes of kernel machines. IEEE Transactions on Neural Networks 15(1), 45–54 (2004)CrossRefGoogle Scholar
  22. 22.
    R Development Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2008) ISBN 3-900051-07-0, http://www.R-project.org
  23. 23.
    Rokach, L.: Pattern classification using ensemble methods. Series in machine perception and artificial intelligence. World Scientific (2010)Google Scholar
  24. 24.
    Schölkopf, B., Smola, A.: Learning with kernels: support vector machines, regularization, optimization, and beyond. In: Adaptive Computation and Machine Learning. MIT Press (2002)Google Scholar
  25. 25.
    Tax, D.M.J., Duin, R.P.W.: Combining One-Class Classifiers. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 299–308. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  26. 26.
    Tax, D., Duin, R.P.W.: Characterizing one-class datasets. In: Proceedings of the Sixteenth Annual Symposium of the Pattern Recognition Association of South Africa, pp. 21–26 (2005)Google Scholar
  27. 27.
    Wilk, T., Wozniak, M.: Soft computing methods applied to combination of one-class classifiers. Neurocomput. 75, 185–193 (2012)CrossRefGoogle Scholar
  28. 28.
    Xu, L., Krzyzak, A., Suen, C.: Methods of combining multiple classifiers and their applications to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics 22(3), 418–435 (1992)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Department of Systems and Computer NetworksWroclaw University of TechnologyWroclawPoland

Personalised recommendations

Cite paper

Buy options