Advertisement

Applied Intelligence

, Volume 48, Issue 10, pp 3263–3279 | Cite as

Improved one-class classification using filled function

  • Javad HamidzadehEmail author
  • Mona Moradi
Article

Abstract

Novelty detection is the identification of new observation that a machine learning system is not aware. Detecting novel instances is one of the interesting topics in recent studies. The problem of the current methods is their high run-time, so often make them unusable for large data sets. This paper presents the proposed method concerning this problem. Focusing on the task of one-class classification, the labeled data are mapped into two hypersphere regions for target and non-target objects. This mapping process is considered as a nonlinear programming. The problem is solved by employing the filled function for finding global minimizer. The global minimizer is considered as a boundary which is fit the target class. In the end, a one-class classifier to detect target class members is obtained. To present the power of the proposed method, several experiments have been conducted based on 10-fold cross-validation over real-world data sets from UCI repository. Experimental results show that the proposed method is superior than the state-of-the-art competing methods regarding applied evaluation metrics.

Keywords

Novelty detection One-class classification Optimization problem Filled function 

References

  1. 1.
    Khan SS, Madden MG (2014) One-class classification: taxonomy of study and review of techniques. Knowl Eng Rev 29:345– 374CrossRefGoogle Scholar
  2. 2.
    Xin-min T, Wan-Hai C, Bao-Xiang D, Yong X, Han-Guang D (2007) A novel model of one-class bearing fault detection using SVDD and genetic algorithm. In: 2007 2nd IEEE Conference Industry of Electronics Applications.  https://doi.org/10.1109/ICIEA.2007.4318518, pp 802–807
  3. 3.
    Luo J, Li B, Wu Cq, Pan Y (2010) A fast SVDD algorithm based on decomposition and combination for fault detection. In: IEEE ICCA 2010, pp 1924–1928.  https://doi.org/10.1109/ICCA.2010.5524160
  4. 4.
    Zhuang J, Luo J, Peng Y, Wu C (2008) On-line fault detection method based on modified SVDD for industrial process system. In: 2008 3rd International Conference on Intelligent Systems of Knowledge Engineering, pp 754–760.  https://doi.org/10.1109/ISKE.2008.4731031
  5. 5.
    Xiao Y, Wang H, Zhang L, Xu W (2014) Two methods of selecting Gaussian kernel parameters for one-class SVM and their application to fault detection. Knowl-Based Syst 59:75–84.  https://doi.org/10.1016/j.knosys.2014.01.020 CrossRefGoogle Scholar
  6. 6.
    Tao H, Yun L, Ke W, Jian X, Fu L (2016) A new weighted SVDD algorithm for outlier detection. In: 2016 Chinese Control Decision Conference, pp 5456–5461.  https://doi.org/10.1109/CCDC.2016.7531972
  7. 7.
    Hao S, Zhou X, Song H (2015) A new method for noise data detection based on DBSCAN and SVDD. In: 2015 IEEE International Conference on Cybernetics Technology Automation and Intelligent Systems pp 784–789.  https://doi.org/10.1109/CYBER.2015.7288042
  8. 8.
    Park J, Kang D, Kim J, Kwok JT, Tsang IW (2007) SVDD-based pattern denoising. Neural Comput 19:1919–1938.  https://doi.org/10.1162/neco.2007.19.7.1919 MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Ritter G, Gallegos MT (1997) Outliers in statistical pattern recognition and an application to automatic chromosome classification. Pattern Recognit Lett 18:525–539.  https://doi.org/10.1016/S0167-8655(97)00049-4 CrossRefGoogle Scholar
  10. 10.
    Liu J, Deng H (2013) Outlier detection on uncertain data based on local information. Knowl-Based Syst 51:60–71.  https://doi.org/10.1016/j.knosys.2013.07.005 CrossRefGoogle Scholar
  11. 11.
    Chaki S, Verma AK, Routray A, Mohanty WK, Jenamani M (2014) A one-class classification framework using SVDD: Application to an imbalanced geological dataset. In: 2014 IEEE Students’ Technology Symposium (TechSym).  https://doi.org/10.1109/TechSym.2014.6807918, pp 76–81
  12. 12.
    Schölkopf B, Williamson RC, Smola AJ, Shawe-Taylor J, Platt JC (2000) Support vector method for novelty detection. In: Advance Neural Information and Processing System, pp 582–588Google Scholar
  13. 13.
    Bicego M, Figueiredo MAT (2009) Soft clustering using weighted one-class support vector machines. Pattern Recognit 42:27–32.  https://doi.org/10.1016/j.patcog.2008.07.004 CrossRefzbMATHGoogle Scholar
  14. 14.
    Huang G, Yang Z, Chen X, Ji G (2017) An innovative one-class least squares support vector machine model based on continuous cognition. Knowl-Based Syst 123:217–228.  https://doi.org/10.1016/j.knosys.2017.02.024 CrossRefGoogle Scholar
  15. 15.
    Utkin LV, Zhuk YA (2017) An one-class classification support vector machine model by interval-valued training data. Knowl-Based Syst 120:43–56.  https://doi.org/10.1016/j.knosys.2016.12.022 CrossRefGoogle Scholar
  16. 16.
    Zhu W, Zhong P (2014) A new one-class SVM based on hidden information. Knowl-Based Syst 60:35–43.  https://doi.org/10.1016/j.knosys.2014.01.002 CrossRefGoogle Scholar
  17. 17.
    Bose A, Beemanapalli K, Srivastava J, Sahar S (2007) Incorporating concept hierarchies into usage mining based recommendations. In: Nasraoui O, Spiliopoulou M, Srivastava J, Mobasher B, Masand B (eds) Advance Web of Mining Web Usage Analysis 8th International Work Knowledge Discovery Web, WebKDD 2006 Philadelphia, USA, August 20, 2006 Revising Papers, Springer, Berlin, pp 110–126.  https://doi.org/10.1007/978-3-540-77485-37
  18. 18.
    Chen Y, Wang K, Zhong P (2016) One-class support tensor machine. Knowl-Based Syst 96:14–28.  https://doi.org/10.1016/j.knosys.2016.01.007 CrossRefGoogle Scholar
  19. 19.
    Folguera L, Zupan J, Cicerone D, Magallanes JF (2015) Self-organizing maps for imputation of missing data in incomplete data matrices. Chemometr Intell Lab Syst 143:146–151.  https://doi.org/10.1016/j.chemolab.2015.03.002 CrossRefGoogle Scholar
  20. 20.
    Sadeghi R, Hamidzadeh J (2016) Automatic support vector data description. Soft Comput.  https://doi.org/10.1007/s00500-016-2317-5
  21. 21.
    Roth V (2006) Kernel fisher discriminants for outlier detection. Neural Comput 18:942–960MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Tax DMJ (2001) One-class classification: Concept learning in the absence of counter-examples, Technische Universiteit Delft. http://proquest.umi.com/pqdweb?did=728104171&Fmt=2&clientId=36097&RQT=309&VName=PQD
  23. 23.
    Chen B, Gao BC, Wu M, Bin Cheng X, Yan ZL (2012) Dimensionality reduction method of training sample set for SVDD based on statistical information. In: Advanced Manufacturing Technology Transactions in Technology Publications, pp 2097–2101.  https://doi.org/10.4028/www.scientific.net/AMM.220-223.2097
  24. 24.
    Cha M, Kim JS, Baek J-G (2014) Density weighted support vector data description. Expert Syst Appl 41:3343–3350.  https://doi.org/10.1016/j.eswa.2013.11.025 CrossRefGoogle Scholar
  25. 25.
    Cha M, Kim JS, Park SH, Baek J-G (2012) Nonparametric control chart using density weighted support vector data description. In: Proceedings of the World Academy of Science, Engineering and Technology, p 1020Google Scholar
  26. 26.
    Guo SM, Chen LC, Tsai JSH (2009) A boundary method for outlier detection based on support vector domain description. Pattern Recognit 42:77–83.  https://doi.org/10.1016/j.patcog.2008.07.003 CrossRefzbMATHGoogle Scholar
  27. 27.
    Cho H-W (2009) Data description and noise filtering based detection with its application and performance comparison. Expert Syst Appl 36:434–441.  https://doi.org/10.1016/j.eswa.2007.09.053 CrossRefGoogle Scholar
  28. 28.
    Le T, Tran D, Ma W, Sharma D (2012) Fuzzy multi-sphere support vector data description. In: 2012 IEEE International Conference on Fuzzy System, pp 1–5.  https://doi.org/10.1109/FUZZ-IEEE.2012.6251336
  29. 29.
    Chen G, Zhang X, Wang ZJ, Li F (2015) Robust support vector data description for outlier detection with noise or uncertain data. Knowl-Based Syst 90:129–137.  https://doi.org/10.1016/j.knosys.2015.09.025 CrossRefGoogle Scholar
  30. 30.
    Lai V, Nguyen D, Nguyen K, Le T (2015) Mixture of support vector data descriptions. In: 2015 2nd National Foundations of Science Technology and Development Conference of Information and Computing Science.  https://doi.org/10.1109/NICS.2015.7302178, pp 135–140
  31. 31.
    Wu T, Liang Y, Varela R, Wu C, Zhao G, Han X (2016) Self-adaptive SVDD integrated with AP clustering for one-class classification. Pattern Recognit Lett 84:232–238.  https://doi.org/10.1016/j.patrec.2016.10.009 CrossRefGoogle Scholar
  32. 32.
    Fung G, Mangasarian OL (2001) Proximal support vector machine classifiers. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery Data Mining. ACM, New York, pp 77–86.  https://doi.org/10.1145/502512.502527
  33. 33.
    Sanyang L, Jinjin L, De W, Wei D (2009) Confidence support vector domain description. J Syst Eng Electron 20:852–857Google Scholar
  34. 34.
    Wang J, Neskovic P, Cooper LN, Wang L (2007) Wang selecting data for fast support vector machines training. In: Chen K (ed) Trends Neural Computer, Springer, Berlin, pp 61–84.  https://doi.org/10.1007/978-3-540-36122-03
  35. 35.
    Liu B, Xiao Y, Cao L, Hao Z, Deng F (2013) SVDD-based outlier detection on uncertain data. Knowl Inf Syst 34:597–618CrossRefGoogle Scholar
  36. 36.
    Tai X, Wang L (2014) Develop an ensemble support vector data description method for improving invasive tamarisk mapping at regional scale. Int J Remote Sens 35:7030–7045.  https://doi.org/10.1080/01431161.2014.965283 Google Scholar
  37. 37.
    Xing H-J, Chen X-F (2012) Selective ensemble of support vector data descriptions for novelty detection. In: Wang J, Yen GG, Polycarpou MM (eds) Advance Neural Networks – ISNN 2012 9th International Symposium. Neural Networks, Shenyang, China, 2012. Proceedings, Part I.  https://doi.org/10.1007/978-3-642-31346-253. Springer, Berlin, pp 468–477
  38. 38.
    Theissler A (2017) Detecting known and unknown faults in automotive systems using ensemble-based anomaly detection. Knowledge-Based Syst 123:163–173.  https://doi.org/10.1016/j.knosys.2017.02.023 CrossRefGoogle Scholar
  39. 39.
    Hooshmand Moghaddam V, Hamidzadeh J (2016) New Hermite orthogonal polynomial kernel and combined kernels in support vector machine classifier. Pattern Recognit 60:921–935.  https://doi.org/10.1016/j.patcog.2016.07.004 CrossRefGoogle Scholar
  40. 40.
    Liang YM, Zhang LS, Li MM, Han BS (2007) A filled function method for global optimization. J. Comput. Appl. Math. 205:16–31.  https://doi.org/10.1016/j.cam.2006.04.038 MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Hamidzadeh J, Monsefi R, Yazdi HS (2014) LMIRA: large margin instance reduction algorithm. Neurocomputing 145:477–487.  https://doi.org/10.1016/j.neucom.2014.05.006 CrossRefGoogle Scholar
  42. 42.
    Renpu G (1990) A filled function method for finding a global minimizer of a function of several variables. Math Program 46:191–204MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Antczak T (2009) Exact penalty functions method for mathematical programming problems involving invex functions. Eur J Oper Res 198:29–36MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Chang C-C, Lin C-J (2011) LIBSVM: A library for support vector machines. ACM Trans Intell Syst Technol 2:27:1–27:27CrossRefGoogle Scholar
  45. 45.
    Tax DMJ, Duin RPW (2004) Support vector data description. Mach Learn 54:45–66.  https://doi.org/10.1023/B:MACH.0000008084.60811.49 CrossRefzbMATHGoogle Scholar
  46. 46.
    Jiang Y, Wang Y, Luo H (2015) Fault diagnosis of analog circuit based on a second map SVDD. Analog Integr Circ Signal Process 85:395–404.  https://doi.org/10.1007/s10470-015-0597-9 CrossRefGoogle Scholar
  47. 47.
    Zheng S (2016) Smoothly approximated support vector domain description. Pattern Recognit 49:55–64.  https://doi.org/10.1016/j.patcog.2015.07.003 CrossRefGoogle Scholar
  48. 48.
    Sheskin DJ (2007) Handbook of parametric and nonparametric statistical procedures, 4th edn. Chapman & Hall/CRC, NYzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Faculty of computer engineering and information technologySadjad University of TechnologyMashhadIran

Personalised recommendations