Advertisement

Generative adversarial network based novelty detection usingminimized reconstruction error

  • Huan-gang Wang
  • Xin Li
  • Tao Zhang
Article

Abstract

Generative adversarial network (GAN) is the most exciting machine learning breakthrough in recent years, and it trains the learning model by finding the Nash equilibrium of a two-player zero-sum game. GAN is composed of a generator and a discriminator, both trained with the adversarial learning mechanism. In this paper, we introduce and investigate the use of GAN for novelty detection. In training, GAN learns from ordinary data. Then, using previously unknown data, the generator and the discriminator with the designed decision boundaries can both be used to separate novel patterns from ordinary patterns. The proposed GAN-based novelty detection method demonstrates a competitive performance on the MNIST digit database and the Tennessee Eastman (TE) benchmark process compared with the PCA-based novelty detection methods using Hotelling’s T2 and squared prediction error statistics.

Keywords

Generative adversarial network (GAN) Novelty detection Tennessee Eastman (TE) process 

CLC number

TP391 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abadi M, Andersen D, 2016. Learning to protect communications with adversarial neural cryptography. https://arxiv.org/abs/1610.06918Google Scholar
  2. Arjovsky M, Chintala S, Bottou L, 2017. Wasserstein generative adversarial networks. Int Conf on Machine Learning, p.214–223.Google Scholar
  3. Berthelot D, Schumm T, Metz L, 2017. BEGAN: boundary equilibrium generative adversarial networks. https://arxiv.org/abs/1703.10717Google Scholar
  4. Clifton L, Clifton D, Watkinson P, et al., 2011. Identification of patient deterioration in vital-sign data using one-class support vector machines. Federated Conf on Computer Science and Information Systems, p.125–131.Google Scholar
  5. Denton E, Chintala S, Fergus R, et al., 2015. Deep generative image models using a Laplacian pyramid of adversarial networks. Advances in Neural Information Processing Systems, p.1486–1494.Google Scholar
  6. Donahue J, Krähenbühl P, Darrell T, 2016. Adversarial feature learning. https://arxiv.org/abs/1605.09782Google Scholar
  7. Downs J, Vogel E, 1993. A plant-wide industrial process control problem. Comput Chem Eng, 17(3):245–255. https://doi.org/10.1016/0098-1354(93)80018-ICrossRefGoogle Scholar
  8. Dumoulin V, Belghazi I, Poole B, et al., 2016. Adversarially learned inference. https://arxiv.org/abs/1606.00704Google Scholar
  9. Ge Z, Song Z, 2013. Bagging support vector data description model for batch process monitoring. J Proc Contr, 23(8):1090–1096. https://doi.org/10.1016/j.jprocont.2013.06.010CrossRefGoogle Scholar
  10. Ge Z, Yang C, Song Z, 2009. Improved kernel PCA-based monitoring approach for nonlinear processes. Chem Eng Sci, 64(9):2245–2255. https://doi.org/10.1016/j.ces.2009.01.050CrossRefGoogle Scholar
  11. Ge Z, Gao F, Song Z, 2011. Batch process monitoring based on support vector data description method. J Proc Contr, 21(6):949–959. https://doi.org/10.1016/j.jprocont.2011.02.004CrossRefGoogle Scholar
  12. Ge Z, Song Z, Gao F, 2013. Review of recent research on data-based process monitoring. Ind Eng Chem Res, 52(10):3543–3562. https://doi.org/10.1021/ie302069qCrossRefGoogle Scholar
  13. Ge Z, Demyanov S, Chen Z, et al., 2017. Generative Open-Max for multi-class open set classification. https://arxiv.org/abs/1707.07418Google Scholar
  14. Goodfellow I, Pouget-Abadie J, Mirza M, et al., 2014. Generative adversarial nets. Advances in Neural Information Processing Systems, p.2672–2680.Google Scholar
  15. Grover A, Ermon S, 2017. Boosted generative models. https://arxiv.org/abs/1702.08484Google Scholar
  16. Hautamaki V, Karkkainen I, Franti P, 2004. Outlier detection using k-nearest neighbour graph. Proc 17th Int Conf on Pattern Recognition, p.430–433. https://doi.org/10.1109/ICPR.2004.1334558Google Scholar
  17. He Z, Deng S, Xu X, 2005. An optimization model for outlier detection in categorical data. LNCS, 3644:400–409. https://doi.org/10.1007/11538059_42Google Scholar
  18. Hoffmann H, 2007. Kernel PCA for novelty detection. Patt Recogn, 40(3):863–874. https://doi.org/10.1016/j.patcog.2006.07.009CrossRefzbMATHGoogle Scholar
  19. Kadurin A, Aliper A, Kazennov A, et al., 2017a. The cornucopia of meaningful leads: applying deep adversarial autoencoders for new molecule development in oncology. Oncotarget, 8(7):10883. https://doi.org/10.18632/oncotarget.14073CrossRefGoogle Scholar
  20. Kadurin A, Nikolenko S, Khrabrov K, et al., 2017b. dru-GAN: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico. Mol Pharmaceut, 14(9):3098–3104. https://doi.org/10.1021/acs.molpharmaceut.7b00346CrossRefGoogle Scholar
  21. Keogh E, Lonardi S, Ratanamahatana C, 2004. Towards parameter-free data mining. Proc 10th ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, p.206–215. https://doi.org/10.1145/1014052.1014077Google Scholar
  22. Kim T, Cha M, Kim H, et al., 2017. Learning to discover cross-domain relations with generative adversarial networks. https://arxiv.org/abs/1703.05192Google Scholar
  23. Ledig C, Theis L, Huszár F, et al., 2016. Photo-realistic single image super-resolution using a generative adversarial network. https://arxiv.org/abs/1609.04802Google Scholar
  24. Li J, Liang X, Wei Y, et al., 2017. Perceptual generative adversarial networks for small object detection. CVPR, p.1951–1959. https://doi.org/10.1109/CVPR.2017.211Google Scholar
  25. Li Y, Maguire L, 2011. Selecting critical patterns based on local geometrical and statistical information. IEEE Trans Patt Anal Mach Intell, 33(6):1189–1201. https://doi.org/10.1109/TPAMI.2010.188CrossRefGoogle Scholar
  26. Li Y, Liu S, Yang J, et al., 2017. Generative face completion. CVPR, p.5892–5900. https://doi.org/10.1109/CVPR.2017.624Google Scholar
  27. Luc P, Couprie C, Chintala S, et al., 2016. Semantic segmentation using adversarial networks. https://arxiv.org/abs/1611.08408Google Scholar
  28. Mahadevan S, Shah S, 2009. Fault detection and diagnosis in process data using one-class support vector machines. J Proc Contr, 19(10):1627–1639. https://doi.org/10.1016/j.jprocont.2009.07.011CrossRefGoogle Scholar
  29. Mao X, Li Q, Xie H, et al., 2016. Least squares generative adversarial networks. https://arxiv.org/abs/1611.04076Google Scholar
  30. Mogren O, 2016. C-RNN-GAN: continuous recurrent neural networks with adversarial training. https://arxiv.org/abs/1611.09904Google Scholar
  31. Patcha A, Park J, 2007. An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput Netw, 51(12):3448–3470. https://doi.org/10.1016/j.comnet.2007.02.001CrossRefGoogle Scholar
  32. Pimentel M, Clifton D, Clifton L, et al., 2014. A review of novelty detection. Signal Process, 99:215–249. https://doi.org/10.1016/j.sigpro.2013.12.026CrossRefGoogle Scholar
  33. Radford A, Metz L, Chintala S, 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. https://arxiv.org/abs/1511.06434Google Scholar
  34. Reed S, Akata Z, Yan X, et al., 2016. Generative adversarial text to image synthesis. Proc 33rd Int Conf on Machine Learning, p.1060–1069.Google Scholar
  35. Schlegl T, Seeböck P, Waldstein S, et al., 2017. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. Int Conf on Information Processing in Medical Imaging, p.146–157. https://doi.org/10.1007/978-3-319-59050-9_12CrossRefGoogle Scholar
  36. Springenberg J, 2015. Unsupervised and semi-supervised learning with categorical generative adversarial networks. https://arxiv.org/abs/1511.06390Google Scholar
  37. Vondrick C, Pirsiavash H, Torralba A, 2016. Generating videos with scene dynamics. Advances in Neural Information Processing Systems, p.613–621.Google Scholar
  38. Wu J, Zhang C, Xue T, et al., 2016. Learning a probabilistic latent space of object shapes via 3D generativeadversarial modeling. Advances in Neural Information Processing Systems, p.82–90.Google Scholar
  39. Xiao Y, Wang H, Xu W, et al., 2016. Robust one-class SVM for fault detection. Chemometr Intell Lab Syst, 151: 15–25. https://doi.org/10.1016/j.chemolab.2015.11.010CrossRefGoogle Scholar
  40. Yang Z, Chen W, Wang F, et al., 2017. Improving neural machine translation with conditional sequence generative adversarial nets. https://arxiv.org/abs/1703.04887Google Scholar
  41. Yeh R, Chen C, Lim T, et al., 2016. Semantic image inpainting with perceptual and contextual losses. https://arxiv.org/abs/1607.07539Google Scholar
  42. Yi Z, Zhang H, Gong P, et al., 2017. DualGAN: unsupervised dual learning for image-to-image translation. https://arxiv.org/abs/1704.02510Google Scholar
  43. Yu J, 2012. Semiconductor manufacturing process monitoring using Gaussian mixture model and Bayesian method with local and nonlocal information. IEEE Trans Semicond Manuf, 25(3):480–493. https://doi.org/10.1109/TSM.2012.2192945CrossRefGoogle Scholar
  44. Yu J, Qin S, 2008. Multimode process monitoring with Bayesian inference-based finite Gaussian mixture models. AIChE J, 54(7):1811–1829. https://doi.org/10.1002/aic.11515CrossRefGoogle Scholar
  45. Yu J, Qin S, 2009. Multiway Gaussian mixture model based multiphase batch process monitoring. Ind Eng Chem Res, 48(18):8585–8594. https://doi.org/10.1021/ie900479gCrossRefGoogle Scholar
  46. Yu L, Zhang W, Wang J, et al., 2017. SeqGAN: sequence generative adversarial nets with policy gradient. 31st AAAI Conf on Artificial Intelligence, p.2852–2858.Google Scholar
  47. Zhao F, Feng J, Zhao J, et al., 2018. Robust LSTMautoencoders for face de-occlusion in the wild. IEEE Trans Image Process, 27(2):778–790. https://doi.org/10.1109/TIP.2017.2771408MathSciNetCrossRefGoogle Scholar
  48. Zhao J, Mathieu M, LeCun Y, 2016. Energy-based generative adversarial network. https://arxiv.org/abs/1609.03126Google Scholar
  49. Zhu J, Park T, Isola P, et al., 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. https://arxiv.org/abs/1703.10593CrossRefGoogle Scholar

Copyright information

© Zhejiang University and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Department of Automation, School of Information Science and TechnologyTsinghua UniversityBeijingChina

Personalised recommendations