Skip to main content
SpringerLink
Log in

Novelty Detection in Time Series Using Self-Organizing Neural Networks: A Comprehensive Evaluation

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In this survey paper, we report the results of a comprehensive study involving the application of dynamic self-organizing neural networks (SONNs) to the problem of novelty detection in time series data. The study is comprised of three main parts. In the first part, we aim at evaluating how the performances of nonrecurrent dynamic SONNs are influenced by the introduction of different short-term memory kernels, such as Gamma, Gamma II and Laguerre, in the network input. In the second part, we analyze the performances of recurrent dynamic SONNs with the goal of inferring if they possess or not any competitive advantage over nonrecurrent architectures for novelty detection in time series. Finally, in the third part of the study, we introduce an alternative approach for dynamic SONN-based novelty detection by revisiting the operator map framework introduced by Lampinen and Oja (Proceedings of the 6th Scandinavian conference on image analysis (SCIA’89), pp 120–127, 1989) and Lehtimäki et al. (Proceedings of the joint international conference on artificial neural networks and neural information processing (ICANN/ICONIP’2003), pp 622–629, 2003). This framework allows the design of dynamic SONNs whose neurons are regarded as adaptive local linear models. In this case, novel/abnormal patterns are detected based on the statistics of prediction errors of the local models. A comprehensive performance comparison involving several nonrecurrent and recurrent dynamic SONN architectures is carried out using both synthetic and real-world time series data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. Also known as Shewhart charts or process-behavior charts.

  2. Difference between the observed and the predicted value of the variable of interest.

  3. By definition, the (unilateral) Z-transform of a discrete-time signal \(\{x(n)\}_{n=0}^{\infty }\) is defined as \(X(z) = \mathcal {Z}\{x(n)\} = \sum _{n=0}^{\infty } x(n)z^{-n}\), where n is an integer and \(z=Ae^{j\phi }=A(\cos \phi + j\sin \phi )\). In words, the Z-transform converts a time series, which is a sequence of real (or complex) numbers, into a complex frequency domain representation.

  4. Roots of the polynomial in the denominator of \(G^0(z)\) and G(z). For Eq. (37), the pole is found by doing \(z-(1-\lambda )=0\).

  5. For the remaining parts of this paper, the label NORMAL will be used for known or ordinary patterns; and ABNORMAL for the novel, anomalous or unknown ones.

  6. The percentile of a distribution of values is a number \(N_{\alpha }\) such that a percentage \(100(1-\alpha )\) of the sample values are less than or equal to \(N_{\alpha }\).

References

  1. Alhoniemi E, Hollmén J, Simula O, Vesanto J (1999) Process monitoring and modeling using the self-organizing map. Integr Comput Aided Eng 6(1):3–14

    Google Scholar 

  2. Alippi C, Roveri M, Trovò F (2014) A self-building and cluster-based cognitive fault diagnosis system for sensor networks. IEEE Trans Neural Netw Learn Syst 25(6):1021–1032

    Article  Google Scholar 

  3. Barreto G, Mota J, Souza L, Frota R (2004) Nonstationary time series prediction using local models based on competitive neural networks. In: Orchard B, Yang C, Ali M (eds) Innovations in applied artificial intelligence. Lecture notes in computer science, vol 3029. Springer, Berlin, pp 1146–1155

    Chapter  Google Scholar 

  4. Barreto GA, Aguayo L (2009) Time series clustering for anomaly detection using competitive neural networks. In: Principe JC, Miikkulainen R (eds) Advances in self-organizing maps, vol LNCS-5629. Springer, New York, pp 28–36

    Chapter  Google Scholar 

  5. Barreto GA, Frota RA (2013) A unifying methodology for the evaluation of neural network models on novelty detection tasks. Pattern Anal Appl 16(1):83–97

    Article  MathSciNet  Google Scholar 

  6. Barreto GA, Mota JCM, Souza LGM, Frota RA, Aguayo L (2005) Condition monitoring of 3G cellular networks through competitive neural models. IEEE Trans Neural Netw 16(5):1064–1075

    Article  Google Scholar 

  7. Brighenti C, Sanz-Bobi MA (2011) Auto-regressive processes explained by self-organized maps: application to the detection of abnormal behavior in industrial processes. IEEE Trans Neural Netw 22(12):2078–2090

    Article  Google Scholar 

  8. Carpenter GA, Grossberg S (2003) Adaptive resonance theory. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, pp 87–90

    Google Scholar 

  9. Carpenter GA, Grossberg S, Rosen DB (1991) Fuzzy ART: fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural Netw 4(6):759–771

    Article  Google Scholar 

  10. Carpinteiro OAS, Lima I, Moreira EM, Pinheiro CAM, Seraphim E, Pinto JVL (2009) A hierarchical hybrid neural model with time integrators in long-term load forecasting. Neural Comput Appl 18(8):1057–1063

    Article  Google Scholar 

  11. Carvalho AX, Tanner MA (2005) Mixtures-of-experts of autoregressive time series: asymptotic normality and model specification. IEEE Trans Neural Netw 16(1):39–56

    Article  Google Scholar 

  12. Chappell GJ, Taylor JG (1993) The temporal Kohonen map. Neural Netw 6(3):441–445

    Article  Google Scholar 

  13. Chawla NV (2005) Data mining for imbalanced datasets: an overview. In: Maimon O, Rokach L (eds) Data mining and knowledge discovery handbook. Springer, New York, pp 853–867

    Chapter  Google Scholar 

  14. Clifton DA, Clifton LA, Bannister PR, Tarassenko L (2008) Automated novelty detection in industrial systems. In: Liu Y, Sun A, Loh HT, Lu WF, Lim EP (eds) Advances of computational intelligence in industrial systems. Studies in computational intelligence, vol 116. Springer, Berlin, pp 269–296

    Chapter  Google Scholar 

  15. Coelho DN, Barreto GA, Medeiros CMS, Santos JDA (2014) Performance comparison of classifiers in the detection of short circuit incipient fault in a three-phase induction motor. In: Proceedings of the 2014 IEEE symposium on computational intelligence for engineering solutions (CIES’2014). Orlando, pp 42–48

  16. Côme E, Cottrell M, Verleysen M, Lacaille J (2010) Aircraft engine health monitoring using self-organizing maps. In: Advances in data mining—applications and theoretical aspects. Springer, vol LNCS-6171, pp 405–417

  17. Dablemont S, Simon G, Lendasse A, Ruttiens A, Blayo F, Verleysen M (2003) Time series forecasting with SOM and local non-linear models—application to the DAX30 index prediction. In: Proceedings of the 4th workshop on self-organizing maps (WSOM’2003), pp 340–345

  18. Estevez PA, Hernandez R (2009) Gamma SOM for temporal sequence processing. In: Proceedings of 7th workshop on self-organizing maps (WSOM’09). Springer, vol LNCS-5629, pp 63–71

  19. Estevez PA, Hernandez R, Perez C, Held C (2011) Gamma-filter self-organising neural networks for unsupervised sequence processing. Electron Lett 47(8):494–496

    Article  Google Scholar 

  20. Fawcett T (2006) An introduction to ROC analysis. Pattern Recogn Lett 27(8):861–874

    Article  MathSciNet  Google Scholar 

  21. Fink O, Zio E, Weidmann U (2015) Novelty detection by multivariate kernel density estimation and growing neural gas algorithm. Mech Syst Signal Process 50–51:427–436

    Article  Google Scholar 

  22. Fritzke B (1994) A growing neural gas network learns topologies. In: Tesauro G, Touretzky DS, Leen TK (eds) Advances in neural information processing systems (NIPS), Denver, Colorado. MIT Press, Cambridge, MA, pp 625–632

  23. Fu TC (2011) A review on time series data mining. Eng Appl Artif Intell 37:164–181

    Article  Google Scholar 

  24. Galar M, Fernández A, Barrenechea E, Bustince H, Herrera F (2011) A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans Syst Man Cybern C-42(4):463–484

  25. Gan M, Peng H, Peng X, Chen X, Inoussa G (2010) A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling. Inf Sci 180(22):4370–4383

    Article  MathSciNet  Google Scholar 

  26. Gollamudi S, Nagaray S, Kapoor S, Huang Y (1998) Set-membership filtering and a set-membership normalized LMS algorithm with an adaptive step size. IEEE Signal Process Lett 5:111–114

    Article  Google Scholar 

  27. Granger E, Grossberg S, Rubin MA, Streilein WW (1999) Familiarity discrimination of radar pulses. In: Kearns MJ, Solla SA, Cohn DA (eds) Advances in neural information processing systems, vol 11. MIT Press, Cambridge, pp 875–881

    Google Scholar 

  28. Hampshire JB II, Weibel A (1992) The Meta-Pi network: building distributed knowledge representations for robust multisource pattern recognition. IEEE Trans Mach Intell Pattern Anal 14(7):751–769

    Article  Google Scholar 

  29. Hodge VJ, Austin J (2004) A survey of outlier detection methodologies. Artif Intell Rev 22(2):85–126

    Article  MATH  Google Scholar 

  30. Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE (1991) Adaptive mixtures of local experts. Neural Comput 3(1):79–87

    Article  Google Scholar 

  31. Jamsa-Jounela SL, Vermasvuori M, Enden P, Haavisto S (2003) A process monitoring system based on the kohonen self-organizing maps. Control Eng Pract 11(1):83–92

    Article  Google Scholar 

  32. Kangas JA, Kohonen TK, Laaksonen J (1990) Variants of self-organizing maps. IEEE Trans Neural Netw 1(1):93–99

    Article  Google Scholar 

  33. Kantz H, Schreiber T (1997) Nonlinear time series analysis. Cambridge University Press, Cambridge

    MATH  Google Scholar 

  34. Kohonen T (2013) Essentials of the self-organizing map. Neural Netw 37:52–65

    Article  Google Scholar 

  35. Kohonen TK (1990) The self-organizing map. Proc IEEE 78(9):1464–1480

    Article  Google Scholar 

  36. Koskela T, Varsta M, Heikkonen J, Kaski K (1998) Time series prediction using recurrent SOM with local linear models. Int J Knowl Based Intell Eng Syst 2(1):60–68

    Google Scholar 

  37. Lampinen J, Oja E (1989) Self-organizing maps for spatial and temporal AR models. In: Proceedings of the 6th Scandinavian conference on image analysis (SCIA’89). Helsinki, Finland, pp 120–127

  38. Lee HJ, Cho S (2005) SOM-based novelty detection using novel data. In: Gallagher M, Hogan JP, Maire F (eds) Intelligent data engineering and automated learning (IDEAL’2005). Lecture notes in computer science, vol 3578. Springer, New York, pp 359–366

    Chapter  Google Scholar 

  39. Lehtimäki P, Raivio K, Simula O (2003) Self-organizing operator maps in complex system analysis. In: Proceedings of the joint international conference on artificial neural networks and neural information processing (ICANN/ICONIP’2003). Istanbul, Turkey, pp 622–629

  40. Liao TW (2007) A clustering procedure for exploratory mining of vector time series. Pattern Recogn 40:2250–2562

    MATH  Google Scholar 

  41. Liao Y, Vemuri R, Pasos A (2007) Adaptive anomaly detection with evolving connectionist systems. J Netw Comput Appl 30:60–80

    Article  Google Scholar 

  42. Markou M, Singh S (2003) Novelty detection: a review—part 1: statistical approaches. Sig Process 83:2481–2497

    Article  MATH  Google Scholar 

  43. Martínez-Rego D, Fontenla-Romero O, Alonso-Betanzos A (2011) Efficiency of local models ensembles for time series prediction. Expert Syst Appl 38(6):6884–6894

    Article  Google Scholar 

  44. Mounce SR, Mounce RB, Boxall J (2011) Novelty detection for time series data analysis in water distribution systems using support vector machines. J HydroInform 13(4):672–686

    Article  Google Scholar 

  45. Nelson LS (1984) Technical aids. J Qual Technol 16(4):238–239

    Google Scholar 

  46. Neme A, Lugo B, Cervera A (2011) Authorship attribution as a case of anomaly detection: a neural network model. Int J Hybrid Intell Syst 8(4):225–235

    Article  Google Scholar 

  47. Neme A, Pulido J, Muñoz A, Hernández S, Dey T (2015) Stylistics analysis and authorship attribution algorithms based on self-organizing maps. Neurocomputing 147:147–159

    Article  Google Scholar 

  48. Ni H, Yin H (2009) A self-organising mixture autoregressive network for FX time series modelling and prediction. Neurocomputing 72:3529–3537

    Article  Google Scholar 

  49. Ni H, Yin H (2014) A neural gas mixture autoregressive network for modelling and forecasting FX time series. Neurocomputing 135:171–179

    Article  Google Scholar 

  50. Oakland JS (2007) Statistical process control, 6th edn. Routledge, London

    Google Scholar 

  51. Oliveira ALI, Meira SRL (2006) Detecting novelties in time series through neural networks forecasting with robust confidence intervals. Neurocomputing 70(1–3):79–92

  52. Oliveira ALI, Lima Neto FB, Meira SRL (2003) Novelty detection for short time series with neural networks. In: Abraham A, Köppen M, Franke K (eds) Design and application of hybrid intelligent systems. IOS Press, pp 66–75

  53. Pimentel MAF, Clifton DA, Clifton L, Tarassenko L (2014) A review of novelty detection. Sig Process 99:215–249

    Article  Google Scholar 

  54. Principe JC, de Vries B, de Oliveira PG (1994) An analysis of the gamma memory in dynamic neural networks. IEEE Trans Neural Netw 5(2):331–337

    Article  Google Scholar 

  55. Principe JC, de Vries B, Oliveira PG (1993) The gamma filter—a new class of adaptive IIR filters with restricted feedback. IEEE Trans Signal Process 41(2):649–656

    Article  MATH  Google Scholar 

  56. Sarasamma ST, Zhu QA (2006) Min-max hyperellipsoidal clustering for anomaly detection in network security. IEEE Trans Syst Man Cybern 36(4):887–901

    Article  Google Scholar 

  57. Singh S, Markou M (2004) An approach to novelty detection applied to the classification of image regions. IEEE Trans Knowl Data Eng 16(4):1041–4347

    Article  Google Scholar 

  58. Strickert M, Hammer B (2005) Merge SOM for temporal data. Neurocomputing 64:39–71

    Article  Google Scholar 

  59. Voegtlin T (2002) Recursive self-organizing maps. Neural Netw 15:979–991

    Article  Google Scholar 

  60. Wälchli M, Braun T (2009) Efficient signal processing and anomaly detection in wireless sensor networks. In: Giacobini M, Brabazon A, Cagnoni S, Di Caro GA, Ekárt A, Esparcia-Alcázar AI, Farooq M, Fink A, Machado P (eds) Applications of evolutionary computing. Lecture notes in computer science, vol 5484. Springer, Berlin, pp 81–86

    Google Scholar 

  61. Weigend AS, Gershenfeld NA (1994) Time series prediction: forecasting the future and understanding the past. Perseus Book, New York

    Google Scholar 

  62. Weigend AS, Mangeas M, Srivastana AN (1995) Nonlinear gated experts for time series: discovering regimes and avoiding overfitting. Int J Neural Syst 6(4):373–399

    Article  Google Scholar 

  63. Werner S, Diniz PSR (2001) Set-membership affine projection algorithm. IEEE Signal Process Lett 8(8):231–235

    Article  Google Scholar 

  64. Wong CS, Li WK (2001) On a logistic mixture autoregressive model. Biometrika 88(3):833–846

    Article  MathSciNet  MATH  Google Scholar 

  65. Wong M, Jack L, Nandi A (2006) Modified self-organising map for automated novelty detection applied to vibration signal monitoring. Mech Syst Signal Process 20(3):593–610

    Article  Google Scholar 

  66. Yu Y, Zhu Y, Li S, Wan D (2014) Time series outlier detection based on sliding window prediction. Math Probl Eng 2014(Article ID 879736):1–14. doi:10.1155/2014/879736

Download references

Acknowledgements

The first author thanks the support from CAPES for this work via a PNPD (National Program of Post-Doctorate) grant. The second author thanks CNPq for the Grant Number 309451/2015-9.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, University of Brasília, Brasília, Brazil

    Leonardo Aguayo

  2. Department of Teleinformatics Engineering, Federal University of Ceará (UFC), Fortaleza, Ceará, Brazil

    Guilherme A. Barreto

Authors
  1. Leonardo Aguayo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guilherme A. Barreto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guilherme A. Barreto.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aguayo, L., Barreto, G.A. Novelty Detection in Time Series Using Self-Organizing Neural Networks: A Comprehensive Evaluation. Neural Process Lett 47, 717–744 (2018). https://doi.org/10.1007/s11063-017-9679-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-017-9679-2

Keywords

Search

Navigation