Robust Neural Network for Novelty Detection on Data Streams

  • Andrzej Rusiecki
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7267)


In the on-line data processing it is important to detect a novelty as soon as it appears, because it may be a consequence of gross errors or sudden change in the analysed system. In this paper we present a framework of novelty detection, based on the robust neural network. To detect novel patterns we compare responses of two autoregressive neural networks. One of them is trained with a robust learning algorithm designed to remove the influence of outliers, while the other uses simple training, based on the least squares error criterion. We present also a simple and easy to use approach that adapts this technique to data streams. Experiments conducted on data containing novelty and outliers have shown promising performance of the new method, applied to analyse temporal sequences.


Data Stream Outlier Detection Feedforward Neural Network Network Output Training Pattern 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Barreto, G.A., Aguayo, L.: Time Series Clustering for Anomaly Detection Using Competitive Neural Networks. In: Príncipe, J.C., Miikkulainen, R. (eds.) WSOM 2009. LNCS, vol. 5629, pp. 28–36. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  2. 2.
    Brotherton, T., Johnson, T., Chadderdon, G.: Classification and Novelty Detection using Linear Models and a Class Dependent - Elliptical Bassi Function Neural Network. In: Proc. of the International Conference on Neural Networks, Anchorage (1998)Google Scholar
  3. 3.
    Chen, D.S., Jain, R.C.: A robust back propagation learning algorithm for function approximation. IEEE Trans. on Neural Networks 5, 467–479 (1994)CrossRefGoogle Scholar
  4. 4.
    Chu, F., Wang, Y., Zaniolo, C.: An Adaptive Learning Approach for Noisy Data Streams. In: Proc. of the 4th IEEE Int. Conf. on Data Mining, pp. 351–354 (2004)Google Scholar
  5. 5.
    Chuang, C., Su, S., Hsiao, C.: The Annealing Robust Backpropagation (ARBP) Learning Algorithm. IEEE Trans. on Neural Networks 11, 1067–1076 (2000)CrossRefGoogle Scholar
  6. 6.
    Crook, P., Hayes, G.: A Robot Implementation of a Biologically Inspired Method for Novelty Detection. In: Proceedings of TIMR 2001, Manchester (2001)Google Scholar
  7. 7.
    Hagan, M.T., Menhaj, M.B.: Training Feedforward Networks with the Marquardt Algorithm. IEEE Trans. on Neural Networks 5(6), 989–993 (1994)CrossRefGoogle Scholar
  8. 8.
    Hawkins, S., He, H., Williams, G.J., Baxter, R.A.: Outlier Detection Using Replicator Neural Networks. In: Kambayashi, Y., Winiwarter, W., Arikawa, M. (eds.) DaWaK 2002. LNCS, vol. 2454, pp. 170–180. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  9. 9.
    Himberg, J., Jussi, A., Alhoniemi, E., Vesanto, J., Simula, O.: The Self-Organizing Map as a Tool in Knowledge Engineering. In: Pattern Recognition in Soft Computing Paradigm, Soft Computing, WSP, pp. 38–65 (2001)Google Scholar
  10. 10.
    Hodge, V.J., Austin, J.: A Survey of Outlier Detection Methodologies. Kluwer Academic Publishers, The Netherlands (2004)Google Scholar
  11. 11.
    Huber, P.J.: Robust Statistics. Wiley, New York (1981)zbMATHCrossRefGoogle Scholar
  12. 12.
    Liano, K.: Robust error measure for supervised neural network learning with outliers. IEEE Transactions on Neural Networks 7, 246–250 (1996)CrossRefGoogle Scholar
  13. 13.
    Liu, J., Gader, P.: Neural Networks with Enhanced Outlier Rejection Ability for Off-line Handwritten Word Recognition. Pattern Recognition 35(10), 2061–2071 (2002)zbMATHCrossRefGoogle Scholar
  14. 14.
    Ma, J., Perkins, S.: Online Novelty Detection on Temporal Sequences. In: Proc. of 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Washington, D.C, pp. 613–618 (2003)Google Scholar
  15. 15.
    Marsland, S.: On-Line Novelty Detection Through Self-Organisation, with Application to Inspection Robotics, Ph.D. thesis, Faculty of Science and Engineering, University of Manchester, UK (2001)Google Scholar
  16. 16.
    Pernia-Espinoza, A.V., Ordieres-Mere, J.B., Martinez-de-Pison, F.J., Gonzalez-Marcos, A.: TAO-robust backpropagation learning algorithm. Neural Networks 18, 191–204 (2005)CrossRefGoogle Scholar
  17. 17.
    Rusiecki, A.: Robust MCD-Based Backpropagation Learning Algorithm. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2008. LNCS (LNAI), vol. 5097, pp. 154–163. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Rusiecki, A.: Fast Robust Learning Algorithm Dedicated to LMLS Criterion. In: Rutkowski, L., Scherer, R., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds.) ICAISC 2010, Part II. LNCS, vol. 6114, pp. 96–103. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  19. 19.
    Seborg, D.E., et al.: WIE Process Dynamics and Control, 2nd edn. Wiley (2004)Google Scholar
  20. 20.
    Taylor, O., Addison, D.: Novelty Detection Using Neural Network Technology. In: Proceedings of the COMADEN Conference (2000)Google Scholar
  21. 21.
    Weekley, R.A., Goodrich, R.K., Cornman, L.B.: An Algorithm for Classification and Outlier Detection of Time-Series Data. Journal of Atmospheric and Oceanic Technology 27(1), 94–107 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Andrzej Rusiecki
    • 1
  1. 1.Wroclaw University of TechnologyWroclawPoland

Personalised recommendations