Anomaly Detection of Streaming Data Based on Deep Learning

Abstract

In today’s information age, with the rapid development of network technology and a large number of applications of sensors in daily life. The number of various types of data has shown a great growth. These data not only bring us convenience and benefits, but also pose a great challenge to us, that is, how to explore the information of the data and effectively analyze the specific significance contained in the data has become a hot research direction. In the large amount of data obtained, most of the data are in the normal state and only represent the basic “value” information. Compared with the data in the normal state, the information carried by a small part of the data is more worthy of attention. The emergence of these data is the so-called “exception”. The occurrence of exceptions indicates that the system that should be running normally has changed, and these changes often have a negative impact on the system. Looking at these applications, it is not difficult to find that the actual frequency of these anomalies accounts for a very small proportion compared with a large number of normal time, but its value is very huge. Therefore, real-time online anomaly detection of convective data has very important value and significance for scientific research and industrial application. In the process of this research, I have a more comprehensive and detailed understanding of the research field of “anomaly detection of stream data” through theoretical learning, apply the deep learning model to anomaly detection of stream data, and test the performance of the model through experiments. By comparing the performance of traditional anomaly detection algorithms, unsupervised and semi supervised machine learning models and neural networks such as LSTM/HTM in stream data anomaly detection, we try to find an algorithm that can detect stream data information in real time and provide anomaly alarm in time.