Approach based on wavelet analysis for detecting and amending anomalies in dataset
- 31 Downloads
It is difficult to detect the anomalies whose matching relationship among some data attributes is very different from others’ in a dataset. Aiming at this problem, an approach based on wavelet analysis for detecting and amending anomalous samples was proposed. Taking full advantage of wavelet analysis’ properties of multi-resolution and local analysis, this approach is able to detect and amend anomalous samples effectively. To realize the rapid numeric computation of wavelet translation for a discrete sequence, a modified algorithm based on Newton-Cores formula was also proposed. The experimental result shows that the approach is feasible with good result and good practicality.
Key wordsdata preprocessing wavelet analysis anomaly detecting data mining
Unable to display preview. Download preview PDF.
- Eskin E. Anomaly detection over noisy data using learned probability distributions [C]// Langley P. Proceedings of the 17th International Conference on Machine Learning. San Francisco: Morgan Kaufmann Publishers Inc, 2000: 255–262.Google Scholar
- Knorr E M, Ng R T. Algorithms for mining distance-based outliers in large datasets [C]// Gupta A, Shmueli O, Widom J. Proceedings of the 24th International Conference on Very Large Data Bases. New York: Morgan Kaufmann, 1998: 392–403.Google Scholar
- Knorr E M, Ng R T. Finding intensional knowledge of distance-based outliers[C]// Atkinson M P, Orlowska M E, Valduriez P. Proceedings of the 25th International Conference on Very Large Data Bases. Edinburgh: Morgan Kaufmann, 1999: 211–222.Google Scholar
- Ramaswamy S, Rastogi R, Kyuseok S. Efficient algorithms for mining outliers from large data sets[C]// Chen W, Naughton J F, Bernstein P A. Proceedings of the ACM SIGMOD International Conference on Management of Data. Dallas: ACM Press, 2000: 427–438.Google Scholar
- Breunig M M, Kriegel H P, Ng R T, et al. OPTICS-OF: identifying local outliers [C]// Zytkow J M, Rauch J. Proceedings of the 3rd European Conference on Principles and Practice of Knowledge Discovery in Databases. Berlin: Springer, 1999: 262–270.Google Scholar
- Breunig M M, Kriegel H P, Ng R. T, et al. LOF: identifying density-based local outliers[C]// Chen W, Naughton J F, Bernstein P A. Proceedings of the ACM SIGMOD International Conference on Management of Data. Dallas: ACM Press, 2000: 93–104.Google Scholar
- Arshad M H, Chan P K. Identifying outliers via clustering for anomaly detection[EB/OL]. [2003-06-13]. http://www.cs.fit.edu/Projects/tech-reports/cs-2003-19.pdfGoogle Scholar
- HE Zeng-you, DENG Sheng-chun, XU Xiao-fei. Outlier detection integrating semantic knowledge [C]// Proceeding of the 3rd International Conference on Web-Age Information Management. London: Springer-verlag, 2002: 126–131.Google Scholar
- YANG Fu-sheng. Wavelet transformation’s analysis and application in engineering[M]. Beijing: Science Press, 2000. (in Chinese)Google Scholar
- LI Qing-yan, WANG Neng-chao, YI Da-yi. Numerical analysis[M]. 3rd ed. Wuhan: Huazhong University of Science and Technology Press, 1986. (in Chinese)Google Scholar