Abstract
In this article we consider the problem of detecting unusual values or outliers from time series data where the process by which the data are created is difficult to model. The main consideration is the fact that data closer in time are more correlated to each other than those farther apart. We propose two variations of a method that uses the median from a neighborhood of a data point and a threshold value to compare the difference between the median and the observed data value. Both variations of the method are fast and can be used for data streams that occur in quick succession such as sensor data on an airplane.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Box GEP, Jenkins GM, Reinsel GC (1994) Time series analysis: forecasting and control, 3rd edn. Prentice-Hall, Englewood Cliffs, NJ
Chang I, Tiao GC, Chen C (1988) Estimation of time series parameters in the presence of outliers. Technometrics 30:193–204
Pearson RK (2002) Data mining in the face of contaminated and incomplete records. In: Second SIAM conference on data mining, Arlington, VA
Peña D (2001) Outliers, influential observations, and missing data. In: Peña D, Tiao GC, Tsay RS (eds) A course in time series analysis. Wiley, New York, pp 136–170
Zhang S, Zhang C, Yang Q (2003) Data preparation for data mining. Appl Artif Intell 17:375–382
Zhang S, Zhang C, Yang Q (2004) Information enhancement for data mining. IEEE Intell Syst March/April 12–13
Author information
Authors and Affiliations
Corresponding author
Additional information
Martin Meckesheimer has been a member of the Applied Statistics Group at Phantom Works, Boeing since 2001. He received a Bachelor of Science Degree in Industrial Engineering from the University of Pittsburgh in 1997, and a Master's Degree in Industrial and Systems Engineering from Ecole Centrale Paris in 1999. Martin earned a Doctorate in Industrial Engineering from The Pennsylvania State University in August 2001, as a student of Professor Russell R. Barton and Dr. Timothy W. Simpson. His primary research interests are in the areas of design of experiments and surrogate modeling.
Sabyasachi Basu received his Ph.D. is Statistics from the University of Wisconsin at Madison in 1990. Since his Ph.D., he has worked in both academia and in industry. He has taught and guided Ph.D. students in the Department of Statistics at the Southern Methodist University. He has also worked as a senior marketing statistician at the J. C. Penney Company. Dr. Basu is also an American Society of Quality certified Six Sigma Black Belt. He is currently an Associate Technical Fellow in Statistics and Data Mining at the Boeing Company. In this capacity, he works as a researcher and a technical consultant within Boeing for data mining, statistics and process improvements. He has published more than 20 papers and technical reports. He has also served as journal referee for several journals, organized conferences and been invited to present at conferences.
Rights and permissions
About this article
Cite this article
Basu, S., Meckesheimer, M. Automatic outlier detection for time series: an application to sensor data. Knowl Inf Syst 11, 137–154 (2007). https://doi.org/10.1007/s10115-006-0026-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-006-0026-6