We discuss moving window techniques for fast extraction of a signal composed of monotonic trends and abrupt shifts from a noisy time series with irrelevant spikes. Running medians remove spikes and preserve shifts, but they deteriorate in trend periods. Modified trimmed mean filters use a robust scale estimate such as the median absolute deviation about the median (MAD) to select an adaptive amount of trimming. Application of robust regression, particularly of the repeated median, has been suggested for improving upon the median in trend periods. We combine these ideas and construct modified filters based on the repeated median offering better shift preservation. All these filters are compared w.r.t. fundamental analytical properties and in basic data situations. An algorithm for the update of the MAD running in time O(log n) for window width n is presented as well.
KeywordsSignal extraction Robust filtering Drifts Jumps Outliers Computational geometry Update algorithm
Unable to display preview. Download preview PDF.
- Cormen T. H., Leiserson C. E., Rivest R. L., and Stein C. 2001. Introduction to Algorithms. Second edition. MIT Press, Cambridge, Massachusetts, and McGraw-Hill Book Company, New York.Google Scholar
- Davies P., Fried R., and Gather U. 2004. Robust signal extraction for on-line monitoring data. Journal of Statistical Planning and Inference 122: 65-78. Einbeck, J. and Kauermann, G. (2003). Online Monitoring with Local Smoothing Methods and Adaptive Ridging. Journal of Statistical Computation and Simulation 73, 913-929. Fitch, J., Coyle, E., Gallagher, N.: (1984) “Median Filtering by Threshold Decomposition”, IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. 32, Nr. 6, S. 1183-1188CrossRefMathSciNetGoogle Scholar
- Fried R., Bernholt T., and Gather U. 2006. Repeated median and hybrid filters. Computational Statistics & Data Analysis, to appear. Technical Report 10/2004, SFB 475, University of Dortmund, Germany.Google Scholar
- Rousseeuw P. J. and Leroy A. M. 1987. Robust Regression and Outlier Detection. Wiley, New York.Google Scholar
- Tukey J. W. 1977. Exploratory Data Analysis. Addison-Wesley, Reading, Mass. (Preliminary edition 1971).Google Scholar