Statistics and Computing

, Volume 16, Issue 2, pp 177–192 | Cite as

Modified repeated median filters

Article

Abstract

We discuss moving window techniques for fast extraction of a signal composed of monotonic trends and abrupt shifts from a noisy time series with irrelevant spikes. Running medians remove spikes and preserve shifts, but they deteriorate in trend periods. Modified trimmed mean filters use a robust scale estimate such as the median absolute deviation about the median (MAD) to select an adaptive amount of trimming. Application of robust regression, particularly of the repeated median, has been suggested for improving upon the median in trend periods. We combine these ideas and construct modified filters based on the repeated median offering better shift preservation. All these filters are compared w.r.t. fundamental analytical properties and in basic data situations. An algorithm for the update of the MAD running in time O(log n) for window width n is presented as well.

Keywords

Signal extraction Robust filtering Drifts Jumps Outliers Computational geometry Update algorithm 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bernholt T. and Fried R. 2003. Computing the update of the repeated median regression line in linear time. Information Processing Letters 88: 111–117.CrossRefMathSciNetGoogle Scholar
  2. Cormen T. H., Leiserson C. E., Rivest R. L., and Stein C. 2001. Introduction to Algorithms. Second edition. MIT Press, Cambridge, Massachusetts, and McGraw-Hill Book Company, New York.Google Scholar
  3. Davies P., Fried R., and Gather U. 2004. Robust signal extraction for on-line monitoring data. Journal of Statistical Planning and Inference 122: 65-78. Einbeck, J. and Kauermann, G. (2003). Online Monitoring with Local Smoothing Methods and Adaptive Ridging. Journal of Statistical Computation and Simulation 73, 913-929. Fitch, J., Coyle, E., Gallagher, N.: (1984) “Median Filtering by Threshold Decomposition”, IEEE Transactions on Acoustics, Speech and Signal Processing, Vol. 32, Nr. 6, S. 1183-1188CrossRefMathSciNetGoogle Scholar
  4. Fried R. 2004. Robust filtering of time series with trends. Journal of Nonparametric Statistics 16: 313–328.CrossRefMATHMathSciNetGoogle Scholar
  5. Fried R., Bernholt T., and Gather U. 2006. Repeated median and hybrid filters. Computational Statistics & Data Analysis, to appear. Technical Report 10/2004, SFB 475, University of Dortmund, Germany.Google Scholar
  6. Gather U. and Fried R. 2003. Robust estimation of scale for local linear temporal trends. Tatra Mountains Mathematical Publications 26: 87–101.MathSciNetGoogle Scholar
  7. Gather U., Fried R., Lanius V., and Imhoff M. 2001. Online monitoring of high dimensional physiological time series–-a case study. Estadística 53: 259–298.MathSciNetGoogle Scholar
  8. Gervini D. and Yohai V. J. 2002. A class of robust and fully efficient regression estimators. Annals of Statistics 30: 583–616.CrossRefMathSciNetGoogle Scholar
  9. Himayat N. and Kassam S. A. 1993. Approximate performance analysis of edge preserving filters. IEEE Transactions on Signal Processing 4: 2764–2776.CrossRefGoogle Scholar
  10. Imhoff M., Bauer M., Gather U., and Fried R. 2002. Pattern detection in intensive care monitoring time series with autoregressive models: Influence of the model order. Biometrical Journal 44: 746–761.CrossRefMathSciNetGoogle Scholar
  11. Lee Y. and Kassam S. 1985. Generalized median filtering and related nonlinear filtering techniques. IEEE Transactions on Acoustics, Speech, and Signal Processing 33: 672–683.CrossRefGoogle Scholar
  12. Matoušek J., Mount D. M. and Netanyahu N. 1998. Efficient randomized algorithms for the repeated median line estimator. Algorithmica 20: 136–150.CrossRefMathSciNetGoogle Scholar
  13. Pitas I. and Venetsanopoulos A. 1992. Order statistics in digital image processing. Proceedings of the IEEE 80: 1893–1921.CrossRefGoogle Scholar
  14. Rousseeuw P. J. 1984. Least median of squares regression. Journal of the American Statistical Association 79: 871–880.MATHMathSciNetGoogle Scholar
  15. Rousseeuw P. J. and Leroy A. M. 1987. Robust Regression and Outlier Detection. Wiley, New York.Google Scholar
  16. Siegel A. F. 1982. Robust regression using repeated medians. Biometrika 69: 242–244.CrossRefMATHGoogle Scholar
  17. Tukey J. W. 1977. Exploratory Data Analysis. Addison-Wesley, Reading, Mass. (Preliminary edition 1971).Google Scholar

Copyright information

© Springer Science + Business Media, LLC 2006

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of DortmundDortmundGermany
  2. 2.Department of StatisticsUniversity Carlos III de MadridGetafeSpain
  3. 3.Department of StatisticsUniversity of DortmundDortmundGermany

Personalised recommendations