Penalized Least Squares for Smoothing Financial Time Series

  • Adrian Letchford
  • Junbin Gao
  • Lihong Zheng
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7106)


Modeling of financial time series data by methods of artificial intelligence is difficult because of the extremely noisy nature of the data. A common and simple form of filter to reduce the noise originated in signal processing, the finite impulse response (FIR) filter. There are several of these noise reduction methods used throughout the financial instrument trading community. The major issue with these filters is the delay between the filtered data and the noisy data. This delay only increases as more noise reduction is desired. In the present marketplace, where investors are competing for quality and timely information, this delay can be a hindrance. This paper proposes a new FIR filter derived with the aim of maximizing the level of noise reduction and minimizing the delay. The model is modified from the old problem of time series graduation by penalized least squares. Comparison between five different methods has been done and experiment results have shown that our method is significantly superior to the alternatives in both delay and smoothness over short and middle range delay periods.


Penalized least squares Time series analysis Financial analysis Finite impulse response Time series data mining 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boudraa, A.O., Cexus, J.C.: EMD-based signal filtering. IEEE Transactions on Instrumentation and Measurement 56(6), 2196–2202 (2007)CrossRefGoogle Scholar
  2. 2.
    Donoho, D.L., Johnstone, I.M.: Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association 90(432), 1200–1224 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  3. 3.
    Donoho, D.L., Johnstone, I.M.: Minimax estimation via wavelet shrinkage. The Annals of Statistics 26(3), 879–921 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Drakakis, K.: Empirical mode decomposition of financial data. International Mathematical Forum 3(25), 1191–1202 (2008)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Ehlers, J.F.: Rocket science for traders: Digital signal processing applications. John Wiley & Sons, Inc., New York (2001)Google Scholar
  6. 6.
    Eilers, P.H.C.: A perfect smoother. Analytical Chemistry 75(14), 3631–3636 (2003)CrossRefGoogle Scholar
  7. 7.
    Ellis, C.A., Parbery, S.A.: Is smarter better? a comparison of adaptive, and simple moving average trading strategies. Research in International Business and Finance 19(3), 399–411 (2005)CrossRefGoogle Scholar
  8. 8.
    Farmer, D.J., Sidorowich, J.J.: Optimal shadowing and noise reduction. Physica D: Nonlinear Phenomena 47(3), 373–392 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Gardner, J.E.S.: Exponential smoothing: The state of the art–part II. International Journal of Forecasting 22(4), 637–666 (2006)CrossRefGoogle Scholar
  10. 10.
    Godsill, S.J., Doucet, A., West, M.: Monte Carlo smoothing for nonlinear time series. Journal of the American Statistical Association 99(465), 156–168 (2004)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Hale, D.: Recursive gaussian filters. Tech. rep., Center for Wave Phenomena (2006)Google Scholar
  12. 12.
    Hassani, H.: Singular spectrum analysis: Methodology and comparison. Journal of Data Sciences 5, 239–257 (2007), mPRA PaperGoogle Scholar
  13. 13.
    Hassani, H., Soofi, A.S., Zhigljavsky, A.A.: Predicting daily exchange rate with singular spectrum analysis. Nonlinear Analysis: Real World Applications 11(3), 2023–2034 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Hodrick, R.J., Prescott, E.C.: Postwar u.s. business cycles: An empirical investigation. Journal of Money, Credit and Banking 29(1), 1–16 (1997)CrossRefGoogle Scholar
  15. 15.
    Hsu, L.-Y., Horng, S.-J., He, M., Fan, P., Kao, T.-W., Khan, M.K., Run, R.-S., Lai, J.-L., Chen, R.-J.: Mutual funds trading strategy based on particle swarm optimization. Expert Systems with Applications 38(6), 7582–7602 (2011)CrossRefGoogle Scholar
  16. 16.
    Hull, A.: Hull moving average HMA (2011),
  17. 17.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. Transactions of the ASME Journal of Basic Engineering 82(Series D), 35–45 (1960)CrossRefGoogle Scholar
  18. 18.
    Kamen, E.W., Su, J.K.: Introduction to optimal estimation. Springer, London (1999)CrossRefzbMATHGoogle Scholar
  19. 19.
    Karunasingha, D.S.K., Liong, S.Y.: Enhancement of chaotic time series prediction with real-time noise reduction. In: International Conference on Small Hydropower - Hydro Sri Lanka (2007)Google Scholar
  20. 20.
    Klaas, M., Briers, M., Freitas, N.d., Doucet, A., Maskell, S., Lang, D.: Fast particle smoothing: if I had a million particles. In: Proceedings of the 23rd International Conference on Machine Learning, pp. 481–488 (2006)Google Scholar
  21. 21.
    Kostelich, E.J., Yorke, J.A.: Noise reduction in dynamical systems. Physical Review A 38(3), 1649 (1988)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Kostelich, E.J., Yorke, J.A.: Noise reduction: Finding the simplest dynamical system consistent with the data. Physica D: Nonlinear Phenomena 41(2), 183–196 (1990)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
  24. 24.
    Li, T., Li, Q., Zhu, S., Ogihara, M.: A survey on wavelet applications in data mining. SIGKDD Explor. Newsl. 4(2), 49–68 (2002), 772870CrossRefGoogle Scholar
  25. 25.
    Liu, Y., Liao, X.: Adaptive chaotic noise reduction method based on dual-lifting wavelet. Expert Systems with Applications 38(3), 1346–1355 (2011)CrossRefGoogle Scholar
  26. 26.
    Moore, J.B.: Discrete-time fixed-lag smoothing algorithms. Automatica 9(2), 163–173 (1973)CrossRefzbMATHGoogle Scholar
  27. 27.
    Nikpour, M., Nadernejad, E., Ashtiani, H., Hassanpour, H.: Using pde’s for noise reduction in time series. International Journal of Computing and ICT Research 3(1), 42–48 (2009)Google Scholar
  28. 28.
    Russell, S.J., Norvig, P.: Artificial intelligence: A modern approach. Pretice Hall series in artificial intelligence. Prentice Hall, N.J (2010)zbMATHGoogle Scholar
  29. 29.
    Sadasivan, P.K., Dutt, D.N.: SVD based technique for noise reduction in electroencephalographic signals. Signal Processing 55(2), 179–189 (1996)CrossRefzbMATHGoogle Scholar
  30. 30.
    Shynk, J.J.: Frequency-domain and multirate adaptive filtering. IEEE Signal Processing Magazine 9(1), 14–37 (1992)CrossRefGoogle Scholar
  31. 31.
    Soofi, A.S., Cao, L.: Nonlinear forecasting of noisy financial data. In: Soofi, A.S., Cao, L. (eds.) Modeling and Forecasting Financial Data: Techniques of Nonlinear Dynamics, pp. 455–465. Kluwer Academic Publishers, Boston (2002)CrossRefGoogle Scholar
  32. 32.
    Vandewalle, N., Ausloos, M., Boveroux, P.: The moving averages demystified. Physica A: Statistical Mechanics and its Applications 269(1), 170–176 (1999)CrossRefGoogle Scholar
  33. 33.
    Whittaker, E.T.: On a new method of graduation. Proceedings of the Edinburgh Mathematical Society 41(-1), 63–75 (1923)Google Scholar
  34. 34.
    Zehtabian, A., Hassanpour, H.: A non-destructive approach for noise reduction in time domain. World Applied Sciences Journal 6(1), 53–63 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Adrian Letchford
    • 1
  • Junbin Gao
    • 1
  • Lihong Zheng
    • 1
  1. 1.School of Computing and MathematicsCharles Sturt UniversityAustralia

Personalised recommendations