Skip to main content

A Continuous Differentiable Wavelet Shrinkage Function for Economic Data Denoising

Abstract

In economic (financial) time series analysis, prediction plays an important role and the inclusion of noise in the time series data is also a common phenomenon. In particular, stock market data are highly random and non-stationary, thus they contain much noise. Prediction of the noise-free data is quite difficult when noise is present. Therefore, removal of such noise before predicting can significantly improve the prediction accuracy of economic models. Based on this consideration, this paper proposes a new shrinkage (thresholding) function to improve the performance of wavelet shrinkage denoising. The proposed thresholding function is an arctangent function with several parameters to be determined and the optimal parameters are determined by ensuring that the thresholding function satisfies the condition of continuously differentiable. The closing price data with the Shanghai Composite Index from January 1991 to December 2014 are used to illustrate the application of the proposed shrinkage function in denoising the stock data. The experimental results show that compared with the classical shrinkage (hard, soft, and nonnegative garrote) functions, the proposed thresholding function not only has the advantage of continuous derivative, but also has a very competitive denoising performance.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

References

  • Afanasyev, D. O., & Fedorova, E. A. (2016). The long-term trends on the electricity markets: Comparison of empirical mode and wavelet decompositions. Energy Economics, 56, 432–442.

    Google Scholar 

  • Aguiar-Conraria, L., Azevedo, N., & Soares, M. J. (2008). Using wavelets to decompose the time–frequency effects of monetary policy. Physica A, 387, 2863–2878.

    Google Scholar 

  • Alzahrani, M., Masih, M., & Al-Titi, O. (2014). Linear and non-linear Granger causality between oil spot and futures prices: A wavelet based test. Journal of International Money and Finance, 48, 175–201.

    Google Scholar 

  • Amasyali, K., & EI-Ghoary, N. M. (2018). A review of data-driven building energy consumption prediction studies. Renewable and Sustainable Energy Reviews, 81, 1192–1205.

    Google Scholar 

  • Bagheri, A., Peyhani, H. M., & Akbari, M. (2014). Financial forecasting using ANFIS networks with quantum-behaved particle swarm optimization. Expert Systems with Applications, 41, 6235–6250.

    Google Scholar 

  • Bai, D. D., & Wei, J. Y. (2015). Time series forecasting model based on wavelet denoising application in manufacturing PMI prediction. In Proceedings of international symposium on social science (ISSS 2015), pp. 434–437.

  • Barboza, F., Kimura, H., & Altman, E. (2017). Machine learning models and bankruptcy prediction. Expert Systems with Applications, 83, 405–417.

    Google Scholar 

  • Barunik, J., & Hlinkova, M. (2016). Revisiting the long memory dynamics of the implied–realized volatility relationship: New evidence from the wavelet regression. Economic Modelling, 54, 503–514.

    Google Scholar 

  • Bayer, F. A., Lorenzen, M., Muller, M. A., & Allgower, F. (2016). Robust economic model predictive control using stochastic information. Automatica, 74, 151–161.

    Google Scholar 

  • Benedetto, F., Giunta, G., & Mastroeni, L. (2015). A maximum entropy method to assess the predictability of financial and commodity prices. Digital Signal Processing, 46, 19–31.

    Google Scholar 

  • Boubaker, H. (2015). Wavelet estimation of Gegenbauer processes: Simulation and empirical application. Computational Economics, 46, 551–574.

    Google Scholar 

  • Boubaker, H., & Raza, S. A. (2017). A wavelet analysis of mean and volatility spillovers between oil and BRICS stock markets. Energy Economics, 64, 105–117.

    Google Scholar 

  • Breiman, L. (1995). Better subset regression using the nonnegative garrote. Technometrics, 37, 373–384.

    Google Scholar 

  • Breiman, L. (2001). Statistical modeling: The two cultures. Statistical Science, 16(3), 199–231.

    Google Scholar 

  • Brislawn, C. M. (1996). Classification of nonexpansive symmetric extension transforms for multirate filter banks. Applied and Computational Harmonic Analysis, 3(4), 337–357.

    Google Scholar 

  • Bruzda, J. (2017). Real and complex wavelets in asset classification: An application to the US stock market. Finance Research Letters, 21, 115–125.

    Google Scholar 

  • Cao, L. Y., Hong, Y. G., Zhao, H. Z., & Deng, S. H. (1996). Predicting economic time series using a nonlinear deterministic technique. Computational Economics, 9, 149–178.

    Google Scholar 

  • Cavalcante, R. C., Brasileiro, R. C., Souza, V. L. F., & Nobrega, J. P. (2016). Computational intelligence and financial markets: A survey and future directions. Expert Systems with Applications, 55, 194–211.

    Google Scholar 

  • Chang, C. P., & Lee, C. C. (2015). Do oil spot and futures prices move together? Energy Economics, 50, 379–390.

    Google Scholar 

  • Chen, W. D., & Li, H. C. (2016). Wavelet decomposition of heterogeneous investment horizon. Journal of Economics and Finance, 40, 713–734.

    Google Scholar 

  • Chiang, W. C., Enke, D., Wu, T., & Wang, R. (2016). An adaptive stock index trading decision support system. Expert Systems with Applications, 59, 195–207.

    Google Scholar 

  • Coifman, R. R., & Wickerhauser, M. V. (1992). Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory, 38(2), 713–718.

    Google Scholar 

  • de Souza e Silva, E. G., Legey, L. F. L., & de Souza e Silva, E. A. (2010). Forecasting oil price trends using wavelets and hidden Markov models. Energy Economics, 32, 1507–1519.

    Google Scholar 

  • Dewandaru, G., Masih, R., & Masih, A. M. M. (2016). What can wavelets unveil about the vulnerabilities of monetary integration? A tale of Eurozone stock markets. Economic Modelling, 52, 981–996.

    Google Scholar 

  • Dhifaoui, Z. (2018). Statistical moments of Gaussian kernel correlation sum and weighted least square estimator of correlation dimension and noise level. Journal of Statistical Planning and Inference, 193, 55–69.

    Google Scholar 

  • Donoho, D. L. (1995). De-noising by soft-thresholding. IEEE Transactions on Information Theory, 41(3), 613–627.

    Google Scholar 

  • Donoho, D. L., & Johnstone, I. M. (1994). Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81(3), 425–455.

    Google Scholar 

  • Donoho, D. L., & Johnstone, I. M. (1995). Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90(432), 1200–1224.

    Google Scholar 

  • Donoho, D. L., & Johnstone, I. M. (1998). Minimax estimation via wavelet shrinkage. Annals of Statistics, 26(3), 879–921.

    Google Scholar 

  • Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., & Picard, D. (1995). Wavelet shrinkage: Asymptopia? Journal of the Royal Statistical Society: Series B, 57, 301–369.

    Google Scholar 

  • Egami, M., & Kevkhishvili, R. (2017). An analysis of simultaneous company defaults using a shot noise process. Journal of Banking & Finance, 80, 135–161.

    Google Scholar 

  • Elad, M. (2010). Sparse and redundant representations: From theory to applications in signal and image processing. New York: Springer.

    Google Scholar 

  • Ferrer, R., Bolos, V. J., & Benitez, R. (2016). Interest rate changes and stock returns: A European multi-country study with wavelets. International Review of Economics and Finance, 44, 1–12.

    Google Scholar 

  • Ferretti, M., & Rizzo, D. (2000). Handling borders in systolic architectures for the 1-D discrete wavelet transform for perfect reconstruction. IEEE Transactions on Signal Processing, 48(5), 1365–1378.

    Google Scholar 

  • Francke, M. K., & Minne, A. (2017). The hierarchical repeat sales model for granular commercial real estate and residential price indices. The Journal of Real Estate Finance and Economics, 55, 511–532.

    Google Scholar 

  • Ftiti, Z., Tiwari, A., Belanes, A., & Guesmi, K. (2015). Tests of financial market contagion: Evolutionary cospectral analysis versus wavelet analysis. Computational Economics, 46, 575–611.

    Google Scholar 

  • Furlaneto, D. C., Oliveira, L. S., Menotti, D., & Cavalcanti, G. D. C. (2017). Bias effect on predicting market trends with EMD. Expert Systems with Applications, 82, 19–26.

    Google Scholar 

  • Gao, H. Y. (1998). Wavelet shrinkage denoising using the non-negative garrote. Journal of Computational and Graphical Statistics, 7(4), 469–488.

    Google Scholar 

  • Gao, H. Y., & Bruce, A. G. (1997). WaveShrink with firm shrinkage. Statistica Sinica, 7, 855–874.

    Google Scholar 

  • Gao, J. B., Sultan, H., Hu, J., & Tung, W. W. (2010). Denoising nonlinear time series by adaptive filtering and wavelet shrinkage: A comparison. IEEE Signal Processing Letters, 17(3), 237–240.

    Google Scholar 

  • Gidea, M., & Katz, Y. (2018). Topological data analysis of financial time series: Landscapes of crashes. Physica A, 491, 820–834.

    Google Scholar 

  • Gomes, J., & Velho, L. (2015). From Fourier analysis to wavelets. Basel, Switzerland: Springer.

    Google Scholar 

  • Gopal, S., & Ramasamy, M. (2017). Hybrid multiple structural break model for stock price trend prediction. The Spanish Review of Financial Economics, 15, 41–51.

    Google Scholar 

  • Greenblatt, S. A. (1998). Atomic decomposition of financial data. Computational Economics, 12, 275–293.

    Google Scholar 

  • Gunduz, H., Yaslan, Y., & Cataltepe, Z. (2017). Intraday prediction of Borsa Istanbul using convolutional neural networks and feature correlations. Knowledge-Based Systems, 137, 138–148.

    Google Scholar 

  • Hammoudeh, S., & McAleer, M. (2015). Advances in financial risk management and economic policy uncertainty: An overview. International Review of Economics and Finance, 40, 1–7.

    Google Scholar 

  • He, K., Lai, K. K., & Yen, J. (2012a). Ensemble forecasting of value at risk via multi resolution analysis based methodology in metals markets. Expert Systems with Applications, 39, 4258–4267.

    Google Scholar 

  • He, K., Yu, L., & Lai, K. K. (2012b). Crude oil price analysis and forecasting using wavelet decomposed ensemble model. Energy, 46, 564–574.

    Google Scholar 

  • Hoesli, M., Kadilli, A., & Reka, K. (2017). Commonality in liquidity and real estate securities. The Journal of Real Estate Finance and Economics, 55, 65–105.

    Google Scholar 

  • Huang, S. C. (2011). Integrating spectral clustering with wavelet based kernel partial least square regressions for financial modeling and forecasting. Applied Mathematics and Computation, 217, 6755–6764.

    Google Scholar 

  • Huang, N. E., Shen, Z., Long, S. R., Wu, M. C., Shih, H. H., Zheng, Q., et al. (1998). The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. In Proceedings of the royal society of London a: Mathematical, physical and engineering sciences (Vol. 454, pp. 903–995). The Royal Society.

  • Hussain, A. J., Al-Jumeily, D., Al-Askar, H., & Radi, N. (2016). Regularized dynamic self-organized neural network in spired by the immune algorithm for financial time series prediction. Neurocomputing, 188, 23–30.

    Google Scholar 

  • In, F., & Kim, S. (2006). Multiscale hedge ratio between the Australian stock and futures markets: Evidence from wavelet analysis. Journal of Multinational Financial Management, 16, 411–423.

    Google Scholar 

  • Jammazi, R. (2012). Cross dynamics of oil-stock interactions: A redundant wavelet analysis. Energy, 44, 750–777.

    Google Scholar 

  • Jammazi, R., & Aloui, C. (2012). Crude oil price forecasting: Experimental evidence from wavelet decomposition and neural network modeling. Energy Economics, 34, 828–841.

    Google Scholar 

  • Jammazi, R., Lahiani, A., & Nguyen, D. K. (2015). A wavelet-based nonlinear ARDL model for assessing the exchange rate pass-through to crude oil prices. Journal of International Financial Markets, Institutions & Money, 34, 173–187.

    Google Scholar 

  • Jardin, P. (2017). Dynamics of firm financial evolution and bankruptcy prediction. Expert Systems with Applications, 75, 25–43.

    Google Scholar 

  • Jiang, M. H., An, H. Z., Jia, X. L., & Sun, X. Q. (2017). The influence of global benchmark oil prices on the regional oil spot market in multi-period evolution. Energy, 118, 742–752.

    Google Scholar 

  • Khashman, A., & Dimililer, K. (2008). Image compression using neural networks and Haar wavelet. WSEAS Transactions on Signal Processing, 4(5), 330–339.

    Google Scholar 

  • Kim, S., & In, F. (2007). On the relationship between changes in stock prices and bond yields in the G7 countries: Wavelet analysis. Journal of International Financial Markets, Institution & Money, 17, 167–179.

    Google Scholar 

  • Kinateder, H., Hofstetter, B., & Wagner, N. (2017). Do liquidity variables improve out-of-sample prediction of sovereign spreads during crisis periods? Finance Research Letters, 21, 144–150.

    Google Scholar 

  • Kolosinska, M. I., & Kolosinskyi, Y. Y. (2013). Analysis and forecast of the basic principles of tourist market development in Ukraine using the methods of economic-mathematical modeling. Actual Problems of Economics, 10(148), 222–227.

    Google Scholar 

  • Konchitchki, Y., & Patatoukas, P. N. (2014). Taking the pulse of the real economy using financial statement analysis: Implications for macro forecasting and stock valuation. The Accounting Review, 89(2), 669–694.

    Google Scholar 

  • Kriechbaumer, T., Angus, A., Parsons, D., & Casado, M. R. (2014). An improved wavelet-ARIMA approach for forecasting metal prices. Resources Policy, 39, 32–41.

    Google Scholar 

  • Kyrtsou, C., & Terraza, M. (2010). Seasonal Mackey–Glass–GARCH process and short-term dynamics. Empirical Economics, 38, 325–345.

    Google Scholar 

  • Lahmiri, S. (2016). A variational mode decomposition approach for analysis and forecasting of economic and financial time series. Expert Systems with Applications, 55, 268–273.

    Google Scholar 

  • Li, S. T., & Kuo, S. C. (2008). Knowledge discovery in financial investment for forecasting and trading strategy through wavelet-based SOM networks. Expert Systems with Applications, 34, 935–951.

    Google Scholar 

  • Li, C., & Shang, P. (2018). Complexity analysis based on generalized deviation for financial markets. Physica A, 494, 118–128.

    Google Scholar 

  • Li, S. T., & Shue, L. Y. (2001). Data mining to aid policy making in air pollution management. Expert Systems with Applications, 27, 331–340.

    Google Scholar 

  • Liang, X., & Lu, Y. (2017). Indifference pricing of a life insurance portfolio with risky asset driven by a shot-noise process. Insurance: Mathematics and Economics, 77, 119–132.

    Google Scholar 

  • Michis, A. A. (2014). Investing in gold: Individual asset risk in the long run. Finance Research Letters, 11, 369–374.

    Google Scholar 

  • Mihcak, M. K., Kozintsev, I., Ramchandran, K., & Moulin, P. (1999). Low-complexity image denoising based on statistical modeling of wavelet coefficients. IEEE Signal Processing Letters, 6(12), 300–303.

    Google Scholar 

  • Morlet, J., Arens, G., Fourgeau, E., & Giard, D. (1982a). Wave propagation and sampling theory, part I: Complex signal land scattering in multilayer media. Geophysics, 47(2), 203–221.

    Google Scholar 

  • Morlet, J., Arens, G., Fourgeau, E., & Giard, D. (1982b). Wave propagation and sampling theory, part II: Sampling theory and complex waves. Geophysics, 47(2), 222–236.

    Google Scholar 

  • Naccache, T. (2011). Oil price cycles and wavelets. Energy Economics, 33, 338–352.

    Google Scholar 

  • Neittaanmäki, P., Repin, S., & Tuovinen, T. (2016). Mathematical Modeling and optimization of complex structures. Basel, Switzerland: Springer.

    Google Scholar 

  • Oet, M. V., Gramlich, D., & Sarlin, P. (2016). Evaluating measures of adverse financial conditions. Journal of Financial Stability, 27, 234–249.

    Google Scholar 

  • Oh, K. J., & Kim, K. J. (2002). Analyzing stock market tick data using piecewise nonlinear model. Expert Systems with Applications, 22, 249–255.

    Google Scholar 

  • Page, L., & Siemroth, C. (2017). An experimental analysis of information acquisition in prediction markets. Games and Economic Behavior, 101, 354–378.

    Google Scholar 

  • Pal, D., & Mitra, S. K. (2017). Time-frequency contained co-movement of crude oil and world food prices: A wavelet-based analysis. Energy Economics, 62, 230–239.

    Google Scholar 

  • Power, G. J., Eaves, J., Turvey, C., & Vedenov, D. (2017). Catching the curl: Wavelet thresholding improves forward curve modeling. Economic Modelling, 64, 312–321.

    Google Scholar 

  • Ramiah, V., Xu, X., & Moosa, I. A. (2015). Neoclassical finance, behavioral finance and noise traders: A review and assessment of the literature. International Review of Financial Analysis, 41, 89–100.

    Google Scholar 

  • Ramsey, J. B., Mar, G., Mau, G., & Semmler, W. (2010). Instrumental variables and wavelet decompositions. Economic Modelling, 27, 1498–1513.

    Google Scholar 

  • Rashid, M. M., Mhaskar, P., & Swartz, C. L. E. (2016). Multi-rate modeling and economic model predictive control of the electric arc furnace. Journal of Process Control, 40, 50–61.

    Google Scholar 

  • Ruan, J., & Ma, T. (2017). Bid-ask spread, quoted depths, and unexpected duration between trades. Journal of Financial Services Research, 51, 385–436.

    Google Scholar 

  • Sehgal, N., & Pandey, K. K. (2015). Artificial intelligence methods for oil price forecasting: A review and evaluation. Energy Systems, 6, 479–506.

    Google Scholar 

  • Sezen, U. (2009). Perfect reconstruction IIR digital filter banks supporting nonexpansive linear signal extensions. IEEE Transactions on Signal Processing, 56(6), 2140–2150.

    Google Scholar 

  • Shahzad, S. J. H., Raza, N., Balcilar, M., Ali, S., & Shabaz, M. (2017). Can economic policy uncertainty and investors sentiment predict commodities returns and volatility? Resources Policy, 53, 208–218.

    Google Scholar 

  • Shin, T., & Han, I. (2000). Optimal signal multi-resolution by genetic algorithms to support artificial neural networks for exchange-rate forecasting. Expert Systems with Applications, 18, 257–269.

    Google Scholar 

  • Singh, L. P., & Challa, R. T. (2016). Integrated forecasting using the discrete wavelet theory and artificial intelligence techniques to reduce the bullwhip effect in a supply chain. Global Journal of Flexible Systems Management, 17(2), 157–169.

    Google Scholar 

  • Su, H., Liu, Q., & Li, J. S. (2012). Boundary effects reduction in wavelet transform for time-frequency analysis. WSEAS Transactions on Signal Processing, 8(4), 169–179.

    Google Scholar 

  • Sui, C., Wang, Z., & Ye, R. (2016). A novel consensus based prediction strategy for data sensing. Neurocomputing, 215, 175–183.

    Google Scholar 

  • Sun, E. W., Chen, Y. T., & Yu, M. T. (2015). Generalized optimal wavelet decomposing algorithm for big financial data. International Journal of Production Economics, 156, 194–214.

    Google Scholar 

  • Sun, E. W., & Meinl, T. (2012). A new wavelet-based denoising algorithm for high-frequency financial data mining. European Journal of Operational Research, 217, 589–599.

    Google Scholar 

  • Svarc, J., & Dabic, M. (2017). Evolution of the knowledge economy: A historical perspective with an application to the case of Europe. Journal of the Knowledge Economy, 8(1), 159–176.

    Google Scholar 

  • Taswell, C. (2000). The what, how, and why of wavelet shrinkage denoising (pp. 12–19). May/June: Computing in Science & Engineering.

    Google Scholar 

  • Tiwari, A. K. (2013). Oil prices and the macroeconomy reconsideration for Germany: Using continuous wavelet. Economic Modelling, 30, 636–642.

    Google Scholar 

  • Tiwari, A. K., & Kyophilavong, P. (2014). New evidence from the random walk hypothesis for BRICS stock indices: A wavelet unit root test approach. Economic Modelling, 43, 38–41.

    Google Scholar 

  • Tiwari, A. K., Mutascu, M. I., & Albulescu, C. T. (2016). Continuous wavelet transform and rolling correlation of European stock markets. International Review of Economics and Finance, 42, 237–256.

    Google Scholar 

  • Tzagkarakis, G., Caicedo-Llano, J., & Dionysopoulos, T. (2016). Time-frequency adapted market integration measure based on hough transformed multiscale decompositions. Computational Economics, 48, 1–27.

    Google Scholar 

  • Vacha, L., & Barunik, J. (2012). Co-movement of energy commodities revisited: Evidence from wavelet coherence analysis. Energy Economics, 34, 241–247.

    Google Scholar 

  • Vacha, L., Janda, K., Kristoufek, L., & Zilberman, D. (2013). Time–frequency dynamics of biofuel–fuel–food system. Energy Economics, 40, 233–241.

    Google Scholar 

  • Vaseghi, S. V. (2008). Advanced digital signal processing and noise reduction (4th ed.). Chichester: John Wiley & Sons.

    Google Scholar 

  • Wang, J. Z., Wang, J. J., Zhang, J. G., & Guo, S. P. (2011). Forecasting stock indices with back propagation neural network. Expert Systems with Applications, 38, 14346–14355.

    Google Scholar 

  • Wang, Y., Wei, G. W., & Yang, S. (2012). Iterative filtering decomposition based on local spectral evolution kernel. Journal of Scientific Computing, 50, 629–664.

    Google Scholar 

  • Wei, Y., Zhang, X., Shi, Y., Xia, L., Pan, S., Wu, J., et al. (2018). A review of data-driven approaches for prediction and classification of building energy consumption. Renewable and Sustainable Energy Reviews, 82, 1027–1047.

    Google Scholar 

  • Wu, Z. H., & Huang, N. E. (2009). Ensemble empirical model decomposition: A noise-assisted data analysis method. Advances in Adaptive Data Analysis, 1(01), 1–14.

    Google Scholar 

  • Yang, L., Cai, X. J., & Hamori, S. (2017). Does the crude oil price influence the exchange rates of oil-importing and oil-exporting countries differently? A wavelet coherence analysis. International Review of Economics and Finance, 49, 536–547.

    Google Scholar 

  • Yildirim-Karaman, S. (2018). Uncertainty in financial markets and business cycles. Economic Modelling, 68, 329–339.

    Google Scholar 

  • Yousefi, S., Weinreich, I., & Reinarz, D. (2005). Wavelet-based prediction of oil prices. Chaos, Solitons & Fractals, 25, 265–275.

    Google Scholar 

  • Yu, L., Zhao, Y., & Tang, L. (2014). A compressed sensing based AI learning paradigm for crude oil price forecasting. Energy Economics, 46, 236–245.

    Google Scholar 

  • Zografidou, E., Petridis, K., Petridis, N. E., & Arabatzis, G. (2017). A financial approach to renewable energy production in Greece using goal programming. Renewable Energy, 108, 37–51.

    Google Scholar 

  • Zu, Y., & Boswijk, H. P. (2014). Estimating spot volatility with high-frequency financial data. Journal of Econometrics, 181, 117–135.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Fan He.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

He, F., He, X. A Continuous Differentiable Wavelet Shrinkage Function for Economic Data Denoising. Comput Econ 54, 729–761 (2019). https://doi.org/10.1007/s10614-018-9849-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10614-018-9849-y

Keywords

  • Economic (financial) time series
  • Wavelet shrinkage denoising
  • Thresholding function
  • Threshold rule
  • Continuously differentiable