Skip to main content
SpringerLink
Log in

Time series prediction using kernel adaptive filter with least mean absolute third loss function

  • Original Paper
  • Published:
Nonlinear Dynamics Aims and scope Submit manuscript

Abstract

In this paper, a novel kernel adaptive filter, based on the least mean absolute third (LMAT) loss function, is proposed for time series prediction in various noise environments. Combining the benefits of the kernel method and the LMAT loss function, the proposed KLMAT algorithm performs robustly against noises with different probability densities. However, an important limitation of the KLMAT algorithm is a trade-off between the convergence rate and steady-state prediction error imposed by the selection of a certain value for the learning rate. Therefore, a variable learning rate version (VLR–KLMAT algorithm) is also proposed based on a Lorentzian function. We analyze the stability and convergence behavior of the KLMAT algorithm and derive a sufficient condition to predict its learning rate behavior. Moreover, a kernel recursive extension of the KLMAT algorithm is further proposed for performance improvement. Simulation results in the context of time series prediction verify the effectiveness of the proposed algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Aboulnasr, T., Mayyas, K.: A robust variable step-size LMS-type algorithm: analysis and simulations. IEEE Trans. Signal Process. 45(3), 631–639 (1997)

    Article  Google Scholar 

  2. Arqub, O.A.: Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations. Neural Comput. Appl. (2015). doi:10.1007/s00521-015-2110-x

  3. Arqub, O.A., Al-Smadi, M., Momani, S., Hayat, T.: Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput. (2016). doi:10.1007/s00500-016-2262-3

    MATH  Google Scholar 

  4. Arqub, O.A., Al-Smadi, M., Momani, S., Hayat, T.: Numerical solutions of fuzzy differential equations using reproducing kernel Hilbert space method. Soft Comput. 20, 3283–3302 (2016)

    Article  MATH  Google Scholar 

  5. Bershad, N.J., Bermudez, J.C.M.: Stochastic analysis of the Least Mean Kurtosis algorithm for Gaussian inputs. Digit. Signal Process. 54, 35–45 (2016)

    Article  Google Scholar 

  6. Black, M.J., Rangarajan, A.: The outlier process: unifying line processes and robust statistics. In: IEEE CS Conference on Computer Vision and Pattern Recognition (CVPR’94), pp. 15–22 (1994)

  7. Cao, Z., Yu, S., Principe, J.C., Xu, G., Chen, B.: Multiple adaptive kernel size KLMS for Beijing PM2.5 prediction. In: International Joint Conference on Neural Networks, pp. 1403–1407 (2016)

  8. Chambers, J.A., Avlonitis, A.: A robust mixed-norm adaptive filter algorithm. IEEE Signal Process. Lett. 4(2), 46–48 (1997)

    Article  Google Scholar 

  9. Chambers, J.A., Tanrikulu, O., Constantinides, A.G.: Least mean mixed-norm adaptive filtering. Electron. Lett. 30(19), 1574–1575 (1994)

    Article  Google Scholar 

  10. Chen, B., Yang, X., Ji, H., Qu, H., Zheng, N., Principe, J.C.: Trimmed affine projection algorithms. In: International Joint Conference on Neural Networks (IJCNN), pp. 1923–1928 (2014)

  11. Chen, B., Yuan, Z., Zheng, N., Principe, J.C.: Kernel minimum error entropy algorithm. Neurocomputing 121, 160–169 (2013)

    Article  Google Scholar 

  12. Chen, B., Zhao, S., Zhu, P., Principe, J.C.: Mean square convergence analysis for kernel least mean square algorithm. Signal Process. 92(11), 2624–2632 (2012)

    Article  Google Scholar 

  13. Chen, B., Zhao, S., Zhu, P., Principe, J.C.: Quantized kernel least mean square algorithm. IEEE Trans. Neural Netw. Learn. Syst. 23(1), 22–32 (2012)

    Article  Google Scholar 

  14. Chen, B., Zhao, S., Zhu, P., Principe, J.C.: Quantized kernel recursive least squares algorithm. IEEE Trans. Neural Netw. Learn. Syst. 24(9), 1484–1491 (2013)

    Article  Google Scholar 

  15. Chouvardas, S., Draief, M.: A diffusion kernel LMS algorithm for nonlinear adaptive networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4164–4168 (2016)

  16. Costa, M.H., Bermudez, J.C.M.: A noise resilient variable step-size LMS algorithm. Signal Process. 88(3), 733–748 (2008)

    Article  MATH  Google Scholar 

  17. Engel, Y., Mannor, S., Meir, R.: The kernel recursive least-squares algorithm. IEEE Trans. Signal Process. 52(8), 2275–2285 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  18. Eweda, E., Bershad, N.J.: Stochastic analysis of a stable normalized least mean fourth algorithm for adaptive noise canceling with a white Gaussian reference. IEEE Trans. Signal Process. 60(12), 6235–6244 (2012)

    Article  MathSciNet  Google Scholar 

  19. García, J.A., Vidal, A.R.F., Sayed, A.H.: Mean-square performance of a convex combination of two adaptive filters. IEEE Trans Signal Process. 54(3), 1078–1090 (2006)

    Article  MATH  Google Scholar 

  20. Gil-Cacho, J.M., Signoretto, M., van Waterschoot, T., Moonen, M., Jensen, S.H.: Nonlinear acoustic echo cancellation based on a sliding-window leaky kernel affine projection algorithm. IEEE Trans. Audio Speech Lang. Process. 21(9), 1867–1878 (2013)

    Article  Google Scholar 

  21. Huang, B., Xiao, Y., Ma, Y., Wei, G.G., Sun, J.: A simplified variable step-size LMS algorithm for Fourier analysis and its statistical properties. Signal Process. 117, 69–81 (2015)

    Article  Google Scholar 

  22. Koike, S.I.: Convergence analysis of adaptive filters using normalized sign-sign algorithm. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. 88(11), 3218–3224 (2005)

    Article  Google Scholar 

  23. Lee, Y.H., Mok, J.D., Kim, S.D., Cho, S.H.: Performance of least mean absolute third (LMAT) adaptive algorithm in various noise environments. Electron. Lett. 34(3), 241–243 (1998)

    Article  Google Scholar 

  24. Li, C., Shen, P., Liu, Y., Zhang, Z.: Diffusion information theoretic learning for distributed estimation over network. IEEE Trans. Signal Process. 61(16), 4011–4024 (2013)

    Article  MathSciNet  Google Scholar 

  25. Liu, Z., Hu, Q., Cui, Y., Zhang, Q.: A new detection approach of transient disturbances combining wavelet packet and Tsallis entropy. Neurocomputing 142, 393–407 (2014)

    Article  Google Scholar 

  26. Liu, W., Park, Il, Wang, Y., Principe, J.C.: Extended kernel recursive least squares algorithm. IEEE Trans. Signal Process. 57(10), 3801–3814 (2009)

    Article  MathSciNet  Google Scholar 

  27. Liu, W., Pokharel, P., Principe, J.C.: The kernel least mean square algorithm. IEEE Trans. Signal Process. 56(2), 543–554 (2008)

    Article  MathSciNet  Google Scholar 

  28. Liu, W., Principe, J.C.: Kernel affine projection algorithms. EURASIP J. Adv. Signal Process. 2008(1), 1–13 (2008)

    Article  MATH  Google Scholar 

  29. Liu, W., Principe, J.C., Haykin, S.: Kernel Adaptive Filtering: A Comprehensive Introduction. Wiley, Hoboken, NJ (2010)

    Book  Google Scholar 

  30. Liu, J., Qu, H., Chen, B., Ma, W.: Kernel robust mixed-norm adaptive filtering. In: International Joint Conference on Neural Networks (IJCNN), pp. 3021–3024 (2014)

  31. Lu, L., Zhao, H.: Adaptive Volterra filter with continuous lp-norm using a logarithmic cost for nonlinear active noise control. J. Sound Vib. 364, 14–29 (2016)

    Article  Google Scholar 

  32. Lu, L., Zhao, H., Chen, B.: Collaborative adaptive Volterra filters for nonlinear system identification in \(\alpha \)-stable noise environments. J. Frankl. Inst. 353(17), 4500–4525 (2016)

    Article  MATH  MathSciNet  Google Scholar 

  33. Lu, L., Zhao, H., Chen, B.: Improved-variable-forgetting-factor recursive algorithm based on the logarithmic cost for Volterra system identification. IEEE Trans. Circuits Syst. II 63(6), 588–592 (2016)

    Article  Google Scholar 

  34. Lu, L., Zhao, H., Chen, B.: KLMAT: A Kernel Least Mean Absolute Third Algorithm (2016). arXiv:1603.03564

  35. Lu, L., Zhao, H., Chen, B.: Variable-Mixing Parameter Quantized Kernel Robust Mixed-Norm Algorithms for Combating Impulsive Interference (2015). arXiv:1508.05232

  36. Lu, L., Zhao, H., Li, K., Chen, B.: A novel normalized sign algorithm for system identification under impulsive noise interference. Circuits Syst. Signal Process. 35(9), 3244–3265 (2015)

    Article  MATH  MathSciNet  Google Scholar 

  37. Ma, W., Duan, J., Man, W., Zhao, H., Chen, B.: Robust kernel adaptive filters based on mean \(p\)-power error for noisy chaotic time series prediction. Eng. Appl. Artif. Intell. 58, 101–110 (2017)

    Article  Google Scholar 

  38. Machado, J.A.T.: Entropy analysis of integer and fractional dynamical systems. Nonlinear Dyn. 62(1), 371–378 (2010)

    MATH  MathSciNet  Google Scholar 

  39. Masry, E., Bullo, F.: Convergence analysis of the sign algorithm for adaptive filtering. IEEE Trans. Inf. Theory 41(2), 489–495 (1995)

    Article  MATH  Google Scholar 

  40. Mathews, V.J., Cho, S.H.: Improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm. IEEE Trans. Acoust. Speech Signal Process. 35(4), 450–454 (1987)

    Article  MATH  Google Scholar 

  41. Miao, Q.Y., Li, C.G.: Kernel least-mean mixed-norm algorithm. In: International Conference on Automatic Control and Artificial Intelligence (ACAI), Mar. 3–5, pp. 1285–1288 (2012)

  42. Mitra, R., Bhatia, V.: Diffusion-KLMS Algorithm and Its Performance Analysis for Non-linear Distributed Networks (2015). arXiv:1509.01352

  43. Papoulis, E.V., Stathaki, T.: A normalized robust mixed-norm adaptive algorithm for system identification. IEEE Signal Process. Lett. 11(1), 56–59 (2004)

    Article  Google Scholar 

  44. Parreira, W.D., Bermudez, J.C.M., Richard, C., Tourneret, J.Y.: Stochastic behavior analysis of the Gaussian kernel least-mean-square algorithm. IEEE Trans. Signal Process. 60(5), 2208–2222 (2012)

    Article  MathSciNet  Google Scholar 

  45. Pei, S.C., Tseng, C.C.: Least mean \(p\)-power error criterion for adaptive FIR filter. IEEE J. Sel. Areas Commun. 12(9), 1540–1547 (1994)

    Article  Google Scholar 

  46. Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)

    Article  Google Scholar 

  47. Tian, J., Gu, H., Gao, C., Lian, J.: Local density one-class support vector machines for anomaly detection. Nonlinear Dyn. 64(1–2), 127–130 (2011)

    Article  Google Scholar 

  48. Walach, E., Widrow, B.: The least mean fourth (LMF) adaptive algorithm and its family. IEEE Trans. Inf. Theory 30(2), 275–283 (1984)

    Article  Google Scholar 

  49. Wang, S., Feng, J., Tse, C.K.: Kernel affine projection sign algorithms for combating impulse interference. IEEE Trans. Circuits Syst. II 60(11), 811–815 (2013)

    Article  Google Scholar 

  50. Wang, S., Zheng, Y., Duan, S., Wang, L., Chi, K.T.: A class of improved least sum of exponentials algorithms. Signal Process. 128, 340–349 (2016)

    Article  Google Scholar 

  51. Wu, P., Duan, F., Guo, P.: A pre-selecting base kernel method in multiple kernel learning. Neurocomputing 165, 46–53 (2015)

    Article  Google Scholar 

  52. Wu, Z., Shi, J., Zhang, X., Ma, W., Chen, B.: Kernel recursive maximum correntropy. Signal Process. 117, 11–16 (2015)

    Article  Google Scholar 

  53. Yang, J., Guan, J., Ning, Y., Liu, Q.: A novel adaptive harmonic detecting algorithm applied to active power filters. In: Proceedings of the 3rd International Congress of Image Signal Processing Oct. 16–18, vol. 7, pp. 3287–3290 (2010)

  54. Yang, X., Qu, H., Zhao, J., Chen, B.: Hybrid affine projection algorithm. In: 13th International Conference on Control Automation Robotics and Vision (ICARCV), pp. 964–968 (2014)

  55. Yazdi, H.S., Pakdaman, M., Modaghegh, H.: Unsupervised kernel least mean square algorithm for solving ordinary differential equations. Neurocomputing 74(12), 2062–2071 (2011)

    Article  Google Scholar 

  56. Yu, Y., Zhao, H.: An improved variable step-size NLMS algorithm based on a Versiera function. In: IEEE International Conference on Signal Processing, Communications and Computing, Aug. 5–8, pp. 1–4 (2013)

  57. Zhang, H., Yang, T., Xu, W., Xu, Y.: Effects of non-Gaussian noise on logical stochastic resonance in a triple-well potential system. Nonlinear Dyn. 76(1), 649–656 (2014)

    Article  MATH  MathSciNet  Google Scholar 

  58. Zhao, S., Chen, B., Cao, Z., Zhu, P., Principe, J.C.: Self-organizing kernel adaptive filtering. EURASIP J. Adv. Signal Process. 2016(1), 106 (2016)

    Article  Google Scholar 

  59. Zhao, S., Chen, B., Principe, J.C.: Kernel adaptive filtering with maximum correntropy criterion. In: International Joint Conference on Neural Networks (IJCNN), pp. 2012–2017 (2011)

  60. Zhao, S., Chen, B., Zhu, P., Principe, J.C.: Fixed budget quantized kernel least-mean-square algorithm. Signal Process. 93(9), 2759–2770 (2013)

    Article  Google Scholar 

  61. Zhao, H., Gao, S., He, Z., Zeng, X., Jin, W., Li, T.: Identification of nonlinear dynamic system using a novel recurrent wavelet neural network-based the pipelined architecture. IEEE Trans. Ind. Electron. 61(8), 4171–4182 (2014)

    Article  Google Scholar 

  62. Zhao, S., Man, Z., Khoo, S., Wu, H.R.: Variable step-size LMS algorithm with a quotient form. Signal Process. 89(1), 67–76 (2009)

    Article  MATH  Google Scholar 

  63. Zhao, X., Shang, P.: Principal component analysis for non-stationary time series based on detrended cross-correlation analysis. Nonlinear Dyn. 84(2), 1033–1044 (2016)

    Article  MATH  MathSciNet  Google Scholar 

  64. Zhao, J., Wang, T., Ma, W., Qu, H.: The kernel-LMS based network traffic prediction. In: Information Science and Control Engineering (ICISCE 2012), pp. 1–5 (2012)

  65. Zhao, H., Yu, Y., Gao, S., Zeng, X., He, Z.: A new normalized LMAT algorithm and its performance analysis. Signal Process. 105, 399–409 (2014)

    Article  Google Scholar 

  66. Zhao, J., Zhang, G.: A robust Prony method against synchrophasor measurement noise and outliers. IEEE Trans. Power Syst. (2016). doi:10.1109/TPWRS.2016.2612887

    Google Scholar 

  67. Zhou, Y., Chan, S.C., Ho, K.L.: New sequential partial-update least mean M-estimate algorithms for robust adaptive system identification in impulsive noise. IEEE Trans. Ind. Electron. 58(9), 4455–4470 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

The authors want to express their deep thanks to the anonymous reviewers for many valuable comments which greatly helped to improve the quality of this work. This work was partially supported by National Science Foundation of P.R. China (Grant: 61571374, 61271340, 61433011). The first author would like to acknowledge the China Scholarship Council (CSC) for providing him with financial support to study abroad (No. 201607000050).

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Southwest Jiaotong University, Chengdu, China

    Lu Lu & Haiquan Zhao

  2. School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, China

    Badong Chen

Authors
  1. Lu Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haiquan Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Badong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiquan Zhao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, L., Zhao, H. & Chen, B. Time series prediction using kernel adaptive filter with least mean absolute third loss function. Nonlinear Dyn 90, 999–1013 (2017). https://doi.org/10.1007/s11071-017-3707-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11071-017-3707-7

Keywords

Search

Navigation