Skip to main content
SpringerLink
Log in

Identification of Hammerstein Systems with Random Fourier Features and Kernel Risk Sensitive Loss

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Identification of Hammerstein systems with polynomial features and mean square error loss has received a lot of attention due to their simplicity in calculation and solid theoretical foundation. However, when the prior information of nonlinear subblock of a Hammerstein system is unknown or some outliers are involved, the performance of related methods may degenerate seriously. The main reason is that the used polynomial just has finite approximation capability to an unknown nonlinear function, and mean square error loss is sensitive to outliers. In this paper, a new identification method based on random Fourier features and kernel risk sensitive loss is therefore proposed. Since the linear combination of random Fourier features can well approximate any continuous nonlinear function, it is expected to be more powerful to characterize the nonlinear behavior of Hammerstein systems. Moreover, since the kernel risk sensitive loss is a similarity measure that is insensitive to outliers, it is expected to have excellent robustness. Based on the mean square convergence analysis, a sufficient condition to ensure the convergence and some theoretical values regarding the steady-state excess mean square error of the proposed method are also provided. Simulation results on the tasks of Hammerstein system identification and electroencephalogram noise removal show that the new method can outperform other popular and competitive methods in terms of accuracy and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. The manner to generate noise-contaminated samples can refer to Sect. 5.1. The only difference is that the ratio of outliers has been changed to \(p=0.1\).

  2. https://www.cs.colostate.edu/eeg/main/data/1989_Keirn_and_Aunon.

References

  1. Pawlak M, Lv J (2022) Nonparametric testing for Hammerstein systems. IEEE Trans Autom Control. https://doi.org/10.1109/TAC.2022.3171389

    Article  MathSciNet  MATH  Google Scholar 

  2. Rahati Belabad A, Sharifian S, Motamedi SA (2018) An accurate digital baseband predistorter design for linearization of RF power amplifiers by a genetic algorithm based Hammerstein structure. Analog Integr Circuits Process 95(2):231–247

    MATH  Google Scholar 

  3. Jurado F (2006) A method for the identification of solid oxide fuel cells using a Hammerstein model. J Power Sources 154(1):145–152

    Google Scholar 

  4. Capobianco E (2002) Hammerstein system represention of financial volatility processes. Eur Phys J B-Condens Matter Complex Syst 27(2):201–211

    Google Scholar 

  5. Liu Z, Li C (2022) Adaptive Hammerstein filtering via recursive non-convex projection. IEEE Trans Signal Process. https://doi.org/10.1109/TSP.2022.3180195

    Article  MathSciNet  Google Scholar 

  6. Umoh IJ, Ogunfunmi T (2010) An affine projection-based algorithm for identification of nonlinear Hammerstein systems. Signal Process 90(6):2020–2030

    MATH  Google Scholar 

  7. Jeraj J, Mathews VJ (2006) A stable adaptive Hammerstein filter employing partial orthogonalization of the input signals. IEEE Trans Signal Process 54(4):1412–1420

    MATH  Google Scholar 

  8. Wang D, Zhang S, Gan M, Qiu J (2020) A novel EM identification method for Hammerstein systems with missing output data. IEEE Trans Ind Inform 16(4):2500–2508

    Google Scholar 

  9. Wang D, Fan Q, Ma Y (2020) An interactive maximum likelihood estimation method for multivariable Hammerstein systems. J Frankl Inst 357(17):12986–13005

    MathSciNet  MATH  Google Scholar 

  10. Greblicki W, Pawlak M (2019) The weighted nearest neighbor estimate for Hammerstein system identification. IEEE Trans Autom Control 64(4):1550–1565

    MathSciNet  MATH  Google Scholar 

  11. Chang W (2022) Identification of nonlinear discrete systems using a new Hammerstein model with Volterra neural network. Soft Comput. https://doi.org/10.1007/s00500-022-07089-6

    Article  Google Scholar 

  12. Wang H, Chen Y (2020) Parameter estimation for dual-rate sampled Hammerstein systems with dead-zone nonlinearity. J Syst Eng Electron 31(1):185–193

  13. Khalifa TR, El-Nagar AM, El-Brawany MA, El-Araby EAG, El-Bardini M (2021) A novel Hammerstein model for nonlinear networked systems based on an interval type-2 fuzzy Takagi–Sugeno–Kang system. IEEE Trans Fuzzy Syst 29(2):275–285

    Google Scholar 

  14. Cheng S, Wei Y, Sheng D, Wang Y (2019) Identification for Hammerstein nonlinear systems based on universal spline fractional order LMS algorithm. Commun Nonlinear Sci Numer Simul 79:104901

    MathSciNet  MATH  Google Scholar 

  15. Cui M, Liu H, Li Z, Tang Y, Guan X (2014) Identification of Hammerstein model using functional link artificial neural network. Neurocomputing 142:419–428

    Google Scholar 

  16. Tang Y, Bu C, Liu M, Zhang L, Lian Q (2018) Application of ELM-Hammerstein model to the identification of solid oxide fuel cells. Neural Comput Appl 29(2):401–411

    Google Scholar 

  17. Scarpiniti M, Comminiello D, Parisi R, Uncini A (2014) Hammerstein uniform cubic spline adaptive filters: learning and convergence properties. Signal Process 100:112–123

    Google Scholar 

  18. Liu C, Zhang Z, Tang X (2019) Sign normalised Hammerstein spline adaptive filtering algorithm in an impulsive noise environment. Neural Process Lett 50(1):477–496

    Google Scholar 

  19. Risuleo RS, Bottegal G, Hjalmarsson H (2017) A nonparametric kernel-based approach to Hammerstein system identification. Automatica 85:234–247

    MathSciNet  MATH  Google Scholar 

  20. Mu B, Chen HF, Wang LY, Yin G, Zheng WX (2017) Recursive identification of Hammerstein systems: convergence rate and asymptotic normality. IEEE Trans Autom Control 62(7):3277–3292

    MathSciNet  MATH  Google Scholar 

  21. Zheng Y, Dong J, Ma W, Chen B (2018) Kernel adaptive Hammerstein filter. In: 26th European Signal Processing Conference, pp 504–508

  22. Van Vaerenbergh S, Azpicueta-Ruiz LA (2014) Kernel-based identification of Hammerstein systems for nonlinear acoustic echo-cancellation. In: 2014 IEEE international conference on acoustics, speech and signal processing, pp 3739–3743

  23. Risuleo RS, Bottegal G, Hjalmarsson H (2015) A new kernel-based approach to overparameterized Hammerstein system identification. In: 2015 54th IEEE conference on decision and control, pp 115–120

  24. Chen M, Xu Z, Zhao J, Zhu Y, Shao Z (2022) Nonparametric identification of batch process using two-dimensional kernel-based Gaussian process regression. Chem Eng Sci 250:117372

    Google Scholar 

  25. Castro-Garcia R, Agudelo OM, Suykens JAK (2019) Impulse response constrained LS-SVM modelling for MIMO Hammerstein system identification. Int J Control 92(4):908–925

    MathSciNet  MATH  Google Scholar 

  26. Ma L, Liu X (2017) A novel APSO-aided weighted LSSVM method for nonlinear Hammerstein system identification. J Frankl Inst 354(4):1892–1906

    MathSciNet  MATH  Google Scholar 

  27. Micchelli CA, Xu Y, Zhang H (2006) Universal kernels. J Mach Learn Res 7:2651–2667

    MathSciNet  MATH  Google Scholar 

  28. Richard C, Bermudez JCM, Honeine P (2009) Online prediction of time series data with kernels. IEEE Trans Signal Process 57(3):1058–1067

    MathSciNet  MATH  Google Scholar 

  29. Liu W, Príncipe JC, Haykin S (2010) Kernel adaptive filtering: a comprehensive introduction. John Wiley and Sons, New York

    Google Scholar 

  30. Coelho DN, Barreto GA (2022) A sparse online approach for streaming data classification via prototype-based kernel models. Neural Process Lett 54(3):1679–1706

    Google Scholar 

  31. Mitra R, Miramirkhani F, Bhatia V, Uysal M (2019) Mixture-kernel based post-distortion in RKHS for time-varying VLC channels. IEEE Trans Veh Technol 68(2):1564–1577

    Google Scholar 

  32. Mitra R, Bhatia V (2016) Adaptive sparse dictionary-based kernel minimum symbol error rate post-distortion for nonlinear LEDs in visible light communications. IEEE Photon J 8(4):1–13

    Google Scholar 

  33. Chen B, Zhao S, Zhu P, Principe JC (2012) Quantized kernel least mean square algorithm. IEEE Trans Neural Netw Learn Syst 23(1):22–32

    Google Scholar 

  34. Zheng Y, Wang S, Feng J, Tse CK (2016) A modified quantized kernel least mean square algorithm for prediction of chaotic time series. Digit Signal Prog 48:130–136

    MathSciNet  Google Scholar 

  35. Singh A, Ahuja N, Moulin P (2012) Online learning with kernels: overcoming the growing sum problem. In: 2012 IEEE international workshop on machine learning for signal processing, pp 1–6

  36. Drineas P, Mahoney MW, Cristianini N (2005) On the Nyström method for approximating a Gram matrix for improved kernel-based learning. J Mach Learn Res 6(12):2153–2175

    MathSciNet  MATH  Google Scholar 

  37. Zhang Q, Shi W, Hoi S, Xu Z (2022) Non-uniform Nyström approximation for sparse kernel regression: theoretical analysis and experimental evaluation. Neurocomputing. https://doi.org/10.1016/j.neucom.2022.05.112

    Article  Google Scholar 

  38. Sun S, Zhao J, Zhu J (2015) A review of Nyström methods for large-scale machine learning. Inf Fus 26:36–48

    Google Scholar 

  39. Rahimi A, Recht B (2007) Random features for large-scale kernel machines. In: 20th International conference on neural information processing systems. Curran Associates Inc., 2981710, pp 1177–1184

  40. Bliek L, Verstraete HRGW, Verhaegen M, Wahls S (2018) Online optimization with costly and noisy measurements using random Fourier expansions. IEEE Trans Neural Netw Learn Syst 29(1):167–182

    MathSciNet  Google Scholar 

  41. Bouboulis P, Chouvardas S, Theodoridis S (2018) Online distributed learning over networks in RKH spaces using random fourier features. IEEE Trans Signal Process 66(7):1920–1932

    MathSciNet  MATH  Google Scholar 

  42. Elias VRM, Gogineni VC, Martins WA, Werner S (2022) Kernel regression over graphs using random Fourier features. IEEE Trans Signal Process 70:936–949

    MathSciNet  Google Scholar 

  43. Shen M, Xiong K, Wang S (2020) Multikernel adaptive filtering based on random features approximation. Signal Process 176:107712

    Google Scholar 

  44. Wu Z, Peng S, Chen B, Zhao H (2015) Robust Hammerstein adaptive filtering under maximum correntropy criterion. Entropy 17(10):7149–7166

    MathSciNet  Google Scholar 

  45. Qian G, Luo D, Wang S (2019) A robust adaptive filter for a complex Hammerstein system. Entropy 21(2):162

    MathSciNet  Google Scholar 

  46. Guan S, Li Z (2017) Normalised spline adaptive filtering algorithm for nonlinear system identification. Neural Process Lett 46(2):595–607

    Google Scholar 

  47. Zheng Y, Chen B, Wang S, Wang W (2021) Broad learning system based on maximum correntropy criterion. IEEE Trans Neural Netw Learn Syst 32(7):3083–3097

    MathSciNet  Google Scholar 

  48. Chen B, Xing L, Xu B, Zhao H, Zheng N, Príncipe JC (2017) Kernel risk-sensitive loss: definition, properties and application to robust adaptive filtering. IEEE Trans Signal Process 65(11):2888–2901

    MathSciNet  MATH  Google Scholar 

  49. Qian G, Dong F, Wang S (2020) Robust constrained minimum mixture kernel risk-sensitive loss algorithm for adaptive filtering. Digit Signal Prog 107:102859

    Google Scholar 

  50. Ma W, Kou X, Hu X, Qi A, Chen B (2022) Recursive minimum kernel risk sensitive loss algorithm with adaptive gain factor for robust power system s estimation. Electr Power Syst Res 206:107788

    Google Scholar 

  51. Luo X, Deng J, Wang W, Wang J, Zhao W (2017) A quantized kernel learning algorithm using a minimum kernel risk-sensitive loss criterion and bilateral gradient technique. Entropy 19(7):365

    Google Scholar 

  52. Luo X, Li Y, Wang W, Ban X, Wang J, Zhao W (2020) A robust multilayer extreme learning machine using kernel risk-sensitive loss criterion. Int J Mach Learn Cybern 11(1):197–216

    Google Scholar 

  53. Wang W, Zhao H, Lu L, Yu Y (2019) Robust nonlinear adaptive filter based on kernel risk-sensitive loss for bilinear forms. Circuits Syst Signal Process 38(4):1876–1888

    MathSciNet  Google Scholar 

  54. Ren L, Liu J, Gao Y, Kong X, Zheng C (2021) Kernel risk-sensitive loss based hyper-graph regularized robust extreme learning machine and its semi-supervised extension for classification. Knowledge-Based Syst 227:107226

    Google Scholar 

  55. Schölkopf B, Herbrich R, Smola AJ (2001) A generalized representer theorem. In: 14th Annual Conference on Computational Learning Theory, pp 416–426

  56. Liu W, Pokharel PP, Principe JC (2007) Correntropy: properties and applications in non-Gaussian signal processing. IEEE Trans Signal Process 55(11):5286–5298

  57. Ma W, Duan J, Zhao H, Chen B (2018) Chebyshev functional link artificial neural network based on correntropy induced metric. Neural Process Lett 47(1):233–252

    Google Scholar 

  58. Stenger A, Kellermann W (2000) Adaptation of a memoryless preprocessor for nonlinear acoustic echo cancelling. Signal Process 80(9):1747–1760

    MATH  Google Scholar 

  59. Sayed AH (2008) Adaptive Filters. Wiley, Hoboken

  60. Lin B, He R, Wang X, Wang B (2008) The excess mean-square error analyses for Bussgang algorithm. IEEE Signal Process Lett 15:793–796

    Google Scholar 

  61. Saeidi M, Karwowski W, Farahani FV, Fiok K, Taiar R, Hancock PA, Al-Juaid A (2021) Neural decoding of EEG signals with machine learning: a systematic review. Brain Sci 11:1525

    Google Scholar 

  62. Tiwari N, Edla DR, Dodia S, Bablani A (2018) Brain computer interface: a comprehensive survey. Biol Inspired Cogn Archit 26:118–129

    Google Scholar 

  63. Mumtaz W, Rasheed S, Irfan A (2021) Review of challenges associated with the EEG artifact removal methods. Biomed Signal Process Control 68:102741

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62071391, and in part by the Natural Science Foundation of Chongqing under Grant cstc2020jcyj-msxmX0234.

Author information

Authors and Affiliations

  1. College of Electronic and Information Engineering, Southwest University, Chongqing, 400715, China

    Yunfei Zheng & Shiyuan Wang

  2. Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University, Xi’an, 710049, China

    Badong Chen

Authors
  1. Yunfei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shiyuan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Badong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YZ: preparation, creation of the work, and writing the initial manuscript. SW: creation of the work by those from the original research group, and supervision of the work. BC: supervision of the work, and revision of the manuscript.

Corresponding author

Correspondence to Shiyuan Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, Y., Wang, S. & Chen, B. Identification of Hammerstein Systems with Random Fourier Features and Kernel Risk Sensitive Loss. Neural Process Lett 55, 9041–9063 (2023). https://doi.org/10.1007/s11063-023-11191-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11191-7

Keywords

Search

Navigation