Improved Optimum Error Nonlinearities Using Cramer–Rao Bound Estimation

  • Muhammad Arif
  • Imran NaseemEmail author
  • Muhammad Moinuddin
  • Muhammad Nasir Iqbal


In this paper, we propose an efficient design of optimum error nonlinearities (OENL) for adaptive filters which minimizes the steady-state excess mean square error and attains the limit mandated by the Cramer–Rao bound (CRB) of the underlying estimation process. Novelty of the work resides in the fact that the proposed improved optimum error nonlinearities (IOENL) design incorporates the effect of CRB which was ignored in the existing literature. To achieve this, we employ two efficient methods to estimate the variance of a priori estimation error. Therefore, the proposed IOENL does not use any assumption on the distribution of input regressor elements and noise sequence. Neither the assumption of independence on the input regressor is made nor any sort of linearization is assumed. Extensive simulations are done to show the efficiency of the proposed algorithm compared to the standard least mean square algorithm and the standard OENL algorithm.


Adaptive filtering Optimum nonlinearity Channel estimation CRLB 



This project was funded by the Center of Excellence in Intelligent Engineering Systems (CEIES), King Abdulaziz University, under grant No. (CEIES-18-04-01). The authors, therefore, acknowledge the technical and financial support of CEIES. The authors also acknowledge the support of Karachi Institute of Economics and Technology (PAF-KIET) Pakistan in facilitating this research.


  1. 1.
    T. Aboulnasr, K. Mayyas, A robust variable step-size LMS type algorithm: analysis and simulation. IEEE Trans. Signal Process. 45, 631–639 (1997)CrossRefGoogle Scholar
  2. 2.
    T. Aboulnasr, A. Zerguine, Variable weight mixed-norm LMS-LMF adaptive algorithm. IEEE Conf. Signals Syst. Comput. 1, 791–794 (1999)Google Scholar
  3. 3.
    T.Y. Al-Naffouri, A.H. Sayed, Optimum error nonlinearities for long adaptive filters, in IEEE International Conference on Acoustic, Speech, and Signal Processing (ICASSP),Orlando, USA (2002)Google Scholar
  4. 4.
    T.Y. Al-Naffouri, A.H. Sayed, Transient analysis of adaptive filters with error nonlinearities. IEEE Trans. Signal Process. 51(3), 653–663 (2003)CrossRefGoogle Scholar
  5. 5.
    T.Y. Al-Naffouri, A.H. Sayed, Transient analysis of data-normalized adaptive filters. IEEE Trans. Signal Process. 51, 639–652 (2003)CrossRefGoogle Scholar
  6. 6.
    T.Y. Al-Naffouri, A.H. Sayed, T. Kailath, On the selection of optimal nonlinearities for stochastic gradient adaptive algorithms, in IEEE International Conference on Acoustic, Speech, and Signal Processing (ICASSP), Istanbul, Turkey, vol 1 (2000), pp. 464–467Google Scholar
  7. 7.
    M. Arif, I. Naseem, M. Moinuddin, U.M. Al-Saggaf, Design of optimum error nonlinearity for channel estimation in the presence of class-a impulsive noise, in 6th International Conference on Intelligent and Advance System (ICIAS2016) Google Scholar
  8. 8.
    N.J. Bershad, On error-saturation nonlinearities in LMS adaptation. IEEE Trans. Acoust. Speech Signal Process. 36(4), 440–452 (1988)CrossRefzbMATHGoogle Scholar
  9. 9.
    N.J. Bershad, M. Bonnet, Saturation effects in LMS adaptive echo cancellation for binary data. IEEE Trans. Acoust. Speech Signal Process. 38, 1687–1696 (1990)CrossRefGoogle Scholar
  10. 10.
    P. Bouboulis, S. Theodoridis, The complex Gaussian Kernel LMS algorithm, in International Conference on Artificial Neural Networks (ICANN) (2010), pp. 11–20Google Scholar
  11. 11.
    P. Bouboulis, S. Theodoridis, Extension of Wirtingers calculus to reproducing kernel Hilbert spaces and the complex kernel LMS. IEEE Trans. Signal Process. 59(3), 964–978 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    J.A. Chambers, O. Tanrikulu, A.G. Constantinides, Least mean mixed-norm adaptive filtering. Electron. Lett. 30(19), 1574–1575 (1994)CrossRefGoogle Scholar
  13. 13.
    Y.R. Chien, W.J. Tseng, Switching-based variable step-size approach for partial update LMS algorithms. Electron. Lett. 9(17), 1081–1083 (2013)MathSciNetCrossRefGoogle Scholar
  14. 14.
    T. Claasen, W. Mecklenbr, Comparison of the convergence of two algorithms for adaptive fir digital filters. IEEE Trans. Circuits Syst. 28(6), 510–518 (1981)CrossRefGoogle Scholar
  15. 15.
    C. Cowan, P. Grant, Adaptive Filters (Prentice Hall, Englewood Cliffs, 1985)zbMATHGoogle Scholar
  16. 16.
    S. Douglas, T. Meng, Stochastic gradient adaptation under general error criteria. IEEE Trans. Signal Process. 42(6), 1352–1365 (1994)CrossRefGoogle Scholar
  17. 17.
    D.L. Duttweiler, Adaptive filter performance with nonlinearities in the correlation multiplier. IEEE Trans. Acoust. Speech Signal Process. 30(4), 578–586 (1982)CrossRefGoogle Scholar
  18. 18.
    E. Eweda, Comparison of RLS, LMS, and sign algorithms for tracking randomly time-varying channels. IEEE Trans. Signal Process. 42, 2937–2944 (1994)CrossRefGoogle Scholar
  19. 19.
    W.A. Gardner, Learning characteristic of stochastic-descent algorithms: a general study, analysis, and critique. Signal Process. 6(2), 113–133 (1984)MathSciNetCrossRefGoogle Scholar
  20. 20.
    J. Gibson, S. Gray, MVSE adaptive filtering subject to a constraint on MSE. IEEE Trans. Circuits Syst. 35(5), 603–608 (1988)CrossRefGoogle Scholar
  21. 21.
    R. Gitlin, H. Meadors, S. Weinstein, The tap-leakage algorithm: an algorithm for the stable operation of a digitally implemented, fractionally spaced adaptive equalizer. Bell Syst. Tech. J. 61, 1817–1839 (1982)CrossRefGoogle Scholar
  22. 22.
    I.S. Gradshteyn, I.M. Ryzhik, Table of Integral, Series, and Products, Corrected and Enlarged Edition (Academic Press, INC, New York, 1980)zbMATHGoogle Scholar
  23. 23.
    R.W. Harris, D. Chabries, F.A. Bishop, A variable stepsize (vs) algorithm. IEEETrans. Acoust. Speech Signal Process. 34, 499–510 (1986)CrossRefGoogle Scholar
  24. 24.
    S. Haykin, Adaptive Filter Theory (Prentice Hall, Englewood Cliffs, 1996)zbMATHGoogle Scholar
  25. 25.
    S. Koike, Convergence analysis of a data echo canceler with a stochastic gradient adaptive fir filter using the sign algorithm. IEEE Trans. Signal Process. 43(12), 2852–2861 (1995)CrossRefGoogle Scholar
  26. 26.
    R.H. Kwong, E.W. Johnston, A variable stepsize LMS algorithm. IEEE Trans. Signal Process. 40, 1633–1642 (1992)CrossRefzbMATHGoogle Scholar
  27. 27.
    W. Liu, J.C. Principe, S. Haykin, Kernel Adaptive Filtering : A Comprehensive Introduction (Wiley, New Jersey, 2010)CrossRefGoogle Scholar
  28. 28.
    V. Mathews, S. Cho, Improved convergence analysis of stochastic gradient adaptive filters using the sign algorithm. IEEE Trans. Acoust. Speech Signal Process. 35(4), 450–454 (1987)CrossRefzbMATHGoogle Scholar
  29. 29.
    V.J. Methews, Z. Xie, A stochastic gradient adaptive filter with gradient adaptive step size. IEEE Trans. Signal Process. 41, 2075–2087 (1993)CrossRefGoogle Scholar
  30. 30.
    M. Moinuddin, A. Zerguine, A.U.H. Sheikh, Multiple access interference plus noise constrained least mean square (MNCLMS) algorithm for CDMA systems. IEEE Trans. Circuit Syst. 55(9), 2870–2883 (2008)MathSciNetCrossRefGoogle Scholar
  31. 31.
    J. Nagumo, A. Noda, A learning method for system identification. IEEE Trans. Autom. Control AC–12, 282–287 (1967)CrossRefGoogle Scholar
  32. 32.
    K. Ozeki, T. Umeda, An adaptive filtering algorithm using an orthogonal projection to an affine subspace and its properties. IEICE Trans. Jpn. J67–A, 126–132 (1984)MathSciNetGoogle Scholar
  33. 33.
    A.H. Sayed, Fundamentals of Adaptive Filtering (Wiley, New Jersey, 2008)Google Scholar
  34. 34.
    A.H. Sayed, M. Rupp, Robustness Issues in Adaptive Filtering, DSP Handbook (CRC Press, Boca Raton, 1998)Google Scholar
  35. 35.
    W.A. Sethares, Adaptive algorithms with nonlinear data and error functions. IEEE Trans. Signal Process. 40(9), 2199–2206 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    T.J. Shan, T. Kailath, Adaptive algorithms with an automatic gain control feature. IEEE Trans. Acoust. Speech Signal Process. 35, 122–127 (1988)Google Scholar
  37. 37.
    E. Walach, B. Widrow, The least mean fourth (LMF) adaptive algorithm and its family. IEEE Trans. Inf. Theory 30(2), 275–283 (1984)CrossRefGoogle Scholar
  38. 38.
    Y. Wei, S.B. Gefand, J.V. Krogmeier, Noise constrained least mean squares algorithm. IEEE Trans. Signal Process. 49, 1961–1970 (2010)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Electrical Engineering DepartmentKarachi Institute of Economics and TechnologyKarachiPakistan
  2. 2.School of Electrical, Electronic and Computer EngineeringUniversity of Western AustraliaCrawleyAustralia
  3. 3.Center of Excellence in Intelligent Engineering Systems (CEIES)King Abdulaziz UniversityJeddahSaudi Arabia
  4. 4.Telecommunication DepartmentIqra UniversityKarachiPakistan

Personalised recommendations