A variable step-size SIG algorithm for realizing the optimal adaptive FIR filter

  • Badong Chen
  • Yu Zhu
  • Jinchun Hu
  • Jose C. Principe
Regular Papers Control Theory


In this paper, we propose an optimal adaptive FIR filter, in which the step-size and error nonlinearity are simultaneously optimized to maximize the decrease of the mean square deviation (MSD) of the weight error vector at each iteration. The optimal step-size and error nonlinearity are derived, and a variable step-size stochastic information gradient (VS-SIG) algorithm is developed to approximately implement the optimal adaptation. Simulation results indicate that this new algorithm achieves faster convergence rate and lower misadjustment error in comparison with other adaptive algorithms.


Adaptive FIR filter optimal error nonlinearity stochastic information gradient (SIG) variable step-size 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    S. Haykin, Adaptive Filtering Theory, 3rd ed., Prentice Hall, NY, 1996.Google Scholar
  2. [2]
    B. Widrow and M. E. Hoff, “Adaptive switching circuits,” IRE WESCON Convention Record, pp. 96–104, 1960.Google Scholar
  3. [3]
    R. H. Kwong and E. W. Johnston, “A variable stepsize LMS algorithm,” IEEE Trans. on Signal Processing, vol. 40, pp. 1633–1642, 1992.CrossRefzbMATHGoogle Scholar
  4. [4]
    T. Aboulnasr and K. Mayyas, “A robust variable step-size LMS-type algorithm: analysis and simulations,” IEEE Trans. on Signal Processing, vol. 45, pp. 631–639, 1997.CrossRefGoogle Scholar
  5. [5]
    D. I. Pazaitis and A. G. Constantinides, “A novel kurtosis driven variable step-size adaptive algorithm,” IEEE Trans. on Signal Processing, vol. 47, pp. 864–872, 1999.CrossRefzbMATHGoogle Scholar
  6. [6]
    A. Mader, H. Puder, and G. U. Schmidt, “Step-size control for acoustic echo cancellation filters — an overview,” Signal Processing, vol. 80, pp. 1697–1719, 2000.CrossRefzbMATHGoogle Scholar
  7. [7]
    H. C. Shin, A. H. Sayed, and W. J. Song, “Variable step-size NLMS and affine projection algorithms,” IEEE Signal Processing Letters, vol. 11, pp. 132–135, 2004.CrossRefGoogle Scholar
  8. [8]
    L. R. Vega, H. Rey, J. Benesty, and S. Tressens, “A new robust variable step-size NLMS algorithm,” IEEE Trans. Signal Processing, vol. 56, pp. 1878–1893, 2008.CrossRefGoogle Scholar
  9. [9]
    T. Shao and Y. R. Zheng, “A new variable step-size fractional lower order moment algorithm for non-Gaussian interference environments,” Proc. IEEE ISCAS, pp. 2065–2068, May 2009.Google Scholar
  10. [10]
    Y. R. Zheng and T. Shao, “A variable step-size LMP algorithm for heavy-tailed interference suppression in phased array radar,” Proc. IEEE Aerospace Conf., pp. 1–6, Mar. 2009.Google Scholar
  11. [11]
    L. R. Vega, H. Rey, and J. Benesty, “A robust variable step-size affine projection algorithm,” Signal Processing, vol. 90, pp. 2806–2810, 2010.CrossRefzbMATHGoogle Scholar
  12. [12]
    E. Walach and B. Widrow, “The least mean fourth (LMF) adaptive algorithm and its family,” IEEE Trans. Inform. Theory, vol. IT-30, no. 2, pp. 275–283, 1984.CrossRefGoogle Scholar
  13. [13]
    S. C. Pei and C. C. Tseng, “Least mean p-power error criterion for adaptive FIR filter,” IEEE Journal on Selected Areas in Communications, vol. 12, no. 9, pp. 1540–1547, 1994.CrossRefGoogle Scholar
  14. [14]
    S. C. Douglas and H. Y. Meng, “Stochastic gradient adaptation under general error criteria,” IEEE Trans. Signal Processing, vol. 42, pp. 1335–1351, 1994.CrossRefGoogle Scholar
  15. [15]
    P. Petrus, “Robust Huber adaptive filter,” IEEE Trans. on Signal Processing, vol. 47, pp. 1129–1133, 1999.CrossRefGoogle Scholar
  16. [16]
    T. Y. Al-Naffouri and A. H. Sayed, “Adaptive filters with error nonlinearities: mean-square analysis and optimum design,” EURASIP Journal on Applied Signal Processing, vol. 4, 2001, 192–205.Google Scholar
  17. [17]
    D. Erdogmus, E. H. Kenneth, and J. C. Principe, “Online entropy manipulation: stochastic information gradient,” IEEE Signal Processing Letters, vol. 10, pp. 242–245, 2003.CrossRefGoogle Scholar
  18. [18]
    N. Bershad and M. Bonnet, “Saturation effects in LMS adaptive echo cancellation for binary data,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 38, pp. 1687–1696, 1990.CrossRefGoogle Scholar
  19. [19]
    R. Price, “A useful theorem for nonlinear devices having Gaussian inputs,” IEEE Trans. Inform. Theory, vol. IT-4, no. 6, pp. 69–72, 1958.CrossRefGoogle Scholar
  20. [20]
    T. M. Cover and J. A. Thomas, Element of Information Theory, Wiley & Son, Inc., Chichester, 1991.CrossRefGoogle Scholar
  21. [21]
    M. Janzura, T. Koski, and A. Otahal, “Minimum entropy of error principle in estimation,” Information Sciences, vol. 79, pp. 123–144, 1994.CrossRefzbMATHGoogle Scholar
  22. [22]
    J. C. Principe, D. Xu, Q. Zhao, and J. W. Fisher III, “Learning from examples with information theoretic criteria,” Journal of VLSI Signal Processing Systems, vol. 26, pp. 61–77, 2000.CrossRefzbMATHGoogle Scholar
  23. [23]
    D. Erdogmus and J. C. Principe, “An error entropy minimization algorithm for supervised training of nonlinear adaptive systems,” IEEE Trans. on Signal Processing, vol. 50, pp. 1780–1786, 2002.CrossRefGoogle Scholar
  24. [24]
    D. Erdogmus and J. C. Principe, “From linear adaptive filtering to nonlinear information processing,” IEEE Signal Processing Magazine, vol. 23, no. 6, pp. 15–33, 2006.CrossRefMathSciNetGoogle Scholar
  25. [25]
    B. Chen, J. Hu, L. Pu, and Z. Sun, “Stochastic gradient algorithm under (h,ϕ)-entropy criterion,” Circuits Systems Signal Processing, vol. 26, pp. 941–960, 2007.CrossRefzbMATHMathSciNetGoogle Scholar
  26. [26]
    B. Chen, Y. Zhu, and J. Hu, “Mean-square convergence analysis of ADALINE training with minimum error entropy criterion,” IEEE Trans. on Neural Networks, vol. 21, no. 7, pp. 1168–1179, 2010.CrossRefGoogle Scholar
  27. [27]
    B. W. Silverman, Density Estimation for Statistic and Data Analysis, Chapman & Hall, NY, 1986.Google Scholar

Copyright information

© Institute of Control, Robotics and Systems and The Korean Institute of Electrical Engineers and Springer-Verlag Berlin Heidelberg  2011

Authors and Affiliations

  • Badong Chen
    • 1
  • Yu Zhu
    • 2
  • Jinchun Hu
    • 2
  • Jose C. Principe
    • 1
  1. 1.Department of Electrical and Computer EngineeringUniversity of FloridaGainesvilleUSA
  2. 2.Institute of Manufacturing Engineering, Department of Precision Instruments and MechanologyTsinghua UniversityBeijingP. R. China

Personalised recommendations