Adaptive Filtering pp 71-131 | Cite as

# The Least-Mean-Square (LMS) Algorithm

## Abstract

The least mean-square (LMS) is a search algorithm in which a simplification of the gradient vector computation is made possible by appropriately modifying the objective function [1]–[2]. The LMS algorithm, as well as others related to it, is widely used in various applications of adaptive filtering due to its computational simplicity [3]–[7]. The convergence characteristics of the LMS algorithm are examined in order to establish a range for the convergence factor that will guarantee stability. The convergence speed of the LMS is shown to be dependent of the eigenvalue spread of the input-signal correlation matrix [2]–[6]. In this chapter, several properties of the LMS algorithm are discussed including the misadjustment in stationary and nonstationary environments [2]– [9], tracking performance, and finite wordlength effects [10]–[12].

### Keywords

Attenuation Covariance Autocorrelation Convolution Seco## Preview

Unable to display preview. Download preview PDF.

### References

- 1.B. Widrow and M. E. Hoff, “Adaptive switching circuits,”
*WESCOM Conv. Rec.*, pt. 4, pp. 96–140, 1960.Google Scholar - 2.B. Widrow, J. M. McCool, M. G. Larimore, and C. R. Johnson, Jr., “Stationary and nonstationary learning characteristics of the LMS adaptive filters,”
*Proceedings of the IEEE*, vol. 64, pp. 1151–1162, Aug. 1976.MathSciNetCrossRefGoogle Scholar - 3.G. Ungerboeck, “Theory on the speed of convergence in adaptive equalizers for digital communication,”
*IBM Journal on Research and Development*, vol. 16, pp. 546–555, Nov. 1972.MATHCrossRefGoogle Scholar - 4.J. E. Mazo, “On the independence theory of equalizer convergence,”
*The Bell System Technical Journal*, vol. 58, pp. 963–993, May 1979.MathSciNetMATHGoogle Scholar - 5.B. Widrow and S. D. Stearns,
*Adaptive Signal Processing*, Prentice Hall, Englewood Cliffs, NJ, 1985.MATHGoogle Scholar - 6.S. Haykin,
*Adaptive Filter Theory*, Prentice Hall, Englewood Cliffs, NJ, 2nd edition, 1991.MATHGoogle Scholar - 7.M. G. Bellanger,
*Adaptive Digital Filters and Signal Analysis*, Marcel Dekker Inc., NY, 1987.Google Scholar - 8.D. C. Farden, “Tracking properties of adaptive signal processing algorithms,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol. ASSP-29, pp. 439–446, June 1981.MathSciNetCrossRefGoogle Scholar - 9.B. Widrow and E. Walach, “On the statistical efficiency of the LMS algorithm with nonstationary inputs,”
*IEEE Trans. on Information Theory*, vol. IT-30, pp. 211–221, March 1984.CrossRefGoogle Scholar - 10.O. Macchi, “Optimization of adaptive identification for time varying filters,”
*IEEE Trans. on Automatic Control*, vol. AC-31, pp. 283–287, March 1986.CrossRefGoogle Scholar - 11.A. Benveniste, “Design of adaptive algorithms for the tracking of time varying systems,”
*Int. J. Adaptive Control and Signal Processing*, vol. 1, pp. 3–29, Jan. 1987.MATHCrossRefGoogle Scholar - 12.W. A. Gardner, “Nonstationary learning characteristics of the LMS algorithm,”
*IEEE Trans. on Circuits and Systems*, vol. CAS-34, pp. 1199–1207, Oct. 1987.CrossRefGoogle Scholar - 13.A. Papoulis,
*Probability, Random Variables, and Stochastic Processes*, McGraw Hill, New York, NY, 3rd edition, 1991.Google Scholar - 14.V. Solo, “The limiting behavior of LMS,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol-37, pp. 1909–1922, Dec. 1989.MathSciNetMATHCrossRefGoogle Scholar - 15.N. J. Bershad and O. M. Macchi, “Adaptive recovery of a chirped sinusoid in noise, Part 2: Performance of the LMS algorithm,”
*IEEE Trans. on Signal Processing*, vol. 39, pp. 595–602, March 1991.CrossRefGoogle Scholar - 16.M. Andrews and R. Fitch, “Finite wordlength arithmetic computational error effects on the LMS adaptive weights,”
*Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing*, pp. 628–631, May 1977.Google Scholar - 17.C. Caraiscos and B. Liu, “A roundoff error analysis of the LMS adaptive algorithm,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol. ASSP-32, pp. 34–41, Feb. 1984.CrossRefGoogle Scholar - 18.S. T. Alexander, “Transient weight misadjustment properties for the finite precision LMS algorithm,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol. ASSP-35, pp. 1250–1258, Sept. 1987.CrossRefGoogle Scholar - 19.A. V. Oppenheim and R. W. Scharfer,
*Discrete-Time Signal Processing*, Prentice Hall, Englewood Cliffs, NJ, 1989.MATHGoogle Scholar - 20.A. Antoniou,
*Digital Filters: Analysis, Design, and Applications*, McGraw Hill, New York, NY, 2nd edition, 1992.Google Scholar - 21.A. B. Spirad and D. L. Snyder, “Quantization errors in floating-point arithmetic,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol. ASSP-26, pp. 456–464, Oct. 1983.Google Scholar - 22.A. Feuer and E. Weinstein, “Convergence analysis of LMS filters with uncorrelated Gaussian data,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol. ASSP-33, pp. 222–230, Jan. 1985.CrossRefGoogle Scholar - 23.D. T. Slock, “On the convergence behavior of the LMS and normalized LMS algorithms,”
*IEEE Trans. on Signal Processing*, vol-40, pp. 2811–2825, Sept. 1993.CrossRefGoogle Scholar - 24.W. A. Sethares, D. A. Lawrence, C. R. Johnson, Jr., and R. R. Bitmead, “Parameter drift in LMS adaptive filters,”
*IEEE Trans. on Acoust., Speech, and Signal Processing*, vol. ASSP-34, pp. 868–878, Aug. 1986.CrossRefGoogle Scholar - 25.V. Solo, “The error variance of LMS with time varying weights,”
*IEEE Trans. on Signal Processing*, vol-40, pp. 803–813, April 1992.MATHCrossRefGoogle Scholar - 26.S. U. Qureshi, “Adaptive Equalization,”
*Proceedings of the IEEE*, vol-73, pp. 1349–1387, Sept. 1985.CrossRefGoogle Scholar - 27.M. L. Honig, “Echo cancellation of voiceband data signals using recursive least squares and stochastic gradient algorithms,”
*IEEE Trans. on Communications*, vol. COM-33, pp. 65–73, Jan. 1985.CrossRefGoogle Scholar - 28.S. Subramanian, D. J. Shpak, P. S. R. Diniz, and A. Antoniou, “The performance of adaptive filtering algorithms in a simulated HDSL environment,”
*Proc. IEEE Canadian Conf. Electrical and Computer Engineering*, Toronto, pp. TA 2.19.1–TA 2.19.5, Sept. 1992.Google Scholar - 29.C. S. Modlin and J. M. Cioffi, “A fast decision feedback LMS algorithm using multiple step sizes,”
*Proc. IEEE SUPERCOMM Inter. Conf. on Communications*, New Orleans, pp. 1201–1205, May 1994.Google Scholar