Skip to main content
SpringerLink
Log in

First-Order Adaptive Algorithms

  • Chapter
  • First Online:
Book cover Fundamentals of Adaptive Signal Processing

Part of the book series: Signals and Communication Technology ((SCT))

  • 3258 Accesses

Abstract

In the two previous chapters, attention was paid on the algorithms for the determination or estimation of filters parameters with a methodology that provides knowledge of the processes statistics or their a priori calculated estimation on an appropriate window signal length. In particular, with regard to the choice of the cost function (CF) to be minimized J(w), the attention has been paid both to the solution methods of the Wiener–Hopf normal equations, which provide a stochastic optimization MMSE solution, and to the form of Yule–Walker that assumed a deterministic (or stochastic approximated) approach, by a least squares error (LSE) solution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note that the subscript n represents an iteration index not necessarily temporal.

  2. 2.

    Subtracting w opt from both members of w n  = w n−1 + μ(g−Rw n−1) = w n−1 + μ R(w opt−w n−1), we get w n  − w opt = w n−1 − w opt + μ R(w opt − w −1)  ⇒ u n  = (I − μ R)u n−1.

  3. 3.

    For independence, it holds that E{u[n]v[n]} = E{u[n]} ⋅ E{v[n]}.

  4. 4.

    The reader will note that, with the direct-averaging approximation, the assumption of independence is not strictly necessary, since it takes into account implicitly.

  5. 5.

    Note that, by making the expectation of both sides of (5.155), we obtain a difference equation identical to (5.142).

  6. 6.

    In LMS the vector u n is an RV, for which (5.65) J n  = J min + u H n Ru n , related to the SDA, in this case must be written as J n  = J min + E{u H n − 1 Ru n−1}.

  7. 7.

    Note that identical result can be reached considering a CF defined as

    w ∴ min δ(w H n−1 w n−1) s.t. |e[n]|2 = |d[n] − w H n − 1 x|2,

    with Lagrangian L(w,λ) = δ w H n − 1 w n−1 + λ|e[n]|2 and considering the descent in the stochastic gradient of the Lagrangian surface \( {\mathbf{w}}_n={\mathbf{w}}_{n-1}-\mu {\scriptscriptstyle \frac{1}{2}}{\nabla}_{\mathbf{w}}L\left(\mathbf{w},\boldsymbol{\uplambda} \right) \).

References

  1. Levenberg K (1944) A method for the solution of certain problems in least squares. Quart Appl Math 2:164–168

    MathSciNet  MATH  Google Scholar 

  2. Marquardt D (1963) An algorithm for least squares estimation on nonlinear parameters. Siam J Appl Math 11:431–441

    Article  MathSciNet  MATH  Google Scholar 

  3. Fletcher R (1986) Practical methods of optimization. Wiley, New York, NY. ISBN 0471278289

    Google Scholar 

  4. Al-Naffouri TY, Sayed AH, Nascimento VH (2003) Energy conservation in adaptive filtering. In: Barner E, Arce G (eds) Nonlinear signal and image processing: theory, methods, and applications. CRC Press, Boca Raton, FL, pp 1–35

    Google Scholar 

  5. Sayed AH (2003) Fundamentals of adaptive filtering. IEEE Wiley - Interscience, New York, NY

    Google Scholar 

  6. Yousef NR, Sayed AH (2001) A unified approach to the steady-state and tracking analyses of adaptive filters. IEEE Trans Signal Proc 49(2)

    Google Scholar 

  7. Al-Naffouri TY, Sayed AH (2003) Transient analysis of adaptive filters with error nonlinearities. IEEE Trans Signal Proc 51(3):653–663

    Article  Google Scholar 

  8. Al-Naffouri TY, Sayed AH (2003) Transient analysis of data-normalized adaptive filters. IEEE Trans Signal Proc 51(3):639–652

    Article  Google Scholar 

  9. Haykin S (1996) Adaptive filter theory, Third Editionth edn. Prentice Hall, Upper Saddle River, NJ

    Google Scholar 

  10. Widrow B, Hoff ME (1960) Adaptive switching circuits. IRE WESCON, Conv. Rec., pt. 4:96–104

    Google Scholar 

  11. Widrow B (1966) Adaptive filters I: fundamentals. Stanford Electron. Labs, Stanford, CA, SEL-66-126

    Google Scholar 

  12. Godara LC, Cantoni A (1986) Analysis of constrained LMS algorithm with application to adaptive beamforming using perturbation sequences. IEEE Trans Antennas Propagat AP-34(3):368–379

    Article  Google Scholar 

  13. Kushner HJ (1984) Approximation and weak convergence methods for random processes, with applications to stochastic systems theory. MIT Press, Cambridge, MA. ISBN 0262110903, 9780262110907

    MATH  Google Scholar 

  14. Clarkson PM, White PR (1987) Simplified analysis of the LMS adaptive filter using a transfer function approximation. IEEE Trans Acoustics Speech Signal Proc ASSP-35(7):987–933

    Article  Google Scholar 

  15. Duttweiler DL (2000) Proportionate normalized least-mean-squares adaptation in echo cancelers. IEEE Trans Speech Audio Proc 8:508–518

    Article  Google Scholar 

  16. Benesty J, Gay SL (2002) An improved PNLMS algorithm. IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '02:1881–1884

    Google Scholar 

  17. Huang Y, Benesty J, Chen J (2006) Acoustic MIMO signal processing. Springer Series on Signal and Communication Technology, ISBN 10 3-540-37630-5

    Google Scholar 

  18. Kamenetsky M, Widrow B (2004) A variable leaky lms adaptive algorithm. IEEE Thirty-Eighth Asilomar Conference on Signals, Systems and Computers 1:125–128

    Google Scholar 

  19. Mayyas K, Tyseer A (1997) Leaky LMS algorithm: MSE analysis for Gaussian data. IEEE Trans Signal Proc 45:927–934

    Article  Google Scholar 

  20. Walach E, Widrow B (1984) The least mean fourth (LMF) adaptive algorithm and its family. IEEE Trans Inform Theor IT-30(2):215

    Google Scholar 

  21. Long G, Ling F, Proakis J (1989) The LMS with delayed coefficient adaptation. IEEE Trans Acoustics Speech Signal Proc 37:1397–1405

    Article  MATH  Google Scholar 

  22. Long G, Ling F, Proakis J (1992) Corrections to the LMS with delayed coefficient adaptation. IEEE Trans Signal Proc 40:230–232

    Article  Google Scholar 

  23. Rupp M, Frenzel R (1994) Analysis of LMS and NLMS algorithms with delayed coefficient update under the presence of spherically invariant processes. IEEE Trans Signal Proc 42:668–672

    Article  Google Scholar 

  24. Tobias OJ, Bermudez JCM, Bershad NJ (2000) Stochastic analysis of the delayed LMS algorithm for a new model. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP '00. Vol. 1, pp 404–407, 15-9 June 2000

    Google Scholar 

  25. Widrow B, Stearns SD (1985) Adaptive signal processing. Prentice Hall, Upper Saddle River, NJ

    MATH  Google Scholar 

  26. Morgan DR (1980) An analysis of multiple correlation cancellation loops with a filter in the auxiliary path. IEEE Trans Acoust Speech Signal Proc ASSP-28(4):454–467

    Article  Google Scholar 

  27. Widrow B, Shur D, Shaffer S (1981) On adaptive inverse control. In 15th Asilomar Conference Circuits, Systems, and Components, 185–189

    Google Scholar 

  28. Elliott SJ, Stothers IM, Nelson PA (1987) A multiple error LMS algorithm and its application to the active control of sound and vibration. lEEE Trans Acoust Speech Signal Proc ASSP-35(10):1423–1434

    Article  Google Scholar 

  29. Tobias OJ, Bermudez JCM, Bershad NJ (2000) Mean weight behavior of the filtered-X LMS algorithm. IEEE Trans Signal Proc 48:1061–1075

    Article  Google Scholar 

  30. Snyder SD, Hansen CH (1994) Effect of transfer function estimation errors on the filtered-X LMS algorithm. IEEE Trans Signal Proc 42(4):950–953

    Article  Google Scholar 

  31. Bjarnason E (1995) Analysis of the filtered-X LMS algorithm. IEEE Trans Speech Audio Proc 3:504–514

    Article  Google Scholar 

  32. Douglas S, Pan W (1995) Exact expectation analysis of the LMS adaptive filter. IEEE Trans Signal Proc 43:2863–2871

    Article  Google Scholar 

  33. Boucher CC, Elliott SJ, Nelson PA (1991) Effect of errors in the plant model on the performance of algorithms for adaptive feed forward control. Proc Inst Elect Eng F 138:313–319

    Google Scholar 

  34. Wan EA (1996) Adjoint LMS: an efficient alternative to the filtered-X LMS and multiple error LMS algorithms. Proc IEEE ICASSP-1996:1842–1845

    Google Scholar 

  35. Widrow B (1971) Adaptive filters, from aspect of networks and system theory. Kalman – De Claris (ed.) Holt, Rinehart and Winston

    Google Scholar 

  36. Widrow B et al (1975) Adaptive noise cancellation: principles and applications. Proc IEEE 63:1691–1717

    Google Scholar 

  37. Wiener N (1949) Extrapolation, interpolation and smoothing of stationary time series, with engineering applications. Wiley, New York, NY

    MATH  Google Scholar 

  38. Bode HW, Shannon CE (1950) A simplified derivation of linear least squares smoothing and prediction theory. Proc IRE 38:417–425

    Article  MathSciNet  Google Scholar 

  39. Kivinen J, Warmuth MK (1997) Exponential gradient versus gradient descent for linear prediction. Inform Comput 132:1–64

    Article  MathSciNet  MATH  Google Scholar 

  40. Widrow B, McCool J, Ball M (1975) The complex LMS algorithm. Proc IEEE 63(4):719–720

    Article  Google Scholar 

  41. Feuer A, Weinstein E (1985) Convergence analysis of LMS filters with uncorrelated Gaussian data. IEEE Trans Acoustics Speech Signal Proc 33(1):222–230

    Article  Google Scholar 

  42. Farhang-Boroujeny B (1998) Adaptive filters: theory and applications. Wiley, New York, NY

    Google Scholar 

  43. Manolakis DG, Ingle VK, Kogon SM (2000) Statistical and adaptive signal processing. McGraw-Hill, New York, NY

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. DIET, Sapienza University of Rome, Rome, Italy

    Aurelio Uncini

Authors
  1. Aurelio Uncini
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Uncini, A. (2015). First-Order Adaptive Algorithms. In: Fundamentals of Adaptive Signal Processing. Signals and Communication Technology. Springer, Cham. https://doi.org/10.1007/978-3-319-02807-1_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-02807-1_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-02806-4

  • Online ISBN: 978-3-319-02807-1

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation