Sparse Nonlinear MIMO Filtering and Identification

Part of the Signals and Communication Technology book series (SCT)


In this chapter system identification algorithms for sparse nonlinear multi input multi output (MIMO) systems are developed. These algorithms are potentially useful in a variety of application areas including digital transmission systems incorporating power amplifier(s) along with multiple antennas, cognitive processing, adaptive control of nonlinear multivariable systems, and multivariable biological systems. Sparsity is a key constraint imposed on the model. The presence of sparsity is often dictated by physical considerations as in wireless fading channel–estimation. In other cases it appears as a pragmatic modelling approach that seeks to cope with the curse of dimensionality, particularly acute in nonlinear systems like Volterra type series. Three identification approaches are discussed: conventional identification based on both input and output samples, Semi-Blind identification placing emphasis on minimal input resources and blind identification whereby only output samples are available plus a–priori information on input characteristics. Based on this taxonomy a variety of algorithms, existing and new, are studied and evaluated by simulations.



This research has been co-financed by the European Union (European Social Fund – ESF) and Greek national funds through the Operational Program “Education and Lifelong Learning” of the National Strategic Reference Framework (NSRF) – Research Funding Program: THALIS–UOA– SECURE WIRELESS NONLINEAR COMMUNICATIONS AT THE PHYSICAL LAYER.


  1. 1.
    Abuthinien M, Chen S, Hanzo L (2008) Semi-blind joint maximum likelihood channel estimation and data detection for MIMO systems. IEEE Signal Process Lett 15:202–205CrossRefGoogle Scholar
  2. 2.
    Andrew G, Gao J (2007) Scalable training of L1 regularized log linear models. International conference on machine learning, InGoogle Scholar
  3. 3.
    Angelosante D, Bazerque JA, Giannakis GB (2010) Online adaptive estimation of sparse signals: Where RLS meets the \(\ell _1\)-norm. IEEE Trans Sig Process 58(7):3436–3447MathSciNetCrossRefGoogle Scholar
  4. 4.
    Angelosante D, Giannakis GB (2009) RLS-weighted lasso for adaptive estimation of sparse signals. IEEE Int Conf Acoust Speech Sig Process 2009:3245–3248Google Scholar
  5. 5.
    Babadi B, Kalouptsidis N, Tarokh V (2010) Sparls: The sparse RLS algorithm. IEEE Trans Sig Process 58(8):4013–4025MathSciNetCrossRefGoogle Scholar
  6. 6.
    Bahl L, Cocke J, Jelinek F, Raviv J (1974) Optimal decoding of linear codes for minimizing symbol error rate (corresp.). IEEE Trans Inf Theory 20(2):284–287.Google Scholar
  7. 7.
    Barry JR, Lee EA, Messerschmitt DG (eds) (2003) Digital communication. Springer, New YorkGoogle Scholar
  8. 8.
    Benedetto S, Biglieri S (1998) Principles of Digital Transmission: with wireless applications. Kluwer Academic, New YorkGoogle Scholar
  9. 9.
    Benesty J, Gnsler T, Huang Y, Rupp M (2004) Adaptive algorithms for MIMO acoustic echo cancellation. In: Huang Y, Benesty J (eds) Audio Signal Processing for Next-Generation Multimedia Communication Systems. Springer, BerlinGoogle Scholar
  10. 10.
    Bertsekas D, Nedic A, Ozdaglar A (2003) Convex Analysis and Optimization. Athena Scientific, CambridgeMATHGoogle Scholar
  11. 11.
    Biglieri E, Calderbank R, Constantinides A, Goldsmith A, Paulraj A, Poor VH (2007) MIMO Wireless Communications. Cambridge University Press, EnglandCrossRefGoogle Scholar
  12. 12.
    Boufounos PT, Raj B, Smaragdis P (2011) Joint sparsity models for broadband array processing. In Proc SPIE Wavelets and Sparsity 14:18–21Google Scholar
  13. 13.
    Boyd S (1985) Volterra Series: Engineering Fundamentals. PhD thesis, UC Berkeley.Google Scholar
  14. 14.
    Boyd S, Chua L (1985) Fading memory and the problem of approximating nonlinear operators with volterra series. IEEE Trans Circ Syst 32(11):1150–1161MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Brewer J (1978) Kronecker products and matrix calculus in system theory. IEEE Trans Circ Syst 25(9):772–781MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Bruckstein AM, Donoho DL, Elad M (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Review 51(1):34–81MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Cands E, Wakin M, Boyd S (2008) Enhancing sparsity by reweighted l1 minimization. J Fourier Anal Appl 14(8):77–905Google Scholar
  18. 18.
    Chen Y, Gu Y, Hero AO (2009) Sparse LMS for system identification. IEEE Int Conf Acoust Speech Sig Process 2:3125–3128Google Scholar
  19. 19.
    Chen Y, Gu Y, Hero AO (2010) Regularized Least-Mean-Square algorithms. Arxiv, preprint stat. ME/1012.5066v2.Google Scholar
  20. 20.
    Fu-Hsuan Chiu. (2006) Transceiver design and performance analysis of bit-interleaved coded MIMO-OFDM systems. PhD thesis, University of Southern California.Google Scholar
  21. 21.
    . Combettes PL, Pesquet JC, (2011) Proximal splitting methods in signal processing. In: Bauschke HH, Burachik RS, Combettes PL, Elser V, Luke DR, Wolkowicz H (eds) Fixed-point algorithms for inverse problems in science and engineering. Springer, New York, pp 185–212Google Scholar
  22. 22.
    Giannakis GB, Angelosante D, Grossi E, Lops M (2010) Sparsity-aware estimation of CDMA system parameters. EURASIP J Adv Sig Process 59(7):3262–3271Google Scholar
  23. 23.
    Dai W, Milenkovic O (2009) Subspace pursuit for compressive sensing signal reconstruction. IEEE Trans Inf Theory 55(5):2230–2249MathSciNetCrossRefGoogle Scholar
  24. 24.
    Daubechies I, Defrise M, De Mol C (2004) An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. commun Pure Appl Math 57(11):1413–1457.Google Scholar
  25. 25.
    Daubechies I, Fornasier M, Loris I (2008) Accelerated projected gradient method for linear inverse problems with sparsity constraints. J Fourier Anal Appl 14:764–792MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Davis S, Mallat GM, Zhang Z (1994) Adaptive time-frequency decompositions. SPIE J Opt Engin 33(7):2183–2191CrossRefGoogle Scholar
  27. 27.
    Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39:1–38MathSciNetMATHGoogle Scholar
  28. 28.
    Ding Z, Li Y (2001) Blind Equalization and Identification. Marcel Dekker, New YorkGoogle Scholar
  29. 29.
    Doicu A, Trautmann T, Schreier F (2010) Numerical Regularization for Atmospheric Inverse Problems. Springer, HeidelbergCrossRefMATHGoogle Scholar
  30. 30.
    Donoho DL, Johnstone IM (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika Trust 81(3):425–455MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Donoho DL, Tsaig Y, Drori I (2013) Starck JL Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit. Submitted for publication.Google Scholar
  32. 32.
    Doyle FJIII, Pearson RK, Ogunnaike BA (2002) Identification and control using Volterra series. Springer, New YorkCrossRefGoogle Scholar
  33. 33.
    Duchi J, Shwartz S.S., Singer Y, Chandra T (2008) Efficient projections onto the l1 ball for learning in high dimensions. In: Proceedings of International Conference on Machine Learning (ICML 08) pp 272–279Google Scholar
  34. 34.
    Duttweiler DL (2000) Proportionate normalized Least-Mean-Squares adaptation in echo cancellers. IEEE Trans Speech Audio Process 8(5):508–518CrossRefGoogle Scholar
  35. 35.
    Eksioglu EM, Tanc AK (2011) RLS algorithm with convex regularization. IEEE Signal Process Lett 18(8):470–473CrossRefGoogle Scholar
  36. 36.
    Elad M (2010) Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, New YorkCrossRefGoogle Scholar
  37. 37.
    Ephraim Y, Merhav N (2002) Hidden markov processes. IEEE Trans Inf Theory 48(6):1518–1569MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Werner S, Riihonen T, Gregorio F (2011) Power amplifier linearization technique with iq imbalance and crosstalk compensation for broadband MIMO-OFDM transmitters. EURASIP J Adv Sig Process 19:1–15Google Scholar
  39. 39.
    Feder M (1987) Statistical signal processing using a class of iterative estimation algorithms. PhD thesis, M.I.T Cambridge MA.Google Scholar
  40. 40.
    Fernandes, CAR (2009) Nonlinear MIMO communication systems: Channel estimation and information recovery using volterra models. PhD thesis, Universite de Nice-Sophia Antipolis.Google Scholar
  41. 41.
    Figueiredo MAT, Nowak RD (2003) An EM algorithm for wavelet-based image restoration. IEEE Trans Image Process 12(8):906–916MathSciNetCrossRefGoogle Scholar
  42. 42.
    Gholami A (2011) A general framework for sparsity-based denoising and inversion. IEEE Trans Sigl Process 59(11):5202–5211MathSciNetCrossRefGoogle Scholar
  43. 43.
    Giannakis G, Liu Z, Ma X (2003) Space Time Coding for Broadband Wireless Communications. Wiley-Interscience, HobokenGoogle Scholar
  44. 44.
    Giannakis GB, Serpedin E (1997) Linear multichannel blind equalizers of nonlinear FIR volterra channels. IEEE Trans Sig Process 45(1):67–81CrossRefGoogle Scholar
  45. 45.
    Gregorio F, Werner S, Laakso TI, Cousseau J (2007) Receiver cancellation technique for nonlinear power amplifier distortion in SDMA-OFDM systems. IEEE Trans Veh Technol 56(5):2499–2516CrossRefGoogle Scholar
  46. 46.
    Haykin SO (2001) Adaptive filter theory, 4th edn. Springer, New YorkGoogle Scholar
  47. 47.
    Huang Y, Benesty J, Chen J (2006) Acoustic MIMO signal processing. Springer, BerlinMATHGoogle Scholar
  48. 48.
    Hurley N, Rickard S (2009) Comparing measures of sparsity. IEEE Trans Inf Theory 55(10):4723–4741MathSciNetCrossRefGoogle Scholar
  49. 49.
    Jin J, Gu Y, Mei S (2010) A stochastic gradient approach on compressive sensing signal reconstruction based on adaptive filtering framework. IEEE J Sel Topics Sig Process 4(2):409–420CrossRefGoogle Scholar
  50. 50.
    Kaleh GK, Vallet R (1994) Joint parameter estimation and symbol detection for linear or nonlinear unknown channels. IEEE Trans Comm 42(7):2406–2413CrossRefMATHGoogle Scholar
  51. 51.
    Kalouptsidis N (1997) Signal Processing Systems Theory and Design. John Wiley and Sons, New YorkGoogle Scholar
  52. 52.
    Nicholas Kalouptsidis, Gerasimos Mileounis, Behtash Babadi, Vahid Tarokh (2011) Adaptive algorithms for sparse system identification. Sig Process 91(8):1910–1919CrossRefMATHGoogle Scholar
  53. 53.
    Koike-Akino T, Molisch AF, Pun MO, Annavajjala R, Orlik P (2011) Order-extended sparse RLS algorithm for doubly-selective MIMO channel estimation. In: IEEE international conference on communications (ICC), 1–6 June 2011.Google Scholar
  54. 54.
    Kopsinis Y, Slavakis K, Theodoridis S (2011) Online sparse system identification and signal reconstruction using projections onto weighted \(\ell _{1}\) balls. IEEE Trans Sig Process 59(3):936–952MathSciNetCrossRefGoogle Scholar
  55. 55.
    Ljung L (1993) General structure of adaptive algorithms: Adaptation and tracking. Kalouptsidis N, Theodoridis S (eds) In adaptive system identification and signal processing algorithmsGoogle Scholar
  56. 56.
    Maleh R, Gilbert AC (2007) Multichannel image estimation via simultaneous orthogonal matching pursuit. Proceedings workshop statistics signal process, InGoogle Scholar
  57. 57.
    Marmarelis PZ, Marmarelis VZ (1978) Analysis of physiological systems. Plenum Press, New YorkCrossRefGoogle Scholar
  58. 58.
    Mileounis G, Babadi B, Kalouptsidis N, Tarokh V (2010) An adaptive greedy algorithm with application to nonlinear communications. IEEE Trans Sig Process 58(6):2998–3007MathSciNetCrossRefGoogle Scholar
  59. 59.
    Mileounis G, Kalouptsidis N (2012) A sparsity driven approach to cumulant based identification. In IEEE International Workshop on Signal Processing Advances in Wireless Communications.Google Scholar
  60. 60.
    Mileounis G Kalouptsidis N, Babadi B, Tarokh V (2011) Blind identification of sparse channels and symbol detection via the EM algorithm. In: International conference on digital signal processing (DSP), 1–5 july 2011.Google Scholar
  61. 61.
    Murakami Y, Yamagishi M, Yukawa M, Yamada I (2010) A sparse adaptive filtering using time-varying soft-thresholding techniques. In: 2010 IEEE international conference on acoustics speech and signal processing (ICASSP), 3734–3737.Google Scholar
  62. 62.
    Needell D, Tropp JA (2009) CoSaMP: Iterative signal recovery from incomplete and inaccurate samples. Appl Comput Harmon Anal 26:301–321MathSciNetCrossRefMATHGoogle Scholar
  63. 63.
    Needell D, Vershynin R (2009) Uniform uncertainty principle and signal recovery via regularized orthogonal matching pursuit. Found Comput Math 9(3):317–334MathSciNetCrossRefMATHGoogle Scholar
  64. 64.
    Paleologu C, Benesty J, Ciochina S (2010) Sparse adaptive filters for echo cancellation. Morgan and Claypool Publishers, San RafaelGoogle Scholar
  65. 65.
    Pati YC, Rezaiifar R, Krishnaprasad PS (1993) Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: 27th asilomar Conference on signals, systems and Computation 40–44.Google Scholar
  66. 66.
    Rabiner LR (1989) A tutorial on hidden markov models and selected applications in speech recognition. IEEE Proc 77(2):257–286CrossRefGoogle Scholar
  67. 67.
    Redfern AJ, Zhou GT (2001) Blind zero forcing equalization of multichannel nonlinear CDMA systems. IEEE Trans Sig Process 49(10):2363–2371CrossRefGoogle Scholar
  68. 68.
    Rizogiannis C, Kofidis E, Papadias CB, Theodoridis S (2010) Semi-blind maximum-likelihood joint channel/data estimation for correlated channels in multiuser MIMO networks. Sig Process 90(4):1209–1224CrossRefMATHGoogle Scholar
  69. 69.
    Rugh WJ, (1981) Nonlinear System Theory. The Johns Hopkins University Press, Baltimore (in press).Google Scholar
  70. 70.
    Adaptive Filters. Wiley-Blackwell, Hoboken.Google Scholar
  71. 71.
    Schetzen M (1980) The volterra and wiener theories of nonlinear systems. Willey and Sons, New YorkMATHGoogle Scholar
  72. 72.
    Seretis C (1997) Control-relevant identification for constrained and nonlinear systems. PhD thesis, University of Maryland.Google Scholar
  73. 73.
    Tidestav C, Lindskog E (1998) Bootstrap equalization. In: IEEE International Conference on Universal Personal Communications (ICUPC), vol 2. 1221–1225 oct 1998.Google Scholar
  74. 74.
    Tong L, Rw Liu, Soon VC, Huang YF (1991) Indeterminacy and identifiability of blind identification. IEEE Trans Circ Syst 38(5):499–509CrossRefMATHGoogle Scholar
  75. 75.
    Tropp JA, Gilbert AC, Strauss MJ (2006) Algorithms for simultaneous sparse approximation. part i: Greedy pursuit. Sig Process 86(3):572–588.Google Scholar
  76. 76.
    Tropp JA, Wright SJ (2010) Computational methods for sparse solution of linear inverse problems. Proceedings of the IEEE 98(6):948–958CrossRefGoogle Scholar
  77. 77.
    Sicuranza GL, Mathews VJ (2000) Polynomial Signal Processing. Wiley-Blackwell, New YorkGoogle Scholar
  78. 78.
    Wang Y, Yagola AG, Yang C (2010) Optimization and regularization for computational inverse problems and applications. Springer, HeidelbergMATHGoogle Scholar
  79. 79.
    Weston J, Elisseeff A, Schölkopf B, Tipping M (2003) Use of the zero norm with linear models and kernel methods. J Mach Learn Res 3:1439–1461MathSciNetMATHGoogle Scholar
  80. 80.
    Westwick DT, Kearney RE (2003) Identification of nonlinear physiological systems. IEEE Press, New YorkCrossRefGoogle Scholar
  81. 81.
    Wu C (1983) On the convergence properties of the EM algorithm. Ann Statist 11:95–103MathSciNetCrossRefMATHGoogle Scholar
  82. 82.
    Yukawa M (2010) Adaptive filtering based on projection method. Block Seminar in Elite Master Study Course SIM.Google Scholar
  83. 83.
    Zheng Q (1995) A volterra series approach to nonlinear process control and control relevant identification. PhD thesis, University of Maryland.Google Scholar
  84. 84.
    Zheng Q, Zafiriou E (2004) Volterra Laguerre models for nonlinear process identification with application to a fluid catalytic cracking unit. Ind Eng Chem Res 43(2):340–348CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.Department of Informatics and Telecommunications, Division of Communications and Signal ProcessingUniversity of AthensPanepistimiopolisGreece

Personalised recommendations