Optimization Viewpoint on Kalman Smoothing with Applications to Robust and Sparse Estimation

  • Aleksandr Y. Aravkin
  • James V. Burke
  • Gianluigi Pillonetto
Part of the Signals and Communication Technology book series (SCT)


In this chapter, we present the optimization formulation of the Kalman filtering and smoothing problems, and use this perspective to develop a variety of extensions and applications. We first formulate classic Kalman smoothing as a least squares problem, highlight special structure, and show that the classic filtering and smoothing algorithms are equivalent to a particular algorithm for solving this problem. Once this equivalence is established, we present extensions of Kalman smoothing to systems with nonlinear process and measurement models, systems with linear and nonlinear inequality constraints, systems with outliers in the measurements or sudden changes in the state, and systems where the sparsity of the state sequence must be accounted for. All extensions preserve the computational efficiency of the classic algorithms, and most of the extensions are illustrated with numerical examples, which are part of an open source Kalman smoothing Matlab/Octave package.


  1. 1.
    Angelosante D, Roumeliotis SI, Giannakis GB (2009) Lasso-kalman smoother for tracking sparse signals. In: 2009 conference record of the 43rd Asilomar conference on signals, systems and computers, pp 181–185Google Scholar
  2. 2.
    Ansley CF, Kohn R (1982) A geometric derivation of the fixed interval smoothing algorithm. Biometrika 69:486–487MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Aravkin A, Burke J, Pillonetto G (2011) Robust and trend-following Kalman smoothers using students t. In: International federation of automaic control (IFAC), 16th symposium of system identification, Oct 2011Google Scholar
  4. 4.
    Aravkin A, Burke J, Pillonetto G (2011) A statistical and computational theory for robust and sparse Kalman smoothing. In: International federation of automaic control (IFAC), 16th symposium of system identification, Oct 2011Google Scholar
  5. 5.
    Aravkin AY ( 2010) Robust methods with applications to Kalman smoothing and bundle adjustment. Ph.D. Thesis, University of Washington, Seattle, June 2010Google Scholar
  6. 6.
    Aravkin AY, Bell BM, Burke JV, and Pillonetto G, (2007–2011) Matlab/Octave package for constrained and robust Kalman smoothingGoogle Scholar
  7. 7.
    Aravkin AY, Bell BM, Burke JV, Pillonetto G (2011) An \(\ell _1\)-laplace robust kalman smoother. IEEE Trans Autom Control 56(12):2898–2911Google Scholar
  8. 8.
    Aravkin AY, Bell BM, Burke JV, Pillonetto G (2011) Learning using state space kernel machines. In: Proceedings of IFAC World congress 2011, MilanGoogle Scholar
  9. 9.
    Bell BM (1994) The iterated Kalman smoother as a Gauss-Newton method. SIAM J Opt 4(3):626–636CrossRefMATHGoogle Scholar
  10. 10.
    Bell BM (2000) The marginal likelihood for parameters in a discrete Gauss-Markov process. IEEE Trans Signal Process 48(3):626–636CrossRefGoogle Scholar
  11. 11.
    Bell BM, Burke JV, Pillonetto G (2009) An inequality constrained nonlinear kalman-bucy smoother by interior point likelihood maximization. Automatica 45(1):25–33MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Bell BM, Cathey F (1993) The iterated Kalman filter update as a Gauss-Newton method. IEEE Trans Autom Control 38(2):294–297MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Bruckstein Alfred M, Donoho David L, Elad Michael (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51(1):34–81MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Burke JV, Han SP (1989) A robust sequential quadratic programming method. Math Program 43:277–303. doi: 10.1007/BF01582294 Google Scholar
  15. 15.
    Burke James V (1985) Descent methods for composite nondifferentiable optimization problems. Math Program 33:260–279MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Carmi A, Gurfil P, Kanevsky D (2010) Methods for sparse signal recovery using kalman filtering with embedded pseudo-measurement norms and quasi-norms. IEEE Trans Signal Process 58:2405–2409Google Scholar
  17. 17.
    Carmi A, Gurfil P, Kanevsky D (2008) A simple method for sparse signal recovery from noisy observations using kalman filtering. Technical report RC24709, Human Language Technologies, IBMGoogle Scholar
  18. 18.
    Van der Merwe R (2004) Sigma-point Kalman filters for probabilistic inference in dynamic state-space models. Ph.D. Thesis, OGI School of Science and Engineering, Oregon Health and Science University, April 2004Google Scholar
  19. 19.
    Dinuzzo F, Neve M, De Nicolao G, Gianazza UP (2007) On the representer theorem and equivalent degrees of freedom of SVR. J Mach Learn Res 8:2467–2495MathSciNetMATHGoogle Scholar
  20. 20.
    Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306Google Scholar
  21. 21.
    Fahrmeir L, Kaufmann V (1991) On Kalman filtering, posterior mode estimation, and Fisher scoring in dynamic exponential family regression. Metrika 38: 37–60Google Scholar
  22. 22.
    Fahrmeir Ludwig, Kunstler Rita (1998) Penalized likelihood smoothing in robust state space models. Metrika 49:173–191MathSciNetCrossRefGoogle Scholar
  23. 23.
    Gao Junbin (2008) Robust L1 principal component analysis and its Bayesian variational inference. Neural Comput 20(2):555–572MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Gillijns V, Mendoza OB, Chandrasekar V, De Moor BLR, Bernstein DS, Ridley A (2006) What is the ensemble Kalman filter and how well does it work? In: Proceedings of the American control conference (IEEE 2006), pp 4448–4453Google Scholar
  25. 25.
    Hewer GA, Martin RD, Judith Zeh (1987) Robust preprocessing for Kalman filtering of glint noise. IEEE Trans Aerosp Electron Syst AES-23(1):120–128Google Scholar
  26. 26.
    Jazwinski A (1970) Stochastic processes and filtering theory. Dover Publications, Inc.Google Scholar
  27. 27.
    Dennis JE Jr, Schnabel. RB (1983) Numerical methods for unconstrained optimiation and nonlinear equations. Computational mathematics, Prentice-Hall, Englewood CliffsGoogle Scholar
  28. 28.
    Julier Simon, Uhlmann Jeffrey, Durrant-White Hugh (2000) A new method for the nonlinear transformation of means and covariances in filters and estimators. IEEE Trans Autom Control 45(3):477–482CrossRefMATHGoogle Scholar
  29. 29.
    Kalman RE (1960) A new approach to linear filtering and prediction problems. Trans AMSE J Basic Eng 82(D):35–45Google Scholar
  30. 30.
    Kandepu R, Foss B, Imsland L (2008) Applying the unscented Kalman filter for nonlinear state estimation. J Process Control 18:753–768CrossRefGoogle Scholar
  31. 31.
    Kim S-J, Koh K, Boyd S, Gorinevsky D (2009) \(\ell _1\) trend filtering. Siam Rev 51(2):339–360Google Scholar
  32. 32.
    Kojima M, Megiddo N, Noma T, Yoshise A (1991) A unified approach to interior point algorithms for linear complementarity problems. Lecture notes in computer science, vol 538. Springer Verlag, BerlinGoogle Scholar
  33. 33.
    Kourouklis S, Paige CC (1981) A constrained least squares approach to the general Gauss-Markov linear model. J Am Stat Assoc 76(375):620–625MathSciNetCrossRefMATHGoogle Scholar
  34. 34.
    Lefebvre T, Bruyninckx H, De Schutter J (2004) Kalman filters for nonlinear systems: A comparison of performance. Intl J Control 77(7):639–653CrossRefMATHGoogle Scholar
  35. 35.
    Liu Jun S, Chen Rong (1998) Sequential Monte Carlo methods for dynamic systems. J Am Stat Assoc 93:1032–1044CrossRefMATHGoogle Scholar
  36. 36.
    Mansour H, Wason H, Lin TTY, Herrmann FJ (2012) Randomized marine acquisition with compressive sampling matrices. Geophys Prospect 60(4):648–662Google Scholar
  37. 37.
    Nemirovskii A, Nesterov Y (1994) Interior-point polynomial algorithms in convex programming. Studies in applied mathematics, vol 13. SIAM, PhiladelphiaGoogle Scholar
  38. 38.
    Oksendal B (2005) Stochastic differential equations, 6th edn. Springer, BerlinGoogle Scholar
  39. 39.
    Paige CC, Saunders MA (1977) Least squares estimation of discrete linear dynamic systems using orthogonal transformations. Siam J Numer Anal 14(2):180–193MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Paige CC (1985) Covariance matrix representation in linear filtering. Contemp Math 47:309–321Google Scholar
  41. 41.
    Pillonetto G, Aravkin AY, Carpin S ( 2010) The unconstrained and inequality constrained moving horizon approach to robot localization. In: 2010 IEEE/RSJ international conference on intelligent robots and systems, Taipei, pp 3830–3835Google Scholar
  42. 42.
    Rauch HE, Tung F, Striebel CT (1965) Maximum likelihood estimates of linear dynamic systems. AIAA J 3(8):1145–1150MathSciNetCrossRefGoogle Scholar
  43. 43.
    Rockafellar RT, Wets RJ-B (1998) Variational analysis. A series of comprehensive studies in mathematics, vol 317. Springer, BerlinGoogle Scholar
  44. 44.
    Schick Irvin C, Mitter Sanjoy K (1994) Robust recursive estimation in the presence of heavy-tailed observation noise. Annal Stat 22(2):1045–1080MathSciNetCrossRefMATHGoogle Scholar
  45. 45.
    Tibshirani R (1996 ) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58(1):267–288Google Scholar
  46. 46.
    van den Berg E, Friedlander MP (2008) Probing the pareto frontier for basis pursuit solutions. SIAM J Sci Comput 31(2):890–912MathSciNetCrossRefMATHGoogle Scholar
  47. 47.
    Vaswani N (2008) Kalman filtered compressed sensing. In: Proceedings of the IEEE international conference on image processing (ICIP)Google Scholar
  48. 48.
    Wahba G (1990) Spline models for observational data. SIAM, PhiladelphiaCrossRefMATHGoogle Scholar
  49. 49.
    Wright SJ (1997) Primal-dual interior-point methods. Siam, Englewood CliffsCrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Aleksandr Y. Aravkin
    • 1
  • James V. Burke
    • 2
  • Gianluigi Pillonetto
    • 3
  1. 1.Numerical Analysis and OptimizationIBM T.J. Watson Research CenterVancouverCanada
  2. 2.Department of MathematicsUniversity of WashingtonSeattleUSA
  3. 3.Control and Dynamic Systems Department of Information EngineeringUniversity of PadovaPadovaItaly

Personalised recommendations