Skip to main content
SpringerLink
Log in

Optimization Viewpoint on Kalman Smoothing with Applications to Robust and Sparse Estimation

  • Chapter
  • First Online:
Compressed Sensing & Sparse Filtering

Part of the book series: Signals and Communication Technology ((SCT))

Abstract

In this chapter, we present the optimization formulation of the Kalman filtering and smoothing problems, and use this perspective to develop a variety of extensions and applications. We first formulate classic Kalman smoothing as a least squares problem, highlight special structure, and show that the classic filtering and smoothing algorithms are equivalent to a particular algorithm for solving this problem. Once this equivalence is established, we present extensions of Kalman smoothing to systems with nonlinear process and measurement models, systems with linear and nonlinear inequality constraints, systems with outliers in the measurements or sudden changes in the state, and systems where the sparsity of the state sequence must be accounted for. All extensions preserve the computational efficiency of the classic algorithms, and most of the extensions are illustrated with numerical examples, which are part of an open source Kalman smoothing Matlab/Octave package.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Angelosante D, Roumeliotis SI, Giannakis GB (2009) Lasso-kalman smoother for tracking sparse signals. In: 2009 conference record of the 43rd Asilomar conference on signals, systems and computers, pp 181–185

    Google Scholar 

  2. Ansley CF, Kohn R (1982) A geometric derivation of the fixed interval smoothing algorithm. Biometrika 69:486–487

    Article  MathSciNet  MATH  Google Scholar 

  3. Aravkin A, Burke J, Pillonetto G (2011) Robust and trend-following Kalman smoothers using students t. In: International federation of automaic control (IFAC), 16th symposium of system identification, Oct 2011

    Google Scholar 

  4. Aravkin A, Burke J, Pillonetto G (2011) A statistical and computational theory for robust and sparse Kalman smoothing. In: International federation of automaic control (IFAC), 16th symposium of system identification, Oct 2011

    Google Scholar 

  5. Aravkin AY ( 2010) Robust methods with applications to Kalman smoothing and bundle adjustment. Ph.D. Thesis, University of Washington, Seattle, June 2010

    Google Scholar 

  6. Aravkin AY, Bell BM, Burke JV, and Pillonetto G, (2007–2011) Matlab/Octave package for constrained and robust Kalman smoothing

    Google Scholar 

  7. Aravkin AY, Bell BM, Burke JV, Pillonetto G (2011) An \(\ell _1\)-laplace robust kalman smoother. IEEE Trans Autom Control 56(12):2898–2911

    Google Scholar 

  8. Aravkin AY, Bell BM, Burke JV, Pillonetto G (2011) Learning using state space kernel machines. In: Proceedings of IFAC World congress 2011, Milan

    Google Scholar 

  9. Bell BM (1994) The iterated Kalman smoother as a Gauss-Newton method. SIAM J Opt 4(3):626–636

    Article  MATH  Google Scholar 

  10. Bell BM (2000) The marginal likelihood for parameters in a discrete Gauss-Markov process. IEEE Trans Signal Process 48(3):626–636

    Article  Google Scholar 

  11. Bell BM, Burke JV, Pillonetto G (2009) An inequality constrained nonlinear kalman-bucy smoother by interior point likelihood maximization. Automatica 45(1):25–33

    Article  MathSciNet  MATH  Google Scholar 

  12. Bell BM, Cathey F (1993) The iterated Kalman filter update as a Gauss-Newton method. IEEE Trans Autom Control 38(2):294–297

    Article  MathSciNet  MATH  Google Scholar 

  13. Bruckstein Alfred M, Donoho David L, Elad Michael (2009) From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev 51(1):34–81

    Article  MathSciNet  MATH  Google Scholar 

  14. Burke JV, Han SP (1989) A robust sequential quadratic programming method. Math Program 43:277–303. doi:10.1007/BF01582294

    Google Scholar 

  15. Burke James V (1985) Descent methods for composite nondifferentiable optimization problems. Math Program 33:260–279

    Article  MathSciNet  MATH  Google Scholar 

  16. Carmi A, Gurfil P, Kanevsky D (2010) Methods for sparse signal recovery using kalman filtering with embedded pseudo-measurement norms and quasi-norms. IEEE Trans Signal Process 58:2405–2409

    Google Scholar 

  17. Carmi A, Gurfil P, Kanevsky D (2008) A simple method for sparse signal recovery from noisy observations using kalman filtering. Technical report RC24709, Human Language Technologies, IBM

    Google Scholar 

  18. Van der Merwe R (2004) Sigma-point Kalman filters for probabilistic inference in dynamic state-space models. Ph.D. Thesis, OGI School of Science and Engineering, Oregon Health and Science University, April 2004

    Google Scholar 

  19. Dinuzzo F, Neve M, De Nicolao G, Gianazza UP (2007) On the representer theorem and equivalent degrees of freedom of SVR. J Mach Learn Res 8:2467–2495

    MathSciNet  MATH  Google Scholar 

  20. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Google Scholar 

  21. Fahrmeir L, Kaufmann V (1991) On Kalman filtering, posterior mode estimation, and Fisher scoring in dynamic exponential family regression. Metrika 38: 37–60

    Google Scholar 

  22. Fahrmeir Ludwig, Kunstler Rita (1998) Penalized likelihood smoothing in robust state space models. Metrika 49:173–191

    Article  MathSciNet  Google Scholar 

  23. Gao Junbin (2008) Robust L1 principal component analysis and its Bayesian variational inference. Neural Comput 20(2):555–572

    Article  MathSciNet  MATH  Google Scholar 

  24. Gillijns V, Mendoza OB, Chandrasekar V, De Moor BLR, Bernstein DS, Ridley A (2006) What is the ensemble Kalman filter and how well does it work? In: Proceedings of the American control conference (IEEE 2006), pp 4448–4453

    Google Scholar 

  25. Hewer GA, Martin RD, Judith Zeh (1987) Robust preprocessing for Kalman filtering of glint noise. IEEE Trans Aerosp Electron Syst AES-23(1):120–128

    Google Scholar 

  26. Jazwinski A (1970) Stochastic processes and filtering theory. Dover Publications, Inc.

    Google Scholar 

  27. Dennis JE Jr, Schnabel. RB (1983) Numerical methods for unconstrained optimiation and nonlinear equations. Computational mathematics, Prentice-Hall, Englewood Cliffs

    Google Scholar 

  28. Julier Simon, Uhlmann Jeffrey, Durrant-White Hugh (2000) A new method for the nonlinear transformation of means and covariances in filters and estimators. IEEE Trans Autom Control 45(3):477–482

    Article  MATH  Google Scholar 

  29. Kalman RE (1960) A new approach to linear filtering and prediction problems. Trans AMSE J Basic Eng 82(D):35–45

    Google Scholar 

  30. Kandepu R, Foss B, Imsland L (2008) Applying the unscented Kalman filter for nonlinear state estimation. J Process Control 18:753–768

    Article  Google Scholar 

  31. Kim S-J, Koh K, Boyd S, Gorinevsky D (2009) \(\ell _1\) trend filtering. Siam Rev 51(2):339–360

    Google Scholar 

  32. Kojima M, Megiddo N, Noma T, Yoshise A (1991) A unified approach to interior point algorithms for linear complementarity problems. Lecture notes in computer science, vol 538. Springer Verlag, Berlin

    Google Scholar 

  33. Kourouklis S, Paige CC (1981) A constrained least squares approach to the general Gauss-Markov linear model. J Am Stat Assoc 76(375):620–625

    Article  MathSciNet  MATH  Google Scholar 

  34. Lefebvre T, Bruyninckx H, De Schutter J (2004) Kalman filters for nonlinear systems: A comparison of performance. Intl J Control 77(7):639–653

    Article  MATH  Google Scholar 

  35. Liu Jun S, Chen Rong (1998) Sequential Monte Carlo methods for dynamic systems. J Am Stat Assoc 93:1032–1044

    Article  MATH  Google Scholar 

  36. Mansour H, Wason H, Lin TTY, Herrmann FJ (2012) Randomized marine acquisition with compressive sampling matrices. Geophys Prospect 60(4):648–662

    Google Scholar 

  37. Nemirovskii A, Nesterov Y (1994) Interior-point polynomial algorithms in convex programming. Studies in applied mathematics, vol 13. SIAM, Philadelphia

    Google Scholar 

  38. Oksendal B (2005) Stochastic differential equations, 6th edn. Springer, Berlin

    Google Scholar 

  39. Paige CC, Saunders MA (1977) Least squares estimation of discrete linear dynamic systems using orthogonal transformations. Siam J Numer Anal 14(2):180–193

    Article  MathSciNet  MATH  Google Scholar 

  40. Paige CC (1985) Covariance matrix representation in linear filtering. Contemp Math 47:309–321

    Google Scholar 

  41. Pillonetto G, Aravkin AY, Carpin S ( 2010) The unconstrained and inequality constrained moving horizon approach to robot localization. In: 2010 IEEE/RSJ international conference on intelligent robots and systems, Taipei, pp 3830–3835

    Google Scholar 

  42. Rauch HE, Tung F, Striebel CT (1965) Maximum likelihood estimates of linear dynamic systems. AIAA J 3(8):1145–1150

    Article  MathSciNet  Google Scholar 

  43. Rockafellar RT, Wets RJ-B (1998) Variational analysis. A series of comprehensive studies in mathematics, vol 317. Springer, Berlin

    Google Scholar 

  44. Schick Irvin C, Mitter Sanjoy K (1994) Robust recursive estimation in the presence of heavy-tailed observation noise. Annal Stat 22(2):1045–1080

    Article  MathSciNet  MATH  Google Scholar 

  45. Tibshirani R (1996 ) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58(1):267–288

    Google Scholar 

  46. van den Berg E, Friedlander MP (2008) Probing the pareto frontier for basis pursuit solutions. SIAM J Sci Comput 31(2):890–912

    Article  MathSciNet  MATH  Google Scholar 

  47. Vaswani N (2008) Kalman filtered compressed sensing. In: Proceedings of the IEEE international conference on image processing (ICIP)

    Google Scholar 

  48. Wahba G (1990) Spline models for observational data. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  49. Wright SJ (1997) Primal-dual interior-point methods. Siam, Englewood Cliffs

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Numerical Analysis and Optimization, IBM T.J. Watson Research Center, Vancouver, BC, Canada

    Aleksandr Y. Aravkin

  2. Department of Mathematics, University of Washington, Seattle, WA, USA

    James V. Burke

  3. Control and Dynamic Systems Department of Information Engineering, University of Padova, Padova, Italy

    Gianluigi Pillonetto

Authors
  1. Aleksandr Y. Aravkin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. James V. Burke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gianluigi Pillonetto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Aleksandr Y. Aravkin .

Editor information

Editors and Affiliations

  1. Department of Mechanical and Aerospace Engineering, Nanyang Technical University, Singapore, Singapore

    Avishy Y. Carmi

  2. School of Computing and Communications, Lancaster University, Lancaster, United Kingdom

    Lyudmila Mihaylova

  3. Department of Engineering, University of Cambridge, Cambridge, United Kingdom

    Simon J. Godsill

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Aravkin, A.Y., Burke, J.V., Pillonetto, G. (2014). Optimization Viewpoint on Kalman Smoothing with Applications to Robust and Sparse Estimation. In: Carmi, A., Mihaylova, L., Godsill, S. (eds) Compressed Sensing & Sparse Filtering. Signals and Communication Technology. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-38398-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-38398-4_8

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-38397-7

  • Online ISBN: 978-3-642-38398-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation