Multidimensional Systems and Signal Processing

, Volume 30, Issue 1, pp 195–217 | Cite as

A simple numerical method based simultaneous stochastic perturbation for estimation of high dimensional matrices

  • H. S. HoangEmail author
  • R. Baraille


We describe a simple algorithm for estimating the elements of a matrix as well as its decomposition under the condition that only the product of this matrix with a vector is accessible. The algorithm is based on application of the stochastic simultaneous perturbation method. Theoretical results on the convergence of the proposed algorithm are proven. Numerical experiments are presented to show the efficiency of the proposed algorithm.


Numerical differentiation Stochastic simultaneous perturbation Matrix decomposition Singular value decomposition Parameter estimation Data assimilation 



The authors would like to thank the anonymous reviewers for their valuable comments and suggestions to improve the quality of the paper.


  1. Anderson, T. W. (2003). An introduction to multivariate statistical analysis. London: Wiley-Interscience.zbMATHGoogle Scholar
  2. Arulampalam, M., Maskell, S., Gordon, N., & Clapp, T. (2002). A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking. IEEE Transactions on Signal Processing, 50(2), 174–188.CrossRefGoogle Scholar
  3. Bai, J., & Shi, S. (2011). Estimating high dimensional covariance matrices and its applications. Annals of Economics and Finances, 12–2, 199–215.Google Scholar
  4. Beu, T. A. (2015). Introduction to numerical programming. Oxfordshire: Taylor and Francis Group.zbMATHGoogle Scholar
  5. Chicone, C. (2006). Ordinary differential equations with applications. New York: Springer.zbMATHGoogle Scholar
  6. Cooper, M., & Haines, K. (1996). Altimetric assimilation with water property conservation. Journal of Geophysical Research, 101, 1059–1077.CrossRefGoogle Scholar
  7. Daley, R. (1991). Atmospheric data analysis. New York: Cambridge University Press.Google Scholar
  8. Del Moral, P. (1996). Non linear filtering: Interacting particle solution. Markov Processes and Related Fields, 2(4), 555–580.MathSciNetzbMATHGoogle Scholar
  9. Del Moral, P., Doucet, A., & Jasra, A. (2012). On adaptive resampling procedures for sequential Monte Carlo methods. Bernoulli, 18(1), 252–278.MathSciNetCrossRefzbMATHGoogle Scholar
  10. Delijani, E. B., Pishvaie, M. R., & Boozarjomehry, R. B. (2014). Subsurface characterization with localized ensemble Kalman filter employing adaptive thresholding. Advances in Water Resources, 69, 181–196.CrossRefGoogle Scholar
  11. Ding, F., & Zhang, H. M. (2014). Gradient-based iterative algorithm for a class of the coupled matrix equations related to control systems. IET Control Theory and Applications, 8(15), 1588–1595.MathSciNetCrossRefGoogle Scholar
  12. El Karoui, N. (2008). Operator norm consistent estimation of large dimensional sparse covariance matrices. Annals of Statistics, 36, 2717–2756.MathSciNetCrossRefzbMATHGoogle Scholar
  13. Evensen, G. (2007). Data assimilation: The ensemble Kalman filter. Berlin: Springer.zbMATHGoogle Scholar
  14. Fukumori, I., & Malanotte-Rizzoli, P. (1995). An approximate Kalman filter for ocean data assimilation: An example with an idealized Gulf Stream model. Journal of Geophysical Research, 100(C4), 6777–6793.CrossRefGoogle Scholar
  15. Golub, G. H., & van Loan, C. F. (1996). Matrix computations. Cambridge: Cambridge University Press.zbMATHGoogle Scholar
  16. Gordon, N. J., Salmond, D. J., & Smith, A. F. M. (1993). Novel approach to nonlinear/non-Gaussian Bayesian state estimation. IEE Proceedings of Radar and Signal Processing, F 140(2), 107–113. (ISSN 0956-375X).CrossRefGoogle Scholar
  17. Hoang, H. S., & Baraille, R. (2011). Prediction error sampling procedure based on dominant Schur decomposition. Application to state estimation in high dimensional oceanic model. Applied Mathematics and Computation, 218(7), 3689–3709.MathSciNetCrossRefzbMATHGoogle Scholar
  18. Hoang, H. S., & Baraille, R. (2017a). On the efficient low cost procedure for estimation of high-dimensional prediction error covariance matrices. Automatica, 83, 317–330.MathSciNetCrossRefzbMATHGoogle Scholar
  19. Hoang, H. S., & Baraille, R. (2017b). A comparison study on performance of an adaptive filter with other estimation methods for state estimation in high-dimensional system, chapter 2. In T. Hokimoto (Ed.), Advances in statistical methodologies and their application to real problems, Chapter 2 (pp. 29–52). Chennai: Intech.Google Scholar
  20. Hoang, H. S., Baraille, R., & Talagrand, O. (2001). On the design of a stable adaptive filter for state estimation in high dimensional system. Automatica, 37, 341–359.MathSciNetCrossRefzbMATHGoogle Scholar
  21. Jazwinski, A. H. (1970). Stochastic processes and filtering theory. New York: Academic.zbMATHGoogle Scholar
  22. Kailath, T. (1991). From Kalman filtering to innovations. In A. C. Antoulas (Ed.), Martingales, scattering and other nice things, mathematical system theory (pp. 55–88). Heidelberg: Springer.Google Scholar
  23. Kivman, G. A. (2003). Sequential parameter estimation for stochastic systems. Nonlinear Processes in Geophysics, 10, 253–259.CrossRefGoogle Scholar
  24. Ledoit, O., & Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 365–411.MathSciNetCrossRefzbMATHGoogle Scholar
  25. Levina, E., Rothman, A. J., & Zhu, J. (2007). Sparse estimation of large covariance matrices via a nested Lasso penalty. Annals of Applied Statistics, 2, 245–263.MathSciNetCrossRefzbMATHGoogle Scholar
  26. Lorenz, E. N. (1963). Deterministic non-periodic flow. Journal of the Atmospheric Sciences, 20(2), 130–141.CrossRefzbMATHGoogle Scholar
  27. Muirhead, R. (2005). Aspects of multivariate statistical theory. London: Wiley.zbMATHGoogle Scholar
  28. Musoff, H., & Zarchan, P. (2005). Fundamentals of Kalman filtering: A practical approach. Reston: American Institute of Aeronautics and Astronautics.CrossRefGoogle Scholar
  29. Pannekoucke, O., Berre, L., & Desroziers, G. (2008). Background-error correlation length-scale estimates and their sampling statistics. Quarterly Journal of the Royal Meteorological Society, 134(631), 497508.CrossRefGoogle Scholar
  30. Ristic, B., Arulampalam, S., & Gordon, N. (2004). Beyond the Kalman Filter: Particle filters for tracking applications. Norwood: Artech House.zbMATHGoogle Scholar
  31. Salimpour, Y., & Soltanian-Zadeh, H. (2009). Particle filtering of point processes observation with application on the modeling of visual cortex neural spiking activity. In 4th International IEEE/EMBS conference on neural engineering, NER09 (pp. 718–721).Google Scholar
  32. Sewell, G. (1988). The numerical solution of ordinary and partial differential equations. London: Academic.zbMATHGoogle Scholar
  33. Simon, D. (2001). Kalman filtering. Embedded systems programming, 16(6), 72–79.Google Scholar
  34. Spall, J. C. (2003). Introduction to stochastic search and optimization. New York: Wiley.CrossRefzbMATHGoogle Scholar
  35. Stewart, G. W., & Sun, J. G. (1990). Matrix perturbation theory. Boston: Academic.zbMATHGoogle Scholar
  36. Talagrand, O., & Courtier, P. (1987). Variational assimilation of meteorological observations with the adjoint vorticity equation. I: Theory. Quarterly Journal of the Royal Meteorological Society, 113, 1311–1328.CrossRefGoogle Scholar
  37. Xie, L., Liu, Y. J., & Yang, H. Z. (2016). Gradient based and least squares based iterative algorithms for matrix equations \(AXB + CX^TD = F\). Applied Mathematics and Computation, 217(5), 2191–2199.CrossRefzbMATHGoogle Scholar
  38. Zhang, H. M., & Ding, F. (2016). Iterative algorithms for \(X+A^TX^{-1}A=I \) by using the hierarchical identification principle. Journal of the Franklin Institute, 353(5), 1132–1146.MathSciNetCrossRefzbMATHGoogle Scholar
  39. Zhang, H. M., & Yin, C. H. (2017). New proof of the gradient-based iterative algorithm for a complex conjugate and transpose matrix equation. Journal of the Franklin Institute, 354(16), 7585–7603.MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Research DepartmentFrench Naval Oceanographic and Hydrographic Service (SHOM/HOM/REC)ToulouseFrance

Personalised recommendations