Abstract
In this chapter, we present four different Newton SPSA algorithms for the long-run average cost objective. The random perturbation technique requiring zero-mean, bounded, symmetric perturbation random variables having a common distribution and mutually independent of one another is used to derive the various Hessian estimates. These algorithms require four, three, two and one simulations, respectively, and are seen to be efficient in practice. Note that, though we discuss Newton SPSA algorithms only for the long-run average cost setting here, all the Hessian estimation schemes discussed below can also be used for the expected cost setting as well.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)
Bhatnagar, S.: Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization. ACM Transactions on Modeling and Computer Simulation 15(1), 74–107 (2005)
Borkar, V.S.: Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge University Press and Hindustan Book Agency (Jointly Published), Cambridge and New Delhi (2008)
Fabian, V.: Stochastic approximation. In: Rustagi, J.J. (ed.) Optimizing Methods in Statistics, pp. 439–470. Academic Press, New York (1971)
Kushner, H.J., Clark, D.S.: Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer, New York (1978)
Kushner, H.J., Yin, G.G.: Stochastic Approximation Algorithms and Applications. Springer, New York (1997)
Lasalle, J.P., Lefschetz, S.: Stability by Liapunov’s Direct Method with Applications. Academic Press, New York (1961)
Patro, R.K., Bhatnagar, S.: A probabilistic constrained nonlinear optimization framework to optimize RED parameters. Performance Evaluation 66(2), 81–104 (2009)
Ruppert, D.: A Newton-Raphson version of the multivariate Robbins-Monro procedure. Annals of Statistics 13, 236–245 (1985)
Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Auto. Cont. 37(3), 332–341 (1992)
Spall, J.C.: Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans. Autom. Contr. 45, 1839–1853 (2000)
Spall, J.C.: Feedback and weighting mechanisms for improving Jacobian estimates in the adaptive simultaneous perturbation algorithm. IEEE Transactions on Automatic Control 54(6), 1216–1229 (2009)
Zhu, X., Spall, J.C.: A modified second-order SPSA optimization algorithm for finite samples. Int. J. Adapt. Control Signal Process. 16, 397–409 (2002)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Springer-Verlag London
About this chapter
Cite this chapter
Bhatnagar, S., Prasad, H., Prashanth, L. (2013). Newton-Based Simultaneous Perturbation Stochastic Approximation. In: Stochastic Recursive Algorithms for Optimization. Lecture Notes in Control and Information Sciences, vol 434. Springer, London. https://doi.org/10.1007/978-1-4471-4285-0_7
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4285-0_7
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4284-3
Online ISBN: 978-1-4471-4285-0
eBook Packages: EngineeringEngineering (R0)