Abstract
When one wishes to solve optimization problems by simulated annealing, the naive mean-field approximation provides a practical way of doing it. Extensions of the naive approximation by including higher-order terms have been proposed in the prospect of improving accuracy of the approximation. It has been reported, however, that higher-order approximations do not work well, especially in low temperature regions. We present an analytical argument and a geometrical view on this contradictory observation based on information-geometry, and give an intuitive explanation as to why the naive approximation does work well when it is applied to solving optimization problems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi, “Optimization by simulated annealing,” Science, vol. 220, no. 4598, pp. 671–680, 1983.
C. Peterson and J. R. Anderson, “A mean field theory learning algorithm for neural networks,” Complex Syst., vol. 1, pp. 995–1019, 1987.
T. Tanaka, “Mean field theory of Boltzmann machine learning,” Phys. Rev. E, vol. 58, no. 2, pp. 2302–2310, Aug. 1998.
C. C. Galland, “The limitations of deterministic Boltzmann machine learning,” Network, vol. 4, no. 3, pp. 355–379, 1993.
H. J. Kappen and F. B. Rodríguez, “Efficient learning in Boltzmann machines using linear response theory,” Neural Computation, vol. 10, no. 5, pp. 1137–1156, 1998.
H. J. Kappen and F. B. Rodríguez, “Boltzmann machine learning using mean field theory and linear response correction,” in M. I. Jordan, M. J. Kearns, and S. A. Solla (Eds.), Advances in Neural Information Processing Systems 10, pp. 280-286, The MIT Press, 1998.
T. Hofmann and J. M. Buhmann, “Pairwise data clustering by deterministic annealing,” IEEE Trans. Patt. Anal. & Machine Intell., vol. 19, no. 1, pp. 1–14, Jan. 1997; Errata, ibid., vol. 19, no. 2, p. 197, Feb. 1997.
T. Tanaka, “A theory of mean field approximation,” M. S. Kearns, S. A. Solla, and D. A. Cohn (Eds.), Advances in Neural Information Processing Systems 11, The MIT Press, pp. 351–357, 1999.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag London
About this paper
Cite this paper
Tanaka, T. (2000). Geometrical View on Mean-Field Approximation for Solving Optimization Problems. In: Suzuki, Y., Ovaska, S., Furuhashi, T., Roy, R., Dote, Y. (eds) Soft Computing in Industrial Applications. Springer, London. https://doi.org/10.1007/978-1-4471-0509-1_34
Download citation
DOI: https://doi.org/10.1007/978-1-4471-0509-1_34
Publisher Name: Springer, London
Print ISBN: 978-1-4471-1155-9
Online ISBN: 978-1-4471-0509-1
eBook Packages: Springer Book Archive