Handbook of Combinatorial Optimization pp 259-293 | Cite as

# Neural Networks Approaches for Combinatorial Optimization Problems

## Abstract

Most of the engineering design problems and applications can be formulated as a nonlinear programming problem in which the objective function is nonlinear and has many local optima in its feasible region. It is desirable to find a local optimum that corresponds to the global optimum. The problem of finding the global optimum is known as the global optimization problem. Most such global optimization problems are difficult to solve. The main difficulties in finding the global optimum are that there are no operationally useful optimality conditions for identifying whether a point is indeed a global optimum, except in cases of special structured problems [33] and so it is computationally intensive to obtain the global optimum. Therefore, it is desirable and sometimes necessary to find a near global optimum in a reasonable time rather than obtaining the global optimum.

## Keywords

Energy Function Travel Salesman Problem Travel Salesman Problem Knapsack Problem Combinatorial Optimization Problem## Preview

Unable to display preview. Download preview PDF.

## References

- [1]S. Abe, J. Kawakami, and K. Hirasawa, Solving inequality constrained combinatorial optimization problems by the Hopfield neural networks,
*Neural Networks*Vol. 5 (1992) pp. 663–670.CrossRefGoogle Scholar - [2]D.H. Ackley, G.E.. Hinton, and T.J. Sejnowski, A learning algorithm for Boltzman machines,
*Cognitive Science*Vol. 9 (1985) pp. 147–169.CrossRefGoogle Scholar - [3]N. Ansari, E.S.H. Hou, and Y. Yu, A new method to optimize the satellite broadcasting schedules using the mean field annealing of a Hopfield neural network,
*IEEE Transactions on Neural Networks*Vol. 6 No. 2 (1995) pp. 470–482.CrossRefGoogle Scholar - [4]M. Bengtsson and P. Roivainen, Using the Potts glass for solving the clustering problem,
*International Journal of Neural Systems*Vol. 6 No. 2 (1995) pp. 119–132.CrossRefGoogle Scholar - [5]G. Bilbro, R. Mann, T.K. Miller, W.E. Snyder, D.E. Van Den Bout, and M. White, Optimization by mean field annealing, in D.S. Touretzky (ed.)
*Proceedings of the Annual Conferences on Advances in Neural Information Processing Systems*,*Volume**1*, (Morgan Kaufmann Publishers, 1988 ) pp. 91–98.Google Scholar - [6]A. Bovier and P. Picco,
*Mathematical Aspects of Spin Glasses and Neural Networks*, (Birkhäuser, 1998 ).Google Scholar - [7]N.E.G. Buchler, E.R.P. Zuiderweg, H. Wang, and R.A. Goldstein, Protein heteronuclear NMR assignments using mean-field simulated annealing,
*Journal of Magnetic Resonance*Vol. 125 (1997) pp. 34–42.CrossRefGoogle Scholar - [8]B. Cabon, G. Verfaillie, D. Martinez, and P. Bourret, Using mean field methods for boosting backtrack search in constraint satisfaction problems, in W. Wahlster (ed.)
*12th European Conference on Artificial Intelligence*, ( John Wiley and Sons, Ltd., 1996 ).Google Scholar - [9]D. Chowdhury,
*Spin Glasses and Other Frustrated Systems*, (Princeton University Press, 1986 ).Google Scholar - [10]D. Chowdhury,
*Spin Glasses and Other Frustrated Systems*, (Princeton University Press, 1986 ).Google Scholar - [11]A. Cichocki and R. Unbehauen,
*Neural Networks for Optimization and Signal Processing*, (John Wiley and Sons, 1993 ).Google Scholar - [12]R. Courant and D. Hilbert,
*Methods of Mathematical Physics*, (Inter-science Publishers Inc., 1953 ).Google Scholar - [13]K.A. Dowsland, Simulated annealing, in C.R. Reeves (ed.)
*Modern Heuristic Techniques for Combinatorial Problems*, (John Wiley and Sons, 1993 ) pp. 20–69.Google Scholar - [14]R. Durbin, R. Szeliski, and A.L. Yuille, An analysis of the elastic net approach to the travelling salesman problem,
*Neural Computation Vold*(1989) pp. 348–358.Google Scholar - [15]I.M. Elfadel, Convex potential and their conjugates in analog mean-field optimization,
*Neural Computation*Vol. 7 (1995) pp. 1079–1104.CrossRefGoogle Scholar - [16]S. Elmohamed, P. Coddington, and G. Fox, A comparison of annealing techniques for academic course scheduling, Northeast Parallel Architectures Center (NPAC) technical report SCCS-777, January 25, 1997.Google Scholar
- [17]L. Fang and T. Li, Design of competition-based neural networks for combinatorial optimization,
*International Journal of Neural Systems*Vold No. 3 (1990) pp. 221–235.Google Scholar - [18]L. Fang, W.H. Wilson, and T. Li, Mean field annealing neural net for quadratic assignment, in
*International Neural Network Conference*,*July 9–13*,*Paris*,*France*,(Kluwer Academic Publishers, 1990) pp. 282286.Google Scholar - [19]L. Fausett,
*Fundamentals of Neural Networks: Architectures*,*Algorithms*,*and Applications*, (Prentice-Hall, 1994 ).Google Scholar - [20]L. Faybusovich, Interior point methods and entropy,
*IEEE Conference on Decision and Control*, (1991) pp. 2094–2095.Google Scholar - [21]K.H. Fischer and J.A. Hertz,
*Spin Glasses*, (Cambridge University Press, 1991 ).Google Scholar - [22]C.A. Floudas and P.M. Pardalos,
*Recent Advances in Global Optimization*, (Princeton University Press, 1994 ).Google Scholar - [23]D. Geiger and F. Girosi, Coupled Markov random fields and mean field theory, in D.S. Touretzky (ed.)
*Proceedings of the Annual Conferences on Advances in Neural Information Processing Systems*,*Volume**2*, (Morgan Kaufmann Publishers, 1989 ) pp. 660–667.Google Scholar - [24]L. Gislen, C. Peterson, and B. Soderberg, Teachers and classes with neural networks,
*International Journal of Neural Systems Vol.1*No. 1 (1989) pp. 167–183.CrossRefGoogle Scholar - [25]L. Gislen, C. Peterson, and B. Soderberg, Complex scheduling with Potts neural networks,
*Neural Computation*Vol. 4 (1992) pp. 805–831.CrossRefGoogle Scholar - [26]R.J. Glauber, Time-dependent statistics of the Ising model,
*Journal of Mathematical Physics*Vol. 4 No. 2 (1963) pp. 294–307.MathSciNetMATHCrossRefGoogle Scholar - [27]S. Haykin,
*Neural Networks: A Comprehensive Foundation*, (Macmillan College Publishing Company, 1994 ).Google Scholar - [28]L. Herault and J.-J. Niez, Neural networks and combinatorial optimization: a study of NP-complete graph problems, in E. Gelenbe (ed.)
*Neural Networks: Advances and Applications*, ( Elsevier Science Publishers B. V., 1991 ) pp. 165–213.Google Scholar - [29]T. Hofmann and J.M. Buhmann, Pairwise data clustering by deterministic annealing,
*IEEE Transactions on Pattern Analysis and Machine Intelligence*Vol. 19 No. 1 (1997) pp. 1–14.CrossRefGoogle Scholar - [30]J.J. Hopfield, Neural networks and physical systems with emergent collective computational abilities, in
*Proceedings of the National Academy of Sciences of the U.S.A.*,*Biophysics*, (1982) pp. 2554–2558.Google Scholar - [31]J.J. Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons, in
*Proceedings of the National Academy of Sciences of the U.S.A.*,*Biophysics*, (1984) pp. 3088–3092.Google Scholar - [32]J.J. Hopfield and D.W. Tank, “Neural” computation of decisions in optimization problems,
*Biological Cybernetics*Vol.52 (1985) pp. 141–152.Google Scholar - [33]R. Horst, P.M. Pardalos, and N.V. Thoai,
*Introduction to Global Optimization*, (Kluwer Academic Publishers, 1995 ).Google Scholar - [34]H. Igarashi, A solution to combinatorial optimization problems using a two-layer random field model: Mean-field approximation, in
*World Congress on Neural Networks*,*July**11–15*,*Portland*,*Oregon*,*Volume**1*, ( Lawrance Erlbaum Associates Inc. Publishers, 1993 ) pp. 283–286.Google Scholar - [35]H. Igarashi, A solution for combinatorial optimization problems using a two-layer random field model: Mean-field approximation,
*Systems and Computers in Japan*Vol. 25 No. 8 (1994) pp. 61–71.CrossRefGoogle Scholar - [36]S. Ishii and M.-A. Sato, Chaotic Potts spin model for combinatorial optimization problems,
*Neural Networks*Vol.10 No. 5 (1997) pp. 941963.Google Scholar - [37]I. Kanter and H. Sompolinsky, Graph optimisation problems and the Potts glass,
*Journal of Physics A: Math. Gen*. Vol. 20 (1987) pp. L673 - L679.MathSciNetCrossRefGoogle Scholar - [38]S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi, Optimization by simulated annealing,
*Science*Vol. 220 No. 4598 (1983) pp. 671–680.MathSciNetMATHCrossRefGoogle Scholar - [39]J.J. Kosowsky and A.L. Yuille, The invisible hand algorithm: Solving the assignment problem with statistical physics,
*Neural Networks*Vol. 7 No. 3 (1994) pp. 477–490.MATHCrossRefGoogle Scholar - [40]N. Kurita and K.-I. Funahashi, On the Hopfield neural networks and mean field theory,
*Neural Networks*Vol. 9 No. 9 (1996) pp. 1531–1540.MATHCrossRefGoogle Scholar - [41]T. Kurokawa and S. Kozuka, Use of neural networks for the optimum frequency assignment problem,
*Electronics and Communications in Japan*,*Part**1*Vol. 77 No. 11 (1994) pp. 106–116.Google Scholar - [42]M. Lagerholm, C. Peterson, and B. Soderberg, Airline crew scheduling with Potts neurons,
*Neural Computation*Vol. 9 (1997) pp. 1589–1599.CrossRefGoogle Scholar - [43]K.-C. Lee and Y. Takefuji, Maximum clique problems: Part 1, in Y. Takefuji and J. Wang (eds.)
*Neural Computing for Optimization and Combinatorics*, (World Scientific, 1996 ) pp. 31–61.CrossRefGoogle Scholar - [44]K.-C. Lee and Y. Takefuji, Maximum clique problems: Part 2, in Y. Takefuji and J. Wang (eds.)
*Neural Computing for Optimization and Combinatorics*, (World Scientific, 1996 ) pp. 63–77.Google Scholar - [45]B.C. Levy and M.B. Adams, Global optimization with stochastic neural networks, in
*Neural Networks for Optimization and Signal Processing*,*Proceedings of the First International Conference on Neural Networks*,*San Diego*,*(1987)*pp. 681–690.Google Scholar - [46]W.A. Little, The existence of persistent states in the brain,
*Mathematical Biosciences*Vol. 19 (1974) pp. 101–120.MATHCrossRefGoogle Scholar - [47]C.-K. Looi, Neural network methods in combinatorial optimization,
*Computers in Operations Research*Vol.19 No. 3 /4 (1992) pp. 191–208.Google Scholar - [48]S. Matsuda, Set-theoretic comparison of mapping of combinatorial optimization problems to Hopfield neural networks,
*Systems and Computers in Japan*Vol. 27 No. 6 (1996) pp. 45–59.CrossRefGoogle Scholar - [49]W.S. McCullough and W. Pitts, A logical calculus of the ideas immanent in nervous activity,
*Bulletin of Mathematical Biophysics*Vol. 5 (1943) pp. 115–133.CrossRefGoogle Scholar - [50]I.I. Melamed, Neural networks and combinatorial optimization,
*Automation and Remote Control*Vol. 55 No. 11 (1994) pp. 1553–1584.MathSciNetMATHGoogle Scholar - [51]M. Mezard, G. Parisi, and M.A. Virasoro,
*Spin Glass Theory and Beyond*, (World Scientific, 1987 ).Google Scholar - [52]B. Muller and J. Reinhardt,
*Neural Networks: An Introduction*, (Springer-Verlag, 1990 ).Google Scholar - [53]M. Ohlsson, C. Peterson, and B. Soderberg, Neural networks for optimization problems with inequality constraints: The knapsack problem,
*Neural Computation*Vol. 5 (1993) pp. 331–339.CrossRefGoogle Scholar - [54]M. Ohlsson and H. Pi, A study of the mean field approach to knapsack problems,
*Neural Networks*Vol. 10 No. 2 (1997) pp. 263–271.CrossRefGoogle Scholar - [55]C.H. Papadimitriou and K. Steiglitz,
*Combinatorial Optimization: Algorithms and Complexity*, (Prentice-Hall, 1982 ).Google Scholar - [56]C. Peterson, Parallel distributed approaches to combinatorial optimization: Benchmark studies on traveling salesman problem,
*Neural Computation*Vol. 2 (1990) pp. 261–269.CrossRefGoogle Scholar - [57]C. Peterson, Mean field theory neural networks for feature recognition, content addressable memory and optimization,
*Connection Science*Vol. 3 No. 1 (1991) pp. 3–33.CrossRefGoogle Scholar - [58]C. Peterson, Solving optimization problems with mean field methods,
*Physica A*Vol. 200 (1993) pp. 570–580.CrossRefGoogle Scholar - [59]C. Peterson and J.R. Anderson, A mean field theory learning algorithm for neural networks,
*Complex Systems Vol.1 (1987)*pp. 995–1019.Google Scholar - [60]C. Peterson and J.R. Anderson, Neural networks and NP-complete optimization problems; a performance study on the graph bisection problem,
*Complex Systems*Vol. 2 (1988) pp. 59–89.MathSciNetMATHGoogle Scholar - [61]C. Peterson and B. Soderberg, A new method for mapping optimization problems onto neural networks,
*International Journal of Neural systems*Vol. 1 No. 1 (1989) pp. 3–22.CrossRefGoogle Scholar - [62]C. Peterson and B. Soderberg, Artificial neural networks, in C.R. Reeves (ed.)
*Modern Heuristic Techniques for Combinatorial Problems*, (John Wiley and Sons, 1993 ) pp. 197–242.Google Scholar - [63]C. Peterson and B. Soderberg, Artificial neural networks, in E. Aarts and J.K. Lenstra (eds.)
*Local Search in Combinatorial Optimization*, (John Wiley and Sons, 1997 ) pp. 177–213.Google Scholar - [64]F. Qian and H. Hirata, A parallel computation based on mean field theory for combinatorial optimization and Boltzman machines,
*Systems and Computers in Japan*Vol. 25 No. 12 (1994) pp. 86–97.Google Scholar - [65]J. Ramanujam and P. Sadayappan, Mapping combinatorial optimization problems onto neural networks,
*Information Sciences*Vol. 82 (1995) pp. 239–255.MathSciNetCrossRefGoogle Scholar - [66]C.R. Reeves,
*Modern Heuristic Techniques for Combinatorial Problems*, (John Wiley and Sons, 1993 ).Google Scholar - [67]L.E. Reichl,
*A Modern Course in Statistical Physics*, (University of Texas Press, 1980 ).Google Scholar - [68]D.E. Rumelhart, G.E. Hinton, and R.J. Williams, Learning representations by back-propagating errors,
*Nature (London)*Vol. 323 (1986) pp. 533–536.CrossRefGoogle Scholar - [69]M.-A. Sato and S. Ishii, Bifurcations in mean field theory,
*Physical Review E*Vol. 53 No. 5 (1996) pp. 5153–5168.CrossRefGoogle Scholar - [70]G.M. Shim, D. Kim, and M.Y. Choi, Potts-glass model of layered feed-forward neural networks,
*Physical Review A*Vol. 45 No. 2 (1992) pp. 1238–1248.MathSciNetCrossRefGoogle Scholar - [71]P. Stolorz,
*Merging constrained optimisation with deterministic annealing to “solve” combinatorially hard problems*, in J.E. Moody, S.J. Hanson, and R.P. Lippmann (eds.) Advances in Neural Information Processing Systems, ( Chapman and Hall, 1991 ) pp. 1025–1032.Google Scholar - [72]Y. Uesaka, Mathematical basis of neural networks for combinatorial optirnization problems,
*Optoelectronics–Devices and Technologies*Vol. 8 No. 1 (1993) pp. 1–9.Google Scholar - [73]K. Urahama, Mathematical programming formulation for neural combinatorial optimization algorithms,
*Electronics and Communications in Japan*Vol. 78 No. 9 (1995) pp. 67–75.CrossRefGoogle Scholar - [74]K. Urahama and S.-I. Ueno, A gradient system solution to Potts mean field equations and its electronic implementation,
*International Journal of Neural Systems*Vol. 4 No. 1 (1993) pp. 27–34.CrossRefGoogle Scholar - [75]K. Urahama and T. Yamada, Constrained Potts mean field systems and their electronic implementation,
*International Journal of Neural Systems*Vol. 5 No. 3 (1994) pp. 229–239.CrossRefGoogle Scholar - [76]D.E. Van Den Bout and T.K. Miller, Graph partitioning using annealed networks,
*IEEE Transactions on Neural Networks*Vol.]. No. 2 (1990) pp. 192–203.CrossRefGoogle Scholar - [77]P.J.M. Van Laarhoven,
*Theoretical and Computational Aspects of Simulated Annealing*, (Stichting Mathematisch Centrum, 1988 ).Google Scholar - [78]J. Wang, Deterministic neural networks for combinatorial optimization, in O.M. Omidvar (ed.)
*Progress in Neural Networks*, (Ablex, 1994 ) pp. 319–340.Google Scholar - [79]F.Y. Wu, The Potts model,
*Reviews of Modern Physics*Vol. 54 No. 1 (1982) pp. 235–268.MathSciNetCrossRefGoogle Scholar - [80]L. Xu and A.L. Yuille, Robust principal component analysis by self-organizing rules based on statistical physics approach,
*IEEE Transactions on Neural Networks*Vol. 6 No. 1 (1995) pp. 131–143.CrossRefGoogle Scholar - [81]A.L. Yuille, Generalized Deformable models, statistical physics, and matching problems,
*Neural Computation*Vol. 2 (1990) pp. 1–24.CrossRefGoogle Scholar - [82]A.L. Yuille and J.J. Kosowsky, Statistical physics algorithms that converge, in R.J. Mammone (ed.)
*Artificial Neural Networksfor Speech and*Vision,(Chapman and Hall, 1993 ) pp. 19–36.Google Scholar - [83]A.L. Yuille and J.J. Kosowsky, Statistical physics algorithms that converge,
*Neural Computation*Vol. 6 (1994) pp. 341–356.CrossRefGoogle Scholar - [84]A.L. Yuille, P. Stolorz, and J. Utans, Statistical physics, mixtures of distributions, and the EM algorithm,
*Neural Computation*Vol. 6 (1994) pp. 334–340.CrossRefGoogle Scholar