Part of the Studies in Computational Intelligence book series (SCI, volume 508)


A brief overview of the Hopfield Neural Network is presented to emphasize the benefits of implementing neural circuits in actual hardware. The limitations associated with the standard Hopfield Network are then discussed, and the need for a better architecture is highlighted. A neural architecture in which the nature of feedback is non-linear, as opposed to the linear feedback in Hopfield Networks, is explained. It is further demonstrated that such non-linear feedback neural networks are capable of providing better solutions to combinatorial problems like graph colouring and sorting. The chapter ends with an overview of pertinent technical literature.


Energy Function Linear Programming Problem Travelling Salesman Problem Recurrent Neural Network Operational Amplifier 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Rahman, S.A., Jayadeva, C., Dutta Roy, S.C.: Neural network approach to graph colouring. Electron. Lett. 35(14), 1173–1175 (1999)CrossRefGoogle Scholar
  2. 2.
    Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Hopfield, J.J., Tank, D.W.: “Neural” computation of decisions in optimization problems. Biol. Cybernet. 52, 141–152 (1985)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Tank, D., Hopfield, J.: Simple ‘neural’ optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit. IEEE Trans. Circ. Syst. 33(5), 533–541 (1986)CrossRefGoogle Scholar
  5. 5.
    Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. 81(10), 3088–3092 (1984)CrossRefGoogle Scholar
  6. 6.
    Vidyasagar, M.: Location and stability of the high-gain equilibria of nonlinear neural networks. IEEE Trans. Neural Netw. 4(4), 660–672 (1993)CrossRefGoogle Scholar
  7. 7.
    Wilson, G.V., Pawley, G.S.: On the stability of the travelling salesman problem algorithm of hopfield and tank. Biol. Cybern. 58(1), 63–70 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Kamgar-Parsi, B., Kamgar-Parsi, B.: Dynamical stability and parameter selection in neural optimization. In Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 566–571, Baltimore, USA (1992)Google Scholar
  9. 9.
    Aiyer, S.V.B., Niranjan, M., Fallside, F.: A theoretical investigation into the performance of the hopfield model. IEEE Trans. Neural Netw. 1(2), 204–215 (1990)CrossRefGoogle Scholar
  10. 10.
    Gee, A.H.: Problem Solving with Optimization Networks. PhD thesis, University of Cambridge, UK (1993)Google Scholar
  11. 11.
    Van den Bout, D.E., Miller, T.K.: A traveling salesman objective function that works. In: Proceedings of IEEE International Conference on Neural Networks, pp. 299–303, San Diego, USA (1988)Google Scholar
  12. 12.
    Nonaka, H., Kobayashi, Y.: Sub-optimal solution screening in optimization by neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 606–611, Baltimore, USA (1992)Google Scholar
  13. 13.
    Foo, Y.P.S., Szu, H.: Solving large-scale optimization problems by divide-and-conquer neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 507–511, Washington, DC, USA (1989)Google Scholar
  14. 14.
    Lo, J.T.-H.: A new approach to global optimization and its applications to neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 600–605, Baltimore, USA (1992)Google Scholar
  15. 15.
    Amartur, S.C., Piraino, D., Takefuji, Y.: Optimization neural networks for the segmentation of magnetic resonance images. IEEE Trans. Med. Imaging 11(2), 215–220 (1992)CrossRefGoogle Scholar
  16. 16.
    Chen, L., Aihara, K.: Chaotic simulated annealing by a neural network model with transient chaos. Neural Netw. 8(6), 915–930 (1995)CrossRefGoogle Scholar
  17. 17.
    Arabas, J., Kozdrowski, S.: Applying an evolutionary algorithm to telecommunication network design. IEEE Trans. Evol. Comput. 5(4), 309–322 (2001)CrossRefGoogle Scholar
  18. 18.
    Rahman, S.A.: A nonlinear synapse neural network and its applications. PhD thesis, Department of Electrical Engineering, Indian Institute of Technology, Delhi, India (2007)Google Scholar
  19. 19.
    Jayadeva, Rahman, S.A.: A neural network with O(N) neurons for ranking N numbers in O(1/N) time. IEEE Trans. Circ. Syst. I: Regular Papers 51(10):2044–2051 (2004)Google Scholar
  20. 20.
    Jensen, T.R., Toft, B.: Graph Coloring Problems. John Wiley & Sons, New York (1994)CrossRefGoogle Scholar
  21. 21.
    Gassen, D.W., Carothers, J.D.: Graph color minimization using neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 1541–1544, Nagoya, Japan (1993)Google Scholar
  22. 22.
    El-Fishawy, N.A., Hadhood, M.M., Elnoubi, S., EL-Sersy, W.: A modified hopfield neural network algorithm for cellular radio channel assignment. In: Proceedings of TENCON, pp. 213–216, Kuala Lumpur, Malaysia (2000)Google Scholar
  23. 23.
    LaPaugh, A.S.: VLSI Layout algorithms. Chapman & Hall/CRC, Boca Raton (2010)Google Scholar
  24. 24.
    Yue, T.-W., Lee, Z. Z.: A Q’tron neural-network approach to solve the graph coloring problems. In: Proceedings of 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI), pp. 19–23, Paris, France (2007)Google Scholar
  25. 25.
    Graph Coloring Instances. Accessed 10 February 2012
  26. 26.
    Gustafson, J.L.: The quest for linear equation solvers and the invention of electronic digital computing. In: Proceedings of IEEE John Vincent Atanasoff 2006 International Symposium on Modern Computing (JVA’06), pp. 10–16, Sofia (2006)Google Scholar
  27. 27.
    Yong, L.L.: Jiu Zhang Suanshu (nine chapters on the mathematical art): an overview. Arch. Hist. Exact Sci. 47(1), 1–51 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Kleiner, I.: History of Linear Algebra, pp. 79–89. Birkhauser, Basel (2007)Google Scholar
  29. 29.
    Peters, G., Wilkinson, J.H., Martin, R.S.: Iterative refinement of the solution of a positive definite system of equations. Numer. Math. 8, 203–216 (1966)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    White, S.: A brief history of computing—complete timeline. Accessed 30 October 2012
  31. 31.
    Kung, S.: VLSI array processors. IEEE ASSP Mag. 2(3), 4–22 (1985)CrossRefGoogle Scholar
  32. 32.
    Wilburn, V.C., Ko, H.-L., Alexander, W.E.: An algorithm and architecture for the parallel solution of systems of linear equations. In: Proceedings of IEEE Fifteenth Annual International Phoenix Conference on Computer and Communications, pp. 392–398, Scottsdale, USA, March 1996Google Scholar
  33. 33.
    Wilbur, J.B.: The mechanical solution of simultaneous equations. J. Franklin Inst. 222, 715–724 (1936)CrossRefGoogle Scholar
  34. 34.
    Walker, R.M.: An analogue computer for the solution of linear simultaneous equations. Proc. IRE Waves Electrons Section 37(12), 1467–1473 (1949)Google Scholar
  35. 35.
    Ackerman, S.: Precise solutions of linear simultaneous equations using a low cost analog. Rev. Sci. Instrum. 22(10), 746–748 (1951)MathSciNetCrossRefGoogle Scholar
  36. 36.
    Many, A., Oppenheim, U., Amitsur, S.: An electrical computer for the solution of linear simultaneous equations. Rev. Sci. Instrum. 24(2), 112–116 (1953)MathSciNetCrossRefGoogle Scholar
  37. 37.
    Mitra, S.K.: Electrical analog computing machine for solving linear equations and related problems. Rev. Sci. Instrum. 26(5), 453–457 (1955)MathSciNetCrossRefGoogle Scholar
  38. 38.
    Hutchinson, J., Koch, C., Luo, J., Mead, C.: Computing motion using analog and binary resistive networks. Computer 21(3), 52–63 (1988)CrossRefGoogle Scholar
  39. 39.
    Jang, J., Lee, S., Shin, S.: An Optimization Network for Matrix Inversion, pp. 397–401. American Institute of Physics, New York (1988)Google Scholar
  40. 40.
    Chakraborty, K., Mehrotra, K., Mohan, C.K., Ranka, S.: An optimization network for solving a set of simultaneous linear equations. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 516–521, Baltimore, USA, June 1992Google Scholar
  41. 41.
    Wang, J.: Electronic realization of recurrent neural network for solving simultaneous linear equations. Electron. Lett. 28(5), 493–495 (1992)CrossRefGoogle Scholar
  42. 42.
    Cichocki, A., Unbehauen, R.: Neural networks for solving systems of linear equations and related problems. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 39(2), 124–138 (1992)CrossRefzbMATHGoogle Scholar
  43. 43.
    Cichocki, A., Unbehauen, R.: Neural networks for solving systems of linear equations. Part ii. Minimax and least absolute value problems. IEEE Trans. Circ. Syst. II: Analog Digital Signal Processing 39(9), 619–633 (1992)CrossRefzbMATHGoogle Scholar
  44. 44.
    Wang, J., Li, H.: Solving simultaneous linear equations using recurrent neural networks. Inf. Sci. Intell. Syst. 76(3), 255–277 (1994)zbMATHGoogle Scholar
  45. 45.
    Zhang, K., Ganis, G., Sereno, M.I.: Anti-hebbian synapses as a linear equation solver. In: Proceedings of International Conference on Neural Networks, pp. 387–389, Houston, TX, USA, June 1997Google Scholar
  46. 46.
    Xia, Y., Wang, J., Hung, D.L.: Recurrent neural networks for solving linear inequalities and equations. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 46(4), 452–462 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Jiang, D.: Analog computing for real-time solution of time-varying linear equations. In: Proceedings of International Conference on Communications, Circuits and Systems (ICCCAS), pp. 1367–1371, June 2004Google Scholar
  48. 48.
    Wang, X., Ziavras, S.G.: Parallel direct solution of linear equations on FPGA-based machines. In: Proceedings of International Parallel and Distributed Processing Symposium (IPDPS’03), pp. 113–120, Nice, France, April 2003Google Scholar
  49. 49.
    Kreyszig, E.: Advanced Engineering Mathematics, 8th edn. Wiley-India (2006)Google Scholar
  50. 50.
    Kambo, N.S.: Mathematical Programming Techniques, revised edn. Affilated East-West Press Pvt Ltd., New Delhi (1991)Google Scholar
  51. 51.
    Anguita, D., Boni, A., Ridella, S.: A digital architecture for support vector machines: theory, algorithm, and FPGA implementation. IEEE Trans. Neural Netw. 14(5), 993–1009 (2003)CrossRefGoogle Scholar
  52. 52.
    Cichocki, A., Unbehauen, R.: Neural Networks for Optimization and Signal Processing. Wiley, Chichester (1993)zbMATHGoogle Scholar
  53. 53.
    Iqbal, K., Pai, Y.C.: Predicted region of stability for balance recovery: motion at the knee joint can improve termination of forward movement. J. Biomech. 13(12), 1619–1627 (2000)CrossRefGoogle Scholar
  54. 54.
    Zhang, Y.: Towards piecewise-linear primal neural networks for optimization and redundant robotics. In: Proceedings of IEEE International Conference on Networking, Sensing and Control, pp. 374–379, Fort Lauderdale, Florida, USA (2006)Google Scholar
  55. 55.
    Zhang, Y., Leithead, W.E.: Exploiting hessian matrix and trust-region algorithm in hyperparameters estimation of gaussian process. Appl. Math. Comput. 171(2), 1264–1281 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  56. 56.
    Young, M.R.: A minimax portfolio selection rule with linear programming solution. Manage. Sci. 44(5), 673–683 (1988)CrossRefGoogle Scholar
  57. 57.
    Bixby, R.E., Gregory, J.W., Lustig, I.J., Marsten, R.E., Shanno, D.F.: Very large-scale linear programming: a case study in combining interior point and simplex methods. Oper. Res. 40(5), 885–897 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Peidro, D., Mula, J., Jimenez, M., del Mar Botella, M.: A fuzzy linear programming based approach for tactical supply chain planning in an uncertainty environment. Eur. J. Oper. Res. 205(16), 65–80 (2010)CrossRefzbMATHGoogle Scholar
  59. 59.
    Chertkov, M., Stepanov, M.G.: An efficient pseudocodeword search algorithm for linear programming decoding of LDPC codes. IEEE Trans. Inf. Theory 54(4), 1514–1520 (2008)MathSciNetCrossRefGoogle Scholar
  60. 60.
    ChvÃtal, V., Cook, W., Dantzig, G.B., Fulkerson, D.R., Johnson, S.M.: Solution of a Large-Scale Traveling-Salesman Problem, pp. 7–28. Springer, Berlin, (2010)Google Scholar
  61. 61.
    Atkociunas, J.: Quadratic programming for degenerate shakedown problems of bar structures. Mech. Res. Commun. 23(2), 195–206 (1996)CrossRefzbMATHGoogle Scholar
  62. 62.
    Bartlett, R.A., Wachter, A., Biegler, L.T.: Active set vs. interior point strategies for model predictive control. In: Proceedings of American Control Conference, pp. 4229–4233, Chicago, USA, June 2000Google Scholar
  63. 63.
    Maier, G., Munro, J.: Mathematical programming applications to engineering plastic analysis. Appl. Mech. Rev. 35, 1631–1643 (1982)Google Scholar
  64. 64.
    Nordebo, S., Zang, Z., Claesson, I.: A semi-infinite quadratic programming algorithm with applications to array pattern synthesis. IEEE Trans. Circ. Syst. II: Analog and Digital Signal Processing 48(3), 225–232 (2001)CrossRefGoogle Scholar
  65. 65.
    Schonherr, S.: Quadratic programming in geometric optimization: theory, implementation, and applications. PhD thesis, Swiss Federal Institute of Technology, Zurich (2002)Google Scholar
  66. 66.
    Borguet, S., Leonard, O.: A quadratic programming framework for constrained and robust jet engine health monitoring. Progr. Propul. Phys. 1, 669–692 (2009)Google Scholar
  67. 67.
    Dembo, R.S., Tulowitzki, U.: Computing equilibria on large multicommodity networks: an application of truncated quadratic programming algorithms. Networks 18(4), 273–284 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  68. 68.
    Censor, Y., Elfving, T.: New methods for linear inequalities. Linear Algebra Appl. 42, 199–211 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  69. 69.
    Wen, U.-P., Lan, K.-M., Shih, H.-S.: A review of hopfield neural networks for solving mathematical programming problems. Eur. J. Oper. Res. 198(3), 675–687 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  70. 70.
    Kennedy, M.P., Chua, L.O.: Neural networks for nonlinear programming. IEEE Trans. Circ. Syst. 35(5), 554–562 (1988)MathSciNetCrossRefGoogle Scholar
  71. 71.
    Rodriguez-Vazquez, A., Rueda, A., Huertas, J.L., Dominguez-Castro, R.: Switched-capacitor neural networks for linear programming. Electron. Lett. 24(8), 496–498 (1988)CrossRefGoogle Scholar
  72. 72.
    Rodriguez-Vazquez, A., Dominguez-Castro, R., Rueda, A., Huertas, J.L., Sanchez-Sinencio, E.: Nonlinear switched capacitor ‘neural’ networks for optimization problems. IEEE Trans. Circ. Syst. 37(3), 384–398 (1990)MathSciNetCrossRefGoogle Scholar
  73. 73.
    Lan, K.-M., Wen, U.-P., Shih, H.-S., Lee, E.S.: A hybrid neural network approach to bilevel programming problems. Appl. Math. Lett. 20(8), 880–884 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  74. 74.
    Maa, C.-Y., Shanblatt, M.A.: Linear and quadratic programming neural network analysis. IEEE Trans. Neural Netw. 3(4), 580–594 (1992)CrossRefGoogle Scholar
  75. 75.
    Chong, E.K.P., Hui, S., Zak, S.H.: An analysis of a class of neural networks for solving linear programming problems. IEEE Trans. Autom. Control 44(11), 1995–2006 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  76. 76.
    Zhu, X., Zhang, S., Constantinides, A.G.: Lagrange neural networks for linear programming. J. Parallel Distrib. Comput. 14(3), 354–360 (1992)CrossRefGoogle Scholar
  77. 77.
    Xia, Y., Wang, J.: Neural network for solving linear programming problems with bounded variables. IEEE Trans. Neural Netw. 6(2), 515–519 (1995)CrossRefGoogle Scholar
  78. 78.
    Malek, A., Yari, A.: Primal-dual solution for the linear programming problems using neural networks. Appl. Math. Comput. 167(1), 198–211 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  79. 79.
    Ghasabi-Oskoei, H., Malek, A., Ahmadi, A.: Novel artificial neural network with simulation aspects for solving linear and quadratic programming problems. Comput. Math. Appl. 53(9), 1439–1454 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  80. 80.
    Wang, J.: Recurrent neural network for solving quadratic programming problems with equality constraints. Electron. Lett. 28(14), 1345–1347 (1992)CrossRefGoogle Scholar
  81. 81.
    Forti, M., Tesi, A.: New conditions for global stability of neural networks with application to linear and quadratic programming problems. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 42(7), 354–366 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  82. 82.
    Wu, X.-Y., Xia, Y.-S., Li, J., Chen, W.-K.: A high-performance neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7(3), 643–651 (1996)CrossRefGoogle Scholar
  83. 83.
    Xia, Y.: A new neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7(6), 1544–1548 (1996)CrossRefGoogle Scholar
  84. 84.
    Tao, Q., Cao, J., Sun, D.: A simple and high performance neural network for quadratic programming problems. Appl. Math. Comput. 124(2), 251–260 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  85. 85.
    Liu, Q., Wang, J.: A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming. IEEE Trans. Neural Netw. 19(4), 558–570 (2008)CrossRefGoogle Scholar
  86. 86.
    Gould, N.I.M., Toint, P.L.: A quadratic programming bibliography. Technical report, RAL Numerical Analysis Group, March 2010Google Scholar

Copyright information

© Springer India 2014

Authors and Affiliations

  1. 1.Department of Electronics EngineeringAligarh Muslim UniversityAligarhIndia

Personalised recommendations