Skip to main content
SpringerLink
Log in

Background

  • Chapter
  • First Online:
Non-Linear Feedback Neural Networks

Part of the book series: Studies in Computational Intelligence ((SCI,volume 508))

  • 1266 Accesses

Abstract

A brief overview of the Hopfield Neural Network is presented to emphasize the benefits of implementing neural circuits in actual hardware. The limitations associated with the standard Hopfield Network are then discussed, and the need for a better architecture is highlighted. A neural architecture in which the nature of feedback is non-linear, as opposed to the linear feedback in Hopfield Networks, is explained. It is further demonstrated that such non-linear feedback neural networks are capable of providing better solutions to combinatorial problems like graph colouring and sorting. The chapter ends with an overview of pertinent technical literature.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Rahman, S.A., Jayadeva, C., Dutta Roy, S.C.: Neural network approach to graph colouring. Electron. Lett. 35(14), 1173–1175 (1999)

    Article  Google Scholar 

  2. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982)

    Article  MathSciNet  Google Scholar 

  3. Hopfield, J.J., Tank, D.W.: “Neural” computation of decisions in optimization problems. Biol. Cybernet. 52, 141–152 (1985)

    MathSciNet  MATH  Google Scholar 

  4. Tank, D., Hopfield, J.: Simple ‘neural’ optimization networks: an A/D converter, signal decision circuit, and a linear programming circuit. IEEE Trans. Circ. Syst. 33(5), 533–541 (1986)

    Article  Google Scholar 

  5. Hopfield, J.J.: Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. 81(10), 3088–3092 (1984)

    Article  Google Scholar 

  6. Vidyasagar, M.: Location and stability of the high-gain equilibria of nonlinear neural networks. IEEE Trans. Neural Netw. 4(4), 660–672 (1993)

    Article  Google Scholar 

  7. Wilson, G.V., Pawley, G.S.: On the stability of the travelling salesman problem algorithm of hopfield and tank. Biol. Cybern. 58(1), 63–70 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  8. Kamgar-Parsi, B., Kamgar-Parsi, B.: Dynamical stability and parameter selection in neural optimization. In Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 566–571, Baltimore, USA (1992)

    Google Scholar 

  9. Aiyer, S.V.B., Niranjan, M., Fallside, F.: A theoretical investigation into the performance of the hopfield model. IEEE Trans. Neural Netw. 1(2), 204–215 (1990)

    Article  Google Scholar 

  10. Gee, A.H.: Problem Solving with Optimization Networks. PhD thesis, University of Cambridge, UK (1993)

    Google Scholar 

  11. Van den Bout, D.E., Miller, T.K.: A traveling salesman objective function that works. In: Proceedings of IEEE International Conference on Neural Networks, pp. 299–303, San Diego, USA (1988)

    Google Scholar 

  12. Nonaka, H., Kobayashi, Y.: Sub-optimal solution screening in optimization by neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 606–611, Baltimore, USA (1992)

    Google Scholar 

  13. Foo, Y.P.S., Szu, H.: Solving large-scale optimization problems by divide-and-conquer neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 507–511, Washington, DC, USA (1989)

    Google Scholar 

  14. Lo, J.T.-H.: A new approach to global optimization and its applications to neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 600–605, Baltimore, USA (1992)

    Google Scholar 

  15. Amartur, S.C., Piraino, D., Takefuji, Y.: Optimization neural networks for the segmentation of magnetic resonance images. IEEE Trans. Med. Imaging 11(2), 215–220 (1992)

    Article  Google Scholar 

  16. Chen, L., Aihara, K.: Chaotic simulated annealing by a neural network model with transient chaos. Neural Netw. 8(6), 915–930 (1995)

    Article  Google Scholar 

  17. Arabas, J., Kozdrowski, S.: Applying an evolutionary algorithm to telecommunication network design. IEEE Trans. Evol. Comput. 5(4), 309–322 (2001)

    Article  Google Scholar 

  18. Rahman, S.A.: A nonlinear synapse neural network and its applications. PhD thesis, Department of Electrical Engineering, Indian Institute of Technology, Delhi, India (2007)

    Google Scholar 

  19. Jayadeva, Rahman, S.A.: A neural network with O(N) neurons for ranking N numbers in O(1/N) time. IEEE Trans. Circ. Syst. I: Regular Papers 51(10):2044–2051 (2004)

    Google Scholar 

  20. Jensen, T.R., Toft, B.: Graph Coloring Problems. John Wiley & Sons, New York (1994)

    Book  Google Scholar 

  21. Gassen, D.W., Carothers, J.D.: Graph color minimization using neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 1541–1544, Nagoya, Japan (1993)

    Google Scholar 

  22. El-Fishawy, N.A., Hadhood, M.M., Elnoubi, S., EL-Sersy, W.: A modified hopfield neural network algorithm for cellular radio channel assignment. In: Proceedings of TENCON, pp. 213–216, Kuala Lumpur, Malaysia (2000)

    Google Scholar 

  23. LaPaugh, A.S.: VLSI Layout algorithms. Chapman & Hall/CRC, Boca Raton (2010)

    Google Scholar 

  24. Yue, T.-W., Lee, Z. Z.: A Q’tron neural-network approach to solve the graph coloring problems. In: Proceedings of 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI), pp. 19–23, Paris, France (2007)

    Google Scholar 

  25. Graph Coloring Instances. http://mat.gsia.cmu.edu/COLOR/instances.html. Accessed 10 February 2012

  26. Gustafson, J.L.: The quest for linear equation solvers and the invention of electronic digital computing. In: Proceedings of IEEE John Vincent Atanasoff 2006 International Symposium on Modern Computing (JVA’06), pp. 10–16, Sofia (2006)

    Google Scholar 

  27. Yong, L.L.: Jiu Zhang Suanshu (nine chapters on the mathematical art): an overview. Arch. Hist. Exact Sci. 47(1), 1–51 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  28. Kleiner, I.: History of Linear Algebra, pp. 79–89. Birkhauser, Basel (2007)

    Google Scholar 

  29. Peters, G., Wilkinson, J.H., Martin, R.S.: Iterative refinement of the solution of a positive definite system of equations. Numer. Math. 8, 203–216 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  30. White, S.: A brief history of computing—complete timeline. http://trillian.randomstuff.org.uk/stephen/history/timeline.html. Accessed 30 October 2012

  31. Kung, S.: VLSI array processors. IEEE ASSP Mag. 2(3), 4–22 (1985)

    Article  Google Scholar 

  32. Wilburn, V.C., Ko, H.-L., Alexander, W.E.: An algorithm and architecture for the parallel solution of systems of linear equations. In: Proceedings of IEEE Fifteenth Annual International Phoenix Conference on Computer and Communications, pp. 392–398, Scottsdale, USA, March 1996

    Google Scholar 

  33. Wilbur, J.B.: The mechanical solution of simultaneous equations. J. Franklin Inst. 222, 715–724 (1936)

    Article  Google Scholar 

  34. Walker, R.M.: An analogue computer for the solution of linear simultaneous equations. Proc. IRE Waves Electrons Section 37(12), 1467–1473 (1949)

    Google Scholar 

  35. Ackerman, S.: Precise solutions of linear simultaneous equations using a low cost analog. Rev. Sci. Instrum. 22(10), 746–748 (1951)

    Article  MathSciNet  Google Scholar 

  36. Many, A., Oppenheim, U., Amitsur, S.: An electrical computer for the solution of linear simultaneous equations. Rev. Sci. Instrum. 24(2), 112–116 (1953)

    Article  MathSciNet  Google Scholar 

  37. Mitra, S.K.: Electrical analog computing machine for solving linear equations and related problems. Rev. Sci. Instrum. 26(5), 453–457 (1955)

    Article  MathSciNet  Google Scholar 

  38. Hutchinson, J., Koch, C., Luo, J., Mead, C.: Computing motion using analog and binary resistive networks. Computer 21(3), 52–63 (1988)

    Article  Google Scholar 

  39. Jang, J., Lee, S., Shin, S.: An Optimization Network for Matrix Inversion, pp. 397–401. American Institute of Physics, New York (1988)

    Google Scholar 

  40. Chakraborty, K., Mehrotra, K., Mohan, C.K., Ranka, S.: An optimization network for solving a set of simultaneous linear equations. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), pp. 516–521, Baltimore, USA, June 1992

    Google Scholar 

  41. Wang, J.: Electronic realization of recurrent neural network for solving simultaneous linear equations. Electron. Lett. 28(5), 493–495 (1992)

    Article  Google Scholar 

  42. Cichocki, A., Unbehauen, R.: Neural networks for solving systems of linear equations and related problems. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 39(2), 124–138 (1992)

    Article  MATH  Google Scholar 

  43. Cichocki, A., Unbehauen, R.: Neural networks for solving systems of linear equations. Part ii. Minimax and least absolute value problems. IEEE Trans. Circ. Syst. II: Analog Digital Signal Processing 39(9), 619–633 (1992)

    Article  MATH  Google Scholar 

  44. Wang, J., Li, H.: Solving simultaneous linear equations using recurrent neural networks. Inf. Sci. Intell. Syst. 76(3), 255–277 (1994)

    MATH  Google Scholar 

  45. Zhang, K., Ganis, G., Sereno, M.I.: Anti-hebbian synapses as a linear equation solver. In: Proceedings of International Conference on Neural Networks, pp. 387–389, Houston, TX, USA, June 1997

    Google Scholar 

  46. Xia, Y., Wang, J., Hung, D.L.: Recurrent neural networks for solving linear inequalities and equations. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 46(4), 452–462 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  47. Jiang, D.: Analog computing for real-time solution of time-varying linear equations. In: Proceedings of International Conference on Communications, Circuits and Systems (ICCCAS), pp. 1367–1371, June 2004

    Google Scholar 

  48. Wang, X., Ziavras, S.G.: Parallel direct solution of linear equations on FPGA-based machines. In: Proceedings of International Parallel and Distributed Processing Symposium (IPDPS’03), pp. 113–120, Nice, France, April 2003

    Google Scholar 

  49. Kreyszig, E.: Advanced Engineering Mathematics, 8th edn. Wiley-India (2006)

    Google Scholar 

  50. Kambo, N.S.: Mathematical Programming Techniques, revised edn. Affilated East-West Press Pvt Ltd., New Delhi (1991)

    Google Scholar 

  51. Anguita, D., Boni, A., Ridella, S.: A digital architecture for support vector machines: theory, algorithm, and FPGA implementation. IEEE Trans. Neural Netw. 14(5), 993–1009 (2003)

    Article  Google Scholar 

  52. Cichocki, A., Unbehauen, R.: Neural Networks for Optimization and Signal Processing. Wiley, Chichester (1993)

    MATH  Google Scholar 

  53. Iqbal, K., Pai, Y.C.: Predicted region of stability for balance recovery: motion at the knee joint can improve termination of forward movement. J. Biomech. 13(12), 1619–1627 (2000)

    Article  Google Scholar 

  54. Zhang, Y.: Towards piecewise-linear primal neural networks for optimization and redundant robotics. In: Proceedings of IEEE International Conference on Networking, Sensing and Control, pp. 374–379, Fort Lauderdale, Florida, USA (2006)

    Google Scholar 

  55. Zhang, Y., Leithead, W.E.: Exploiting hessian matrix and trust-region algorithm in hyperparameters estimation of gaussian process. Appl. Math. Comput. 171(2), 1264–1281 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  56. Young, M.R.: A minimax portfolio selection rule with linear programming solution. Manage. Sci. 44(5), 673–683 (1988)

    Article  Google Scholar 

  57. Bixby, R.E., Gregory, J.W., Lustig, I.J., Marsten, R.E., Shanno, D.F.: Very large-scale linear programming: a case study in combining interior point and simplex methods. Oper. Res. 40(5), 885–897 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  58. Peidro, D., Mula, J., Jimenez, M., del Mar Botella, M.: A fuzzy linear programming based approach for tactical supply chain planning in an uncertainty environment. Eur. J. Oper. Res. 205(16), 65–80 (2010)

    Article  MATH  Google Scholar 

  59. Chertkov, M., Stepanov, M.G.: An efficient pseudocodeword search algorithm for linear programming decoding of LDPC codes. IEEE Trans. Inf. Theory 54(4), 1514–1520 (2008)

    Article  MathSciNet  Google Scholar 

  60. ChvÃtal, V., Cook, W., Dantzig, G.B., Fulkerson, D.R., Johnson, S.M.: Solution of a Large-Scale Traveling-Salesman Problem, pp. 7–28. Springer, Berlin, (2010)

    Google Scholar 

  61. Atkociunas, J.: Quadratic programming for degenerate shakedown problems of bar structures. Mech. Res. Commun. 23(2), 195–206 (1996)

    Article  MATH  Google Scholar 

  62. Bartlett, R.A., Wachter, A., Biegler, L.T.: Active set vs. interior point strategies for model predictive control. In: Proceedings of American Control Conference, pp. 4229–4233, Chicago, USA, June 2000

    Google Scholar 

  63. Maier, G., Munro, J.: Mathematical programming applications to engineering plastic analysis. Appl. Mech. Rev. 35, 1631–1643 (1982)

    Google Scholar 

  64. Nordebo, S., Zang, Z., Claesson, I.: A semi-infinite quadratic programming algorithm with applications to array pattern synthesis. IEEE Trans. Circ. Syst. II: Analog and Digital Signal Processing 48(3), 225–232 (2001)

    Article  Google Scholar 

  65. Schonherr, S.: Quadratic programming in geometric optimization: theory, implementation, and applications. PhD thesis, Swiss Federal Institute of Technology, Zurich (2002)

    Google Scholar 

  66. Borguet, S., Leonard, O.: A quadratic programming framework for constrained and robust jet engine health monitoring. Progr. Propul. Phys. 1, 669–692 (2009)

    Google Scholar 

  67. Dembo, R.S., Tulowitzki, U.: Computing equilibria on large multicommodity networks: an application of truncated quadratic programming algorithms. Networks 18(4), 273–284 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  68. Censor, Y., Elfving, T.: New methods for linear inequalities. Linear Algebra Appl. 42, 199–211 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  69. Wen, U.-P., Lan, K.-M., Shih, H.-S.: A review of hopfield neural networks for solving mathematical programming problems. Eur. J. Oper. Res. 198(3), 675–687 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  70. Kennedy, M.P., Chua, L.O.: Neural networks for nonlinear programming. IEEE Trans. Circ. Syst. 35(5), 554–562 (1988)

    Article  MathSciNet  Google Scholar 

  71. Rodriguez-Vazquez, A., Rueda, A., Huertas, J.L., Dominguez-Castro, R.: Switched-capacitor neural networks for linear programming. Electron. Lett. 24(8), 496–498 (1988)

    Article  Google Scholar 

  72. Rodriguez-Vazquez, A., Dominguez-Castro, R., Rueda, A., Huertas, J.L., Sanchez-Sinencio, E.: Nonlinear switched capacitor ‘neural’ networks for optimization problems. IEEE Trans. Circ. Syst. 37(3), 384–398 (1990)

    Article  MathSciNet  Google Scholar 

  73. Lan, K.-M., Wen, U.-P., Shih, H.-S., Lee, E.S.: A hybrid neural network approach to bilevel programming problems. Appl. Math. Lett. 20(8), 880–884 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  74. Maa, C.-Y., Shanblatt, M.A.: Linear and quadratic programming neural network analysis. IEEE Trans. Neural Netw. 3(4), 580–594 (1992)

    Article  Google Scholar 

  75. Chong, E.K.P., Hui, S., Zak, S.H.: An analysis of a class of neural networks for solving linear programming problems. IEEE Trans. Autom. Control 44(11), 1995–2006 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  76. Zhu, X., Zhang, S., Constantinides, A.G.: Lagrange neural networks for linear programming. J. Parallel Distrib. Comput. 14(3), 354–360 (1992)

    Article  Google Scholar 

  77. Xia, Y., Wang, J.: Neural network for solving linear programming problems with bounded variables. IEEE Trans. Neural Netw. 6(2), 515–519 (1995)

    Article  Google Scholar 

  78. Malek, A., Yari, A.: Primal-dual solution for the linear programming problems using neural networks. Appl. Math. Comput. 167(1), 198–211 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  79. Ghasabi-Oskoei, H., Malek, A., Ahmadi, A.: Novel artificial neural network with simulation aspects for solving linear and quadratic programming problems. Comput. Math. Appl. 53(9), 1439–1454 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  80. Wang, J.: Recurrent neural network for solving quadratic programming problems with equality constraints. Electron. Lett. 28(14), 1345–1347 (1992)

    Article  Google Scholar 

  81. Forti, M., Tesi, A.: New conditions for global stability of neural networks with application to linear and quadratic programming problems. IEEE Trans. Circ. Syst. I: Fundam. Theory Appl. 42(7), 354–366 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  82. Wu, X.-Y., Xia, Y.-S., Li, J., Chen, W.-K.: A high-performance neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7(3), 643–651 (1996)

    Article  Google Scholar 

  83. Xia, Y.: A new neural network for solving linear and quadratic programming problems. IEEE Trans. Neural Netw. 7(6), 1544–1548 (1996)

    Article  Google Scholar 

  84. Tao, Q., Cao, J., Sun, D.: A simple and high performance neural network for quadratic programming problems. Appl. Math. Comput. 124(2), 251–260 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  85. Liu, Q., Wang, J.: A one-layer recurrent neural network with a discontinuous hard-limiting activation function for quadratic programming. IEEE Trans. Neural Netw. 19(4), 558–570 (2008)

    Article  Google Scholar 

  86. Gould, N.I.M., Toint, P.L.: A quadratic programming bibliography. Technical report, RAL Numerical Analysis Group, March 2010

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electronics Engineering, Aligarh Muslim University, Aligarh, Uttar Pradesh, 202002, India

    Mohd. Samar Ansari

Authors
  1. Mohd. Samar Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohd. Samar Ansari .

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer India

About this chapter

Cite this chapter

Ansari, M.S. (2014). Background. In: Non-Linear Feedback Neural Networks. Studies in Computational Intelligence, vol 508. Springer, New Delhi. https://doi.org/10.1007/978-81-322-1563-9_2

Download citation

  • DOI: https://doi.org/10.1007/978-81-322-1563-9_2

  • Published:

  • Publisher Name: Springer, New Delhi

  • Print ISBN: 978-81-322-1562-2

  • Online ISBN: 978-81-322-1563-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation