Advertisement

Multi-layer Perceptron Neural Networks in Geospatial Analysis

  • Duong Dang Khoi
  • Yuji Murayama
Chapter

Abstract

Geospatial analysis involves using a variety of approaches. Deciding on a suitable approach depends on the complexity of the problem being addressed and the degree to which the problem is understood. Several algebraic and numerical computing techniques can be used to describe the behavior and nature of real geographical processes and develop mathematical models to represent them. Such methods require accurate knowledge of the process dynamics to emulate the processes. However, in practice, the knowledge required to solve a problem may be incomplete because the source of the knowledge is unknown, or because the complexity of the problem may introduce uncertainties and inaccuracies that make modeling unrealistic. In this case, an approximate analysis approach can be used. Artificial neural networks (ANNs) are an open concept that allows for the continuous refinement and acquisition of new knowledge and can provide solutions to such problems. It offers an alternative for dealing with solutions with a tolerance of imprecision, uncertainty, and approximation. It is known as an information-processing paradigm inspired by the interconnected and parallel structure of the human brain. The initial concepts of ANNs were attempts to depict the characteristics of biological neural networks in order to address a series of information-processing issues. This research domain has been extensively studied and applied during the last three decades. ANNs provide a flexible data analysis framework for appropriate nonlinear mappings from a variety of input variables. In particular, they are distribution-free and thus have an advantage over most statistical methods that require knowledge of the distribution function. In particular, they can learn complex functional relationships between input and output data that are not envisioned by researchers (Kim and Nelson 1998). ANNs are applied in a wide range of fields, such as medicine, molecular biology, ecology, environmental sciences, and image classification (Atkinson and Tatnall 1997).

References

  1. Abid S, Fnaiech F, Najim M (2001) A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. IEEE Trans Neural Network 12:424–430CrossRefGoogle Scholar
  2. Aitkenhead MJ, Aalders IH (2008) Classification of Landsat Thematic Mapper imagery for land cover using neural networks. Int J Rem Sens 29:2075–2084CrossRefGoogle Scholar
  3. Atkinson PM, Tatnall ARL (1997) Neural networks in remote sensing. Int J Rem Sens 18:699–709CrossRefGoogle Scholar
  4. Baum EB, Haussler D (1989) What size net gives valid generalization? In: Touretzky DS (ed) Advances in neural information processing systems I. Morgan Kaufmann, San Mateo, pp 81–90Google Scholar
  5. Benediktsson JA, Swain PH, Ersoy OK (1990) Neural network approaches versus statistical methods in classification of multisource remote sensing data. IEEE Trans Geosci Rem Sens 28:540–552CrossRefGoogle Scholar
  6. Berberoglu S, Curran PJ, Lloyd CD, Atkinson PM (2007) Texture classification of Mediterranean land cover. Int J Appl Earth Observation Geoinformation 9:322–334CrossRefGoogle Scholar
  7. Bi W, Wang X, Tang Z, Tamura H (2005) Avoiding the local minima problem in backpropagation algorithm with modified error function. Trans Fund Electron Comm Comput Sci E88-A:3645–3653CrossRefGoogle Scholar
  8. Bishop C (1995) Neural networks for pattern recognition. Clarendon, OxfordGoogle Scholar
  9. Blamire PA (1996) The influence of relative sample size in training artificial neural networks. Int J Rem Sens 17:223–230CrossRefGoogle Scholar
  10. Brown M, Harris C (1994) Neuro-fuzzy adaptive modelling and control. PrenticeHall, New YorkGoogle Scholar
  11. Buckley JJ, Hayashi Y (1994) Fuzzy neural networks. In: Yager R, Zadeh L (eds) Fuzzy sets, neural networks and soft computing. Van Nostrand Reinhold, New YorkGoogle Scholar
  12. Cho SW, Chow TWS (1999) Training multilayer neural networks using fast global learning algorithm-least-squares and penalized optimization methods. Neurocomputing 25:115–131CrossRefGoogle Scholar
  13. Chowdhury PR, Singh YP, Chansarkar RA (1999) Dynamic tunneling technique for efficient training of multilayer perceptrons. IEEE Trans Neural Network 10:48–55CrossRefGoogle Scholar
  14. Civco DL (1993) Artificial neural networks for land cover classification and mapping. Int J Geogr Inform Syst 7:173–186CrossRefGoogle Scholar
  15. Cronin J (1987) Mathematical aspects of Hodgkin–Huxley theory. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  16. Drago GP, Ridella S (1992) Statistically controlled activation weight initialization. IEEE Trans Neural Network 3:627–631CrossRefGoogle Scholar
  17. Dreyer P (1993) Classification of land cover using optimized neural networks on SPOT data. Photogramm Eng Rem Sens 5:617–621Google Scholar
  18. Foody GM, Mathur A, Sanchez-Hernandez C, Boyd DS (2006) Training set size requirements for the classification of a specific class. Rem Sens Environ 104:1–14CrossRefGoogle Scholar
  19. Garson GD (1998) Neural networks: an introductory guide for social scientists. Sage, LondonGoogle Scholar
  20. Ghosh AK, Bose S (2004) Backfitting neural networks. Comput Stat 19:193–210CrossRefGoogle Scholar
  21. Girosi F, Poggio T (1990) Networks and the best approximation property. Biol Cybern 63:169–176CrossRefGoogle Scholar
  22. Gong P (1996) Integrated analysis of spatial data from multiple sources: using evidential reasoning and artificial neural network techniques for geological mapping. Photogramm Eng Rem Sens 62:513–523Google Scholar
  23. Gupta MM (1994) Fuzzy neural networks: theory and Applications, Proceedings of SPIE, pp 303–325Google Scholar
  24. Hagan MT, Menhaj MB (1994) Training feedforward neural networks with the Marquardt algorithm. IEEE Trans Neural Network 5:989–993CrossRefGoogle Scholar
  25. Hartigan JA, Wong MA (1979) A k-means clustering algorithm. Appl Stat 28:100–108CrossRefGoogle Scholar
  26. Hebb DO (1949) The organization of behavior. Wiley, New YorkGoogle Scholar
  27. Hecht-Nielsen R (1987) Kolmogorov’s mapping neural network existence theorem. In: Caudill M, Butler C (eds) Proceedings of the first IEEE international conference on neural networks, pp 11–14Google Scholar
  28. Heermann PD, Khazenie N (1992) Classification of multispectral remote sensing data using a back-propagation neural network, LEEE Transactions on Geoscience and Remote Sensing, 30(1):81–88Google Scholar
  29. Hepner GF, Logan T, Ritter N, Bryant N (1990) ANN classification using a minimal training set: comparison to conventional supervised classification. Photogramm Eng Rem Sens 56:469–473Google Scholar
  30. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79(8):2554–2558CrossRefGoogle Scholar
  31. Horikawa S, Furuhashi T, Uchikawa Y (1992) On fuzzy modelling using fuzzy neural networks with backpropagation algorithm. IEEE Trans Neural Network 3:801–806CrossRefGoogle Scholar
  32. Hornik K, Stinchcombe M, White H (1989) Multi-layer feedforward networks are universal approximators. Neural Network 2:359–366CrossRefGoogle Scholar
  33. Hush DR (1989) Classification with neural networks: a performance analysis. In: Proceedings of the IEEE international conference on systems engineering, pp 277–280Google Scholar
  34. Jacobs RA (1988) Increasing rate of convergence through learning rate adaptation. Neural Network 1:295–307CrossRefGoogle Scholar
  35. Jeenbekov AA, Sarybaeva AA (2000) Conditions of convergence of backpropagation learning algorithm. In: Proceedings of SPIE on optoelectronic and hybrid optical/digital systems for image and signal processing, pp 12–18Google Scholar
  36. Kamarthi SVC, Pittner S (1999) Accelerating neural network training using weight extrapolations. Neural Network 12:1285–1299CrossRefGoogle Scholar
  37. Kanellopoulos I, Wilkinson GG (1997) Strategies and best practice for neural network image classification. Int J Rem Sens 18:711–725CrossRefGoogle Scholar
  38. Kanellopoulos I, Varfis A, Wilkinson GG, Megier J (1990) Land-cover discrimination in SPOT HRV imagery using an artificial neural network a 20-class experiment. Int J Rem Sens 13:917–924CrossRefGoogle Scholar
  39. Karras DA, Perantonis SJ (1995) An Efficient constrained training algorithm for feedforward networks. IEEE Transactions on Neural Networks 6:1420–1434CrossRefGoogle Scholar
  40. Kavzoglu T (2001) An investigation of the design and use of feed-forward artificial neural networks in the classification of remotely sensed images. Ph.D. Dissertation, The University of NottinghamGoogle Scholar
  41. Kavzoglu T (2008) Increasing the accuracy of neural network classification using refined training data. Environ Model Softw 24:850–858CrossRefGoogle Scholar
  42. Kavzoglu T, Mather PM (2002) The role of feature selection in artificial neural network applications. International Journal of Remote Sensing 23:2919–2937CrossRefGoogle Scholar
  43. Key J, Maslanik JA, Schweiger AJ (1989) Classification of merged AVHRR and SMMR Arctic data with neural networks. Photogramm Eng Rem Sens 55:1331–1338Google Scholar
  44. Kim DS, Nelson RF (1998) Attributes of neural networks for extracting continuous vegetation variables from optical and radar measurements. Int J Rem Sens 19:2639–2663CrossRefGoogle Scholar
  45. Klimasauskas CC (1993) Applying neural networks. In: Trippi RR, Turban E (eds) Neural networks in finance and investing. Probus, Cambridge, pp 47–72Google Scholar
  46. Kohonen T (1984) Self-organization and associative memory. Springer, BerlinGoogle Scholar
  47. Kohonen T (1995) Self-organizing maps. Spinger, BerlinCrossRefGoogle Scholar
  48. Kohonen T (1990) The self-organizing map. Proc. IEEE 78:1464–1480CrossRefGoogle Scholar
  49. Kurkova V (1992) Kolmogorov’s theorem and multilayer neural networks. Neural Network 5:501–506CrossRefGoogle Scholar
  50. Kwok TY, Yeung DY (1997) Objective functions for training new hidden units in constructive neural networks. IEEE Trans Neural Network 8:1131–1147CrossRefGoogle Scholar
  51. Martens JP (1996) A stochastically motivated random initialization of pattern classifying MLPs. Neural Process Lett 3:23–29CrossRefGoogle Scholar
  52. Mather PM (1999) Computer processing of remotely-sensed images: an introduction. John Wiley, ChichesterGoogle Scholar
  53. McCulloch WW, Pitts W (1943) A logical calculus of ideas imminent in nervous activity. Bull Math Biophys 5:115–133CrossRefGoogle Scholar
  54. Mead C (1989) Analog VLSI and neural systems. Addison-Wesley, ReadingCrossRefGoogle Scholar
  55. Minsky M, Papert SA (1969) Perceptrons: an introduction to computational geometry. MIT Press, CambridgeGoogle Scholar
  56. Nie J, Linkens D (1992) Neural network-based approximate reasoning: principles and implementation. Int J Control 56:399–413CrossRefGoogle Scholar
  57. Ooyen AO, Neinhuis B (1992) Improving the convergence of the backpropagation algorithm. Neural Network 5:465–471CrossRefGoogle Scholar
  58. Paola JD (1994) Neural network classification of multispectral imagery. M.Sc. Thesis, The University of ArizonaGoogle Scholar
  59. Paola JD, Schowengerdt RA (1995) A detailed comparison of backpropagation neural network and maximum likelihood classifiers for urban land use classification. IEEE Trans Geosci Rem Sens 33:981–996CrossRefGoogle Scholar
  60. Paola JD, Schowengerdt RA (1997) The effect of neural network structure on a multispectral land-use/land-cover classification. Photogramm Eng Rem Sens 63:535–544Google Scholar
  61. Pedrycz W (1995) Fuzzy sets engineering. CRC Press, Boca RatonGoogle Scholar
  62. Poggio T, Girosi F (1987) Networks for approximation and learning. In: Proceedings of IEEE, pp 1481–1497Google Scholar
  63. Ripley BD (1993) Statistical aspects of neural networks. In: Barndorff-Nielsen OE, Jensen JL, Kendall WS (eds) Networks and chaos: statistical and probabilistic aspect. Chapman & Hall, London, pp 40–123Google Scholar
  64. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386–408CrossRefGoogle Scholar
  65. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing: explorations in the microstructures of cognition. MIT Press, Cambridge, pp 318–362Google Scholar
  66. Salomon R, Hemmen JL (1996) Accelerating backpropagation through dynamic self-adaptation. Neural Network 9:589–601CrossRefGoogle Scholar
  67. Sarle WS (2000) Neural network, FAQ. ftp://ftp.sas.com/pub/neural/FAQ.html
  68. Seber GAF, Wild CJ (1989) Nonlinear regression. Wiley, New YorkCrossRefGoogle Scholar
  69. Staufer P, Fisher MM (1997) Spectral pattern recognition by a two-layer perceptron: effects of training set size. In: Kanellopoulos I, Wilkinson GG, Roli F, Austin J (eds) Neurocomputation in remote sensing data analysis. Springer, London, pp 105–116CrossRefGoogle Scholar
  70. Takagi T, Hayashi I (1991) Neural network driven fuzzy reasoning. Int J Approx Reason 5:191–212CrossRefGoogle Scholar
  71. Vogel TP, Mangis JK, Rigler AK, Zink WT, Alkon DL (1988) Accelerating the convergence of the backpropagation method. Biol Cybern 59:257–263CrossRefGoogle Scholar
  72. Wang J (1994) A deterministic annealing neural network for Conex Programming. Neural networks 7:629–641CrossRefGoogle Scholar
  73. Wang XG, Tang Z, Tamura H, Ishii M, Sun WD (2004) An improved backpropagation algorithm to avoid the local minima problem. Neurocomputing 56:455–460CrossRefGoogle Scholar
  74. Widrow B, Hoff ME (1960) Adaptive switching circuits. IRE WESCOM Convention Record 4:96–104Google Scholar
  75. Yam JYF, Chow TWS (1997) Extended least squares based algorithm for training feedforward networks. IEEE Trans Neural Network 8:806–810CrossRefGoogle Scholar
  76. Yam JYF, Chow TWS (2000) A weight initialization method for improving training speed in feedforward neural networks. Neurocomputing 30:219–232CrossRefGoogle Scholar
  77. Yam JYF, Chow TWS (2001) Feedforward networks training speed enhancement by optimal initialization of the synaptic coefficients. IEEE Trans Neural Network 12:430–434CrossRefGoogle Scholar
  78. Yam JYF, Chow TWS, Leung CT (1997) A new method in determining initial weights of feedforward neural networks for training enhancement. Neurocomputing 16:23–32CrossRefGoogle Scholar
  79. Yu XH, Chen GA (1997) Efficient backpropagation learning using optimal learning rate and momentum. Neural Network 10:517–527CrossRefGoogle Scholar
  80. Yu X, Loh NK, Miller WC (1993) A new acceleration technique for the backpropagation algorithm. In: Proceedings of IEEE international conference on neural networks, pp 1157–1161Google Scholar
  81. Yu XH, Chen GA, Cheng DSX (1995) Dynamic learning rate optimization of the backpropagation algorithm. IEEE Trans Neural Network 6:669–677CrossRefGoogle Scholar
  82. Zhu Y, He Y (2006) Short-term load forecasting model using fuzzy c means based radial basis function network. In: Proceedings of 6th international conference on intelligence systems design and applications, pp 579–582Google Scholar

Copyright information

© Springer Japan 2012

Authors and Affiliations

  1. 1.Faculty of Land AdministrationHanoi University of Natural Resources and EnvironmentHanoiVietnam
  2. 2.Division of Spatial Information Science, Graduate School of Life and Environmental SciencesUniversity of TsukubaTsukubaJapan

Personalised recommendations