Progress in Geospatial Analysis pp 125-141 | Cite as

# Multi-layer Perceptron Neural Networks in Geospatial Analysis

## Abstract

Geospatial analysis involves using a variety of approaches. Deciding on a suitable approach depends on the complexity of the problem being addressed and the degree to which the problem is understood. Several algebraic and numerical computing techniques can be used to describe the behavior and nature of real geographical processes and develop mathematical models to represent them. Such methods require accurate knowledge of the process dynamics to emulate the processes. However, in practice, the knowledge required to solve a problem may be incomplete because the source of the knowledge is unknown, or because the complexity of the problem may introduce uncertainties and inaccuracies that make modeling unrealistic. In this case, an approximate analysis approach can be used. Artificial neural networks (ANNs) are an open concept that allows for the continuous refinement and acquisition of new knowledge and can provide solutions to such problems. It offers an alternative for dealing with solutions with a tolerance of imprecision, uncertainty, and approximation. It is known as an information-processing paradigm inspired by the interconnected and parallel structure of the human brain. The initial concepts of ANNs were attempts to depict the characteristics of biological neural networks in order to address a series of information-processing issues. This research domain has been extensively studied and applied during the last three decades. ANNs provide a flexible data analysis framework for appropriate nonlinear mappings from a variety of input variables. In particular, they are distribution-free and thus have an advantage over most statistical methods that require knowledge of the distribution function. In particular, they can learn complex functional relationships between input and output data that are not envisioned by researchers (Kim and Nelson 1998). ANNs are applied in a wide range of fields, such as medicine, molecular biology, ecology, environmental sciences, and image classification (Atkinson and Tatnall 1997).

### References

- Abid S, Fnaiech F, Najim M (2001) A fast feedforward training algorithm using a modified form of the standard backpropagation algorithm. IEEE Trans Neural Network 12:424–430CrossRefGoogle Scholar
- Aitkenhead MJ, Aalders IH (2008) Classification of Landsat Thematic Mapper imagery for land cover using neural networks. Int J Rem Sens 29:2075–2084CrossRefGoogle Scholar
- Atkinson PM, Tatnall ARL (1997) Neural networks in remote sensing. Int J Rem Sens 18:699–709CrossRefGoogle Scholar
- Baum EB, Haussler D (1989) What size net gives valid generalization? In: Touretzky DS (ed) Advances in neural information processing systems I. Morgan Kaufmann, San Mateo, pp 81–90Google Scholar
- Benediktsson JA, Swain PH, Ersoy OK (1990) Neural network approaches versus statistical methods in classification of multisource remote sensing data. IEEE Trans Geosci Rem Sens 28:540–552CrossRefGoogle Scholar
- Berberoglu S, Curran PJ, Lloyd CD, Atkinson PM (2007) Texture classification of Mediterranean land cover. Int J Appl Earth Observation Geoinformation 9:322–334CrossRefGoogle Scholar
- Bi W, Wang X, Tang Z, Tamura H (2005) Avoiding the local minima problem in backpropagation algorithm with modified error function. Trans Fund Electron Comm Comput Sci E88-A:3645–3653CrossRefGoogle Scholar
- Bishop C (1995) Neural networks for pattern recognition. Clarendon, OxfordGoogle Scholar
- Blamire PA (1996) The influence of relative sample size in training artificial neural networks. Int J Rem Sens 17:223–230CrossRefGoogle Scholar
- Brown M, Harris C (1994) Neuro-fuzzy adaptive modelling and control. PrenticeHall, New YorkGoogle Scholar
- Buckley JJ, Hayashi Y (1994) Fuzzy neural networks. In: Yager R, Zadeh L (eds) Fuzzy sets, neural networks and soft computing. Van Nostrand Reinhold, New YorkGoogle Scholar
- Cho SW, Chow TWS (1999) Training multilayer neural networks using fast global learning algorithm-least-squares and penalized optimization methods. Neurocomputing 25:115–131CrossRefGoogle Scholar
- Chowdhury PR, Singh YP, Chansarkar RA (1999) Dynamic tunneling technique for efficient training of multilayer perceptrons. IEEE Trans Neural Network 10:48–55CrossRefGoogle Scholar
- Civco DL (1993) Artificial neural networks for land cover classification and mapping. Int J Geogr Inform Syst 7:173–186CrossRefGoogle Scholar
- Cronin J (1987) Mathematical aspects of Hodgkin–Huxley theory. Cambridge University Press, CambridgeCrossRefGoogle Scholar
- Drago GP, Ridella S (1992) Statistically controlled activation weight initialization. IEEE Trans Neural Network 3:627–631CrossRefGoogle Scholar
- Dreyer P (1993) Classification of land cover using optimized neural networks on SPOT data. Photogramm Eng Rem Sens 5:617–621Google Scholar
- Foody GM, Mathur A, Sanchez-Hernandez C, Boyd DS (2006) Training set size requirements for the classification of a specific class. Rem Sens Environ 104:1–14CrossRefGoogle Scholar
- Garson GD (1998) Neural networks: an introductory guide for social scientists. Sage, LondonGoogle Scholar
- Ghosh AK, Bose S (2004) Backfitting neural networks. Comput Stat 19:193–210CrossRefGoogle Scholar
- Girosi F, Poggio T (1990) Networks and the best approximation property. Biol Cybern 63:169–176CrossRefGoogle Scholar
- Gong P (1996) Integrated analysis of spatial data from multiple sources: using evidential reasoning and artificial neural network techniques for geological mapping. Photogramm Eng Rem Sens 62:513–523Google Scholar
- Gupta MM (1994) Fuzzy neural networks: theory and Applications, Proceedings of SPIE, pp 303–325Google Scholar
- Hagan MT, Menhaj MB (1994) Training feedforward neural networks with the Marquardt algorithm. IEEE Trans Neural Network 5:989–993CrossRefGoogle Scholar
- Hartigan JA, Wong MA (1979) A k-means clustering algorithm. Appl Stat 28:100–108CrossRefGoogle Scholar
- Hebb DO (1949) The organization of behavior. Wiley, New YorkGoogle Scholar
- Hecht-Nielsen R (1987) Kolmogorov’s mapping neural network existence theorem. In: Caudill M, Butler C (eds) Proceedings of the first IEEE international conference on neural networks, pp 11–14Google Scholar
- Heermann PD, Khazenie N (1992) Classification of multispectral remote sensing data using a back-propagation neural network, LEEE Transactions on Geoscience and Remote Sensing, 30(1):81–88Google Scholar
- Hepner GF, Logan T, Ritter N, Bryant N (1990) ANN classification using a minimal training set: comparison to conventional supervised classification. Photogramm Eng Rem Sens 56:469–473Google Scholar
- Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79(8):2554–2558CrossRefGoogle Scholar
- Horikawa S, Furuhashi T, Uchikawa Y (1992) On fuzzy modelling using fuzzy neural networks with backpropagation algorithm. IEEE Trans Neural Network 3:801–806CrossRefGoogle Scholar
- Hornik K, Stinchcombe M, White H (1989) Multi-layer feedforward networks are universal approximators. Neural Network 2:359–366CrossRefGoogle Scholar
- Hush DR (1989) Classification with neural networks: a performance analysis. In: Proceedings of the IEEE international conference on systems engineering, pp 277–280Google Scholar
- Jacobs RA (1988) Increasing rate of convergence through learning rate adaptation. Neural Network 1:295–307CrossRefGoogle Scholar
- Jeenbekov AA, Sarybaeva AA (2000) Conditions of convergence of backpropagation learning algorithm. In: Proceedings of SPIE on optoelectronic and hybrid optical/digital systems for image and signal processing, pp 12–18Google Scholar
- Kamarthi SVC, Pittner S (1999) Accelerating neural network training using weight extrapolations. Neural Network 12:1285–1299CrossRefGoogle Scholar
- Kanellopoulos I, Wilkinson GG (1997) Strategies and best practice for neural network image classification. Int J Rem Sens 18:711–725CrossRefGoogle Scholar
- Kanellopoulos I, Varfis A, Wilkinson GG, Megier J (1990) Land-cover discrimination in SPOT HRV imagery using an artificial neural network a 20-class experiment. Int J Rem Sens 13:917–924CrossRefGoogle Scholar
- Karras DA, Perantonis SJ (1995) An Efficient constrained training algorithm for feedforward networks. IEEE Transactions on Neural Networks 6:1420–1434CrossRefGoogle Scholar
- Kavzoglu T (2001) An investigation of the design and use of feed-forward artificial neural networks in the classification of remotely sensed images. Ph.D. Dissertation, The University of NottinghamGoogle Scholar
- Kavzoglu T (2008) Increasing the accuracy of neural network classification using refined training data. Environ Model Softw 24:850–858CrossRefGoogle Scholar
- Kavzoglu T, Mather PM (2002) The role of feature selection in artificial neural network applications. International Journal of Remote Sensing 23:2919–2937CrossRefGoogle Scholar
- Key J, Maslanik JA, Schweiger AJ (1989) Classification of merged AVHRR and SMMR Arctic data with neural networks. Photogramm Eng Rem Sens 55:1331–1338Google Scholar
- Kim DS, Nelson RF (1998) Attributes of neural networks for extracting continuous vegetation variables from optical and radar measurements. Int J Rem Sens 19:2639–2663CrossRefGoogle Scholar
- Klimasauskas CC (1993) Applying neural networks. In: Trippi RR, Turban E (eds) Neural networks in finance and investing. Probus, Cambridge, pp 47–72Google Scholar
- Kohonen T (1984) Self-organization and associative memory. Springer, BerlinGoogle Scholar
- Kohonen T (1995) Self-organizing maps. Spinger, BerlinCrossRefGoogle Scholar
- Kohonen T (1990) The self-organizing map. Proc. IEEE 78:1464–1480CrossRefGoogle Scholar
- Kurkova V (1992) Kolmogorov’s theorem and multilayer neural networks. Neural Network 5:501–506CrossRefGoogle Scholar
- Kwok TY, Yeung DY (1997) Objective functions for training new hidden units in constructive neural networks. IEEE Trans Neural Network 8:1131–1147CrossRefGoogle Scholar
- Martens JP (1996) A stochastically motivated random initialization of pattern classifying MLPs. Neural Process Lett 3:23–29CrossRefGoogle Scholar
- Mather PM (1999) Computer processing of remotely-sensed images: an introduction. John Wiley, ChichesterGoogle Scholar
- McCulloch WW, Pitts W (1943) A logical calculus of ideas imminent in nervous activity. Bull Math Biophys 5:115–133CrossRefGoogle Scholar
- Mead C (1989) Analog VLSI and neural systems. Addison-Wesley, ReadingCrossRefGoogle Scholar
- Minsky M, Papert SA (1969) Perceptrons: an introduction to computational geometry. MIT Press, CambridgeGoogle Scholar
- Nie J, Linkens D (1992) Neural network-based approximate reasoning: principles and implementation. Int J Control 56:399–413CrossRefGoogle Scholar
- Ooyen AO, Neinhuis B (1992) Improving the convergence of the backpropagation algorithm. Neural Network 5:465–471CrossRefGoogle Scholar
- Paola JD (1994) Neural network classification of multispectral imagery. M.Sc. Thesis, The University of ArizonaGoogle Scholar
- Paola JD, Schowengerdt RA (1995) A detailed comparison of backpropagation neural network and maximum likelihood classifiers for urban land use classification. IEEE Trans Geosci Rem Sens 33:981–996CrossRefGoogle Scholar
- Paola JD, Schowengerdt RA (1997) The effect of neural network structure on a multispectral land-use/land-cover classification. Photogramm Eng Rem Sens 63:535–544Google Scholar
- Pedrycz W (1995) Fuzzy sets engineering. CRC Press, Boca RatonGoogle Scholar
- Poggio T, Girosi F (1987) Networks for approximation and learning. In: Proceedings of IEEE, pp 1481–1497Google Scholar
- Ripley BD (1993) Statistical aspects of neural networks. In: Barndorff-Nielsen OE, Jensen JL, Kendall WS (eds) Networks and chaos: statistical and probabilistic aspect. Chapman & Hall, London, pp 40–123Google Scholar
- Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386–408CrossRefGoogle Scholar
- Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing: explorations in the microstructures of cognition. MIT Press, Cambridge, pp 318–362Google Scholar
- Salomon R, Hemmen JL (1996) Accelerating backpropagation through dynamic self-adaptation. Neural Network 9:589–601CrossRefGoogle Scholar
- Sarle WS (2000) Neural network, FAQ. ftp://ftp.sas.com/pub/neural/FAQ.html
- Seber GAF, Wild CJ (1989) Nonlinear regression. Wiley, New YorkCrossRefGoogle Scholar
- Staufer P, Fisher MM (1997) Spectral pattern recognition by a two-layer perceptron: effects of training set size. In: Kanellopoulos I, Wilkinson GG, Roli F, Austin J (eds) Neurocomputation in remote sensing data analysis. Springer, London, pp 105–116CrossRefGoogle Scholar
- Takagi T, Hayashi I (1991) Neural network driven fuzzy reasoning. Int J Approx Reason 5:191–212CrossRefGoogle Scholar
- Vogel TP, Mangis JK, Rigler AK, Zink WT, Alkon DL (1988) Accelerating the convergence of the backpropagation method. Biol Cybern 59:257–263CrossRefGoogle Scholar
- Wang J (1994) A deterministic annealing neural network for Conex Programming. Neural networks 7:629–641CrossRefGoogle Scholar
- Wang XG, Tang Z, Tamura H, Ishii M, Sun WD (2004) An improved backpropagation algorithm to avoid the local minima problem. Neurocomputing 56:455–460CrossRefGoogle Scholar
- Widrow B, Hoff ME (1960) Adaptive switching circuits. IRE WESCOM Convention Record 4:96–104Google Scholar
- Yam JYF, Chow TWS (1997) Extended least squares based algorithm for training feedforward networks. IEEE Trans Neural Network 8:806–810CrossRefGoogle Scholar
- Yam JYF, Chow TWS (2000) A weight initialization method for improving training speed in feedforward neural networks. Neurocomputing 30:219–232CrossRefGoogle Scholar
- Yam JYF, Chow TWS (2001) Feedforward networks training speed enhancement by optimal initialization of the synaptic coefficients. IEEE Trans Neural Network 12:430–434CrossRefGoogle Scholar
- Yam JYF, Chow TWS, Leung CT (1997) A new method in determining initial weights of feedforward neural networks for training enhancement. Neurocomputing 16:23–32CrossRefGoogle Scholar
- Yu XH, Chen GA (1997) Efficient backpropagation learning using optimal learning rate and momentum. Neural Network 10:517–527CrossRefGoogle Scholar
- Yu X, Loh NK, Miller WC (1993) A new acceleration technique for the backpropagation algorithm. In: Proceedings of IEEE international conference on neural networks, pp 1157–1161Google Scholar
- Yu XH, Chen GA, Cheng DSX (1995) Dynamic learning rate optimization of the backpropagation algorithm. IEEE Trans Neural Network 6:669–677CrossRefGoogle Scholar
- Zhu Y, He Y (2006) Short-term load forecasting model using fuzzy c means based radial basis function network. In: Proceedings of 6th international conference on intelligence systems design and applications, pp 579–582Google Scholar