Skip to main content
SpringerLink
Log in

Extracting the contribution of independent variables in neural network models: a new approach to handle instability

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

One of the main limitations of artificial neural networks (ANN) is their high inability to know in an explicit way the relations established between explanatory variables (input) and dependent variables (output). This is a major reason why they are usually called “black boxes.” In the last few years, several methods have been proposed to assess the relative importance of each explanatory variable. Nevertheless, it has not been possible to reach a consensus on which is the best-performing method. This is largely due to the different relative importance obtained for each variable depending on the method used. This importance also varies with the designed network architecture and/or with the initial random weights used to train the ANN. This paper proposes a procedure that seeks to minimize these problems and provides consistency in the results obtained from different methods. Essentially, the idea is to work with a set of neural networks instead of a single one. The proposed procedure is validated using a database collected from a customer satisfaction survey, which was conducted on the public transport system of Granada (Spain) in 2007. The results show that, when each method is applied independently, the variable’s importance rankings are similar and, in addition, coincide with the hierarchy established by researchers who have applied other techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Akin D, Akbaç B (2010) A neural network (NN) model to predict intersection crashes based upon driver, vehicle and roadway surface characteristics. Sci Res Essays 5(19):2837–2847

    Google Scholar 

  2. Azadeh A, Rouzbahman M, Saberi M, Fam IM (2011) An adaptative neural network algorithm for assessment and improvement of job satisfaction with respect to HSE and ergonomics program: the case of a gas refinery. J Loss Prev Process Ind 24:361–370

    Article  Google Scholar 

  3. Beale MH, Hagan MT, Demuth HB (2007) Neural Network Toolbox 7. User’s Guide. MathWorks, Inc. 3 Apple Hill Drive Natic, MA 01760-2098

  4. Cao M, Qiao P (2008) Neural network committee-based sensitivity analysis strategy for geotechnical engineering problems. Neural Comput Appl 17:509–519

    Article  Google Scholar 

  5. Cortez P, Embrechts MJ (2011) Opening black box data mining models using sensitivity analysis. IEEE symposium series in computational intelligence, Paris, France, 4, 2011

  6. Cortez P, Embrechts MJ (2013) Using sensitivity analysis and visualization techniques to open black box data mining models. Inf Sci 225:1–17

    Article  Google Scholar 

  7. De Oña J, De Oña R, Calvo FJ (2012) A classification tree approach to identify key factors of transit service quality. Expert Syst Appl 39:11164–11171

    Article  Google Scholar 

  8. De Oña J, De Oña R, Eboli L, Mazzulla G (2013) Perceived service quality in bus transit service: a structural equation approach. Transp Policy 29:219–226

    Article  Google Scholar 

  9. Delen D, Sharda R, Bessonov M (2006) Identifying significant predictors of injury severity in traffic accidents using a series of artificial neural networks. Accid Anal Prev 38:434–444

    Article  Google Scholar 

  10. Dell’Olio L, Ibeas A, Cecín P (2010) Modelling user perception of bus transit quality. Transp Policy 17(6):388–397

    Article  Google Scholar 

  11. Dell’Olio L, Ibeas A, Cecín P (2011) The quality of service desired by public transport users. Transp Policy 18(1):217–227

    Article  Google Scholar 

  12. Dimopoulos Y, Bourret P, Lek S (1995) Use of some sensitivity criteria for choosing networks with good generalization ability. Neural Process Lett 2:1–4

    Article  Google Scholar 

  13. Eboli L, Mazulla G (2008) Willingness-to-pay of public transport users for improvement in service quality. Eur Transp 38:107–118

    Google Scholar 

  14. Eboli L, Mazulla G (2010) How to capture the passengers’ point of view on a transit service through rating and choice opinions. Transp Rev 30:435–450

    Article  Google Scholar 

  15. Engelbrecht AP, Cloete I, Zurada JM (1995) Determining the significance of input parameters using sensitivity analysis, from natural to artificial neural computation. In: proceedings of International Workshop on Artificial Neural Networks. Málaga-Torremolinos, Springer, Spain, pp 382–388

  16. Funahashi KI (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192

    Article  Google Scholar 

  17. Garson GD (1991) Interpreting neural-network connection weights. Artif Intell Expert 6:47–51

    Google Scholar 

  18. Gedeon, T.D., Wong, P.M. & Harris, D., (1995). Balancing the bias and variance: network topology and pattern set reduction techniques. In: proceedings of International Workshop on Artificial Neural Networks, IWANN95, Torremolinos, España, pp 550–558

  19. Gedeon TD (1997) Data mining of inputs: analyzing magnitude of functional measures. Int J Neural Syst 8(2):209–218

    Article  MathSciNet  Google Scholar 

  20. Gevrey M, Dimopoulos I, Lek S (2003) Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecol Model 160:249–264

    Article  Google Scholar 

  21. Gevrey M, Dimopoulos I, Lek S (2006) Two-way interaction of input variables in the sensitivity analysis of neural network models. Ecol Model 195:43–50

    Article  Google Scholar 

  22. Goh ATC (1995) Back-propagation neural networks for modeling complex systems. Artif Intell Eng 9:143–151

    Article  Google Scholar 

  23. Hagan MT, Demuth HB, Beale MH (1996) Neural network design. Campus Publishing Service, Colorado University Bookstore, Colorado. ISBN 0-9717321-0-8

    Google Scholar 

  24. He F, Sung AH, Guo B (1997) A neural network for prediction of oil well cement bonding quality. In: proceedings of IASTED international conference on control, IASTED-ACTA Press, Cancun-Mexico, pp 417–420

  25. Hunter A, Kennedy L, Henry J, Ferguson I (2000) Application of neural networks and sensitivity analysis to improved prediction of trauma survival. Comput Methods Progr Biomed 62:11–19

    Article  Google Scholar 

  26. Hornik K, Stichcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2:359–366

    Article  Google Scholar 

  27. Kemp SL, Zaradic P, Hansen F (2007) An approach for determining relative input parameter importance and significance in artificial neural networks. Ecol Model 204:326–334

    Article  Google Scholar 

  28. Kewley R, Embrechts M, Breneman C (2000) Data strip mining for the virtual design of pharmaceuticals with neural networks. IEEE Trans Neural Netw 11(3):668–679

    Article  Google Scholar 

  29. Lek S, Beland A, Dimopoulos I, Lauga J, Moreau J (1995) Improved estimation, using neural networks, of the food consumption of fish populations. Mar Freshw Res 46(8):1229–1236

    Article  Google Scholar 

  30. Lek S, Delacoste M, Baran P, Dimopoulos I, Lauga J, Aulagnier S (1996) Application of neural networks to modeling nonlinear relationships in ecology. Ecol Model 90:39–52

    Article  Google Scholar 

  31. Lek S, Beland A, Baran P, Dimopoulos I, Delacoste M (1996) Role of some environmental variables in trout abundance models using neural networks. Aquat Living Resour 9:23–29

    Article  Google Scholar 

  32. Lin Y, Cunningham GA (1995) A new approach to fuzzy-neural system modeling. IEEE Trans Fuzzy Syst 3(2):190–198

    Article  Google Scholar 

  33. Martín del Bío B, Sanz Molina A (2006) Neural networks and fuzzy systems. Editorial RA-MA

  34. Masters T (1993) Practical neural networks recipes in C ++. Academic Press, Waltham

    Google Scholar 

  35. Moghaddam FR, Afandizadeh S, Ziyadi M (2010) Prediction of accident severity using artificial neural networks. Int J Civ Eng 9:1

    Google Scholar 

  36. Mohammadipour AH, Alavi SH (2009) The optimization of the geometric cross-section dimensions of raised pedestrian crosswalks: a case of study in Qazvin. Accid Anal Prev 41:314–326

    Article  Google Scholar 

  37. Mussone L, Ferrari A, Oneta M (1999) An analysis of urban collisions using an artificial intelligence model. Accid Anal Prev 31:705–718

    Article  Google Scholar 

  38. Olden JD, Jackson DA (2002) Illuminating the “black-box”: a randomization approach for understanding variable contributions in artificial neural networks. Ecol Model 154:135–150

    Article  Google Scholar 

  39. Olden JD, Joy MK, Death RG (2004) An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecol Model 178:389–397

    Article  Google Scholar 

  40. Özesmi SL, Özesmi U (1999) An artificial neural network approach to spatial habitat modeling with interspecific interaction. Ecol Model 116:15–31

    Article  Google Scholar 

  41. Paliwal M, Kumar UA (2011) Assessing the contribution of variables in feed forward neural network. Appl Soft Comput 11:3690–3696

    Article  Google Scholar 

  42. Palmer A, Montaño JJ (2002a) Redes neuronales artificiales aplicadas al análisis de datos. Doctoral Dissertation. University of Palma de Mallorca

  43. Palmer A, Montaño JJ (2002) Redes neuronales artificiales: abriendo la caja negra. Metodología de las ciencias del comportamiento 4(1):77–93

    Google Scholar 

  44. Palmer A, Montaño JJ (2003) Numeric sensitivity analysis applied to feed forward neural networks. Neural Comput Appl 12:119–125

    Article  Google Scholar 

  45. Rzempoluk EJ (1998) Neural network data analysis using simulnet. Springer, New York

    Book  Google Scholar 

  46. Rumelhart DE, McClelland JL (1986) Parallel distributed processing, vol 1: foundations. MIT Press, Cambridge

    Google Scholar 

  47. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by backpropagation errors. Nature 323:533–536

    Article  Google Scholar 

  48. Scardi M, Harding LW (1999) Developing an empirical model of phytoplankton primary production: a neural networks case study. Ecol Model 120(2–3):213–223

    Article  Google Scholar 

  49. Sung AH (1998) Ranking importance of input parameters of neural networks. Expert Syst Appl 15:405–411

    Article  Google Scholar 

  50. Watts MJ, Worner SP (2008) Using artificial neural networks to determine the relative contribution of abiotic factors influencing the establishment of insect pest species. Ecol Inform 3:64–74

    Article  Google Scholar 

  51. Werbos PJ (1974) Beyond regression: new tools for prediction and analysis in behavioral sciences. Doctoral Dissertation. Applied Mathematics, Harvard University

  52. Yao J, Teng N, Poh HL, Tan CL (1998) Forecasting and analysis of marketing data using neural networks. J Inf Sci Eng 14:843–862

    Google Scholar 

  53. Yeh I, Cheng W (2010) First and second order sensitivity analysis of MLP. Neurocomputing 73:2225–2233

    Article  Google Scholar 

  54. Zhou ZH, Wu J, Tang W (2002) Ensembling neural networks: many could be better than all. Artif Intell 137(1–2):239–263

    Article  MATH  MathSciNet  Google Scholar 

  55. Zurada JM, Malinowski A, Cloete I (1994) Sensitivity analysis for minimization of input data dimensión for feed forward neural network. In: proceedings of IEEE international symposium on circuits and systems, IEEE Press, London

Download references

Acknowledgments

Support from Consejería de Innovación, Ciencia y Economía of the Junta de Andalucía (Spain) (Research Project P08-TEP-03819, co-funded by FEDER) is gratefully acknowledged. The authors also acknowledge the Granada Consorcio de Transportes for making the data set available for this study.

Author information

Authors and Affiliations

  1. TRYSE Research Group, Department of Civil Engineering, University of Granada, ETSI Caminos, Canales y Puertos, c/Severo Ochoa, s/n, 18071, Granada, Spain

    Juan de Oña & Concepción Garrido

Authors
  1. Juan de Oña
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Concepción Garrido
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan de Oña.

Rights and permissions

Reprints and permissions

About this article

Cite this article

de Oña, J., Garrido, C. Extracting the contribution of independent variables in neural network models: a new approach to handle instability. Neural Comput & Applic 25, 859–869 (2014). https://doi.org/10.1007/s00521-014-1573-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-014-1573-5

Keywords

Search

Navigation