In general, it is not possible to have access to all variables that determine the behavior of a system. Once a number of measurable variables is identified, there might still exist hidden variables which influence the behavior of the system. The result is model ambiguity in the sense that, for the same (or very similar) input values, distinct outputs are obtained. In addition, the degree of ambiguity may vary across the range of input values. Therefore, to evaluate the accuracy of a model it is important to devise a method to obtain the degree of reliability for each output result. In this paper, we present such a scheme composed of two coupled neural networks, the first one computing the average predicted value and the other the reliability of the output, which is learned from the error values of the first one. As an illustration, the scheme is applied to a model for tracking slopes in a straw chamber and to a credit scoring model.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Albalate A, Suchindranath A, Soenmez MM, Suendermann D (2010) On ambiguity detection and postprocessing schemes using cluster ensembles, ICAART 2010. In: Proceedings of the international conference on agents and artificial intelligence. INSTICC Press, pp 623–630
Alfaro-Ponce M, Cruz AA, Chairez I (2014) Adaptive identifier for uncertain complex nonlinear systems based on continuous neural networks. IEEE Trans Neural Netw Learn Syst 25:483–494
Alippi C (1999) FPE-based criteria to dimension feedforward neural topologies. IEEE Trans Circuits Syst I 46:962–973
Alippi C, Piuri V, Sami M (1995) Sensitivity to errors in artificial neural networks: a behavioural approach. IEEE Trans Circuits Syst I 42:358–361
Bailey-Kellogg C, Ramakrishnan N (2001) Ambiguity-directed sampling for qualitative analysis of sparse data from spatially-distributed physical systems. In: Proceedings of the IJCAI
Bartlett PL, Maass W (2003) Vapnik–Chervonenkis dimension of neural nets. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, pp 1188–1192
Cawley GC, Janacek GJ, Haylock MR, Dorling SR (2007) Predictive uncertainty in environmental modelling. Neural Netw 20:537–549
Christensen S (2013) Optimal decision under ambiguity for diffusion processes. Math Meth Oper Res 77:207–226
Clarke CLA, Kolla M, Vechtomova O (2009) An effectiveness measure for ambiguous and underspecified queries. In: Advances in information retrieval theory, lecture notes in computer science, vol 5766, pp 188–199
Cox E (2005) Fuzzy modeling and genetic algorithms for data mining and exploration. Elsevier, Amsterdam
dos Santos E, Sabourin R, Maupin P (2007) Ambiguity-guided dynamic selection of ensemble of classifiers. In: 10th international conference on information fusion
Denby B (1999) Neural networks in high energy physics: a ten year perspective. Comput Phys Commun 119:219–231
Denby B, Lessner E, Lindsey CS (1990) Test of track segment and vertex finding with neural networks. In: Proceedings of the 1990 conference on computing in high energy physics, Sante Fe, NM. AIP conference proceedings, vol 209, p 211
Dente JA, Vilela Mendes R (1996) Unsupervised learning in general connectionist systems. Netw Comput Neural Syst 7:123–139
Dente JA, Vilela Mendes R (1997) Characteristic functions and process identification from neural networks. Neural Netw 10:1465–1471
Doyne J (1990) Farmer. A Rosetta stone for connectionism. Phys D 42:153–187
Gabrys B, Bargiela A (1999) Neural network based decision support in presence of uncertainties. ASCE J Water Resour Plan Manag 125:272–280
Gernoth KA, Clark JW (1995) A modified backpropagation algorithm for training neural networks on data with error bars. Comput Phys Commun 88:1–22
Hao Xu, Jagannathan S (2013) Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming. IEEE Trans Neural Netw Learn Syst 24:471–484
Huang G, Song S, Wu C, You K (2012) Robust support vector regression for uncertain input and output data. IEEE Trans Neural Netw Learn Syst 23:1690–1700
Inukai K, Takahashi T (2009) Decision under ambiguity: effects of sign and magnitude. Int J Neurosci 119:1170–1178
Klir GJ (2006) Uncertainty and information: foundations of generalized information theory. Wiley, Hoboken
Klir GJ, Smith RM (2001) On measuring uncertainty and uncertainty-based information: recent developments. Ann Math Artif Intell 32:5–33
Lando D (2004) Credit risk modeling. Princeton University Press, Princeton
Lin Y-M, Wang X, Ng WWY, Chang Q, Yeung DS, Wang X-L (2006) Sphere classification for ambiguous data. In: 2006 International conference on machine learning and cybernetics, IEEE conference publications, pp 2571–2574
Martins J, Vilela Mendes R (2001) Neural networks and logical reasoning systems. A translation table. Int J Neural Syst 11:179–186
Ortega J, Koppel M, Argamon S (2001) Arbitrating among competing classifiers using learned referees. Knowl Inf Syst 3:470–490
Pacelli V, Azzollini M (2011) An artificial neural network approach for credit risk management. J Intell Learn Syst Appl 3:103–112
Reppa V, Polycarpou MM, Panayiotou CG (2014) Adaptive approximation for multiple sensor fault detection and isolation of nonlinear uncertain systems. IEEE Trans Neural Netw Learn Syst 25:137–153
Roul RK, Sahay SK (2012) An effective information retrieval for ambiguous query. Asian J Comput Sci Inf Technol 2:26–30
Svendsen JGG (2003) A SearchFree approach to ambiguity resolution. In: Proceedings of the 16th international technical meeting of the satellite Division of the Institute of Navigation. Portland, pp 769–774
Thomas LC, Edelman DB, Crook JN (2002) Credit scoring and its applications. SIAM, Philadelphia
UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/
Van Gestel T, Baesens B (2009) Credit risk management: basic concepts: financial risk components. Rating analysis, models, economic and regulatory capital. Oxford University Press, Oxford
Vapnik V (2000) The nature of statistical learning theory. Springer, New York
Vapnik V, Chervonenkis A (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16:264–280
Vapnik V, Levin E, Le Cun Y (1994) Measuring the VC-dimension of a learning machine. Neural Comput 6:851–876
Vigo R (2013) Complexity over uncertainty in generalized representational information theory: a structure-sensitive general theory of information. Information 4:1–30
West D (2000) Neural network credit scoring models. Comput Oper Res 27:1131–1152
Wong SKM, Wang ZW (1993) Qualitative measures of ambiguity. In: Proceedings of the 9th conference on uncertainty in artificial intelligence, pp 443–450
IEEE (1993) Proceedings of second international symposium on uncertainty modeling and analysis. In: IEEE conference publications. doi:10.1109/ISUMA.1993.366801
Yan Z, Wang J (2014) Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks. IEEE Trans Neural Netw Learn Syst 25:457–469
Zhang Q, Dong C, Cui Y, Yang Z (2014) Dynamic uncertain causality graph for knowledge representation and probabilistic reasoning: statistics base, matrix and application. IEEE Trans Neural Netw Learn Syst 25:645–663
This study was funded by Fundação para a Ciência e Tecnologia, Portugal.
Conflict of interest
The authors declare that they have no conflict of interest.
This article does not contain any studies with human participants performed by any of the authors.
Communicated by V. Loia.
Appendix: Evolution of the mean square error when the number of neurons in the hidden layer changes
About this article
Cite this article
Ligeiro, R., Vilela Mendes, R. Detecting and quantifying ambiguity: a neural network approach. Soft Comput 22, 2695–2703 (2018). https://doi.org/10.1007/s00500-017-2525-7
- Neural networks