# Detecting and quantifying ambiguity: a neural network approach

- 152 Downloads

## Abstract

In general, it is not possible to have access to all variables that determine the behavior of a system. Once a number of measurable variables is identified, there might still exist hidden variables which influence the behavior of the system. The result is model ambiguity in the sense that, for the same (or very similar) input values, distinct outputs are obtained. In addition, the degree of ambiguity may vary across the range of input values. Therefore, to evaluate the accuracy of a model it is important to devise a method to obtain the degree of reliability for each output result. In this paper, we present such a scheme composed of two coupled neural networks, the first one computing the average predicted value and the other the reliability of the output, which is learned from the error values of the first one. As an illustration, the scheme is applied to a model for tracking slopes in a straw chamber and to a credit scoring model.

## Keywords

Uncertainty Ambiguity Neural networks## Notes

### Acknowledgements

This study was funded by Fundação para a Ciência e Tecnologia, Portugal.

### Compliance with ethical standards

### Conflict of interest

The authors declare that they have no conflict of interest.

### Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

## References

- Albalate A, Suchindranath A, Soenmez MM, Suendermann D (2010) On ambiguity detection and postprocessing schemes using cluster ensembles, ICAART 2010. In: Proceedings of the international conference on agents and artificial intelligence. INSTICC Press, pp 623–630Google Scholar
- Alfaro-Ponce M, Cruz AA, Chairez I (2014) Adaptive identifier for uncertain complex nonlinear systems based on continuous neural networks. IEEE Trans Neural Netw Learn Syst 25:483–494CrossRefGoogle Scholar
- Alippi C (1999) FPE-based criteria to dimension feedforward neural topologies. IEEE Trans Circuits Syst I 46:962–973CrossRefGoogle Scholar
- Alippi C, Piuri V, Sami M (1995) Sensitivity to errors in artificial neural networks: a behavioural approach. IEEE Trans Circuits Syst I 42:358–361CrossRefGoogle Scholar
- Bailey-Kellogg C, Ramakrishnan N (2001) Ambiguity-directed sampling for qualitative analysis of sparse data from spatially-distributed physical systems. In: Proceedings of the IJCAIGoogle Scholar
- Bartlett PL, Maass W (2003) Vapnik–Chervonenkis dimension of neural nets. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, pp 1188–1192Google Scholar
- Cawley GC, Janacek GJ, Haylock MR, Dorling SR (2007) Predictive uncertainty in environmental modelling. Neural Netw 20:537–549CrossRefMATHGoogle Scholar
- Christensen S (2013) Optimal decision under ambiguity for diffusion processes. Math Meth Oper Res 77:207–226MathSciNetCrossRefMATHGoogle Scholar
- Clarke CLA, Kolla M, Vechtomova O (2009) An effectiveness measure for ambiguous and underspecified queries. In: Advances in information retrieval theory, lecture notes in computer science, vol 5766, pp 188–199Google Scholar
- Cox E (2005) Fuzzy modeling and genetic algorithms for data mining and exploration. Elsevier, AmsterdamMATHGoogle Scholar
- dos Santos E, Sabourin R, Maupin P (2007) Ambiguity-guided dynamic selection of ensemble of classifiers. In: 10th international conference on information fusionGoogle Scholar
- Denby B (1999) Neural networks in high energy physics: a ten year perspective. Comput Phys Commun 119:219–231CrossRefGoogle Scholar
- Denby B, Lessner E, Lindsey CS (1990) Test of track segment and vertex finding with neural networks. In: Proceedings of the 1990 conference on computing in high energy physics, Sante Fe, NM. AIP conference proceedings, vol 209, p 211Google Scholar
- Dente JA, Vilela Mendes R (1996) Unsupervised learning in general connectionist systems. Netw Comput Neural Syst 7:123–139MATHGoogle Scholar
- Dente JA, Vilela Mendes R (1997) Characteristic functions and process identification from neural networks. Neural Netw 10:1465–1471CrossRefGoogle Scholar
- Doyne J (1990) Farmer. A Rosetta stone for connectionism. Phys D 42:153–187CrossRefGoogle Scholar
- Gabrys B, Bargiela A (1999) Neural network based decision support in presence of uncertainties. ASCE J Water Resour Plan Manag 125:272–280CrossRefGoogle Scholar
- Gernoth KA, Clark JW (1995) A modified backpropagation algorithm for training neural networks on data with error bars. Comput Phys Commun 88:1–22CrossRefMATHGoogle Scholar
- Hao Xu, Jagannathan S (2013) Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming. IEEE Trans Neural Netw Learn Syst 24:471–484CrossRefGoogle Scholar
- Huang G, Song S, Wu C, You K (2012) Robust support vector regression for uncertain input and output data. IEEE Trans Neural Netw Learn Syst 23:1690–1700CrossRefGoogle Scholar
- Inukai K, Takahashi T (2009) Decision under ambiguity: effects of sign and magnitude. Int J Neurosci 119:1170–1178CrossRefGoogle Scholar
- Klir GJ (2006) Uncertainty and information: foundations of generalized information theory. Wiley, HobokenMATHGoogle Scholar
- Klir GJ, Smith RM (2001) On measuring uncertainty and uncertainty-based information: recent developments. Ann Math Artif Intell 32:5–33MathSciNetCrossRefMATHGoogle Scholar
- Lando D (2004) Credit risk modeling. Princeton University Press, PrincetonMATHGoogle Scholar
- Lin Y-M, Wang X, Ng WWY, Chang Q, Yeung DS, Wang X-L (2006) Sphere classification for ambiguous data. In: 2006 International conference on machine learning and cybernetics, IEEE conference publications, pp 2571–2574Google Scholar
- Martins J, Vilela Mendes R (2001) Neural networks and logical reasoning systems. A translation table. Int J Neural Syst 11:179–186Google Scholar
- Ortega J, Koppel M, Argamon S (2001) Arbitrating among competing classifiers using learned referees. Knowl Inf Syst 3:470–490CrossRefMATHGoogle Scholar
- Pacelli V, Azzollini M (2011) An artificial neural network approach for credit risk management. J Intell Learn Syst Appl 3:103–112Google Scholar
- Reppa V, Polycarpou MM, Panayiotou CG (2014) Adaptive approximation for multiple sensor fault detection and isolation of nonlinear uncertain systems. IEEE Trans Neural Netw Learn Syst 25:137–153CrossRefGoogle Scholar
- Roul RK, Sahay SK (2012) An effective information retrieval for ambiguous query. Asian J Comput Sci Inf Technol 2:26–30Google Scholar
- Svendsen JGG (2003) A SearchFree approach to ambiguity resolution. In: Proceedings of the 16th international technical meeting of the satellite Division of the Institute of Navigation. Portland, pp 769–774Google Scholar
- Thomas LC, Edelman DB, Crook JN (2002) Credit scoring and its applications. SIAM, PhiladelphiaCrossRefMATHGoogle Scholar
- UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/
- Van Gestel T, Baesens B (2009) Credit risk management: basic concepts: financial risk components. Rating analysis, models, economic and regulatory capital. Oxford University Press, OxfordGoogle Scholar
- Vapnik V (2000) The nature of statistical learning theory. Springer, New YorkCrossRefMATHGoogle Scholar
- Vapnik V, Chervonenkis A (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16:264–280CrossRefMATHGoogle Scholar
- Vapnik V, Levin E, Le Cun Y (1994) Measuring the VC-dimension of a learning machine. Neural Comput 6:851–876CrossRefGoogle Scholar
- Vigo R (2013) Complexity over uncertainty in generalized representational information theory: a structure-sensitive general theory of information. Information 4:1–30CrossRefGoogle Scholar
- West D (2000) Neural network credit scoring models. Comput Oper Res 27:1131–1152CrossRefMATHGoogle Scholar
- Wong SKM, Wang ZW (1993) Qualitative measures of ambiguity. In: Proceedings of the 9th conference on uncertainty in artificial intelligence, pp 443–450Google Scholar
- IEEE (1993) Proceedings of second international symposium on uncertainty modeling and analysis. In: IEEE conference publications. doi: 10.1109/ISUMA.1993.366801
- Yan Z, Wang J (2014) Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks. IEEE Trans Neural Netw Learn Syst 25:457–469CrossRefGoogle Scholar
- Zhang Q, Dong C, Cui Y, Yang Z (2014) Dynamic uncertain causality graph for knowledge representation and probabilistic reasoning: statistics base, matrix and application. IEEE Trans Neural Netw Learn Syst 25:645–663CrossRefGoogle Scholar