Detecting and quantifying ambiguity: a neural network approach

Abstract

In general, it is not possible to have access to all variables that determine the behavior of a system. Once a number of measurable variables is identified, there might still exist hidden variables which influence the behavior of the system. The result is model ambiguity in the sense that, for the same (or very similar) input values, distinct outputs are obtained. In addition, the degree of ambiguity may vary across the range of input values. Therefore, to evaluate the accuracy of a model it is important to devise a method to obtain the degree of reliability for each output result. In this paper, we present such a scheme composed of two coupled neural networks, the first one computing the average predicted value and the other the reliability of the output, which is learned from the error values of the first one. As an illustration, the scheme is applied to a model for tracking slopes in a straw chamber and to a credit scoring model.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

References

  1. Albalate A, Suchindranath A, Soenmez MM, Suendermann D (2010) On ambiguity detection and postprocessing schemes using cluster ensembles, ICAART 2010. In: Proceedings of the international conference on agents and artificial intelligence. INSTICC Press, pp 623–630

  2. Alfaro-Ponce M, Cruz AA, Chairez I (2014) Adaptive identifier for uncertain complex nonlinear systems based on continuous neural networks. IEEE Trans Neural Netw Learn Syst 25:483–494

    Article  Google Scholar 

  3. Alippi C (1999) FPE-based criteria to dimension feedforward neural topologies. IEEE Trans Circuits Syst I 46:962–973

    Article  Google Scholar 

  4. Alippi C, Piuri V, Sami M (1995) Sensitivity to errors in artificial neural networks: a behavioural approach. IEEE Trans Circuits Syst I 42:358–361

    Article  Google Scholar 

  5. Bailey-Kellogg C, Ramakrishnan N (2001) Ambiguity-directed sampling for qualitative analysis of sparse data from spatially-distributed physical systems. In: Proceedings of the IJCAI

  6. Bartlett PL, Maass W (2003) Vapnik–Chervonenkis dimension of neural nets. In: Arbib MA (ed) The handbook of brain theory and neural networks. MIT Press, Cambridge, pp 1188–1192

    Google Scholar 

  7. Cawley GC, Janacek GJ, Haylock MR, Dorling SR (2007) Predictive uncertainty in environmental modelling. Neural Netw 20:537–549

    Article  MATH  Google Scholar 

  8. Christensen S (2013) Optimal decision under ambiguity for diffusion processes. Math Meth Oper Res 77:207–226

    MathSciNet  Article  MATH  Google Scholar 

  9. Clarke CLA, Kolla M, Vechtomova O (2009) An effectiveness measure for ambiguous and underspecified queries. In: Advances in information retrieval theory, lecture notes in computer science, vol 5766, pp 188–199

  10. Cox E (2005) Fuzzy modeling and genetic algorithms for data mining and exploration. Elsevier, Amsterdam

    Google Scholar 

  11. dos Santos E, Sabourin R, Maupin P (2007) Ambiguity-guided dynamic selection of ensemble of classifiers. In: 10th international conference on information fusion

  12. Denby B (1999) Neural networks in high energy physics: a ten year perspective. Comput Phys Commun 119:219–231

    Article  Google Scholar 

  13. Denby B, Lessner E, Lindsey CS (1990) Test of track segment and vertex finding with neural networks. In: Proceedings of the 1990 conference on computing in high energy physics, Sante Fe, NM. AIP conference proceedings, vol 209, p 211

  14. Dente JA, Vilela Mendes R (1996) Unsupervised learning in general connectionist systems. Netw Comput Neural Syst 7:123–139

    MATH  Google Scholar 

  15. Dente JA, Vilela Mendes R (1997) Characteristic functions and process identification from neural networks. Neural Netw 10:1465–1471

    Article  Google Scholar 

  16. Doyne J (1990) Farmer. A Rosetta stone for connectionism. Phys D 42:153–187

    Article  Google Scholar 

  17. Gabrys B, Bargiela A (1999) Neural network based decision support in presence of uncertainties. ASCE J Water Resour Plan Manag 125:272–280

    Article  Google Scholar 

  18. Gernoth KA, Clark JW (1995) A modified backpropagation algorithm for training neural networks on data with error bars. Comput Phys Commun 88:1–22

    Article  MATH  Google Scholar 

  19. Hao Xu, Jagannathan S (2013) Stochastic optimal controller design for uncertain nonlinear networked control system via neuro dynamic programming. IEEE Trans Neural Netw Learn Syst 24:471–484

    Article  Google Scholar 

  20. Huang G, Song S, Wu C, You K (2012) Robust support vector regression for uncertain input and output data. IEEE Trans Neural Netw Learn Syst 23:1690–1700

    Article  Google Scholar 

  21. Inukai K, Takahashi T (2009) Decision under ambiguity: effects of sign and magnitude. Int J Neurosci 119:1170–1178

    Article  Google Scholar 

  22. Klir GJ (2006) Uncertainty and information: foundations of generalized information theory. Wiley, Hoboken

    Google Scholar 

  23. Klir GJ, Smith RM (2001) On measuring uncertainty and uncertainty-based information: recent developments. Ann Math Artif Intell 32:5–33

    MathSciNet  Article  MATH  Google Scholar 

  24. Lando D (2004) Credit risk modeling. Princeton University Press, Princeton

    Google Scholar 

  25. Lin Y-M, Wang X, Ng WWY, Chang Q, Yeung DS, Wang X-L (2006) Sphere classification for ambiguous data. In: 2006 International conference on machine learning and cybernetics, IEEE conference publications, pp 2571–2574

  26. Martins J, Vilela Mendes R (2001) Neural networks and logical reasoning systems. A translation table. Int J Neural Syst 11:179–186

    Google Scholar 

  27. Ortega J, Koppel M, Argamon S (2001) Arbitrating among competing classifiers using learned referees. Knowl Inf Syst 3:470–490

    Article  MATH  Google Scholar 

  28. Pacelli V, Azzollini M (2011) An artificial neural network approach for credit risk management. J Intell Learn Syst Appl 3:103–112

    Google Scholar 

  29. Reppa V, Polycarpou MM, Panayiotou CG (2014) Adaptive approximation for multiple sensor fault detection and isolation of nonlinear uncertain systems. IEEE Trans Neural Netw Learn Syst 25:137–153

    Article  Google Scholar 

  30. Roul RK, Sahay SK (2012) An effective information retrieval for ambiguous query. Asian J Comput Sci Inf Technol 2:26–30

    Google Scholar 

  31. Svendsen JGG (2003) A SearchFree approach to ambiguity resolution. In: Proceedings of the 16th international technical meeting of the satellite Division of the Institute of Navigation. Portland, pp 769–774

  32. Thomas LC, Edelman DB, Crook JN (2002) Credit scoring and its applications. SIAM, Philadelphia

    Google Scholar 

  33. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml/

  34. Van Gestel T, Baesens B (2009) Credit risk management: basic concepts: financial risk components. Rating analysis, models, economic and regulatory capital. Oxford University Press, Oxford

    Google Scholar 

  35. Vapnik V (2000) The nature of statistical learning theory. Springer, New York

    Google Scholar 

  36. Vapnik V, Chervonenkis A (1971) On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab Appl 16:264–280

    Article  MATH  Google Scholar 

  37. Vapnik V, Levin E, Le Cun Y (1994) Measuring the VC-dimension of a learning machine. Neural Comput 6:851–876

    Article  Google Scholar 

  38. Vigo R (2013) Complexity over uncertainty in generalized representational information theory: a structure-sensitive general theory of information. Information 4:1–30

    Article  Google Scholar 

  39. West D (2000) Neural network credit scoring models. Comput Oper Res 27:1131–1152

    Article  MATH  Google Scholar 

  40. Wong SKM, Wang ZW (1993) Qualitative measures of ambiguity. In: Proceedings of the 9th conference on uncertainty in artificial intelligence, pp 443–450

  41. IEEE (1993) Proceedings of second international symposium on uncertainty modeling and analysis. In: IEEE conference publications. doi:10.1109/ISUMA.1993.366801

  42. Yan Z, Wang J (2014) Robust model predictive control of nonlinear systems with unmodeled dynamics and bounded uncertainties based on neural networks. IEEE Trans Neural Netw Learn Syst 25:457–469

    Article  Google Scholar 

  43. Zhang Q, Dong C, Cui Y, Yang Z (2014) Dynamic uncertain causality graph for knowledge representation and probabilistic reasoning: statistics base, matrix and application. IEEE Trans Neural Netw Learn Syst 25:645–663

    Article  Google Scholar 

Download references

Acknowledgements

This study was funded by Fundação para a Ciência e Tecnologia, Portugal.

Author information

Affiliations

Authors

Corresponding author

Correspondence to R. Vilela Mendes.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Additional information

Communicated by V. Loia.

Appendix: Evolution of the mean square error when the number of neurons in the hidden layer changes

Appendix: Evolution of the mean square error when the number of neurons in the hidden layer changes

In two figures (Figs. 1011), we display the mean square error after training, of the networks in the two examples, when the number of neurons in the hidden layer changes

Fig. 10
figure10

Mean square error after training when the number of neurons in the hidden layer changes (straw chamber)

Fig. 11
figure11

Mean square error after training when the number of neurons in the hidden layer changes (credit scoring)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ligeiro, R., Vilela Mendes, R. Detecting and quantifying ambiguity: a neural network approach. Soft Comput 22, 2695–2703 (2018). https://doi.org/10.1007/s00500-017-2525-7

Download citation

Keywords

  • Uncertainty
  • Ambiguity
  • Neural networks