A Method for Uncertainty Elicitation of Experts Using Belief Function

  • Tuan Nha HoangEmail author
  • Tien Tuan Dao
  • Marie-Christine Ho Ba Tho
Part of the Studies in Computational Intelligence book series (SCI, volume 769)


The reliability of the biomedical data plays an essential role in the translation of the computational models and simulations of the human body systems into clinical decision support. Numerical models are commonly linked to the hypotheses on the data range of values due to the lack of in vivo data for some biomaterial variables. However, the reliability of these data is still not fully understood due to a lack of a systematic evaluation approach. The objective of this present study was to assess the reliability of biomedical data using expert judgment and belief theory. A systematic evaluation framework was developed using belief theory to perform the expert elicitation process. Seven parameters related to the muscle morphology and mechanics and motion analysis were selected. Twenty data sources related to these parameters were acquired using a systematic review process on the reliable search engines. A questionnaire was established including four main questions and four complementary questions related to the confidence levels. Eleven experts participated into the evaluation process via Google Form. A transformation process was developed to convert qualitative expert judgments to the numeric representations of the mass functions in the framework of belief theory. Two combination rules (Demspter and Dubois-Prade) were used to fuse the responses of multiple experts. At the end, data reliability was assessed using the pignistic probability to select the sources that correspond to some on-demand levels of confidence.


Biomechanical data reliability Expert opinion Expert elicitation Belief theory 



The authors would like to thank all anonymous experts participating into the evaluation process.


This work was carried out and funded in the framework of the Labex MS2T. It was supported by the French Government, through the program “Investments for the future” managed by the National Agency for Research (Reference ANR-11-IDEX-0004-02).


  1. 1.
    Aboal, J.R., Boquete, M.T., Carballeira, A., Casanova, A., Fernández, J.A.: Quantification of the overall measurement uncertainty associated with the passive moss biomonitoring technique: sample collection and processing. Environ. Pollut. 224, 235–242 (2017)CrossRefGoogle Scholar
  2. 2.
    Boone, I., Van der Stede, Y., Bollaerts, K., Messens, W., Mintiens, K.: Expert judgement in a risk assessment model for Salmonella spp. in pork: the performance of different weighting schemes. Prev. Vet. Med. 92(3), 224–234 (2009)CrossRefGoogle Scholar
  3. 3.
    Charles Osuagwu, C., Okafor, E.C.: Framework for eliciting knowledge for a medical laboratory diagnostic expert system. Expert Syst. Appl. 37(7), 5009–5016 (2010)CrossRefGoogle Scholar
  4. 4.
    Chatterjee, S., Bhattacharyya, M.: Judgment analysis of crowdsourced opinions using biclustering. Inf. Sci. 375(1), 138–154 (2017)CrossRefGoogle Scholar
  5. 5.
    Cobb, J.B.R., Shenoy, P.P.: On the plausibility transformation method for translating belief function models to probability models. Int. J. Approx. Reason. 41(3), 314–330 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Dubois D., Prade, H.: Representation and combination of uncertainty with belief functions and possibility measures. Comput. Intell. (1988)Google Scholar
  7. 7.
    Hanea, D.M., Jagtman, H.M., van Alphen, L.L.M.M., Ale, B.J.M.: Quantitative and qualitative analysis of the expert and non-expert opinion in fire risk in buildings. Reliab. Eng. Syst. Saf. 95(7), 729–741 (2010)CrossRefGoogle Scholar
  8. 8.
    Jörg, E., Julia, H., Valentin, Q., Markus, T., Björn, R.: Biomechanical model based evaluation of Total Hip Arthroplasty therapy outcome. J. Orthop. 14(4), 582–588 (2017)CrossRefGoogle Scholar
  9. 9.
    Lev, V.U.: A method for processing the unreliable expert judgments about parameters of probability distributions. Eur. J. Oper. Res. 175(1), 385–398 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Nicholas, T., Danielle, P., Nikhil, V.D., Robert, P.L.: Biomechanical analysis of gait waveform data: exploring differences between shod and barefoot running in habitually shod runners. Gait Posture 58, 274–279 (2017)CrossRefGoogle Scholar
  11. 11.
    Nicolas, R., Didier, P., Julie, C., Johanna, R., Raphael, Z.: Categorization of gait patterns in adults with cerebral palsy: a clustering approach. Gait Posture 39(1), 235–240 (2014)CrossRefGoogle Scholar
  12. 12.
    Pauk, J., Minta-Bielecka, K.: Gait patterns classification based on cluster and bicluster analysis. Biocybern. Biomed. Eng. 36(2), 391–396 (2016)CrossRefGoogle Scholar
  13. 13.
    Rustem, B., Robin Becker, G., Wolfgang, M.: Robust min–max portfolio strategies for rival forecast and risk scenarios. J. Econ. Dyn. Control 24(11), 1591–1621 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Samuel, T.R., Alejandro, S.: Error correction in multi-fidelity molecular dynamics simulations using functional uncertainty quantification. J. Comput. Phys. 334(1), 207–220 (2017)MathSciNetGoogle Scholar
  15. 15.
    Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press (1976)Google Scholar
  16. 16.
    Skinner, D.J.C., Rocks, S.A., Pollard, S.J.T.: Where do uncertainties reside within environmental risk assessments? Expert opinion on uncertainty distributions for pesticide risks to surface water organisms. Sci. Total Environ. 572, 23–33Google Scholar
  17. 17.
    Smets, P.: Data fusion in the transferable belief model. In: Proceedings of 3rd International Conference on Information Fusion, Paris, France, pp. 21–33 (2000)Google Scholar
  18. 18.
    Smets, P.: Belief functions: the disjunctive rule of combination and the generalized Bayesian theorem. Int. J. Approx. Reason. 9(1), 1–35 (1993)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Wang, P., Ma, Z., Tian, Y.: Application of expert judgment method in the aircraft wiring risk assessment. Proc. Eng. 17, 440–445 (2011)CrossRefGoogle Scholar
  20. 20.
    Yun, Z., Norman, F., Martin, N.: Bayesian network approach to multinomial parameter learning using data and expert judgments. Int. J. Approx. Reason. 55(5), 1252–1268 (2014)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Tuan Nha Hoang
    • 1
    Email author
  • Tien Tuan Dao
    • 2
  • Marie-Christine Ho Ba Tho
    • 2
  1. 1.Quang Binh UniversityQuang BinhVietnam
  2. 2.Sorbonne University, Université de Technologie de CompiègneCompiègneFrance

Personalised recommendations