Validation workflow for a clinical Bayesian network model in multidisciplinary decision making in head and neck oncology treatment

  • Mario A. CypkoEmail author
  • Matthaeus Stoehr
  • Marcin Kozniewski
  • Marek J. Druzdzel
  • Andreas Dietz
  • Leonard Berliner
  • Heinz U. Lemke
Original Article



Oncological treatment is being increasingly complex, and therefore, decision making in multidisciplinary teams is becoming the key activity in the clinical pathways. The increased complexity is related to the number and variability of possible treatment decisions that may be relevant to a patient. In this paper, we describe validation of a multidisciplinary cancer treatment decision in the clinical domain of head and neck oncology.


Probabilistic graphical models and corresponding inference algorithms, in the form of Bayesian networks, can support complex decision-making processes by providing a mathematically reproducible and transparent advice. The quality of BN-based advice depends on the quality of the model. Therefore, it is vital to validate the model before it is applied in practice.


For an example BN subnetwork of laryngeal cancer with 303 variables, we evaluated 66 patient records. To validate the model on this dataset, a validation workflow was applied in combination with quantitative and qualitative analyses. In the subsequent analyses, we observed four sources of imprecise predictions: incorrect data, incomplete patient data, outvoting relevant observations, and incorrect model. Finally, the four problems were solved by modifying the data and the model.


The presented validation effort is related to the model complexity. For simpler models, the validation workflow is the same, although it may require fewer validation methods. The validation success is related to the model’s well-founded knowledge base. The remaining laryngeal cancer model may disclose additional sources of imprecise predictions.


Therapy decision support system Bayesian network Model validation Laryngeal cancer Head and neck oncology Multidisciplinary tumor board 



The authors would like to thank J. Gaebel, Y. Deng, S. Oeltze-Jafra, and A. Oniśko for their valuable comments and suggestions that lead to improvements in the quality of the paper.

Funding ICCAS is funded by the German Federal Ministry of Education and Research (BMBF). The statements made herein are solely the responsibility of the authors.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

For this type of study, formal consent is not required. This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

This articles does not contain patient information.


  1. 1.
    Chatenoud L, GaravelloW Pagan E, Bertuccio P, Gallus S, La Vecchia C, Negri E, Bosetti C (2016) Laryngeal cancermortality trends in European countries. Int J Cancer 138(4):833–842. doi: 10.1002/ijc.29833 CrossRefPubMedGoogle Scholar
  2. 2.
    Cypko MA, Hirsch D, Koch L, Stoehr M, Strauss G, K D (2015) Web-tool to support medical experts in probabilistic modelling using large Bayesian networks with an example of rhinosinusitis. Stud Health Technol Inform 216:259–263PubMedGoogle Scholar
  3. 3.
    Cypko MA, Stoehr M, Denecke K (2015) Web-based guiding of clinical experts through the modelling of treatment decision models using MEBN with an example of laryngeal cancer. Int J CARS 10(1):163–164Google Scholar
  4. 4.
    Cypko MA, Stoehr M, Denecke K, Dietz A, Lemke HU (2014) User interaction with MEBNs for large patient-specific treatment decision models with an example for laryngeal cancer. In: Proceeding of the 28th conference for computer assisted radiology and surgery. Fukuoka, JapanGoogle Scholar
  5. 5.
    DeGroot MH, Fienberg SE (1983) The comparison and evaluation of forecasters. J Roy Stat Soc. Series D (The Statistician) 32(1/2):12–22Google Scholar
  6. 6.
    Diez FJ, Mira J, Iturralde E, Zubillaga S (1997) DIAVAL, a Bayesian expert system for echocardiography. Artif Intell Med 10(1):59–73CrossRefPubMedGoogle Scholar
  7. 7.
    Druzdzel MJ (1999) GeNIe: a development environment for graphical decision-analytic models. In: Proceedings of the 1999 annual symposium of the American medical informatics association (AMIA-1999), p 1206. Washington, DCGoogle Scholar
  8. 8.
    Druzdzel MJ, Oniśko A, Schwartz D, Dowling JN, Wasyluk H (1999) Knowledge engineering for very large decision-analytic medical models. In: Proceedings of the 1999 annual meeting of the American medical informatics association, p 1049. Washington, DCGoogle Scholar
  9. 9.
    Ferlay J, Soerjomataram I, Ervik M, Dikshit R, Eser S, Mathers C, Rebelo M, Parkin DM, Forman D, Bray F (2014) Globocan 2014 v1.0. Technical report on cancer incidence and mortality worldwide: IARC CancerBase No. 11. International Agency for Research on Cancer, Lyon, FranceGoogle Scholar
  10. 10.
    Forastiere A, Weber R, Trotti A (2015) Organ preservation for advanced larynx cancer: issues and outcomes. J Clin Oncol 33(29):3262–3268CrossRefPubMedPubMedCentralGoogle Scholar
  11. 11.
    Kahneman D, Slovic P, Tversky A (eds) (1982) Judgment under uncertainty: Heuristics and biases. Cambridge University Press, Cambridge, EnglandGoogle Scholar
  12. 12.
    Lemke HU, Golubnitschaja O (2014) Towards personal health care with model-guided medicine: long-term PPPM-related strategies and realisation opportunities within ’Horizon 2020’. EPMA J 5(1):1–9CrossRefGoogle Scholar
  13. 13.
    Manoogian J, Benson B (2017) Cognitive bias codex.
  14. 14.
    Moore AW, Lee MS (1994) Efficient algorithms for minimizing cross validation error. In: 11th international conference on machine learning, pp 842–846. Morgan Kaufmann, San Francisco, CaliforniaGoogle Scholar
  15. 15.
    National Comprehensive Cancer Network (2016) Head and Neck Cancer Cancers. v1. 2016Google Scholar
  16. 16.
    Oniśko A (2003) Probabilistic causal models in medicine: application to diagnosis of liver disorders. Ph.D. thesis, Institute of Biocybernetics and Biomedical Engineering, Polish Academy of Science, WarsawGoogle Scholar
  17. 17.
    Pearl J (1998) Probabilistic reasoning in intelligent systems. Morgan Kaufmann, BurlingtonGoogle Scholar
  18. 18.
    Pitchforth J, Mengersen K (2013) A proposed validation framework for expert elicited Bayesian networks. Expert Syst Appl 40(1):162–167CrossRefGoogle Scholar
  19. 19.
    Stoehr M, Cypko MA, Denecke K, Lemke HU, Dietz A (2014) A model of the decision-making process: therapy of laryngeal cancer. Int J CARS 9(Suppl 1):217–218Google Scholar
  20. 20.
    Witten IH, Eibe F (2005) Data mining: practical machine learning tools and techniques. Morgan Kaufmann, BurlingtonGoogle Scholar

Copyright information

© CARS 2017

Authors and Affiliations

  • Mario A. Cypko
    • 1
    Email author
  • Matthaeus Stoehr
    • 2
  • Marcin Kozniewski
    • 3
    • 4
  • Marek J. Druzdzel
    • 3
    • 4
  • Andreas Dietz
    • 2
  • Leonard Berliner
    • 5
  • Heinz U. Lemke
    • 1
  1. 1.Innovation Center Computer Assisted SurgeryUniversity of LeipzigLeipzigGermany
  2. 2.Clinic of Otolaryngology, Head and Neck Surgery, Department of Head Medicine and Oral HealthUniversity of LeipzigLeipzigGermany
  3. 3.School of Information SciencesUniversity of PittsburghPittsburghUSA
  4. 4.Faculty of Computer Science Bialystok University of TechnologyBialystokPoland
  5. 5.New York Methodist HospitalBrooklynUSA

Personalised recommendations