A Methodology to Involve Domain Experts and Machine Learning Techniques in the Design of Human-Centered Algorithms
Machine learning techniques are increasingly applied in Decision Support Systems. The selection processes underlying a conclusion often become black-boxed. Thus, the decision flow is not always comprehensible by developers or end users. It is unclear what the priorities are and whether all of the relevant information is used. In order to achieve human interpretability of the created algorithms, it is recommended to include domain experts in the modelling phase. Their knowledge is elicited through a combination of machine learning and social science techniques. The idea is not new, but it remains a challenge to extract and apply the experts’ experience without overburdening them. The current paper describes a methodology set to unravel, define and categorize the implicit and explicit domain knowledge in a less intense way by making use of co-creation to design human-centered algorithms, when little data is available. The methodology is applied to a case in the health domain, targeting a rheumatology triage problem. The domain knowledge is obtained through dialogue, by alternating workshops and data science exercises.
KeywordsHuman-centered algorithms Decision Support Systems Knowledge elicitation methods Knowledge engineering
We would like to thank Klaas Vandevyvere, MD, for his support in this research, both as project initiator and rheumatologist.
- 9.Becerra-Fernandez, I., Sabherwal, R.: Knowledge management systems and processes (2010)Google Scholar
- 12.Ribeiro, M.T., Singh, S., Guestrin, C.: “Why should I trust you?”: explaining the predictions of any classifier (2016)Google Scholar
- 13.Druzdzel, M.J., Flynn, R.R.: Decision support systems. In: Encyclopedia of Library and Information Science, pp. 1–15 (2002)Google Scholar
- 14.Dalinger, E.: A framework for design of an integrated system for decision support and training. In: Proceedings of the 31st European Conference on Cognitive Ergonomics, ECCE 2013, p. 11 (2013)Google Scholar
- 16.Power, D.J.: A Brief History of Decision Support Systems (2007). http://dssresources.com/history/dsshistoryv28.html
- 17.Garcia-Taylor, M.C.: Development of a knowledge-based framework for demand management for refrigerated and shelf-life constrained food supply chains (2016)Google Scholar
- 20.Vandewiele, G.: Enhancing white-box machine learning processes by incorporating semantic background knowledge. In: Blomqvist, E., Maynard, D., Gangemi, A., Hoekstra, R., Hitzler, P., Hartig, O. (eds.) ESWC 2017, Part II. LNCS, vol. 10250, pp. 267–278. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58451-5_21CrossRefGoogle Scholar
- 21.Turban, E., Volonio, L., Mclean, E., Wetherbe, J.: Information Technology for Management: Transforming Organizations in the Digital Economy. John Wiley & Sons, New York (2009)Google Scholar
- 22.Lindgren, H.: Decision support system supporting clinical reasoning process – an evaluation study in dementia care. Stud. Health Technol. Inform. 136, 315–320 (2008). eHealth Beyond Horiz. – Get IT ThereGoogle Scholar
- 23.Sanders, L., Stappers, P.J.: Convivial Toolbox: Generative Research for the Front End of Design. BIS Publishers, Amsterdam (2013)Google Scholar
- 27.Quinlan, J.R.: C4.5: Programs for Machine Learning. Kaufmann Publishers, San Francisco (1993)Google Scholar
- 28.Is See5/C5.0 Better Than C4.5? Springer (2013)Google Scholar