What does it mean to provide decision support to a responsible and competent expert?

The case of diagnostic decision support systems

Abstract

Decision support consists in helping a decision-maker to improve his/her decisions. However, clients requesting decision support are often themselves experts and are often taken by third parties and/or the general public to be responsible for the decisions they make. This predicament raises complex challenges for decision analysts, who have to avoid infringing upon the expertise and responsibility of the decision-maker. The case of diagnosis decision support in healthcare contexts is particularly illustrative. To support clinicians in their work and minimize the risk of medical error, various decision support systems have been developed, as part of information systems that are now ubiquitous in healthcare contexts. To develop, in collaboration with the hospitals of Lyon, a diagnostic decision support system for day-to-day customary consultations, we propose in this paper a critical analysis of current approaches to diagnostic decision support, which mainly consist in providing them with guidelines or even full-fledged diagnosis recommendations. We highlight that the use of such decision support systems by physicians raises responsibility issues, but also that it is at odds with the needs and constraints of customary consultations. We argue that the historical choice to favor guidelines or recommendations to physicians implies a very specific vision of what it means to support physicians, and we argue that the flaws of this vision partially explain why current diagnostic decision support systems are not accepted by physicians in their application to customary situations. Based on this analysis, we propose that decision support to physicians for customary cases should be deployed in an “adjustive” approach, which consists in providing physicians with the data on patients they need, when they need them, during consultations. The rationale articulated in this article has a more general bearing than clinical decision support and bears lessons for decision support activities in other contexts where decision-makers are competent and responsible experts.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    www.mims.co.

  2. 2.

    www.nice.org.uk/guidance/ng28.

  3. 3.

    www.nice.org.uk.

  4. 4.

    www.has-sante.fr.

  5. 5.

    http://www.mghlcs.org/projects/dxplain.

  6. 6.

    Some exceptions exist to the two predominant subsets of DDSSs (Guideline-based and ML-based DDSSs). Gräßer et al. (2017) proposed a DDSS dedicated to providing therapy recommendations, based not on expert guidelines or machine learning algorithms, but on similarity measures between the current case and previous ones, computed for each new cases, without any learning process involved. Whereas this system is akin to ML-based DDSSs, it does not use ML algorithms. Similarly, Giordanengo et al. (2019) proposed a DDSS dedicated to presenting self-collected data on patients and reminders of actions to do to physicians during the consultations of patients with diabetes. In this work, Giordanengo et al. (2019) did not use the guidelines of any health authority but included physicians in the development process of the DDSS to establish rules to apply in specific situations. In addition, the recommendations established by consensus among the physicians involved are not intended for other physicians, but to developers adding needed features into the DDSS. Lastly, the ML-based DDSS proposed by Simon et al. (2019) does not use ML algorithms to make recommendations but to detect complex concepts in medical documents, facilitating access to information on patients or to reference documents. With this DDSS, Simon et al. (2019) showed that it is possible to use ML algorithms in other ways than by producing recommendations, while still providing support to physicians in practice.

  7. 7.

    The emerging field of Explainable AI (Doran et al. 2017; Gunning 2017; Rudin and Radin 2019) holds promises to mitigate this problem.

  8. 8.

    https://futureoflife.org/ai-principles/.

References

  1. Anand V, Biondich PG, Liu GC, Rosenman MB, Downs SM et al (2004) Child health improvement through computer automation: the chica system. Medinfo. https://doi.org/10.3233/978-1-60750-949-3-187

    Article  Google Scholar 

  2. Barnett GO, Cimino JJ, Hupp JA, Hoffer EP (1987) Dxplain: an evolving diagnostic decision-support system. JAMA 258(1):67–74. https://doi.org/10.1001/jama.1987.03400010071030

    Article  Google Scholar 

  3. Bernasconi A, Crabbé F, Adedeji AM, Bello A, Schmitz T, Landi M, Rossi R (2019) Results from one-year use of an electronic clinical decision support system in a post-conflict context: An implementation research. PLoS One. https://doi.org/10.1371/journal.pone.0225634

    Article  Google Scholar 

  4. Berner ES (2016) Clinical decision support systems, 3rd edn. Springer, New York

    Google Scholar 

  5. Berner ES, Graber ML (2008) Overconfidence as a cause of diagnostic error in medicine. Am J Med 121(5):S2–S23. https://doi.org/10.1016/j.amjmed.2008.01.001

    Article  Google Scholar 

  6. Bertillot H (2016) Comment l’évaluation de la qualité transforme l’hôpital. les deux visages de la rationalisation par les indicateurs. Cahiers internationaux de sociologie de la gestion 15:11–48

    Google Scholar 

  7. Bessat C, Zonon NA, D’Acremont V (2019) Large-scale implementation of electronic integrated management of childhood illness (eimci) at the primary care level in burkina faso: a qualitative study on health worker perception of its medical content, usability and impact on antibiotic prescription and resistance. BMC Public Health 19(1):449. https://doi.org/10.1186/s12889-019-6692-6

    Article  Google Scholar 

  8. Beynon-Davies P, Lloyd-Williams M (1998) Health information systems, ‘safety’ and organizational learning. Health Inform J 4(3–4):128–137. https://doi.org/10.1177/146045829800400303

    Article  Google Scholar 

  9. Bowman S (2013) Impact of electronic health record systems on information integrity: quality and safety implications. Perspect Health Inf Manag 10(Fall):1c

    Google Scholar 

  10. Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PAC, Rubin HR (1999) Why don’t physicians follow clinical practice guidelines?: A framework for improvement. JAMA 282(15):1458–1465. https://doi.org/10.1001/jama.282.15.1458

    Article  Google Scholar 

  11. Cabitza F, Rasoini R, Gensini GF (2017) Unintended consequences of machine learning in medicine. JAMA 318(6):517–518. https://doi.org/10.1001/jama.2017.7797

    Article  Google Scholar 

  12. Carroll AE, Biondich P, Anand V, Dugan TM, Downs SM (2013) A randomized controlled trial of screening for maternal depression with a clinical decision support system. J Am Med Inform Assoc 20(2):311–316. https://doi.org/10.1136/amiajnl-2011-000682

    Article  Google Scholar 

  13. Chadwick D, Hall C, Rae C, Rayment M, Branch M, Littlewood J, Sullivan A (2017) A feasibility study for a clinical decision support system prompting HIV testing. HIV Med 18(6):435–439. https://doi.org/10.1111/hiv.12472

    Article  Google Scholar 

  14. Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K (2019) Artificial intelligence, bias and clinical safety. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2018-008370

    Article  Google Scholar 

  15. Chang PL, Yc Li, Huang ST, Wang TM, Hsien ML (1996) Effects of a medical expert system on differential diagnosis of renal masses: A prospective study. Comput Med Imaging Graph 20(1):43–48. https://doi.org/10.1016/0895-6111(96)00029-8

    Article  Google Scholar 

  16. Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG (2006) Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 144(10):742–752. https://doi.org/10.7326/0003-4819-144-10-200605160-00125

    Article  Google Scholar 

  17. Choulak M, Marage D, Gisbert M, Paris M, Meinard Y (2019) A meta-decision-analysis approach to structure operational and legitimate environmental policies-with an application to wetland prioritization. Sci Total Environ 655:384–394. https://doi.org/10.1016/j.scitotenv.2018.11.202

    Article  Google Scholar 

  18. Clancy CM (2009) Ten years after to err is human. Am J Med Qual 24(6):525–528. https://doi.org/10.1177/1062860609349728

    Article  Google Scholar 

  19. Dalaba MA, Akweongo P, Williams J, Saronga HP, Tonchev P, Sauerborn R, Mensah N, Blank A, Kaltschmidt J, Loukanova S (2014) Costs associated with implementation of computer-assisted clinical decision support system for antenatal and delivery care: case study of kassena-nankana district of northern ghana. PLoS One. https://doi.org/10.1371/journal.pone.0106416

    Article  Google Scholar 

  20. De Dombal F (1987) Ethical considerations concerning computers in medicine in the 1980s. J Med Ethics 13(4):179–184. https://doi.org/10.1136/jme.13.4.179

    Article  Google Scholar 

  21. De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S, Askham H, Glorot X, O’Donoghue B, Visentin D et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24(9):1342–1350. https://doi.org/10.1038/s41591-018-0107-6

    Article  Google Scholar 

  22. De Marchi G, Lucertini G, Tsoukiàs A (2016) From evidence-based policy making to policy analytics. Ann Oper Res 236(1):15–38. https://doi.org/10.1007/s10479-014-1578-6

    Article  Google Scholar 

  23. Deig CR, Kanwar A, Thompson RF (2019) Artificial intelligence in radiation oncology. Hematol Oncol Clin 33(6):1095–1104. https://doi.org/10.1016/j.hoc.2019.08.003

    Article  Google Scholar 

  24. Donaldson MS, Corrigan JM, Kohn LT et al (2000) To err is human: building a safer health system, vol 6. National Academies Press, Washington, D.C

    Google Scholar 

  25. Doran D, Schulz S, Besold TR (2017) What does explainable ai really mean? a new conceptualization of perspectives. arXiv preprint arXiv:171000794

  26. Dua S, Acharya UR, Dua P (2014) Machine learning in healthcare informatics, vol 56. Springer, New York

    Google Scholar 

  27. Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, Purves I (2002) Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ 325(7370):941. https://doi.org/10.1136/bmj.325.7370.941

    Article  Google Scholar 

  28. Elsner P, Bauer A, Diepgen TL, Drexler H, Fartasch M, John SM, Schliemann S, Wehrmann W, Tittelbach J (2018) Position paper: Telemedicine in occupational dermatology-current status and perspectives. JDDG: Journal der Deutschen Dermatologischen Gesellschaft 16(8):969–974. https://doi.org/10.1111/ddg.13605

    Article  Google Scholar 

  29. Elstein AS, Friedman CP, Wolf FM, Murphy G, Miller J, Fine P, Heckerling P, Miller T, Sisson J, Barlas S et al (1996) Effects of a decision support system on the diagnostic accuracy of users: a preliminary report. J Am Med Inform Assoc 3(6):422–428. https://doi.org/10.1136/jamia.1996.97084515

    Article  Google Scholar 

  30. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118. https://doi.org/10.1038/nature21056

    Article  Google Scholar 

  31. Etter DJ, McCord A, Ouyang F, Gilbert AL, Williams RL, Hall JA, Tu W, Downs SM, Aalsma MC (2018) Suicide screening in primary care: use of an electronic screener to assess suicidality and improve provider follow-up for adolescents. J Adolesc Health 62(2):191–197. https://doi.org/10.1016/j.jadohealth.2017.08.026

    Article  Google Scholar 

  32. Ferretti V, Pluchinotta I, Tsoukiàs A (2019) Studying the generation of alternatives in public policy making processes. Eur J Oper Res 273(1):353–363. https://doi.org/10.1016/j.ejor.2018.07.054

    Article  Google Scholar 

  33. Field MJ, Lohr KN et al (1990) Clinical practice guidelines: directions for a new program. National Academies Press, Washington, D.C

    Google Scholar 

  34. Fieschi M (1986) Expert systems for medical consultation. Health Policy 6(2):159–173. https://doi.org/10.1016/0168-8510(86)90005-9

    Article  Google Scholar 

  35. Garcia M (2016) Racist in the machine: The disturbing implications of algorithmic bias. World Policy J 33(4):111–117

    Article  Google Scholar 

  36. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux P, Beyene J, Sam J, Haynes RB (2005) Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293(10):1223–1238. https://doi.org/10.1001/jama.293.10.1223

    Article  Google Scholar 

  37. Giordanengo A, Årsand E, Woldaregay AZ, Bradway M, Grottland A, Hartvigsen G, Granja C, Torsvik T, Hansen AH (2019) Design and prestudy assessment of a dashboard for presenting self-collected health data of patients with diabetes to clinicians: Iterative approach and qualitative case study. JMIR Diabetes 4(3):e14002. https://doi.org/10.2196/14002

    Article  Google Scholar 

  38. Gonzalvo CM, de Wit HA, van Oijen BP, Hurkens KP, Janknegt R, Schols JM, Mulder WJ, Verhey FR, Winkens B, van der Kuy PHM (2017) Supporting clinical rules engine in the adjustment of medication (scream): protocol of a multicentre, prospective, randomised study. BMC Geriatr 17(1):35. https://doi.org/10.1186/s12877-017-0426-3

    Article  Google Scholar 

  39. Goodman KW (2016) Ethical and legal issues in decision support. Clinical decision support systems. Springer, New York, pp 131–146. https://doi.org/10.1007/978-3-319-31913-1_8

    Google Scholar 

  40. Gordon ES, Babu D, Laney DA (2018) The future is now: Technology’s impact on the practice of genetic counseling. Am J Med Genet Part C Semin Med Genet 178:15–23. https://doi.org/10.1002/ajmg.c.31599 (Wiley Online Library)

    Article  Google Scholar 

  41. Gräßer F, Beckert S, Küster D, Schmitt J, Abraham S, Malberg H, Zaunseder S (2017) Therapy decision support based on recommender system methods. J Healthc Eng. https://doi.org/10.1155/2017/8659460

    Article  Google Scholar 

  42. Guah M (1998) Evaluation and analysis of multimedia information system design and implementation at the Coloscopy unit of St. James university hospital, Leeds, UK. PhD thesis, MSc Dissertation, School of Management, UMIST, Manchester

  43. Gunning D (2017) Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web 2

  44. Hall LH, Johnson J, Watt I, Tsipa A, O’Connor DB (2016) Healthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One 11(7):e0159015. https://doi.org/10.1371/journal.pone.0159015

    Article  Google Scholar 

  45. Heckerling P, Elstein A, Terzian C, Kushner M (1991) The effect of incomplete knowledge on the diagnoses of a computer consultant system. Med Inform 16(4):363–370. https://doi.org/10.3109/14639239109067658

    Article  Google Scholar 

  46. Heeks R (2006) Health information systems: failure, success and improvisation. Int J Med Inform 75(2):125–137. https://doi.org/10.1016/j.ijmedinf.2005.07.024

    Article  Google Scholar 

  47. Heeks R, Mundy D, Salazar A (1999) Why health care information systems succeed or fail. Inform Syst Public Sector Manag. https://doi.org/10.2139/ssrn.3540062

    Article  Google Scholar 

  48. Hoffer EP, Feldman MJ, Kim RJ, Famiglietti KT, Barnett GO (2005) Dxplain: patterns of use of a mature expert system. In: AMIA annual symposium proceedings, American Medical Informatics Association, vol 2005, p 321

  49. Hollon TC, Lewis S, Pandian B, Niknafs YS, Garrard MR, Garton H, Maher CO, McFadden K, Snuderl M, Lieberman AP et al (2018) Rapid intraoperative diagnosis of pediatric brain tumors using stimulated raman histology. Cancer Res 78(1):278–289. https://doi.org/10.1158/0008-5472.CAN-17-1974

    Article  Google Scholar 

  50. Honaker SM, Dugan T, Daftary A, Davis S, Saha C, Baye F, Freeman E, Downs SM (2018) Unexplained practice variation in primary care providers’ concern for pediatric obstructive sleep apnea. Acad Pediatr 18(4):418–424. https://doi.org/10.1016/j.acap.2018.01.011

    Article  Google Scholar 

  51. Hornik K, Stinchcombe M, White H et al (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366. https://doi.org/10.1016/0893-6080(89)90020-8

    Article  Google Scholar 

  52. Horrocks M, Michail M, Aubeeluck A, Wright N, Morriss R (2018) An electronic clinical decision support system for the assessment and management of suicidality in primary care: protocol for a mixed-methods study. JMIR Res Protoc 7(12):e11135. https://doi.org/10.2196/11135

    Article  Google Scholar 

  53. Hunt DL, Haynes RB, Hanna SE, Smith K (1998) Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 280(15):1339–1346. https://doi.org/10.1001/jama.280.15.1339

    Article  Google Scholar 

  54. Itani S, Lecron F, Fortemps P (2019) Specifics of medical data mining for diagnosis aid: a survey. Expert Syst Appl 118:300–314. https://doi.org/10.1016/j.eswa.2018.09.056

    Article  Google Scholar 

  55. Jaroszewski AC, Morris RR, Nock MK (2019) Randomized controlled trial of an online machine learning-driven risk assessment and intervention platform for increasing the use of crisis services. J Consult Clin Psychol 87(4):370. https://doi.org/10.1037/ccp0000389

    Article  Google Scholar 

  56. Jeanmougin M, Dehais C, Meinard Y (2017) Mismatch between habitat science and habitat directive: lessons from the french (counter) example. Conserv Lett 10(5):634–644. https://doi.org/10.1111/conl.12330

    Article  Google Scholar 

  57. Johnston ME, Langton KB, Haynes RB, Mathieu A (1994) Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med 120(2):135–142. https://doi.org/10.7326/0003-4819-120-2-199401150-00007

    Article  Google Scholar 

  58. Joo S, Yang YS, Moon WK, Kim HC (2004) Computer-aided diagnosis of solid breast nodules: use of an artificial neural network based on multiple sonographic features. IEEE Trans Med Imaging 23(10):1292–1300. https://doi.org/10.1109/TMI.2004.834617

    Article  Google Scholar 

  59. Kataria S, Ravindran V (2018) Digital health: a new dimension in rheumatology patient care. Rheumatol Int 38(11):1949–1957. https://doi.org/10.1007/s00296-018-4037-x

    Article  Google Scholar 

  60. Kaushal R, Shojania KG, Bates DW (2003) Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 163(12):1409–1416. https://doi.org/10.1001/archinte.163.12.1409

    Article  Google Scholar 

  61. Keen J (1994) Information management in health services. Open University Press, Berkshire. https://doi.org/10.1016/j.jacr.2017.08.033

    Google Scholar 

  62. Kirby AM, Kruger B, Jain R, Daniel P, Granger BB (2018) Using clinical decision support to improve referral rates in severe symptomatic aortic stenosis: a quality improvement initiative. CIN: Comput Inform Nurs 36(11):525–529. https://doi.org/10.1097/CIN.0000000000000471

    Article  Google Scholar 

  63. Kostopoulou O, Porat T, Corrigan D, Mahmoud S, Delaney BC (2017) Diagnostic accuracy of GPS when using an early-intervention decision support system: a high-fidelity simulation. Br J Gen Pract 67(656):e201–e208. https://doi.org/10.3399/bjgp16X688417

    Article  Google Scholar 

  64. Kulikowski CA (1988) Artificial intelligence in medical consultation systems: A review. IEEE Eng Med Biol Mag 7(2):34–39. https://doi.org/10.1109/51.1972

    Article  Google Scholar 

  65. Kulikowski CA (2019) Beginnings of artificial intelligence in medicine (aim): computational artifice assisting scientific inquiry and clinical art-with reflections on present aim challenges. Yearb Med Inform 28(01):249–256. https://doi.org/10.1055/s-0039-1677895

    Article  Google Scholar 

  66. Kuperman GJ, Gibson RF (2003) Computer physician order entry: benefits, costs, and issues. Ann Intern Med 139(1):31. https://doi.org/10.7326/0003-4819-139-1-200307010-00010

    Article  Google Scholar 

  67. Leape LL (2000) Institute of medicine medical error figures are not exaggerated. JAMA 284(1):95–97. https://doi.org/10.1001/jama.284.1.95

    Article  Google Scholar 

  68. Leape LL, Berwick DM (2005) Five years after to err is human: what have we learned? JAMA 293(19):2384–2390. https://doi.org/10.1001/jama.293.19.2384

    Article  Google Scholar 

  69. López MM, López MM, de la Torre Díez I, Jimeno JCP, López-Coronado M (2017) Mhealth app for IOS to help in diagnostic decision in ophthalmology to primary care physicians. J Med Syst 41(5):81. https://doi.org/10.1007/s10916-017-0731-6

    Article  Google Scholar 

  70. Masud R, Al-Rei M, Lokker C (2019) Computer-aided detection for breast cancer screening in clinical settings: scoping review. JMIR Med Inform 7(3):e12660. https://doi.org/10.2196/12660

    Article  Google Scholar 

  71. Meinard Y, Cailloux O (2020) On justifying the norms underlying decision support. Eur J Oper Res. https://doi.org/10.1016/j.ejor.2020.02.022

    Article  Google Scholar 

  72. Meinard Y, Thébaud G (2019) L’identification syntaxonomique dans les démarches de gestion et/ou de restauration d’espaces naturels en france: pour ou contre? Naturae 6:165–173

    Google Scholar 

  73. Meinard Y, Tsoukiàs A (2019) On the rationality of decision aiding processes. Eur J Oper Res 273(3):1074–1084. https://doi.org/10.1016/j.ejor.2018.09.009

    Article  Google Scholar 

  74. Miller R, Masarie F Jr (1990) The demise of the “greek oracle” model for medical diagnostic systems. Methods Inf Med 29(01):1–2

    Article  Google Scholar 

  75. Miller RA (1994) Medical diagnostic decision support systems-past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc 1(1):8–27. https://doi.org/10.1136/jamia.1994.95236141

    Article  Google Scholar 

  76. Miller RA (2010) A history of the internist-1 and quick medical reference (QMR) computer-assisted diagnosis projects, with lessons learned. Yearb Med Inform 19(01):121–136. https://doi.org/10.1055/s-0038-1638702

    Article  Google Scholar 

  77. Miller RA (2016) Diagnostic decision support systems. Clinical decision support systems. Springer, New York, pp 181–208. https://doi.org/10.1007/978-3-319-31913-1_11

    Google Scholar 

  78. Miller RA, McNeil MA, Challinor SM, Masarie FE Jr, Myers JD (1986) The internist-1/quick medical reference project-status report. West J Med 145(6):816

    Google Scholar 

  79. Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform 19(6):1236–1246. https://doi.org/10.1093/bib/bbx044

    Article  Google Scholar 

  80. Mitchell I, Schuster A, Smith K, Pronovost P, Wu A (2016) Patient safety incident reporting: a qualitative study of thoughts and perceptions of experts 15 years after ‘to err is human’. BMJ Qual Saf 25(2):92–99. https://doi.org/10.1136/bmjqs-2015-004405

    Article  Google Scholar 

  81. Mokdad AH, Marks JS, Stroup DF, Gerberding JL (2004) Actual causes of death in the united states, 2000. JAMA 291(10):1238–1245. https://doi.org/10.1001/jama.291.10.1238

    Article  Google Scholar 

  82. Morelli RA, Bronzino JD, Goethe JW (1987) Expert systems in psychiatry. J Med Syst 11(2–3):157–168. https://doi.org/10.1007/BF00992350

    Article  Google Scholar 

  83. Murphy GC, Friedman CP, Elstein AS, Wolf FM, Miller T, Miller J (1996) The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. In: Proceedings of the AMIA annual fall symposium, American Medical Informatics Association, p 219

  84. Musen MA, Middleton B, Greenes RA (2014) Clinical decision-support systems. Biomed Inform. Springer, New York, pp 643–674. https://doi.org/10.1007/978-1-4471-4474-8_22

    Google Scholar 

  85. Nadarzynski T, Bayley J, Llewellyn C, Kidsley S, Graham CA (2020) Acceptability of artificial intelligence (ai)-enabled chatbots, video consultations and live webchats as online platforms for sexual health advice. BMJ Sex Reprod Health. https://doi.org/10.1136/bmjsrh-2018-200271

    Article  Google Scholar 

  86. Onega T, Bowles EJA, Miglioretti DL, Carney PA, Geller BM, Yankaskas BC, Kerlikowske K, Sickles EA, Elmore JG (2010) Radiologists’ perceptions of computer aided detection versus double reading for mammography interpretation. Acad Radiol 17(10):1217–1226. https://doi.org/10.1016/j.acra.2010.05.007

    Article  Google Scholar 

  87. Osheroff JA, Teich JM, Levick D, Saldana FT Luis ans Velasco, Sittig DF, Rogeers KM, Jenders RA (2012) Improving outcomes with clinical decision support: an implementer’s guide. HIMSS

  88. Overhage JM, Tierney WM, Zhou XH, McDonald CJ (1997) A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc 4(5):364–375. https://doi.org/10.1136/jamia.1997.0040364

    Article  Google Scholar 

  89. Ozaydin B, Hardin JM, Chhieng DC (2016) Data mining and clinical decision support systems. Clinical decision support systems. Springer, New York, pp 45–68. https://doi.org/10.1007/978-3-319-31913-1_3

    Google Scholar 

  90. Pare G, Elam JJ (1998) Introducing information technology in the clinical setting: lessons learned in a trauma center. Int J Technol Assess Health Care 14(2):331–343. https://doi.org/10.1017/S0266462300012290

    Article  Google Scholar 

  91. Pasquali P, Sonthalia S, Moreno-Ramirez D, Sharma P, Agrawal M, Gupta S, Kumar D, Arora D et al (2020) Teledermatology and its current perspective. Indian Dermatol Online J 11(1):12. https://doi.org/10.4103/idoj.IDOJ_241_19

    Article  Google Scholar 

  92. Patel VL, Kushniruk AW, Yang S, Yale JF (2000) Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning. J Am Med Inform Assoc 7(6):569–585. https://doi.org/10.1136/jamia.2000.0070569

    Article  Google Scholar 

  93. Pearce C, McLeod A, Rinehart N, Patrick J, Fragkoudi A, Ferrigi J, Deveny E, Whyte R, Shearer M (2019) Polar diversion: using general practice data to calculate risk of emergency department presentation at the time of consultation. Appl Clin Inform 10(01):151–157. https://doi.org/10.1055/s-0039-1678608

    Article  Google Scholar 

  94. Peiffer-Smadja N, Rawson TM, Ahmad R, Buchard A, Pantelis G, Lescure FX, Birgand G, Holmes AH (2019) Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect. https://doi.org/10.1016/j.cmi.2019.09.009

    Article  Google Scholar 

  95. Pivovarov R, Elhadad N (2015) Automated methods for the summarization of electronic health records. J Am Med Inform Assoc 22(5):938–947. https://doi.org/10.1093/jamia/ocv032

    Article  Google Scholar 

  96. Pluchinotta I, Pagano A, Giordano R, Tsoukiàs A (2018) A system dynamics model for supporting decision-makers in irrigation water management. J Environ Manage 223:815–824. https://doi.org/10.1016/j.jenvman.2018.06.083

    Article  Google Scholar 

  97. Pluchinotta I, Kazakçi AO, Giordano R, Tsoukiàs A (2019) Design theory for generating alternatives in public decision making processes. Group Decis Negot 28(2):341–375. https://doi.org/10.1007/s10726-018-09610-5

    Article  Google Scholar 

  98. Poels PJ, Schermer TR, Schellekens DP, Akkermans RP, de Vries Robbé PF, Kaplan A, Bottema BJ, Van Weel C (2008) Impact of a spirometry expert system on general practitioners’ decision making. Eur Respir J 31(1):84–92. https://doi.org/10.1183/09031936.00012007

    Article  Google Scholar 

  99. Porat T, Delaney B, Kostopoulou O (2017) The impact of a diagnostic decision support system on the consultation: perceptions of gps and patients. BMC Med Inform Decis Mak 17(1):79. https://doi.org/10.1186/s12911-017-0477-6

    Article  Google Scholar 

  100. Povyakalo AA, Alberdi E, Strigini L, Ayton P (2013) How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography. Med Decis Mak 33(1):98–107. https://doi.org/10.1177/0272989X12465490

    Article  Google Scholar 

  101. Reider JM (2016) Impact of national policies on the use of clinical decision support. Clinical decision support systems. Springer, New York, pp 111–130. https://doi.org/10.1007/978-3-319-31913-1_7

    Google Scholar 

  102. Richard A, Mayag B, Meinard Y, Talbot F, Tsoukiàs A (2018) How AI could help physicians during their medical consultations: an analysis of physicians’ decision process to develop efficient decision support systems for medical consultations. In: PFIA 2018, Nancy, France

  103. Rudin C, Radin J (2019) Why are we using black box models in ai when we don’t need to? a lesson from an explainable ai competition. Harvard Data Sci Rev. https://doi.org/10.1162/99608f92.5a8a3a3d

    Article  Google Scholar 

  104. Sandvig C, Hamilton K, Karahalios K, Langbort C (2016) When the algorithm itself is a racist: diagnosing ethical harm in the basic components of software. Int J Commun 10:19

    Google Scholar 

  105. Shortliffe E (2012) Computer-based medical consultations: MYCIN, vol 2. Elsevier, Amsterdam

    Google Scholar 

  106. Shortliffe EH, Cimino JJ (2014) Biomedical informatics, 4th edn. Springer, New York

    Google Scholar 

  107. Simon G, DiNardo CD, Takahashi K, Cascone T, Powers C, Stevens R, Allen J, Antonoff MB, Gomez D, Keane P et al (2019) Applying artificial intelligence to address the knowledge gaps in cancer care. Oncologist 24(6):772–782. https://doi.org/10.1634/theoncologist.2018-0257

    Article  Google Scholar 

  108. Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL (2006) A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak 6(1):6. https://doi.org/10.1186/1472-6947-6-6

    Article  Google Scholar 

  109. Slain T, Rickard-Aasen S, Pringle JL, Hegde GG, Shang J, Johnjulio W, Venkat A (2014) Incorporating screening, brief intervention, and referral to treatment into emergency nursing workflow using an existing computerized physician order entry/clinical decision support system. J Emerg Nurs 40(6):568–574. https://doi.org/10.1016/j.jen.2013.10.007

    Article  Google Scholar 

  110. Taylor B, Dinh M, Kwok R, Dinh D, Chu M, Tang E (2008) Electronic interface for emergency department management of asthma: a randomized control trial of clinician performance. Emerg Med Austral 20(1):38–44. https://doi.org/10.1111/j.1742-6723.2007.01040.x

    Article  Google Scholar 

  111. Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD (2003) Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med 18(12):967–976. https://doi.org/10.1111/j.1525-1497.2003.30635.x

    Article  Google Scholar 

  112. Titano JJ, Badgeley M, Schefflein J, Pain M, Su A, Cai M, Swinburne N, Zech J, Kim J, Bederson J et al (2018) Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat Med 24(9):1337–1341. https://doi.org/10.1038/s41591-018-0147-y

    Article  Google Scholar 

  113. Tsai TL, Fridsma DB, Gatti G (2003) Computer decision support as a source of interpretation error: the case of electrocardiograms. J Am Med Inform Assoc 10(5):478–483. https://doi.org/10.1197/jamia.M1279

    Article  Google Scholar 

  114. Tsoukiàs A, Montibeller G, Lucertini G, Belton V (2013) Policy analytics: an agenda for research and practice. EURO J Decis Process 1(1–2):115–134. https://doi.org/10.1007/s40070-013-0008-3

    Article  Google Scholar 

  115. Van Der Sijs H, Aarts J, Vulto A, Berg M (2006) Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 13(2):138–147. https://doi.org/10.1197/jamia.M1809

    Article  Google Scholar 

  116. Verdoorn S, Kwint HF, Hoogland P, Gussekloo J, Bouvy ML (2018) Drug-related problems identified during medication review before and after the introduction of a clinical decision support system. J Clin Pharm Ther 43(2):224–231. https://doi.org/10.1111/jcpt.12637

    Article  Google Scholar 

  117. Voigt P, Von dem Bussche A (2017) The EU general data protection regulation (GDPR). SpringerSpringer, New York. https://doi.org/10.1007/978-3-319-57959-7

    Google Scholar 

  118. Wachter RM (2004) The end of the beginning: patient safety five years after ‘to err is human’ amid signs of progress, there is still a long way to go. Health Aff 23(Suppl1):W4–534. https://doi.org/10.1377/hlthaff.W4.534

    Article  Google Scholar 

  119. Warner HR, Haug P, Bouhaddou O, Lincoln M, Warner Jr H, Sorenson D, Williamson JW, Fan C (1988) Iliad as an expert consultant to teach differential diagnosis. In: Proceedings of the annual symposium on computer application in medical care, American Medical Informatics Association, p 371

  120. Warner H Jr (1989) Iliad: moving medical decision-making into new frontiers. Methods Inf Med 28(04):370–372. https://doi.org/10.1055/s-0038-1636792

    Article  Google Scholar 

  121. Watrous RL, Thompson WR, Ackerman SJ (2008) The impact of computer-assisted auscultation on physician referrals of asymptomatic patients with heart murmurs. Clin Cardiol 31(2):79–83. https://doi.org/10.1002/clc.20185

    Article  Google Scholar 

  122. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS (2003) Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 163(21):2625–2631. https://doi.org/10.1001/archinte.163.21.2625

    Article  Google Scholar 

  123. West CP, Dyrbye LN, Shanafelt TD (2018) Physician burnout: contributors, consequences and solutions. J Intern Med 283(6):516–529. https://doi.org/10.1111/joim.12752

    Article  Google Scholar 

  124. Woolf SH (1993) Practice guidelines: a new reality in medicine: Iii. Impact on patient care. Arch Intern Med 153(23):2646–2655. https://doi.org/10.1001/archinte.1993.00410230060008

    Article  Google Scholar 

  125. World Health Organization et al (1992) The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. World Health Organization, Geneva

    Google Scholar 

  126. Yanase J, Triantaphyllou E (2019) A systematic survey of computer-aided diagnosis in medicine: past and present developments. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2019.112821

    Article  Google Scholar 

  127. Yang MS, Choi SI, Song WJ, Kim SH, Cho SH, Min KU, Kim JH, Chang YS (2018) Electronic consultation support system for radiocontrast media hypersensitivity changes clinicians’ behavior. Allergy Asthma Immunol Res 10(2):165–171. https://doi.org/10.4168/aair.2018.10.2.165

    Article  Google Scholar 

  128. Zhang K, Liu X, Liu F, He L, Zhang L, Yang Y, Li W, Wang S, Liu L, Liu Z et al (2018) An interpretable and expandable deep learning diagnostic system for multiple ocular diseases: qualitative study. J Med Internet Res 20(11):e11144. https://doi.org/10.2196/11144

    Article  Google Scholar 

  129. Zier J, Spaulding A, Finch M, Verschaetse T, Tarrago R (2017) Improved time to notification of impending brain death and increased organ donation using an electronic clinical decision support system. Am J Transplant 17(8):2186–2191. https://doi.org/10.1111/ajt.14312

    Article  Google Scholar 

  130. Zou J, Schiebinger L (2018) Ai can be sexist and racist-it’s time to make it fair. Nature 559:324–326. https://doi.org/10.1038/d41586-018-05707-8

    Article  Google Scholar 

Download references

Acknowledgements

This work was made in collaboration with employees of the hospitals of Lyon. Thanks to all of them. Special thanks to Pr. Moulin and Dr. Riou for their suggestions and instructive discussions. Special thanks also to J. Rouchier, O. Cailloux, and P. Grill for their advices and comments on earlier versions of this manuscript, and to P. Castets for his support in implementing this project. We also thank two anonymous reviewers of the journal for their powerful and exacting comments and criticisms.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Yves Meinard.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Richard, A., Mayag, B., Talbot, F. et al. What does it mean to provide decision support to a responsible and competent expert?. EURO J Decis Process 8, 205–236 (2020). https://doi.org/10.1007/s40070-020-00116-7

Download citation

Keywords

  • Decision analysis
  • Decision support systems
  • Diagnostic decision support systems

Mathematics Subject Classification

  • 68U35