Abstract
Decision support consists in helping a decision-maker to improve his/her decisions. However, clients requesting decision support are often themselves experts and are often taken by third parties and/or the general public to be responsible for the decisions they make. This predicament raises complex challenges for decision analysts, who have to avoid infringing upon the expertise and responsibility of the decision-maker. The case of diagnosis decision support in healthcare contexts is particularly illustrative. To support clinicians in their work and minimize the risk of medical error, various decision support systems have been developed, as part of information systems that are now ubiquitous in healthcare contexts. To develop, in collaboration with the hospitals of Lyon, a diagnostic decision support system for day-to-day customary consultations, we propose in this paper a critical analysis of current approaches to diagnostic decision support, which mainly consist in providing them with guidelines or even full-fledged diagnosis recommendations. We highlight that the use of such decision support systems by physicians raises responsibility issues, but also that it is at odds with the needs and constraints of customary consultations. We argue that the historical choice to favor guidelines or recommendations to physicians implies a very specific vision of what it means to support physicians, and we argue that the flaws of this vision partially explain why current diagnostic decision support systems are not accepted by physicians in their application to customary situations. Based on this analysis, we propose that decision support to physicians for customary cases should be deployed in an “adjustive” approach, which consists in providing physicians with the data on patients they need, when they need them, during consultations. The rationale articulated in this article has a more general bearing than clinical decision support and bears lessons for decision support activities in other contexts where decision-makers are competent and responsible experts.
Similar content being viewed by others
Notes
Some exceptions exist to the two predominant subsets of DDSSs (Guideline-based and ML-based DDSSs). Gräßer et al. (2017) proposed a DDSS dedicated to providing therapy recommendations, based not on expert guidelines or machine learning algorithms, but on similarity measures between the current case and previous ones, computed for each new cases, without any learning process involved. Whereas this system is akin to ML-based DDSSs, it does not use ML algorithms. Similarly, Giordanengo et al. (2019) proposed a DDSS dedicated to presenting self-collected data on patients and reminders of actions to do to physicians during the consultations of patients with diabetes. In this work, Giordanengo et al. (2019) did not use the guidelines of any health authority but included physicians in the development process of the DDSS to establish rules to apply in specific situations. In addition, the recommendations established by consensus among the physicians involved are not intended for other physicians, but to developers adding needed features into the DDSS. Lastly, the ML-based DDSS proposed by Simon et al. (2019) does not use ML algorithms to make recommendations but to detect complex concepts in medical documents, facilitating access to information on patients or to reference documents. With this DDSS, Simon et al. (2019) showed that it is possible to use ML algorithms in other ways than by producing recommendations, while still providing support to physicians in practice.
References
Anand V, Biondich PG, Liu GC, Rosenman MB, Downs SM et al (2004) Child health improvement through computer automation: the chica system. Medinfo. https://doi.org/10.3233/978-1-60750-949-3-187
Barnett GO, Cimino JJ, Hupp JA, Hoffer EP (1987) Dxplain: an evolving diagnostic decision-support system. JAMA 258(1):67–74. https://doi.org/10.1001/jama.1987.03400010071030
Bernasconi A, Crabbé F, Adedeji AM, Bello A, Schmitz T, Landi M, Rossi R (2019) Results from one-year use of an electronic clinical decision support system in a post-conflict context: An implementation research. PLoS One. https://doi.org/10.1371/journal.pone.0225634
Berner ES (2016) Clinical decision support systems, 3rd edn. Springer, New York
Berner ES, Graber ML (2008) Overconfidence as a cause of diagnostic error in medicine. Am J Med 121(5):S2–S23. https://doi.org/10.1016/j.amjmed.2008.01.001
Bertillot H (2016) Comment l’évaluation de la qualité transforme l’hôpital. les deux visages de la rationalisation par les indicateurs. Cahiers internationaux de sociologie de la gestion 15:11–48
Bessat C, Zonon NA, D’Acremont V (2019) Large-scale implementation of electronic integrated management of childhood illness (eimci) at the primary care level in burkina faso: a qualitative study on health worker perception of its medical content, usability and impact on antibiotic prescription and resistance. BMC Public Health 19(1):449. https://doi.org/10.1186/s12889-019-6692-6
Beynon-Davies P, Lloyd-Williams M (1998) Health information systems, ‘safety’ and organizational learning. Health Inform J 4(3–4):128–137. https://doi.org/10.1177/146045829800400303
Bowman S (2013) Impact of electronic health record systems on information integrity: quality and safety implications. Perspect Health Inf Manag 10(Fall):1c
Cabana MD, Rand CS, Powe NR, Wu AW, Wilson MH, Abboud PAC, Rubin HR (1999) Why don’t physicians follow clinical practice guidelines?: A framework for improvement. JAMA 282(15):1458–1465. https://doi.org/10.1001/jama.282.15.1458
Cabitza F, Rasoini R, Gensini GF (2017) Unintended consequences of machine learning in medicine. JAMA 318(6):517–518. https://doi.org/10.1001/jama.2017.7797
Carroll AE, Biondich P, Anand V, Dugan TM, Downs SM (2013) A randomized controlled trial of screening for maternal depression with a clinical decision support system. J Am Med Inform Assoc 20(2):311–316. https://doi.org/10.1136/amiajnl-2011-000682
Chadwick D, Hall C, Rae C, Rayment M, Branch M, Littlewood J, Sullivan A (2017) A feasibility study for a clinical decision support system prompting HIV testing. HIV Med 18(6):435–439. https://doi.org/10.1111/hiv.12472
Challen R, Denny J, Pitt M, Gompels L, Edwards T, Tsaneva-Atanasova K (2019) Artificial intelligence, bias and clinical safety. BMJ Qual Saf. https://doi.org/10.1136/bmjqs-2018-008370
Chang PL, Yc Li, Huang ST, Wang TM, Hsien ML (1996) Effects of a medical expert system on differential diagnosis of renal masses: A prospective study. Comput Med Imaging Graph 20(1):43–48. https://doi.org/10.1016/0895-6111(96)00029-8
Chaudhry B, Wang J, Wu S, Maglione M, Mojica W, Roth E, Morton SC, Shekelle PG (2006) Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 144(10):742–752. https://doi.org/10.7326/0003-4819-144-10-200605160-00125
Choulak M, Marage D, Gisbert M, Paris M, Meinard Y (2019) A meta-decision-analysis approach to structure operational and legitimate environmental policies-with an application to wetland prioritization. Sci Total Environ 655:384–394. https://doi.org/10.1016/j.scitotenv.2018.11.202
Clancy CM (2009) Ten years after to err is human. Am J Med Qual 24(6):525–528. https://doi.org/10.1177/1062860609349728
Dalaba MA, Akweongo P, Williams J, Saronga HP, Tonchev P, Sauerborn R, Mensah N, Blank A, Kaltschmidt J, Loukanova S (2014) Costs associated with implementation of computer-assisted clinical decision support system for antenatal and delivery care: case study of kassena-nankana district of northern ghana. PLoS One. https://doi.org/10.1371/journal.pone.0106416
De Dombal F (1987) Ethical considerations concerning computers in medicine in the 1980s. J Med Ethics 13(4):179–184. https://doi.org/10.1136/jme.13.4.179
De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S, Askham H, Glorot X, O’Donoghue B, Visentin D et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24(9):1342–1350. https://doi.org/10.1038/s41591-018-0107-6
De Marchi G, Lucertini G, Tsoukiàs A (2016) From evidence-based policy making to policy analytics. Ann Oper Res 236(1):15–38. https://doi.org/10.1007/s10479-014-1578-6
Deig CR, Kanwar A, Thompson RF (2019) Artificial intelligence in radiation oncology. Hematol Oncol Clin 33(6):1095–1104. https://doi.org/10.1016/j.hoc.2019.08.003
Donaldson MS, Corrigan JM, Kohn LT et al (2000) To err is human: building a safer health system, vol 6. National Academies Press, Washington, D.C
Doran D, Schulz S, Besold TR (2017) What does explainable ai really mean? a new conceptualization of perspectives. arXiv preprint arXiv:171000794
Dua S, Acharya UR, Dua P (2014) Machine learning in healthcare informatics, vol 56. Springer, New York
Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, Purves I (2002) Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ 325(7370):941. https://doi.org/10.1136/bmj.325.7370.941
Elsner P, Bauer A, Diepgen TL, Drexler H, Fartasch M, John SM, Schliemann S, Wehrmann W, Tittelbach J (2018) Position paper: Telemedicine in occupational dermatology-current status and perspectives. JDDG: Journal der Deutschen Dermatologischen Gesellschaft 16(8):969–974. https://doi.org/10.1111/ddg.13605
Elstein AS, Friedman CP, Wolf FM, Murphy G, Miller J, Fine P, Heckerling P, Miller T, Sisson J, Barlas S et al (1996) Effects of a decision support system on the diagnostic accuracy of users: a preliminary report. J Am Med Inform Assoc 3(6):422–428. https://doi.org/10.1136/jamia.1996.97084515
Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118. https://doi.org/10.1038/nature21056
Etter DJ, McCord A, Ouyang F, Gilbert AL, Williams RL, Hall JA, Tu W, Downs SM, Aalsma MC (2018) Suicide screening in primary care: use of an electronic screener to assess suicidality and improve provider follow-up for adolescents. J Adolesc Health 62(2):191–197. https://doi.org/10.1016/j.jadohealth.2017.08.026
Ferretti V, Pluchinotta I, Tsoukiàs A (2019) Studying the generation of alternatives in public policy making processes. Eur J Oper Res 273(1):353–363. https://doi.org/10.1016/j.ejor.2018.07.054
Field MJ, Lohr KN et al (1990) Clinical practice guidelines: directions for a new program. National Academies Press, Washington, D.C
Fieschi M (1986) Expert systems for medical consultation. Health Policy 6(2):159–173. https://doi.org/10.1016/0168-8510(86)90005-9
Garcia M (2016) Racist in the machine: The disturbing implications of algorithmic bias. World Policy J 33(4):111–117
Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux P, Beyene J, Sam J, Haynes RB (2005) Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 293(10):1223–1238. https://doi.org/10.1001/jama.293.10.1223
Giordanengo A, Årsand E, Woldaregay AZ, Bradway M, Grottland A, Hartvigsen G, Granja C, Torsvik T, Hansen AH (2019) Design and prestudy assessment of a dashboard for presenting self-collected health data of patients with diabetes to clinicians: Iterative approach and qualitative case study. JMIR Diabetes 4(3):e14002. https://doi.org/10.2196/14002
Gonzalvo CM, de Wit HA, van Oijen BP, Hurkens KP, Janknegt R, Schols JM, Mulder WJ, Verhey FR, Winkens B, van der Kuy PHM (2017) Supporting clinical rules engine in the adjustment of medication (scream): protocol of a multicentre, prospective, randomised study. BMC Geriatr 17(1):35. https://doi.org/10.1186/s12877-017-0426-3
Goodman KW (2016) Ethical and legal issues in decision support. Clinical decision support systems. Springer, New York, pp 131–146. https://doi.org/10.1007/978-3-319-31913-1_8
Gordon ES, Babu D, Laney DA (2018) The future is now: Technology’s impact on the practice of genetic counseling. Am J Med Genet Part C Semin Med Genet 178:15–23. https://doi.org/10.1002/ajmg.c.31599 (Wiley Online Library)
Gräßer F, Beckert S, Küster D, Schmitt J, Abraham S, Malberg H, Zaunseder S (2017) Therapy decision support based on recommender system methods. J Healthc Eng. https://doi.org/10.1155/2017/8659460
Guah M (1998) Evaluation and analysis of multimedia information system design and implementation at the Coloscopy unit of St. James university hospital, Leeds, UK. PhD thesis, MSc Dissertation, School of Management, UMIST, Manchester
Gunning D (2017) Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web 2
Hall LH, Johnson J, Watt I, Tsipa A, O’Connor DB (2016) Healthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One 11(7):e0159015. https://doi.org/10.1371/journal.pone.0159015
Heckerling P, Elstein A, Terzian C, Kushner M (1991) The effect of incomplete knowledge on the diagnoses of a computer consultant system. Med Inform 16(4):363–370. https://doi.org/10.3109/14639239109067658
Heeks R (2006) Health information systems: failure, success and improvisation. Int J Med Inform 75(2):125–137. https://doi.org/10.1016/j.ijmedinf.2005.07.024
Heeks R, Mundy D, Salazar A (1999) Why health care information systems succeed or fail. Inform Syst Public Sector Manag. https://doi.org/10.2139/ssrn.3540062
Hoffer EP, Feldman MJ, Kim RJ, Famiglietti KT, Barnett GO (2005) Dxplain: patterns of use of a mature expert system. In: AMIA annual symposium proceedings, American Medical Informatics Association, vol 2005, p 321
Hollon TC, Lewis S, Pandian B, Niknafs YS, Garrard MR, Garton H, Maher CO, McFadden K, Snuderl M, Lieberman AP et al (2018) Rapid intraoperative diagnosis of pediatric brain tumors using stimulated raman histology. Cancer Res 78(1):278–289. https://doi.org/10.1158/0008-5472.CAN-17-1974
Honaker SM, Dugan T, Daftary A, Davis S, Saha C, Baye F, Freeman E, Downs SM (2018) Unexplained practice variation in primary care providers’ concern for pediatric obstructive sleep apnea. Acad Pediatr 18(4):418–424. https://doi.org/10.1016/j.acap.2018.01.011
Hornik K, Stinchcombe M, White H et al (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366. https://doi.org/10.1016/0893-6080(89)90020-8
Horrocks M, Michail M, Aubeeluck A, Wright N, Morriss R (2018) An electronic clinical decision support system for the assessment and management of suicidality in primary care: protocol for a mixed-methods study. JMIR Res Protoc 7(12):e11135. https://doi.org/10.2196/11135
Hunt DL, Haynes RB, Hanna SE, Smith K (1998) Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 280(15):1339–1346. https://doi.org/10.1001/jama.280.15.1339
Itani S, Lecron F, Fortemps P (2019) Specifics of medical data mining for diagnosis aid: a survey. Expert Syst Appl 118:300–314. https://doi.org/10.1016/j.eswa.2018.09.056
Jaroszewski AC, Morris RR, Nock MK (2019) Randomized controlled trial of an online machine learning-driven risk assessment and intervention platform for increasing the use of crisis services. J Consult Clin Psychol 87(4):370. https://doi.org/10.1037/ccp0000389
Jeanmougin M, Dehais C, Meinard Y (2017) Mismatch between habitat science and habitat directive: lessons from the french (counter) example. Conserv Lett 10(5):634–644. https://doi.org/10.1111/conl.12330
Johnston ME, Langton KB, Haynes RB, Mathieu A (1994) Effects of computer-based clinical decision support systems on clinician performance and patient outcome: a critical appraisal of research. Ann Intern Med 120(2):135–142. https://doi.org/10.7326/0003-4819-120-2-199401150-00007
Joo S, Yang YS, Moon WK, Kim HC (2004) Computer-aided diagnosis of solid breast nodules: use of an artificial neural network based on multiple sonographic features. IEEE Trans Med Imaging 23(10):1292–1300. https://doi.org/10.1109/TMI.2004.834617
Kataria S, Ravindran V (2018) Digital health: a new dimension in rheumatology patient care. Rheumatol Int 38(11):1949–1957. https://doi.org/10.1007/s00296-018-4037-x
Kaushal R, Shojania KG, Bates DW (2003) Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 163(12):1409–1416. https://doi.org/10.1001/archinte.163.12.1409
Keen J (1994) Information management in health services. Open University Press, Berkshire. https://doi.org/10.1016/j.jacr.2017.08.033
Kirby AM, Kruger B, Jain R, Daniel P, Granger BB (2018) Using clinical decision support to improve referral rates in severe symptomatic aortic stenosis: a quality improvement initiative. CIN: Comput Inform Nurs 36(11):525–529. https://doi.org/10.1097/CIN.0000000000000471
Kostopoulou O, Porat T, Corrigan D, Mahmoud S, Delaney BC (2017) Diagnostic accuracy of GPS when using an early-intervention decision support system: a high-fidelity simulation. Br J Gen Pract 67(656):e201–e208. https://doi.org/10.3399/bjgp16X688417
Kulikowski CA (1988) Artificial intelligence in medical consultation systems: A review. IEEE Eng Med Biol Mag 7(2):34–39. https://doi.org/10.1109/51.1972
Kulikowski CA (2019) Beginnings of artificial intelligence in medicine (aim): computational artifice assisting scientific inquiry and clinical art-with reflections on present aim challenges. Yearb Med Inform 28(01):249–256. https://doi.org/10.1055/s-0039-1677895
Kuperman GJ, Gibson RF (2003) Computer physician order entry: benefits, costs, and issues. Ann Intern Med 139(1):31. https://doi.org/10.7326/0003-4819-139-1-200307010-00010
Leape LL (2000) Institute of medicine medical error figures are not exaggerated. JAMA 284(1):95–97. https://doi.org/10.1001/jama.284.1.95
Leape LL, Berwick DM (2005) Five years after to err is human: what have we learned? JAMA 293(19):2384–2390. https://doi.org/10.1001/jama.293.19.2384
López MM, López MM, de la Torre Díez I, Jimeno JCP, López-Coronado M (2017) Mhealth app for IOS to help in diagnostic decision in ophthalmology to primary care physicians. J Med Syst 41(5):81. https://doi.org/10.1007/s10916-017-0731-6
Masud R, Al-Rei M, Lokker C (2019) Computer-aided detection for breast cancer screening in clinical settings: scoping review. JMIR Med Inform 7(3):e12660. https://doi.org/10.2196/12660
Meinard Y, Cailloux O (2020) On justifying the norms underlying decision support. Eur J Oper Res. https://doi.org/10.1016/j.ejor.2020.02.022
Meinard Y, Thébaud G (2019) L’identification syntaxonomique dans les démarches de gestion et/ou de restauration d’espaces naturels en france: pour ou contre? Naturae 6:165–173
Meinard Y, Tsoukiàs A (2019) On the rationality of decision aiding processes. Eur J Oper Res 273(3):1074–1084. https://doi.org/10.1016/j.ejor.2018.09.009
Miller R, Masarie F Jr (1990) The demise of the “greek oracle” model for medical diagnostic systems. Methods Inf Med 29(01):1–2
Miller RA (1994) Medical diagnostic decision support systems-past, present, and future: a threaded bibliography and brief commentary. J Am Med Inform Assoc 1(1):8–27. https://doi.org/10.1136/jamia.1994.95236141
Miller RA (2010) A history of the internist-1 and quick medical reference (QMR) computer-assisted diagnosis projects, with lessons learned. Yearb Med Inform 19(01):121–136. https://doi.org/10.1055/s-0038-1638702
Miller RA (2016) Diagnostic decision support systems. Clinical decision support systems. Springer, New York, pp 181–208. https://doi.org/10.1007/978-3-319-31913-1_11
Miller RA, McNeil MA, Challinor SM, Masarie FE Jr, Myers JD (1986) The internist-1/quick medical reference project-status report. West J Med 145(6):816
Miotto R, Wang F, Wang S, Jiang X, Dudley JT (2017) Deep learning for healthcare: review, opportunities and challenges. Brief Bioinform 19(6):1236–1246. https://doi.org/10.1093/bib/bbx044
Mitchell I, Schuster A, Smith K, Pronovost P, Wu A (2016) Patient safety incident reporting: a qualitative study of thoughts and perceptions of experts 15 years after ‘to err is human’. BMJ Qual Saf 25(2):92–99. https://doi.org/10.1136/bmjqs-2015-004405
Mokdad AH, Marks JS, Stroup DF, Gerberding JL (2004) Actual causes of death in the united states, 2000. JAMA 291(10):1238–1245. https://doi.org/10.1001/jama.291.10.1238
Morelli RA, Bronzino JD, Goethe JW (1987) Expert systems in psychiatry. J Med Syst 11(2–3):157–168. https://doi.org/10.1007/BF00992350
Murphy GC, Friedman CP, Elstein AS, Wolf FM, Miller T, Miller J (1996) The influence of a decision support system on the differential diagnosis of medical practitioners at three levels of training. In: Proceedings of the AMIA annual fall symposium, American Medical Informatics Association, p 219
Musen MA, Middleton B, Greenes RA (2014) Clinical decision-support systems. Biomed Inform. Springer, New York, pp 643–674. https://doi.org/10.1007/978-1-4471-4474-8_22
Nadarzynski T, Bayley J, Llewellyn C, Kidsley S, Graham CA (2020) Acceptability of artificial intelligence (ai)-enabled chatbots, video consultations and live webchats as online platforms for sexual health advice. BMJ Sex Reprod Health. https://doi.org/10.1136/bmjsrh-2018-200271
Onega T, Bowles EJA, Miglioretti DL, Carney PA, Geller BM, Yankaskas BC, Kerlikowske K, Sickles EA, Elmore JG (2010) Radiologists’ perceptions of computer aided detection versus double reading for mammography interpretation. Acad Radiol 17(10):1217–1226. https://doi.org/10.1016/j.acra.2010.05.007
Osheroff JA, Teich JM, Levick D, Saldana FT Luis ans Velasco, Sittig DF, Rogeers KM, Jenders RA (2012) Improving outcomes with clinical decision support: an implementer’s guide. HIMSS
Overhage JM, Tierney WM, Zhou XH, McDonald CJ (1997) A randomized trial of “corollary orders” to prevent errors of omission. J Am Med Inform Assoc 4(5):364–375. https://doi.org/10.1136/jamia.1997.0040364
Ozaydin B, Hardin JM, Chhieng DC (2016) Data mining and clinical decision support systems. Clinical decision support systems. Springer, New York, pp 45–68. https://doi.org/10.1007/978-3-319-31913-1_3
Pare G, Elam JJ (1998) Introducing information technology in the clinical setting: lessons learned in a trauma center. Int J Technol Assess Health Care 14(2):331–343. https://doi.org/10.1017/S0266462300012290
Pasquali P, Sonthalia S, Moreno-Ramirez D, Sharma P, Agrawal M, Gupta S, Kumar D, Arora D et al (2020) Teledermatology and its current perspective. Indian Dermatol Online J 11(1):12. https://doi.org/10.4103/idoj.IDOJ_241_19
Patel VL, Kushniruk AW, Yang S, Yale JF (2000) Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning. J Am Med Inform Assoc 7(6):569–585. https://doi.org/10.1136/jamia.2000.0070569
Pearce C, McLeod A, Rinehart N, Patrick J, Fragkoudi A, Ferrigi J, Deveny E, Whyte R, Shearer M (2019) Polar diversion: using general practice data to calculate risk of emergency department presentation at the time of consultation. Appl Clin Inform 10(01):151–157. https://doi.org/10.1055/s-0039-1678608
Peiffer-Smadja N, Rawson TM, Ahmad R, Buchard A, Pantelis G, Lescure FX, Birgand G, Holmes AH (2019) Machine learning for clinical decision support in infectious diseases: a narrative review of current applications. Clin Microbiol Infect. https://doi.org/10.1016/j.cmi.2019.09.009
Pivovarov R, Elhadad N (2015) Automated methods for the summarization of electronic health records. J Am Med Inform Assoc 22(5):938–947. https://doi.org/10.1093/jamia/ocv032
Pluchinotta I, Pagano A, Giordano R, Tsoukiàs A (2018) A system dynamics model for supporting decision-makers in irrigation water management. J Environ Manage 223:815–824. https://doi.org/10.1016/j.jenvman.2018.06.083
Pluchinotta I, Kazakçi AO, Giordano R, Tsoukiàs A (2019) Design theory for generating alternatives in public decision making processes. Group Decis Negot 28(2):341–375. https://doi.org/10.1007/s10726-018-09610-5
Poels PJ, Schermer TR, Schellekens DP, Akkermans RP, de Vries Robbé PF, Kaplan A, Bottema BJ, Van Weel C (2008) Impact of a spirometry expert system on general practitioners’ decision making. Eur Respir J 31(1):84–92. https://doi.org/10.1183/09031936.00012007
Porat T, Delaney B, Kostopoulou O (2017) The impact of a diagnostic decision support system on the consultation: perceptions of gps and patients. BMC Med Inform Decis Mak 17(1):79. https://doi.org/10.1186/s12911-017-0477-6
Povyakalo AA, Alberdi E, Strigini L, Ayton P (2013) How to discriminate between computer-aided and computer-hindered decisions: a case study in mammography. Med Decis Mak 33(1):98–107. https://doi.org/10.1177/0272989X12465490
Reider JM (2016) Impact of national policies on the use of clinical decision support. Clinical decision support systems. Springer, New York, pp 111–130. https://doi.org/10.1007/978-3-319-31913-1_7
Richard A, Mayag B, Meinard Y, Talbot F, Tsoukiàs A (2018) How AI could help physicians during their medical consultations: an analysis of physicians’ decision process to develop efficient decision support systems for medical consultations. In: PFIA 2018, Nancy, France
Rudin C, Radin J (2019) Why are we using black box models in ai when we don’t need to? a lesson from an explainable ai competition. Harvard Data Sci Rev. https://doi.org/10.1162/99608f92.5a8a3a3d
Sandvig C, Hamilton K, Karahalios K, Langbort C (2016) When the algorithm itself is a racist: diagnosing ethical harm in the basic components of software. Int J Commun 10:19
Shortliffe E (2012) Computer-based medical consultations: MYCIN, vol 2. Elsevier, Amsterdam
Shortliffe EH, Cimino JJ (2014) Biomedical informatics, 4th edn. Springer, New York
Simon G, DiNardo CD, Takahashi K, Cascone T, Powers C, Stevens R, Allen J, Antonoff MB, Gomez D, Keane P et al (2019) Applying artificial intelligence to address the knowledge gaps in cancer care. Oncologist 24(6):772–782. https://doi.org/10.1634/theoncologist.2018-0257
Sittig DF, Krall MA, Dykstra RH, Russell A, Chin HL (2006) A survey of factors affecting clinician acceptance of clinical decision support. BMC Med Inform Decis Mak 6(1):6. https://doi.org/10.1186/1472-6947-6-6
Slain T, Rickard-Aasen S, Pringle JL, Hegde GG, Shang J, Johnjulio W, Venkat A (2014) Incorporating screening, brief intervention, and referral to treatment into emergency nursing workflow using an existing computerized physician order entry/clinical decision support system. J Emerg Nurs 40(6):568–574. https://doi.org/10.1016/j.jen.2013.10.007
Taylor B, Dinh M, Kwok R, Dinh D, Chu M, Tang E (2008) Electronic interface for emergency department management of asthma: a randomized control trial of clinician performance. Emerg Med Austral 20(1):38–44. https://doi.org/10.1111/j.1742-6723.2007.01040.x
Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD (2003) Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med 18(12):967–976. https://doi.org/10.1111/j.1525-1497.2003.30635.x
Titano JJ, Badgeley M, Schefflein J, Pain M, Su A, Cai M, Swinburne N, Zech J, Kim J, Bederson J et al (2018) Automated deep-neural-network surveillance of cranial images for acute neurologic events. Nat Med 24(9):1337–1341. https://doi.org/10.1038/s41591-018-0147-y
Tsai TL, Fridsma DB, Gatti G (2003) Computer decision support as a source of interpretation error: the case of electrocardiograms. J Am Med Inform Assoc 10(5):478–483. https://doi.org/10.1197/jamia.M1279
Tsoukiàs A, Montibeller G, Lucertini G, Belton V (2013) Policy analytics: an agenda for research and practice. EURO J Decis Process 1(1–2):115–134. https://doi.org/10.1007/s40070-013-0008-3
Van Der Sijs H, Aarts J, Vulto A, Berg M (2006) Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 13(2):138–147. https://doi.org/10.1197/jamia.M1809
Verdoorn S, Kwint HF, Hoogland P, Gussekloo J, Bouvy ML (2018) Drug-related problems identified during medication review before and after the introduction of a clinical decision support system. J Clin Pharm Ther 43(2):224–231. https://doi.org/10.1111/jcpt.12637
Voigt P, Von dem Bussche A (2017) The EU general data protection regulation (GDPR). SpringerSpringer, New York. https://doi.org/10.1007/978-3-319-57959-7
Wachter RM (2004) The end of the beginning: patient safety five years after ‘to err is human’ amid signs of progress, there is still a long way to go. Health Aff 23(Suppl1):W4–534. https://doi.org/10.1377/hlthaff.W4.534
Warner HR, Haug P, Bouhaddou O, Lincoln M, Warner Jr H, Sorenson D, Williamson JW, Fan C (1988) Iliad as an expert consultant to teach differential diagnosis. In: Proceedings of the annual symposium on computer application in medical care, American Medical Informatics Association, p 371
Warner H Jr (1989) Iliad: moving medical decision-making into new frontiers. Methods Inf Med 28(04):370–372. https://doi.org/10.1055/s-0038-1636792
Watrous RL, Thompson WR, Ackerman SJ (2008) The impact of computer-assisted auscultation on physician referrals of asymptomatic patients with heart murmurs. Clin Cardiol 31(2):79–83. https://doi.org/10.1002/clc.20185
Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS (2003) Physicians’ decisions to override computerized drug alerts in primary care. Arch Intern Med 163(21):2625–2631. https://doi.org/10.1001/archinte.163.21.2625
West CP, Dyrbye LN, Shanafelt TD (2018) Physician burnout: contributors, consequences and solutions. J Intern Med 283(6):516–529. https://doi.org/10.1111/joim.12752
Woolf SH (1993) Practice guidelines: a new reality in medicine: Iii. Impact on patient care. Arch Intern Med 153(23):2646–2655. https://doi.org/10.1001/archinte.1993.00410230060008
World Health Organization et al (1992) The ICD-10 classification of mental and behavioural disorders: clinical descriptions and diagnostic guidelines. World Health Organization, Geneva
Yanase J, Triantaphyllou E (2019) A systematic survey of computer-aided diagnosis in medicine: past and present developments. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2019.112821
Yang MS, Choi SI, Song WJ, Kim SH, Cho SH, Min KU, Kim JH, Chang YS (2018) Electronic consultation support system for radiocontrast media hypersensitivity changes clinicians’ behavior. Allergy Asthma Immunol Res 10(2):165–171. https://doi.org/10.4168/aair.2018.10.2.165
Zhang K, Liu X, Liu F, He L, Zhang L, Yang Y, Li W, Wang S, Liu L, Liu Z et al (2018) An interpretable and expandable deep learning diagnostic system for multiple ocular diseases: qualitative study. J Med Internet Res 20(11):e11144. https://doi.org/10.2196/11144
Zier J, Spaulding A, Finch M, Verschaetse T, Tarrago R (2017) Improved time to notification of impending brain death and increased organ donation using an electronic clinical decision support system. Am J Transplant 17(8):2186–2191. https://doi.org/10.1111/ajt.14312
Zou J, Schiebinger L (2018) Ai can be sexist and racist-it’s time to make it fair. Nature 559:324–326. https://doi.org/10.1038/d41586-018-05707-8
Acknowledgements
This work was made in collaboration with employees of the hospitals of Lyon. Thanks to all of them. Special thanks to Pr. Moulin and Dr. Riou for their suggestions and instructive discussions. Special thanks also to J. Rouchier, O. Cailloux, and P. Grill for their advices and comments on earlier versions of this manuscript, and to P. Castets for his support in implementing this project. We also thank two anonymous reviewers of the journal for their powerful and exacting comments and criticisms.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Richard, A., Mayag, B., Talbot, F. et al. What does it mean to provide decision support to a responsible and competent expert?. EURO J Decis Process 8, 205–236 (2020). https://doi.org/10.1007/s40070-020-00116-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40070-020-00116-7