Advances in Health Sciences Education

, Volume 14, Supplement 1, pp 89–106 | Cite as

Computer-assisted diagnostic decision support: history, challenges, and possible paths forward

  • Randolph A. MillerEmail author


This paper presents a brief history of computer-assisted diagnosis, including challenges and future directions. Some ideas presented in this article on computer-assisted diagnostic decision support systems (CDDSS) derive from prior work by the author and his colleagues (see list in Acknowledgments) on the INTERNIST-1 and QMR projects. References indicate the original sources of many of these ideas.


Computers Decision support systems Management/trends Diagnosis Computer-assisted Research User-computer interface 



Disclosure of (Non) Conflicts of Interest: Dr. Miller previously received royalties from the University of Pittsburgh in recognition of his work there in authoring the Quick Medical Reference (QMR)® program and knowledge base for diagnostic decision support in Internal Medicine; he donated most of those royalties to charity. The program is no longer being sold. Dr. Miller receives royalties from Vanderbilt University based on Vanderbilt’s commercialization of the WizOrder CPOE system, and the Star electronic health record system, both of which he helped to develop. The majority of income from licensing these systems goes directly to Vanderbilt School of Medicine, per se. Dr. Miller does not own stock in, or serve as an employee or officer of, any of those vendors’ companies. Work on INTERNIST-1 and QMR has been supported over the years by NIH grants from DRR and NLM; grants from Paul Mongerson and his CAMDAT Foundation; and, by internal funding from the University of Pittsburgh. In addition to the author, major contributors to the INTERNIST-1 and QMR projects include: Jack D. Myers, MD; Harry E. Pople, Jr., PhD; Griff Smith, PhD; Zachary Moraitis, MD; Craig Dean; Casey Quayle; Chuck Olson, PhD; Fred E. Masarie, Jr., MD; Nunzia B. Giuse, MD, MLS; Dario A. Giuse, Dr. Ing; Richard A. Bankowitz, MD; and numerous medical students and faculty members from the University of Pittsburgh and other institutions.


  1. Adams, I. D., Chan, M., Clifford, P. C., et al. (1986). Computer aided diagnosis of acute abdominal pain: A multicentre study. British Medical Journal (Clinical research ed.), 293, 800–804.CrossRefGoogle Scholar
  2. Aliferis, C. F., Cooper, G. F., Miller, R. A., Buchanan, B. G., Bankowitz, R., & Giuse, N. B. (1996). A temporal analysis of QMR. Journal of the American Medical Informatics Association, 3, 79–91.Google Scholar
  3. Bankowitz, R. A., McNeil, M. A., Challinor, S. M., Parker, R. C., Kapoor, W. N., & Miller, R. A. (1989a). A computer-assisted medical diagnostic consultation service: Implementation and prospective evaluation of a prototype. Annals of Internal Medicine, 110, 824–832.Google Scholar
  4. Bankowitz, R. B., McNeil, M. A., Challinor, S. M., & Miller, R. A. (1989b). Effect of a computer-assisted general medicine diagnostic consultation service on housestaff diagnostic strategy. Methods of Information in Medicine, 28, 352–356.Google Scholar
  5. Barnett, G. O., Cimino, J. J., Hupp, J. A., & Hoffer, E. P. (1987). DXplain. An evolving diagnostic decision-support system. JAMA, 258, 67–74.CrossRefGoogle Scholar
  6. Bates, D. W., et al. (2003). Ten Commandments for effective clinical decision support. Journal of the American Medical Informatics Association, 10(6), 523–530.CrossRefGoogle Scholar
  7. Berner, E. S., Webster, G. D., Shugerman, A. A., et al. (1994). Performance of four computer-based diagnostic systems. New England Journal of Medicine, 330(25), 1792–1796.CrossRefGoogle Scholar
  8. Berner, E. S., Maisiak, R. S., Cobbs, C. G., & Taunton, O. D. (1999). Effects of a decision support system on physicians’ diagnostic performance. Journal of the American Medical Informatics Association, 6(5), 420–427.Google Scholar
  9. Bleich, H. L. (1969). Computer evaluation of acid-base disorders. Journal of Clinical Investigation, 48, 1689–1696.CrossRefGoogle Scholar
  10. Blois, M. S., Tuttle, M. S., & Sherertz, D. D. (1981). RECONSIDER: A program for generating differential diagnoses. In: Hefferman, H. G. (Ed.). Proceedings of the Fifth Annual Symposium On Computer Applications in Health Care (pp. 263–268). Washington, DC: IEEE Computer Society Press.Google Scholar
  11. Collen, M. F. (1995). A history of medical informatics in the United States: 1950 to 1990. Bethesda (p. 489). Amer Med Informatics Assoc: MD. ISBN 0-9647743-0-5.Google Scholar
  12. Cooper, G. F. (1986). A diagnostic method that uses causal knowledge and linear programming in the application of Bayes’ formula. Computer Methods and Programs in Biomedicine, 22(2), 223–237.CrossRefGoogle Scholar
  13. de Dombal, F. T., Hortocks, J. C., Staniland, J. R., & Gill, P. W. (1971). Simulation of clinical diagnosis: A comparative study. British Medical Journal, 2, 575–577.CrossRefGoogle Scholar
  14. Durack, D. T. (1978). The weight of medical knowledge. New England Journal of Medicine, 298(14), 773–775.Google Scholar
  15. Feldman, M. J., & Bartlett, G. O. (1991). An approach to evaluating the accuracy of’ DXplain. Computer Methods and Programs in Biomedicine, 35, 261–266.CrossRefGoogle Scholar
  16. Friedman, C. P., & Wyatt, J. (1997). Evaluation methods in medical informatics (p. 311). New York: Springer. ISBN 0387942289, 9780387942285.Google Scholar
  17. Friedman, C. P., Elstein, A. S., Wolf, F. M., et al. (1999). Enhancement of clinicians’ diagnostic reasoning by computer-based consultation: A multisite study of 2 systems. JAMA, 282(19), 1851–1856.CrossRefGoogle Scholar
  18. Fryback, D. G. (1978). Bayes’ theorem and conditional nonindependence of data in medical diagnosis. Computers and Biomedical Research, 11, 423–434.CrossRefGoogle Scholar
  19. Giuse, N. B., Bankowitz, R. A., Giuse, D. A., Parker, R. C., & Miller, R. A. (1989a). Medical Knowledge Base Acquisition: The Role of Expert Review Process in Disease Profile Construction. Proceedings of Thirteenth Annual Symposium on Computer Applications in Medical Care (pp. 105–109). Washington, DC: IEEE Press.Google Scholar
  20. Giuse, N. B., Giuse, D. A, & Miller, R. A. (1989b). Medical knowledge base construction as a means of introducing students to medical informatics. Proceedings of the International Symposium on Medical Informatics and Education (pp. 228–232). Victoria, BC.Google Scholar
  21. Giuse, D. A., Giuse, N. B., & Miller, R. A. (1990). Towards computer assisted maintenance of medical knowledge bases. Artificial Intelligence in Medicine, 2, 21–33.CrossRefGoogle Scholar
  22. Giuse, N, B, Giuse, D. A, Bankowitz, R. A., & Miller, R. A. (1993a). Comparing contents of a knowledge base to traditional information sources. Proceedings of the Seventeenth Annual Symposium on Computer Applications in Medical Care. Washington DC: McGraw-Hill, Nov. 1993.Google Scholar
  23. Giuse, D. A., Giuse, N. B., & Miller, R. A. (1993b). Consistency enforcement in medical knowledge base construction. Artificial Intelligence in Medicine, 5, 245–252.CrossRefGoogle Scholar
  24. Giuse, N. B., Giuse, D. A., Miller, R. A., Bankowitz, R. A., Janosky, J. E., Davidoff, F., et al. (1993c). Evaluating consensus among physicians in medical knowledge base construction. Methods of Information in Medicine, 32, 137–145.Google Scholar
  25. Giuse, D. A., Giuse, N. B., & Miller, R. A. (1995). Evaluation of long-term maintenance of a large medical knowledge base. Journal of the American Medical Informatics Association, 2, 297–306.Google Scholar
  26. Gorman, P. N., & Helfand, M. (1995). Information seeking in primary care: How physicians choose which clinical questions to pursue and which to leave unanswered. Medical Decision Making, 15(2), 113–119.CrossRefGoogle Scholar
  27. Gorry, A. (1968). Strategies for computer-aided diagnosis. Mathematical Biosciences, 2, 293–318.CrossRefGoogle Scholar
  28. Gorry, G. A., & Barnett, G. O. (1968). Experience with a model of sequential diagnosis. Computers and Biomedical Research, 1, 490–507.CrossRefGoogle Scholar
  29. Horrocks, J. C., McCann, A. P., Staniland, J. R., Leaper, D. J., & de Dombal, F. T. (1972). Computer-aided diagnosis: Description of an adaptable system, and operational experience with 2, 034 cases. British Medical Journal, 2(5804), 5–9.CrossRefGoogle Scholar
  30. Hupp, J. A., Cimino, J. J., Hoffer, E. F., Lowe, H. J., & Barnett, G. O. (1986). Explain-A computer-based diagnostic knowledge base. In: Proc Fifth World Conference on Medical Informatics, MEDINFO 86 (pp. 3117–3121). Amsterdam: North-Holland.Google Scholar
  31. Kingsland, L. C. III, Sharp, G. C., Kay, D. R., Weiss, S. M., Roeseler, G. C., & Lindberg, D. A. B. (1982). An expert consultant system in rheumatology: AI/RHEUM. Proc Sixth Ann Symp Comput Appl Med Care (pp. 748–752).Google Scholar
  32. Kingsland, L., Sharp, G., & Capps, R. (1983). Testing of a criteria-based consultant system in rheumatology. In J. van Bemmel, M. Ball, O. Wigertz, et al. (Eds.), Proceedings of MEDlNFO-83 (pp. 514–517). Amsterdam, The Netherlands: North-Holland.Google Scholar
  33. Lau, L. M., & Warner, H. R. (1992). Performance of a diagnostic system (Iliad) as a tool for quality assurance. Computers and Biomedical Research, 25, 314–323.CrossRefGoogle Scholar
  34. Ledley, R. S., & Lusted, L. B. (1959). Reasoning foundations of medical diagnosis. Science, 130, 9–21.CrossRefGoogle Scholar
  35. Leigh, T. M., Young, P. R., & Haley, J. V. (1993). Performances of family practice diplomates on successive mandatory recertification examinations. Academic Medicine, 68(12), 912–918.CrossRefGoogle Scholar
  36. Lindbeg, D. A. B., Rowland, L. R., Buch, C. R. Jr., Morse, W. F., & Morse, S. S. (1968). CONSIDER: A computer program for medical instruction. Proc Ninth IBM Med Symp.Google Scholar
  37. Lipkin, M., & Hardy, J. D. (1958). Mechanical correlation of data in differential diagnosis of hematological diseases. JAMA, 166, 113–123.Google Scholar
  38. Madlon-Kay, D. J. (1989). The weight of medical knowledge: Still gaining. New England Journal of Medicine, 321(13), 908.Google Scholar
  39. Masarie, F. E., Jr., Miller, R. A., & Myers, J. D. (1985). INTERNIST-I PROPERTIES: Representing common sense and good medical practice in a computerized medical knowledge base. Computers and Biomedical Research, 18, 458–479.CrossRefGoogle Scholar
  40. Miller, R. A. (1984). Internist-1/CADUCEUS: Problems facing expert consultant programs. Methods of Information in Medicine, 23, 9–14.Google Scholar
  41. Miller, R. A. (1990). Why the standard view is standard: People, not machines, understand patients’ problems. Journal of Medicine and Philosophy, 15, 581–591.Google Scholar
  42. Miller, R. A. (1994). Medical diagnostic decision support systems past, present, and future. Journal of the American Medical Informatics Association, 1, 8–27.Google Scholar
  43. Miller, R. A. (1996). Evaluating evaluations of medical diagnostic systems. Journal of the American Medical Informatics Association, 3, 429–431.Google Scholar
  44. Miller, R. A. (1997). A heuristic approach to the multiple diagnoses problem. In E. Keravnou, C. Garbay, R. Baud, & J. Wyatt (Eds.), Artificial intelligence in medicine, proceedings of AIME 97. Lecture notes in artificial intelligence series, #1211 (pp. 187–198). Berlin: Springer.Google Scholar
  45. Miller, R. A., & Gardner, R. M. (1997). Summary recommendations for responsible monitoring and regulation of clinical software systems. Annals of Internal Medicine, 127(9), 842–845.Google Scholar
  46. Miller, R. A., & Masarie, F. E., Jr. (1989). Use of the quick medical reference (QMR) (R) program as a tool for medical education. Methods of Information in Medicine, 28, 340–345.Google Scholar
  47. Miller, R. A., & Masarie, F. E., Jr. (1990). The demise of the “greek oracle” model for medical diagnostic systems. Methods of Information in Medicine, 29, 1–2.Google Scholar
  48. Miller, R. A., & Masarie, F. E. (1992). The Quick Medical Reference (QMR) Relationships function: Description and evaluation of a simple, efficient “multiple diagnoses” algorithm. Proc MEDINFO 92 (pp. 512–518). Geneva, Switzerland.Google Scholar
  49. Miller, R. A., & Schaffner, K. F. (1982). The logic of problem-solving in clinical diagnosis: A course for second-year medical students. Journal of Medical Education, 57, 63–65.Google Scholar
  50. Miller, R. A., Pople, H. E., Jr., & Myers, J. D. (1982). INTERNIST-1, an experimental computer-based diagnostic consultant for general internal medicine. New England Journal of Medicine, 307, 468–476.CrossRefGoogle Scholar
  51. Miller, R. A., Schaffner, K. F., & Meisel, A. (1985). Ethical and legal issues related to the use of computer programs in clinical medicine. Annals of Internal Medicine, 102, 529–536.Google Scholar
  52. Miller, R. A., Masarie, F. E., & Myers, J. D. (1986a). “Quick medical reference” for diagnostic assistance. MD Computing, 3, 34–48.Google Scholar
  53. Miller, R. A., McNeil, M. A., Challinor, S., Masarie, F. E., & Myers, J. D. (1986b). Status report: The INTERNIST-1/quick medical reference project. Western Journal of Medicine, 145, 816–822.Google Scholar
  54. Nash, F. A. (1954). Differential diagnosis: An apparatus to assist the logical faculties. Lancet, 1, 874.CrossRefGoogle Scholar
  55. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  56. Newman-Toker, D. E., & Pronovost, P. J. (2009). Diagnostic errors—The next frontier for patient safety. JAMA, 301(10), 1060–1062.CrossRefGoogle Scholar
  57. Nordyke, R. A., Kulikowski, C. A., & Kulikowski, C. W. (1971). A comparison of methods for the automated diagnosis of thyroid dysfunction. Computers and Biomedical Research, 4(4), 374–389.CrossRefGoogle Scholar
  58. Osheroff, J. A., Forsythe, D. E., Buchanan, B. G., Bankowitz, R. A., Blumenfeld, B. H., & Miller, R. A. (1991). Physicians’ information needs: An analysis of questions posed during clinical teaching in internal medicine. Annals of Internal Medicine, 114, 576–581.Google Scholar
  59. Parker, R. C., & Miller, R. A. (1989). Creation of a knowledge base adequate for simulating patient cases: Adding deep knowledge to the INTERNIST-1/QMR knowledge base. Methods of Information in Medicine, 28, 346–351.Google Scholar
  60. Pauker, S. G., Gorry, G. A., Kassirer, J. P., & Schwartz, W. B. (1976). Towards the simulation of clinical cognition. Taking a present illness by computer. American Journal of Medicine, 60(7), 981–996.CrossRefGoogle Scholar
  61. Pearl, J. (1987). Evidential reasoning using stochastic simulation of causal models. Artificial Intelligence, 32, 245–252.CrossRefGoogle Scholar
  62. Pople, H. E., Jr. (1982). Heuristic methods for imposing structure on ill-structured problems: The structuring of medical diagnostics. In P. Szolovits (Ed.), Artificial intelligence in medicine (pp. 119–190). Boulder, Co: Westview Press. AAAS Symposium Series, no. 51.Google Scholar
  63. Pople, H. E., Myers, J. D., & Miller, R. A (1975). DIALOG: A model of diagnostic logic for internal medicine. In: Proceedings of the fourth International Joint Conference on Artificial Intelligence (pp. 848–855). Cambridge, Massachusetts: MIT Artificial Intelligence Laboratory Publications.Google Scholar
  64. Porter, J. F., Kingsland, L. C., I. I. I., Lindbeerg, D. A., et al. (1988). The AI/RHEUM knowledge-based computer consulatant system in rheumatology. Performance in the diagnosis of 59 connective tissue disease patients from Japan. Arthritis and Rheumatism, 31, 219–226.CrossRefGoogle Scholar
  65. Ramnarayan, P., Kapoor, R. R., Coren, M., Nanduri, V., Tomlinson, A. L., Taylor, P. M., et al. (2003). Measuring the impact of diagnostic decision support on the quality of clinical decision making: Development of a reliable and valid composite score. Journal of the American Medical Informatics Association, 10(6), 563–572.CrossRefGoogle Scholar
  66. Ramnarayan, P., Tomlinson, A., Kulkarni, G., Rao, A., & Britto, J. (2004). A novel diagnostic aid (ISABEL): Development and preliminary evaluation of clinical performance. Studies in Health Technology and Informatics, 107(Pt 2), 1091–1095.Google Scholar
  67. Ramnarayan, P., Winrow, A., Coren, M., Nanduri, V., Buchdahl, R., Jacobs, B., et al. (2006). Diagnostic omission errors in acute paediatric practice: Impact of a reminder system on decision-making. BMC Medical Informatics and Decision Making, 6, 37–39.CrossRefGoogle Scholar
  68. Ramsey, P. G., Carline, J. D., Inui, T. S., Larson, Lo. Gerfo. J. P., Norcini, J. J., & Wenrich, M. D. (1991). Changes over time in the knowledge base of practicing internists. JAMA, 266(8), 1103–1107.CrossRefGoogle Scholar
  69. Shiffman, R. N. (1995). Guideline maintenance and revision. 50 years of the Jones criteria for diagnosis of rheumatic fever. Archives of Pediatrics and Adolescent Medicine, 149(7), 727–732.Google Scholar
  70. Shortliffe, E. H. (1976). Computcr-based medical consultations: MYCIN. Artificial Intelligence Series. New York: Elsevier Computer Science Library.Google Scholar
  71. Statnikov, A., Aliferis, C. F., Tsamardinos, I., Hardin, D., & Levy, S. (2005). A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis. Bioinformatics, 21(5), 631–643.CrossRefGoogle Scholar
  72. Stead, W. W., Haynes, R. B., Fuller, S., Friedman, C. P., et al. (1994). Designing medical informatics research and library–Resource projects to increase what is learned. Journal of the American Medical Informatics Association, 1(1), 28–33.Google Scholar
  73. Szolovits, P., Patil, R. S., & Schwartz, W. B. (1988). Artificial intelligence in medical diagnosis. Annals of Internal Medicine, 108, 7.Google Scholar
  74. Warner, H. R., Jr. (1989). Iliad: Moving medical decision-making into new frontiers. Methods of Information in Medicine, 28, 370–372.Google Scholar
  75. Warner, H. R., Toronto, A. F., Veasey, L. G., & Stephenson, R. A. (1961). Mathematical approach to medical diagnosis. JAMA, 177, 75–81.Google Scholar
  76. Warner, H. R, Haug, P., & Bouhaddou, O., et al. (1987). ILIAD as an expert consultant to teach differential diagnosis. In: Proceedings of the Twelfth Annual Symposium on Computer Applications in Medical Care (pp. 371–376). New York: IEEE Computer Society Press.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2009

Authors and Affiliations

  1. 1.Department of Biomedical Informatics, Eskind Biomedical LibraryVanderbilt University Medical CenterNashvilleUSA

Personalised recommendations