Journal of Medical Systems

, Volume 8, Issue 5, pp 419–429 | Cite as

A taxonomy: Hospital information systems evaluation methodologies

  • John R. Krobock


During the past decade, computers and information systems and their resultant products have pervaded hospitals. Not only have the number of and investments in computers and their associated systems increased, but the types of hospital systems and affected functions have also multiplied. Unfortunately, methodologies to measure impacts of these systems have not evolved at the same pace. There have been few developments of new techniques. Existing techniques have not been adapted for use in hospitals. As a result, evaluations are not performed or are based upon generalizations, often with users minimally involved, if at all. Most techniques emphasize the cost-benefit approach, but this should not be the only tool used. This paper presents a taxonomy of methodologies for the evaluation of hospital information systems. The analyses of methodologies and systems are accomplished from user's perspectives. Effectiveness of system products is emphasized. Users are provided with an additional tool to plan, organize, control, and direct the information resources and achieve the hospital's goals and objectives.


Information System System Product System Evaluation Information Resource Resultant Product 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Austin, C. J.,Information Systems for Hospital Administration, University of Michigan, Ann Arbor, 1979.Google Scholar
  2. 2.
    Lerner, M. A., Hager, M., Coppola, V., and Tesener, M., New war on health costs.Newsweek May 9: 24, 29, 1983.Google Scholar
  3. 3.
    Story, A., A computer is only as good as its managers.Heath Care (Don Mills) 23(6): 57–58, 1981.Google Scholar
  4. 4.
    Collen, M. F., ed.,Hospital Computer Systems. Wiley, New York, 1974.Google Scholar
  5. 5.
    Connell, J. J., The fallacy of information resource management.Infosystems May: 78–84, 1980.Google Scholar
  6. 6.
    Barrett, J. P., Barnum R. A., Gordon, B. B., and Pesut, R. N., Evaluation of the implementation of a medical information system in a general community hospital: Final report. NTIS Document NO. PB 248340, Batelle Columbus Laboratories, Columbus, Ohio, 1975.Google Scholar
  7. 7.
    LaViolette, S., Too much technology, not enough control.Mod. Healthcare 11(5):102, 104, 1981.Google Scholar
  8. 8.
    Jay, S. J. and Anderson, J. C., Computerized hospital information systems; their future role in medicine.J. R. Soc. Med. 75(5):303–304, 1982.Google Scholar
  9. 9.
    Whitted, G. S., What to expect from electronic data processing in medical centers.Health Care Management Rev. 3(1):53–60, 1978.Google Scholar
  10. 10.
    Schmitz, H. H., A protocol for evaluating hospital information systems.Hospital and Health Services Administration Winter; 45–56, 1977.Google Scholar
  11. 11.
    Deland, E. C., and Waxman, B. D., Review of hospital information systems.Hospital Information Systems (C. A. Bekey and M. D. Schwartz, eds.), Marcel Dekker, New York, 1972, pp. 1–28.Google Scholar
  12. 12.
    Isaacs, H. H., Objectives, criteria, and techniques for evaluation.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press, Boulder, Colorado, 1974, pp. 329–340.Google Scholar
  13. 13.
    Suchman, E. A.,Evaluative Research, Russell Sage Foundation, New York, 1967.Google Scholar
  14. 14.
    Weiss, C. H., Evaluation Research Methods of Assessing Program Effectiveness, Prentice-Hall, Englewood Cliffs, New Jersey, 1972.Google Scholar
  15. 15.
    Perkins, H. K., III, Production services.Managing the Data Resource Function, 2nd edition, R. L. Nolan, West Publishing, St. Paul, Minnesota, 1982, pp. 254–267.Google Scholar
  16. 16.
    O'Brien, J. F., Methodology for Assessing the Impact of Computing and Information Systems on Users. Technical Report APR-205461TR-6, Westinghouse Research and Development Center, 1977.Google Scholar
  17. 17.
    Murdick, R. G.,MIS Concepts and Desing, Prentice-Hall, Englewood Cliffs, New Jersey, 1980.Google Scholar
  18. 18.
    Axelrod, C. W., How effective is your computer?Infosystem. February: 50–52, 1979.Google Scholar
  19. 19.
    Barnard, C. I.,The Functions of the Executive. Harvard University Press, Cambridge, Massachusetts, 1968.Google Scholar
  20. 20.
    Morris, M. F., and Roth, F.,Computer Performance Evaluation: Tools and Techniques for Effective Analysis, Van Nostrand Reinhold, New York, 1982.Google Scholar
  21. 21.
    Carlson, E. D., Evaluating the impact of information systems.Management Informatics 3(2):57–67, 1974.Google Scholar
  22. 22.
    Collen, M., ed.,Medical Information Systems. NTIS PB 195-998, National Center for Health Services Research, Washington, D.C., 1970.Google Scholar
  23. 23.
    Drazen, E., and Metzger, J., Methods for Evaluating Costs of Automated Hospital Information Systems, DHHS (PHS) 81-3283, July 1981.Google Scholar
  24. 24.
    Fairman, W. L., and Dickhaus, E. A., Technology evaluation: A case study of MARSMed. Care 15 (1):79–92, 1977.Google Scholar
  25. 25.
    Flagle, C. D., Evaluation and control of technology in health services.Conference on Technology and Health Care Systems in the 1980's, January 1972, pp. 213–224.Google Scholar
  26. 26.
    Gall, J. E., Jr., et al., Demonstration and Evaluation of a Total Hospital Information System (Final Report), DHEW PB-262106, 1977.Google Scholar
  27. 27.
    Gall, J., Cost-benefit analysis; total hospital information systems.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press Boulder, Colorado, 1974, pp. 299–327.Google Scholar
  28. 28.
    Grams, R. R., Evaluation—A word for all seasons.J. Med. Sys. 3:3–6, 1979.Google Scholar
  29. 29.
    Priest, S. L.,Managing Hospital Information Systems Aspen Systems, Rockville, Maryland, 1982.Google Scholar
  30. 30.
    Richart, R. H., Evaluation of a medical data system.Comput. Biomed. Res. 3:415–425, 1970.Google Scholar
  31. 31.
    Richart, R. H., Evaluation of a hospital computer system.Hospital Computer Systems (M. F. Collen, ed.), J. Wiley, New York, 1974, pp. 341–417.Google Scholar
  32. 32.
    Schmitz, H. H.,Hospital Information Systems Aspen Systems Rockville, Maryland, 1979.Google Scholar
  33. 33.
    Simborg, D. W., and Whiting-O'Keefe, Q. E., Evaluation methodology for ambulatory care information systems.Med. Care 20(3):255–265, 1982.Google Scholar
  34. 34.
    Beggs-Baker, S., Evaluation method: The experience of the Casa De Amigos computerized patient management system.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press, Boulder, Colorado, 1974, pp. 115–154.Google Scholar
  35. 35.
    Coffey, R. M., How a medical information system affects hospital costs: The El Camino Hospital experience. National Center for Health Services Research, DHEW Publication No. (PHS) 80-3265, March 1980.Google Scholar
  36. 36.
    U.S. Department of Commerce, Demonstration and Evaluation of a Total Hospital Information System, PB-271079, March 1977.Google Scholar
  37. 37.
    Doyle, O., et al.,Analysis Manual for Hospital Information Systems, Association of University Professors in Health Administration, Ann Arbor, Michigan, 1980.Google Scholar
  38. 38.
    Hodge, M. H.,Health Planning Review of Medical Information Systems, National Center for Health Services Research, Department of Health and Human Services, 1981.Google Scholar
  39. 39.
    Evaluating Automated Hospital Information Systems Instruction Manual. The Lutheran Hospital Society of Southern California, Center Publications, South Pasadena, California, 1982.Google Scholar
  40. 40.
    Evaluating Automated Hospital Information Systems: Executive Overview. PB 82-213 620, NTIS, 1982.Google Scholar
  41. 41.
    Krobock, J. R., and Wilson, G. R.,One hospital: User satisfaction and perceived utility of data processing reports. Presented at the 2nd International and 5th National Conference on Computers and Industrial Engineering, Orlando, Florida, March 16–18, 1983.Google Scholar
  42. 42.
    Pearson, S. W., and Bailey, J. E., Measurement of Computers User Satisfaction. Proceedings of the American Institute of Industrial Engineers, Fall Industrial Engineering Conference, Atlanta, Georgia, 1978, pp. 219–227.Google Scholar
  43. 43.
    Synott, W. R., and Gruber, W. H.,Information Resource Mangement Opportunities and Strategies for the 1980's, Wiley, New York, 1981.Google Scholar
  44. 44.
    Hurst, L. L., Evaluation plan: Urban comprehensive health care information system.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press, Boulder, Colorado, 1974, pp. 155–196.Google Scholar
  45. 45.
    Simborg, D. W., MacDonald, L. K., Liebman, J. S., and Musco, P., Ward information-management system—An evaluation.Comp. Biomed. Res. 5:484–497, 1972.Google Scholar
  46. 46.
    Hofman, M. N., An evaluation methodology for office practice information systems.J. of Med. System. 5:305–319, 1981.Google Scholar
  47. 47.
    Blackwelder, R. E.,Taxonomy A Text and Reference Book., Wiley, New York, 1967.Google Scholar
  48. 48.
    Nolan, R. L., and Finch, F. E., Towards a theory of data processing leadership.Managing the Data Resource Function., 2nd edition, R. L. Nolan, West Publishing, St. Paul, Minnesota, 1982, pp. 362–369.Google Scholar

Copyright information

© Plenum Publishing Corporation 1984

Authors and Affiliations

  • John R. Krobock
    • 1
  1. 1.Department of Management Information ScienceCalifornia State UniversitySacramento

Personalised recommendations