During the past decade, computers and information systems and their resultant products have pervaded hospitals. Not only have the number of and investments in computers and their associated systems increased, but the types of hospital systems and affected functions have also multiplied. Unfortunately, methodologies to measure impacts of these systems have not evolved at the same pace. There have been few developments of new techniques. Existing techniques have not been adapted for use in hospitals. As a result, evaluations are not performed or are based upon generalizations, often with users minimally involved, if at all. Most techniques emphasize the cost-benefit approach, but this should not be the only tool used. This paper presents a taxonomy of methodologies for the evaluation of hospital information systems. The analyses of methodologies and systems are accomplished from user's perspectives. Effectiveness of system products is emphasized. Users are provided with an additional tool to plan, organize, control, and direct the information resources and achieve the hospital's goals and objectives.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Austin, C. J.,Information Systems for Hospital Administration, University of Michigan, Ann Arbor, 1979.
Lerner, M. A., Hager, M., Coppola, V., and Tesener, M., New war on health costs.Newsweek May 9: 24, 29, 1983.
Story, A., A computer is only as good as its managers.Heath Care (Don Mills) 23(6): 57–58, 1981.
Collen, M. F., ed.,Hospital Computer Systems. Wiley, New York, 1974.
Connell, J. J., The fallacy of information resource management.Infosystems May: 78–84, 1980.
Barrett, J. P., Barnum R. A., Gordon, B. B., and Pesut, R. N., Evaluation of the implementation of a medical information system in a general community hospital: Final report. NTIS Document NO. PB 248340, Batelle Columbus Laboratories, Columbus, Ohio, 1975.
LaViolette, S., Too much technology, not enough control.Mod. Healthcare 11(5):102, 104, 1981.
Jay, S. J. and Anderson, J. C., Computerized hospital information systems; their future role in medicine.J. R. Soc. Med. 75(5):303–304, 1982.
Whitted, G. S., What to expect from electronic data processing in medical centers.Health Care Management Rev. 3(1):53–60, 1978.
Schmitz, H. H., A protocol for evaluating hospital information systems.Hospital and Health Services Administration Winter; 45–56, 1977.
Deland, E. C., and Waxman, B. D., Review of hospital information systems.Hospital Information Systems (C. A. Bekey and M. D. Schwartz, eds.), Marcel Dekker, New York, 1972, pp. 1–28.
Isaacs, H. H., Objectives, criteria, and techniques for evaluation.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press, Boulder, Colorado, 1974, pp. 329–340.
Suchman, E. A.,Evaluative Research, Russell Sage Foundation, New York, 1967.
Weiss, C. H., Evaluation Research Methods of Assessing Program Effectiveness, Prentice-Hall, Englewood Cliffs, New Jersey, 1972.
Perkins, H. K., III, Production services.Managing the Data Resource Function, 2nd edition, R. L. Nolan, West Publishing, St. Paul, Minnesota, 1982, pp. 254–267.
O'Brien, J. F., Methodology for Assessing the Impact of Computing and Information Systems on Users. Technical Report APR-205461TR-6, Westinghouse Research and Development Center, 1977.
Murdick, R. G.,MIS Concepts and Desing, Prentice-Hall, Englewood Cliffs, New Jersey, 1980.
Axelrod, C. W., How effective is your computer?Infosystem. February: 50–52, 1979.
Barnard, C. I.,The Functions of the Executive. Harvard University Press, Cambridge, Massachusetts, 1968.
Morris, M. F., and Roth, F.,Computer Performance Evaluation: Tools and Techniques for Effective Analysis, Van Nostrand Reinhold, New York, 1982.
Carlson, E. D., Evaluating the impact of information systems.Management Informatics 3(2):57–67, 1974.
Collen, M., ed.,Medical Information Systems. NTIS PB 195-998, National Center for Health Services Research, Washington, D.C., 1970.
Drazen, E., and Metzger, J., Methods for Evaluating Costs of Automated Hospital Information Systems, DHHS (PHS) 81-3283, July 1981.
Fairman, W. L., and Dickhaus, E. A., Technology evaluation: A case study of MARSMed. Care 15 (1):79–92, 1977.
Flagle, C. D., Evaluation and control of technology in health services.Conference on Technology and Health Care Systems in the 1980's, January 1972, pp. 213–224.
Gall, J. E., Jr., et al., Demonstration and Evaluation of a Total Hospital Information System (Final Report), DHEW PB-262106, 1977.
Gall, J., Cost-benefit analysis; total hospital information systems.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press Boulder, Colorado, 1974, pp. 299–327.
Grams, R. R., Evaluation—A word for all seasons.J. Med. Sys. 3:3–6, 1979.
Priest, S. L.,Managing Hospital Information Systems Aspen Systems, Rockville, Maryland, 1982.
Richart, R. H., Evaluation of a medical data system.Comput. Biomed. Res. 3:415–425, 1970.
Richart, R. H., Evaluation of a hospital computer system.Hospital Computer Systems (M. F. Collen, ed.), J. Wiley, New York, 1974, pp. 341–417.
Schmitz, H. H.,Hospital Information Systems Aspen Systems Rockville, Maryland, 1979.
Simborg, D. W., and Whiting-O'Keefe, Q. E., Evaluation methodology for ambulatory care information systems.Med. Care 20(3):255–265, 1982.
Beggs-Baker, S., Evaluation method: The experience of the Casa De Amigos computerized patient management system.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press, Boulder, Colorado, 1974, pp. 115–154.
Coffey, R. M., How a medical information system affects hospital costs: The El Camino Hospital experience. National Center for Health Services Research, DHEW Publication No. (PHS) 80-3265, March 1980.
U.S. Department of Commerce, Demonstration and Evaluation of a Total Hospital Information System, PB-271079, March 1977.
Doyle, O., et al.,Analysis Manual for Hospital Information Systems, Association of University Professors in Health Administration, Ann Arbor, Michigan, 1980.
Hodge, M. H.,Health Planning Review of Medical Information Systems, National Center for Health Services Research, Department of Health and Human Services, 1981.
Evaluating Automated Hospital Information Systems Instruction Manual. The Lutheran Hospital Society of Southern California, Center Publications, South Pasadena, California, 1982.
Evaluating Automated Hospital Information Systems: Executive Overview. PB 82-213 620, NTIS, 1982.
Krobock, J. R., and Wilson, G. R.,One hospital: User satisfaction and perceived utility of data processing reports. Presented at the 2nd International and 5th National Conference on Computers and Industrial Engineering, Orlando, Florida, March 16–18, 1983.
Pearson, S. W., and Bailey, J. E., Measurement of Computers User Satisfaction. Proceedings of the American Institute of Industrial Engineers, Fall Industrial Engineering Conference, Atlanta, Georgia, 1978, pp. 219–227.
Synott, W. R., and Gruber, W. H.,Information Resource Mangement Opportunities and Strategies for the 1980's, Wiley, New York, 1981.
Hurst, L. L., Evaluation plan: Urban comprehensive health care information system.Health Information Systems Evaluation (R. C. Koza, ed.), Associated University Press, Boulder, Colorado, 1974, pp. 155–196.
Simborg, D. W., MacDonald, L. K., Liebman, J. S., and Musco, P., Ward information-management system—An evaluation.Comp. Biomed. Res. 5:484–497, 1972.
Hofman, M. N., An evaluation methodology for office practice information systems.J. of Med. System. 5:305–319, 1981.
Blackwelder, R. E.,Taxonomy A Text and Reference Book., Wiley, New York, 1967.
Nolan, R. L., and Finch, F. E., Towards a theory of data processing leadership.Managing the Data Resource Function., 2nd edition, R. L. Nolan, West Publishing, St. Paul, Minnesota, 1982, pp. 362–369.
About this article
Cite this article
Krobock, J.R. A taxonomy: Hospital information systems evaluation methodologies. J Med Syst 8, 419–429 (1984). https://doi.org/10.1007/BF02285254
- Information System
- System Product
- System Evaluation
- Information Resource
- Resultant Product