Design, Conduct, and Analysis of Demonstration Studies
Demonstration studies answer questions about an information resource, exploring such issues as the resource’s value to a certain professional group or its impact on the processes and outcomes of health care.1 Recall from Chapter 4 that measurement studies are required to test, refine and validate measurement processes before they can be used to answer questions about a resource or its impact. Chapters 5 and 6 explained these ideas and how to conduct measurement studies in more detail. In this chapter we assume that measurement methods are available and have been verified by appropriate measurement studies. To answer questions via a demonstration study, appropriate evaluation strategies and study designs must be formulated, the sample of subjects and tasks defined, any threats to validity identified and either eliminated or controlled for, and the results analyzed.2, 3 These issues are discussed in this chapter.
KeywordsCare Provider Decision Support System Receiver Operating Characteristic Information Resource Hawthorne Effect
Unable to display preview. Download preview PDF.
- 7.Airman D: Practical Statistics for Medical Research. London: Chapman & Hall, 1991.Google Scholar
- 8.Cochran WG, Cox GM: Experimental Designs. New York: Wiley, 1957.Google Scholar
- 9.Winer BJ: Statistical Principles in Experimental Design. New York: McGraw Hill, 1991.Google Scholar
- 24.Cohen J: Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum, 1988.Google Scholar
- 25.Wyatt J: Lessons learned from the field trial of ACORN, an expert system to advise on chest pain. In: Barber B, Cao D, Qin D (eds) Proceedings of the Sixth World Conference on Medical Informatics, Singapore. Amsterdam: North Holland 1989:111–115.Google Scholar
- 28.Roethligsburger FJ, Dickson WJ: Management and the Worker. Cambridge, MA: Harvard University Press, 1939.Google Scholar
- 38.Wellwood J, Spiegelhalter DJ, Johannessen S: How does computer-aided diagnosis improve the management of acute abdominal pain? Ann R Coll Surg Engl 1992;74:140–146.Google Scholar
- 39.Indurkhya N, Weiss SM: Models for measuring performance of medical expert systems. AI Med 1989;1:61–70.Google Scholar
- 41.Wald N: Rational use of investigations in clinical practice. In: Hopkins A (ed) Appropriate Investigation and Treatment in Medical Practice. London: Royal College of Physicians, 1990:7–20.Google Scholar
- 49.Friedman CP, Wildemuth BM, Gant SP, Muriuki M, File DD: A comparison of Boolean and Hypertext access to a basic science database. Presented to the Annual Conference on Research in Medical Education, November 1995 [abstract].Google Scholar
- 50.Habbema JDF, Hilden J, Bjerregaard B: The measurement of performance in probabilistic diagnosis; general recommendations. Meth Inf Med 1981;20:97–100.Google Scholar
- 51.Murphy AH, Winkler RL: Probability forecasting in meteorology. J Amer Statist Assoc 1984;79:489–500.Google Scholar