Avoid Missing the Effect
Given the amount of time, money and effort that is spent on developing new algorithms and information systems, surprisingly little time is used to teach how to conduct a valuable evaluation. When evaluation is discussed, the focus is often on avoiding Type I errors, where a nonexistent effect is incorrectly accepted as existing. In contrast, this chapter focuses on Type II errors, where an existing effect is missed.
Type II errors often receive less attention even though, especially in medicine, making such errors may affect people’s quality of life. When a new information system is built, it is intended to improve an existing medical problem. Missing an existing effect wastes time and effort of one group and may incorrectly put others off the trail, thereby stopping further research and development of a potentially promising approach. With good experimental design, the chances of this can be minimized. To avoid missing an effect, the difference between means of experimental conditions should be sufficiently large. The within-group variation should be relatively small so that the difference between groups is larger than that within groups. And a large enough number of observations should be made in each experimental condition, especially with smaller effect sizes.
KeywordsTraining Time Bell Curve Extreme Score Nuisance Variable Valuable Evaluation
- 1.Rosenthal R, Rosnow RL (1991) Essentials of behavioral research: methods and data analysis. McGraw-Hill, BostonGoogle Scholar
- 2.Kirk RE (1995) Experimental design: procedures for the behavioral sciences, 3rd edn. Brooks/Cole Publishing Company, Pacific GroveGoogle Scholar
- 3.Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd edn. L. Erlbaum Associates, HillsdaleGoogle Scholar