, Volume 66, Issue 3, pp 406-421

The sampling distributions of Gaussian ROC statistics

Abstract

For a discrimination experiment, a plot of the hit rate against the false-alarm rate—the ROC curve— summarizes performance across a range of confidence levels. In many content areas, ROCs are well described by a normal-distribution model and thez-transformed hit and false-alarm rates are approximately linearly related. We examined the sampling distributions of three parameters of this model when applied to a ratings procedure: the area under the ROC (A z), the normalized difference between the means of the underlying signal and noise distributions (d a), and the slope of the ROC onz-coordinates (s). Statistical bias (the degree to which the mean of the sampling distribution differed from the true value) was trivial forA z, small but noticeable ford a, and substantial fors. Variability of the sampling distributions decreased with the number of trials and was also affected by the number of response categories available to the participant and by the overall sensitivity level. Figures in the article and tables available on line can be used to construct confidence intervals around ROC statistics and to test statistical hypotheses.

This research was supported in part by National Institutes of Health Research Grant MH60274 to C.M.R. and N.A.M.