Receiver Operating Characteristic (ROC) in Nondestructive Inspection

Abstract

A suitable and objective method to investigate the performance of inspection systems is the Receiver Operating Characteristic (ROC) which is based on the general theory of signal detection. All diagnostic systems include the task of separating a signal from a background of noise. The idea of the ROC method is to characterise the accuracy of an inspection system by evaluating the correct detection rate versus the false alarm rate for a set of different thresholds between signal and noise applied during this task. This method is called detection ROC because it deals only with the pure detection of a signal (defect). If there are several types of defects one can also measure the accuracy of the defect classification and the correctness of the indicated defect importance v/ith the help of the so-called Joint ROC. For the industries it is particularly important to find the unacceptable defects which has to be repaired. For this case the masked ROC is calculated where only the correct or false indications of critical defects are taken into account.

The power of the method is demonstrated in terms of three examples: ultrasonic testing of fibre reinforced plastic materials used in space- and aircraft manufacturing and of welds typical for railway systems and radiographic weld inspection. Of interest was here the dependence of the testing performance from the ultrasonic frequency, from the experience of the inspectors and from the digitisation procedure, respectively. Special tests — reducing the number of samples — were undertaken to show until which minimum amount of statistical basis the result is stable.