A New Pixel-Based Quality Measure for Segmentation Algorithms Integrating Precision, Recall and Specificity
There are several approaches for performance evaluation of image processing algorithms in video-based surveillance systems: Precision/ Recall, Receiver Operator Characteristics (ROC), F-measure, Jaccard Coefficient, etc. These measures can be used to find good values for input parameters of image segmentation algorithms. Different measures can give different values of these parameters, considered as optimal by one criterion, but not by another. Most of the times, the measures are expressed as a compromise between two of the three aspects that are important for a quality assessment: Precision, Recall and Specificity. In this paper, we propose a new 3-dimensional measure (D prs ), which takes into account all of the three aspects. It can be considered as a 3D generalization of 2D ROC analysis and Precision/Recall curves. To estimate the impact of parameters on the quality of the segmentation, we study the behavior of this measure and compare it with several classical measures. Both objective and subjective evaluations confirm that our new measure allows to determine more stable parameters than classical criteria, and to obtain better segmentations of images.
KeywordsSegmentation quality measures F-measure Jaccard Coefficient(JC) Percentage of Correct Classification(PCC)
Unable to display preview. Download preview PDF.
- 2.Rosin, P.L., Ioannidis, E.: Evaluattion of Global Image Thresholding for Change Detection. Pattern Recognition Letter 24, 345–2356 (2003)Google Scholar
- 4.Gao, X., Boult, T.E., Coetzee, F., Ramesh, V.: Error analysis of background adaptation. In: Computer Vision and Pattern Recognition, vol. 1, pp. 503–510 (2000)Google Scholar
- 5.Davis, J., Burnside, E., Dutra, I., Page, D., Ramakrishnan, R., Santos Costa, V., Shavlik, J.: View learning for statistical relational learning: With an application to mammography. In: Proceeding of the 19th International Joint Conference on Artificial Intelligence, pp. 677–683 (2005)Google Scholar
- 6.Davis, J., Goadrich, M.: The Relationship between Precision-Recall and ROC Curves. In: International Conference on Machine Learning, pp. 233–240 (2006)Google Scholar
- 7.Wang, S., Chang, C.I., Yang, S.C., Hsu, G.C., Hsu, H.H., Chung, P.C., Guo, S.M., Lee, S.K.: 3D ROC Analysis for Medical Imaging Diagnosis. Engineering in Medicine and Biology Society, 7545–7548 (2005)Google Scholar
- 8.Chen, Y.T., Chen, C.S., Huang, C.R., Hung, Y.P.: Efficient hierarchical method for background subtraction. Pattern Recognition, 2706–2715 (2007)Google Scholar
- 9.Dhome, Y., Tronson, N., Vacavant, A.: A Benchmark for Background Subtraction Algorithms in Monocular Vision: A Comparative Study. Image Processing Theory Tools and Applications (IPTA), 7–10 (2010)Google Scholar
- 10.Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 246–252 (1999)Google Scholar
- 11.Huang, C.R., Chen, C.S., Chung, P.C.: Contrast context histogram A discriminating local descriptor for image matching. In: Proceedings of IEEE International Conference on Pattern Recognition, vol. 4, pp. 53–56 (2006)Google Scholar
- 12.Chateau, T.: Visage Challenge: Video-surveillance Intelligente: Systemes et AlGorithmES (2012), http://visage.univ-bpclermont.fr/?q=node/2