A New Pixel-Based Quality Measure for Segmentation Algorithms Integrating Precision, Recall and Specificity

  • Kannikar Intawong
  • Mihaela Scuturici
  • Serge Miguet
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8047)


There are several approaches for performance evaluation of image processing algorithms in video-based surveillance systems: Precision/ Recall, Receiver Operator Characteristics (ROC), F-measure, Jaccard Coefficient, etc. These measures can be used to find good values for input parameters of image segmentation algorithms. Different measures can give different values of these parameters, considered as optimal by one criterion, but not by another. Most of the times, the measures are expressed as a compromise between two of the three aspects that are important for a quality assessment: Precision, Recall and Specificity. In this paper, we propose a new 3-dimensional measure (D prs ), which takes into account all of the three aspects. It can be considered as a 3D generalization of 2D ROC analysis and Precision/Recall curves. To estimate the impact of parameters on the quality of the segmentation, we study the behavior of this measure and compare it with several classical measures. Both objective and subjective evaluations confirm that our new measure allows to determine more stable parameters than classical criteria, and to obtain better segmentations of images.


Segmentation quality measures F-measure Jaccard Coefficient(JC) Percentage of Correct Classification(PCC) 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Nascimento, J., Marques, J.: Performance evaluation of object detection algorithms for video surveillance. IEEE Transactions on Multimedia 8, 761–777 (2006)CrossRefGoogle Scholar
  2. 2.
    Rosin, P.L., Ioannidis, E.: Evaluattion of Global Image Thresholding for Change Detection. Pattern Recognition Letter 24, 345–2356 (2003)Google Scholar
  3. 3.
    Elhabian, S., El-Sayed, K.: Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art. Recent Patents on Computer Science 1, 32–54 (2008)CrossRefGoogle Scholar
  4. 4.
    Gao, X., Boult, T.E., Coetzee, F., Ramesh, V.: Error analysis of background adaptation. In: Computer Vision and Pattern Recognition, vol. 1, pp. 503–510 (2000)Google Scholar
  5. 5.
    Davis, J., Burnside, E., Dutra, I., Page, D., Ramakrishnan, R., Santos Costa, V., Shavlik, J.: View learning for statistical relational learning: With an application to mammography. In: Proceeding of the 19th International Joint Conference on Artificial Intelligence, pp. 677–683 (2005)Google Scholar
  6. 6.
    Davis, J., Goadrich, M.: The Relationship between Precision-Recall and ROC Curves. In: International Conference on Machine Learning, pp. 233–240 (2006)Google Scholar
  7. 7.
    Wang, S., Chang, C.I., Yang, S.C., Hsu, G.C., Hsu, H.H., Chung, P.C., Guo, S.M., Lee, S.K.: 3D ROC Analysis for Medical Imaging Diagnosis. Engineering in Medicine and Biology Society, 7545–7548 (2005)Google Scholar
  8. 8.
    Chen, Y.T., Chen, C.S., Huang, C.R., Hung, Y.P.: Efficient hierarchical method for background subtraction. Pattern Recognition, 2706–2715 (2007)Google Scholar
  9. 9.
    Dhome, Y., Tronson, N., Vacavant, A.: A Benchmark for Background Subtraction Algorithms in Monocular Vision: A Comparative Study. Image Processing Theory Tools and Applications (IPTA), 7–10 (2010)Google Scholar
  10. 10.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 246–252 (1999)Google Scholar
  11. 11.
    Huang, C.R., Chen, C.S., Chung, P.C.: Contrast context histogram A discriminating local descriptor for image matching. In: Proceedings of IEEE International Conference on Pattern Recognition, vol. 4, pp. 53–56 (2006)Google Scholar
  12. 12.
    Chateau, T.: Visage Challenge: Video-surveillance Intelligente: Systemes et AlGorithmES (2012),

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Kannikar Intawong
    • 1
  • Mihaela Scuturici
    • 1
  • Serge Miguet
    • 1
  1. 1.LIRIS UMR5205Université de Lyon, CNRS, Université Lumière Lyon 2BronFrance

Personalised recommendations