Normalized Joint Mutual Information Measure for Image Segmentation Evaluation with Multiple Ground-Truth Images

  • Xue Bai
  • Yibiao Zhao
  • Yaping Huang
  • Siwei Luo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6854)


Supervised or ground-truth-based image segmentation evaluation paradigm plays an important role in objectively evaluating segmentation algorithms. So far, many evaluation methods in terms of comparing clusterings in machine learning field have been developed. Being different from recognition task, image segmentation is considered an ill-defined problem. In a hand-labeled segmentations dataset, for the same image, different human subjects always produce various segmented results, leading to more than one ground-truth segmentations for an image. Thus, it is necessary to extend the traditional pairwise similarity measures that compare a machine generated clustering and a “true” clustering to handle multiple ground-truth clusterings. In this paper, based on the Normalized Mutual Information (NMI) which is a popular information theoretic measure for clustering comparison, we propose to utilize the Normalized Joint Mutual Information (NJMI), an extension of the NMI, to achieve the goal mentioned above. We illustrate the effectiveness of NJMI for objective segmentation evaluation with multiple ground-truth segmentations by testing it on images from Berkeley segmentation dataset.


image segmentation evaluation similarity measure joint mutual information 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics. In: ICCV (2001)Google Scholar
  2. 2.
    Rand, W.M.: Objective Criteria for the Evaluation Clustering Methods. Journal of the American Statistical Association 66(336), 846–850 (1971)CrossRefGoogle Scholar
  3. 3.
    Meilǎ, M., Heckerman, D.: An Experimental Comparison of Model-based Clustering Methods. Machine Learning 42(1-2), 9–29 (2001)CrossRefzbMATHGoogle Scholar
  4. 4.
    Strehl, A., Ghosh, J.: Cluster Ensembles—A Knowledge Reuse Framework for Combining Multiple Partitions. J. Machine Learning Research 3, 583–617 (2002)MathSciNetzbMATHGoogle Scholar
  5. 5.
    Meilǎ, M.: Comparing Clusterings by the Variation of Information. In: Conf. Learning Theory (2003)Google Scholar
  6. 6.
    Jiang, X., Marti, C., Irniger, C., Bunke, H.: Distance Measures for Image Segmentation Evaluation. EURASIP Journal on Applied Signal Processing, 1–10 (2006)Google Scholar
  7. 7.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: Toward Objective Evaluation of Image Segmentation Algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence 29(6), 929–944 (2007)CrossRefGoogle Scholar
  8. 8.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, Chichester (1991)CrossRefzbMATHGoogle Scholar
  9. 9.
    Slonim, N., Atwal, G.S., Tkačik, G., Bialek, W.: Information-based Clustering. PNAS 102(51), 18297–18302 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Zhou, Z., Li, N.: Multi-information Ensemble Diversity. In: El Gayar, N., Kittler, J., Roli, F. (eds.) MCS 2010. LNCS, vol. 5997, pp. 134–144. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Ge, F., Wang, S., Liu, T.: New Benchmark for Image Segmentation Evaluation. Journal of Electronic Imaging 16(3) (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Xue Bai
    • 1
  • Yibiao Zhao
    • 1
  • Yaping Huang
    • 1
  • Siwei Luo
    • 1
  1. 1.School of Computer and Information TechnologyBeijing Jiaotong UniversityChina

Personalised recommendations