Skip to main content
SpringerLink
Log in

Conditional random field with the multi-granular contextual information for pixel labeling

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

To make full use of the contextual information object recognition and scene understanding, a multi-granular context conditional random field (MGCCRF) model is presented to combine context information in a variety of scales. It is efficiently implemented through extending the pairwise clique to the multi-granular context windows. In the fine-granular context window, the label consistency of similar features can be obtained with the probability of the label transferring between two adjacent pixels. At the same time, the spatial relationships among different classes in the coarse-granular context window are explicated in details. To train the MGCCRF model, a piecewise training method with the bound optimization algorithm is designed to improve the performance. Experiments on two real-world image databases show that compared with other methods, the modified conditional random field model is more competitive and effective in terms of the quantitative and qualitative labeling performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bao BK, Li T, Yan S (2012) Hidden-concept driven multilabel image annotation and label ranking. IEEE Trans Multimed 14(1):199–210

    Article  Google Scholar 

  2. Chen Y, Bi J, Wang JZ (2006) MILES: multiple-instance learning via embedded instance selection. IEEE Trans Pattern Anal Mach Intell 28(12):1931–1947

    Article  Google Scholar 

  3. Chen G, Song Y, Wang F et al (2008) Semi-supervised multi-label learning by solving a sylvester equation. In: SIAM International Conference on Data Mining: 410–419

  4. Dai J, He K, Sun J (2015) Convolutional feature masking for joint object and stuff segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: 3992–4000

  5. Gould S, Fulton R, Koller D (2009) Decomposing a scene into geometric and semantically consistent regions. In: IEEE 12th International Conference on Computer Vision: 1–8

  6. He X, Zemel RS, Carreira-Perpiñán MÁ (2004) Multiscale conditional random fields for image labeling. In: IEEE computer society conference on Computer vision and pattern recognition (2): 695–702

  7. Heili A, Lopez-Mendez A, Odobez JM (2014) Exploiting long-term connectivity and visual motion in CRF-based multi-person tracking. IEEE Trans Image Process 23(7):3040–3056

    Article  MathSciNet  Google Scholar 

  8. Huang Q, Han M, Wu B et al (2011) A hierarchical conditional random field model for labeling and segmenting images of street scenes. In: IEEE Conference on Computer Vision and Pattern Recognition: 1953–1960

  9. Kae A, Sohn K, Lee H et al (2013) Augmenting CRFs with Boltzmann machine shape priors for image labeling. In: IEEE Conference on Computer Vision and Pattern Recognition: 2019–2026

  10. Krishnapuram B, Carin L, Figueiredo MAT et al (2005) Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Trans Pattern Anal Mach Intell 27(6):957–968

    Article  Google Scholar 

  11. Kumar S, Hebert M (2003) Man-made structure detection in natural images using a causal multiscale random field. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1): 119–126

  12. Kumar S, Hebert M (2005) A hierarchical field framework for unified context-based classification. In: IEEE International Conference on Computer Vision: 1284–1291

  13. Liu Y, Jin R, Yang L (2006) Semi-supervised multi-label learning by constrained non-negative matrix factorization. In: Proceedings of the national conference on artificial intelligence 21(1): 421

  14. Long J, Shelhamer E, Darrell T (2015) In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: 3431–3440

  15. Mensink T, Verbeek J, Csurka G (2013) Tree-structured CRF models for interactive image labeling. IEEE Trans Pattern Anal Mach Intell 35(2):476–489

    Article  Google Scholar 

  16. Nasierding G, Tsoumakas G, Kouzani AZ (2009) Clustering based multi-label classification for image annotation and retrieval. In: IEEE International Conference on Systems, Man and Cybernetics: 4514–4519

  17. Neher R, Srivastava A (2005) A Bayesian MRF framework for labeling terrain using hyperspectral imaging. IEEE Trans Geosci Remote Sens 43(6):1363–1374

    Article  Google Scholar 

  18. Nguyen TM, Wu QMJ (2012) Gaussian-mixture-model-based spatial neighborhood relationships for pixel labeling problem. IEEE Trans Syst Man Cybern 42(1):193–202

    Article  Google Scholar 

  19. Nowozin S, Lampert CH (2011) Structured learning and prediction in computer vision. Foundations and Trends in Computer Graphics and Vision 6(3–4):185–365

    MATH  Google Scholar 

  20. Poggi G, Scarpa G, Zerubia JB (2005) Supervised segmentation of remote sensing images based on a tree-structured MRF model. IEEE Trans Geosci Remote Sens 43(8):1901–1911

    Article  Google Scholar 

  21. Posner I, Cummins M, Newman P (2009) A generative framework for fast urban labeling using spatial and temporal context. Auton Robot 26(2–3):153–170

    Article  Google Scholar 

  22. Provost JN, Collet C, Rostaing P et al (2004) Hierarchical Markovian segmentation of multispectral images for the reconstruction of water depth maps. Comput Vis Image Underst 93(2):155–174

    Article  Google Scholar 

  23. Roig G, Boix X, De la Torre F et al (2011) Hierarchical crf with product label spaces for parts-based models. In: IEEE International Conference on Automatic Face & Gesture Recognition and Workshops: 657–664

  24. Russell C, Kohli P, Torr PHS (2009) Associative hierarchical CRFs for object class image segmentation. In: IEEE 12th International Conference on Computer Vision: 739–746

  25. Shotton J, Winn J, Rother C et al (2006) Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: Proceedings of the European on Computer Vision: 1–15

  26. Sutton C, McCallum A (2005) Piecewise training of undirected models. In: Proceedings of UAI: 568–575

  27. Sutton C, McCallum A (2007) Piecewise pseudolikelihood for efficient training of conditional random fields. In: Proceedings of the 24th international conference on Machine learning: 863–870

  28. Wilson R, Li CT (2003) A class of discrete multiresolution random fields and its application to image segmentation. IEEE Trans Pattern Anal Mach Intell 25(1):42–56

    Article  Google Scholar 

  29. Xu X, Frank E (2004) Logistic regression and boosting for labeled bags of instances. In: Proceedings of the Pacific Aisa Conference on Knowledge Discovery and Data Mining: 272–281.

  30. Yang MY, Förstner W (2011) A hierarchical conditional random field model for labeling and classifying images of man-made scenes. In: IEEE International Conference on Computer Vision Workshops: 196–203

  31. Yao B, Yang X, Zhu SC (2007) Introduction to a large-scale general purpose ground truth database: methodology, annotation tool and benchmarks. Energy Minimization Methods in Computer Vision and Pattern Recognition: 169–183

  32. Yu Y, Pedrycz W, Miao D (2013) Neighborhood rough sets based multi-label classification for automatic image annotation. Int J Approx Reason 54(9):1373–1387

    Article  MATH  Google Scholar 

  33. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048

    Article  MATH  Google Scholar 

  34. Zhong P, Wang R (2006) Object detection based on combination of conditional random field and markov random field. In: IEEE 18th International Conference on Pattern Recognition: 160–163

Download references

Acknowledgments

This work was supported by Innovation Foundations of Education for Graduate Students of Shanxi Province (No. 2015BY23).

Author information

Authors and Affiliations

  1. College of Information Engineering, Taiyuan University of Technology, No.79 West Yingze Street, Taiyuan, 030024, Shanxi, China

    Jie Zhao & Gang Xie

  2. Department of Computer Engineering, Taiyuan college, No.18 South Dachang Road, Taiyuan, 030032, Shanxi, China

    Jie Zhao

  3. National Plant Phenomics Center, Aberystwyth University, Aberystwyth, SY23 3EE, UK

    Jiwan Han

Authors
  1. Jie Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiwan Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Xie.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, J., Xie, G. & Han, J. Conditional random field with the multi-granular contextual information for pixel labeling. Multimed Tools Appl 76, 9169–9194 (2017). https://doi.org/10.1007/s11042-016-3513-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3513-0

Keywords

Search

Navigation