Skip to main content
SpringerLink
Log in

Semantic Image Segmentation

  • Reference work entry
  • First Online:
Computer Vision

Synonyms

Object segmentation; Scene/image parsing

Definition

Semantic image segmentation describes the task of partitioning an image into regions that delineate meaningful objects and labeling those regions with an object category label. Some example semantic segmentations are given in Fig. 1. It can be seen as a generalization of figure-ground segmentation [1] where one segments a particular object, say a horse, from the background.

Semantic Image Segmentation, Fig. 1
figure 177 figure 177

Top row: three input images. Bottom row: the corresponding semantic segmentations where colors represent object categories

Full size image

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 649.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 899.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Borenstein E, Ullman S (2002) Class-specific, top-down segmentation. In: Heyden A, Sparr G, Johansen P (eds) Proceedings of the European conference on computer vision. Lecture notes in computer science (ECCV), vol 2351. Springer, Berlin, pp 109–124

    Google Scholar 

  2. . Wright W (1989) Image labelling with a neural network. In: Proceedings of the 5th Alvey vision conference, Reading

    Google Scholar 

  3. Everingham M, Thomas B, Troscianko T (1999) Head-mounted mobility aid for low vision using scene classification techniques. Int J Virtual Real 3(4):3–12

    Google Scholar 

  4. . Konishi S, Yuille AL (2000) Statistical cues for domain specific image segmentation with performance analysis. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Hilton Head, vol 1, pp 125–132

    Google Scholar 

  5. Shotton J, Winn J, Rother C, Criminisi A (2009) Textonboost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81(1):2–23

    Article  Google Scholar 

  6. . Shotton J, Johnson M, Cipolla R (2008) Semantic texton forests for image categorization and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Anchorage

    Google Scholar 

  7. Campbell N, Mackeown W, Thomas B, Troscianko T (1997) Interpreting image databases by region classification. Pattern Recognit 30:555–563

    Article  Google Scholar 

  8. . Shi J, Malik J (1997) Normalized cuts and image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), San Juan, pp 731–737

    Google Scholar 

  9. Felzenszwalb P, Huttenlocher D (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2): 167–181

    Article  Google Scholar 

  10. . Carreira J, Sminchisescu C (2010) Constrained parametric min-cuts for automatic object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), San Francisco

    Google Scholar 

  11. Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(6):721–741

    Article  MATH  Google Scholar 

  12. . Lafferty J, McCallum A, Pereira F (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the international conference on machine learning, Williams College, pp 282–289

    Google Scholar 

  13. . Kumar S, Hebert M (2003) Discriminative random fields: a discriminative framework for contextual interaction in classification. In: Proceedings of the international conference on computer vision, Kerkyra, vol 2, pp 1150–1157

    Google Scholar 

  14. . Winn J, Shotton J (2006) The layout consistent random field for recognizing and segmenting partially occluded objects. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Miami, vol 1, pp 37–44

    Google Scholar 

  15. Kohli P, Ladický L, Torr P (2009) Robust higher order potentials for enforcing label consistency. Int J Comput Vis 82:302–324

    Article  Google Scholar 

  16. . Ladický L, Russell C, Kohli P, Torr P (2010) Graph cut based inference with co-occurrence statistics. In: Proceedings of the European conference on computer vision (ECCV), Heraklion

    Google Scholar 

  17. . Ladický L, Sturgess P, Alahari K, Russell C, Torr P (2010) What, where and how many? Combining object detectors and CRFs. In: Proceedings of the European conference on computer vision (ECCV), Heraklion

    Google Scholar 

  18. Tu Z, Chen X, Yuille A, Zhu S (2003) Image parsing: unifying segmentation, detection, and recognition. In: Proceedings of the IEEE conference on computer vision, Nice, France, vol 1, pp 18–25

    Google Scholar 

  19. . Liu C, Yuen J, Torralba A (2009) Nonparametric scene parsing: label transfer via dense scene alignment. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Miami

    Google Scholar 

  20. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  21. Belongie S, Malik J, Puzicha J (2002) Shape matching and object recognition using shape contexts. IEEE Trans Pattern Anal Mach Intell 24(24):509–522

    Article  Google Scholar 

  22. . Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), San Diego, vol 2, pp 886–893

    Google Scholar 

  23. . Brostow G, Shotton J, Fauqueur J, Cipolla R (2008) Segmentation and recognition using structure from motion point clouds. In: Proceedings of the European conference on computer vision (ECCV), Marseille

    Google Scholar 

  24. . Anguelov D, Taskar B, Chatalbashev V, Koller D, Gupta D, Ng A (2005) Discriminative learning of markov random fields for segmentation of 3D scan data. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Beijing

    Google Scholar 

  25. Torralba A, Murphy K, Freeman W, Rubin M (2003) Context-based vision system for place and object recognition. In: Proceedings of the conference on computer vision, Nice, France, vol 2, pp 273–280

    Google Scholar 

  26. . Hoiem D, Efros A (2006) Objects in perspective. In: Proceedings of the conference on computer vision and pattern recognition (CVPR), New York

    Google Scholar 

  27. . Rabinovich A, Vedaldi A, Galleguillos C, Wiewiora E, Belongie S (2007) Objects in context. In: Proceedings of the international conference on computer vision, Rio de Janeiro

    Google Scholar 

  28. . Tu Z (2008) Auto-context and its application to high-level vision tasks. In: Proceedings of the international conference on computer vision and pattern recognition (CVPR), Anchorage

    Google Scholar 

  29. . Russell BC, Efros AA, Sivic J, Freeman WT, Zisserman A (2006) Using multiple segmentations to discover objects and their extent in image collections. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), New York

    Google Scholar 

  30. . Rother C, Minka T, Blake A, Kolmogorov V (2006) Cosegmentation of image pairs by histogram matching – incorporating a global constraint into MRFs. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), New York

    Google Scholar 

  31. Barnard K, Duygulu P, de Freitas N, Forsyth D, Blei D, Jordan MI (2003) Matching words and pictures. J Mach Learn Res 3:1107–1135

    MATH  Google Scholar 

  32. . Verbeek J, Triggs B (2007) Region classification with markov field aspect models. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Minneapolis

    Google Scholar 

  33. . He X, Zemel R, Carreira-Perpińán M (2004) Multiscale conditional random fields for image labeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Washington, DC, vol 2, pp 695–702

    Google Scholar 

  34. Russell B, Torralba A, Murphy K, Freeman WT (2008) Labelme: a database and web-based tool for image annotation. Int J Comput Vis 77:157–173

    Article  Google Scholar 

  35. . Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A, The PASCAL VOC challenge. http://www.pascal-network.org/challenges/VOC/

  36. . Choi M, Lim J, Torralba A, Willsky A (2010) Exploiting hierarchical context on a large database of object categories. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), San Francisco

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Microsoft Research Ltd, Cambridge, UK

    Jamie Shotton

  2. Department of Computer Science And Applied Mathematics, Weizmann Institute of Science, Rehovot, Israel

    Pushmeet Kohli

Authors
  1. Jamie Shotton
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pushmeet Kohli
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Industrial Science, The University of Tokyo, Tokyo, Japan

    Katsushi Ikeuchi

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this entry

Cite this entry

Shotton, J., Kohli, P. (2014). Semantic Image Segmentation. In: Ikeuchi, K. (eds) Computer Vision. Springer, Boston, MA. https://doi.org/10.1007/978-0-387-31439-6_251

Download citation

Publish with us

Policies and ethics

Search

Navigation