Skip to main content
SpringerLink
Log in
Book cover

Video Segmentation and Its Applications pp 1–23Cite as

Image/Video Segmentation: Current Status, Trends, and Challenges

  • Chapter
  • First Online:

Abstract

Segmentation plays an important role in digital media processing, pattern recognition, and computer vision. The task of image/video segmentation emerges in many application areas, such as image interpretation, video analysis and understanding, video summarization and indexing, and digital entertainment. Over the last two decades, the problem of segmenting image/video data has become a fundamental one and had significant impact on both new pattern recognition algorithms and applications.This chapter has several objectives: (1) to survey the current status of research activities including graph-based, density estimator-based, and temporal-based segmentation algorithms. (2) To discuss recent developments while providing a comprehensive introduction to the fields of image/video segmentation. (3) To identify challenges ahead, and outline perspectives for the years to come.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ngan King N., Li H.: Semantic Object Segmentation. IEEE Communications Society Multimedia Communications Technical Committee E-Letter, 4(6), 6–8 (2009)

    Google Scholar 

  2. Meier T., Ngan K. N.: Automatic segmentation of moving objects for video objects plane generation. IEEE Trans. Circuits and Systems for Video Technology, 8(5), 525–538 (1998)

    Article  Google Scholar 

  3. Li H., Ngan King N.: Automatic video segmentation and tracking for content-based applications, IEEE Communications Magazine, 45(1), 27–33 (2007)

    Google Scholar 

  4. Bornhovd C., BuchmannA. P.: A Prototype for metadata-based integration of internet sources, Advanced Information Systems Engineering, Lecture Notes in Computer Science, 1626, 439–445 (2010)

    Google Scholar 

  5. Kokkinos I., Maragos P.: Synergy between Object Recognition and Image Segmentation Using the Expectation-Maximization Algorithm, IEEE Trans. Pattern Analysis and Machine Intelligence, 31(8), 1486–1501 (2009)

    Article  Google Scholar 

  6. Todorovic S., Ahuja N.: Unsupervised category modeling, recognition, and segmentation in images, IEEE Trans. Pattern Analysis and Machine Intelligence, 30(12), 2158– 2174 (2008)

    Article  Google Scholar 

  7. Gentile, C., Camps, O., Sznaier, M.: Segmentation for robust tracking in the presence of severe occlusion. IEEE Trans. Image Processing, 13(2), 166–178 (2004)

    Article  Google Scholar 

  8. Mezaris V., Kompatsiaris I., Boulgouris N. V., Strintzis M. G.: Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval, IEEE Trans. Circuits and Systems for Video Technology, 14(5), 606–621 (2004)

    Article  Google Scholar 

  9. Ko, B., Byun, H.: FRIP: a region-based image retrieval tool using automatic image segmentation and stepwise Boolean AND matching, IEEE Trans. Multimedia, 7(1), 105–113 (2005)

    Article  Google Scholar 

  10. Meier T., Ngan K.N.: Video segmentation for content-based coding, IEEE Transactions on Circuits and Systems for Video Technology, 9(8), 1190–1203 (1999)

    Article  Google Scholar 

  11. Zitnick C. L., Kang S. B.: Stereo for Image-based rendering using image over-segmentation, International Journal of Computer Vision, 75(1), 49–65 (2007)

    Article  Google Scholar 

  12. Chai D., Ngan K.N.: Face segmentation using skin color map in videophone applications, IEEE Transactions on Circuits and Systems for Video Technology, 9(4), 551–564 (1999)

    Article  Google Scholar 

  13. Ren X., Malik J.: Learning a classification model for segmentation, Intl Conf. Computer Vision (ICCV), vol. 1, 10–17 (2003)

    Google Scholar 

  14. Nikhil R. P., Sankar K. P.: A review on image segmentation techniques, Pattern Recognition, 26(9), 1277–1294 (1993)

    Article  Google Scholar 

  15. Cremers D., Rousson M., Deriche R.: A Review of statistical approaches to level set segmentation: integrating color, texture, motion and shape, International Journal of Computer Vision, 72(2), 195–215, (2007)

    Article  Google Scholar 

  16. Noble J.A., Boukerroui, D.: Ultrasound image segmentation: a survey, IEEE Trans. Medical Imaging, 25(8), 987–1010 (2006)

    Article  Google Scholar 

  17. Koprinska I., Carrato S.: Temporal video segmentation: A survey, Signal processing: Image communication, 16(5), 477–500 (2001)

    Article  Google Scholar 

  18. Greig D. M., Porteous B.T., Seheult A.H.: Exact maximum a posteriori estimation for binary images, Journal of the Royal Statistical Society Series B (Methodological), 271–279 (1989)

    Google Scholar 

  19. Boykov Y., Jolly M.-P.: “Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images”, Proc.ICCV 2001, vol. 1, 105–112 (2001)

    Google Scholar 

  20. Rother C., Kolmogorov V., Blake A.: GrabCut: interactive foreground extraction using iterated graph cuts, Proc. SIGGRAPH 2004, (2004)

    Google Scholar 

  21. Li Y., Sun J., Tang C.-K., Shum H.-Y.: Lazy snapping, Proc. SIGGRAPH 2004, (2004)

    Google Scholar 

  22. Wang J., Bhat P., Colburn R. A., Agrawala M., Cohen M. F.: Interactive video cutout, Proc. SIGGRAPH 2005, (2005)

    Google Scholar 

  23. Boykov Y., Kolmogorov V.: An experimental comparison of Min-Cut/Max-Flow algorithms for energy minimization in vision, IEEE Trans. Pattern Analysis and Machine Intelligence, 26(9), 1124–1137 (2004)

    Article  Google Scholar 

  24. Felzenszwalb P. F., Huttenlocher D. P.: Efficient Graph-Based Image Segmentation, International Journal of Computer Vision, 59(2), 167–181 (2004)

    Article  Google Scholar 

  25. Wechsler H., Kidode M.: A random walk procedure for texture discrimination,IEEE Trans. Pattern Analysis and Machine Intelligence, 1(3), 272–280 (1979)

    Article  Google Scholar 

  26. Grady L., Funka-Lea G.: Multi-Label image segmentation for medical applications based on graph-theoretic electrical potentials, Proc. Workshop Computer Vision and Math. Methods in Medical and Biomedical Image Analysis, 230–245 (2004)

    Google Scholar 

  27. Grady L.: Random walks for image segmentation, IEEE Trans. Pattern Analysis and Machine Intelligence, 28(11), 1768–1783 (2006)

    Article  Google Scholar 

  28. Luxburg U. V.: A tutorial on spectral clustering, Statistics and Computing, 17(4), 395-416 (2007)

    Article  MathSciNet  Google Scholar 

  29. Shi J., Malik J.: Normalized cuts and image segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905 (2000)

    Article  Google Scholar 

  30. Ng, A., Jordan, M., Weiss, Y.: (2002). On spectral clustering: analysis and an algorithm, Advances in Neural Information Processing Systems, 14, 849–856, MIT Press.

    Google Scholar 

  31. Fukunaga K., Hostetler L.D.: The estimation of the gradient of a density function, with applications in pattern recognition, IEEE Trans. Information Theory, 21, 32–40 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  32. Fashing M., Tomasi C.: Mean shift is a bound optimization, IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(3), 471–474 (2005)

    Article  Google Scholar 

  33. Cheng Y.: Mean shift, mode seeking, and clustering, IEEE Trans. Pattem Anal. Machine Intell., 17, 790–799 (1995)

    Article  Google Scholar 

  34. Comaniciu D., Meer P. (1997). Robust analysis of feature spaces: color image segmentation, IEEE Conf. Comp. Vis. and Pattern Recogn., Puerto Rico, 750–755.

    Google Scholar 

  35. Comaniciu D., Meer P.: Mean shift: a robust analysis of feature spaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(5), 603–619 (2002)

    Article  Google Scholar 

  36. Li Y. Sun J., Shum H.-Y.: Video object cut and paste. SIGGRAPH 2005, 24, 595–600 (2005)

    Google Scholar 

  37. Mori G., Ren X., Efros A., Malik J.: Recovering Human body configurations: combining segmentation and recognition, IEEE Conf. Comp. Vis. and Pattern Recogn., 2, 326–333 (2004)

    Google Scholar 

  38. Ugarriza L. G., Saber E., Vantaram S. R., Amuso V., Shaw M., Bhaskar R.: Automatic image segmentation by dynamic region growth and multiresolution merging, IEEE Trans. Image Processing, 18(10), 2275–2288 (2009)

    Article  Google Scholar 

  39. Yu Q., Clausi D. A.: IRGS: image segmentation using edge penalties and region growing, IEEE Trans. Pattern Analysis and Machine Intelligence, 30(12), 2126–2139, (2008)

    Article  Google Scholar 

  40. Gould S., Fulton R., Koller D.: Decomposing a scene into geometric and semantically consistent regions, Intl Conf. Computer Vision (ICCV), Kyoto, Japan, 2009.

    Google Scholar 

  41. Ma Y., Derksen H., Hong W., Wright J.: Segmentation of multivariate mixed data via lossy coding and compression, IEEE Trans. Pattern Analysis and Machine Intelligence, 29(9), 1546–1562 (2007)

    Article  Google Scholar 

  42. Blei D. M., Ng A. Y., Jordan M. I.: Latent dirichlet allocation, Journal of Machine Learning Research, 3:993–1022 (2003)

    Article  MATH  Google Scholar 

  43. Sivic J., Russell B. C., Efros A. A., Zisserman A., Freeman W. T.: Discovering object categories in image collections. Intl Conf. Computer Vision (ICCV), 2005.

    Google Scholar 

  44. Lowe D.: Object recognition from local scale-invariant features. In Proc. ICCV, 1150–1157 (1999)

    Google Scholar 

  45. Cao L., Fei-Fei L.: Spatially coherent latent topic model for concurrent object segmentation and classification, In Proc. IEEE Intern. Conf. in Computer Vision (ICCV). 2007.

    Google Scholar 

  46. Wang X. Grimson E.: Spatial Latent Dirichlet Allocation, in Proceedings of Neural Information Processing Systems Conference (NIPS), 2007

    Google Scholar 

  47. Itti L., Koch C., Niebur E.: “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  48. Han J., Ngan K. N., Li M., Zhang H.-J.: Unsupervised extraction of visual attention objects in color images, IEEE Trans. Circuits and Systems for Video Technology, 16(1), 141–145 (2006)

    Article  Google Scholar 

  49. Achanta R., Hemami S. S., Estrada F. J., Ssstrunk S.: Frequency-tuned salient region detection, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009.

    Google Scholar 

  50. Hou X., Zhang L.: Saliency detection: a spectral residual approach, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009.

    Google Scholar 

  51. Wang W., Wang Y., Huang Q., Gao W.: Measuring visual saliency by site entropy rate, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

    Google Scholar 

  52. Goferman S., Zelnik-Manor L.: Context-aware saliency detection, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

    Google Scholar 

  53. Li H., Ngan K. N.: Saliency model based face segmentation in head-and-shoulder video sequences, Journal of Visual Communication and Image Representation, Elsevier Science, 19(5), 320–333 (2008)

    Google Scholar 

  54. Li H., Ngan K. N.: Unsupervised Video Segmentation with Low Depth of Field, IEEE Transactions on Circuits and Systems for Video Technology, 17(12), 1742–1751 (2007)

    Article  Google Scholar 

  55. Rother C., Minka T., Blake A., and Kolmogorov V.: Cosegmentation of image pairs by histogram matching-incorporating a global constraint into MRFs, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, pp. 993–1000, 2006.

    Google Scholar 

  56. Mukherjee L., Singh V., and Dyer C. R.: Half-integrality based algorithms for cosegmentation of images, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009.

    Google Scholar 

  57. Hochbaum D. S., and Singh V., An efficient algorithm for co-segmentation, IEEE International Confernce on Computer Vision (ICCV), 2009.

    Google Scholar 

  58. Joulin A., Bach F., and Ponce J.: Discriminative clustering for image co-segmentation, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

    Google Scholar 

  59. Vicente S., Kolmogorov V., and Rother C., Cosegmentation revisited: models and optimization, European Conference on Computer Vision (ECCV), 2010.

    Google Scholar 

  60. Borenstein E., Ullman S.: Class-specific, top-down segmentation, European Conference on Computer Vision (ECCV), 2002.

    Google Scholar 

  61. Lee Y. J., Grauman K.: Collect-Cut: segmentation with top-down cues discovered in multi-object images, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

    Google Scholar 

  62. Li H., Ngan K. N.: FaceSeg: Automatic Face Segmentation for Real-Time Video, IEEE Transactions on Multimedia, 11(1), 77–88 (2009)

    Article  Google Scholar 

  63. Unnikrishnan R., and Hebert M., Measures of similarity, IEEE Workshop on Computer Vision Applications, pp. 394–400, 2005.

    Google Scholar 

  64. Martin D., Fowlkes C., Tal D., and Malik J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in Proc. Intl Conf. Computer Vision (ICCV), vol. 2, pp. 416–423, 2001.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic Engineering, University of Electronic Science and Technology of China, Chengdu, China

    Hongliang Li

Authors
  1. Hongliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. King Ngi Ngan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongliang Li .

Editor information

Editors and Affiliations

  1. , Department of Electronic Engineering, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR, China, People's Republic

    King Ngi Ngan

  2. Technology of China, School of Electronic Engineering, University of Electronic Science &, Chengdu, 610054, China, People's Republic

    Hongliang Li

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer Science+Business Media, LLC

About this chapter

Cite this chapter

Li, H., Ngan, K.N. (2011). Image/Video Segmentation: Current Status, Trends, and Challenges. In: Ngan, K., Li, H. (eds) Video Segmentation and Its Applications. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-9482-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-1-4419-9482-0_1

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4419-9481-3

  • Online ISBN: 978-1-4419-9482-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation