Multimedia Tools and Applications

, Volume 69, Issue 3, pp 717–741 | Cite as

Systematic skin segmentation: merging spatial and non-spatial data

  • Rehanullah Khan
  • Allan Hanbury
  • Robert Sablatnig
  • Julian Stöttinger
  • F. Ali Khan
  • F. Alam Khan
Article

Abstract

Skin detection is used in applications ranging from face detection, tracking of body parts, hand gesture analysis, to retrieval and blocking objectionable content. We present a systematic approach for robust skin segmentation using graph cuts. The skin segmentation process starts by exploiting the local skin information of detected faces. The detected faces are used as foreground seeds for calculating the foreground weights of the graph. If local skin information is not available, we opt for the universal seed. To increase the robustness, the decision tree based classifier is used to augment the universal seed weights when no local information is available in the image. With this setup, we achieve robust skin segmentation, outperforming off-line trained classifiers. The setup also provides a generic skin detection system, using positive training data only. With face detection, we take advantage of the contextual information present in the scene. With the weight augmentation, we provide a setup for merging spatial and non-spatial data. Experiments on two datasets with annotated pixel-level ground truth show that the systematic skin segmentation approach outperforms other approaches and provides robust skin detection.

Keywords

Skin detection Skin segmentation Classification Graph cuts for skin detection Neighborhood relationship Systematic skin detection approach Classifiers integration 

References

  1. 1.
    Albiol A, Torres L, Delp EJ (2001) Optimum color spaces for skin detection. In: Proceedings of the ICIP, pp 122–124Google Scholar
  2. 2.
    Argyros AA, Lourakis MIA (2004) Real-time tracking of multiple skin-colored objects with a possibly moving camera. In: ECCV, pp 368–379Google Scholar
  3. 3.
    Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans Pattern Anal Mach Intell (TPAMI) 26(9):1124–1137CrossRefGoogle Scholar
  4. 4.
    Boykov YY, Jolly MP (2001) Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: ICCV 2001, vol 1, pp 105–112Google Scholar
  5. 5.
    Brown D, Craw I, Lewthwaite J (2001) A SOM based approach to skin detection with application in real time systems. In: BMVC’01, pp 491–500Google Scholar
  6. 6.
    Cai J, Goshtasby A (1999) Detecting human faces in color images. Image Vis Comput 18:63–75CrossRefGoogle Scholar
  7. 7.
    Cao L-L, Li X-L, Yu N-H, Liu Z-K (2002) Naked people retrieval based on adaboost learning. In: International conference on machine learning and cybernetics, pp 1133–1138Google Scholar
  8. 8.
    Chai D, Ngan KN (1998) Locating facial region of a head-and-shoulders color image. In: Int. conf. automatic face and gesture recognition, pp 124–129Google Scholar
  9. 9.
    Fleck MM, Forsyth DA, Bregler C (1996) Finding naked people. In: ECCV, pp 593–602Google Scholar
  10. 10.
    Fowlkes C, Martin D, Malik J (2003) Learning affinity functions for image segmentation: combining patch-based and gradient-based approaches. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, vol 2, pp 54–64Google Scholar
  11. 11.
    Fu Z, Yang J, Hu W, Tan T (2004) Mixture clustering using multidimensional histograms for skin detection. In: ICPR. Washington, DC, USA, pp 549–552Google Scholar
  12. 12.
    Garcia C, Tziritas G (1999) Face detection using quantized skin color regions merging and wavelet packet analysis. IEEE Trans Multimedia 1(3):264–27CrossRefGoogle Scholar
  13. 13.
    Hanbury A (2008) Constructing cylindrical coordinate colour spaces. Pattern Recognition (PR) Letters 29(4):495–500Google Scholar
  14. 14.
    Hsu RL, Abdel-Mottaleb M, Jain AK (2002) Face detection in color images. IEEE Trans Pattern Anal Mach Intell (TPAMI) 24:696–706CrossRefGoogle Scholar
  15. 15.
    Hu Z, Wang G, Lin X, Yan H (2009) Skin segmentation based on graph cuts. Sci Technol 14(4):478–486Google Scholar
  16. 16.
    Jones MJ, Rehg JM (2002) Statistical color models with application to skin detection. Int J Comput Vis (IJCV) 46(1):81–96CrossRefMATHGoogle Scholar
  17. 17.
    Kakumanu P, Makrogiannis S, Bourbakis N (2007) A survey of skin-color modeling and detection methods. Pattern Recognition (PR) Journal 40(3):1106–1122CrossRefMATHGoogle Scholar
  18. 18.
    Khan R, Hanbury A, Stoettinger J (2010) Skin detection: a random forest approach. In: ICIP, pp 4613–4616Google Scholar
  19. 19.
    Khan R, Hanbury A, Stöttinger J (2010) Universal Seed Skin Segmentation. ISVC 2:75–84Google Scholar
  20. 20.
    Khan R, Hanbury A, Stöttinger J, Bais A (2012) Color based skin classification. Pattern Recogn Lett 33(2):157–163CrossRefGoogle Scholar
  21. 21.
    Khan R, Stöttinger J, Kampel M (2008) An adaptive multiple model approach for fast content-based skin detection in on-line videos. In: ACM MM, AREA workshop, pp 89–96Google Scholar
  22. 22.
    Kuo YM, Lee J-S, Chung P-C (2007) The naked image detection based on automatic white balance method. In: 2006 ICS International Computer Conference, pp 990–994Google Scholar
  23. 23.
    Lee JY, Yoo S (2002) An elliptical boundary model for skin color detection. In: ISST, pp 579–584Google Scholar
  24. 24.
    Lee J-S, Kuo Y-M, Chung P-C, Chen E-L (2007 ) Naked image detection based on adaptive and extensible skin color model. Pattern Recognition (PR) Journal 40(8):2261–2270CrossRefMATHGoogle Scholar
  25. 25.
    Li Y, Sun J, Tang CK, Shum HY (2004) Lazy snapping. In: SIGGRAPH. New York, NY, USA, pp 303–308Google Scholar
  26. 26.
    Liensberger C, Stöttinger J, Kampel M (2009) Color-based and context-aware skin detection for online video annotation. In: MMSP, pp 1–6Google Scholar
  27. 27.
    Malik J, Belongie S, Leung TK, Shi J (2001) Contour and texture analysis for image segmentation. Int J Comput Vis 43(1):7–27CrossRefMATHGoogle Scholar
  28. 28.
    Martin DR, Fowlkes CC, Malik J (2004) Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans Pattern Anal Mach Intell (TPAMI) 26(5):530–549CrossRefGoogle Scholar
  29. 29.
    Micusík B, Hanbury A (2005) Steerable semi-automatic segmentation of textured images. In: SCIA, pp 35–44Google Scholar
  30. 30.
    Micusík B, Hanbury A (2005) Supervised texture detection in images. In: CAIP, pp 441–448Google Scholar
  31. 31.
    Micusík B, Hanbury A (2006) Automatic image segmentation by positioning a seed. In: ECCV (2), pp 468–480Google Scholar
  32. 32.
    Pavlovic V (2001) Boosted detection of objects and attributes. In: CVPR, pp 1–8Google Scholar
  33. 33.
    Peer P, Kovac J, Solina F (2003) Human skin colour clustering for face detection. In: EUROCON, vol 2, pp 144–148Google Scholar
  34. 34.
    Phung SL, Bouzerdoum A, Chai D (2005) Skin segmentation using color pixel classification: analysis and comparison. IEEE Trans Pattern Anal Mach Intell (TPAMI) 27(1):148–154CrossRefGoogle Scholar
  35. 35.
    Phung SL, Chai D, Bouzerdoum A (2001) A universal and robust human skin color model using neural networks. In: IJCNN, pp 2844–2849Google Scholar
  36. 36.
    Quinlan RJ (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers IncGoogle Scholar
  37. 37.
    Schettini R, Brambilla C, Cusano C, Ciocca G (2003) On the detection of pornographic digital images. In: SPIE vol 5150, pp 2105–2113Google Scholar
  38. 38.
    Schmugge SJ, Jayaram S, Shin MC, Tsap LV (2007) Objective evaluation of approaches of skin detection using ROC analysis. Comput Vis Image Underst 108(1–2):41–51CrossRefGoogle Scholar
  39. 39.
    Sebe N, Cohen I, Huang TS, Gevers T (2004) Skin detection: a Bayesian network approach. In: ICPR, pp 903–906Google Scholar
  40. 40.
    Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell (TPAMI) 22(8):888–905CrossRefGoogle Scholar
  41. 41.
    Sigal L, Sclaroff S, Athitsos V (2004) Skin color-based video segmentation under time-varying illumination. IEEE Trans Pattern Anal Mach Intell (TPAMI) 26(7):862–877CrossRefGoogle Scholar
  42. 42.
    Stokman H, Gevers T (2005) Selection and fusion of color models for feature detection. In: Proceedings of the CVPR. IEEE Computer Society, Washington, DC, USA, pp 560–565Google Scholar
  43. 43.
    Stokman H, Gevers T (2007) Selection and fusion of color models for image feature detection. IEEE Trans Pattern Anal Mach Intell 29(3):371–381CrossRefGoogle Scholar
  44. 44.
    Störring M, Andersen HJ, Granum E (2000) Estimation of the illuminant colour from human skin colour. In: IEEE international conference on automatic face and gesture recognition, pp 64–69Google Scholar
  45. 45.
    Stöttinger J, Hanbury A, Liensberger C, Khan R (2009) Skin paths for contextual flagging adult videos. In: International symposium on visual computing, pp 303–314Google Scholar
  46. 46.
    Stöttinger J, Uijlings J, Pandey AK, Sebe N, Giunchiglia F (2012) Event recognition via composition. In: CVPR. http://disi.unitn.it/~stottinger/downloads/publications/Stottinger-et-al_Unseen_Event_Recognition_via_Semantic_Compositionality_CVPR2012.pdf. Accessed 1 Mar 2012
  47. 47.
    Terrillon J-C, Akamatsu S (2000) Comparative performance of different chrominance spaces for color segmentation and detection of human faces in complex scene images. In: Proceedings of the 12th conference on vision interface, pp 180–187Google Scholar
  48. 48.
    Vezhnevets V, Sazonov V, Andreev A (2003) A survey on pixel-based skin color detection techniques. In: GraphiCon, pp 85–92Google Scholar
  49. 49.
    Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. CVPR 1:I–511–I–518Google Scholar
  50. 50.
    Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis (IJCV) 57(2):137–154CrossRefGoogle Scholar
  51. 51.
    Wang S, Hui1 H, Li SH, Zhang H, Shi YY, Qu WT (2005 ) Exploring content-based and image-based features for nude image detection. In: Fuzzy systems and knowledge discovery, pp 324–328Google Scholar
  52. 52.
    Wong KW, Lam KM, Siu WC (2003) A robust scheme for live detection of human faces in color images. Signal Process, Image Commun 18(2):103–114CrossRefGoogle Scholar
  53. 53.
    Yang J, Lu W, Waibel A (1997) Skin-color modeling and adaptation. In: ACCV, pp 687–694Google Scholar
  54. 54.
    Yang M, Ahuja N (1999) Gaussian mixture model for human skin color and its application in image and video databases. In: SPIE, pp 458–466Google Scholar
  55. 55.
    Zabih R, Kolmogorov V (2004) Spatially coherent clustering using graph cuts. In: CVPR (2), pp 437–444Google Scholar
  56. 56.
    Zarit BD, Super BJ, Quek FKH (1999) Comparison of five color models in skin pixel classification. In: RATFG-RTS ’99: proceedings of the international workshop on recognition, analysis, and tracking of faces and gestures in real-time systems. IEEE Computer Society, Washington, DC, USA, pp 58–63Google Scholar
  57. 57.
    Zheng Q-F, Zeng W, Wen G, Wang W-Q (2004) Shape-based adult image detection. In: ICIG ’04: proceedings of the third international conference on image and graphics. IEEE Computer Society, Washington, DC, USA, pp 150–153CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Rehanullah Khan
    • 1
  • Allan Hanbury
    • 2
  • Robert Sablatnig
    • 3
  • Julian Stöttinger
    • 4
  • F. Ali Khan
    • 2
    • 5
  • F. Alam Khan
    • 6
  1. 1.Sarhad University of Science and Information TechnologyPeshawarPakistan
  2. 2.Institute of Software Technology and Interactive SystemsTU-WienViennaAustria
  3. 3.Institute of Computer Aided AutomationTU-WienViennaAustria
  4. 4.Department of Information Engineering and Computer ScienceUniversity of TrentoTrentoItaly
  5. 5.Institute of Management SciencesPeshawarPakistan
  6. 6.COMSATS Institute of Information TechnologyIslamabadPakistan

Personalised recommendations