Skip to main content
SpringerLink
Log in

Skin detection in video under uncontrolled illumination

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Many vision-based human-computer interaction (HCI) applications require skin detection. However, their performance relies on accuracy in detecting skin regions in video, which is difficult under uncontrolled illumination. The chromatic appearance of skin changes because of shading, often caused by body movement. To address this, we propose a dynamic adaptation method to detect skin regions affected by local color deformations. Static and dynamic skin regions are detected by a corresponding module. The static module includes a facial skin distribution model (FSDM) and a fusion-based background distribution model (FBDM). The FBDM is obtained from a local background distribution model (LBDM) and a global background distribution model (GBDM). The LBDM is obtained by comparing a frame pixel distribution model with the FSDM and GBDM. Next, the FBDM is derived from the LBDM and the GBDM. The dynamic module includes a moving skin distribution model (MSDM), derived from a set of moving skin samples. Initially, moving skin regions are detected using a modified double frame-difference method and then modeled using a Gaussian mixture model. To avoid misidentifying background regions as skin, the final MSDM is obtained by comparing the initial moving skin model to the FSDM and FBDM. Finally, the static and the dynamic models are fused by applying a maximization rule. Experimental results shows that the proposed method can detect skin regions more accurately than state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Avola D, Bernardi M, Cinque L, Foresti G L, Massaroni C (2019) Exploiting recurrent neural networks and leap motion controller for the recognition of sign language and semaphoric hand gestures. IEEE Trans Multimed 21 (1):234–245

    Article  Google Scholar 

  2. Awad G, Han J, Sutherland A (2006) A unified system for segmentation and tracking of face and hands in sign language recognition. In: Proc. Int. Conf. Pattern Recogn., vol 1, pp 239–242

  3. Bianco S, Gasparini F, Schettini R (2015) Adaptive skin classification using face and body detection. IEEE Trans Image Process 24(12):4756–4765

    Article  MathSciNet  Google Scholar 

  4. Carrara F, Elias P, Sedmidubsky J (2019) Lstm-based real-time action detection and prediction in human motion streams. Multimed Tools Appl 78:27309–27331

    Article  Google Scholar 

  5. Chakraborty B K, Bhuyan M (2020) Image specific discriminative feature extraction for skin segmentation. Multimed Tools Appl:1573–7721

  6. Chakraborty B K, Bhuyan MK, Kumar S (2017) Combining image and global pixel distribution model for skin colour segmentation. Pattern Recogn Lett 88:33–40

    Article  Google Scholar 

  7. Chung JK, Kannappan PL, Ng CT, Sahoo PK (1989) Measures of distance between probability distributions. J Math Anal Appl 138(1):280–292

    Article  MathSciNet  Google Scholar 

  8. Chyad M A, Alsattar H A, Zaidan B B, Zaidan A A, Al Shafeey G A (2019) The landscape of research on skin detectors: Coherent taxonomy, open challenges, motivations, recommendations and statistical analysis, future directions. IEEE Access 7:106536–106575

    Article  Google Scholar 

  9. Danyang C, Menggui Z, Yanchao Z (2018) Research on a facial feature points detection method based on skin color model. In: 2018 Chinese Control And Decision Conference (CCDC), pp 5688–5693

  10. Fouad R M, Omer O A, Aly M H (2019) Optimizing remote photoplethysmography using adaptive skin segmentation for real-time heart rate monitoring. IEEE Access 7:76513–76528

    Article  Google Scholar 

  11. George Y, Aldeen M, Garnavi R (2017) A pixel-based skin segmentation in psoriasis images using committee of machine learning classifiers. In: Int. Conf. Digit. Image Comput.: Tech. and Appl. (DICTA), pp 1–8

  12. Habili N, Lim C-C, Moini A (2004) Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Trans Circ Syst Video Technol 14(8):1086–1097

    Article  Google Scholar 

  13. Han J, Awad G, Sutherland A (2009) Automatic skin segmentation and tracking in sign language recognition. IET Comput Vis 3(1):24–35

    Article  Google Scholar 

  14. Hettiarachchi R, Peters JF (2016) Multi-manifold-based skin classifier on feature space voronoï regions for skin segmentation. J Vis Commun Image R 41:123–139

    Article  Google Scholar 

  15. Jones M J, Rehg J (2002) Statistical color models with application to skin detection. Int J Comput Vis 46(1):81–96

    Article  Google Scholar 

  16. Kakkoth S S, Gharge S (2018) Real time hand gesture recognition its applications in assistive technologies for disabled. In: Int. Conf. Comput. Comm. Control and Auto. (ICCUBEA), pp 1–6

  17. Kameda Y, Michihiko M (1996) A human motion estimation method using 3-successive video frames. In: Proc. Conf. on Virtual Systems and Multimedia, pp 135–140. http://ci.nii.ac.jp/naid/10024346616/en/

  18. Kawulok M (2013) Fast propagation-based skin regions segmentation in color images. In: Proc. 10th IEEE Int. Conf. and Workshops Autom. Face and Gesture Recogn. (FG), pp 1–7

  19. Kawulok M, Kawulok J, Nalepa J, Papiez M (2013) Skin detection using spatial analysis with adaptive seed. In: Proc. IEEE ICIP, pp 3720–3724

  20. Kawulok M, Kawulok J, Nalepa J (2014) Spatial-based skin detection using discriminative skin-presence features. Pattern Recogn Lett 41:3–13

    Article  Google Scholar 

  21. Lei Q, Zhang H, Xia Z, Yang Y, He Y, Liu S (2019) Applications of hand gestures recognition in industrial robots: a review . In: Verikas A, Nikolaev D P, Radeva P, Zhou J (eds) Int. Conf. Machine Vis. (ICMV 2018), vol 11041. International Society for Optics and Photonics, SPIE, pp 455–465

  22. Lei Y, Yuan W, Wang H, Wenhu Y, Bo W (2017) A skin segmentation algorithm based on stacked autoencoders. IEEE Trans Multimed 19 (4):740–749

    Article  Google Scholar 

  23. Liu L, Sang N, Yang S, Huang R (2011) Real-time skin color detection under rapidly changing illumination conditions. IEEE Trans Consum Electron 57(3):1295–1302

    Article  Google Scholar 

  24. McBride T J, Vandayar N, Nixon K J (2019) A comparison of skin detection algorithms for hand gesture recognition. In: 2019 Southern African Universities Power Engineering Conference/Robotics and Mechatronics/Pattern Recognition Association of South Africa (SAUPEC/RobMech/PRASA), pp 211–216

  25. Mo T, Sun P (2019) Research on key issues of gesture recognition for artificial intelligence. Soft Comput 24:5795–5803

    Article  Google Scholar 

  26. Moreira D C, Fechine J M (2018) A machine learning-based forensic discriminator of pornographic and bikini images. In: 2018 Int. Joint Conf. Neural Net. (IJCNN), pp 1–8

  27. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66

    Article  MathSciNet  Google Scholar 

  28. Pennisi A, Bloisi D D, Nardi D, Giampetruzzi A R, Mondino C, Facchiano A (2016) Skin lesion image segmentation using delaunay triangulation for melanoma detection. Comput Med Imag Graph 52:89–103

    Article  Google Scholar 

  29. Phung SL, Bouzerdoum A, Chai S (2005) Skin segmentation using color pixel classification: Analysis and comparison. IEEE Trans Pattern Anal Mach Intell 27(1):148–154

    Article  Google Scholar 

  30. Ramasinghe S, Rajasegaran J, Jayasundara V, Ranasinghe K, Rodrigo R, Pasqual A A (2019) Combined static and motion features for deep-networks-based activity recognition in videos. IEEE Trans Circ Syst Vid Tech 29(9):2693–2707

    Article  Google Scholar 

  31. Rautaray S, Agrawal A (2012) Vision based hand gesture recognition for human computer interaction: a survey. Artificial Intell Rev:1–54

  32. Sandbach G, Zafeiriou S, Pantic M, Yin L (2012) Static and dynamic 3d facial expression recognition: A comprehensive survey. Image Vis Comput 30(10):683–697

    Article  Google Scholar 

  33. SanMiguel J C, Suja S (2013) Skin detection by dual maximization of detectors agreement for video monitoring. Pattern Recogn Lett 34(16):2102–2109

    Article  Google Scholar 

  34. Sigal L, Sclaroff S, Athitsos V (2004) Skin color-based video segmentation under time-varying illumination. IEEE Trans Pattern Anal Mach Intell 26 (7):862–877

    Article  Google Scholar 

  35. Soriano M, Martinkauppi B, Huovinen S, Laaksonen M (2003) Adaptive skin color modeling using the skin locus for selecting training pixels. Pattern Recogn 36(3):681–690

    Article  Google Scholar 

  36. Tang C, Lu J, Liu J (2018) Non-contact heart rate monitoring by combining convolutional neural network skin detection and remote photoplethysmography via a low-cost camera. In: 2018 IEEE/CVF Conf. Comput. Vis. Pattern Recogn. Work. (CVPRW), pp 1390–13906

  37. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: Proc. IEEE Comp. Society Conf. Comp. Vis. and Pattern Recognition (CVPR), vol 1, pp 511–518

  38. Wachs J P, Kölsch M, Stern H, Edan Y (February 2011) Vision-based hand-gesture applications. Commun ACM 54(2):6071

    Article  Google Scholar 

  39. Yang J, Wang Y, Lv Z, Jiang N, Steed A (2018) Interaction with three-dimensional gesture and character input in virtual reality: Recognizing gestures in different directions and improving user input. IEEE Consum Electron Mag 7(2):64–72

    Article  Google Scholar 

  40. Zuo H, Fan H, Blasch E, Ling H (2017) Combining convolutional and recurrent neural networks for human skin detection. IEEE Signal Process Let 24(3):289–293

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Sankhyasutra Labs, Karnataka, India

    Biplab Ketan Chakraborty

  2. Department of Electronics and Electrical Engineering, IIT Guwahati, Guwahati, 781039, India

    M. K. Bhuyan

  3. Indiana University School of Informatics and Computing, 535 West Michigan St., Indianapolis, IN, 46202, USA

    Karl F. MacDorman

Authors
  1. Biplab Ketan Chakraborty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. K. Bhuyan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karl F. MacDorman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. K. Bhuyan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chakraborty, B.K., Bhuyan, M.K. & MacDorman, K.F. Skin detection in video under uncontrolled illumination. Multimed Tools Appl 80, 24319–24341 (2021). https://doi.org/10.1007/s11042-021-10728-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10728-z

Keywords

Search

Navigation