Skip to main content
SpringerLink
Log in

Optimizable face detection and tracking model with occlusion resolution for high quality videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recent state-of-the-art Face Detection algorithms in the field of Computer Vision focus greatly on real-time processing and results. The applications using these algorithms deal with low quality video feeds having less Pixels Per Inch (ppi) and/or low frame rate. The algorithms perform well with such video feeds, but their performance deteriorates towards high quality, high data-per-frame videos. Such video files mostly exist in offline mode, that could be used for post processing by the Computer Vision applications. This paper focuses on developing such an algorithm that gives faster results on high quality videos, at par with the algorithms working on live low quality video feeds. The proposed algorithm uses Convolutional-MTCNN as base algorithm, and speeds it up for high definition videos. This paper also presents a novel solution to the problem of occlusion and detecting partial or fully hidden faces in the videos. This is achieved by using probabilistic approaches, given that the face has been identified in first few frames, to give the algorithm an estimate of where the face should be in the occluded region.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. In: IEEE conference on computer vision and pattern recognition, pp 2887–2894

  2. Chrysos G G, Antonakos E, Zafeiriou S, Snape P (2015) Offline deformable face tracking in arbitrary videos. In: IEEE international conference on computer vision workshop (ICCVW), pp 954–962

  3. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: IEEE Computer society conference on computer vision and pattern recognition (CVPR’05), vol 1, pp 886–893

  4. Ismail N, Sabri MIM (2009) Review of existing algorithms for face detection and recognition. In: World scientific and engineering academy and society (WSEAS), pp 30–39

  5. Jia Y-B (2017) Polynomial interpolation. National Taiwan Ocean University Pub Scientific Computing

  6. Li J, Song L, Liu C (2018) The cubic trigonometric automatic interpolation spline. IEEE/CAA J Autom Sin 5(6):1136–1141

    Article  MathSciNet  Google Scholar 

  7. Luo J, Liu J, Lin J, Wang Z (2020) A lightweight face detector by integrating the convolutional neural network with the image pyramid. Pattern Recogn Lett 133:180–187. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0167865520300805

    Article  Google Scholar 

  8. Meijering E (2002) A chronology of interpolation: from ancient astronomy to modern signal and image processing. Proc IEEE 90(3):319–342

    Article  Google Scholar 

  9. Nguyen D T, Nguyen T N, Kim H, Lee H (2019) A high-throughput and power-efficient fpga implementation of yolo cnn for object detection. IEEE Trans Very Large Scale Integr (VLSI) Syst 27(8):1861–1873

    Article  Google Scholar 

  10. Pairo W, Loncomilla P, Ruiz-del Solar J (2019) A delay-free and robust object tracking approach for robotics applications. J Intell Robot Syst 95:07

    Article  Google Scholar 

  11. Raja R, Sinha D T, Dubey R (2015) Recognition of human-face from side-view using progressive switching pattern and soft-computing technique. Adv Model Anal B 58:14–34, 01

    Google Scholar 

  12. Raja R, Sinha T S, Patra R K, Tiwari S (2018) Physiological trait-based biometrical authentication of human-face using lgxp and ann techniques. Int J Inf Comput Secur 10(2–3):303–320. [Online]. Available: https://www.inderscienceonline.com/doi/abs/10.1504/IJICS.2018.091468

    Google Scholar 

  13. Ranftl A, Alonso-Fernandez F, Karlsson S, Bigun J (2017) A real-time adaboost cascade face tracker based on likelihood map and optical flow. IET Biom 6:05

    Article  Google Scholar 

  14. Ranjan R, Patel V M, Chellappa R (2019) Hyperface: a deep multi-task learning framework for face detection, landmark localization, pose estimation, and gender recognition. IEEE Trans Pattern Anal Mach Intell 41(1):121–135

    Article  Google Scholar 

  15. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: 2015 IEEE Conference on computer vision and pattern recognition (CVPR). [Online]. Available: https://doi.org/10.1109/CVPR.2015.7298682

  16. Shen J, Zafeiriou S, Chrysos G G, Kossaifi J, Tzimiropoulos G, Pantic M (2015) The first facial landmark tracking in-the-wild challenge: Benchmark and results. In: IEEE international conference on computer vision workshop (ICCVW), pp 1003–1011

  17. Singh S, Singh D, Yadav V (2020) Face recognition using hog feature extraction and svm classifier. 8:6437–6440, 09

  18. Tomasi C, Kanade T (1991) Detection and tracking of point features. Int J Comput Vis

  19. Tzimiropoulos G (2015) Project-out cascaded regression with an application to face alignment. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 3659–3667

  20. Viola P, Jones M (2004) Robust real-time face detection. Int J Comput Vis 57:137–154, 05

    Article  Google Scholar 

  21. Yang M -H, Kriegman D J, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mac Intell 24(1):34–58

    Article  Google Scholar 

  22. Yang B, Yan J, Lei Z, Li S Z (2014) Aggregate channel features for multi-view face detection. In: IEEE International joint conference on biometrics, pp 1–8

  23. Yu B, Tao D (2019) Anchor cascade for efficient face detection. IEEE Trans Image Process 28(5):2490–2501

    Article  MathSciNet  Google Scholar 

  24. Zeng D, Zhao F, Ge S, Shen W (2019) Fast cascade face detection with pyramid network. Pattern Recogn Lett 119:180–186. [Online]. Available: http://www.sciencedirect.com/science/article/pii/S0167865518302125

    Article  Google Scholar 

  25. Zhang K, Zhang Z, Li Z, Qiao Y (2016) Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process Lett 23(10):1499–1503

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Technology, Delhi Technological University, Shahbad Daulatpur, Main Bawana Road, Delhi, 110042, India

    Akshay Mool & Kapil Sharma

  2. Department of Electronics and Communications Engineering, Delhi Technological University, Shahbad Daulatpur, Main Bawana Road, Delhi, 110042, India

    J. Panda

Authors
  1. Akshay Mool
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. Panda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kapil Sharma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Akshay Mool.

Ethics declarations

Conflict of Interest

Akshay Mool, J. Panda and Kapil Sharma declare that they have no conflict of interest.

Additional information

Competing interest

The authors declare that they do not have any competing financial interests nor any personal relationships that could seem to have influenced the work presented by this paper.

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mool, A., Panda, J. & Sharma, K. Optimizable face detection and tracking model with occlusion resolution for high quality videos. Multimed Tools Appl 81, 10391–10406 (2022). https://doi.org/10.1007/s11042-022-11958-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11958-5

Keywords

Search

Navigation