Multiple features fusion based video face tracking

  • Tianping Li
  • Pingping Zhou
  • Hui LiuEmail author


With the development of monitoring equipment and artificial intelligence technology, video face tracking under the big data background has become an important research hot spot in the field of public security. In order to track robustly under the circumstances of illumination variation, background clutter, fast motion, partial occlusion and so on, this paper proposed an algorithm combining a multi-feature fusion in the frame of particle filter and an improved mechanism, which consists of three main steps. At first, the color and edge features of human face were extracted from the video sequence. Meanwhile, color histograms and edge orientation histograms (EOH) were used to describe the facial features and beneficial to improve the efficiency of calculation. Then we employed a self-adaptive features fusion strategy to calculate the particle weight, which can effectively enhance the reliability of face tracking. Moreover, in order to solve the computational efficiency problem caused by too many particles, we added the integral histogram method to simplify the calculation complexity. At last, the object model was updated between the current object model and the initial model for alleviating the model drifts. Experiments conducted on testing dataset show that this proposed approach can robustly track single face with the cases of complex backgrounds, such as similar skin color, illumination change and occlusion, and perform better than color-based and edge-based methods in terms of both quantitative metrics and visual quality.


Video face tracking Particle filter (PF) Features fusion Updating model Template drift 



This work was supported in part by NSFC (61572286 and 61472220), NSFC Joint with Zhejiang Integration of Informatization and Industrializaiton under Key Project (U1609218), and the Fostering Project of Dominant Discipline a Talent Team of Shandong Province Higher Education.


  1. 1.
    Arulampalam MS, Maskell S, Gordon N, Clapp T (2002) A tutorial on particle filters for on-line non-linear/non-Gaussian Bayesian tracking. IEEE Trans Sign Proc 50(2):174–188CrossRefGoogle Scholar
  2. 2.
    Bos R, De Waele S, Broersen PMT (2002) Autoregressive spectral estimation by application of the burg algorithm to irregularly sampled data. IEEE Trans Instrum Meas 51(6):1289–1294CrossRefGoogle Scholar
  3. 3.
    Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Patt Anal Mach Intel 25(5):564–575CrossRefGoogle Scholar
  4. 4.
    Dou J, Li J, Zhang Z, Han S (2012) Face tracking with an adaptive Adaboost-based particle filter. 24th IEEE Chin Control Dec Conf (CCDC):3626–3631Google Scholar
  5. 5.
    Gustafsson F, Gunnarsson F, Bergman N, Forssell U, Jansson J, Karlsson R, Nordlund PJ (2002) Particle filters for positioning, navigation, and tracking. IEEE Trans Signal Process 50(2):425–437CrossRefGoogle Scholar
  6. 6.
    Jiang X, Yu H, Lu Y, Liu H (2016) A fusion method for robust face tracking. Multimed Tools Appl 75(19):11801–11813CrossRefGoogle Scholar
  7. 7.
    Kailath T (1967) The Divergence and Bhattacharyya Distance Measures in Signal Selection. IEEE Trans Commun Technol 15(1):52–60CrossRefGoogle Scholar
  8. 8.
    Lee J, Park G-L, Kwak H-Y (2011) Two-directional two-dimensional random projection and its variations for face and palmprint recognition. ICCSA. Int Conf 32(10):707–727Google Scholar
  9. 9.
    Leng L, Li M, Leng L, Teoh ABJ (2013) Conjugate 2DPalmHash code for secure palm-print-vein verification. 2013 6th Int Cong Image Sign Proc (CISP) 70(1):495–523Google Scholar
  10. 10.
    Leng L, Li M, Kim C, Bi X (2015) Dual-source discrimination power analysis for multi-instance contactlesLeng, L., Li, M., Leng, L., & Teoh, A. B. J. (2013). Conjugate 2DPalmHash code for secure palm-print-vein verification. 2013 6th Int Cong Image Sign Proc (CISP) 70(1):495–523Google Scholar
  11. 11.
    Li X, Dick A, Shen C, Zhang Z, Hengel AVD, Wang H (2013) Visual tracking with spatio-temporal Dempster-Shafer information fusion. IEEE Trans Image Proc 22(8):3028–3040MathSciNetCrossRefGoogle Scholar
  12. 12.
    Li Y, Wang G, Nie L, Wang Q (2018) Distance metric optimization driven convolutional neural network for age invariant face recognition. Pattern Recogn 75:51–62CrossRefGoogle Scholar
  13. 13.
    Lin Y, Shen J, Cheng S, Pantic M (2018) Mobile Face Tracking: A Survey and Benchmark. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), arXiv:1805.09749Google Scholar
  14. 14.
    Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing 181:108–115CrossRefGoogle Scholar
  15. 15.
    Liu Y, Zheng Y, Liang Y, Liu S, David S (2016) UrbanWater quality prediction based on multi-task multi-view. Learning. 34(4):44–48Google Scholar
  16. 16.
    Lu L, Zhang J, Xu J, Khan MK, Alghathbar K (2010) Dynamic weighted discrimination power analysis in DCT domain for face and palmprint recognition. 2010. Int Conf Inform Commun Technol Conver (ICTC) 34(4):44–48Google Scholar
  17. 17.
    Sadeghian A, Alahi A, Savarese S (2017) Tracking The Untrackable: Learning To Track Multiple Cues with Long-Term Dependencies. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), arXiv:1701.01909Google Scholar
  18. 18.
    SOBEL I (1990) An Isotropic 3×3 Gradient Operator, Machine Vision for Three – Dimensional Scenes, Freeman, H., Academic Press, 376–379Google Scholar
  19. 19.
    Swain MJ, Ballard DH (1991) Color indexing. Int J Comput Vis 7(1):11–32CrossRefGoogle Scholar
  20. 20.
  21. 21.
    Wang J, Yagi Y (2008) Integrating color and shape-texture features for adaptive real-time object tracking. IEEE Trans Image Process 17(2):235–240CrossRefGoogle Scholar
  22. 22.
    Wang J, Jiang YX, Tang CH (2012) Face tracking based on particle filter using color histogram and contour distributions. Poto-Electron Eng 39(10):32–39Google Scholar
  23. 23.
    Wang L, Yan H, Lv K, Pan C (2014) Visual Tracking Via Kernel Sparse Representation With Multikernel Fusion. IEEE Trans Circ Syst Video Technol 24(7):1132–1141CrossRefGoogle Scholar
  24. 24.
    Welch G, Bishop G (2017) An introduction to the Kalman filter. Univ North Carol Chapel Hill 8(7):127–132Google Scholar
  25. 25.
    Yang D, Zhang Y, Ji R, Li Y, Huangfu L, Yang Y (2015) An improved spatial histogram and particle filter face tracking. Genetic and evolutionary computing, advances in intelligent systems. Computing 329:257–267Google Scholar
  26. 26.
    Yoon JH, Yang MH, Lim J, Yoon KJ (2015) Bayesian Multi-object Tracking Using Motion Context from Multiple Objects. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp.33–40Google Scholar
  27. 27.
    Zhu W, Levinson S (2000) Edge Orientation-Based Multi-View Object Recognition. Proc 15th Int Conf Patt Recog (ICPR):936–939

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Shandong Province Key Laboratory of Medical Physics and Image Processing TechnologyJinanChina
  2. 2.Department of Physics and ElectronicsShandong Normal UniversityJinanChina
  3. 3.Yancheng Biological Engineering Higher Vocational Technology SchoolYanchengChina
  4. 4.Department of Computer Science & TechnologyShandong University of Finance and EconomicsJinanChina
  5. 5.Medical Physics Division in the Department of Radiation OncologyStanford UniversityPalo AltoUSA

Personalised recommendations