Selective multi-descriptor fusion for face identification

  • Xin WeiEmail author
  • Hui Wang
  • Bryan Scotney
  • Huan Wan
Original Article


Over the last 2 decades, face identification has been an active field of research in computer vision. As an important class of image representation methods for face identification, fused descriptor-based methods are known to lack sufficient discriminant information, especially when compared with deep learning-based methods. This paper presents a new face representation method, multi-descriptor fusion (MDF), which represents face images through a combination of multiple descriptors, resulting in hyper-high dimensional fused descriptor features. MDF enables excellent performance in face identification, exceeding the state-of-the-art, but it comes with high memory and computational costs. As a solution to the high cost problem, this paper also presents an optimisation method, discriminant ability-based multi-descriptor selection (DAMS), to select a subset of descriptors from the set of 65 initial descriptors whilst maximising the discriminant ability. The MDF face representation, after being refined by DAMS, is named selective multi-descriptor fusion (SMDF). Compared with MDF, SMDF has much smaller feature dimension and is thus usable on an ordinary PC, but still has similar performance. Various experiments are conducted on the CAS-PEAL-R1 and LFW datasets to demonstrate the performance of the proposed methods.


Face identification Face recognition Feature extraction Feature selection Objective optimisation 



  1. 1.
    Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. University of Massachusetts, Amherst, Tech. Rep. pp 07–49Google Scholar
  2. 2.
    Sun Y, Liang D, Wang X, Tang X (2015) Deepid3: Face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873
  3. 3.
    Schroff F, Kalenichenko D, Philbin J (2015) FaceNet: a unified embedding for face recognition and clustering. In: pp 815–823Google Scholar
  4. 4.
    Liu J, Deng Y, Bai T, Wei Z, Huang C (2015) Targeting ultimate accuracy: face recognition via deep embedding, pp 06–24. arXiv:1506.07310[cs]
  5. 5.
    Kumar N, Berg A C, Belhumeur P N, Nayar S K (2009) Attribute and simile classifiers for face verification. In: 2009 IEEE 12th international conference on computer vision, Sep., pp 365–372Google Scholar
  6. 6.
    Learned-Miller E, Huang GB, RoyChowdhury A, Li H, Hua G (2016) Labeled faces in the wild: a survey. In: Kawulok M, Celebi ME, Smolka B (eds) Advances in face detection and facial image analysis. Springer International Publishing, Berlin, pp 189–248. CrossRefGoogle Scholar
  7. 7.
    Xie S, Shan S, Chen X, Chen J (2010) Fusing local patterns of gabor magnitude and phase for face recognition. IEEE Trans Image Process 19(5):1349–1361MathSciNetCrossRefGoogle Scholar
  8. 8.
    Wei X, Wang H, Guo G, Wan H (2015) Multiplex image representation for enhanced recognition. Int J Mach Learn Cybern 9:1–10Google Scholar
  9. 9.
    Chan CH, Tahir MA, Kittler J, Pietikäinen M (2013) Multiscale local phase quantization for robust component-based face recognition using kernel fusion of multiple descriptors. IEEE Trans Pattern Anal Mach Intell 35(5):1164–1177CrossRefGoogle Scholar
  10. 10.
    Taigman Y, Yang M, Ranzato M, Wolf L (2014) DeepFace: closing the gap to human-level performance in face verification. In: pp 1701–1708Google Scholar
  11. 11.
    Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: pp 1891–1898Google Scholar
  12. 12.
    Ding C, Tao D (2017) Trunk-Branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell PP(99):1–1Google Scholar
  13. 13.
    Huang GB, Learned-Miller E (2014) Labeled faces in the wild: Updates and new reporting procedures. Dept. Comput. Sci., Univ. Massachusetts Amherst, Amherst, MA, USA, Tech. Rep, pp 14–003Google Scholar
  14. 14.
    Huang KK, Dai DQ, Ren CX, Yu YF, Lai ZR (2017) Fusing landmark-based features at kernel level for face recognition. Pattern Recognit 63:406–415CrossRefGoogle Scholar
  15. 15.
    Ding C, Choi J, Tao D, Davis LS (2016) Multi-directional multi-level dual-cross patterns for robust face recognition. IEEE Trans Pattern Anal Mach Intell 38(3):518–531CrossRefGoogle Scholar
  16. 16.
    Visual descriptor (2017) Page Version ID: 788982618. [Online].
  17. 17.
    Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041CrossRefGoogle Scholar
  18. 18.
    Qi X, Xiao R, Li CG, Qiao Y, Guo J, Tang X (2014) Pairwise rotation invariant co-occurrence local binary pattern. IEEE Trans Pattern Anal Mach Intell 36(11):2199–2213CrossRefGoogle Scholar
  19. 19.
    Kim J, Yu S, Kim D, Toh K-A, Lee S (2017) An adaptive local binary pattern for 3d hand tracking. Pattern Recognit 61(Supplement C):139–152. [Online]. CrossRefGoogle Scholar
  20. 20.
    Karczmarek P, Kiersztyn A, Pedrycz W, Dolecki M (2017) An application of chain code-based local descriptor and its extension to face recognition. Pattern Recognit 65:26–34CrossRefGoogle Scholar
  21. 21.
    Zhen X, Zheng F, Shao L, Cao X, Xu D (2017) Supervised local descriptor learning for human action recognition. IEEE Trans Multimed 19(9):2056–2065CrossRefGoogle Scholar
  22. 22.
    Lan R, Zhou Y, Tang YY (2016) Quaternionic local ranking binary pattern: a local descriptor of color images. IEEE Trans Image Process 25(2):566–579MathSciNetCrossRefGoogle Scholar
  23. 23.
    Yan P, Liang D, Tang J, Zhu M (2016) Local feature descriptor using entropy rate. Neurocomputing 194:157–167CrossRefGoogle Scholar
  24. 24.
    Mangai UG, Samanta S, Das S, Chowdhury PR (2010) A survey of decision fusion and feature fusion strategies for pattern classification. IETE Tech Rev 27(4):293–307. [Online]. CrossRefGoogle Scholar
  25. 25.
    Nikan S, Ahmadi M (2014) Local gradient-based illumination invariant face recognition using local phase quantisation and multi-resolution local binary pattern fusion. IET Image Process 9(1):12–21CrossRefGoogle Scholar
  26. 26.
    Gao Z, Ding L, Xiong C, Huang B (2014) A robust face recognition method using multiple features fusion and linear regression. Wuhan Univ J Nat Sci 19(4):323–327CrossRefGoogle Scholar
  27. 27.
    Ruggieri S (2002) Efficient C4.5 [classification algorithm]. IEEE Trans Knowl Data Eng 14(2):438–444CrossRefGoogle Scholar
  28. 28.
    Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227CrossRefGoogle Scholar
  29. 29.
    Weinberger KQ, Saul LK (2009) Distance metric learning for large margin nearest neighbor classification. J Mach Learn Res 10:207–244 [Online].
  30. 30.
    Heisele B, Ho P, Poggio T (2001) Face recognition with support vector machines: global versus component-based approach. In: Proceedings of eighth IEEE international conference on computer vision, vol 2. ICCV 2001, pp 688–694Google Scholar
  31. 31.
    Lawrence S, Giles CL, Tsoi AC, Back AD (1997) Face recognition: a convolutional neural-network approach. IEEE Trans Neural Netw 8(1):98–113CrossRefGoogle Scholar
  32. 32.
    Sebe N, Lew MS, Cohen I, Garg A, Huang TS (2002) Emotion recognition using a Cauchy Naive Bayes classifier. In: Object recognition supported by user interaction for service robots, vol 1, pp 17–20Google Scholar
  33. 33.
    Li XX, Dai DQ, Zhang XF, Ren CX (2013) Structured sparse error coding for face recognition with occlusion. IEEE Trans Image Process 22(5):1889–1900MathSciNetCrossRefGoogle Scholar
  34. 34.
    Yang M, Zhang L, Yang J, Zhang D (2013) Regularized robust coding for face recognition. IEEE Trans Image Process 22(5):1753–1766MathSciNetCrossRefGoogle Scholar
  35. 35.
    Yang M, Zhang L, Shiu SCK, Zhang D (2013) Robust kernel representation with statistical local features for face recognition. IEEE Trans Neural Netw Learn Syst 24(6):900–912CrossRefGoogle Scholar
  36. 36.
    Huang KK, Dai DQ, Ren CX, Lai ZR (2016) Learning kernel extended dictionary for face recognition. IEEE Trans Neural Netw Learn Syst PP(99):1–13Google Scholar
  37. 37.
    Deng W, Hu J, Guo J (2012) Extended SRC: undersampled face recognition via intraclass variant dictionary. IEEE Trans Pattern Anal Mach Intell 34(9):1864–1870CrossRefGoogle Scholar
  38. 38.
    Deng W, Hu J, Guo J (2013) In defense of sparsity based face recognition. In The IEEE conference on computer vision and pattern recognition (CVPR)Google Scholar
  39. 39.
    Gao W, Cao B, Shan S, Chen X, Zhou D, Zhang X, Zhao D (2008) The CAS-PEAL large-scale chinese face database and baseline evaluations. IEEE Trans Syst Man Cybern Part A Syst Hum 38(1):149–161CrossRefGoogle Scholar
  40. 40.
    Xiong X, De la Torre F (2013) Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 532–539Google Scholar
  41. 41.
    Gross R, Matthews I, Cohn J, Kanade T, Baker S (2010) Multi-pie. Image Vis Comput 28(5):807–813CrossRefGoogle Scholar
  42. 42.
    Saragih J (2011) Principal regression analysis. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2881–2888Google Scholar
  43. 43.
    Bartlett MS, Littlewort G, Frank MG, Lainscsek C, Fasel IR, Movellan JR (2006) Automatic recognition of facial actions in spontaneous expressions. J Multimed 1(6):22–35CrossRefGoogle Scholar
  44. 44.
    Tan H, Yang B, Ma Z (2013) Face recognition based on the fusion of global and local hog features of face images. IET Comput Vis 8(3):224–234CrossRefGoogle Scholar
  45. 45.
    Fierro-Radilla AN, Nakano-Miyatake M, Perez-Meana H, Cedillo-Hernandez M, Garcia-Ugalde F (2013) An efficient color descriptor based on global and local color features for image retrieval. In: IEEE 2013 10th international conference on electrical engineering, computing science and automatic control (CCE), pp 233–238Google Scholar
  46. 46.
    Shabanzade M, Zahedi M, Aghvami SA (2011) Combination of local descriptors and global features for leaf recognition. Signal Image Process 2(3):23Google Scholar
  47. 47.
    Swets DL, Weng JJ (1996) Using discriminant eigenfeatures for image retrieval. IEEE Trans Pattern Anal Mach Intell 18(8):831–836CrossRefGoogle Scholar
  48. 48.
    Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650MathSciNetCrossRefGoogle Scholar
  49. 49.
    Ahonen T, Rahtu E, Ojansivu V, Heikkila J (2008) Recognition of blurred faces using local phase quantization. In 2008 19th international conference on pattern recognition, vol 12, pp 1–4Google Scholar
  50. 50.
    Vu NS, Caplier A (2012) Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans Image Process 21(3):1352–1365MathSciNetCrossRefGoogle Scholar
  51. 51.
    Ojala T, Pietikainen M, Maenpaa T (2002) Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intell 24(7):971–987CrossRefGoogle Scholar
  52. 52.
    Trefnỳ J, Matas J (2010) Extended set of local binary patterns for rapid object detection. In: Computer vision winter workshop, pp 1–7Google Scholar
  53. 53.
    Ren CX, Dai DQ, Li XX, Lai ZR (2014) Band-reweighed gabor kernel embedding for face image representation and recognition. IEEE Trans Image Process 23(2):725–740MathSciNetCrossRefGoogle Scholar
  54. 54.
    Best-Rowden L, Han H, Otto C, Klare BF, Jain AK (2014) Unconstrained face recognition: identifying a person of interest from a media collection. IEEE Trans Inf Forensics Secur 9(12):2144–2157CrossRefGoogle Scholar
  55. 55.
    Taigman Y, Yang M, Ranzato M, Wolf L (2015) Web-scale training for face identification. In: pp 2746–2754. [Online].
  56. 56.
    Sun Y, Wang X, Tang X (2015) Deeply learned face representations are sparse, selective, and robust. In: pp 2892–2900. [Online].

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.School of ComputingUlster UniversityBelfastUK

Personalised recommendations