Multimedia Tools and Applications

, Volume 76, Issue 3, pp 4635–4650 | Cite as

Modality identification for heterogeneous face recognition

  • Muhammad Khurram Shaikh
  • Ashref Lawgaly
  • Muhammad Atif Tahir
  • Ahmed Bouridane


Identifying the type of modalities of the query image which can be of types visual, NIR, digital camera, web camera etc. have been assumed to be available before face matching. This leads to a major drawback in achieving fully automated heterogeneous face recognition as real world scenarios cannot be reflected. Therefore, modality identification is an important component of the heterogeneous face recognition system which is being overlooked by majority of the state-of-the-art methods. This component should be given similar attention when comparing with other face recognition modules identifying pose, gesture, camera source etc. In this paper inspired from sensor pattern noise (SPN) estimation based approaches, a novel image sharpening based modality pattern noise technique is proposed for modality identification. The proposed system has been evaluated on three challenging benchmarks of heterogeneous face databases. The proposed technique has produced outstanding results and will open new avenues of research for automated HFR methods in future.


Heterogeneous face recognition Modality pattern noise Modality identification 


  1. 1.
    Cai X, Wang C, Xiao B, Chen X, Lv Z, Shi Y (2013) Coupled latent least squares regression for heterogeneous face recognition. IEEE Inter Conf Image Process:2772–2776Google Scholar
  2. 2.
    Chen M, Fridrich J, Goljan M, Luks̈ J (2008) Determining image origin and integrity using sensor noise. IEEE Trans Inf Forensics Secur 3:74–90CrossRefGoogle Scholar
  3. 3.
    Chingovska I, Anjos A, Marcel S (2013) Anti-spoofing in action joint operation with a verification system. In: IEEE International on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 98–104Google Scholar
  4. 4.
    de Freitas Pereira T, Anjos A, de Martino JM, Marcel S (2012) LBP-TOP based countermeasure against face spoofing attacks. Computer Vision-ACCV Workshops:121–132Google Scholar
  5. 5.
    de Freitas Pereira T, Anjos A, de Martino JM, Marcel S (2013) Can face anti-spoofing countermeasures work in a real world scenario. In: IEEE International conference on Biometrics (ICB), pp 1–8Google Scholar
  6. 6.
    Galbally J, Marcel S, Fierrez J (2014) Image quality assessment for fake biometric detection application to iris, fingerprint, and face recognition. IEEE Trans Image Process 23:710–724MathSciNetCrossRefGoogle Scholar
  7. 7.
    Garcia JO, Fierrez J, Fernandez FA, Galbally J, Freire MR, Gonzalez-Rodriguez J, Garcia-Mateo C, Alba-Castro JL, Gonzalez-Agulla E, Otero-Muras E, Garcia-Salicetti S, Allano L, Ly-Van B, Dorizzi B, Kittler J, Bourlai T, Poh N, Deravi F, Ng M, Fairhurst M, Hennebert J, Humm A, Tistarelli M, Brodo L, Richiardi J, Drygajlo A, Ganster H, Sukno FM, Pavani SK, Frangi A, Akarun L, Savran A (2010) The multiscenario multienvironment biosecure multimodal database (bmdb). IEEE Trans Pattern Anal Mach Intell 32:1097–1111CrossRefGoogle Scholar
  8. 8.
    Han J, Shao L, Xu D, Shotton J (2013) Enhanced computer vision with microsoft kinect sensor A review. IEEE Trans Cybernetics 43:1318–1334CrossRefGoogle Scholar
  9. 9.
    Huang X, Lei Z, Wang X, Li SZ (2013) Regularized discriminative spectral regression method for heterogeneous face matching. IEEE Trans Image Process 22:1513–1516MathSciNetGoogle Scholar
  10. 10.
    Jain AK, Li SZ (2005) Handbook of Face Recognition. Springer, NewYorkMATHGoogle Scholar
  11. 11.
    Kang X, Li Y, Qu Z, Huang J (2012) Enhancing source camera identification performance with a camera reference phase sensor pattern noise. IEEE Trans Inf Forensics Secur 7:393–402CrossRefGoogle Scholar
  12. 12.
    Klare BF, Jain AK (2013) Heterogeneous face recognition using kernel prototype similarities. IEEE Trans Pattern Anal Mach Intell 35:1410–1422CrossRefGoogle Scholar
  13. 13.
    Kollreider K, Fronthaler H, Bigun J (2009) Non-intrusive liveness detection by face images. Image Vision Computing J 27:233244CrossRefGoogle Scholar
  14. 14.
    Komulainen J, Hadid A, Pietikainen M, Anjos A, Marcel S (2013) Complementary countermeasures for detecting scenic face spoofing attacks. In: International Conference on Biometrics (ICB), pp 1–7Google Scholar
  15. 15.
    Kose N, Dugelay JL (2013) Countermeasure for the protection of face recognition systems against mask attacks. In: IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp 1–6Google Scholar
  16. 16.
    Lawgaly A, Khelifi F, Bouridane A (2013) Image sharpening for efficient source camera identification based on sensor pattern noise estimation. In: Fourth IEEE International Conference In Emerging Security Technologies (EST), pp 113–116Google Scholar
  17. 17.
    Lee YH, Park SY (1990) A study of convex/concave edges and edge-enhancing operators based on the laplacian. IEEE Trans Circuits and Syst 37:940–946MathSciNetCrossRefGoogle Scholar
  18. 18.
    Lei Z, Pietikainen M, Li SZ (2014) Learning discriminant face descriptor. IEEE Trans Pattern Anal Mach Intell 36:289–302CrossRefGoogle Scholar
  19. 19.
    Lei Z, Liao S, Jain A.K, Li SZ (2012) Coupled discriminant analysis for heterogeneous face recognition. IEEE Trans Inf Forensics Secur 7:1707–1716CrossRefGoogle Scholar
  20. 20.
    Li CT (2010) Source camera identification using enhanced sensor pattern noise. IEEE Trans Inf Forensics Secur 5:280–287CrossRefGoogle Scholar
  21. 21.
    Li SZ, Lei Z, Ao M (2009) The HFB face database for heterogeneous face biometrics researchGoogle Scholar
  22. 22.
    Li SZ, Yi D, Lei Z, Liao S (2013) The CASIA NIR-VIS 2.0 face database. In: IEEE International Conference In Computer Vision and Pattern Recognition Workshops, pp 348–353Google Scholar
  23. 23.
    Li Z, Gong D, Qiao Y, Tao D (2014) Common feature discriminant analysis for matching infrared face images to optical face images. IEEE Trans Image Process 23:2436–2445MathSciNetCrossRefGoogle Scholar
  24. 24.
    Lin D, Tang X (2006) Inter-modality face recognition. European Conference on Computer Vision:13–26Google Scholar
  25. 25.
    Liu L, Shao L (2013) Learning discriminative representations from RGB-D video data. In: International Joint Conference on Artificial IntelligenceGoogle Scholar
  26. 26.
    Liu L, Shao L, Li X, Lu K (2016) Learning spatio-temporal representations for action recognition: A genetic programming approach. IEEE Trans Cybernetics 46:158–170CrossRefGoogle Scholar
  27. 27.
    Lukas J, Fridrich J, Goljan M (2006) Digital camera identification from sensor pattern noise. IEEE Trans Inf Forensics Secur 1:205–214CrossRefGoogle Scholar
  28. 28.
    Mtt J, Hadid A, Pietikainen M (2011) Face spoofing detection from single images using micro-texture analysis. In: IEEE International joint conference on Biometrics, pp 1–7Google Scholar
  29. 29.
    Pan G, Sun L, Wu Z, Lao S (2007) Eyeblink-based anti-spoofing in face recognition from a generic webcamera. In: IEEE 11th International Conference on Computer Viion (ICCV), pp 1–8Google Scholar
  30. 30.
    Pavlidis I, Symosek P (2000) The imaging issue in an automatic face/disguise detection system. In: IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, pp 15– 24Google Scholar
  31. 31.
    Shao L, Wu D, Li X (2014) Learning deep and wide: A spectral method for learning deep networks. IEEE Transactions on Neural Networks and Learning Systems 25:2303–2308CrossRefGoogle Scholar
  32. 32.
    Sutcu Y, Bayram S, Sencar HT, Memon N (2007) Improvements on sensor noise based source camera identification. In: IEEE International Conference on Multimedia and Expo, pp 24–27Google Scholar
  33. 33.
    Van Lanh T, Chong KS, Emmanuel S, Kankanhalli MS (2007) A survey on digital camera image forensic methods. In: IEEE International Conference on Multimedia and Expo, pp 16–19Google Scholar
  34. 34.
    Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31:1955–1967CrossRefGoogle Scholar
  35. 35.
    Yi D, Lei Z, Li SZ (2015) Shared representation learning for heterogenous face recognition. In: Proceedings of 11th IEEE international conference on automatic face and gesture recognitionGoogle Scholar
  36. 36.
    Yu M, Liu L, Shao L (2015) Structure-preserving binary representations for RGB-D action recognition. IEEE Trans Pattern Anal Mach IntellGoogle Scholar
  37. 37.
    Zhang Z, Yan J, Liu S, Lei Z, Yi D, Li SZ (2012) A face antispoofing database with diverse attacks. In: IEEE International conference on Biometrics (ICB), pp 26–31Google Scholar
  38. 38.
    Zhu JY, Zheng WS, Lai JH, Li SZ (2014) Matching NIR face to VIS face using transduction. IEEE Trans Inf Forensics Secur 9:501–514CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Muhammad Khurram Shaikh
    • 1
  • Ashref Lawgaly
    • 1
  • Muhammad Atif Tahir
    • 1
    • 2
  • Ahmed Bouridane
    • 1
  1. 1.Department of Computer Science and Digital TechnologiesNorthumbria University at NewcastleNewcastleUK
  2. 2.Systems Research Laboratory Computer ScienceFAST- National University of Computer and Emerging SciencesKarachiPakistan

Personalised recommendations