Skip to main content
SpringerLink
Log in

DCCA and DMCCA framework for multimodal biometric system

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep networks have been successfully applied to unsupervised feature learning for single modalities such as Text, Image, or Audio. Here, we present the Deep Network that learns nonlinear transformations of two or more biometric modalities. The resultant representation is a combination of maximally correlated features of the input modalities. It can be viewed as a feature fusion technique with the nonlinear extension of the linear multiset canonical correlation analysis (MCCA). The Foundation of a multibiometric system is information fusion. Feature level fusion is an important area of research owing to the information richness of features. This Deep learning model outlines the integrated correlation among multiple feature vectors and forms a new fused feature vector. This paper presents the Deep Canonical Correlation Analysis (DCCA) for bimodal biometric system and Deep Multiset Canonical Correlation Analysis (DMCCA) framework for the multimodal biometric system outperforming the traditional CCA based fusion methods. Experiments are carried out with the SDUMLA-HMT multimodal dataset. Using the proposed DCCA method performance of a bi-modal biometric system with iris and fingerprint modalities in terms of EER (equal error rate) is 3.78% and 0.116% (single modality recognition). While for the multibiometric system with four modalities (iris, fingerprint, finger-vein, and face) with DMCCA, EER is 0.717%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Ammour B, Boubchir L, Bouden T, Ramdani M (2020) Face–iris multimodal biometric identification system. Electron 9(1):85

    Article  Google Scholar 

  2. An L, Yang S, Bhanu B (2015) Person re-identification by robust canonical correlation analysis. IEEE Signal Process Lett 22(8):1103–1107

    Article  Google Scholar 

  3. Anderson TW (1958) An introduction to multivariate statistical analysis, vol 2. Wiley, New York

    Google Scholar 

  4. Andrew G, Arora R, Bilmes J, Livescu K (2013) Deep canonical correlation analysis. In: International conference on machine learning, PMLR, pp 1247–1255

  5. Bach FR, Jordan MI (2005) A probabilistic interpretation of canonical correlation analysis

  6. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720. https://doi.org/10.1109/34.598228

    Article  Google Scholar 

  7. Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. In: Neural networks: Tricks of the trade, Springer, pp 437–478

  8. Bilenko NY, Gallant JL (2016) Pyrcca: Regularized kernel canonical correlation analysis in python and its applications to neuroimaging. Frontiers in Neuroinformatics 10:49

    Article  Google Scholar 

  9. Borga M, Knutsson H (2001) A canonical correlation approach to blind source separation. Report liu-IMT-EX-0062 Department of Biomedical Engineering Linkping University

  10. Chakraborty S, Mitra M, Pal S (2016) Biometric analysis using fused feature set from side face texture and electrocardiogram. IET Sci, Meas Technol 11(2):226–233

    Article  Google Scholar 

  11. Correa NM, Li YO, Adali T, Calhoun VD (2008) Canonical correlation analysis for feature-based fusion of biomedical imaging modalities and its application to detection of associative networks in schizophrenia. IEEE J Sel Top Sign Process 2(6):998–1007

    Article  Google Scholar 

  12. Eskandari M, Toygar Ö. (2014) Fusion of face and iris biometrics using local and global feature extraction methods. Signal Image and Video Process 8 (6):995–1006

    Article  Google Scholar 

  13. Gao X, Sun Q, Xu H (2017) Multiple-rank supervised canonical correlation analysis for feature extraction, fusion and recognition. Expert Syst Appl 84:171–185

    Article  Google Scholar 

  14. Haghighat M, Abdel-Mottaleb M, Alhalabi W (2016) Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition. IEEE Trans Inf Forensic Secur 11(9):1984–1996

    Article  Google Scholar 

  15. Hammad M, Liu Y, Wang K (2018) Multimodal biometric authentication systems using convolution neural network based on different level fusion of ecg and fingerprint. IEEE Access 7:26527–26542

    Article  Google Scholar 

  16. Hardoon DR, Mourao-Miranda J, Brammer M, Shawe-Taylor J (2007) Unsupervised analysis of fmri data using kernel canonical correlation. NeuroImage 37(4):1250–1259

    Article  Google Scholar 

  17. Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: An overview with application to learning methods. Neural Comput 16 (12):2639–2664

    Article  Google Scholar 

  18. Hel-Or Y (2004) The canonical correlations of color images and their use for demosaicing. HP Laboratories Israel, Tech. Rep HPL-2003-164r1

  19. Hotelling H (1992) Relations between two sets of variates. In: Breakthroughs in statistics, Springer, pp 162–190

  20. Hsieh WW (2000) Nonlinear canonical correlation analysis by neural networks. Neural Netw 13(10):1095–1105

    Article  Google Scholar 

  21. Huo G, Liu Y, Zhu X, Dong H, He F (2015) Face–iris multimodal biometric scheme based on feature level fusion. J Electron Imaging 24 (6):063020

    Article  Google Scholar 

  22. Jain A, Nandakumar K, Ross A (2005) Score normalization in multimodal biometric systems. Pattern Recogn 38(12):2270–2285

    Article  Google Scholar 

  23. Kettenring JR (1971) Canonical analysis of several sets of variables. Biometrika 58(3):433–451

    Article  MathSciNet  Google Scholar 

  24. Kim TK, Kittler J (2005) Locally linear discriminant analysis for multimodally distributed classes for face recognition with a single model image. IEEE Trans Pattern Anal Mach Intell 27(3):318–327

    Article  Google Scholar 

  25. Kim TK, Kittler J, Cipolla R (2007) Discriminative learning and recognition of image set classes using canonical correlations. IEEE Trans Pattern Anal Mach Intell 29(6):1005–1018

    Article  Google Scholar 

  26. Lai PL, Fyfe C (2000) Kernel and nonlinear canonical correlation analysis. Int J Neural Syst 10(05):365–377

    Article  Google Scholar 

  27. Li YO, Adali T, Wang W, Calhoun VD (2009) Joint blind source separation by multiset canonical correlation analysis. IEEE Trans Signal Process 57 (10):3918–3929

    Article  MathSciNet  Google Scholar 

  28. Loog M, van Ginneken B, Duin RP (2005) Dimensionality reduction of image features using the canonical contextual correlation projection. Pattern Recogn 38(12):2409–2418

    Article  Google Scholar 

  29. Nielsen AA (2002) Multiset canonical correlations analysis and multispectral, truly multitemporal remote sensing data. IEEE Trans Image Process 11 (3):293–305

    Article  Google Scholar 

  30. Regouid M, Touahria M, Benouis M, Costen N (2019) Multimodal biometric system for ecg, ear and iris recognition based on local descriptors. Multimed Tools Appl 78(16):22509–22535

    Article  Google Scholar 

  31. Ross A, Jain A (2003) Information fusion in biometrics. Pattern Recogn Lett 24(13):2115–2125

    Article  Google Scholar 

  32. Ross A, Jain AK (2004) Multimodal biometrics: An overview. In: 2004 12Th european signal processing conference, IEEE, pp 1221–1224

  33. Ross A, Poh N (2009) Multibiometric systems: Overview, case studies, and open issues. Handbook of Remote Biometrics, pp 273–292

  34. Ross AA, Nandakumar K, Jain AK (2006) Handbook of multibiometrics, vol 6, Springer Science & Business Media

  35. Sun QS, Liu ZD, Heng PA, Xia DS (2005) A theorem on the generalized canonical projective vectors. Pattern Recogn 38(3):449–452

    Article  Google Scholar 

  36. Sun QS, Zeng SG, Heng PA, Xia DS (2004) Feature fusion method based on canonical correlation analysis and handwritten character recognition. In: ICARCV 2004 8Th control, automation, robotics and vision conference, 2004, vol 2, IEEE, pp 1547–1552

  37. Sun QS, Zeng SG, Liu Y, Heng PA, Xia DS (2005) A new method of feature fusion and its application in image recognition. Pattern Recogn 38(12):2437–2448

    Article  Google Scholar 

  38. Sun T, Chen S (2007) Locality preserving cca with applications to data visualization and pose estimation. Image Vis Comput 25(5):531–543

    Article  Google Scholar 

  39. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cogn Neurosci 3(1):71–86

    Article  Google Scholar 

  40. Vía J, Santamaría I, Pérez J (2007) A learning algorithm for adaptive canonical correlation analysis of several data sets. Neural Netw 20(1):139–152

    Article  Google Scholar 

  41. Wang L, Xiao L, Wei Z (2015) Image dehazing using two-dimensional canonical correlation analysis. IET Comput Vis 9(6):903–913

    Article  Google Scholar 

  42. Xing X, Wang K, Yan T, Lv Z (2016) Complete canonical correlation analysis with application to multi-view gait recognition. Pattern Recogn 50:107–117

    Article  Google Scholar 

  43. Yang J, Yang JY, Zhang D, Lu JF (2003) Feature fusion: Parallel strategy vs. serial strategy. Pattern Recogn 36(6):1369–1381

    Article  Google Scholar 

  44. Yin Y, Liu L, Sun X (2011) Sdumla-hmt: A multimodal biometric database. In: Chinese conference on biometric recognition, Springer, pp 260–268

  45. Yuan YH, Shen X, Li Y, Li B, Gou J, Qiang J, Zhang X, Sun QS (2019) Composite nonlinear multiset canonical correlation analysis for multiview feature learning and recognition. Concurrency and Computation: Practice and Experience, pp e5476

  46. Yuan YH, Sun QS, Zhou Q, Xia DS (2011) A novel multiset integrated canonical correlation analysis framework and its application in feature fusion. Pattern Recogn 44(5):1031–1040

    Article  Google Scholar 

  47. Zheng W, Zhou X, Zou C, Zhao L (2006) Facial expression recognition using kernel canonical correlation analysis (kcca). IEEE Trans Neural Netw 17 (1):233–238

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Technology, Savitribai Phule Pune University, Pune, India

    Shubhangi Deshmukh & Aditya Abhyankar

  2. Persistent Systems, Pune, India

    Shubhangi Kelkar

Authors
  1. Shubhangi Deshmukh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aditya Abhyankar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shubhangi Kelkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shubhangi Deshmukh.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deshmukh, S., Abhyankar, A. & Kelkar, S. DCCA and DMCCA framework for multimodal biometric system. Multimed Tools Appl 81, 24477–24491 (2022). https://doi.org/10.1007/s11042-022-12435-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12435-9

Keywords

Search

Navigation