Federated learning for biometric recognition: a survey

Abstract

Recently, deep learning (DL) has achieved great success in biometric recognition. The application of DL has also led to a high demand for biometric data. However, as people attach more importance to privacy protection, biometric data have become increasingly difficult to obtain and access, leading to significant limitations in the development and application of DL-based biometric recognition. Federated learning (FL), a distributed learning technique with privacy protection, provides a potential solution to this problem. Several researchers have attempted to integrate FL into biometric recognition. These studies have shown that the introduction of FL not only solves the conflict between privacy and accessibility of biometric data but also improves the accuracy and generalizability of local recognition systems. Therefore, the combination of FL and biometric recognition techniques has become a new research hotspot. In this survey, we comprehensively review the latest advances regarding the application of FL to biometric recognition, biometric presentation attack detection and the related fields to provide new researchers with a quick and systematic overview of this emerging cross-disciplinary field. This paper also summarizes the future opportunities and challenges of this field. To our knowledge, this is the first survey that systematically organizes and analyses federated biometric recognition and related fields to provide suggestions and references for future research.

1 Introduction

Biometric recognition, as an emerging identification technology, has received extensive attention from researchers in recent years (Liu et al. 2022; Buciu and Gacsadi 2016; Mir et al. 2011). Studies have shown that, compared with traditional identification technologies such as passwords and ID cards, biometric recognition has outstanding advantages in terms of convenience, security, and uniqueness (Guennouni et al. 2019; Gafurov 2010). This approach has led to wide use in identity verification, user recognition, and security authentication (Minaee et al. 2023; Rattani and Derakhshani 2018). Currently, common human biometric features can be divided into two categories. The first category consists of human physiological features, such as fingerprints (Yang et al. 2019), palmprints (Balakrishnan et al. 2021), faces (Liu et al. 2020), irises (Balashanmugam et al. 2023), and veins (Liu et al. 2023), and the second category includes human behavioral features, e.g., gait (Hsu et al. 2023) and signatures (Elhoseny et al. 2018).

Fig. 1

Sample images for various biometric features. The data were collected from the publicly available face dataset LFW (Huang et al. 2008), fingerprint dataset NUPT-FVP (Ren et al. 2022), finger vein dataset HKPU (Kumar and Zhou 2011), iris dataset CASIA lamp (Qiu et al. 2007) and gait dataset CASIA-B (Yu et al. 2006a)

Some common biometric data are presented in Fig. 1. In real life, each biometric feature has unique advantages and can be applied to different situations. Therefore, numerous researchers have focused on the earliest studied biometrics, such as fingerprints and faces, and emerging biometrics, such as veins and gaits.

According to the differences in feature extraction methods, biometric recognition methods can be divided into methods based on traditional feature extraction and methods based on deep learning (DL). In recent years, DL-based biometric recognition methods have gradually become the mainstream methods due to their obvious superiority in accuracy (Parashar et al. 2023; Yang et al. 2022; Chaudhuri 2020). DL-based methods usually require a large amount of biometric data for training models; however, obtaining biometric datasets is very difficult and costly. In practice, biometric data are often small and fragmented at various institutions or research organizations. Some methods try to address the problem by sharing data. However, whether using traditional biometric methods or deep learning-based biometric methods, sharing data can lead to user privacy leakages. Therefore, the data sharing solution is not feasible. This leads to the failure to address the problem of lacking data, which results in insufficient generalization capabilities for the constructed biometric recognition models. To address this problem, in previous studies, public datasets were usually employed to pretrain models. However, with the recent increase in privacy protection worldwide, several large public biometric datasets, such as the Ms-celeb-1 m (Guo et al. 2016) face dataset, that can be used for pretraining are no longer available. Similarly, other fields that are closely related to biometric recognition, e.g., biometric presentation attack detection (Kolberg et al. 2021; Wang et al. 2023) and biometric image quality assessment (Shaheed and Qureshi 2022), also suffer from this problem. In addition, in real life, a series of techniques, such as biometric presentation attack detection, are difficult to apply. Therefore, solving the above problems is highly important in biometric recognition and related fields.

Fig. 2

Schematic diagram of the traditional centralized method and the distributed method based on FL. In traditional methods, terminal devices cannot independently train effective recognition models due to data and privacy limitations. The federated biometric recognition method assists terminals in learning collaboratively via servers and can protect users’ data and privacy

Currently, the biometrics field faces conflicts between data privacy and accessibility. Federated learning (Díaz and García 2023; Cao et al. 2021) provides a distributed solution to this problem through privacy preservation. Therefore, the incorporation of FL into the field of biometrics is very necessary. FL is an emerging distributed learning framework that aims to address the problem of collaborative multiterminal learning while protecting privacy. Since the distributed architecture and user privacy preservation properties of FL match well with the requirements of biometric recognition, FL has gained increasing attention from scholars. Numerous researchers have applied FL and improved upon it to realize distributed biometric recognition, achieving greater success. Compared with traditional DL-based biometric systems, FL-based frameworks can train effective biometric recognition models without violating users’ privacy. These methods solve the conflict between privacy and accessibility of biometric data. Figure 2 shows the comparison of traditional centralized and emerging FL-based distributed biometric recognition systems. In the traditional biometric system, local clients train the recognition model independently. However, the lack of local data and the inability to collaborate between clients result in limited performance of models trained independently by clients. The FL-based biometric recognition framework solves this problem. Multi-end cooperative learning for higher recognition accuracy is achieved by first uploading local models to a server and then aggregating the models via the server while not sharing user data. These advantages have led to FL-based biometric recognition methods, which received extensive attention soon after they were proposed (Ding et al. 2022; Kim et al. 2021). Currently, the application of FL has become a research hotspot in biometric recognition.

Numerous works have been proposed in federated biometric recognition; therefore, this paper provides a comprehensive review of recent advances in this cross-cutting field and aims to both summarize the existing related research and determine future research directions. For each work, we provide information on its research content, main contributions, etc. Additionally, the results and performance of some studies are compared and summarized in this paper. Finally, the challenges and opportunities for research in federated biometric recognition are explored. The main contributions of this paper are summarized as follows.

• This paper provides a comprehensive review of recent advances in FL applications for seven types of biometric recognition (face, iris, palm print, finger vein, gait, speech recognition, and multibiometric). To our knowledge, this is the first review paper on federated biometric recognition.

• This paper also reviews the progress of FL applied to related fields of biometric recognition, including but not limited to biometric presentation attack detection and biometric quality evaluation. Likewise, the contents, contributions and experimental results of FL are comprehensively analyzed and summarized.

• This paper highlights the challenges and potential research directions for federated biometric recognition, and these suggestions can provide valuable references for future researchers.

The remainder of this paper is organized as follows. In Sect. 2, we provide an overview of biometric recognition and FL and describe the feasibility and necessity of applying FL to biometric recognition. In Sect. 3, we summarize in detail the recent works on federated biometric recognition and analyze the results of these studies. Later, in Sect. 4, we summarize and compare these existing works and analyze their key issues and core techniques. Afterward, we explore the challenges and future directions of federated biometric recognition in Sect. 5. Finally, we conclude the paper in Sect. 6.

2 Biometric recognition and federated learning

2.1 Biometric recognition

Fig. 3

Technological process of most classical biometric recognition algorithms

Biometric recognition aims to identify and authenticate the user through human physiological or behavioral features (Ahmadkhani and Adibi 2016; Liu et al. 2020). As an emerging identification technology, this technology is superior in terms of convenience and security compared with traditional technology. Most biometric recognition methods can be divided into four steps, namely, data capture, data preprocessing, feature extraction and feature matching, as shown in Fig. 3.

Currently, according to the different features, biometric recognition can be classified into face recognition (Min et al. 2019), fingerprint recognition (Lin and Kumar 2018), palmprint recognition (Yang et al. 2023), vein recognition (Qin et al. 2023), iris recognition (Wang et al. 2020), and gait recognition (Liao et al. 2020). Each of these recognition methods has unique advantages and is suitable for application in different situations. For example, face recognition is highly convenient, finger vein recognition is performed while the patient is alive and rooted subcutaneously, and gait recognition can be applied to long-distance identification tasks.

Biometric recognition methods can be divided into feature engineering-based recognition (Zhang and Wang 2022) and DL-based recognition (Tsai et al. 2023) approaches based on their recognition techniques. In feature engineering-based recognition methods, biometric data are processed via a manually designed algorithm to obtain biometric representations. For example, Chaabane et al. (2022) proposed a facial recognition method based on statistical features and the support vector machine algorithm, where a statistical analysis was used to extract and select the target statistical features. Krishnan and Thomas (2023) designed six feature representations based on the anatomy of vein patterns, including the fork, bridge, and other vein patterns. However, manually crafted filters have problems such as single scales, and they have difficulty adapting to different datasets. Thus, Li et al. (2024) proposed a neural network with a learnable variable curvature-based Gabor convolutional layer for finger vein recognition, which solves the problem that the traditional Gabor filtering scale is singular and difficult to adjust.

DL-based recognition methods involve feature extraction through DL models. For example, Deng et al. (2023) proposed a highly efficient and compact deep learning model for facial recognition, which achieved state-of-the-art performance on various facial recognition benchmarks. Additionally, Ma et al. (2023) proposed a dynamic aggregation network to learn more discriminative gait features by creating a dynamic attention mechanism that operates between the features of neighboring pixels. With the development of DL technology, DL-based biometric recognition methods have shown greater advantages in terms of accuracy and have gradually become the mainstream solution in biometric recognition. Since biometric data are related to user privacy, obtaining a large biometric dataset for training DL models is highly difficult and costly. This leads to a great limitation in the applicability of these DL-based recognition methods in reality.

Moreover, the lack of data also exists in various fields related to biometric recognition, such as biometric image quality assessment and biometric image presentation attack detection. Taking biometric presentation attack detection (Mehboob et al. 2023) as an example, in practical applications, fake biometric data are often distributed in various research centers rather than in end devices that are facing direct threats. Moreover, the performance of presentation attack detection models obtained directly from research centers is difficult to guarantee at end devices (Shao et al. 2021). Therefore, training or acquiring effective attack detection models for end devices is difficult. Similar to biometric presentation attack detection, other biometrics-related fields also suffer from the dilemma of lacking data at the end device. The lack of data leads to restrictions in the application of these related technologies in real devices. Therefore, solving this problem is particularly important in these biometric recognition-related fields.

2.2 Federated learning

Federated learning (Drainakis et al. 2023; Wasilewska et al. 2023; Zhang et al. 2023), as a distributed learning technique with privacy-preserving, provides a potential solution to resolve the conflict between user privacy and data accessibility. In 2016, McMahan et al. (2017) first proposed and implemented the most classical FL method, FedAvg. The core of FedAvg is to collect models from local clients via a third-party server and aggregate all local models to achieve distributed learning. The proposed model aggregation method is shown in Eq. 1.

$$\begin{aligned} w_{t} = ~{\sum \limits _{k = 1}^{K}{\frac{1}{n}w_{t}^{k}}}, \end{aligned}$$

(1)

where $w_t$ denotes the model obtained by aggregation in round t, ${w^k}_t$ denotes the local model of the k-th client in round t, K denotes the index set of all clients, and $k \in K$. n denotes the total number of clients. Since FedAvg implements a distributed learning technique with privacy preservation, it has received great attention from researchers in various fields (Coelho et al. 2023; Arisdakessian et al. 2022; Pokhrel and Choi 2020).

For biometric recognition, FL is highly suitable for its development requirements. On the one hand, FL meets the demand of organizations for high-performance recognition models by enabling individual organizations or terminals to collaborate in training; on the other hand, FL does not require clients to upload or share their user data, which meets the demand of protecting user privacy. Therefore, FL can resolve the conflict between user privacy and data accessibility that the biometric field is currently facing. Moreover, this approach enables clients to train models with high performance even when they have only a small amount of data. These advantages have led FL to receive extensive attention from scholars in biometric recognition, and some excellent results have been achieved (Bai et al. 2021; Luo et al. 2022; Shao and Zhong 2020; Lian et al. 2023), which have greatly enhanced the accuracy, usability, and security of biometric systems.

To summarize, the combination of FL and biometric recognition is the latest research hotspot. However, classical FL performs poorly when directly applied to biometric recognition. This is due to the strong heterogeneity of biometric data. Therefore, numerous scholars in this field have initiated research on various aspects of relieving the heterogeneity of biometric data, personalizing federated biometric frameworks, and generalizing biometric recognition models. These studies are discussed and analyzed in detail in Sect. 3. In addition, with further cross-studies, we can observe that valuable topics and directions, such as the application of FL to address security problems related to biometric recognition and the task of biometric quality assessments, have gradually surfaced. The exploration of these issues helps to further enhance the recognition performance and application prospects of federated biometric recognition systems. In Sect. 5 of this paper, based on a comprehensive summary of the existing results in federated biometric recognition, the challenges and potential research directions are proposed and analyzed.

Fig. 4

Statistics of seven federated biometric recognition

3 Reviews of federated biometric recognition and related fields

This section provides a comprehensive review of recent works on federated biometric recognition. In this section, almost all the relevant research in this field is covered. First, we summarize in detail the existing FL research on seven biometric recognition methods and analyze the experimental results. For each work, this section provides information on the research content, main contributions, etc. Then, we also provide a review of the research on FL applications for biometric security problems. Similarly, detailed information on their research content, main contributions, etc., is provided in this section. Finally, this section further summarizes the works of FL applications in other related fields of biometric recognition.

3.1 FL for biometric recognition

In recent years, several researchers have achieved good results in the field of federated biometric recognition. Next, we review the research on FL applied to seven biometric techniques: face recognition, iris recognition, palmprint recognition, finger vein recognition, gait recognition, speech recognition, and multibiometric recognition. There is almost no research on combining other biometric recognition techniques with FL. We have conducted a comprehensive count of the number of existing studies on FL in each biometric recognition field, and the results are shown in Fig. 4.

Table 1 Review of existing federated face recognition works

3.1.1 Federated face recognition

• a. Methods

Face recognition, as one of the most popular biometric recognition techniques, has received substantial attention from researchers. The results in Fig. 4 show that federated face recognition is the most popular research direction. This section provides a comprehensive summary of these studies, and the detailed results are presented in Table 1.

Fig. 5

Personalized federated face recognition framework, referred from (Liu et al. 2022)

As shown in Table 1, to date, there have been 11 studies on federated face recognition. In 2021, Bai et al. (2021) proposed the first method of applying FL to face recognition, and since then this method has rapidly gained widespread use. In the same year, other researchers explored various aspects of federated face recognition more deeply. For example, Hosseini et al. (2021) considered the problem of user-authenticated model training under an FL setup and proposed the FedUV framework to solve this problem. Experiments were also conducted on three datasets comprising face, voice, and handwritten data. ZHuang et al. (2021) proposed a new unsupervised federated face recognition method with a new domain constraint loss (DCL) to normalize the source domain training. Kim et al. (2021) considered the distributed face recognition task "continuous" and proposed strategies for positive data labeling, data management, updated model evaluation, and training consideration.

In 2022, many researchers have investigated the potential of federated face recognition. First, Liu et al. (2022) explored the application of FL to face authentication systems and proposed a new equivalence class of embedding vectors. Moreover, Liu et al. (2022) noted the necessity of personalized settings in federated face recognition and proposed the FedFR framework to improve generic face representations. This was the first study to explore personalized face recognition with FL settings. The personalized FL framework is shown in Fig. 5. Shang et al. (2022), from the perspective of the loss function, noted that existing studies have not explored the loss function for local training. To fill this gap, a joint face recognition system was constructed, and the performances of five classical loss functions were tested. This work provides useful suggestions for selecting loss functions. Moreover, in an artificial intelligence of things (AIoT) network, Ding et al. (2022) proposed an efficient industrial FL framework for the AIoT that utilizes transfer learning to accelerate the FL process on devices.

Face recognition, as a classic and popular biometric recognition technique, has gained more widespread attention than other biometric recognition methods. Existing studies have made excellent contributions to federated face recognition. For new researchers, it is important to build on these studies for further exploration.

• b. Performance

The commonly used and publicly available datasets for federated face recognition include MS-Celeb-1 M (Guo et al. 2016), LFW (Huang et al. 2008), VGGFace2 (Cao et al. 2018), CASIA-Webface (Yi et al. 2014), AgeDB (Moschoglou et al. 2017), CFP-FP (Sengupta et al. 2016), and IJB-C (Maze et al. 2018). Moreover, accuracy (ACC) and TPR@FPR are often used to verify the method’s performance. Here, ACC is the percentage of correct classifications. TPR@FPR is the true positive rate (TPR) for a certain false positive rate (FPR). This paper provides a performance comparison of several federated face recognition algorithms in Table 2.

Table 2 Performance of several federated face recognition algorithms

As shown in Table 2. Although these methods cannot be directly compared, they are still able to perform meaningful analyses. For example, the methods in (Niu and Deng 2022) and (Liu et al. 2022) use overlapping datasets as well as the same validation metric, i.e., ACC. Based on the available results, it is possible to determine that method proposed in (Niu and Deng 2022) performs better than that in (Liu et al. 2022). This is because the approach in (Niu and Deng 2022) presented an updated research result that sets more advanced modules for federated face recognition. Moreover, the results of (Liu et al. 2022) showed that personalized settings are effective at improving local performance in federated face recognition tasks.

Compared to those on face recognition, relatively few studies have been conducted on other federated biometric recognition fields. Therefore, we present a unified review of these studies in Table 3, and a detailed description of these methods follows.

Table 3 Review of existing research on the application of FL to biometric recognition (excluding face recognition)

3.1.2 Federated iris recognition

• a. Methods

For iris recognition, Luo et al. (2022) were the first to propose the concept of applying FL to iris recognition and proposed the FedIris framework in 2022. FedIris proposed the Fed-Triplet loss by applying federated communication templates to iris recognition. The Fed-Triplet loss, which is upgraded from the triple loss function, was proposed for use in client template communication. The Fed-Triplet model is shown in Eq. 2.

$$\begin{aligned} L_{FT} = max\left( {\parallel f_{i}\left( x_{i}^{a} \right) - f_{i}\left( x_{i}^{p} \right) \parallel }_{2}^{2} - {\parallel f_{j}\left( x_{j}^{a} \right) - f_{j}\left( x_{j}^{n} \right) \parallel }_{2}^{2} + margin,0 \right) , \end{aligned}$$

(2)

where $f_{i}(x_{i}^{a}), f_{i}(x_{i}^{p}), and~f_{j}(x_{j}^{n})$ denote the iris feature templates obtained from the i-th client and the j-th client, respectively, and the margin denotes the set interval size. Fed-Triplet combines the advantages of federated learning with Triplet loss to acquire nonidentical representations from other clients, enhancing the robustness of the iris representations. The experimental results on several iris datasets demonstrate the superiority of the FedIris method. The proposed method provides a useful template for subsequent research on federated iris recognition. In 2023, Gupta et al. (2022), noting the sensitivity of iris data, developed a privacy-preserving, high-performance convolutional neural network model with federated iris recognition. The model implements distributed iris recognition from the perspective of privacy-preserving security.

• b. Performance

The commonly used datasets for iris recognition include CASIA lamp (Qiu et al. 2007), CASIA thousand (Zhang et al. 2010), CSIR (Zhang et al. 2018), and ICE (Phillips et al. 2008). Moreover, the equal error rate (EER) is used as a verification metric, which is the rate of error determined by a threshold that yields an equal false negative rate and false positive rate.

Since the introduction of federated learning in biometric recognition fields (other than face) has been relatively limited, few methods exist. Therefore, in this paper, the performances of representative methods in these biometric recognition fields are unified and organized in Table 4.

Table 4 Performance of several federated biometric recognition methods

As shown in Table 4, the method in (Luo et al. 2022) achieves excellent results on four iris datasets. The EERs for the CASIA lamp, CASIA thousand, CSIR, and ICE datasets reach 6.06, 1.41, 6.41, and 1.91, respectively. Notably, the results outperform those of centralized training, which indicates that the FedIris method demonstrates superior performance.

3.1.3 Federated palmprint recognition

• a. Methods

Palmprint recognition, as a popular biometric recognition technique, has received widespread attention. In 2020, Shao and Zhong (2020) proposed the application of FL in palmprint recognition, and to our knowledge, this is the first study of federated palmprint recognition. In this work, a new federated hash learning (FHL) method is proposed for distributed palmprint recognition. The findings of this study serve as a reference for subsequent work on federated palmprint recognition and even other biometric fields. In 2023, Yang et al. (2023) focused on solving multispectral palmprint recognition and proposed a physics-driven spectrum-consistent federated learning method for palmprint verification (PSFed-Palm). This method utilizes the inherent physical properties of different wavelength spectra to classify clients based on the wavelength range of local spectral images. Furthermore, anchor models for short and long spectra are introduced to restrict the optimization direction of the local model. A classical proximal loss (Li et al. 2020) is used to optimize the model, which is specifically shown in Eq. 3.

$$\begin{aligned} L_{prox}\left( {\theta _{local},\theta _{A}} \right) = \frac{\mu }{2}{\parallel \theta _{local} - \theta _{A} \parallel }_{2}^{2}, \end{aligned}$$

(3)

where $\theta _{local}$ and $\theta _{A}$ denote the parameters of the local and global models, respectively, and $\mu$ denotes the factor setting in advance.

• b. Performance

The commonly used publicly available datasets for palmprint recognition are PolyU (Zhang et al. 2009), XJTU-UP (Shao et al. 2019), and Multi-Spectral (Hao et al. 2008). The ACC and EER are commonly used evaluation metrics in palmprint recognition.

As Table 4 shows, the methods in (Shao and Zhong 2020) and (Yang et al. 2023) were tested on different palmprint datasets. Among them, the approach in (Shao and Zhong 2020) achieved 99.22% accuracy on PolyU and 87.33% accuracy on XJTU-UP. The approach in (Yang et al. 2023) achieved an EER of 1.22% on the Multi-Spectral dataset. Since these methods were not tested with the same dataset and metrics, it is not possible to directly compare their performance. However, it can be noted that method in (Shao and Zhong 2020) achieved good recognition accuracy on the PolyU dataset.

3.1.4 Federated vein recognition

• a. Methods

Finger vein recognition is an emerging biometric recognition technique, and few existing studies have evaluated combining finger vein recognition with FL. The only known study was by (Lian et al. 2023) in 2023. In this study, a federated learning-based finger vein authentication framework (FedFV) was proposed to solve the problem of small-sample finger vein recognition while protecting privacy. The flow chart of the method is illustrated in Fig. 6. Based on the introduction of classical FL, FedFV proposed an efficient personalized federation aggregation algorithm to solve the problem of non-independent data among client data. The FedFV, as the first federation finger vein recognition method, is of reference significance for subsequent research.

Fig. 6

Flowchart of the FedFV protocol, where each client divides the model into a federated module for federated learning and a local module for local training only (referenced from (Lian et al. 2023)

• b. Performance

The commonly used datasets for finger vein recognition include HKPU-FV (Kumar and Zhou 2011), SDUMLA (Yin et al. 2011), FV-USM (Asaari et al. 2014), VERA (Tome et al. 2014), and MMCBNU (Lu et al. 2013). The EER is a commonly used metric in finger vein recognition.

In Table 4, the method in (Lian et al. 2023) was tested via many experiments on several finger vein datasets. Among them, FedFV achieved EERs of 0.48%, 1.52%, 0.07%, and 4.54% on four finger veins: HKPU-FV, SDUMLA, FV-USM, and VERA, respectively. Notably, the FedFV algorithm outperforms several state-of-the-art centralized learning methods. This finding suggests that FL is very feasible for application in finger vein recognition. We believe that the success achieved is due to the personalized adaptation of traditional FL, which seems very effective in federated finger vein recognition.

3.1.5 Federaetd voice recognition

In voice recognition, there is no specialized study that combines voice recognition with FL. However, in a recent study by (Hosseini et al. 2021), FL is applied to user verification models. In this study, the authors addressed the problem of training user authentication models in a federated setting by proposing the Federated User Verification (FedUV) framework. The proposed method was validated in three separate domains: voice, face and handwritten data. Although this was not a specialized study on federated voice recognition, we believe that it still provides a reference for subsequent work.

3.1.6 Federated gait recognition

• a. Methods

Gait recognition is a biometric recognition technique that can be used for long-distance identification. Existing gait recognition algorithms depend largely on the size of the gait dataset; large datasets are very difficult to collect. To solve this problem, (Li et al. 2022) proposed a federated gait recognition method (FedGait). Two scenarios using the federated gait recognition method are introduced: an institution-based scenario (IBS) and a device-based scenario (DBS). This is the first study of federated gait recognition, and it provides a benchmark for subsequent research.

• b. Performance

The commonly used publicly available datasets for gait recognition are CASIA-B (Yu et al. 2006b), OU-MVLP (Takemura et al. 2018), ReSGait (Mu et al. 2021), and CASIA-E (Yu et al. 2021). The ACC is commonly used as an evaluation metric for gait recognition. As shown in Table 4, the ACC of gait recognition has yet to improve compared to that of other federated biometric recognition methods. Moreover, gait recognition, as a biometric recognition technique with long-range recognition characteristics, has considerable unexplored potential.

3.1.7 Federated multibiometric recognition

• a. Methods

With the increasing number of studies of FL applications in various biometric fields, some scholars have started to explore the feasibility of federated multibiometric recognition. In addition, in 2023, several breakthrough innovations have occurred. In the health IoT field, Coelho et al. (2023) proposed a secure authentication method combining photoplethysmography and electrocardiogram signals to solve the problem of accessing sensitive data. This method is based on FL implementation and is an attempt to apply FL to multimodal biometric recognition. Similarly, in the field of biometric IoT, Lin et al. (2023) proposed an attention-based multimodal biometric recognition (AMBR) network combined with FL for face and voice multibiometric recognition. The approach trains the AMBR network via FL to overcome the data privacy and regulatory problems in collecting training data. The experiments proved that this research achieved good results. These studies demonstrated the feasibility of applying FL to multibiometric recognition.

• b. Performance

Multibiometric recognition involves several combinations of biometric traits. Hence, many dataset combinations can be selected. In (Coelho et al. 2023), the CapnoBase (Karlen 2021), BIDMC (Pimentel et al. 2016), and TROIKA (Zhang et al. 2014) datasets were selected for testing, where the five metrics ACC, Precision, Recall, FAR, and FRR were used. In addition, in (Lin et al. 2023), VoxCeleb1 (Boulkenafet et al. 2017) and VoxCeleb2 (Zhang et al. 2012) were chosen as the experimental datasets, and the EER was used as the validation metric.

Table 5 Performance of federated multibiometric recognition

The results of two existing federated multibiometric recognition methods are shown in Table 5. Since these results are the first in this field, they provide a baseline for subsequent research.

In summary, all of the above studies of FL applied to biometric recognition have achieved excellent results, and these results have proven the feasibility of federated biometric recognition. However, there are still many problems in terms of performance, efficiency, and security. For performance, compared with other fields in which FL is applied, biometric data exhibit greater heterogeneity due to factors such as collection equipment, collection conditions, and subject differences. This heterogeneity limits the ability of FL to be directly applied to biometric recognition. The ability to solve this heterogeneity problem is important for ensuring the accuracy of federated biometric recognition. For system efficiency, the introduction of FL increases the time and space overhead of terminal systems. This additional overhead may lead to a decrease in efficiency. Therefore, it is also important to achieve a balance between overhead and performance. For system security, FL inevitably introduces security threats such as byzantine attacks and inference attacks while realizing distributed biometric recognition systems. These security threats cause a decrease in system security. However, there is currently very little research on defense against these threats in federated biometric recognition. Finally, in practical applications, edge terminals may experience communication delays or even dropouts, and this problem will directly impact all the devices in FL. Existing studies on federated biometric recognition have been conducted in ideal scenarios (no devices drop out), and no solutions have been designed for this real-world situation. We believe that these problems are well worth exploring in future research. This section provides a comprehensive review of all currently known federated biometric methods, and we believe that all of the above research is of considerable importance for the development and promotion of federated biometric recognition.

3.2 FL for biometric security

Fig. 7

Comparison between a classical federated PAD method and a traditional PAD method

As the research on biometric recognition continues to deepen, there are concerns about the security of biometric data. Among them, biometric presentation attacks (including printing attacks, video attacks, forgery attacks, etc.), as one of the most representative attacks, pose a significant threat to the security of existing biometric systems (George et al. 2019; Ramachandra et al. 2021). The existing methods for biometric presentation attack detection (PAD) are difficult to apply directly in real life due to the problems of data privacy and uneven distribution. To address this issue, scholars have started to apply FL to biometric PAD. A comparison between a federated biometric PAD method and the traditional biometric PAD method is shown in Fig. 7.

Fig. 8

Statistics of existing studies on the application of FL to different biometric presentation attack detection tasks

Table 6 Review of existing research on the application of FL to biometric security

Currently, almost all existing FL applications for biometric security are focused on federated biometric PAD. This section reviews these recent works in detail. Figure 8 shows the statistics of the existing federated biometric PAD studies. All the studies related to federated biometric security are summarized in detail in Table 6.

3.2.1 FL for face security

• a. Methods

Face presentation attack detection (fPAD) is one of the most popular research problems in biometrics. The introduction of FL in fPAD has also been the subject of more research than has other biometric security fields. As shown in Table 5, there are six existing studies on federated fPAD, including studies on a variety of fPAD schemes, such as face printing and video forgery attacks.

In 2021, Shao et al. (2021) first noted the feasibility and necessity of the application of FL in fPAD and proposed a two-stage federated fPAD method. In addition, in 2022, these researchers further proposed a more generalized federated fPAD method by studying this problem in more depth (Shao et al. 2022). Rui Shao et al., as the first researchers to propose this concept, received attention from other researchers in fPAD and even biometric PAD. Hu et al. (2022) proposed a face forgery video detection framework, which marked the first time that FL was applied to a face forgery video detection task. In 2023, an increasing number of scholars were inspired to explore federated fPAD tasks in depth. For instance, Liu et al. (2023) proposed a new generalized residual FL for a face forgery detection framework. Sarwar (2023) utilized a local binary pattern or Gabor filter to extract features and proposed a hybrid ResNet50-SVM-based FL model. In addition, Akimoto et al. (2023) proposed a federated split learning with an intermediate representation sampling framework.

In addition to the research on the application of FL to fPAD, some researchers have explored other security problems in federated face recognition. For example, Meng et al. (2022) proposed the PrivacyFace framework to solve the privacy-utility paradox, which improves the security of federated face recognition by propagating auxiliary information between clients. Niu and Deng (2022) introduced the FedGC framework to solve the privacy leakage problem to facilitate federated face recognition. Zheng et al. (2022) proposed a method to enhance user data privacy in federated face recognition and to solve the problems of a single point of failure and insufficient client incentives. In reference (Zheng et al. 2022), blockchain technology was innovatively introduced to achieve a decentralized federated face recognition framework. Moreover, in terms of face image privacy protection, Yang et al. (2021) proposed a face image privacy protection method based on FL and aggregate modeling. The method enables each client to generate private face images with high portability and utility. This is the first method that applies FL to face image privacy protection. These methods effectively improve the security of federated face recognition and provide valuable references for subsequent research.

• b. Performance

Commonly used datasets for federated fPAD research include Oulu-NPU (Boulkenafet et al. 2017), CASIA-MFSD (Zhang et al. 2012), Idiap Replay-Attack (Chingovska et al. 2012), and MSU-MFSD (Wen et al. 2015). The half total error rate (HTER), area under the curve (AUC), and EER are used as metrics to validate performance. Here, HTER denotes half of the sum of the false acceptance rate and false rejection rate and AUC denotes the coverage area of the ROC curve.

Table 7 Performance of federated fPAD

As shown in Table 7, the methods in (Shao et al. 2021; Akimoto et al. 2023; Shao et al. 2022) were tested on the same four datasets. Based on these results, reference (Akimoto et al. 2023), as the most advanced research in federated fPAD, shows performance superiority over the other methods. Compared with the approaches in (Shao et al. 2021; Akimoto et al. 2023; Shao et al. 2022) reduces the HTER by 16.75% and 12.16% and improves the AUC by 14.39% and 8.54%, respectively, on average, on the four datasets. Moreover, in these studies, federated fPAD methods were compared with traditional fPAD methods, and the results all showed that the application of FL in fPAD can effectively improve the accuracy of detection models where these methods are better than traditional methods. Existing research on the application of FL to face recognition-related security has focused mainly on federated fPAD tasks. As an emerging research direction, the above research results play a crucial role in promoting the exploration of FL application for face recognition security. Notably, the number of studies in this area has been on an upward trend since 2021. This finding demonstrates that increasing attention has been given to this emerging field.

3.2.2 FL for fingerprint security

• a. Methods

To date, no research has been focused only on federated fingerprint recognition and related security fields. Only one recent study by Sarwar (2023) explored the application of FL to both fPAD and fingerprint PAD. For federated fingerprint PAD, SM Sarwar proposed feature extraction via local binary patterns or histograms of oriented gradients to obtain fingerprint features. Afterward, a hybrid CNN-SVM-based FL model is proposed for federated fingerprint PAD tasks. We consider this an attempt in this field, and it has reference value for subsequent research.

• b. Performance

In (Sarwar 2023), experiments were conducted using the LivDet dataset (Mura et al. 2015), and ACC was used as a validation metric. With respect to LivDet, the method achieved 93.89% recognition ACC. Compared with four other methods, the proposed method achieved the best result. In reference (Sarwar 2023), numerous and detailed experiments were conducted to verify the validity of the proposed method. However, relatively few evaluation indices and datasets were selected for this method, and these data need to be supplemented by subsequent studies.

3.2.3 FL for voice security

• a. Methods

Voice live detection (VLD) has become a popular topic. FL, as a solution to the problems of data silos and privacy leakage in traditional VLD methods, has received increased attention from researchers. In 2023, Zong et al. (2023) successfully applied FL to the VLD task for the first time. In this study, a novel word-level VLD framework, called FL-VLD, based on asynchronous FL, is proposed. FL-VLD solves the data siloing problem in VLD scenarios without threatening users’ privacy. We believe that the success of this research has greatly motivated researchers in the field and has made an excellent contribution to the promotion of FL in VLD.

• b. Performance

Table 8 Performance of Federated VLD

In the literature (Zong et al. 2023), experiments were conducted using two datasets, POCO (Akimoto et al. 2020) and ASV (Wang et al. 2020), and the ACC was used as an evaluation metric. The experimental results are shown in Table 8.

In summary, FL applications for addressing security issues in the face, fingerprint, and voice fields have attracted the attention of researchers. Especially in fPAD, there are now six studies addressing this issue. However, in other biometric fields, the application of FL for security enhancement has not yet been explored. Therefore, researchers in related fields can focus on exploring these gap areas in the future.

3.3 FL for other biometric fields

Due to the specificity of biometric data, the problem of difficult data collection also exists in other biometric fields. Therefore, the success of applying FL to biometric recognition has also attracted attention in these fields, which are related to biometrics. We believe that the ideas and methods of introducing FL to process biometric data can provide references and help to guide future researchers in biometrics. Therefore, we also review these techniques in this section.

Table 9 Review of FL applied to other biometric field studies

Table 9 shows the currently available applications of FL in other biometric fields. In the field of biometric quality assessment, Tianyi et al. (2022) extended the two-stage framework, which is widely used in full-reference image quality assessment, by improving the traditional methods with DL methods and proposed a nonlinear two-stage framework based on FL. In face pain detection, Rudovic et al. (2021) proposed a personalized FL-based pain detection method for face images. The method is based on a modification of the traditional FL method but applies it to the task of face pain detection with additional data protection. This is the first study in this area that utilizes FL, and the findings of this study have considerable reference value for subsequent related studies. In speech recognition, He et al. (2021) proposed a federated data architecture based on blockchain and regulated the process of exchange at the feature level to promote distributed machine learning. In voice conversion, Hirai et al. (2023) proposed a method to train a many-to-many voice conversion model that additionally learns the user’s voice while protecting the privacy of user data. Jorge et al. (2022) introduced FL to address the difficulty and privacy of gait data collection in detecting freezing of gait in Parkinson’s disease patients.

Currently, very few studies on FL have been performed in biometrics-related fields. These studies are scattered in various fields and have not yet formed a complete system that can be practically applied. Therefore, there are still great opportunities for exploration in these fields. This paper reviews these studies and hopes to assist new researchers in gaining a quick understanding of these fields.

4 Comparison and analysis

Fig. 9

Taxonomy of federated biometrics

In Sect. 3, we provide a comprehensive review of the existing work on federated learning methods for biometrics and the related fields. To further summarize and compare these studies, we categorize them in this section, as shown in Fig. 9. Furthermore, we analyze the key issues, core techniques, and other aspects of such studies.

As shown in Fig. 9, we categorize the existing studies into three categories, FL for biometric recognition, FL for biometric security and FL for other biometric applications, depending on their target tasks. For each category, unlike in Sect. 3 where we classified the methods based on their types of biometric traits, we classify the works based on their technical details to better compare them and to summarize and analyze the key issues in the field.

4.1 FL for biometric recognition

In this section, we categorize the existing methods into those that apply to unimodal tasks and those that apply to multimodal tasks. Additionally, we classify the unimodal methods according to their technology types: those that apply traditional FL frameworks, use embedding vectors, use unsupervised learning, implement personalized frameworks, and add blockchains. Furthermore, we analyze the key issues faced by these methods.

4.1.1 Unimodal methods

The integration of FL into the field of unimodal biometric recognition has been the focus of research in recent years. Many existing studies have incorporated the traditional federated learning framework into the biometric field to form distributed biometric recognition methods and to solve the privacy protection problem faced by the traditional centralized system. For example, in the field of facial recognition, Bai et al. (2021), Kim et al. (2021), and Shang et al. (2022) all conducted research based on traditional federated learning. Similarly, this style of directly introducing the traditional FL framework can be found in the fields of iris recognition (Li et al. 2022), etc. These studies, as pioneers in this domain, have indeed laid the cornerstone of federated biometric recognition. However, since most of these methods are directly introduced to the traditional FL framework, room for improvement remains with regard to their recognition accuracy. Typically, the achieved recognition accuracy decreases when switching from a centralized system to a distributed system. This phenomenon is due to the fact that centralized training methods are able to collect all the necessary data for gradient updating, which enables them to better train their models. In contrast, in distributed systems based on FL, the available data are distributed across clients, and data heterogeneity is present between the clients, which affects the training process. To solve the impact of non-IID data, researchers have improved the FL framework to improve its recognition accuracy through approaches such as implementing a personalized FL framework, introducing embedded vectors, and so on.

To address such data heterogeneity issues and attain improved recognition accuracy, Liu et al. (2022) proposed a personalized FL framework for facial recognition that jointly optimizes personalized models for the corresponding clients by decoupling the feature customization process. As a personalized FL method, their approach is more adaptable to the heterogeneity of the data among different clients than is a method that directly applies traditional federated learning (such as the technique in (Bai et al. 2021)), thus achieving better recognition performance. Similarly, in the field of federated finger vein recognition, Lian et al. (2023) implemented a personalized FL framework that performs finger vein recognition by aggregating differently weighted models for its clients. Although both (Liu et al. 2022) and (Lian et al. 2023) implemented personalized FL frameworks, the former performed personalized FL based more on local design, while the latter relied more on the server-level aggregation of personalized models. However, both experiments showed that a personalized framework makes it possible to overcome the data distribution differences between clients and thus achieve higher accuracy.

Another solution is to achieve improved accuracy by transferring auxiliary embedding vectors. (Hosseini et al. 2021), (Luo et al. 2022), and (Liu et al. 2022) are some of the studies that used embedding vectors for this purpose. Among them, (Luo et al. 2022) and (Liu et al. 2022) employed embedding vectors for communication. (Luo et al. 2022) attempted to employ the iris template as a communication carrier and formulated federated triplets for knowledge transfer. This approach is very different from personalized methods. Typically, this embedding vector-based approach does not require many localization settings. (Hosseini et al. 2021) also developed an approach related to embedding vectors, but this method aims more at solving the problem of training user-verified models in a federated setting.

In addition, several studies have improved traditional FL frameworks for applications in biometrics, aiming to solve problems related to edge computing, training overhead, and secure communication, etc. For example, (ZHuang et al. 2021) proposed a novel unsupervised federated facial recognition approach. Compared to other supervised approaches, the method in (ZHuang et al. 2021) does not require labels for training, which enables it to be applied to many edge terminals that lack labeled data or have higher privacy needs. Additionally, (Zheng et al. 2022) addressed the single-point-of-failure and user incentive problems by incorporating blockchain techniques into a federated biometric approach. In this new framework, the use of blockchain instead of a central server is distinct from the traditional FL strategy and effectively relieves the data processing pressure imposed on the central server. These methods extend the usability and security of federated biometric systems beyond that provided by the direct introduction of FL methods, such as the approach in (Bai et al. 2021).

In summary, the key issue in the field of federated unimodal biometric recognition has now begun to shift from the introduction of traditional FL frameworks to the realization of FL techniques with higher accuracy, more security, or less overhead. We believe that this indicates that the field is receiving increasing attention from researchers and is in a phase of rapid development.

4.1.2 Multimodal methods

Compared to the research on unimodal methods, relevant research on FL applications involving multimodal biometrics is scarce. The existing federated unimodal biometrics methods also cannot be directly migrated to the multimodal field. This is due to the fact that multibiometric recognition methods require additional modules such as multimodal feature extraction and feature fusion modules.

Currently, the only existing federated multibiometric recognition studies are those of (Coelho et al. 2023) and (Lin et al. 2023), both of which were conducted in the field of the Internet of Things (IoT). Coelho et al. (2023) introduced FL on the basis of implementing multiple biometrics photoplethysmography and electrocardiogram signals. However, this work was only a simple introduction to FL, so the recognition accuracy of their approach still needs to be improved. Similarly, Lin et al. (2023) also involved a simple application of FL in a multibiometric scenario. Compared with Coelho et al. (2023), Lin et al. (2023) improved the feature fusion module by proposing an attention-based multimodal biometric recognition network. However, these methods still have numerous shortcomings in comparison with unimodal federated biometric recognition methods. For example, they cannot address the heterogeneity between clients as the methods in (Liu et al. 2022) and (Bai et al. 2021) can, nor can they solve the single-point-of-failure problem as the approach in (Zheng et al. 2022) does.

Notably, several research gaps remain in the field of federated multibiometric recognition. In fact, multibiometric data are more difficult to obtain than single-biometric data, so the application of FL in this field is an even more urgent need. We believe that there is a great research potential in the federated multibiometric recognition field.

4.2 FL for biometric security

In this section, we categorize the existing work into two aspects of FL in biometric PAD and FL for biometric privacy protection. And, a detailed categorization is done based on the key techniques used in each study.

4.2.1 FL for biometric PAD

Due to the characteristics of biometric presentation attack data, such as their uneven distributions and direct acquisition difficulty, the application of FL in this field is particularly important. Therefore, FL research for biometric PAD has received more extensive attention.

Many studies have been conducted in this field. (Shao et al. 2021) was the first study to introduce FL to biometric PAD. In this study, Shao et al. proposed a two-stage approach to solve the uneven data distribution problem. In fact, their method is a test-time FL framework. This method of introducing FL at test time can be a good solution for the biometric PAD task involving edge terminals with missing data. However, this method imposes certain computing power requirements on the edge terminal. Thus, it is difficult to apply this approach to certain terminals for which inference-based training is not possible.

To solve this problem, the concept of training generalizable models that can be directly applied to edge terminals has attracted the attention of researchers. (Shao et al. 2022) further proposed a federated generalized face presentation attack detection framework based on the previous work. Compared with the previous method (Shao et al. 2021), the new framework with generalization does not need to be trained at the edge terminal. This indicates that the model has a stronger generalization ability than that of (Shao et al. 2021). Furthermore, Liu et al. (2023) proposed a federated learning framework with generalizability. Compared with the approach of (Shao et al. 2022), which utilizes deep images to perform assisted training and recognition, the method of (Liu et al. 2023) is more feasible via a variational autoencoder and can learn robust and discriminative residual feature maps.

Additionally, edge terminals may have extremely limited computing power levels, preventing them from running full models. The framework based on a split FL framework proposed by Akimoto et al. (2023) is highly applicable. Unlike methods such as those of (Shao et al. 2022) and (Liu et al. 2023), the method presented in (Akimoto et al. 2023) deploys a portion of its model to the server, which is not only beneficial for enabling the server to perform domain alignment to achieve enhanced generalization but also reduces the overhead of the edge terminals. We believe that this split FL solution is also applicable to the federated biometric recognition field, for which no similar method is currently available.

Overall, FL is currently achieving great success in the field of biometric PAD, and many interesting improvements to the traditional FL framework have emerged. These studies are highly transferrable to the related fields.

4.2.2 Privacy protection

Compared to the research conducted in the field of biometric PAD, relatively few existing FL studies have focused on improving privacy, and we categorize them into two categories: recognition-related privacy and image privacy preservation works.

The first class is the recognition-related privacy issues, i.e., the consideration of privacy issues while implementing a federated biometric recognition system. (Meng et al. 2022), (Niu and Deng 2022), and (Zheng et al. 2022) all implemented federated biometrics while further considering privacy issues. Among them, unlike the other approaches, the method of (Zheng et al. 2022) is decentralized through blockchain technology. This decentralized structure is naturally immune to attacks from malicious servers. Additionally, (Meng et al. 2022) resolved the privacy-utility paradox. This idea of realizing distributed recognition while considering user privacy fits well with the current needs in the field of biometrics.

On the other hand, unlike the above methods, Yang et al. (2021) enables each client to generate private face images with high transferability and practicality. Compared to (Niu and Deng 2022) and (Zheng et al. 2022), (Yang et al. 2021) focuses more on achieving privacy protection for face image generation, transmission, etc. It also presents a solution for introducing FL in the field of face image generation.

Although user privacy and data privacy are particularly important in distributed biometric recognition systems, there is still very scarce research on this topic. Privacy protection issues still need to be explored further than the research on federated biometric PAD.

4.3 FL for other biometric applications

Finally, with the development of federated biometrics, related methods have also been introduced to some areas related to biometrics to address data and privacy challenges. In this section, these methods are categorized into three groups according to their task categories: evaluation, detection and others.

In addition to the recognition task, the estimation task is always the focus of the biometric field. Currently, two assessment tasks are examined in the field of biometrics combined with FL: face pain estimation (Rudovic et al. 2021) and biometric image quality assessment (Tianyi et al. 2022). These two studies, as pioneers, introduced FL to their respective tasks and provided excellent references for subsequent studies. Unfortunately, however, (Tianyi et al. 2022) involved only a limited exploration of FL. We believe that it is of considerable importance to introduce the FL framework to the field of image quality assessment and to improve it. Unlike it, (Rudovic et al. 2021) not only introduced the FL framework but also implemented a novel personalized federated deep learning approach. This enabled the developed approach to achieve better performance than that of the standard FL algorithm.

Additionally, research incorporating FL is beginning to emerge in the field of detection and other related areas. Jorge et al. (2022) proposed an FL-based healthcare application that enables wearable devices to detect FoG symptoms. In our opinion, this is quite an interesting study, which addressed the gait detection task with wearable devices via FL. In addition, two other speech-related studies (He et al. 2021; Hirai et al. 2023) have addressed the problems encountered when using FL frameworks. However, again, all three above frameworks are direct applications of traditional FL, thereby possessing much room for exploration.

In summary, we horizontally and vertically compare the existing methods in this section. Furthermore, we analyze and summarize the key issues related to federated biometrics. Notably, federated biometrics has become a hot research topic. However, it still has much content that has not been research and issues that are worth exploring. We will discuss the future research directions in this field in the subsequent section.

5 Opportunities and challenges

Fig. 10

Chord diagram of FL applied in biometrics

The application of federated learning in biometric recognition and related fields has become the latest research hotspot. In recent years, an increasing number of new researchers have started to devote themselves to exploring this field. To better help future researchers quickly understand existing research in this field, we statistically present these studies in this section and analyze the possible challenges and potential opportunities that federated biometrics may face in the future.

5.1 Research interests of federated biometrics

Figure 10 shows the statistics of existing research directions on FL applications to biometrics. In this paper, the statistics of the existing research can be categorized into the following five research directions: FL applied to biometrics, FL applied to solving biometric security problems, FL applied to solving biometric-related fields, personalization exploration of FL in biometrics, and generalization exploration of FL in biometrics. The first three directions aim to explore the application of FL to original problems in the biometric field, while the personalization and generalization directions aim to explore the modification of the FL framework to adapt to specific biometric-related tasks.

Among the eight biometrics in Fig. 10, the combination of face and FL has received the most attention, and the number of studies in this field is much greater than that in other fields. Among the studies related to federated faces, the application of FL for face recognition and its security issues has received the most consideration from researchers. Notably, only the federated face field has been explored in all five research directions mentioned in this paper. As FL technology has made breakthroughs in face recognition and related areas, it has also gained attention for use in biometrics, such as those of the iris, voice, gait, and vein. We believe that the application of FL in these emerging biometric fields will be the target of subsequent research.

Fig. 11

Taxonomy of FL applied in biometrics

Figure 11 further illustrates the representative results achieved in the five directions of FL application to biometrics. Among the five research directions, to date, the application of FL to solve the problems of biometric recognition and security has received the most attention. This is because the exploration of biometric recognition and its security has been the most critical problem in biometrics. Moreover, additional research on FL for biometric recognition has attempted to explore personalized FL to improve the adaptability of FL frameworks to highly heterogeneous biometric data. Previous studies have demonstrated that personalization can effectively address the heterogeneity between different datasets when FL is introduced into biometric recognition to improve the accuracy of recognition. Moreover, in the biometric security field, existing studies have explored the use of FL to improve model generalization. This is because these studies in the biometric security-related field focused on introducing FL into the PAD task. In this particular task, the generalizability of the model is particularly important because the test data are usually unavailable. Research in this field has shown that the introduction of FL improves the generalization of biometric PAD models to obtain more robust and accurate generalized models. Both personalization and generalization have been explored with the aim of enabling FL to achieve better performance when applied to biometrics and related fields. These changes and explorations of the FL framework have made it applicable to specific tasks in biometric recognition and important for the development of federated biometrics.

Federated biometric recognition, as an emerging cross-cutting field, still has many unexplored issues. In the remainder of this section, we analyze the opportunities and challenges from four aspects, namely, (1) applying FL to solve biometric recognition problems, (2) solving security problems when FL is applied to biometric recognition, (3) exploring the personalization and generalization of FL in biometric recognition and (4) applying FL to solve biometric recognition problems.

5.2 FL for biometric recognition

Fig. 12

A taxonomy of FL applied in biometric recognition from 2021-2023

Figure 12 illustrates the research statistics of FL applied to biometric recognition over the last three years. As shown in Fig. 12, in 2021 and 2022, more than half of the FL studies involved face recognition. However, in the latest research, increasing amounts of work on FL applied to non-face recognition fields, such as iris and finger vein recognition, has emerged. Especially in multibiometric recognition, the introduction of FL can greatly relieve the problem of acquiring multibiometric data, which is of great practical significance. However, research combined with FL has not yet been conducted in a minority of biometric fields, such as palm vein recognition and ear recognition. This finding indicates that there are still many problems in federated biometric recognition that have not been fully investigated, and we believe that future research will focus on the following aspects.

• (1) How to utilize FL to further improve the accuracy of biometric recognition systems. In biometric recognition, recognition accuracy is one of the most important evaluation indices. When combined with FL, the accuracy of a biometric recognition system depends on both the local recognition algorithm and the federated learning algorithm. Thus, it is especially important to improve biometric recognition and federated learning algorithms so that they can collaborate and complement each other to realize improved performance. We believe that continuously improving the recognition performance of federated biometric systems to approach or even surpass centralized learning systems will be a very important research direction in the future.

• (2) How to optimize the time, space, and communication costs of federated biometric systems. With the introduction of FL, these biometric systems for performing federated learning have been converted from traditional local learning to multiend collaborative learning to improve recognition performance. However, in the process of multistage collaboration, additional overhead such as communication, waiting and storage are incurred. In practice, some edge devices may even experience dropouts and delays. This will lead to the other devices in FL being affected and can drastically reduce learning efficiency. However, the exploration of these problems in the field of federated biometric recognition currently has been very limited.

• (3) How to introduce FL to solve some minor and emerging biometric recognition fields. In face recognition, the introduction of FL has largely solved the contradiction between data privacy and accessibility. However, in some emerging biometric recognition fields, it is more difficult to obtain biometric data, and the amount of data is limited. This also leads to greater requirements for the application of FL in these fields. However, to date, there has been almost no research combining FL with these emerging methods, such as palm vein recognition and ear recognition. We believe that the combination of these fields with FL will be the focus of future investigations.

• (4) How to utilize FL to solve multibiometric recognition problems. Compared to other fields where FL is introduced, multibiometric data are unique to the biometric recognition field and can be obtained by collecting more than two human biometric traits simultaneously. Since multibiometric recognition often involves operations such as extracting features from multiple models and performing feature fusion, existing FL algorithms cannot be directly applied in the field. Moreover, there have been very few studies related to federated multibiometric recognition. Thus, this field still has very large research potential.

Overall, there are many problems to be explored in the federated biometric recognition field. As an identification technology, biometric recognition has begun to be popularized, and the introduction of FL to improve its performance in terms of security, usability, recognition accuracy and efficiency is a hot research topic.

5.3 Security of federated biometrics

Security-related research on federated biometric recognition can be divided into two categories. The first category includes the application of FL to solve the inherent security problems in biometric recognition, e.g., biometric PAD and biometric data transmission problems. The second category is the security risks caused by the process of introducing FL, e.g., byzantine attacks and malicious server attacks. Therefore, future research related to security in federated biometric recognition can be divided into two groups.

• (1) How to apply FL to solve the inherent security problems in the biometric recognition field. Here, FL is introduced to solve specific tasks. In particular, some progress has been made in the application of FL to biometric PAD. For example, with the introduction of FL in fPAD, the detection accuracy has substantially improved. However, in other security-related fields in the biometric field, such as biometric template protection and biometric privacy leakage, research involving FL is rare. Therefore, we believe that filling these gaps is quite important for future research.

• (2) How to solve the security problems inevitably brought about by the introduction of FL. With the introduction of FL, the biometric recognition system has changed from a local learning system to a distributed learning system, which inevitably leads to security threats in the distributed mode. These security threats can affect the security of devices participating in FL, e.g., byzantine attacks, inference attacks, backdoor attacks, and malicious server attacks (Mothukuri et al. 2021). Therefore, it is important to protect against these security threats while introducing FL. However, most of the current research on these security threats has concentrated on the traditional FL field. In the field of federated biometric recognition, almost no research has been carried out to address these problems. It is also highly valuable to determine whether it is possible to utilize the characteristics of the biometric recognition field to defend against these security attacks. Research on this topic will not only contribute to the development of federated biometric recognition but also be important for FL security research.

5.4 FL for personalized or generalized biometrics

Research on personalization and generalization related to FL has attracted increased interest. When FL is applied to biometrics, the study of personalization and generality becomes particularly important due to the non-independent and identically distributed (non-IID) characteristics of biometric data. Exploring the personalization and generality of FL frameworks are two effective technical routes for solving the problem of non-IID data and improving the accuracy of biometric recognition systems.

• (1) Personalization exploration usually involves designing private local modules for each client to enable the FL framework to adapt the local data distribution. In federated biometric recognition, some studies aim to overcome the non-IID characteristics of biometric data through personalization settings, which improve the recognition accuracy of biometric recognition systems. In federated face and finger vein recognition, studies have attempted to relieve the impact of the heterogeneity of biometric data on performance through personalized FL. These personalization-related studies have resulted in varying degrees of performance improvement for local clients. However, space still exists for improving the recognition performance. Moreover, in many other biometric recognition fields, the non-IID problem involving data has not been studied or solved, and personalization-related studies still have great research value.

• (2) Generalization exploration involves improving the generality of models through FL. Improving model generalizability is highly important for many task-specific tasks in certain biometrics fields, such as biometric PAD, which have received much attention. A generalized model can be applied to biometric PAD tasks involving uneven data distributions and non-IID characteristics. However, in other fields of biometrics, FL generalization has been less explored. For example, most of the current research on biometric recognition has focused on exploring personalized settings to improve recognition performance. However, we believe that generalizable models are also of great research value. Therefore, there is still much research significance and potential for exploring model generalizability in biometric recognition and related fields.

Table 10 Summary of the opportunities and challenges

5.5 FL for other biometric fields

As FL has continued to gain popularity in biometric recognition, researchers in related fields have begun to notice FL techniques. These fields do not directly overlap with biometric recognition but are related to it to different degrees. Currently, relatively few studies have combined these fields with FL, and we believe that the introduction of FL in the following fields has great research significance.

• (1) The application of FL in the field of biometric quality assessment. A biometric quality assessment enables the quality of biometric traits to be assessed to ensure greater recognition accuracy. Compared with the field of biometric recognition, biometric quality assessment often requires a large amount of high-quality and low-quality data, and the acquisition of such specific biometric data is quite difficult. Therefore, introducing FL into biometric quality assessment is highly important for overcoming the problem of lacking data.

• (2) The application of FL in biometric preprocessing. Among the four key steps of biometric recognition mentioned above, biometric data preprocessing is one of them. Effective preprocessing of biometric data can greatly improve the accuracy of subsequent recognition. Most of the current preprocessing methods are designed for local data and are not generalizable. However, the introduction of FL can provide a potential solution to this problem. Therefore, the combination of FL with biometric preprocessing has great research value.

• (3) In addition, there are many biometrics-related fields where it is difficult to collect a large amount of specified biometric data. For example, facial masking recognition, biometric image restoration, and so on. The lack of data has led to limitations in the application of these fields in practice. Therefore, introducing FL into these fields has become especially critical.

The application of FL in biometric-related domains can not only address data and privacy problems but also improve the feasibility of these methods for edge devices. However, FL applications in these fields are currently very sparse. Future studies in these fields can further complement the system of federated biometrics, which is highly important to the whole field of biometrics.

In summary, the combination of FL and biometric recognition, as an emerging research hotspot, still has many opportunities and challenges. These opportunities and challenges are briefly summarized in Table 10. And, we believe that future research can explore the accuracy, applicability, safety, etc. In this paper, we hope to contribute to promoting subsequent research and filling research gaps by summarizing the existing research developments.

6 Conclusion

The application of FL in biometric recognition and related fields has become a vibrant and booming research topic. From the perspective of protecting biometric data security and user privacy, the introduction of FL solves the problems in the existing biometric-related fields caused by the difficulty of data collection and the restriction of user privacy. In recent years, FL has gradually received increasing attention in the field of biometrics (including biometric recognition, biometric PAD, etc.). In this paper, we comprehensively review and summarize these recent advances. To our knowledge, this covers almost all the existing studies on federated biometric recognition. We also summarize the future research directions and possible challenges in this field. For example, the security concerns of distributed biometric systems are challenges that must be faced, and more personalized and generalized frameworks can facilitate the development of this field. We hope that this survey can provide valuable reference and help for researchers in this field.