Finger-vein authentication based on deformation-tolerant feature-point matching

Abstract

A novel method for finger-vein authentication based on feature-point matching is proposed and evaluated. A finger-vein image captured by infrared light contains artifacts such as irregular shading and vein posture deformation that can degrade accuracy of finger-vein authentication. Therefore, a method is proposed for extracting features from vein patterns and for matching feature points that is robust against irregular shading and vein deformation. In the proposed method, curvature of image-intensity profiles is used for feature point extraction because such image profiles are a robust feature against irregular shading. To increase the number of feature points, these points are extracted from any positions where vein shape is non-linear. Moreover, a finger-shape model and non-rigid registration method are proposed. Both the model and the registration method correct a deformation caused by the finger-posture change. It is experimentally shown that the proposed method achieves more robust matching than conventional methods. Furthermore, experiments on finger-vein identification show that the proposed method provides higher identification accuracy than conventional methods.

1 Introduction

Biometrics is a technology for uniquely identifying humans on the basis of intrinsic physical features such as our fingerprints, iris, or face. Compared with the conventional biometric characteristics, finger-vein patterns exhibit some excellent advantages in real application. Each finger vein has a unique vein pattern [1], which is found inside the body and is extremely difficult to forge. The use of infrared light allows for non-invasive, contactless imaging that ensures the user experience is both convenient and clean. Moreover, vein patterns are stable and clearly defined, allowing the use of low-resolution cameras to capture vein images for small-size, simple data image processing [2]. Finger veins are, therefore, used for authentication.

As shown in Fig. 1, near-infrared light passes through the tissue of a finger. The transmitted light is captured by projecting the surface of the finger onto a 2D plane. However, the light is blocked by hemoglobin in the blood vessels, so the veins near the surface of the skin appear as dark shadow lines in a captured image. A person can be identified on the basis of that finger-vein image.

Finger-vein authentication has some issues and can still be improved in some way. Irregular shading and deformation of the vein patterns are the main causes of performance degradation in finger-vein authentication. The irregular shading can be caused by a shift of the finger position because the thickness of the finger at the illuminated point varies with the shift. Finger posture also varies in three dimensions, and the change in finger posture deforms the vein patterns. Both the irregular shading and the deformation of vein patterns prevent robust feature extraction/matching and reduce the accuracy of the finger-vein authentication. To address those issues, in the present study, a method is, therefore, proposed for finger-vein authentication that is robust against irregular shading and deformation of vein patterns.

Fig. 1

Principle of personal identification using finger-vein images

2 Finger-vein authentication

2.1 Related work

A vein authentication system generally consists of two processes: feature extraction and matching. Extraction of finger-vein patterns has attracted a lot of interest [3–6]. Vein images contain irregular shading and noise. It is preferable to extract effective vein patterns from these unclear images. Miura et al. extracted vein patterns using repeated line tracking [3]. They also proposed a vein extraction method that is robust against irregular shading [4]. This method employs the curvature of an image profile to extract vein patterns. Kumar et al. used Gabor filters and morphological processing for extracting vein patterns [5]. Yang and Yang extracted directional information concerning vein patterns in addition to the location of vein patterns [6]. All the above-mentioned methods utilize “template matching” for matching extracted vein patterns and attained high accuracy. However, these matching methods are essentially sensitive to vein deformation.

When qualities of finger-vein images are low, segmentation errors can occur during the extraction (binarization) of the vein patterns. To solve this problem, [8] proposed a method for finger-vein recognition using a local binary pattern (LBP) and a local derivative pattern (LDP). Rosdi et al. [9] applied a new variant of LBP called local a line binary pattern (LLBP) to finger-vein authentication. However, these methods do not solve the problem of vein deformation either. To improve the visibility of finger-vein imageries in the pre-processing of the feature extraction method, scattering removal and vein enhancement methods have been proposed [10]. To localize finger-vein ROI of a feature extraction method, scattering removal and vein ridge enhancement methods have been proposed [7].

Other applied point-matching methods including Minutiae [11, 12] and SIFT [13–17] for vein authentication. A point-matching method is generally robust against deformation because each point corresponds to another point independently. These methods consist of two processes: feature-point detection and feature-point matching. In the first process, feature points are detected from binarized vein patterns or gray-scale images, and feature descriptors are calculated at each point. In the second process, each feature point corresponds to a feature point. A matching score is then calculated on the basis of the ratio of the number of pairs of matching points and the number of all feature points. Yu et al. [12] utilizes MHD [18] for calculating the matching score.

Minutiae matching is commonly used for finger print authentication. Minutiae points are the major features of fingerprint ridges such as the bifurcation point and end point. When it is applied to vein images, it suffers a problem that only a few feature points are detected from a vein image (Fig. 2b), because typical vein patterns have fewer minutiae points in vein images.

The SIFT method for feature-point detection [16, 17] can extract more feature points than minutiae extraction. SIFT methods detect most feature points from boundaries of veins (Fig. 2c). These boundaries are blurred and unclear due to light scattering in the skin layer. Changes of intensity are large near these boundaries, so they easily cause false detection of feature points.

As for fingerprint matching, registration aligns two fingerprints as much as possible to be robust against deformation of fingerprints. Some algorithms use a non-rigid transformation for registration. [22] proposed a minutiae matching method that describes elastic deformations in fingerprints by means of a thin-plate spline model, which is estimated using a local and a global matching stage. In finger-vein-point matching, this non-rigid registration has not been used until now.

Fig. 2

Problems concerning conventional methods for feature-point detection. a Original image. b Minutiae. c SIFT

2.2 Technical issues

As shown in Fig. 3, a finger-vein image contains several kinds of feature points, namely bifurcation points and end points on vein patterns. Because these points vary widely from person to person, they can be used for personal identification.

Fig. 3

Technical issues concerning finger-vein authentication. a Original image in good condition. b Irregular shading. c Deformation of vein patterns

A finger-vein image, however, has the following three undesirable characteristics that could degrade the accuracy of authentication.

• Irregular shading

• Deformation of vein patterns

• Blurring of veins

Irregular shading is caused by biased light distribution in the human body due to changes in finger posture (Fig. 3b). That is, when the relative position between a finger and a light source varies, non-uniform light is distributed in the finger. The resulting irregular shading degrades contrast of the vein image and prevents stable feature extraction.

Finger-vein images captured as a 2D projected pattern of a 3D vein structure contain deformation of vein patterns. The deformation is caused by a change in the spatial posture of the finger such as bending of the finger (Fig. 3c) and rotation around a longitudinal axis of the finger. The deformation degrades the reproducibility of features extracted from vein images; thus, the accuracy of the feature matching is also reduced.

Vein images contain a large amount of blur due to scattering of infrared light inside skin tissues. Because the blur reduces the contrast between a vein and its background, which are the edges of veins, feature points are difficult to extract stably and precisely.

Relationships between the above-mentioned issues and conventional methods are as follows. Miura et al. [4] employed the curvature of an image profile to extract finger-vein patterns and conducted vein-pattern matching by using template matching. This vein extraction method is robust against irregular shading. On the other hand, the template matching is essentially sensitive to deformation.

Point-matching methods such as minutiae methods [11, 12] and SIFT methods [13–17] are generally more robust against deformation than template matching because each point corresponds to another point independently.

However, minutiae methods have a problem that the number of feature points extracted from vein images is quite small, because typical vein images have fewer minutiae points such as bifurcation points and end points. SIFT methods for feature-point detection [16, 17] can extract more feature points. These methods detect most feature points from blurred and unclear boundaries of veins and easily cause false detection due to irregular shading and blurring of veins.

None of the above-mentioned minutiae and SIFT point-matching methods for finger-vein authentication considers the deformation of veins in point-matching process. However, vein patterns are easily deformed by posture change of finger such as finger bending and finger rolling. These large deformations negatively affect these conventional methods.

In this study, a new finger-vein authentication method is proposed to solve the above-mentioned problems. The main contributions of our proposed method can be summarized as follows:

• Feature points are stably detected from any position where vein shape is non-linear.

• A finger-shape model is proposed to effectively correct deformation of veins.

• Non-rigid registration is applied to finger-vein authentication for the first time.

To increase the number of feature points, these points are stably detected from any position where vein shape is non-linear. Feature-point detection utilizes curvatures of an image profile for robust processing against irregular shading. Curvatures are calculated on the basis of eigenvalue analysis. Detected points are located at the center line of vein patterns, and the effect of blurring of vein is relatively small.

In the procedure for calculating feature descriptors, the descriptors robust against deformations of veins are computed on the basis of a combination of vein shape and vein direction. A finger-shape model is defined that represents a deformation due to a “finger rolling” which is a rotation around a longitudinal axis of the finger. The deformation is corrected by using this model.

In the feature-point-matching process, non-rigid registration not used conventionally in finger-vein-matching is proposed to realize an authentication system robust against deformation of vein patterns.

Fig. 4

Block diagram for proposed finger-vein authentication

3 Proposed method

3.1 Computation flow of proposed method

The flow of the proposed feature-point-matching method is composed of the following five steps. The block diagram of the proposed method is shown in Fig. 4.

Step 1 Acquisition of image

Figure 5a,b show the image acquisition device used in the proposed method and its configuration, respectively. To acquire a finger-vein image, a finger is put on the touching parts of the device, and a camera located on the palm side of the hand captures the infrared light transmitted through the finger. The LEDs are arranged on the back of the hand, and the intensity of the LED light is adjusted so that the veins in the vein image can be seen clearly.

Fig. 5

Imaging device. a Appearance of imaging device. b Configuration of imaging device

Step 2 Normalization of image

After the vein image is acquired, the location, rotation angle around the optical axis, and scale of enlargement of the finger are normalized by using the outline of the finger. The original image consists of 240 ${\times }$ 140 pixels with 8 bits per pixel. Each normalized image is clipped from a predetermined ROI of the original image and consists of 220 ${\times }$ 120 pixels. The length of the finger is in the horizontal direction.

Step 3 Pre-processing

A vein image is smoothed and vein patterns are emphasized for accurate feature point extraction. Yang and Yang compute outputs of multi-directional Gabor filtering and then select the largest output to emphasize vein patterns [19]. In a preliminary experiment, however, it was found that the largest output converts the non-linear structure of the vein shape into a linear structure. Feature points can only be extracted from the non-linear structure. To reduce image noise while maintaining the non-linear structure, the mean of the third- and fourth largest outputs is then selected.

Step 4 Feature extraction

Feature points are detected from the pre-processed finger-vein image, and the descriptor is calculated with respect to each point.

Step 5 Matching

Non-linear deformation of vein patterns is corrected by non-rigid registration. After the registration, feature points in one image are matched with the corresponding points in another image on the basis of the feature-point position and descriptors.

Fig. 6

Linear and non-linear shape of vein patterns. a Linear. b Bifurcation. c Non-linear

In the following section, the methods proposed for feature extraction and feature-point matching are explained.

3.2 Algorithm for deformation-tolerant feature-point matching

3.2.1 Feature-point detection using eigenvalue analysis

In a finger-vein image, the vein pattern that appears to have a linear shape is in reality meandering at the local level. We focus on the non-linear shape of the vein pattern. Positions where the vein-pattern shape is non-linear can be candidates of feature points. In the proposed method, feature points are detected from any non-linear-shape part of vein patterns to increase the number of feature points. These feature points are extracted by utilizing intensity profile curvatures that are robust against irregular shading. Curvatures that stand for non-linear-shape points of veins are efficiently calculated on the basis of eigenvalue analysis. As mentioned above, Gabor-based pre-processing is applied to a vein image for accurate feature-point extraction.

Figure 6 shows an example of some vein shapes. The intensity profile curvatures are calculated in the directions indicated by the two-way arrows. As shown in Fig. 6a, when intensity profile A intersects a boundary line between the veins and the background, the curvature value is large. However, intensity profile B runs parallel to the blood-flow direction, and the profile has little luminance fluctuation. Therefore, the curvature value is quite small. In contrast, as shown in Fig. 6b, all directional profiles intersect boundary lines between the veins and the background at the point located in a branch of the veins. Therefore, the values of all directional curvatures are large. At the point where the blood-flow direction changes, the curvature values in every direction are also large (Fig. 6c). The procedure for detecting these points that have large curvature values in all directions is explained as follows.

First, intensity profile curvatures are calculated in all directions for a pixel in a pre-processed vein image. Next, a minimum curvature is selected from all the curvatures calculated in the previous step. Finally, these two processes are conducted with respect to each pixel, and a minimum-curvature map is obtained. An example of a minimum-curvature map is shown in Fig. 7b. As mentioned above, the pixel value of the minimum-curvature map at the linear-shape part of vein patterns is quite small, whereas the value at the non-linear-shape part is large. Feature points can therefore be obtained from the minimum-curvature map (Fig. 7c).

Fig. 7

Feature-point detection. a Original image. b Minimum-curvature map. c Extracted feature points

In the proposed method, a minimum-curvature map is calculated on the basis of eigenvalue analysis. f(x, y) is taken as a vein image. The second derivative of image f(x, y) in any direction $\theta$ is expressed as

$$\begin{aligned}&\frac{\partial ^2 f}{\partial x^2}\mathrm{cos^2}\theta + 2 \frac{\partial ^2 f}{\partial x \partial y}\mathrm{cos}\theta \mathrm{sin}\theta + \frac{\partial ^2 f}{\partial y^2}\mathrm{sin^2}\theta \nonumber \\&\quad =\left( \begin{array}{cc} \mathrm{cos}\theta&\mathrm{sin}\theta \end{array} \right) \left( \begin{array}{cc} \frac{\partial ^2 f}{\partial x^2} &{}\quad \frac{\partial ^2 f}{\partial x \partial y} \\ \frac{\partial ^2 f}{\partial y \partial x} &{}\quad \frac{\partial ^2 f}{\partial y^2} \end{array} \right) \left( \begin{array}{c}\mathrm{cos}\theta \\ \mathrm{sin}\theta \end{array} \right) , \end{aligned}$$

(1)

where the Hessian matrix of the second derivatives can be defined as

$$\begin{aligned} H(x,y) = \left( \begin{array}{cc} \frac{\partial ^2 f(x,y)}{\partial x^2} &{}\quad \frac{\partial ^2 f(x,y)}{\partial x \partial y} \\ \frac{\partial ^2 f(x,y)}{\partial y \partial x} &{}\quad \frac{\partial ^2 f(x,y)}{\partial y^2} \end{array} \right) . \end{aligned}$$

(2)

Hessian matrix diagonalization makes non-diagonal elements of the matrix $H'(x, y)$ come close to zero. That is, the intensity gradient in x-direction or y-direction is brought close to zero.

$$\begin{aligned} H'(x, y) = \mathrm{Diag}(H) = \left( \begin{array}{cc} \lambda _1 &{}\quad 0 \\ 0 &{}\quad \lambda _2 \end{array} \right) ,\quad \left( \lambda _1 > \lambda _2 \right) . \end{aligned}$$

(3)

Therefore, after the diagonalization, element $\lambda _2$ (small eigenvalue) indicates a second derivative (curvature) of image f(x, y) in the direction of the smallest curvature among all curvatures in all directions (180 degrees). The minimum-curvature map $\mathrm{MCM}(x, y)$ is expressed as

$$\begin{aligned} \mathrm{MCM}(x, y) = \mathrm{max} (\lambda _2, 0). \end{aligned}$$

(4)

Here, a position where the value of $\lambda _2$ is negative is regarded as a non-blood area, so the value of $\mathrm{MCM}(x, y)$ is set to zero. As shown in Fig. 7b, the value of a pixel increases as the position of the pixel approaches feature points in $\mathrm{MCM}(x, y)$. These feature points can, therefore, be obtained by detecting local maximum points from $\mathrm{MCM}(x, y)$ as shown in Fig. 7c.

Feature points of $\mathrm{MCM}(x, y)$ are located at the center line of vein patterns, and the effect of luminance fluctuation is relatively small at these positions.

3.2.2 Calculation of feature descriptor

A feature descriptor is proposed that is robust against irregular shading and deformation of vein patterns. The descriptor is calculated in accordance with a combination of vein patterns and blood-flow direction. A descriptor-normalization method is also proposed that uses a finger-shaped model to deal with deformation due to finger rolling.

Fig. 8

Vein pattern extraction. a Original image. b Vein-pattern map

The minimum-curvature map $\mathrm{MCM}(x, y)$ is effective for detecting feature points stably. Pixels around the feature point have small values in $\mathrm{MCM}(x, y)$ and little distinctive information. Hence, it is difficult to make a distinctive descriptor by using $\mathrm{MCM}(x, y)$.

On the other hand, a large eigenvalue $\lambda _1$ expresses a curvature in a direction orthogonal to the direction where the curvature is minimum. Hence, a large eigenvalue $\lambda _1$ expresses curvature of a cross-sectional-intensity profile at vein patterns, and the map calculated by selecting a large eigenvalue for each pixel expresses vein patterns. The map defined as vein-pattern map $\mathrm{VPM}(x, y)$ is calculated as follows.

$$\begin{aligned} \mathrm{VPM}(x, y) = \mathrm{max} (\lambda _1, 0). \end{aligned}$$

(5)

An example of $\mathrm{VPM}(x, y)$ is shown in Fig. 8b. The feature descriptor is calculated by using a vein-pattern map because the vein-pattern map has more distinctive and complicated patterns than $\mathrm{MCM}(x, y)$. The eigenvector of the large eigenvalue $\lambda _1$ represents the direction perpendicular to the travel direction of the veins. To further increase the distinctive performance of the feature descriptor, the proposed feature descriptor is computed by using both $\mathrm{VPM}(x, y)$ and the eigenvector for each pixel. The descriptor is calculated as follows.

Fig. 9

Calculation of feature-point descriptor

First, the square area around a feature point is divided into $W{\times }H$ blocks. A histogram of the vein direction is computed with respect to each block by using N bins. Each bin covers an angle of 180 / N degrees. The pixel value of the vein-pattern map $\mathrm{VPM}(x, y)$ is added to the bin that corresponds to the orientation of the pixel’s eigenvector. This process is executed for all the pixels in each block so as to make histograms. All histograms are calculated from $W{\times }H$ blocks, so a $W{\times }H{\times }N$ dimensional descriptor is generated per feature point (Fig. 9). The vector is then normalized to the unit length to enhance invariance to affine changes in illumination.

3.2.3 Normalization of descriptor with finger-shaped model

Vein-pattern deformation is normalized as follows. When the size of a descriptor area is fixed, the descriptor is affected by a large deformation of vein patterns due to a change in finger posture caused by, for example, finger rolling. In general, the cross section shape of a finger can be approximated to a circle.

We, therefore, define a finger-shaped model as shown in Fig. 10. The model assumes that the cross-sectional shape of a finger is a circle, and the deformation is normalized on the basis of this model. In the proposed method, a is the radius of finger cross section. This radius is expressed as a half width between the upper and lower positions of finger outlines in a finger-vein image. The deformation of finger-vein patterns increases as the position of the veins approaches the finger outline. As shown in Fig. 10a, b, when the finger rotates around its longitudinal axis, the descriptor area A approaches the finger outline, so the area is projected onto a 2D plane as area B with a deformation. At this time, area B is smaller than area A. The size of the descriptor area is therefore adjusted in accordance with the position of the feature point. By normalizing the size of the descriptor area, the two descriptors, one calculated before and one calculated after the finger rolling, are approximately equal.

Fig. 10

Vein deformation caused by finger rolling. a Descriptor area A. b Descriptor area B with deformation

Fig. 11

Normalization of descriptor area

Fig. 12

Block diagram for proposed feature point matching

The size of the descriptor area is normalized by using the displacement of a feature point from a reference position, which is the intermediate position between the upper and lower positions of finger outlines at a certain point on the x-axis as shown in Fig. 11. In practice, both lower and upper boundaries of the descriptor area were determined with respect to each feature point p by using a displacement of the feature point in the y-direction, $\mathrm{d}y$, which is the distance between any feature point p and the feature point c located in the reference position. $y_{c_{\mathrm{low}}}$ and $y_{c_{\mathrm{upp}}}$ are taken as the lower and upper boundaries of c’s descriptor area, and $y_{p_{\mathrm{low}}}$ and $y_{p_{\mathrm{upp}}}$ are the lower and upper boundaries of p’s descriptor area.

Angle $\theta _p$ of point p from the reference position is calculated by substituting displacement $\mathrm{d}y$ for y in the following equation.

$$\begin{aligned} \theta = \mathrm{arcsin} \left( \frac{y}{a} \right) . \end{aligned}$$

(6)

Angles of the lower and upper boundaries of c’s descriptor area, namely $\theta _{c_{\mathrm{low}}}$ and $\theta _{c_{\mathrm{upp}}}$, are also calculated from Eq. (6) by using $y_{c_{\mathrm{low}}}$ and $y_{c_{\mathrm{upp}}}$. The lower and upper boundaries of feature point p’s descriptor area ($y_{p_{\mathrm{low}}}$, $y_{p_{\mathrm{upp}}}$) are then calculated from the following equations:

$$\begin{aligned} y_{p_{\mathrm{low}}} = a\mathrm{sin}\left( \theta _p + \theta _{c_{\mathrm{low}}}\right) ,\quad y_{p_{\mathrm{upp}}} = a\mathrm{sin}\left( \theta _p + \theta _{c_{\mathrm{upp}}}\right) . \end{aligned}$$

(7)

The feature descriptor is calculated from the descriptor area determined by these processes. The length of the descriptor area oriented in the direction of the long axis of a finger (x-direction) is fixed regardless of the position of the feature point since the size of the descriptor area in the x-direction only varies slightly due to finger rolling.

3.2.4 Elastic feature-point matching using non-rigid registration

The proposed elastic feature-point matching is composed of two steps. The flow of the point matching process is shown in Fig. 12.

Step 1 Non-rigid registration

To correct non-linear deformation of vein patterns, feature-point coordinates are aligned in three steps. First, all feature points in an enrollment vein are compared with those in an input vein on the basis of the feature descriptor (step 1-1). Next, some correspondences are selected to improve registration accuracy (step 1-2). In step 1-3, a non-rigid transformation is estimated by using selected correspondences. All points in the enrollment image are aligned by the non-rigid transformation.

Step 2 Feature-point matching

After the registration, each enrollment point searches for a matching input point on the basis of distance of the image and similarity of feature-descriptor space. The matching score is calculated on the basis of the ratio of the number of matching points and the number of all enrollment or input points.

The details of each step are described as follows.

Step 1 Non-rigid registration

Step 1-1 Searching for corresponding point

In this step, two images are compared: the enrollment image E and input image I. Each enrollment feature point i of image E is compared to input feature point j of image I, where $i = 1, 2, \ldots , n_1$ and $j = 1, 2, \ldots , n_2$. Each point i then corresponds to feature point j with the minimum distance of their feature descriptors. The corresponding point $\mathrm{CP}_{i}$ is defined as:

$$\begin{aligned} \mathrm{CP}_{i}=\mathop {\arg \min }\limits _{j} (FD_{ij}), \end{aligned}$$

(8)

where $\mathrm{FD}_{ij}$ is the distance of feature descriptors and is calculated as:

$$\begin{aligned} \mathrm{FD}_{ij}=\sqrt{\sum _{d=1}^{\mathrm{dim}} (\mathrm{Dsc}_{i, d}-Dsc_{j, d})^2 }. \end{aligned}$$

(9)

$\mathrm{Dsc}_{i, d}$ and $\mathrm{Dsc}_{j, d}$, $(d=1, 2, \ldots , \mathrm{dim})$, are feature descriptors of points i and j, respectively. Here, dim expresses the number of dimensions of the feature descriptor, and $\mathrm{dim}=W{\times }H{\times }N$. An example of corresponding points (correspondences) is shown in Fig. 13a. The upper and lower parts are the enrollment and input images, respectively.

Fig. 13

Elastic feature-point matching. a Searching for corresponding points. b GL-filtering for correspondences. c Feature-point matching

Fig. 14

Global filtering a Histogram before global filtering. b Histogram after global filtering

Step 1-2 GL-filtering for correspondences

To enhance registration accuracy, correspondences are selected by global and local filtering (GL-filtering). Correspondences whose direction differs greatly from many other directions are removed as shown in Fig. 13b. Selected correspondences are turned toward almost the same direction, and this enables accurate estimation of non-rigid transformation.

Hereafter, $(x_a, y_a)$ and $(x_b, y_b)$ mean the coordinates of corresponding points. The displacement vector between the correspondences points $V=(\mathrm{d}x,\mathrm{d}y)$ is calculated as $\mathrm{d}x=x_a-x_b$, $\mathrm{d}y=y_a-y_b$. As shown in Fig. 14a, a 2D-histogram of $\mathrm{d}x$ and $\mathrm{d}y$ is computed from all correspondences. The histogram represents the frequency distribution of the displacement vector. It is assumed that most correspondences are associated with higher frequency in the histogram. Correspondences existing at an area surrounding the peak in the histogram are referred to as inliers. The other correspondences are referred to as outliers. The proposed GL-filtering extracts inliers for accurate non-rigid registration. The GL-filtering method is composed of two sequential phases: global filtering and local filtering.

In the global-filtering procedure, outliers that can be easily distinguished from inliers are removed. First, the 2D-histogram of the displacement vector is computed from all correspondences (Fig. 14a). The displacement vector of outliers differs greatly from that of inliers. By extracting correspondences within radius $C_{\mathrm{th}_1}$ from the mode of the histogram, outliers are removed (Fig. 14b).

Local filtering is applied to the remaining correspondences that cannot be filtered by global filtering. The local filtering can remove outliers robust against deformation by restricting the filtering area in a vein image.

The details are as follows. First, a set of enrollment feature points within radius R from a certain enrollment point i in a vein image is selected. Next, a 2D-histogram of a displacement vector is computed by using only the selected correspondences. Finally, correspondences within radius $C_{\mathrm{th}_2}$ from the mode of the histogram are regarded as inliers. If the correspondence of enrollment point i is outside $C_{\mathrm{th}_2}$, the correspondence is removed as an outlier. These processes are conducted with respect to each correspondence. Local filtering restricts the range of correspondences that can be selected. Variations in inliers in a histogram are then made smaller. Distinguishing inliers from outliers is therefore easier than the global filtering.

Step 1-3 Estimation of non-rigid transformation

A non-rigid transformation is estimated by using the possible correspondences extracted in step 1-2. The thin-plate spline (TPS) model [20] is used for the non-rigid transformation. TPS is often used for fingerprint recognition [22, 23].

The non-rigid transformation is explained as follows. Here, $u_k=(u_{x_k}, u_{y_k})$ and $v_k=(v_{x_k}, v_{y_k})$ denote the location of the enrollment point and input point of the k th correspondence, and n is the total number of correspondences. The coordinates of the enrollment image (x, y) are transformed into the coordinates $(x', y')$ by using correspondence points $u_k$ and $v_k$. The TPS model gives the transformed coordinates, $(x', y')$, independently as a function of coordinates (x, y) as

$$\begin{aligned} x'=a_{x_1} + a_{x_2}x + a_{x_3}y + \sum _{k=1}^n (w_{x_k} U(|P_k-(x, y)|)), \end{aligned}$$

(10)

$$\begin{aligned} y'=a_{y_1} + a_{y_2}x + a_{y_3}y + \sum _{k=1}^n (w_{y_k} U(|P_k-(x, y)|)), \end{aligned}$$

(11)

where $U(r) = r^2log(r)$ is the basis function, $a_1$, $a_2$, and $a_3$ present the affine part of the transformation, $w_k$ gives an additional non-linear deformation, and $P_k$ is the position of the kth enrollment point.

Equations (10) and (11) can be denoted as the following matrix equation:

$$\begin{aligned} \varvec{V_{\Lambda }} = \left( \begin{array}{cc} \varvec{K} &{}\quad \varvec{P} \\ \varvec{P}^{\mathrm {T}} &{}\quad \varvec{O} \end{array} \right) \left( \begin{array}{c} \varvec{W_{\Lambda }} \\ \varvec{a_{\Lambda }} \end{array} \right) , \end{aligned}$$

(12)

where

$$\begin{aligned} \varvec{V_{\Lambda }} = \left( \begin{array}{ccccccc} v_{\Lambda _1}&v_{\Lambda _2}&\ldots&v_{\Lambda _n}&0&0&0 \end{array} \right) ^{\mathrm {T}}, \end{aligned}$$

(13)

$$\begin{aligned} \varvec{W_{\Lambda }} = \left( \begin{array}{cccc} w_{\Lambda _1}&w_{\Lambda _2}&\ldots&w_{\Lambda _n} \end{array} \right) ^{\mathrm {T}}, \end{aligned}$$

(14)

$$\begin{aligned} \varvec{a_{\Lambda }} = \left( \begin{array}{ccc} a_{\Lambda _1}&a_{\Lambda _2}&a_{\Lambda _3} \end{array} \right) ^{\mathrm {T}}, \end{aligned}$$

(15)

$$\begin{aligned} \varvec{P} = \left( \begin{array}{ccc} 1 &{}\quad u_{x_1} &{}\quad u_{y_1} \\ 1 &{}\quad u_{x_2} &{}\quad u_{y_2} \\ \vdots &{}\quad \vdots &{}\quad \vdots \\ 1 &{}\quad u_{x_n} &{}\quad u_{y_n} \end{array} \right) , \end{aligned}$$

(16)

$\Lambda$ stands for x or y, and $(\varvec{K})_{pq}$ is the $n \times n$ matrix with entries $U(|P_p - P_q|)$. It is further assumed that coefficients $\varvec{W_{\Lambda }}$ and $\varvec{a_{\Lambda }}$ satisfy $\varvec{K}\varvec{W_{\Lambda }} + \varvec{P} \varvec{a_{\Lambda }} = \varvec{V_{\Lambda }}$ and $\varvec{P}^{\mathrm {T}} \varvec{W_{\Lambda }}=0$. Thus, the coefficients of the TPS model can be obtained from matrix equation Eq. (12).

In the TPS model, it is assumed that no correspondences include outliers. If outliers are included, the coordinate transformation cannot work correctly. Approximating thin-plate splines [21] were therefore utilized that allow a few outliers by introducing a smoothness term in Eq. (12). The optimal TPS parameters can be found by solving the following equation:

$$\begin{aligned} (\varvec{K} + \lambda \varvec{I})\varvec{W_{\Lambda }} + \varvec{P} \varvec{a_{\Lambda }} = \varvec{V_{\Lambda }}, \end{aligned}$$

(17)

where $\varvec{I}$ is the $n \times n$ identity matrix and the parameter $\lambda$ controls the degrees of smoothing. The coefficients $\varvec{W_{\Lambda }}$ satisfy $\varvec{P}^{\mathrm {T}} \varvec{W_{\Lambda }}=0$.

After the coordinate transformation, coordinates of all enrollment points are transformed into new coordinates. Enrollment points are aligned so that the positions of enrollment and input points match each other.

Step 2 Feature-point matching

Each transformed enrollment point newly searches for a matching input point. After the non-rigid registration, the points with high similarity of feature descriptors are positioned close to each other if these points are on the same finger. These highly similar points are regarded as matching points. If the number of matching points is large, it is highly likely that the enrollment image E and input image I are of the same finger.

Specifically, points i and j are matched when the following two conditions are satisfied.

• $\bullet$ Condition1: $\mathrm{ED}_{ij} {\le } R_{\mathrm{ED}}$

• $\bullet$ Condition2: $\mathrm{FD}_{ij} < T_{\mathrm{FD}_i}$

Condition 1 is a constraint of distance between feature points on the vein image. Condition 2 is set to match feature points that have high similarity of feature descriptors.

$\mathrm{ED}_{ij}$ represents the Euclidean distance between the transformed enrollment point $i=(x'_i, y'_i)$ and input point $j=(x_j,y_j)$ as

$$\begin{aligned} \mathrm{ED}_{ij} = \sqrt{(x'_i-x_j)^2 + (y'_i-x_j)^2}. \end{aligned}$$

(18)

When distance $\mathrm{ED}_{ij}$ is within a disk with radius $R_{\mathrm{ED}}$, the two points satisfy condition 1.

$T_{\mathrm{FD}_i}$ judges whether a similarity between two descriptors is high. When a descriptor distance $\mathrm{FD}_{ij}$ computed in Eq. (9) is smaller than $T_{\mathrm{FD}_i}$, the two points satisfy condition 2. $T_{\mathrm{FD}_i}$ of each point i is calculated as follows.

$$\begin{aligned} T_{\mathrm{FD}_i} = \mathrm{FD}_{\mathrm{av}_i} - \alpha \mathrm{FD}_{\sigma _i}, \end{aligned}$$

(19)

where $\mathrm{FD}_{\mathrm{av}_i}$ and $\mathrm{FD}_{\sigma _i}$ represent the average and variance of the descriptor distance, respectively. Parameter $\alpha$ controls the threshold $T_{\mathrm{FD}_i}$. $\mathrm{FD}_{\mathrm{av}_i}$ and $\mathrm{FD}_{\sigma _i}$ are calculated as follows:

$$\begin{aligned} \mathrm{FD}_{\mathrm{av}_i} = \frac{1}{n_2} \sum _{j=1}^{n_2} \mathrm{FD}_{ij}, \end{aligned}$$

(20)

$$\begin{aligned} \mathrm{FD}_{\sigma _i}^2 = {\frac{1}{n_2} \sum _{j=1}^{n_2} (\mathrm{FD}_{ij}-\mathrm{FD}_{\mathrm{av}_i})^2 }. \end{aligned}$$

(21)

Each enrollment point i is compared with all input points j and then multiple values of $\mathrm{FD}_{ij}$ are calculated. It is assumed that the histogram of these $\mathrm{FD}_{ij}$ has normal distribution. Shapes (variance) and positions (average) of the distribution change depending on each point i. The threshold $T_{\mathrm{FD}_i}$ used for condition 2 is therefore normalized with average $\mathrm{FD}_{\mathrm{av}_i}$ and variance $\mathrm{FD}_{\sigma _i}$ in Eq. (19). An example of feature-point matching based on step 2 is shown in Fig. 13c.

A matching score is calculated on the basis of the ratio of the number of matching points and the number of all enrollment/input points. Finger-vein authentication is then performed on the basis of the matching score described as

$$\begin{aligned} \mathrm{Matching~Score} = \frac{m}{n_1 + n_2}, \end{aligned}$$

(22)

Table 1 Details of collected images in databases

where m is the number of matching points, and $n_1$ and $n_2$ are the numbers of enrollment feature points and input feature points, respectively.

4 Experimental results

The efficiency of the proposed method was evaluated in the following three experiments.

• Exp. 1: Robustness against irregular shading

• Exp. 2: Robustness against deformation

• Exp. 3: Evaluation of accuracy

The purpose of Exp. 1 and Exp. 2 is to evaluate the robustness against irregular shading and deformation of vein patterns. In Exp. 1 and Exp. 2, the proposed method is compared with the conventional method which is based on SIFT method and Miura et al.’s method which is a template matching method. In the point-matching procedure, both the proposed and SIFT methods use a matching method described in Sect. 3.2.4.

In Exp. 3, accuracies of the proposed and conventional methods were evaluated. The proposed method is compared with four conventional methods that are based on template matching: Miura et al.’s method, Kumar et al.’s method, Yang et al.’s method, and SIFT. In all experiments, each method is evaluated in terms of false rejection rate (FRR), false acceptance rate (FAR), and equal error rate (EER). EER is obtained by calculating the error rate where FAR $=$ FRR.

4.1 Database

Multiple databases are prepared to achieve the objective of each experiment as shown in Table 1. Enrollment data are collected by capturing fingers that are stretched naturally and placed on the predetermined standard position in all experiments. In Exp. 1 and Exp. 2, matching data is required that includes irregular shading and deformation of vein shape. Matching data in Exp. 1 (DB1) is collected by capturing fingers that are placed on distant positions from the predetermined standard position. Matching data in Exp. 2 (DB2-1 and DB2-2) is collected by capturing fingers of various postures. Robustness against irregular shading and deformation is evaluated by matching enrollment data with matching data. Databases (DB1, DB2-1 and DB2-2) are acquired from 5 subjects (4 males and 1 female) with ages ranging from their twenties to sixties. Thirty fingers are captured in Exp. 1 and Exp. 2. Each finger in the databases (DB1, DB2-1 and DB2-2) is captured 10 times. Both the number of enrollment data and the number of matching data (DB1, DB2-1 and DB2-2) are 300. To increase variation of irregular shading and deformation, a finger is taken off an imaging device every time capturing is completed.

Table 1 shows how to create DB1, DB2-1 and DB2-2. To capture matching data in DB1, a finger is placed on the position that is 15 mm from the standard position in the longitudinal direction of the finger. The placed finger is captured 10 times. As the collected images in DB1 have a large finger displacement, the common area between compared images becomes small; therefore, FRR due to the irregular shading tends to be increased. For a fair comparison, DB1 also includes the effect of large finger displacement and so is not suitable for comparing point matching methods and template matching methods. The proposed method is therefore only compared with the SIFT method. In DB2-1, the captured finger is placed on the standard position and bends forwards or backwards. Therefore, vein shape is deformed as shown in Table 1. The bent finger is placed within the range of 30 mm above the standard position in the thickness direction of the finger, and the finger is captured 5 times. A finger bent backwards is placed within the range of 5 mm below the standard position in the thickness direction of the finger, and 5 images are acquired. In DB2-2, the captured finger is placed on the standard position and rotates around a longitudinal axis of the finger. The finger rotated about 30 degrees is captured 5 times. Furthermore, the finger rotated about $-$30 degrees is also captured 5 times.

The database used in Exp. 3 (DB3) is collected without special conditions such as irregular shading or deformation of veins. Both enrollment data and matching data in DB3 are acquired by capturing a finger that is placed on the standard position with as little posture change as possible. To evaluate accuracies, more finger data is collected than DB1, DB2-1, and DB2-2. DB3 is acquired from 676 subjects (approximately 70 % male) with ages ranging from their twenties to forties. Both the numbers of enrollment data and matching data (DB3) are 676.

Fig. 15

Evaluation of robustness against irregular shading

4.2 Robustness against irregular shading

In this evaluation experiment, DB1 is used to evaluate robustness against irregular shading. The detection error tradeoff (DET) curve, which shows the relationship between FAR and FRR, is shown in Fig. 15. Both FRR and FAR obtained by the proposed method are lower than the corresponding values obtained by SIFT methods for all the plotted points. EER is 0.236 % for the proposed method, while it is 1.302 % for the SIFT method (Table 2). These results show that the proposed method is more robust against irregular shading then the SIFT method.

Table 2 Experiment results in DB1

4.3 Robustness against deformation of vein patterns

Robustness against deformation of vein patterns caused by a finger-posture change such as finger bending and finger rolling was evaluated next.

Table 3 Details of each method used in Exp. 2-1

Fig. 16

Evaluation of feature-point detection methods and feature-descriptor methods

4.3.1 Evaluation of tolerance to finger bending

Robustness against deformation of vein patterns caused by forwards and backwards finger bending was evaluated in Exp. 2-1. That is to say, the proposed method was compared with the SIFT feature-point detection and SIFT descriptor by using DB2-1. The accuracies were calculated for four combinations of the proposed method, SIFT feature-point detection method, and SIFT feature-descriptor method. The details of the four methods are listed in Table 3. The accuracy is also calculated for Miura et al.’s method which is a template matching method. The calculated FRR/FAR and EER are shown in Fig. 16 and Table 4, respectively. Method 1 shows greater robustness against deformation than the other methods. Method 4, which uses SIFT only, is as good as or better than methods 2 and 3, which use combinations of the proposed method and SIFT point detection and a feature descriptor . However, method 4 is inferior to method 1. It is, therefore, concluded that the combination of the proposed feature-point-detection method and feature descriptor shows superior robustness against deformation of vein patterns.

4.3.2 Evaluation of tolerance to finger rolling

To evaluate the tolerance to finger rolling, the proposed method that uses the normalization of the descriptor and a method that does not are compared with SIFT method by using DB2-2. The results of calculating FRR/FAR are plotted in Fig. 17 as an DET curve, and EER is shown in Table 5. Both FRR and FAR obtained by the method without the normalization of finger deformation are lower than the corresponding values obtained by the template matching method [4] and SIFT method at all plotted points. Those of the proposed method are even lower than the method without the normalization. Moreover, EER obtained by the proposed method which using normalization of the descriptor is the lowest.

Table 4 Experiment results in DB2-1

Fig. 17

Evaluation of tolerance to finger rolling

Table 5 Experiment results in DB2-2

Fig. 18

Accuracy comparison on DET curve

4.4 Accuracy comparison with conventional methods

To examine the accuracy of the proposed method for personal identification, an experiment using DB3 was conducted. The FRR and FAR of proposed and conventional methods are plotted in Fig. 18, and the EER values are listed in Table 6. Both FRR and FAR obtained by the proposed method are lower than the corresponding values obtained by the other conventional methods at all plotted points. Moreover, EER obtained by the proposed method is the lowest, meaning that the accuracy of finger-vein identification using the proposed method is higher than that of the other methods.

Table 6 Experiment results in DB3

5 Conclusion

A new authentication method was proposed, that is based on feature-point matching and is robust against both irregular shading and deformation of vein patterns. Matching of finger-vein images with irregular shading and deformation of vein patterns by the proposed method attained higher accuracy than the SIFT method. Experimental results acquired with a database of 676 subjects demonstrate that accuracy of the proposed method significantly improves on that of conventional finger-vein identification methods.

Future work will focus on the following issues. The first issue is to reduce processing time of the proposed method. Average computation times of three finger-vein authentication methods mentioned in this paper are shown in Table 7. Tests were run on an Intel Core i7-4770 CPU processor at 3.40 GHz, with 16 GB memory. The processing time of feature-point extraction/matching is the average time per processing of 1 image/matching. The proposed method achieves faster processing than SIFT. However, the processing speed of proposed method is much slower than that of the method of Miura et al. [4]. To ensure an authentication system with high usability, the processing time must be shortened.

Table 7 Average computation time in Exp. 3

The second issue is normalization of a rotation angle around an optical axis and scale of a finger-vein image. For the proposed method, it is assumed that an imaging device can capture the outlines of a finger. The angle and scale of enlargement of the finger are normalized by these outlines of the finger. However, if the authentication device were to be used in mobile equipment such as a smartphone or tablet terminal, the device would have to be scaled down [24]. Because such a small device cannot capture the outlines of a finger, the angle and the scale of a finger-vein image need to be normalized.