Introduction

Image matching is the basic component of the field of machine vision, which is now widely used in many fields, such as medicine, agriculture, remote sensing, machinery and artificial intelligence, etc. (Uchiyama et al. 2015; Schmid et al. 2000; Reese et al. 2015; Sedaghat and Ebadi 2015; Lee et al. 2016; Ye et al. 2017). For image matching, it is mainly divided into gray-based matching and feature-based matching (Moon and Loh 2015). The matching method based on image gray information need a large amount of calculation and easy to be affected by illumination and noise. The matching method based on image feature can overcome the shortcomings of matching with image gray information. For the gray scale change, image deformation and image occlusion, feature matching method has a better ability to adapt. Therefore, the method based on feature information matching is more widely used in practical application. The common matching method based on image feature is to use the local feature descriptor, which has made a lot of researches on the local feature descriptor of the image. Lowe (2002) proposed the Scale invariant feature transform algorithm (SIFT) which has good robustness, but the amount of computation is large. Therefore, scholars have studied and improved SIFT. Ke and Sukthankar (2004) proposed PCA-SIFT to reduce the dimension of descriptor and reduce the computational complexity. Speed up robust feature (SURF) algorithm is first presented in 2006 by Bay et al. (2006), it is faster and more robust than the SIFT. However, these algorithms still do not achieve real-time requirements. Rublee et al. (2011) proposed the Oriented Brief algorithm (ORB) in 2011. Although the algorithm is fast, the ORB does not have the scale invariance, so the image matching effect is poor when the image scale changes. Thus, the research and improvement of ORB has great practical significance. In recent years, there has been a lot of improvements for SIFT and SURF algorithms (Chen et al. 2011; Liu 2016; Yu et al. 2015; Ma 2015), but little improvement has been made on ORB. Gao and Zhu (2016) used the improved ORB algorithm to detect moving object for driving assistance system which improved the matching accuracy by establishing a matching optimization strategy to eliminate the mismatched pairs; Li et al. (2016) combined Hamming distance and cosine similarity measurements to describe the binary eigenvectors, effectively reducing the error matching rate; Zou (2015) combined the ORB algorithm with the nearest-neighbor (NN) search method, which replaced the Hamming distance by the nearest-neighbor (NN) search method, and improves the speed of ORB matching by this method. However, none of these improved algorithms can solve the problem that the ORB algorithm does not have scale invariance.

In summary, the ORB algorithm is improved in this paper. The scale invariance of the SURF algorithm is used to obtain a new ORB-SURF algorithm (abbreviated as OR-SURF), which not only retains the fast superiority of the ORB algorithm, but also makes the ORB algorithm scale invariance; nonsubsampled contourlet (NSCT) is used to reduce the impact of high-frequency noise on the matching results; and the random sample consensus algorithm (RANSAC) is applied to eliminate false matching points to complete image matching. The improved algorithm can effectively solve the problem that the ORB algorithm has no scale invariance, and improves the accuracy, speed, and robustness of remote sensing image matching.

The Theory of NSCT

NSCT is an improvement of contourlet transform, which is built upon a 2D nonsubsampled pyramid and nonsubsampled directional filter bank (Da et al. 2006). The nonsubsampled pyramid can be obtained by removing the downsamplers and upsamplers in the Laplacian pyramid, which decides the multiscale decomposition of the NSCT. The nonsubsampled directional filter bank is constructed by eliminating the downsamplers and upsamplers in the directional filter bank, which decides the directional decomposition of the NSCT. The NSCT can suppress the pseudo-Gibbs phenomena effectively, capture the intrinsic geometrical structures accurately, and separate the noise from the weak edge. The overall structure of the NSCT is shown in Fig. 1.

Fig. 1
figure 1

NSCT decomposition diagram

Full size image

Improved ORB Algorithm

SURF Feature Point Detection

SURF extracts feature points based on Hessian matrix. The feature point position corresponds to the local maxima of the Hessian matrix determinant. Given a point x = (x, y) in an image I, the Hessian matrix H is defined as follows:

$$ H\left( {x,\delta } \right) = \left[ {\begin{array}{*{20}c} {L_{xx} \left( {x,\delta } \right)} & {L_{xy} \left( {x,\delta } \right)} \\ {L_{xy} \left( {x,\delta } \right)} & {L_{yy} \left( {x,\delta } \right)} \\ \end{array} } \right] $$
(1)

where \( L_{xx} \left( {x,\delta } \right) \) is the convolution of the Gaussian second-order derivative with the image I in point X = (x, y), and similarly for \( L_{xy} \left( {x,\delta } \right) \) and \( L_{yy} \left( {x,\delta } \right) \).

In order to facilitate the calculation, SURF algorithm uses box filter to approximate the second-order Gaussian filter. The determinant of the Hessian matrix can be approximated by formula (2):

$$ \det \left( {H_{\text{approx}} } \right) = D_{xx} D_{yy} - \left( {\omega D_{xy} } \right)^{2} $$
(2)

In order to ensure that the image matching process has scale invariance, the scale space is constructed on the image, and the feature points are extracted on different scale images. In the establishment of scale space, the size of the original image is not changed, and the integral image of the original image is filtered by changing the size of the box filter so as to form the scale space of the image. In order to determine the SURF feature point, the Hessian threshold H is set and a non-maximum suppression in a 3 × 3 × 3 neighborhood is applied. Such points are used as feature points, when the characteristic value of the feature point is larger than the threshold value H and is larger than the response value of the adjacent 26 points. Finally, accurate interpolation is performed to obtain stable feature points.

Generating OR-SURF Feature Descriptor

ORB algorithm adopts the improved binary robust independent elementary features algorithm (BRIEF) to achieve the description of the feature points (Rublee et al. 2011). The principle of BRIEF is to compare the gray value of pixels of the limited pairs of points. For noise-resistant interference, a 5 × 5 processing domain is selected in the 31 × 31 pixel area of the feature point, and the average of the pixel gray levels in the processing area is compared. In the neighborhood of feature points, m pairs of contrast points are taken, and gray level are compared by formula (3):

$$ \tau \left( {p;x,y} \right) = \left\{ {\begin{array}{*{20}l} {1,} \hfill & {p\left( x \right) < p\left( y \right)} \hfill \\ {0,} \hfill & {p\left( x \right) \ge p\left( y \right)} \hfill \\ \end{array} } \right. $$
(3)

where p(x) is the gray level at the area x around the image feature point. When m contrast point pairs are selected, the length of m binary tests with no rotation invariance is defined as formula (4) (the number of pairs of points m can choose 128,256,512, etc., usually m is selected 256 in the calculation):

$$ f_{m} = \mathop \sum \limits_{1 \le i \le m} 2^{i - 1} \tau \left( {p;x_{i} ,y_{i} } \right) $$
(4)

In order to solve the problem that the descriptor described in (4) has no rotation invariance, the steered BRIEF operator is used in the ORB algorithm. That is, the OR-SURF descriptor has the direction information according to the orientation of feature points in (5), and the corresponding rotation matrix \( R_{\theta } \) is obtained. For m contrast point pairs, defines the matrix Q:

$$ Q = \left[ {\begin{array}{*{20}c} {x_{1} ,x_{2} , \ldots ,x_{m} } \\ {y_{1} ,y_{2} , \ldots ,y_{m} } \\ \end{array} } \right] $$
(5)

To correct, the final corrected version is \( Q_{\theta } :Q_{\theta } = R_{\theta } Q \), where \( R_{\theta } \) is as follows:

$$ R_{\theta } = \left( {\begin{array}{*{20}c} {\cos \theta } & {\sin \theta } \\ { - \sin \theta } & {\cos \theta } \\ \end{array} } \right) $$
(6)

Thus, a directional descriptor is obtained:

$$ g_{m} \left( {p,\theta } \right) = f_{m} \left( p \right)\left| {\left( {x_{i} ,y_{i} } \right)} \right. \in Q_{\theta } . $$
(7)

Feature Point Matching

After using the OR-SURF algorithm to get the feature point, then image matching is the next work. Aiming at the problem that the K-Dimension tree (K-D tree) nearest-neighbor search algorithm has poor performance in processing high-dimensional datasets, the improved K-D tree nearest-neighbor search algorithm Best Bin First (BBF) is used to process the data in this paper. Compared with the K-D tree algorithm, the BBF algorithm adds the priority queue in the backtracking query, that is, the backtrack search is according to the priority queue. In addition, the BBF algorithm also sets the running timeout limit and improves the search efficiency. In this paper, the similarity distance between feature points can be obtained by calculating the Euclidean distance between feature points in two images, and then establish a 64-dimensional K-D tree. The BBF algorithm is adopted to find out the feature highlights of two nearest Euclidean distance points between basic image and matching image. Then, compare the ratio of two nearest Euclidean distance with given threshold, if the result is less than the threshold (0.55 in this article), the match is correct matching. Generally, the threshold range is between 0.4 and 0.6 (Jia et al. 2016). If the threshold is less than 0.4, there is a problem that the matching point is few. If the threshold is greater than 0.6, there is a large number of error matching points. Therefore, 0.55 is chosen as the threshold.

Principle of Improved Remote Sensing Image Matching Algorithm

The improved ORB algorithm based on the SURF algorithm solves the problem that the ORB algorithm has no scale invariance and low matching accuracy in the image matching, and improves the matching performance by the NSCT algorithm. NSCT decomposes the original remote sensing image, in which the low-frequency component contains most of the information of the image, which basically maintains the characteristics of the original image, while the high-frequency detail component contains a lot of noise. Therefore, the low-frequency component can be used as the input image of the OR-SURF algorithm. After pre-matching the two images by OR-SURF algorithm, the RANSAC method is used to eliminate the error matching points for improving the accuracy and obtaining the precise matching point set. The specific steps of the algorithm are as follows:

  • Step 1: The reference image and image to be matched are decomposed by NSCT.

  • Step 2: The low-frequency components of two images decomposed by NSCT are regarded as the input images of improved ORB algorithm.

  • Step 3: The feature highlights of two nearest-neighbor points are found out by using the improved K-D tree nearest-neighbor search algorithm BBF. The preliminary match point is obtained.

  • Step 4: RANSAC algorithm is used to filter the matching points and image matching is completed.

The general process of image matching is shown in Fig. 2.

Fig. 2
figure 2

Algorithm flowchart

Full size image

Results and Discussion

To compare the proposed algorithm with the SIFT, SURF and Harris-SURF algorithms (Wang et al. 2015) in terms of matching performance and various anti-interference capabilities in detail, a large number of experiments of the aforementioned algorithms on actual remote sensing images are conducted, and a quantitative evaluation and analysis are performed. The quantitative evaluation includes feature point extraction performance, matching performance, anti-rotation capability, anti-scale variation capability and anti-noise capability.

Feature Point Extraction Performance Analysis

In order to verify the rapidity and effectiveness of the proposed algorithm in the feature points selection, the feature points are extracted from the remote sensing images shown in Fig. 3 by SIFT, SURF, Harris-SURF and OR-SURF algorithms, and the number of feature points and the detection time are calculated, as shown in Table 1.

Fig. 3
figure 3

Remote sensing image

Full size image
Table 1 Comparisons in feature point extraction performance
Full size table

It can be seen from Table 1 that the number of feature points detected by the OR-SURF algorithm in this paper is less than that of other algorithms, which can effectively extract the image information to highlight the image details, while in the feature points detection by the SIFT, SURF and Harris-SURF algorithm, the large number of feature points will lead to the redundancy of effective information and increase the computational complexity. In terms of detection time, the OR-SURF algorithm takes much less time than SIFT, SURF and Harris-SURF algorithms. In general, the OR-SURF algorithm can effectively detect the image information in a short time and highlight the details of the image, which shows the speediness of the algorithm in the feature points detection and the validity of the feature points selection.

Matching Performance Analysis

In order to verify the matching performance of the proposed algorithm, the SIFT, SURF, Harris-SURF and OR-SURF algorithms are used to match the remote sensing images as shown in Fig. 4. The accuracy and time of the matching are analyzed as shown in Table 2.

Fig. 4
figure 4

Matching image

Full size image
Table 2 Comparisons in matching performance
Full size table

Compared with SURF algorithm, Harris-SURF algorithm has improved its performance. The algorithm proposed in the paper has the fastest matching speed, the highest matching accuracy and the best capability.

It is known from Table 2 that in the four algorithms, the SURF algorithm has greatly reduced the matching time compared with the SIFT algorithm, but in the matching accuracy, the performance is not good, there are more mismatches; the Harris-SURF algorithm is slightly better than the SURF algorithm in matching performance; and the algorithm proposed in this paper has the fastest matching speed, the highest matching accuracy and the best performance.

Anti-rotation Capability Analysis

In order to verify the performance of the algorithm in the rotation invariance, the rotation processing of the image are carried out, and the matching experiment is carried out. First, for the verification of rotation invariance, four classical digital images are selected as the experimental samples, each image is rotated from 0° to 180° at intervals of 30°. Images are verified and matched among SIFT algorithm, SURF algorithm, Harris-SURF algorithm and the proposed matching algorithm. And the results of the accuracy and matching time are recorded.

The statistical results of the algorithm performance comparison have shown in Table 3. As can be seen from Table 3: although the SIFT algorithm has the strong robustness in the rotation invariance which accuracy is more than 96%, the time efficiency is the lowest. The time efficiency of SURF compared to SIFT is larger increase. But the accuracy declines which is only about 70% in dealing with image rotation. The matching accuracy and time efficiency of Harris-SURF algorithm has improved compared with SURF algorithm, but the effect is still poor. The matching accuracy of the proposed algorithm is more than 96%, the matching accuracy and time efficiency of the proposed algorithm has increased compared with the SURF and Harris-SURF algorithms. Considering the matching effect and matching time, the algorithm as an improvement of ORB and SURF algorithm not only keeps the advantages of speed and accuracy of ORB algorithm, but also improves the matching ability of SURF algorithm for rotation invariance, and obtains good effect.

Table 3 Comparisons in anti-rotation capability
Full size table

Anti-scale Variation Capability Analysis

In order to verify the performance of the algorithm in scale invariance, the scale of the image is changed, and the matching experiment is conducted. First of all, for the scale invariance verification, we scale 50% and 200% for experimental images, and then use the SIFT, SURF, Harris-SURF and the proposed algorithm to match images. The matching accuracy and matching time are recorded as shown in Table 4.

Table 4 Comparisons in anti-scale variation capability
Full size table

It can be seen from Table 4 that the performance of SIFT, SURF, Harris-SURF and the algorithm proposed in this paper are very good in terms of image scale changes, but the algorithm has obvious advantages in running time. Since the ORB algorithm does not have scale invariance, although the algorithm runs faster, the matching performance is worse. Considering the matching effect and matching time comprehensively, this algorithm is an improvement of ORB and SURF algorithm. It not only maintains the superiority and accuracy of the ORB algorithm, but also compensates for the defect that the ORB algorithm does not have scale invariance and obtains better results.

Anti-noise Capability Analysis

The Gaussian noise of different variances and salt and pepper noise of different noise density are added into Fig. 4b, and then they are matched with Fig. 4a. The comparisons of four algorithms in terms of anti-noise capability are shown in Fig. 5.

Fig. 5
figure 5

Comparisons in anti-noise capability

Full size image

As shown in Fig. 5, the matching rate of Harris-SURF is higher than that of SURF, but they are both lower than that of the SIFT algorithm without noise. However, with increasing Gaussian noise variance/salt and pepper noise density, the matching rate of the SIFT algorithm decreases dramatically compared with that of SURF and Harris-SURF. The performance of the proposed algorithm is the best, and the matching rate is always superior to the other three algorithms. Because in this paper NSCT is used to decompose the image, and only the low-frequency component of image is matched, which can significantly reduce the effect of noise and other details, thus accelerating the matching and improving its precision.

Conclusions

This paper proposed an algorithm based on the improved ORB algorithm and the NSCT algorithm. The experimental results show that: 1. The algorithm can make up for the lack of scale invariance of ORB algorithm and effectively improve the matching speed and accuracy of the scale change of the image. 2. The algorithm has more robust and comprehensive consideration in the complex situation. The algorithm performs best in terms of the speed and matching effect of the algorithm.