Lagrange Detector in Image Processing

Abstract

Edge detection is a basic operation in the field of image processing and computer vision. However, to the best of our knowledge, there is less mathematical work has been proposed beyond the first- and the second-order derivative operator for edge detection in the past decays. We propose a mathematical model called Lagrange detector for edge detection. Based on Lagrange polynomial interpolation theory, this detector calculates Lagrange remainder as the strength of edge and points at the features in various orders of discrete data or signal. Lagrange remainder combines the first-order derivation and the first- and second-order derivative of neighborhood by multiplication. We use the truncation error of polynomial interpolation to estimate Lagrange remainder. Lagrange detector performs well in detecting both outlines and tiny details. Furthermore, Lagrange detector can be used to detect high-frequency information like as corner, point, Moiré pattern and etc. The research of Lagrange detector opens an new window for low level image processing, and will be used as the basis for further studies on image processing.

1 Introduction

Edge detection is a first-step operation used in image processing and computer vision applications, resulting in significantly reducing the amount of data, filtering out useless information and preserving the important structural properties in an image. Edge detection refers to the process of identifying and locating sharp discontinuities in an image. The discontinuities are obviously changes in pixel intensity which characterize boundaries of objects in a scene.

This field possesses many mathematical models, including Sobel operator, Roberts operator, Prewitt operator, Laplace operator, LoG detector [1], Canny detector [2] and etc. [3]. These operators are based on first-order derivative or second-order derivative. The methods based on first-order derivative detect edges by first computing a measure of edge strength, usually first-order derivative expression such as the gradient magnitude, and then searching for local directional maxima of the gradient magnitude using a computed estimate of the local orientation of the edge, usually the gradient direction. The methods based on second-order derivative detect edges by searching for zero crossings in a second-order derivative expression, usually the zero-crossings of the Laplace operator or the zero-crossings of a non-linear differential expression.

Many methods and techniques have been developed based on these mathematical theories. Data driven methods is the newly and hottest studies, by learning the possibility distributions of features, including methods based on CNN [4]. However, to the best of our knowledge, there is less mathematical work has been proposed beyond the first- and the second-order derivative operator for edge detection in the past decays.

We propose a mathematical theory called Lagrange detector, which is the Lagrange form of the remainder of Taylor’s theorem. Polynomial interpolation theory provides basis for the calculation of Lagrange reminder. Lagrange detector identifies the high-order components in images, charactering, extracting and processing points that present edges or curves.

2 Related Work

2.1 Edge Detection

Edge detection aims at capturing boundaries and curves by detecting sharp changes in image brightness. All of mathematical operators detecting edge are based on first-order derivative or second-order derivative. Essentially we need to get the extremum value of brightness variations that is the first-order derivative. The methods based on calculating the second-order derivative are just to get the extremum point of first-order derivative [3] (Fig. 1).

2.2 First-Order Methods

The methods based on first-order derivative include Sobel operator, Roberts operator, Prewitt operator and Canny detector. One of them Sobel operator has concise mathematical expressions and Canny detector uses Sobel operator as pretreatment. So we mainly explore the mathematical theory of Sobel operator [3].

Fig. 1.

The figure indicates that the mathematical model of edges. (a) The black curve: the function of the original signal; (b) the blue curve: the first-order derivative of the signal; (c) the red curve: the second-order derivative of the signal. The green cross-shaped mark is the point that edge operators detect, of which the first-order derivative takes the extremum value and the second-order derivative crosses over zero. (Color figure online)

Sobel operator has clear and definite mathematical expressions. Sobel operator is defined as

$$ S_{x,y} = \sqrt {d_{x}^{2} + d_{y}^{2} } $$

(1)

where $d_{x}$ is the gradient in x, $d_{y}$ is the gradient in y.

Practically we use two $3 \times 3$ matrices to calculate the differences $\Delta _{\text{x}} ,\Delta _{\text{y}}$ as approximate the values of ${\text{d}}_{\text{x}}$, ${\text{d}}_{\text{y}}$. We have that

$$ \Delta _{x} = \left( {\begin{array}{*{20}c} { - 1} & 0 & { + 1} \\ { - 2} & 0 & { + 2} \\ { - 1} & 0 & { + 1} \\ \end{array} } \right) \times \varvec{A } $$

(2)

$$ \Delta _{y} = \left( {\begin{array}{*{20}c} { + 1} & { + 2} & { + 1} \\ 0 & 0 & 0 \\ { - 1} & { - 2} & { - 1} \\ \end{array} } \right) \times \varvec{A } $$

(3)

where A is the data matrix of the image. While the gradient orientation can be estimated as

$$ \theta = \tan^{ - 1} \left( {{{\Delta _{y} } \mathord{\left/ {\vphantom {{\Delta _{y} } {\Delta _{x} }}} \right. \kern-0pt} {\Delta _{x} }}} \right) $$

(4)

2.3 Second-Order Methods

The methods based on second-order derivative include Laplace operator and LoG detector. Laplace operator also has clear and definite mathematical expressions [3]. LoG detector combines Laplace operator and Gaussian filter [1].

Laplace operator is defined as

$$ \nabla^{2} f\left( {x,y} \right) = \frac{{\partial^{2} f}}{{\partial x^{2} }} + \frac{{\partial^{2} f}}{{\partial y^{2} }} $$

(5)

We also use a 3 × 3 matrix to calculate the second-order difference as the value of $\nabla^{2} f\left( {x,y} \right)$. So we have

$$ \nabla^{2} f \approx \left( {\begin{array}{*{20}c} 0 & { + 1} & 0 \\ { + 1} & { - 4} & { + 1} \\ 0 & { + 1} & 0 \\ \end{array} } \right) \times \varvec{A } $$

(6)

where A is the data matrix of the image.

3 Lagrange Detector

Lagrange detector is the Lagrange form of the remainder of Taylor’s theorem. The reminder can be calculated via the difference between the polynomial interpolation and the true value. For polynomial interpolation, Lagrange Polynomial is

$$ L_{n} \left( x \right) = \sum\nolimits_{k = 0}^{n} {f(x_{k} )l_{k} (x)} $$

(7)

where $l_{k} \left( x \right) = \mathop \prod \limits_{j = 0, j \ne k}^{n} \frac{{x - x_{j} }}{{x_{k} - x_{j} }}\left( {{\text{k}} = 0,{ 1},{ 2}, \, \ldots ,{\text{ n}}} \right)$, given $\left( {x_{i} , f(x_{i} )} \right)$ $\left( {{\text{i}} = 0, 1, 2, \ldots , {\text{n}}} \right)$. The Lagrange remainder is

$$ R_{n} \left( x \right) = f\left( x \right) - L_{n} \left( x \right) = \frac{{f^{{\left( {n + 1} \right)}} ( \epsilon )}}{{\left( {n + 1} \right)!}}\prod\nolimits_{i = 0}^{n} {(x - x_{i} )} $$

(8)

where $\forall x \in \left( {a,b} \right), f^{{\left( {n + 1} \right)}} \left( x \right) {\text{exists}}, \exists \epsilon \,\, \in \left( {a,b} \right) {\text{makes}}$ the formula (8) be workable [5,6,7]. We have proved that $R_{n}$ has obviously positive correlation with signal frequency. So we are able to use $R_{n}$ to characterize and extract the high-frequency components. After attaining the high-frequency data, we can do the works well about detecting edge, corner, point, Moiré pattern [8] and etc.

3.1 1D Signal

From Taylor’s Formula, a 1-dimension signal $\varvec{S}({\text{t}})$ can be decomposed into a set of polynomial expressions:

$$ \varvec{S}\left( {\text{t}} \right) = \sum\nolimits_{n = 0}^{ + \infty } {\frac{{\varvec{S}^{\left( n \right)} (t_{0} )}}{n!} \cdot (t - t_{0} )^{n} } $$

(9)

with $\varvec{S}^{\left( n \right)} (t_{0} )$ is the n-order derivative of $\varvec{S}\left( {\text{t}} \right)$ in $t = t_{0}$ and $\varvec{S}^{\left( n \right)} (t_{0} )$ exists. The Lagrange remainder during Lagrange polynomial interpolation is

$$ R_{n} \left( t \right) = \frac{{\varvec{S}^{{\left( {n + 1} \right)}} ( \epsilon )}}{{\left( {n + 1} \right)!}}\prod\nolimits_{i = 0}^{n} {(t - t_{i} )} $$

(10)

In $t \in {\mathbf{R}}$ and $\epsilon \,\, \in (\varvec{min}\left( {t,t_{0} , \ldots ,t_{n} } \right),\varvec{max}\left( {t,t_{0} , \ldots ,t_{n} } \right))$, the n + 1-order derivative of $\varvec{S}\left( {\text{t}} \right)$ exists. When n = 1, the remainder is

$$ R_{1} \left( t \right) = \frac{{\varvec{S}^{\left( 2 \right)} \left( \epsilon \right)}}{2}(t - t_{0} )(t - t_{1} ) $$

(11)

We take three adjacent points as subject investigated. By linear interpolation with points on side, we get the valuation of the middle point. Figure 2 indicates that errors are significantly different between edges and smooth parts. The truncation error δ:

$$ \delta = \left| {R_{1} \left( t \right)} \right| = \left| {\frac{{\varvec{S}^{\prime\prime} \left( \epsilon \right)}}{2}(t - t_{0} )(t - t_{1} )} \right| $$

(12)

Fig. 2.

The figure indicates that the mathematical model of Lagrange detector. The polynomial curves pass through three adjacent points. The black circles mean the value of the middle point that is got by linear interpolation with points on side. (a) and (b) indicate the deviation is large, corresponding to edges; (c) the deviation is tiny, corresponding to smooth parts. (Color figure online)

We take the difference between true value and interpolation value as

$$ diff = \left| {S_{mid, n = 1} - S_{mid} } \right| $$

(13)

We can use $diff$ to detect edges and curves. The method is the formula (13).

3.2 Experiment on 1D Signals

Exp.01:

We take $\sin (2\pi \omega t)$ as source signal and $\omega_{s} = 100.0$ as sampling rate. In an interval $\upomega \in [1, 9]$, we calculate $\overline{\delta }$ by the formula (13) and get the relationship curves of $\overline{\delta }$ and $\upomega$. The results follow as Fig. 3. When $\omega$ is larger, the feature of edges is more obvious.

Fig. 3.

The figure indicates the relationship curves of $\overline{\delta }$ and $\upomega$, that is the relationship of diff and edges. (a) source signals; (b) the relationship curves of $\overline{\delta }$ and $\upomega$. Using linear interpolation and (13), we get $\overline{\delta }$. (Color figure online)

3.3 Experiments on 2D Signals

Exp. 02:

Use a Gabor filter [9] to get the source image that is Fig. 4(a). After processing this image by l = 1 Lagrange detector, we get the result Fig. 4(b). To give firm analyses, we calculate $\overline{\delta }$ that is the average of $\varvec{diff}_{{{\mathbf{2}}\varvec{D}}}$. Figure 4(b) indicates the relation of $\overline{\delta }$ and the frequency. The results indicate that our approach is an isotropy detector.

Fig. 4.

The figure indicates the results of processing images via Gabor filter. (a) a matrix of the source images; (b) a matrix of the results in which top-to-bottom the column correspond to 0, π⁄4, π⁄2, 3π⁄4 on phase.

$$ \varvec{diff}_{{{\mathbf{2}}\varvec{D}}} = \left| {\varvec{I}_{l}^{'} - \varvec{I}} \right| (l = 0,1, \ldots ,N) $$

(14)

Figure 4(b) indicates Lagrange detector has the feature of isotropy on capturing edges, so we can just take one direction to contrast results of using Sobel operator, Laplace operator and Lagrange detector to process these images. Follow as Fig. 5:

Fig. 5.

Figures (1), (2), (3) and (4) are source images of which frequencies increase successively, that means the features of these patterns change from smooth to sharp. Figures (5), (6), (7) and (8) are the results by Sobel operator; for based on first-order derivative, operation region is two adjacent points, Sobel operator is sensitive on both smooth components and edges. Figures (9), (10), (11) and (12) are the results by Laplace operator of which processing on three adjacent points to calculate second-order derivative, resulting in losing some important information. Figures (13), (14), (15) and (16) are the results by Lagrange detector. From formula (12), Lagrange detector combines first-order derivative and second-order derivative. As a result, Lagrange detector can filter smooth components to some extent and capture as much edge information that change sharply as possible.

4 Lagrange Detector Processes Images to Capture Edges

4.1 Approach Description

After above all the analyses and experiments, we have the approach (Fig. 6):

Fig. 6.

The figure is the flow chart of Lagrange detector. (1) Get the gray image or RGB single channel from the source image, and use Gaussian filter to depress the noise; (2) take 1-order Lagrange interpolation at four directions 0, $\pi /4$, $\pi /2$, $3\pi /4$; (3) then get the sum of their absolute values and remove weak points by taking a threshold; (4) search edges at directions 0, $\pi /4$, $\pi /2$, $3\pi /4$, to remove short and tiny curves or points. (Color figure online)

4.2 Experimental Results

We perform some tests for making comparison between different edge detectors and Lagrange detector on the images, of which from left to right 1 is from MCM dataset [10], 2 is from Kodak dataset [11], 3 is from Sandwich dataset [12], 4 is from McGill dataset [13] and 5 is standard Lena image. The results are Fig. 7. Canny detector and LoG detector have been developed for many years, improved by researchers and developer for lots of times. So they perform well in presenting edges by curves, which means their strategies at processing the data from Sobel operator or Laplace operator. To a certain extent, Lagrange detector performs well in both outlines and tiny details.

Fig. 7.

These figures are test results of Canny detector, LoG detector and Lagrange detector. (a) source images; (b) results of using Canny detector; (c) results of using LoG detector; (d) results of Lagrange detector. (Color figure online)

5 Lagrange Detector Analysis and Conclusion

We propose a novel mathematical theory named Lagrange detector for edge detection. Based on Lagrange polynomial interpolation theory, this detector takes Lagrange remainder as an operator. The formulas (12) and (13) are the mathematical basis of detecting edges. By combining the first-order derivative and the second-order derivative, Lagrange detector can filter smooth components to some extent and capture as much edge information that change sharply as possible.

Like as Sobel operator and Laplace operator, Lagrange detector also has solid and definite mathematical expression shown in formula (2). While used to detect edge, Lagrange detector can presented as formula (12). At this case, the expression of Lagrange detector is equivalent to the truncation error $\varvec{S}^{\prime\prime} \left( \epsilon \right)$, which is the second-order derivation of neighborhood. Lagrange detector includes the characteristics of the first-order derivative and the second-order derivative. The distinction between traditional edge detection operators and Lagrange detector is that the traditional operators aim at get the extremum value of brightness variations that is the first-order derivative, but Lagrange detector uses Lagrange remainder to combine the first-order derivation and the first- and second-order derivative of neighborhood by multiplication. By calculating the truncation error of polynomial interpolation, we can estimate Lagrange remainder. Lagrange detector performs well in detecting both outlines and tiny details.

Lagrange detector provides a new theoretical approach for edge detection, which open a new window for low level image processing, and will be used as the basis for further studies on image processing.