Abstract
We present the implementation and use of filters based on masks and on statistical functions. All filters here considered operate on the image domain of finite images, so special care is taken to present actual implementations of practical algorithms. A generic convolution filter is implemented, and many instances of this kind of filters are shown: low-pass (mean, binomial and Gaussian) and high-pass filters (Laplacian) are applied to a test image which presents flat areas along with small details. A function for producing masks with arbitrary functions of the coordinates is provided, and then applied to building Gaussian masks. The relationship between blurring and variance in Gaussian masks is discussed and illustrated by examples. Image enhancement by unsharp masking is also discussed. The effect of filters is assessed by means of the resulting image and by the analysis of a profile. The minimum, median, and maximum filters are presented, along with a summary of the theoretical properties of order statistics.
Filtro de Amor: 20 vellos tuyos, 3 gramos de polvo de tus uñas, o tres gotas de tu sangre (imprescindible; es la clave del filtro de amor, sirve para que el efecto de amor del filtro vaya dirigido hacia ti). http://www.tarot-amor-gratis.com/filtro_amor.htm
In this chapter we will see how to build filters defined on the image domain. They all share the property of being functions of values around the pixel being processed, so we will not consider approaches based on transformations as, for instance, Fourier or Wavelet domain. Albeit limited, this approach will allow the user to build and experiment with image filters. A more model-based approach can be found in the book by Velho et al. (2008). Other important references are the works by Barrett and Myers (2004); Jain (1989); Lim (1989); Lira Chávez (2010); Gonzalez and Woods (1992); Myler and Weeks (1993); Russ (1998) among many others.
The reader is invited to recall the definitions of local operations, neighborhood and mask presented in Chap. 1 (p. x and 6). All the filters we consider will be defined on a mask: if \(f\) is the input image, then \(g=\Upsilon _M(f)\) is the result of applying the filter \(\Upsilon \) with respect to the mask \(M\) to \(f\). Each element of \(g\) is a function of the values observed in \(f\) locally with respect to the mask \(M\) and the values it conveys. For the sake of simplicity, all considered masks will be of the form presented in Eq. (2.3):
i.e., masks are squared sets of coordinates of (odd) side \(\ell \). Oftentimes, we will need to define values in each mask coordinate, i.e., we will work with matrices of the form \((m_{i,j})_{(\ell -1)/2\le i,j \le (\ell +1)/2}\). The term “mask” and the notation \(M\) will be employed for both the support and the values.
For the sake of simplicity, in all the theoretical descriptions we will assume that the input and output images are defined on the infinite support \(S=\mathbb{Z }^2\). When implementing the filters, the finite nature of the images needs to be taken into account. This can be done in at least two ways, namely, modifying the mask whenever needed (close to the edges of the original image), or applying the transformation only to those coordinates where the mask fits in. The latter will be used in our examples, as illustrated in the following code that will be common to all filters here discussed.
Listing 5.1 presents the general convolution filter. It takes two arguments as input, the image to be filtered and the mask. The first operations consist in discovering the number of lines and columns of the original image (lines 6 and 7, respectively), and the side of the mask (line 8). Line 10 creates the container for the output image g by copying the input image f; g is created with the dimensions, type, and additional attributes f has.
Assume f has m lines and n columns, and that it will be filtered by a mask of (even) side k. If we choose to filter only those pixels over which the mask fits, then our filter must start in line and column \((k-1)/2+1\), and stop in line \(m-(k-1)/2-1\) and column \(n-(k-1)/2-1\). Listing 5.1 performs this in lines 13 and 14; please notice the use of parenthesis, they are mandatory due to the operations precedence. Lines 15 and 16 are the core of the filtering procedure. The former captures the values in the image which are relevant, i.e., those which correspond to the mask centered at coordinate \((i,j)\). These values retain their matrix nature, and are multiplied, value by value, by the ones in the mask (line 16). Once the product has been performed, the sum command adds all the values.
Notice that Listing 5.1 does not perform any check on the dimensions of either the image f or the mask m. The reader is invited to make this function more robust by verifying, for instance, that the side of \(m\) is odd and that there are enough coordinates in f with respect to the size of m for the filter to be applied.
Figure 5.1a presents the image we will use to illustrate the results of applying filters. It is a \(320\times 403\) pixels scanned image of an ex libris in shades of gray. A line80 has been drawn in black, and the values are plotted in Fig. 5.1b; notice how bright areas intersected by this strip appear as values close to 1, whereas dark regions correspond to values close to 0.
It is noteworthy how some image features translate into profile variation. See, for instance, the tightly packed vertical dark strips which cross the profile to the left in Fig. 5.1a. They appear as a rapidly varying signal in Fig. 5.1b. This will be one of the most affected features due to its relatively small size and high contrast. The high values around column \(200\) correspond to the light region below the “O”, with small variations due to noise.
The following sections will deal with two of the main types of filters that can be defined on the data domain: convolutional and order statistics.
5.1 Convolutional Filters
Listing 5.1 presents the general structure of a convolutional filter. Any convolutional filter is defined by means of the values of the mask \(m=(m_{ij})_{{(\ell -1)/2\le i,j \le (\ell +1)/2}}\), with \(\ell \) the (odd) side of the mask, provided these values are held constant regardless the coordinate they are applied to and the values of the input image f. We say that \(g=f*m\) is the result of applying to f the convolution filter defined by the mask \(m\) when the output image is defined by
for every \((i,j)\). This operation consists in overlaying the mask on each coordinate of f, making the product of each value in the mask with the corresponding value in the image, and then adding all the products to compute the corresponding value in the output image \(g\). R performs this in a rather economic way; in fact, f[(i-km1d2):(i+km1d2),(j-km1d2):(j+km1d2)] (line 15 of Listing 5.1) captures all the required values in the input image f. Then * m (line 16) performs the pointwise product of these values with those in the mask m. Finally, the command sum returns the sum of all these products.
The result will depend only on the definition of the values in the mask. There are many ways to specify masks depending on the desired output. The reader is referred to the book by Goudail and Réfrégier (2003) for a comprehensive account of approaches. Particular masks will be denoted by the capital letter \(M\) with a mnemonic subscript. The identity mask \(M_\mathbb{1 }\), defined as \(m_{00}=1\) and zero elsewhere, produces a copy of f, i.e., \(f=f*M_\mathbb{1 }\!.\)
If all the entries of the mask are nonnegative, the filter is usually called “low-pass” because, when analyzed in the frequency domain, any filter of this class will reduce the high-frequency content (mainly due to noise and abrupt changes as, for instance, edges and small features) enhancing the low-frequency component (which is associated to flat or slowly varying areas). The books by Jain (1989) and Lim (1989) are excellent references for the analysis of images in the frequency domain, of which we will only borrow some terminology. In the following we will see the effect of a number of low-pass filters.
The first class of low-pass filters we will analyze is formed by those which consist of computing the mean using a some or all the elements of the mask. Consider the following masks:
They all add up to 1, a very convenient normalization in light of the following result.
claim
(The mean value of images filtered by convolution) Let f be an image such that its mean value is finite, and m a finite mask conveying finite values. The mean value of \(g=f*m\) is the product of the mean values of f and of m.
Since more often than not we will be working with images defined on a compact set of values K, the unitary cube \([0,1]^3\) for instance, this result allows us to transform f into g, whose values are not too far from K.
Listing 5.2 shows how to create the masks and the use of the ConvolutionFilter function already defined in Listing 5.1.
Figure 5.2 shows the result of applying the masks defined in Eqs. (5.2)indexMask and (5.3) The blurring effect in the first three is subtle, and becomes evident in the last one.
Figure 5.3 shows the profile of the original and filtered images restricted to the vertical strips. Notice the smoothing effect which consists in reducing the width of the peak, up to the point of almost eliminating the variability when the \(M_{121}\) mask is applied. The mean value, as expected, is preserved. Notice also that the characters become almost solid, but heavily blurred. The untouched region close to the edges is noticeable in this last example.
Convolution is quite general and is not limited to equal values, as considered so far. Besides the mean filter, another important low-pass convolution filter is the Gaussian mask. A Gaussian mask of side \(\ell \) and standard deviation \(\sigma \) is defined by first computing
for \(-(\ell -1)/2\le i,j\le (\ell -1)/2\) and then using the normalized mask \(m=m^{\prime }/\sum m(i,j)\). These values decrease exponentially fast from the center to the edges of the mask. Gaussian convolution filters are at the core of multilevel techniques (Medeiros et al. 2010).
Listing 5.3 presents the code that returns the Gaussian mask of side side and standard deviation s. Lines 3–5 define the auxiliary function DExp; it was defined within the scope of GaussianMask in order to make it aware of the input parameter s since, as required later, it has to have as arguments the two variables that will comprise the grid. This auxiliary function implements Eq. (5.4).
Line 7 builds one of the variables that will define the support of the mask, say i. There is no need to build the other variable (\(j\)) since they are equal: \(-(\ell -1)/2\le i, j\le (\ell -1)/2\). The function outer used in line 8 takes as input two vectors (the same in our case) and returns a grid whose values are the result of instantiating the function DExp in each of all the possible pairs of values of the input vectors. Line 9 normalizes the result in order to make the sum one.
The following code illustrates three examples of Gaussian masks as computed by the function presented in Listing 5.3, all of them of side five with varying standard deviations. Notice that when \(\sigma \rightarrow 0\) the Gaussian mask converges to the identity mask, while when \(\sigma \rightarrow \infty \) it becomes the mean over all the elements of the mask.
Figure 5.4 shows the result of applying two Gaussian filters with the same window (\(\ell =5\)) and two different values of standard variation: Figure 5.4a is the result for \(\sigma =1\), while Fig. 5.4b is the result for \(\sigma =10\). The difference is noticeable: the latter is more blurred than the former.
Somewhere between the mean filters, which has binary coefficients, and the Gaussian filter, whose values decay exponentially with the Euclidean distance to the center of the mask, there is the binomial filter. The binomial mask is defined as the product of all the binomial coefficients \(C^\ell _ i = \ell ! /(i!(\ell -i)!)\), where \(\ell \) is even (so the vector and the mask are odd) and \(0\le i\le \ell \).
The code presented in Listing 5.4 shows how to compute this masks in R using the choose function, the product of vectors (and matrices) %*%, and the transpose operator t. The values are shown before dividing by the sum of the mask, but the operation commented in line 4 must be activated when building binomial masks.
A few binomial masks are shown in the following.
Figure 5.5 presents the results of applying the binomial masks of sides 7 and 13 to the input image. Notice that the contrast is reduced in the second with respect to the first due to the extent of the mask; it “mixes” the values and both pure black and white tend to disappear.
Figure 5.6 presents the profile of line \(80\) at the strips (columns \(30\)–\(70\)) of the original image, and of the images filtered by the Gaussian (with \(\sigma =1\) and \(\sigma =10\)) and by the Binomial (of sides \(7\) and \(13\)) masks.
So far we have seen low-pass filters, i.e., those defined by nonnegative values of the filter mask. In the following we will see an important application of high-pass filters, namely edge detection. This will require the use of negative values.
The first class of high-pass filters we will describe is known as unsharp masking. The idea is to enhance the rapid variations in the image, which correspond to edges and sharp transitions, by subtracting a fraction \(\alpha \), \(0<\alpha <1\), of an smoothed version \(\varUpsilon (f)\) of the original image to the original image f. The result is \(|(1-\alpha )f+\alpha \varUpsilon (f)|\), conveniently scaled to the \([0,1]\) range.
Figure 5.7 presents the results of applying the unsharp masking technique with two different blurred images and the same value of \(\alpha =3/10\). Figure 5.7a was obtained applying the mask defined in Eq. (5.3), i.e., a \(11\times 11\) mask of ones, while Fig. 5.7b shows the result of applying the binomial mask of side 17. Albeit the difference between the masks, the results are alike. This powerful technique requires trial-and-error until the desired result is obtained.
Figure 5.7c shows the profile of the original and transformed images. Notice how relatively flat (smooth) areas had their values reduced, while sharp transitions are left unaltered. This transformation leads to a perceptual edge enhancement.
As previously said, oftentimes it is desirable to enhance edges and sharp transitions. If we envision the image \(f:S\rightarrow \mathbb{R }\) as the discrete version of a continuous function \(\widetilde{f}:\mathbb{R }^2\rightarrow \mathbb{R }\). Those features could be detected in \(\widetilde{f}\) by applying the gradient operator \(\nabla =(\partial /\partial x, \partial /\partial y)\), but since we do not possess the analytic (functional) description of \(\widetilde{f}\), a discrete version of \(\nabla \) could be applied to \(f\) instead. This can be performed by convolving the original image with the Laplacian mask:
The edges detected by the Laplacian mask are shown in Fig. 5.8a. They can be used to enhance the original image by adding the filtered version to it. Replacing the value 4 by 5 in Eq. (5.5) does the trick, and produces the image shown in Fig. 5.8b. Figure 5.8c presents the profiles of the original and enhanced images at line 80. We notice the same effect already described in the unsharp masking technique: a reduction of smoothly varying areas. The Laplacian filter tends to produce noisy images, since it enhances every little variation, even those due to noise.
5.2 Filters Based on Order Statistics
Order statistics can be used in a number of ways to define filters. If \(X_1,\dots ,X_n\) is a sample of size \(n\ge 2\) from the joint distribution characterized by the density \(h^*_{X_1,\dots ,X_n}(x_1,\dots ,x_n)\), then the ordered sample is the vector \(\varvec{X} = (X_{1:n},\dots ,X_{n:n})\) such that \(X_{1:n}\le \dots \le X_{n:n}\). A frequent, albeit seldom verified in practice, assumption is that the random variables are independent and identically distributed, so their joint distribution is the product of the marginal one \(h^*_{X_1,\dots ,X_n}(x_1,\dots ,x_n)=\prod \nolimits _{i=1}^n h(x_i)\). Assume also that the random variables are continuous, so the distribution of each is also characterized by the cumulative distribution function \(H(x)=\int _{-\infty }^x h(t) dt\), then the following results hold:
-
The joint distribution of the ordered sample is characterized by the joint density \(h^*_{\varvec{X}}(x_1,\dots ,x_n) = n! \prod \nolimits _{i=1}^n h(x_i)\mathbb{1 }_{x_1\le \dots \le x_n}(x_1,\dots , x_n)\).
-
The joint distribution of any pair of ordered samples \((X_{j:n},X_{k:n})\), \(j<k\), is characterized by the joint density
$$\begin{aligned} h^*_{X_{j:n},X_{k:n}}(x_j, x_k) =&{} \frac{n!}{(j-1)! (k-1-j)! (n-k)!} h(x_j) h(x_k)\cdot \\&{} H^{j-1}(x_j) (H(x_k)-H(x_j))^{k-1-j} (1-H(x_k))^{n-k} \mathbb{1 }_{(-\infty ,x_k)}(x_j). \end{aligned}$$ -
The distribution of the \(k\)th sample is characterized by the density
$$\begin{aligned} h_{X_{k:n}}(x) = \frac{n!}{(k-1)! (n-k)!} h(x) H^{k-1}(x) (1-H(x))^{n-k}. \end{aligned}$$
The main idea behind using order statistics is that, if well designed, they may lead to robust techniques. By “robust” we mean procedures which behave well if the underlying hypothesis are verified, but that also produce sensible results if they are not true for the data at hand. The reader is referred to the books by Huber (1981) and by Maronna et al. (2006) for a comprehensive introduction to the statistical theory of robustness.
One of the first “robust” approaches to image filtering consists in replacing the mean by the median. This can be easily done modifying the code which implements the convolution filter; cf. Listing 5.1, p. xx.
The only relevant difference between Listing 5.1 and the median filter presented in Listing 5.5 is in line 14 of the latter. First, the data from the image, which retain their matrix nature, are converted into a vector in order to be able to appy the median function. Then the median is computed and stored.
Figure 5.9 presents the result of applying median filters. Figure 5.9a shows the image filtered by the median on \(3\times 3\) windows, i.e., using nine observations, while Fig. 5.9b shows the result of using windows of side \(11\times 11\), i.e. \(121\) observations.
The difference between the mean and the median is clear when comparing Figs. 5.2d (p. xx) and 5.9a. The information in the former is almost gone, while it is quite well preserved in the latter (in particular, compare the dark pine tree to the right of the image). This is due to the fact that the mean does not spread the values as intensely as the mean, as can be assessed by comparing Figs. 5.3 (p. xx) and 5.9c. The price one pays for this is that the median is not as effective as the mean at reducing noise.
The reader is invited to write the code that implements the convex combination of the mean and the median filter, i.e., assume \(0\le \alpha \le 1\) and a mask of side k. Compute the mean over the \(k^2\) observations, say \(\overline{f}(i,j)\) and the median using the same observations, say \(\widetilde{f}(i,j)\), and return \(g(i,j) = \alpha \overline{f}(i,j) + (1-\alpha )\widetilde{f}(i,j)\) in every fit coordinate \((i,j)\). If \(\alpha =1\) the filter reduces to the mean, and if \(\alpha =0\) it becomes the median. Intermediate values will retain part of the good properties of both filters. An excellent reference for this particular subject is the work by Arias-Castro and Donoho (2009)
Rather than using the median, the minimum and maximum values observed in the mask can also be employed. The former tends to leave the image darker, while the latter usually leaves the image lighter.
Notice in Fig. 5.10c that the result of these filters provides a kind of nonlinear envelope to the original data, and that small details have been completely removed. These operations form the basis of one of the most general nonlinear frameworks for signal and image processing: Mathematical Morphology (Shih 2009).
Order statistics can also be used in adaptive procedures, as the ones discussed by Lopes et al. (1990). Consider, for instance, the \(\sigma \) filter. It consists of estimating the mean \(\bar{x}\) and standard deviation s on each window, and the computing the filtered value as the mean of only those observations which belong to the \([\bar{x}\pm k s]\) interval, with \(k>0\).
References
Arias-Castro, E., & Donoho, D. L. (2009). Does median filtering truly preserve edges better than linear filtering? Annals of Statistics, 37(3), 1172–1206.
Barrett, H. H., & Myers, K. J. (2004). Foundations of image science. Wiley-Interscience, NJ: Pure and Applied Optics.
Gonzalez, R. C., & Woods, R. E. (1992). Digital image processing. MA: Addison-Wesley.
Goudail, F., & Réfrégier, P. (2003). Statistical image processing techiques for noisy images: an application-oriented approach. Kluwer: New York.
Huber, P. J. (1981). Robust statistics. New York: Wiley.
Jain, A. K. (1989). Fundamentals of digital image processing. Englewood Cliffs, NJ: Prentice-Hall International Editions.
Lim, J. S. (1989). Two-dimensional signal and image processing: prentice hall signal processing series. Prentice Hall: Englewood Cliffs.
Lira Chávez, J. (2010). Tratamiento digital de imágenes multiespectrales (2 nd ed.). Universidad Nacional Autónoma de México. URL http://www.lulu.com..
Lopes, A., Touzi, R., & Nezry, E. (1990). Adaptive speckle filters and scene heterogeneity. IEEE Transactions on Geoscience and Remote Sensing, 28(6), 992–1000.
Maronna, R. A., Martin, R. D., & Yohai, V. J. (2006). Robust statistics: theory and methods. Wiley, England: Wiley series in Probability and Statistics.
Medeiros, M. D., Gonçalves, L. M. G. & Frery, A. C. (2010). Using fuzzy logic to enhance stereo matching in multiresolution images. Sensors, 10(2), 1093–1118. URL http://www.mdpi.com/1424-8220/10/2/1093, (Special issue: Instrumentation, Signal Treatment and Uncertainty Estimation in Sensors).
Myler, H. R., & Weeks, A. R. (1993). The pocket handbook of image processing algorithms in C. Prentice Hall: Englewood Cliffs NJ.
Russ, J. C. (1998). The image processing handbook (3rd ed.). CRC Press: USA.
Shih, F. Y. (2009). Image processing and mathematical morphology: fundamentals and applications. CRC Press: USA.
Velho, L., Frery, A. C., & Miranda, J. (2008). Image processing for computer graphics and vision (2nd ed.). London: Springer.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2013 Alejandro C. Frery
About this chapter
Cite this chapter
Frery, A.C., Perciano, T. (2013). Filters in the Image Domain. In: Introduction to Image Processing Using R. SpringerBriefs in Computer Science. Springer, London. https://doi.org/10.1007/978-1-4471-4950-7_5
Download citation
DOI: https://doi.org/10.1007/978-1-4471-4950-7_5
Published:
Publisher Name: Springer, London
Print ISBN: 978-1-4471-4949-1
Online ISBN: 978-1-4471-4950-7
eBook Packages: Computer ScienceComputer Science (R0)