Automated ground-based cloud recognition

Abstract

Recognition of naturally occurring objects is a challenging task. In particular, the recognition of clouds is particularly challenging as the texture of such objects is extremely variable under different atmospheric conditions. There are several benefits of a practical system that can detect and recognise clouds in natural images especially for applications such as air traffic control. In this paper, we test well-known texture feature extraction approaches for automatically training a classifier system to recognise cumulus, towering cumulus, cumulo-nimbus clouds, sky and other clouds. For cloud recognition, we use a total of five different feature extraction methods, namely autocorrelation, co-occurrence matrices, edge frequency, Law’s features and primitive length. We use the k-nearest neighbour and neural network classifiers for identifying cloud types in test images. This exhaustive testing gives us a better understanding of the strengths and limitations of different feature extraction methods and classification techniques on the given problem. In particular, we find that no single feature extraction method is best suited for recognising all classes. Each method has its own merits. We discuss these merits individually and suggest further improvements in this difficult area.

Appendices

Appendix Texture features used for cloud recognition

1.1 Autocorrelation

An autocorrelation function can be evaluated, that measures this coarseness. This function evaluates the linear spatial relationships between primitives. If the primitives are large, the function decreases slowly with increasing distance whereas it decreases rapidly if texture consists of small primitives. However, if the primitives are periodic, then the autocorrelation increases and decreases periodically with distance. The set of autocorrelation coefficients C shown below are used as texture features:

$$ C_{ff} (p,q) = \frac{{MN}} {{(M - p)(N - q)}}\frac{{\sum\nolimits_{i = 1}^{M - p} {\sum\nolimits_{j = 1}^{N - q} {f(i,j)f(i + p,j + q)} } }} {{\sum\nolimits_{i = 1}^M {\sum\nolimits_{j = 1}^N {f^2 (i,j)} } }}, $$

where p, q is the positional difference in the i, j direction, and M, N are image dimensions.

1.2 Co-occurrence matrices

Co-occurrence matrix is the joint probability occurrence of grey levels i and j for two pixels with a defined spatial relationship in an image. The spatial relationship is defined in terms of distance d and angle θ. If the texture is coarse, and distance d is small compared to the size of the texture elements, the pairs of points at distance d should have similar grey levels. Conversely, for a fine texture, if distance d is comparable to the texture size, then the grey levels of points separated by distance d should often be quite different, so that the values in the co-occurrence matrix should be spread out relatively uniformly. Hence, a good way to analyse texture coarseness would be, for various values of distance d, some measure of scatter of the co-occurrence matrix around the main diagonal. Similarly, if the texture has some direction, that is, is coarser in one direction than another, then the degree of spread of the values about the main diagonal in the co-occurrence matrix should vary with the direction θ. Thus, texture directionality can be analysed by comparing spread measures of co-occurrence matrices constructed at various distances d and direction θ.

From co-occurrence matrices, a variety of features can be extracted. From each co-occurrence matrix, 14 statistical measures are extracted proposed by Haralick et al. [13]. These are angular second moment, contrast, variance, inverse different moment, sum average, sum variance, sum entropy, entropy, difference variance, difference entropy, two information measures of correlation, and maximum correlation coefficient.

1.3 Edge frequency

A number of edge detectors can be used to yield an edge image from an original image. We can compute an edge-dependent texture description function E as follows:

$$ E(d) = |f(i,j) - f(i + d,j)| + |f(i,j) - f(i - d,j)| + |f(i,j) - f(i,j + d)| + f(i,j) - f(i,j - d)|. $$

This function is inversely related to the autocorrelation function. Texture features can be evaluated by choosing specified distances d (pixel distance).

1.4 Law’s method

Laws observed that certain gradient operators such as Laplacian and Sobel operators accentuated the underlying microstructure of texture within an image. This was the basis for a feature extraction scheme based on a series of pixel impulse response arrays obtained from combinations of 1-D vectors shown below. Each 1-D array is associated with an underlying microstructure and labelled using an acronym accordingly. The arrays are convolved with other arrays in a combinatorial manner to generate a total of 25 masks, typically labelled as L5L5 for the mask resulting from the convolution of the two L5 arrays.

Five 1-D arrays identified by Laws

$$ \begin{aligned} {\text{Level L}}5 & = [\begin{array}{*{20}l} 1 \hfill & 4 \hfill & 6 \hfill & 4 \hfill & 1 \hfill \\ \end{array} ] \\ {\text{Edge E}}5 & = [\begin{array}{*{20}l} { - 1} \hfill & { - 2} \hfill & 0 \hfill & 2 \hfill & 1 \hfill \\ \end{array} ] \\ {\text{Spot S}}5 & = [ - \begin{array}{*{20}l} 1 \hfill & 0 \hfill & 2 \hfill & 0 \hfill & { - 1} \hfill \\ \end{array} ] \\ {\text{Wave W}}5 & = [\begin{array}{*{20}l} { - 1} \hfill & 2 \hfill & 0 \hfill & { - 2} \hfill & 1 \hfill \\ \end{array} ] \\ {\text{Ripple R}}5 & = [\begin{array}{*{20}l} 1 \hfill & { - 4} \hfill & 6 \hfill & { - 4} \hfill & 1 \hfill \\ \end{array} ]. \\ \end{aligned} $$

These masks are subsequently convolved with a texture field to accentuate its microstructure giving an image from which the energy of the microstructure arrays is measured together with other statistics. The energy measure for a neighbourhood centred at F(j,k), S(j,k), is based on the neighbourhood standard deviation computed from the mean image amplitude:

$$ S(j,k) = \frac{1} {{W^2 }}\left[ {\sum\limits_{m = - w}^w {\sum\limits_{n = - w}^w {[F(j + m,k + n) - M(j + m,k + n)]^2 } } } \right]^{1/2} , $$

where W × W is the pixel neighbourhood and the mean image amplitude M(j,k) is defined as:

$$ M(j,k) = \frac{1} {{W^2 }}\sum\limits_{m = - w}^w {\sum\limits_{n = - w}^w {F(j + m,k + n)} } . $$

1.5 Run length encoding

A large number of neighbouring pixels of the same grey level represent a coarse texture, a small number of these pixels represent a fine texture, and the lengths of texture primitives in different directions can be used for texture description. A primitive is a maximum contiguous set of constant grey level pixels located in a line. These can then be described by grey level, length and direction. The texture description features are then based on computation of continuous probabilities of the length and the grey level of primitives in the texture.

Let B(a,r) be the number of primitives of all directions having length r and grey level a. Let A be the area of the region in question, let L be the number of grey level within that region and let N _r be the maximum primitive length within the image. The texture description features can then be determined as follows. Let K be the total number of runs:

$$ K = \sum\limits_{a = 1}^L {\sum\limits_{r = 1}^{N_r } {B(a,r)} } . $$

The features are defined as:

• (a) Short primitives emphasis

  $$ \frac{1} {K}\sum\limits_{a = 1}^L {\sum\limits_{r = 1}^{N_r } {\frac{{B(a,r)}} {{r^2 }}} } . $$

• (b) Long primitives emphasis

  $$ \frac{1} {K}\sum\limits_{a = 1}^L {\sum\limits_{r = 1}^{N_r } {B(a,r)r^2 } } . $$

• (c) Grey level uniformity

  $$ \frac{1} {K}\sum\limits_{a = 1}^L {\left[ {\sum\limits_{r = 1}^{N_r } {B(a,r)r^2 } } \right]} . $$

• (d) Primitive length uniformity

  $$ \frac{1} {K}\sum\limits_{a = 1}^L {\left[ {\sum\limits_{r = 1}^{N_r } {B(a,r)} } \right]} ^2 . $$

• (e) Primitive percentage

  $$ \frac{K} {{\sum\nolimits_{a = 1}^L {\sum\nolimits_{r - 1}^{N_r } {rB(a,r)} } }} = \frac{K} {A}. $$

Author biographies

Dr. Maneesha Singh is currently a Research Fellow at the Research School of Informatics, Loughborough University. She received her M.Phil. and Ph.D. degrees from the University of Exeter. Her past research investigated image processing optimisation for aviation security for analysing dual X-ray luggage images for explosive detection. Her current work focuses on developing novel machine vision-based inspection for railway and steel industry. She is a member of IEEE, and Membership Secretary of BCS Specialist group on Pattern Analysis and Robotics. She was the Organising Chair of the third International Conference on Advances in Pattern Recognition, 2005, and is the Organising Chair of Summer Schools on Pattern Recognition organised annually in the UK. She has published more than 25 papers in the area of computer vision and machine learning.

Mr. Matthew Glennen did his B.Sc. and M.Sc. degrees from the University of Exeter, UK. He is currently working at Home Office Scientific Development Branch in UK.