State Estimation of the Performance of Gravity Tables Using Multispectral Image Analysis

Abstract

Gravity tables are important machinery that separate dense (healthy) grains from lighter (low yielding varieties) aiding in improving the overall quality of seed and grain processing. This paper aims at evaluating the operating states of such tables, which is a critical criterion required for the design and automation of the next generation of gravity separators. We present a method capable of detecting differences in grain densities, that as an elementary step forms the basis for a related optimization of gravity tables. The method is based on a multispectral imaging technology, capable of capturing differences in the surface chemistry of the kernels. The relevant micro-properties of the grains are estimated using a Canonical Discriminant Analysis (CDA) that segments the captured grains into individual kernels and we show that for wheat, our method correlates well with control measurements ($R^2 = 0.93$).

1 Introduction

Agriculture has been the key ingredient in the emergence and rise of human civilization as farming of domesticated plants/crops have fostered the modern world. Along with the improvement of modern farming techniques, advances in technology have aided in the growth and development of conventional grain and seed cleaning practices sch as gravity sorting. This fundamental technique aids in removing less dense material such as immature, low germination, and insect damaged seeds from the healthier seed lot. Typically this step is repeatedly undertaken in the final grain cleaning stages using equipment referred to as gravity tables [1]. These tables employ specific weight (measure of buoyancy or specific density) to separate material. The table has to be adjusted on a regular basis in order to perform optimally (every 15 min), and whereas gravity tables have improved in design and increased in capacity, the need for very skilled and experienced manual labor for the essential task of monitoring and adjustment, has remained the same. Unfortunately, optimizing the table is difficult, due to grains being undistinguishable to the naked eye. Only operators with years of experience are capable of optimizing the settings of the table and maintain optimal operation. Hence, this is the next bottleneck that should be overcome in order to increase the overall productivity of grain and seed cleaning operations. So far, automation of these machinery have not been addressed, and this work represents a nascent foray into this domain by attempting to develop a self regulating gravity table that measures its current operating state using an advanced vision system. The first step in solving this problem is to prove that cameras are capable of measuring significant (and relevant) differences in the grain product profiles, and furthermore show that this difference actually correlates with grain density. It should be noted that the grains that are fed are pre-cleaned in several preceding operations and the final material processed on a gravity table is (assumed to be) homogeneous in shape,size and distribution. Hence, the table will sort material according to weight, and consequently the objective of this first study is to develop a suitable methodology for the estimation of kernel weights in batches using multispectral image analysis. We further demonstrate the feasibility of such a vision driven process optimization using wheat as a model crop.

In the following, we will introduce the instruments used in the study, followed by a description of the experimental setup, and the chosen strategy. We finally show the results, and illustrate how they can be used in the optimization process of the table parameters.

Fig. 1.

The Westrup KA-1200 gravity table (1), and the VideometerLab2.0 (2).

2 Instrumentation

Before presenting the methods and results, we give a short introduction to the apparatus used in this study.

2.1 Gravity Tables

Figure 1.(1) shows an industrial gravity table, and Fig. 2 illustrates the working principle [1]. Feed material, $\mathbf {X}$, is introduced on a perforated deck, through which air is blowing. The table is tilted along two directions (“lengthwise" and “across" in Fig. 2). The deck is placed on springs, vibrating “lengthwise" with a constant frequency through an eccentric. As soon as material enters the deck, the air starts stratifying it vertically due to the inherent differences in specific weight (lighter on top of heavier). The “across"-angle is forces material to flow off the table during operation while the “lengthwise" tilt forces material to flow downwards (due to gravity). In addition, the vibrating action throws the bottom (heavy) material upwards, resulting in a horizontal separation of lower (heavy) from upper (lighter) material along the deck length. Finally, material exits the table as segregated fractions, $\varvec{Y}$.

Fig. 2.

Principle of operation of a gravity table.

2.2 Camera System

Images were captured using a VideometerLab vision system. The instrument captures images at 19 different wavelengths (from UV (365 nm) to NIR (970 nm)) with an image resolution of $2056\times 2056$ pixels ($44.5\,\upmu \mathrm{m}$ per pixel and $\approx$306 Mb per image). The original technology behind the instrument was concieved at the Technical University of Denmark (DTU) and this system is now being developed and commercialized by Videometer A/S [2, 4] (Fig. 1.(2)). The camera acquires images in which pixels contain a spectrum of (reflectance) values obtained at the different wavelengths. This information, is representative of the chemistry of an area above (and below) the surface. The slightest differences can be captured, through the use of multivariate statistical methods, capable of outperforming the human eye in both sensitivity, specificity and consistency. In this study, we use a VideometerLab (3.0) and the latest version of the accompanying software [2].

3 Materials and Methods

3.1 Gravity Table Setup

The table used in this study is a Westrup KA-1200 (Fig. 1.(1), having $N=4$ outlets (labelled a, b, c, and d in Fig. 2). To simulate a production line, a hopper is placed next to the table, feeding the deck with material, recycled through a conveyor belt collecting (separated) material that leaves the table. This material is further transported to the side of the table, where a vacuum pump lifts the seeds back into the hopper. A 100 kilos of wheat from a grain cleaning plant is filled into the hopper and the table is started. We invited an expert to set the table, making sure it was running in an optimal state: $P^{opt} = [F^{opt}, \varGamma ^{opt}, \theta ^{opt}_L, \theta ^{opt}_T] = [85,40,15,8]$ (see Sect. 1). The parameter values are entered as percentage-of-maximum on the machine.

3.2 Image Acquisition and Analysis

All images are acquired the same way using a VideometerLab (v.3.0). For equal volumes (cups) of grain, images are acquired sequentially using the hopper, feeder and conveyor system described earlier. First, the instrument is calibrated for intensity and geometric distortions and these settings are kept constant for all subsequent acquisitions. Images of the reflectance are captured at 19 different wavelengths (from UV, through visual, to NIR), into one single high-resolution spectral image ($2056\,\times \,2056\,\times \,19$ pixels). A canonical discriminant analysis (CDA) model [5, 6] is created to detect wheat kernels in the images. Pixel regions are labelled in the background and foreground (kernels), after which the between- and within covariance matrices, $\varvec{S}_\text {B}$ and $\varvec{S}_\text {W}$, are calculated for the two classes and used to minimize the Rayleigh quotient

$$\begin{aligned} \min _{\varvec{w}} J(\varvec{w}) = \frac{\varvec{w}^T \varvec{S}_\text {B} \varvec{w}}{\varvec{w}^T \varvec{S}_\text {W} \varvec{w}}. \end{aligned}$$

The solution, $\varvec{w}$, is a new projection (score) with maximum separation between foreground and background pixels. This is used in the segmentation that is done in two steps: (1) a pre-segmentation, where a threshold is applied to the score image, returning regions containing the kernels, (2) a watershed segmentation [7] is applied to the segmented regions in order to secure separate objects (if they are touching each other). From each kernel features are extracted relating to shape, intensity and texture, for each of the wavelengths. A total of 54 features were extracted from all kernel images.

3.3 Data Sampling

Two sets of samples were taken from the table: for the single kernel analysis, and for the density profiling (see Fig. 3). Few estimates of specific weight are generally accepted: The cup-method, measuring the weight of a fixed volume, recalculated to kg/m$^{\text {2}}$, and 2) The 1000 kernel weight, calculating the average weight per kernel $\times ~1000$. Both methods give the same result, but during the course of development, we found that the cup-method was sensitive to variations in grain compaction within the cup (less important for increasing volumes). Therefore, we chose the 1000 kernel weight as the relevant metric, as it was robust and less sensitive towards sampling variation.

Fig. 3.

Processing “pathway" of the analysis.

Single Kernel Modelling. A total of $4 \times 48 = 192$ wheat kernels were sampled randomly from the outlets of the table, and their weight found on an A&D HR-100AZ (102g/0.1 mg) scale, $\varvec{M} = \{ m_1,\ldots , m_{192} \}^T$. Images of the single kernels, were acquired on both sides (i.e. ventral and dorsal) using the VideometerLab and key features extracted. In the analysis, when creating a data set, we randomly select one side and use the representative features, such that we obtain a matrix of independent variables, $\varvec{X}_M \in \mathbb {R}^{192 \times 54}$. Next, kernel weight was modelled using

$$\begin{aligned} \varvec{M} = f(\varvec{X}_M,\hat{\varvec{\beta }}^{*}) \end{aligned}$$

assumed to be linear, for simplicity. We applied stepwise regression [3] combined with a scoring schema. The scoring is done based on counting the number of times a feature has been selected, and for different models this is created randomly by picking kernel sides randomly (randomly picking only 90% of the kernels). We repeated this 1000 times, and choose the features that were selected for more than 30% of the times. The final generated regression model is used to estimate $\hat{\varvec{\beta }}^{*}$ and this is further used in modelling the table density.

Table Density Profiling. Table parameters are varied individually, and 5 points are sampled $\pm 5\%$ around the optimal setting, $P^{opt}$. A total of 80 samples are taken (4 parameters $\times$ 5 parameter values $\times$ 4 outlets). We keep the load fixed (typically for production). Samples are collected in 0.147 L cups, taken at four positions equally distributed along the output side of the table. The weight of each sample (minus cup) is measured, $\varvec{W} = \{ w_1,\ldots , w_{80} \}^T$, and images of the kernels in each cup are acquired using the VideometerLab, with an attached feeder. Figures 4 and 7 show the resulting output for the varying parameters and the profile across the table.

4 Results

The kernels were analyzed as illustrated in Fig. 3. After the image analysis, features were exported as a comma separated (CSV) file into Matlab$\copyright$ (R2014a), from where the analysis was done. First, we create a model capable of estimating the weight of single kernels, based on visual characteristics only. Next, we use this model to estimate the weight of batches of kernels based on images of the individual kernels.

Fig. 4.

The sampled wheat kernels and their measured weight.

Fig. 5.

Result after model generation and feature selection by stepwise regression (only 5 features).

Fig. 6.

Samples from the gravity table, and their measured weight. The difference between light (Outlet a) and heavy (Outlet d) material is 7% (if volume is assumed to be constant).

4.1 Single Kernel Modelling

Figures 4.(1) and 4.(2) shows the kernels picked for the single kernel weight model generation, and their measured weight. The average kernel weight was $45.15 (\pm 11.90)$ mg. And although the kernels were sampled equally from each of the four outlets of the gravity table, no significant (p = 0.05) difference between the weight of kernels could be detected (a and d are different at p = 0.11). In addition, no significant difference could be observed in the features from the different sides. A total of five out of 54 features were selected (with score): $\text {Area} (70\%)$, $\text {Roundness} (37\%)$, $\text {Mean}_{890\,nm} (75\%)$, $\text {Mean}_{940\,nm} (90\%)$, $\text {Volume} (85\%)$. Figure 5 shows the final result, with the predicted weight as function of the real kernel weight ($R^2 = 0.91$). The remaining features all were chosen less than $10\%$ of the times.

Fig. 7.

Plot of the estimated 1000 kernel weight as function of the measured 1000 kernel weight.

4.2 Table Density Profiling

A total of 4964 images (1595 GB) were acquired (${\approx }20$ images per cup) and 364 389 kernels were detected (${\approx }73$ per image). Using the weight model, $\hat{\varvec{\beta }}^{*}$, we estimated the average weight of the kernels in each cup and multiplied it with a 1000 to obtain the thousand kernel weight. As a reference we calculated the 1000 kernel weight based on the measured cup weights. Figure 7 shows the correlation between measurements and the estimates based on image features alone, showing a high correlation ($R^2 = 0.93$). A slight bias (difference in the means) can be observed, coming from an inexact number of kernels. Some kernels were not visible in the camera field of view, or removed (half kernels at the border) during the segmentation process. A decrease in detected kernels relative to the actual, causes the estimated weight of the kernels to be less than the actual weight and the measured weight to be slightly higher. Figure 6 shows that we can use the estimated weight to optimize the process. If we had increased the fluidization conditions slightly, we would have obtained a much better separation between the ends (Outlets a and d) of the table. A similar trend is noted for the vibrational conditions as well, while changing the two angles would have had lower impacts. Figure 8 show the resulting output profiles from the table for the varying table settings. Figure 9 show the output profile for the four different sample points along the table.

Fig. 8.

Variation in the density output for varying parameters.

Fig. 9.

Profile of density on the output.

5 Discussion

We have shown that the density profile of the output of gravity tables can be estimated using a multispectral camera setup coupled with simple CDA and regression modelling. As can be seen, there is a bias in the estimate of the weight due to missing seeds in the images that are outside the optical field of the camera (kernels being positioned at the edges of the image). This bias was noted to be constant and the measured metric can hence be utilized to quantify the relevant grain micro-properties (specific gravity). Such an analysis could be extended to other crop types such as maize, barley etc.