Introduction

One of the key agrotechnical treatments used for cereals and other cultivated plants determining their growth and satisfying crop is sowing1. In Poland, sowing is carried out with the use of mechanical and pneumatic sowers with working widths greater than 3 m. The development of sustainable agriculture in Europe and the world causes that agrotechnical processes, including sowing, are carried out more and more often by machines with huge working widths from 4 to 6 m or even up to 9 or 12 m1 with foldable frame systems2,3,4. Currently, sowers and cultivation and sowing aggregates are the most commonly used sowing machines5. By applying integrated soil cultivation6 one can come across the problems related to quality control and sowing breaks caused by a coulter blockage7 (blocked by wet soil or straw), which in turn may lead to blockage of the sowing pipe. The foldable frames of the sowers require application of pneumatic systems to transport the sowing material to the coulters.

In view of the above, machine operators face problems with installing pneumatical sowers, caused by insufficient evenness of seed division into separate rows1,8,9,10,11,12. This problem can lead to seed transport delays and difficulties to control sowing in terms of detecting sowing brakes caused by blocked feed pipes (transporting seeds).

In recent years, one can observe the development of various sowing control systems, mainly those based on photoelectric sensors capable of detecting individual seeds. One of the most popular solutions devised by the Vaderstad company is a system known under the commercial name of “seed eye”13. So far it has not been widely used for controlling the movement of sowing material in mechanical or pneumatic sowers14. The available structures, do not commonly use solutions such as placing sensors for blockages detection or sowing material counting at the end of a sowing pipe or in a coulter. In addition, no research results have been found to provide new techniques for detecting kernels in sowing material in order to count them and detect blockage of sowing pipes.

However, in specialized literature one can find examples of application of optical sensors used as markers, which allow fast, non-destructive and reliable identification of the best doses of gamma radiation in order to stimulate the parameters of soya growth15. What is more, one can also find research focusing on perceptibility of those sensors16. There are also systems with piezoelectric sensors inside the sower distribution head17 and on the outlet stub pipes18 as well as cases of using capacity sensors U. S. Patent 4782282 A. Unfortunately, insufficient capacity changes possible with this solution, indicate poor future perspectives for this method. In view of the above, it seems justified to search for online solutions that will enable to detect blockages as well as count seeds that are sowed, that can eliminate the necessity of carrying the so called calibration test. It will allow efficient calibration of the sowing unit both in fertilizers and seed sowers and other cultivation devices19.

Lack of a cheap sowing control system of cereal kernels to indicate blocked pipes results in occurrence of sowing gaps, (breaks) and subsequently leaves some parts of acreage unsown (dominated by weeds). Application of secondary sowing in the unsown area is possible but expensive, and this solution is not used by farm owners in Poland or in Europe20.

The aim of this study is to determine the velocity of kernels to be used for defining technical conditions and forces occurring when a kernel collides into the sensor, a bar equipped with a control system designed for the needs of the sensor construction, or a control bar used in mechanical and pneumatic sower units.

In standard multiple row sowers and automated sowers21,22 for sowing corn and other cultivated plants, the above mentioned system could be applied in order to control blockages of sowing pipes. The potential of the above solution can be utilized for detecting blockages in pipes transporting seeds and fertilizers according to a method called strip-till. In order to provide a correct study of such a system it is crucial to obtain all necessary technical data to construct the sensor and the whole unit. The first step is to determine the velocity of kernels in order to estimate forces occurring when a kernel collides into a sensor or a control bar. Until now, there has been no database of velocity distribution of kernels in the sowing pipe outlet (air and seed), or a method determining velocity of objects (kernels).

Currently, there are attempts to find solutions, that will allow to control the velocity of objects (seeds) in real time.

In this study, the authors have used one of the artificial intelligence methods, i.e. artificial neural networks ANN whose functioning involves proper using of mathematical or programing formulas23,24. Artificial neural networks are most often used for data with no ordered or simple computing structure25,26. ANNs are mostly used in information including processing of image (bitmap)27,28,29, sound (acoustic wave)30,31 and text (numerical data)32,33,34,35,36. The simplest form of a neural model consists of layers of neurons, in which a neuron from one layer generates a signal constituting one of the arguments for each neuron of another layer.

A decision was made to examine the velocity of two medium size seeds (wheat and barley) and one fine-grained seed (rape). It should be added that Multi-Layer Perceptron (MLP) neural networks were used in order to detect and count the number of kernels in the sowing material for the most popular varieties of cereals cultivated in Poland. The practical goal was to develop neural networks capable of fast identification of various types of cereal kernels. The use of an ultra-high-speed camera to register images for the needs of designing new sensors and sowing control systems can be viewed as an innovative approach.

Materials and methods

Preparation of samples

It should be said that for the needs of the research, the authors used varieties of cereals that are most commonly sown in Poland. It is also worth noting that, based on statistical data from 2019, including the last ten years (https://www.fao.org/faostat/en), Poland is ranked seventh in the global ranking of rape production. The research material consisted of winter wheat seeds Elixer with the initial moisture level of 9.8%, winter barley seeds LG Veronika with the initial moisture level of 12.2% and winter rape seeds Kite with the initial moisture level of 9.0%. All the research material came from the Main Seed Warehouse Top Farms Seeds, Production Plant in Runów located in Greater Poland Province. Based on the announcement of the Marshal of the Sejm of the Republic of Poland about the Legal Protection of Plant Varieties of January 22, 2021 (Journal of Laws of 2021, item 213) and the breeder's declaration that the indicated varieties: Kite (rape), LG Veronika (barley) and Elixer (wheat) are protected by law by the breeder; the authors have received this permission. The breeder agreed to provide the above-mentioned plant material, which complies with the national guidelines of the Main Seed Warehouse Top Farms Seeds, a production plant in Runów, located in the Greater Poland Province. With the breeder's consent, the authors were allowed to use their plant material only for the purposes of scientific research, including carrying out tests of e.g. seeding simulation.

Prior to the experiment, the sowing dose (ration) was determined for each variety of cereals. The sowing dose (ration) was determined based on the most commonly used dose in Poland. For the needs of the research, a dose (ration) of 115 kg ha−1 and 185 kg ha−1 was established for wheat, a dose of 170 kg ha−1 was established for barley and a dose of 4.4 kg ha−1 was established for rape, respectively. A dose of 115 kg ha−1 is used for wheat seeds in fertile soils, whereas a dose of 185 kg ha−1 is used in medium fertile soils. It should be added that all kernels were properly selected prior to the research.

Next stage of the research involved adapting the stream of kernel mass transported via pneumatic pipe for selected structures: I and II. To check a possibility of controlling the sowing material movement in the sower elements, irrespectively of the seed dose, two dose levels were examined for wheat kernels, i.e. 280 seeds/m2 and 450 seeds/m2 (structure I and II). In the case of barley and rape seeds, only one dose level was used. In terms of barley, sowing density was set at the level of 375 seeds/m2 (structure I and II). For rape, sowing density was set at the level of 80 seeds/m2 (structure I and II).

Test stand

In order to provide the seed blockage control system with real working conditions, the author’s original research stand was constructed, which consisted of (Fig. 1):

  • Sowing unit consisting of seed crate, sowing rollers driven by electric engine with adjustable rotation speed (2a, 2b),

  • Air and seed pipe with configuration I (2a),

  • Air and seed pipe with configuration II (2b),

  • Chronos camera 1.4. (3a),

  • A tripod camera stands with adjustable height (4a),

  • Screen with scale (10 mm) (5a),

  • Two LED lamps with stabilization system (6a),

  • Lamp stand with adjustable height (7a),

  • Seed container (8a).

  • Vacuum cleaner with blower and rotation regulation (9a),

  • Air pipe (10a),

  • Load-bearing construction of the stand (11a).

Figure 1
figure 1

The scheme of the test stand by Gierz et al.31 (a) pipe configuration I, (b) pipe configuration II,, 1- drilling unit (a feeder), 2-seed pipe, 3- high-speed camera, 4- adjustable camera stand, 5—Screen with scale , 6- LED lamps with stabilization system, 7- adjustable lamp stand, 8- seed container, 9-a vacuum cleaner with a blower function, 10- air pipe, 11- load-bearing construction of the stand.

Full size image

Like in the previous research37, velocity of air stream transporting kernel in a pneumatic tube was 15, 20, 25 m s−1 (those values of air stream velocities are typically used in pneumatic sowers). It should be observed that various configurations of the pneumatic tube arrangements and lengths are used in pneumatic sowers. With reference to the earlier research37, it was decided to compare the two most common seed-air tube configurations. The first of them (a) in the shape of extended “S” letter with the tube length of 1.5 m. and the second (b) in the shape of tightened “S” letter with the tube length of 2.0 m.

The kernel dispenser was devised and built according to the pattern of a standard dispenser used in sowers, which was described in an earlier experiment37.

The tests involved providing the dispenser with proper rotation velocity in order to obtain sowing density, that is, 280 grains of wheat/m2 and 450 grains of wheat/m2 respectively, 375 grains of barley/m2 and 80 grains of rapeseed/m2.

As part of the control process of kernel transporting air stream velocity, A pressure Anemometer called VOLTCRAFT VPT−100 was used . The measuring range of the device is 1 to 80 m s−1 and measuring accuracy + /− 2.5%. Air stream transporting seeds in a seed-air tube was generated by vacuum STANLEY SXVC20PTE with rated power of 1200 W, which has also the function of blower. The air stream velocity within the range from 5 to 50 m s−1 was regulated with potentiometer, which controlled the vacuum engine speed.

The following factors were accepted for the needs of this study:

  1. (a)

    Fixed:

    • Dose of seeds to be sown: barley 526.50 g min−1, which corresponds to 170.0 kg ha−1 and rapeseed 13.85 g min−1, which corresponds to 4.4 kg ha−1;

    • Sowing velocity: 15 km h−1,

    • Tube configuration: (a) and (b) (Fig. 2),

    Figure 2
    figure 2

    The scheme of image processing and creation of Artificial Neural Networks.

    Full size image
  2. (b)

    Variable:

    • Velocity of air stream: 15 m s−1, 20 m s−1, 25 m s−1,

    • Grain variety: wheat, barley, rapeseed

    • Dose of seeds to be sown: wheat 359.0 g min−1 , which corresponds to 115.0 kg ha−1 and 575.65 g min−1, which corresponds to 185.0 kg ha−1

  3. (c)

    Result factors:

    • Kernel velocity m s−1,

    • Kernel acceleration m s-2.

Preparation of research material with camera

Graphic data of wheat, barley and rapeseed with different variables (velocity and grain variety, configuration of seed-air tube) was acquired using a high-speed camera HighSpeedStar5, which is characterized by high image projection frequency. This device is equipped with an image converter of CMOS type. This fast device enables taking videos at 3000 frames per second, with the resolution of 1024 × 1024 and taking photos at 30,000 photos per second with the resolution of 256 × 256.

As a result, 18 videos in .im7 format were taken, for which data recording speed was about 9400 kb/s with the image resolution of 640 × 360. Six research variants were created in order to develop learning sets, and the next step was to design neural networks for them. Each research variant consisted of 3 videos with 3 classes of grain varieties (wheat, barley, rapeseed). The same parameters of image projection were used, i.e. 58 frames per second, to provide measuring repeatability of the results and compare velocities.

Image processing in MATLAB

At the first stage of the research, image conversion was carried out to obtain a 24-bitmap image with 1437 × 1253 resolution in .jpg format (Fig. 2). It should be added that the above conversion applied to each class (grain variety) consisting of 212 images in .im7 format. In order to do that the author used software called MATLAB 2021b, which enabled image conversion from .jm7 to .jpg. To acquire image in .jpg format, the author used software called The PIVMat Toolbox for MATLAB by Frederic Moisy (https://www.mathworks.com) and library readimx-v2.1.9 (https://www.lavision.de). PIVMat Toolbox contains a set of functions enabling to import images in .im7 format. Library readimx allowed to read a given file format effectively with MATLAB. Another step was to crop the 24-bitmap 1437 × 1253 resolution images in jpg format to 931 × 931 resolution images.

The next stage was to process the images so as to highlight characteristics of the texture. The texture is used to interpret the image details33,38,39, it carries information about its surface, color and other parameters related to the lighting model including: color of reflected and stray light, degree of transparency and light refraction coefficient. An analysis of the texture was carried out using a grey-level co-occurrence matrix (GLCM)27,40. In order to apply GLCM to present details of the surface with procedural textures, it was necessary to carry out image conversion from 24-bitmap to 8-bitmap image depth in MATLAB. The prepared database consisting of 8-bitmap images was imported again to MATLAB in order to isolate 20 descriptors of texture from Haralick library41,42,43,44: contrast (contr)45, correlation (corrm), cluster prominence (cprom), cluster shade (cshad), dissimilarity (dissi), angular second moment (energ)37, entropy (entro)37, homogeneity (homom), homop, maxiumum probability (maxpr), sum of square variance (sosvh), sum of average (savgh), sum of variance (svarh), sum of entropy (senth), difference variance (dvarh), difference entropy (denth), info. measure of correlation 1 (inf1h), info. measure of correlation 2 (inf21h), inverse difference normalized (indnc), inverse difference moment (idmnc).

Multilayer perceptron networks

The research included the process of network machine learning. Multi-Layer Perceptron (MLP) neural networks were designed. Each structure of Multi-Layer Perceptron Layer Network (MLPN) consisted of 20 neurons in the input layer, 15 neurons in the hidden layer and 3 neurons in the output layer (Fig. 3). The input layer of MLPN defined 20 texture descriptors and determined classes of kernels (variety of grain). Each class of the learning set included 212 learning cases. In the end, the set consisted of 636 cases (Table 2). Six neural networks were prepared for the comparison process to assess the effectiveness of velocity distribution recognition for the selected classes of kernels. The configurations of networks with different kernel transporting air stream velocities (15 m s−1, 20 m s−1, 25 m s−1) and type of the seed-air tube (p1, p2) are presented in Table 1.

Figure 3
figure 3

Artificial Neural Network used for the experimental data.

Full size image
Table 1 Configuration of Artificial Neural Networks based on air velocity and position of the seminal-air tube.
Full size table

Statistical analysis

A statistical analysis was carried out for individual texture descriptors, and Tukey test was carried out for p value of 0.05. It should be added that Statistica 13.3 software was applied in order to carry out a statistical analysis.

Results and discussion

Machine learning

According to the research on determination of the number of the network hidden layers, 15 hidden layers were identified. After aggregation of the input data including weights, it was also necessary to determine the summary signal of stimulation. In order to obtain high effectiveness of the kernel class recognition, a hyperbolic tangent was selected as an activation function for neurons in the output layer. Wag reduction was also applied for neurons in the output layer. 1000 epochs were determined in order to train the network. Determining the number of epochs allows to present all cases for a given network, one after another. The output values will be compared with the assumed values by determining an error value. A simulation of the designed networks was carried out in Statistica v.13.3. software. In the process of learning, an adequate neural model was selected for each research variant, which was characterized by the highest classification capability. The research results obtained for the remaining networks are presented in Table 2. It should be added that Broyden–Fletcher-Goldfarb–Shanno (BFGS)46 algorithm was used in the prepared networks like in the case of the research on recognizing a raspberry powder carrier. According to the research results, BFGS algorithm becomes quite effective for the functions used in the author’s previous original research33. The lowest Root Mean Square Error (RMSE)47 at the level of 0.050 for the sets: training, testing and validation was reached by MLP-p2_20 with the seed-air tube type II. The lowest RMSE at the level of 0.060 was reached by MLP-p1_15 with the seed-air tube of type 1. The research on triticale37 indicates that the use of the seed-air time of type 1 along with an increase in the velocity of kernel transporting air stream (rapeseed, wheat and barley), leads to an observable increase in the network error, which in turn leads to deterioration of classification effectiveness. Like in the previous research, classification accuracy of most results (Table 2) achieved for the tested set was above 0.9037. It is worth highlighting that in the case of wheat kernels, barley kernels and rapeseed kernels, classification accuracy was found to be excellent for the air stream flow level of 15 and 20 m s−1 in the seed-air tube type 1.

Table 2 Results of training process Artificial Neural Networks.
Full size table

One of the difficulties to be coped with is distinguishing between the classes of kernels. It results from the fact that grains of different classes differ in sizes. The above translated into the results of network training, taking into consideration MLP-p1_25 and MLP-p2_25, for which RMSE reached 0.146 and 0.10, respectively. Unfortunately, these results are less satisfying. However, it should be said that the highest effectiveness in cereal kernel recognition was achieved for the seed-air tube type 2 (MLP-p2_20). In this solution the classification coefficient for air stream at the working level of 20 m s−1 was 0.92.

As already mentioned, when designing each network variant, the same parameters were determined, i.e. 20 texture descriptors, number of epochs, number of hidden layers and the output layer activation function. As a result, high effectiveness was obtained for 3 network variants. It turns out that the optimal solution for cereal kernel image recognition can be obtained with air stream velocity at the level of 20 m s−1, for the proposed configuration I and II of the seed-air tubes.

Statistical analysis

Tables 3, 4, 5, 6, 7, 8 present an analysis of variants (ANOVA) of 20 texture descriptors for 9 kernel classes, which differ in terms of the seed-air tube structure used. The analysis allowed to compare different classes of cereal kernels on the basis of texture descriptors (indirectly acquired from digital images). When analyzing variables Contr, Corrm, Cprom, Dissi, Entro, Senth, Dvarh, Denth and inf21h, one can observe a similarity between the classes of wheat kernels transported with the air stream velocity at the level of 15 m s-1 (Table 3) and 20 m s−1 (Table 4) via the seed-air time of type 1. In the case of variables such as Cshad, Energ, Homom, Homop, Maxpr, Sosvh, Svarh, Savgh, inf1h, Indnc and Idmnc, an analysis of variance showed similarity between the classes of barley for the seed-air tube of type 1, transporting kernels with the velocity of air stream at the level of 20 m s-1 (Table 4) and 25 m s−1 (Table 5). By carrying out a statistical comparison of texture descriptors, one can observe similarities in the research groups containing medium size kernels of wheat and barley.

Table 3 Texture analysis of kernels while measuring selected air stream velocities (15 m s−1) and the position of the seed-air tube type I.
Full size table
Table 4 Texture analysis of kernels for selected air stream velocities (20 m s−1) and the position of the seed-air tube type I.
Full size table
Table 5 Texture analysis of kernels for selected air stream velocities (25 m s−1) and the position of the seed-air tube type I.
Full size table
Table 6 Texture analysis of kernels for selected air stream velocities (15 m s−1) and the position of the seed-air tube of type II.
Full size table
Table 7 Texture analysis of kernels for selected air stream velocities (20 m s−1) and the position of the seed-air tube of type II.
Full size table
Table 8 Texture analysis of kernels for selected air stream velocities (25 m s−1) and the position of the seed-air tube of type II.
Full size table

Similarity measured for the seed-air tube of type II occurred for variables: contr, corrm, cprom, dissi, entro, senth and dvarh between research classes in rapeseed when the air stream velocity was set at 15 m s-1 (Table 6), 20 m s-1 (Table 7), 25 m s−1 (Table 8). In the case of variables cshad, energ, homom, homop, maxpr, sosvh, savgh, svarh, indnc and idmnc it was observed that the most significant research group was barely with the type II tube, for air stream velocity equal to 15 m s−1. A statistical comparison of texture descriptors (Tables 6, 7, 8) for the seed-air tube of type II, made it possible to find similarities between fine-grained kernels of rapeseed and medium size barley kernels.

Taking into consideration the seed-air tubes of type I and II it can be concluded that, according to the texture descriptors, the biggest similarity occurred in the research group including medium size kernels i.e. barley. It is worth noting that the air stream velocity has the major influence on the image object recognition. Comparing the results of the research, for the needs of which artificial neural networks were used to identify the sowing material (triticale) contamination and taking into consideration velocity of the sowing material, it was possible to recognize the kernel contamination classes on the basis of an image37. Analyzing the degree of difficulty in fast identification of wheat, barley and rape kernels, like in the analysis of triticale contamination37 , velocity variants and different devices supporting transport of seeds were taken into consideration. Apart from kernel variants, the techniques that were used supported by MLPN allowed to obtain satisfying results characterized by high coefficient of classification, especially for sowing velocity equal to 20 m s−1. ANN created during the research can be a useful device supporting measurement of velocity of kernels transported pneumatically.

Other studies have proven that the machine learning technique allows to evaluate effectively filamentous fungus affected fine-grained kernels (rapeseed) by using the microscopic technique48. Thanks to further and deeper analyses it was noticed that high classification effectiveness was also possible for various food products and different grain varieties28,48. Properties of kernels properly determined on the basis of a bitmat including: thousand grain mass, geometrical diameter, coefficient of spherical shape, surface area, porosity, color and texture, enable fast and non-invasive assessment of the grain class for a given velocity of sowing.

Conslusions

MLPNs capable of noninvasive recognition of wheat, barley and rape kernels were devised based on texture descriptors. The tests were performed using the author’s original test stand, which was supposed to emulate real conditions of the sowing process. Only 2 structural solutions of transporting kernels via a seed-air tube were tested.

It was found that an increase in air stream velocity had a negative impact on the effectiveness in recognizing individual classes of kernels transported via seed-air tubes with the use of structures I and II. It turned out that, the most optimal air stream velocity for kernels transported via the pipe (for solution I) was equal to15 m s−1. It should be added that the most optimal MLP was MLP-p1_15, which reached RMSE value of 0.060 and classification accuracy coefficient at the level of 0.94. In the case of II type air and seed pipe, the best MLP turned out to be MLP-p2_20, which reached the best result for RMSE value of 0.050.

An analysis of variations allowed to group effectively all research classes. It also allowed to determine the impact of texture variables for individual grain varieties including the air stream velocity and the type of air and seed pipe. In the future, the obtained results can be used for further research on the author’s original system for detecting and measuring the amount of sowing material.