Introduction

Bread wheat (Triticum aestivum L.), commonly called wheat. It is an annual plant belonging to the tribe Triticeae in the grass family (Poaceae) [1]. Wheat is one of the most widely consumed and healthiest foods due to its vitamin, amino acid and fibre content [2]. Wheat, which plays a key role in human nutrition, is an important cereal product used in bread, cakes, pasta and cookies [3]. It is grown in many parts of the world as it can be grown in both wet and dry soils [4]. Due to its nutritional value and affordability, wheat ranks third in total food production worldwide after rice and maize [5]. Some important factors such as variety-specific plant diseases, pests, and weeds may affect the safety, availability, and quality of crop production [6]. To improve product quality and yield in wheat production, seed producers need pure, high-quality seed characterised by high productivity. Therefore, it is important for seed producers to analyse varieties accurately, qualitatively and objectively. However, it is quite difficult for non-specialists to determine the type of wheat. Currently, wheat seeds are separated and analysed using traditional methods. This results in more labour, higher costs, and longer wasted time. These traditional methods can also lead to misclassification due to human factors. In parallel with the developments in information technology, computer vision, deep learning, and machine learning methods can be used to find and classify the characteristics of seeds [7]. This makes the process much more accurate, cheaper, faster and objective [2, 8]. Figure 1 shows the five bread wheat varieties approved and grown in Turkey, “Bayraktar 2000”, “Tosunbey”, “Şanlı”, “Ayten Abla” and “Hamitbey”, from the Central Field Crop Research Institute of the Ministry of Agriculture and Forestry of the Republic of Turkey [9]. The shape and structure of wheat differ from wheat varieties of different quality. Therefore, it is important to classify and breed these wheat varieties in terms of cultivation, quality and yield.

Fig. 1
figure 1

Bread wheat varieties

Full size image

The development of high-speed processors and specialised graphics processing units (GPUs) in today’s computer technologies has popularised the use of image processing techniques in artificial intelligence applications and the development of deep learning approaches. Deep learning approaches typically use pre-trained CNN models. Researchers can use their own datasets in these CNN models with minor adjustments by performing transfer learning on CNN models. Pre-trained CNN models have been used in many computer vision applications, such as agriculture, medicine, virus detection, natural language processing, robotics, etc. Deep learning is now used in image processing and computer vision because much more information can be extracted, than with machine learning techniques [10]. Image processing, a new agricultural research programme, uses computer vision, artificial intelligence, and deep learning. Nie et al. (2019) classified 6136 hybrid okra seeds and 4128 hybrid loofah seeds using hyperspectral imaging and achieved 95% classification accuracy with deep CNN-based discriminant analysis [11]. Lopes et al. (2019) proposed Computer Vision Systems (CVS) based on Spatial Pyramid Partition Ensemble (SPPe) technique to discriminate between naked and malt types of 22 flour varieties using machine learning and 55 image features [12]. Xu et al. [13] classified 8080 maize images of five cultivars using machine learning and deep learning methods. Boa and Bambil [14] collected 150 seeds from twelve aquatic plant species and used deep learning, SVM, and Random Forest to classify the seeds, with SVM being the most successful with 97.91%. Ashqar et al. [15] trained about 5000 seedling images of 12 species using CNN algorithms and found that seedling classification was successful with an accuracy rate of 99.48%. Sabanci et al. (2021) enlarged 767 images of pepper seeds by data augmentation to 3068 images. They trained these images with CNN models and classified the features obtained in the CNN models using SVM, one of the machine learning techniques, and analysed the two models. They obtained the highest accuracy of 98.05% in the CNN model and 99.02% in the CNN-SVM model [16]. Koklu et al. (2022) classified the images of 500 grapevine leaves belonging to 5 species with MobileNet v2 and classified them with SVM, one of the machine learning techniques, using the features obtained from MobileNet v2. As a result of the classification process, they obtained 97.2% accuracy with MobileNet v2 and 97.6% accuracy with CNN-SVM [17]. Verma [18] used VGG19, CNN, and SVM methods for rice disease localization, seedling health, and grain quality using artificial intelligence, deep learning, and image processing techniques.

Taner (2018) classified eleven physical property values of cereal grains of species and varieties of bread wheat, barley, durum wheat, triticale, and oats with ANN and made species and variety classification in their study. They obtained R values of 99% variation in two of them for species and variety [19].

Ali et al. (2020) identified 55 hybrid traits consisting of histogram, texture, and spectral traits using images of 6 maize cultivars. They selected nine features using correlation-based feature selection (CFS). These features were classified using four machine learning classifiers (ML). MLP achieved a classification accuracy of 98.93% [20].

Javanmardi et al. (2021) used linear discriminant analysis (LDA), artificial neural network (ANN), quadratic SVM, cubic support vector machine (SVM), k-nearest-neighbour (kNN), bagged tree, and boosted tree using features obtained from CNN models from 2250 images of nine different maize cultivars and obtained the best classification result with the CNN-ANN model with 98.1% [21].

Luo et al. (2021) took 47,696 images of 140 weed species and weed seeds. Six CNNs were used to classify 140 species of weed seeds. Amongst the six models, AlexNet and GoogLeNet achieved the highest classification accuracy quantitatively [22].

In his study, Tuğrul (2022) classified fiver different rice varieties grown in Turkey using four CNN models. In the classification, the VGG CNN model showed the best classification performance [23].

In their study, Kishore et al. (2022) extracted deep traits from the SqueezeNet CNN model using 14,469 maize images of 4 maize varieties. The extracted features were classified using ML classifiers, with mSVM achieving the highest classification success of 89.4%. The Whale Optimization (WOA), Bat Optimization (BA) and Grey Wolf Optimization (GWO) optimization features were selected for feature selection (FS) and the mSVM algorithm achieved 88.72%, 88.82% and 88.95%, respectively [24].

Sabancı et al. (2022) created a dataset consisting of 1200 wheat images by duplicating 300 wheat images, 150 of which represented healthy wheat grains and 150 of which represented wheat grains damaged by sunburn. These images were subjected to classification using CNN's AlexNet and AlexNet + BLST models. Whilst a classification accuracy of 98.5% was achieved with AlexNet, the classification accuracy with AlexNet + BLST was 99.5% [25].

The aim of this study is to develop an identification system for the classification of wheat seeds, which is very difficult to visually distinguish bread wheat seeds and requires expert knowledge in seed images. Since the human factor comes into play in expert knowledge, factors such as fatigue and distraction for time make errors inevitable. In this study, three systems that automatically classify bread wheat seeds according to their varieties are proposed to solve these problems. In the first system, a current CNN model, ResNet’models, was used to classify the varieties. In the second system, attributes were extracted from ResNet CNN models and classified with SVM algorithms. In the third system, the determined part of the features extracted from ResNet CNN models with the mRMR feature selection method is classified with SVM algorithms. In this respect, this study differs from other studies in the literature. The performance of the three models was compared. This work successfully combines various approaches, concepts, techniques, models and components such as image acquisition of Wheat seed images, Image denoising, Transfer learning in CNN models, Feature Extraction, SVM models, Image classification, feature selection, mRMR and Wheat seed classification. This is a typical combination innovation that can be highlighted to show the contributions and/or advantages of the proposed method.

The contributions of our study to the literature can be summarised as follows:

  • Three popular pre-trained CNN models, fine-tuned with five different bread baking moulds, classified wheat seeds

  • Before classifying the ResNet18, ResNet50, and ResNet101 models, the features extracted from the fc1000 layer were classified using the SVM classifier, i.e. CNN-SVM.

  • SVM classifiers by combining ResNet CNN models with different variations and categorised with.

  • After combining the features obtained from all ResNet models, the MRMR feature selection algorithm was applied.

  • ResNet models have been successfully used for bread wheat seed classification. can be used.

  • Five different varieties of bread wheat seed have been presented in the literature.

Materials and methods

Our study consists of five wheat varieties bred and registered by the Central Research Institute of Field Crops of the Ministry of Agriculture and Forestry of the Republic of Turkey and wheat obtained from the same institution. These are five types of bread wheat seeds, namely Bayraktar ‘2000’, ‘Tosun Bey’,’'Şanlı’, ‘Ayten Abla’ and ‘Hamitbey’.

In this section, different methods for bread wheat seed classification are presented. The block diagram of the method proposed in our study is shown in Fig. 2. In the first step, images of wheat seeds were acquired in groups. Using the required image processing and filtering techniques, each wheat grain image was extracted from the grouped images. Each wheat grain was saved as a single image with a size of 250 × 250 pixels, centred on a black background. The wheat grains were classified using the fine-tuned ResNet CNN models. The features from the fc1000 layer, the feature layer of the ResNet CNN models, were classified as different variants using SVM classifiers. Finally, SVM was used to classify the features selected by the feature selection algorithm.

Fig. 2
figure 2

The block diagram of proposed methods

Full size image

This study, image processing and deep learning applications were performed using a powerful laptop with Intel Core i7-10750H-2.60 GHz CPU, 32 GB RAM 2.93 GHz, 4 GB NVIDIA GeForce GTX 1650 Ti and 500 GB NVMe2 SSD HDD.

Figure 3 shows the confusion matrix for multiclass classification. Performance The formulas used in the metrics are given in Eqs. 1–5 [26].

$${\text{Accuracy}} = \frac{TP+TN}{TP+TN+FP+FN}*100,$$
(1)
$${\text{Precision}}=\frac{TP}{TP+FP},$$
(2)
$${\text{Sensitivity}} =\frac{TP}{TP+FN},$$
(3)
$${\text{Specificity}} =\frac{TN}{TN+FP},$$
(4)
$$F1{\text{-Score}}=2* \frac{{\text{Sensitivity}}*{\text{Precision}}}{{\text{Sensitivity}}+{\text{Precision}}},$$
(5)
$$MCC=\frac{(TP*TN)-(FN*FP)}{\sqrt{(TP+FN)*(TN+FP)*(TP+FP)+(TN+FN)}},$$
(6)

where TP is the true positive, TN is the true negative, FP isthe false positive and FN is the false negative.

Fig. 3
figure 3

Multiclass classification problem confusion matrix

Full size image

The whole process is explained in detail below.

Image acquisition

Bread wheat seeds were imaged using the computer vision system shown in Fig. 4. The system has a size of 20 × 20 × 15 cm, an illumination system that prevents the formation of shadows in the background of the wheat seeds, and an image acquisition system with 4640 × 2880 pixels that allows the acquisition of images in a closed box with a camera system as standard.

Fig. 4
figure 4

Computer vision system

Full size image

Creating a bread wheat image dataset

Computer vision system was used to capture images of these wheat seeds with a size of 4640 × 2880 pixels. Noise was removed from the wheat seed images using image processing techniques. Each wheat seed was detected by object detection. The obtained wheat images were separated so that each wheat image was 250 × 250 pixels in size.For more reliable and accurate classification of the separated wheat seeds, the background was coloured black and the wheat seeds were placed in the centre of the image. In this way, 8354 images of wheat grains from all wheat varieties were obtained and a dataset was created. The features and the number of grains are given below.

Bayraktar 2000: Registered in 2000. Morphological: white spiny, leafy, medium height, white and semi-hardy variety. Consists of 1850 images.

Tosun Bey: Registered in 2004. Morphological: it is a medium sized, white and hard-grained variety with awns and white scab. It consists of 1648 frames.

Şanlı: Registered in 2016. Morphological: It is a variety with coloured ears, awnless, medium size, red and hard-grained. It consists of 1600 images.

Ayten Abla: Registered in 2019, morphologically it is a variety with white ears, awns, medium length ears, red grains and medium height. It consists of 1632 images.

Hamitbey: Registered in 2018. Morphologically with white ear, sword, and is a semi-hardy and medium variety. It onsists of 1624 images.

The block diagram of the process is shown in Fig. 5.

Fig. 5
figure 5

Breaad wheat seed data set blog diagram

Full size image

Classification of wheat grains with ResNet models

After the dataset was created, the three pre-trained ResNet models were fed wheat grain images. In our study, we did not develop a new CNN model, but instead fine-tuned existing ResNet models and trained them with wheat images. The recently preferred deep learning architectures ResNet18, ResNet50, and ResNet101 were used in the experimental study. The architecture of each model has its own number and layers. Therefore, they offer different advantages in terms of size, depth, and parameter range. In our study, we compare and contrast these CNN models’ analysis is important. For the data input of all models, 224 × 224 pixel images are used. Therefore, the wheat grain images are reused without deep learning. dimensioned and passed to the models. 8354 wheat images, 70% training and 30% testing for the CNN models. The architecture of the CNN models is shown in Fig. 6.

Fig. 6
figure 6

CNN architecture

Full size image

By combining the deep features of ResNet CNN models with different variants classification of wheat seeds

In this phase, features were extracted from the fc1000 layer of the pre-trained ResNet CNN models before classification. For each wheat grain image, 1000 features were extracted from the feature layer of all ResNet models. These features were combined to binary variations and triples, with 2000 features for each wheat grain image and finally 3,000 features for triple. Figure 7 shows the block diagram of the system.

Fig. 7
figure 7

Deep feature extraction and classification stage

Full size image

MRMR feature selection algorithm

The minimum redundancy maximum relevance (mRMR) method was proposed by Peng, Long, and Ding (2005) and quickly became a popular and widely used method [27, 28]. It allows finding the most relevant set of attributes that can maximize the representation of the classification process. The goal of the algorithm is to minimize the number of attributes whilst extracting the attributes that contribute most to the classification of the attribute set. Feature selection algorithms are used to measure and discriminate between different class labels of a feature or feature set for data evaluation. These include consistency, adherence, categorization error, information or uncertainty and distance [29, 30]. A detailed description of the algorithm and the mathematical modelling used can be found in Peng, Long, and Ding (2005) [27]. For feature selection, the Genetic Algorithm (GA) [31, 32], the Particle Swarm Optimization (PSO) [33, 34], Tree-Seed Algorithm (TSA) [35], Whale Optimization Algorithm (WOA) [36, 37] and many other optimization techniques are used.

The extracted features from all ResNet models are combined by collecting 1000 features from each model output, a total of 3000 features are created for each wheat grain image. Using all of these features can cause the system to run slowly and decrease classification performance. To determine this, the image with the 3000 features obtained from the three CNN models was classified using SVM with different kernel features. Then, 1000 features representing the strongest target were selected using the mRMR feature selection algorithm and classified using SVMs with different kernel functions. Figure 8 shows the block diagram.

Fig. 8
figure 8

Feature selection and classification with mRMR algorithm in deep features

Full size image

Results and discussion

In this section, we investigate the performance of the pre-trained ResNet CNN models ResNet18, ResNet50, and ResNet101 for wheat grain classification. These models have a structure consisting of 18, 50, and 101 deep layers, respectively, all using 224 × 224 × 3 images as input.

Of the 8354 images, 70% were used for training and 30% for testing. In all studies, the parameters used in the CNN models were assumed to be the same to avoid performance differences. The optimization algorithm is stochastic gradient descent with impulse: ‘sgdm’, review frequency: 10, maximum number of epocs: 5, mini-batch size: 128, learning rate: 0.001, learning rate decay factor: 0.1, and learning rate decay time: 10.

The confusion matrix of the first stage classification results is shown in Fig. 9. Also, the performance metrics obtained from the confusion matrix are shown in Table 1.

Fig. 9
figure 9

Confusion matrices

Full size image
Table 1 Performance metrics of CNN-based methods
Full size table

According to Table 1, the results were 97.57%, 97.29% and 97.49% for ResNet18, ResNet50 and ResNet101 models respectively. The highest accuracy was obtained with ResNet18. Figure 10 shows the accuracy (%) and loss graph of the training and testing (validation) steps of the wheat grain classification performed with ResNet models.

Fig. 10
figure 10

Training, validation and loss graphics of ResNet models

Full size image

In the second stage, instead of feeding the Resnet CNN models directly to deep learning, deep features of each model were extracted from the fc1000 feature layer before the classification layer. The extracted features were classified by SVM with different kernel functions including single, binary variations and all. The classification accuracy of ResNet18 + SVM, ResNet50 + SVM and ResNet101 + SVM models were 89.7%, 92.1% and 92%, respectively. ResNet50 model showed the best performance in SVM classification of deep features obtained with a single architecture. In the variations of the combination of two CNN models, the classification accuracy of ResNet18 + ResNet50 + SVM, ResNet18 + ResNet101 + SVM and ResNet50 + ResNet101 + SVM models were 91.9%, 92.3% and 94.0%, respectively. The most successful combination in binary variations was ResNet50 + ResNet101 + SVM. Finally, the classification accuracy of ResNet18 + ResNet50 + ResNet101 + SVM model for wheat seeds was 94.1%. According to the deep attributes of all ResNet models, ResNet18 + ResNet50 + ResNet101 + SVM model has the most successful classification accuracy of wheat seeds with 94.1%.

Finally, the most effective 1000 features in classification amongst 3000 features formed by combining the features obtained from ResNet models were selected by mRMR feature selection algorithm. These features were then classified by SVM. As a result of the classification, ResNet18 + ResNet50 + ResNet101 + mRMR + SVM showed the best classification performance with 94.5% compared to these deep features. The performance metric results of the best classification results obtained in the whole study are given in Table 2. Deep learning algorithms can take a long time depending on the deep learning method used and the characteristics of the computer used. However, the classification of the features obtained by taking deep features with machine learning algorithms such as SVM can be completed in much shorter times as can be seen from the study.

Table 2 Best classification results
Full size table

According to Table 2, it is seen that the most successful method in the classification of wheat grains is the ResNet18 model with 97.57%. In the classification using the attributes obtained from ResNet CNN models, ResNet18 + ResNet50 + ResNet101 + Quadratic SVM model was the most successful model with 94.08%. The accuracy was found to be 94.51% in the classification made with the most effective 1000 attributes selected by the feature selection algorithm. This result confirms the great effectiveness of CNN models for the classification of wheat grains.

The results obtained proved the great usefulness of the Fusion ResNet + mRMR + SVM model and selected ResNet CNN models in the successful classification of bread wheat seeds of different varieties. The procedure combining classification of images through deep learning and deep feature extraction from seed images can be very promising in the research of seed varieties.

Many studies in the literature have proven the effectiveness of distinguishing seed varieties from different species; for example, chickpeas [38], beans [39], wheat [40,41,42], rice [43], tomatoes[44], etc. In this paper, ResNet CNN models, deep features obtained from them, and SVM models as a classifier using the mRMR algorithm to select the most effective features have provided a new approach for the classification of wheat seeds. Deep learning approaches and traditional machine learning methods continue to be used in the purity of seed varieties, non-destructive seed classification and evaluation of seed quality, and new approaches are sought. In this article, ResNet CNN models, classification of deep features with SVM models, classification of deep features with SVM models by fusing them with different combinations, and classification of effective features with SVM models by determining the effective features with the mRMR algorithm have provided a new approach to the classification of wheat seed varieties. The proposed approach may also have practical and useful applications in different seed types. To find the purity of wheat seeds or grain seeds, automatic seed classification systems are desired, using computer vision and image processing to increase the degree of accuracy [45].

Conclusion and future work

In this study, the classification of wheat grains by fine-tuning the pre-trained CNN models ResNet18, ResNet50 and ResNet101 was successfully performed. Unlike the classification of wheat grains by CNN models, this study classifies the deep features obtained from the models with SVM. Then, the most effective 1000 attributes amongst the obtained attributes were determined by the mRMR feature selection algorithm. The mRMR feature selection algorithm gave more accurate results with fewer features in the classification according to the attributes. In addition, the results of the study showed the success of ResNet models amongst the pre-trained CNN models for wheat grain classification. In the future work, we plan to develop a mobile application-based application that will perform a broader seed classification. This application is planned to help students who do not have expert knowledge, individuals who are not engaged in agriculture and people with limited knowledge, to identify and classify seed types.