Introduction

Grape (Vitis L.) is a fruit tree widely grown in the world with great economic importance and wide harvesting area [1, 2]. Grapes are fleshy berries [2, 3] that can be consumed fresh or in processed forms such as wine, vinegar, juice, seed oil, raisins, jam, and jelly. Fresh grapes are considered as non-climacteric fruit and characterized by a short shelf life and processing facilitates their storage [4]. Table grapes consumed in fresh form have a bright color, pleasant flavor, and abundant juice and are rich in nutrients such as vitamins, sugar, and minerals that can eliminate free radicals and reduce the cell senility [5]. The firmness of the pericarp tissue is an attribute of table grapes appreciated by consumers. However, changes in the firmness of the grape berries can occur after harvest [6]. In addition, table grape berries are susceptible to mechanical damage, like rupture and abscission during post-harvest handling and storage [7]. Furthermore, table grapes are facing quality losses due to spoilage and microbial alteration during post-harvest storage. Grapes are considered as perishable and with water and firmness losses, desiccation, decay, berry drop, and stem discoloration during storage [8, 9]. To maintain the postharvest quality of table grapes, storage at a low temperature of around 0 °C and high relative humidity is commonly used. Also, the modification of the atmosphere by changing the O2 or CO2 concentration can be applied [9].

The quality assessment of grapes using traditional analytical methods is destructive, laborious, time-consuming, expensive, and requires high technicity skills [10]. Therefore, the use of new approaches based on non-destructive methods and techniques (e.g., image analysis) constitutes an alternative option seeing its multitude of benefits [11, 12]. As a result, image analysis can be useful to assess and ensure the quality, safety, and freshness of agri-food products [13]. Digital image analysis is a powerful and simple method that guarantees the determination of surface properties of food products and an objective assessment of their quality [14]. The analysis of image parameters can be performed using machine learning methods (defined as a branch of artificial intelligence) that include data analysis, learning, and decision-making [15]. In the case of grapes, the combination of imaging and machine learning was used, among others, for the classification based on the maturation stage [16], identification of varieties [17], biophysical lesion assessment [18], prediction of chemical properties [19], bunch identification, and picking points location [20]. However, there is a scarcity of previous studies focused on the application of image analysis involving texture parameters from images in individual color channels as a new instrument to evaluate and assess the effect of storage conditions on grape berry features.

Thus, the main objective of this study was the development of a non-invasive technique for evaluating grape quality using image analysis and traditional machine learning involving classification models based on texture parameters selected from a set of 1629 attributes (from images in color channels R, G, B, X, Y, Z, L, a, and b). This study aimed also to evaluate the effect of storage in the freezer ( – 18 °C), the refrigerator (+ 4 °C), and the room (ambient temperature) for 4 weeks on the outer structure of grapes. Furthermore, the contributions of the current study are:

  • Non-destructive assessment of grape berry changes under different storage conditions.

  • Application of image processing and traditional machine learning algorithms for the quality determination of stored grapes.

  • Distinguishing fresh grapes and fruit stored at different conditions in terms of image texture features.

Materials and methods

Materials

Forty bunches of mature grapes were collected from the backyard garden in north-eastern Poland (53°14′10″N 20°10′40″E). Grapes were harvested at an early stage of maturity and were subjected (immediately after harvest) to manual sorting, to eliminate not fully developed and damaged grape berries, followed by storage. In this experiment, three storage conditions were tested:

  • Freezing at  – 18 ± 1 °C in the freezer (Whirlpool, Michigan, USA).

  • Chilling at + 4 ± 1 °C in the fridge (Beko, Istanbul, Turkey).

  • Ambient temperature (+ 21 ± 1 °C).

For this, ten grape bunches were used for each storage condition in addition to those reserved for assessing fresh grapes. Moreover, the grape bunches were stored as a single layer in plastic boxes with perforated walls and the remaining part of the material was subjected to imaging as a fresh form. The storage experiments were stopped when distinct changes in the overall appearance including the shape, color, and surface structure of grapes stored in the room were visible. These changes were noticeable after 4 weeks of storage. The approach to the assessment of grape berry behavior under different storage conditions using image analysis and machine learning is summarized in Fig. 1.

Fig. 1
figure 1

The procedure of the assessment of grape berries’ behavior under different storage conditions

Full size image

Digital color imaging and image processing

Individual berries were extracted as ten from each of ten bunches belonging to four classes: fresh, stored in the freezer for 4 weeks, stored in the refrigerator for 4 weeks and stored at a temperature room for 4 weeks. Thus, 100 berries from each class were imaged as individual objects. The image acquisition was carried out using an Epson flatbed scanner (Suwa, Nagano, Japan) placed in a box. The acquired images were saved in the TIFF file format. Images of both bunches and individual berries in fresh and stored state are presented (Fig. 2). Before processing, the background was changed from white to black and the file format of grape images was converted to BMP allowing image segmentation and image feature extraction using MaZda software (Łódź University of Technology, Institute of Electronics, Łódź, Poland) [21,22,23]. The grape images were converted to color channels R, G, B, X, Y, Z, L, a, and b. Due to the black background of the images, image segmentation was facilitated, and lighter grapes were separated from the background. Each fruit was considered as an individual ROI (region of interest). For each ROI, 1629 image texture parameters were determined including 181 textures for each color channel. The image textures were computed based on the histogram, run-length matrix, co-occurrence matrix, Haar wavelet transform, gradient map, and autoregressive model.

Fig. 2
figure 2

Grape image of bunches and individual berries in fresh and stored state

Full size image

The classification of fresh and stored grapes

The classification of fresh grapes (first class) and samples stored in the freezer for 4 weeks (second class), stored in the refrigerator for 4 weeks (third class) and stored in the room for 4 weeks (fourth class) were carried out using WEKA machine learning software (Machine Learning Group, University of Waikato, New Zealand) [24,25,26]. The models for distinguishing all four classes based on selected texture parameters were developed using machine learning algorithms. In the first step, the attribute selection was performed using the Best first and Correlation-based Feature Selection (CFS) subset evaluator. The image textures were selected for a set of combined textures extracted from images in color channels R, G, B, X, Y, Z, L, a, and b and separately for sets of textures from each color channel of images. The selected attributes were used to develop classification models using algorithms from the groups of Bayes, Functions, Lazy, Meta, Rules, and Trees. A test mode of tenfold cross-validation was applied. In the case of each group, one algorithm providing the highest overall accuracy was chosen. The machine learning algorithms from each group providing the highest overall accuracies were Bayes Net from the group of Bayes, Multilayer Perceptron from Functions, KStar from Lazy, Random Committee from Meta, PART from Rules, and Random Forest from Trees. The algorithms were characterized by the following parameters:

  • Bayes Net—doNotCheckCapabilities: False; batchSize: 100; debug: False; estimator: SimpleEstimator -A 0.5; searchAlgorithm: K2 -P 1 -S BAYES;

  • Multilayer Perceptron—doNotCheckCapabilities: False; batchSize: 100; debug: False; decay: False; hiddenLayers: a; normalizeAttributes: True; normalizeNumericClass: True; momentum: 0.2; learningRate: 0.3; NominalToBinaryFilter: True; resume: False; reset: False; seed: 0; trainingTime:500; validationThreshold: 20;

  • KStar—doNotCheckCapabilities: False; batchSize: 100; debug: False; globalBlend: 20; entropicAutoBlend: False; missingMode: Average column entropy curves;

  • Random Committee—doNotCheckCapabilities: False; batchSize: 100; debug: False; numExecutionSlots: 1; numIterations: 10; seeds: 1;

  • PART—doNotCheckCapabilities: False; batchSize: 100; debug: False; confidenceFactor: 0.25; binarySplits: False; numFolds: 3; seeds: 1; unpruned: False; useMDLcorrection: True;

  • Random Forest—doNotCheckCapabilities: False; batchSize: 100; debug: False; calcOutOfBag: False; breakTiesRandomly: False; numIterations: 100; numExecutionSlots: 1; storeOutOfBagPredictions: False; seeds: 1.

For models built separately for individual color channels, one channel with the most satisfactory results was selected. The confusion matrices, overall accuracies, and the values of True Positive (TP) Rate, False Positive (FP) Rate, Precision, F-Measure, Matthews Correlation Coefficient (MCC), Receiver Operating Characteristic (ROC) Area, and Precision-Recall (PRC) Area were determined [27, 28].

Results

In the case of a combined set of textures from images in all color channels, 35 attributes were selected. The selected textures with the highest power for distinguishing fresh grapes and fruit stored at different conditions belonged to the following color channels: R (15 textures: RHPerc10, RHPerc50, RHPerc90, RHDomn01, RHDomn10, RS5SH3DifEntrp, RS5SV3DifVarnc, RS5SZ3SumEntrp, RS5SH5DifEntrp, RS5SZ5Contrast, RS5SZ5DifEntrp, RS5SN5DifEntrp, RS4RHRLNonUni, RS4RZRLNonUni, RATeta2), G (3 textures: GHPerc90, GS5SN5DifEntrp, GATeta1), and B (1 texture: BHPerc10), L (6 textures: LS5SZ3SumOfSqs, LS5SZ3DifVarnc, LS5SV5Entropy, LS5SZ5DifVarnc, LS5SZ5DifEntrp, LS5SN5DifEntrp), a (6 textures: aHMean, aHSkewness, aHPerc10, aHDomn01, aSGSkewness, aS5SV1DifEntrp), b (4 textures: bHMean, bS5SZ3DifEntrp, bS5SH5DifVarnc, bATeta1). No textures from color channels X, Y, and Z were characterized by discriminatory power.

The results of the classification of fresh and stored grapes in the form of confusion matrices with the correctly and incorrectly classified cases are presented in Table 1 and overall accuracies are shown in Fig. 3. The overall accuracy of the classification of fresh grapes and fruit samples stored in the freezer for 4 weeks, stored in the refrigerator for 4 weeks, and stored in the room for 4 weeks ranged from 90.5% for a model built using the PART algorithm to 96% for a model developed using Random Forest. In the case of each algorithm, samples stored in the freezer for 4 weeks and stored in the room for 4 weeks were classified with the highest correctness. In the case of storage in a room, grapes were distinguished from other classes with an accuracy reaching 100% for models developed using Multilayer Perceptron and KStar. Whereas grapes stored in the freezer were completely correctly classified (100%) for Multilayer Perceptron, Random Committee, and Random Forest. For the model providing the highest overall accuracy of 96% built using Random Forest, besides grapes stored in the freezer for 4 weeks which were correctly classified in 100%, the other classes were correctly distinguished in 96% of cases for fresh grapes, 90% for fruit stored in the refrigerator and 98% for samples stored in the room. In the case of a model producing the lowest overall accuracy of 90.5% developed using the PART algorithm, the greatest mixing of cases was also between fresh grapes and samples stored in the refrigerator.

Table 1 The confusion matrices of the classification of fresh and stored grapes based on models including the selected texture parameters from images in color channels R, G, B, L, a, and b built using different machine learning algorithms
Full size table
Fig. 3
figure 3

The overall accuracies of the distinguishing of fresh and stored grapes using models including the selected texture parameters from images in color channels R, G, B, L, a, and b developed using different algorithms

Full size image

Other performance metrics, such as TP Rate, FP (False Positive) Rate, Precision, F-Measure, MCC, ROC Area, and PRC Area of the classification of fresh grapes and samples stored in the freezer, refrigerator and room for 4 weeks are shown in Table 2. The obtained values of TP Rate (Table 2) reflected the number of correctly classified cases presented in confusion matrices (Table 1). The TP Rate equal to 1.000 (Table 2) was observed for grapes stored in the freezer and the room for 4 weeks for Multilayer Perceptron, fruit stored in the room for 4 weeks for KStar, and samples stored in the freezer for 4 weeks for Random Committee and Random Forest. These samples were completely correctly distinguished from other samples. In the case of Bayes Net and PART, no samples with the TP Rate of 1.000 were found. The most desired FP Rate equal to 0.000 was determined for samples stored in the room for 4 weeks for Bayes Net, Multilayer Perceptron, KStar, and Random Committee. It meant that no case from other groups was incorrectly classified as grapes stored in the room for 4 weeks for the selected classifier. The results confirmed significant differences in texture parameters of the outer surface of grapes stored at room temperature compared with fresh fruit and samples stored under other conditions (refrigerator and freezer). Furthermore, the grape samples stored in the room for 4 weeks were distinguished by the highest values of F-Measure and MCC reaching 1.000 for Multilayer Perceptron and KStar, as well as ROC Area and PRC Area equal to 1.000 for Bayes Net, Multilayer Perceptron, KStar, Random Committee, and Random Forest.

Table 2 The performance metrics of the classification of fresh and stored grapes using models based on the selected texture parameters from images in color channels R, G, B, L, a, and b developed using different machine learning algorithms
Full size table

Among the individual color channels, the classifications performed for the selected textures from images in color channel R were the most accurate. These selected textures were RHPerc10, RHPerc50, RHPerc90, RHDomn01, RSGKurtosis, RS5SH3DifEntrp, RS5SZ3DifVarnc, RS5SV5SumVarnc, RS5SZ5Contrast, RS5SZ5DifEntrp, RS5SN5DifEntrp, RS4RHRLNonUni, RATeta2. The confusion matrices are presented in Table 3 and the overall accuracies—in Fig. 4. The overall accuracy was in the range of 85.75% (Multilayer Perceptron) to 92.5% (Random Forest). In the case of models built based on selected image textures belonging to color channel R, grapes stored in the freezer and the room for 4 weeks were correctly classified in 100% of cases for selected machine learning algorithms, such as KStar for fruit stored at room conditions and Random Committee and Random Forest for samples stored in the freezer. The samples stored in the refrigerator for 4 weeks were characterized by the lowest classification accuracy of 65% (Multilayer Perceptron) to 82% (Random Forest), and the highest mixing of cases occurred between fruit stored in the refrigerator and fresh grapes that indicated the greatest similarity of these classes in terms of image textures from color channel R.

Table 3 The confusion matrices of the classification of fresh and stored grapes using models developed based on the selected texture parameters from images in color channel R
Full size table
Fig. 4
figure 4

The overall accuracies of the distinguishing of fresh and stored grapes using models including the selected texture parameters from images in color channel R built using different algorithms

Full size image

The highest accuracies of distinguishing the grape samples stored in the room and the freezer were confirmed for the highest values of TP Rate, Precision, F-Measure, MCC, ROC Area, and PRC Area reaching 1.000 and the lowest value of FP Rate equal to 0.000 and for the sample stored in the room. The most satisfactory results were obtained for the KStar algorithm. In the case of the sample stored in the freezer, the most effective models were built using Random Committee and Random Forest providing the value of 1.000 for TP Rate (Table 4).

Table 4 The results of the classification of fresh and stored grapes using models built based on the selected textures from images in color channel R
Full size table

Discussion

Fresh grape berries and fruit samples stored in the freezer, refrigerator, and in the room for 4 weeks were successfully distinguished using the combination of image analysis and machine learning. In previous studies, image processing and traditional machine learning algorithms also allowed for the correct classification of red currant and black currant [29, 30]. Furthermore, imaging and image analysis combined with machine learning were used for the quality assessment of other fruit. For example, imaging was applied for the identification of fungal infection of stored apples [31]. The changes in the kiwifruit stored under various conditions were determined by Zhao et al. [32] using hyperspectral imaging and deep learning. Hyperspectral imaging combined with machine learning was also used for the detection of the storage time of yellow peaches after mild bruises [33]. Infrared thermal imaging and machine learning were used by Mohd Ali et al. [34] for the evaluation of stored pineapple. Moreover, the ripeness of avocados during storage was determined using smartphone images coupled with machine learning [35]. The examples mentioned above have shown that various imaging techniques can be combined with machine learning to assess the quality of stored fruit.

The current study confirmed the usefulness of the applied approach for the quality assessment of stored berries. The development of innovative models using selected image texture parameters and traditional machine learning algorithms allowed for correct distinguishing fresh grape berries and samples stored at different conditions in an objective and non-destructive manner. The applied approach can have practical applications for the determination of changes in grape berries caused by the storage. However, future research may be performed involving more in-depth use of texture parameters extracted from images acquired using various techniques coupled with traditional machine learning and deep learning algorithms to determine changes in the structure of grapes stored with advanced technologies.

Conclusion

The current study evaluated the feasibility of the comparison of three storage conditions ( – 18 °C, + 4 °C, and room temperature) of grapes based on textural image analysis. The results revealed that grape berries stored under different conditions can be distinguished using models based on selected image textures developed using machine learning algorithms. The most accurate models for the classification of fresh grape samples and fruit stored in a freezer, refrigerator and room for 4 weeks involved selected textures extracted from images in color channels R, G, B, L, a, and b. The Random Forest machine learning algorithm turned out to be the most effective and accurate providing an overall accuracy of 96%. The innovative models developed using selected textures and machine learning algorithms can be used in practice for the non-destructive and objective assessment of changes occurring under different storage conditions of fruits and vegetables.