1 Introduction

Climacteric fruits like mangoes, papaya, banana etc. are plucked in unripe and allowed to ripen post-harvest [1]. Ripening of climacteric fruits is an irreversible phenomenon and is guided by a series of physiological and biochemical changes which lead to the transformation of a hard, unripe fruit to edible, ripe fruit with soft pulp and desirable taste and flavour [2]. Moreover, peel colour of climacteric fruits has a direct correlation with the fruit quality which also plays an important role in consumer acceptability [3,4,5]. For example, ripening of most of the mango varieties including ‘Kesar’ cultivar leads to the change in colour of peel from green to yellow. Ripe mangoes are soft to touch, with a pleasant aroma and exhibit a flavour often described as a combination of peach and pineapple [6]. However, non-climacteric fruits such as oranges, grapes, watermelons, litchis, strawberries, raspberries, cherries etc. are harvested only when they are fully ripe and do not demonstrate significant changes in peel colour [1].

India is the largest producer of mangoes in the world accounting to about 50% of worldwide production [7]. The leading mango producing states in India are Uttar Pradesh, Telangana, Maharashtra and Karnataka. Mangoes grown in India are widely exported to countries like UAE, Bangladesh, UK, Saudi Arabia, and Nepal (FAOSTAT). Delicious ‘Kesar’ cultivar of mangoes are grown in the states of Gujrat and Maharashtra and exported to different parts of the world owing to their excellent taste and flavour.

The most important factors determining the quality of mangoes are harvest conditions, optimum temperatures maintained during post-harvest handling and storage. Mangoes are very sensitive to cold temperatures and green unripe mangoes, when stored at temperatures below 10 °C suffer from chilling injuries like discoloration on the skin, pitting, uneven ripening, and poor flavour and colour development. Chilling injuries causes delay in the natural ripening process [4, 6]. Ideally, mangoes should be stored at lower temperature only when they are fully ripe to retain their freshness for longer duration [3, 6].

Owing to this ripening phenomenon, post-harvest handling becomes very important in mangoes [7, 8]. Export of naturally ripened juicy mangoes to distant places pose difficulties as mangoes tend to become overripe and inedible when shipped over large distances, resulting in huge economic losses. Thus, to minimize economic losses due to over-ripening under harsh transportation conditions, fruit sellers prefer artificial ripening of unripe mangoes at destination before selling them to consumers [1]. The, producers, distributors, retailers, commercial packaging houses as well as pulping industries need to assess the ripeness level, predict the duration until they reach the ideally ripe stage and take dynamic decisions in real-time so as to reprice, repurpose, reroute, and recycle mangoes to reduce wastage and economic losses.

This process of dynamic decision making when enabled will further reduce the need of artificial ripening of mangoes by using harmful chemicals and assist key stake holders in judicious planning of logistics thus ensuring that the consumers receive the best quality mangoes in stipulated time.

Research has been conducted in the past to study the changes over time in the quality of mangoes [9, 10]. Researches have also been done on compositional changes of mangoes as it degrades over time [11, 12]. However, to the best of our knowledge, no specific study was found on the prediction of shelf life and ripening duration of ‘Kesar’ cultivar of mangoes under different storage conditions.

This paper illustrates prediction of shelf-life of ‘Kesar’ cultivar of mangoes based on their respiration rate and ripeness level under different storage conditions. An in-house sensing system and temperature controlled enclosure was developed to record the variation in released carbon di-oxide (CO2), oxygen (O2) intake and emitted ethylene (C2H4), during respiration of mangoes along with the changes in their peel colour from green to yellow through different ripening stages under different storage conditions. Different storage conditions replicated were—(1) the conditions within a household refrigerator (4 °C), (2) ambient conditions (18 °C, 20 °C) and (3) elevated temperatures (25 °C, 30 °C) as experienced in large storage houses. Deep Convolutional Neural Networks (CNNs) ‘VGG-16’, ‘ResNet-50’ and ‘Inception-v3’ pre-trained on natural image dataset ‘Imagenet’ was further fine-tuned on mango images to classify ripeness level of mangoes as ‘unripe’, ‘early-ripe’, ‘partially-ripe’ and ‘ideally-ripe’. Respiration rate (RRCO2) was calculated based on the variation in released CO2 and consumed O2 using principles of Enzyme Kinetics and Michaelis–Menten (MM) equation. A correlation was established between RRCO2 and ripeness level obtained for different storage conditions. A Support Vector Regression model (SVM) was further employed to predict the shelf life of ‘Kesar’ mangoes based on its current RRCO2 and ripeness level when stored under different conditions. The models thus developed can work as soft sensor in prediction of shelf life of mangoes traversing through any node in supply chains.

The outline of the paper on use of AI based soft sensor for prediction of shelf life of ‘Kesar’ cultivar mangoes is as follows. Section 2 depicts the ‘Material and Methods’. Sections 2.1, 2.2 and 2.3 presents the ‘Experimental Methods’, ‘Theoretical Considerations’ and ‘Computation Methods’ respectively that were used for developing the AI based soft sensor for the prediction of shelf life. Section 3 depicts the Experimental and Computational ‘Results’ obtained on the predicted shelf life of mangoes under different storage conditions. Section 4 is the ‘Discussion’ section which presents the analysis of the results obtained and relevance of this research to the stake holders involved in fruit supply chain. Section 5 presents the ‘Conclusion’.

2 Material and methods

‘Kesar’ cultivar of mangoes were classified into four ripening stages namely ‘unripe’; ‘early-ripe’; ‘partially-ripe’ and ‘ideally-ripe’ based on previously published study [13, 14]. ‘Unripe’ mangoes refer to hard, green and unripe mangoes, ‘early-ripe’ were the predominantly green mangoes with yellow patches at early stage of ripening, ‘partially-ripe’ were predominantly yellow mangoes with green patches ripened partially and ‘ideally-ripe’ mangoes refer to soft yellow mangoes which are ideally ripe for consumption. Mangoes at different ripening stages were kept at 18 °C, 20 °C, 25 °C, 30 °C storage temperatures for calculating the shelf life. The ‘ideally-ripe’ mangoes were further kept at 4 °C, similar to the temperature observed in household refrigerators to calculate freshness duration at ‘ideally-ripe’ stage.

2.1 Experimental methods

An in-house internet of things (IoT) and Cloud enabled sensing system and temperature controlled custom enclosure were developed for the quantitative sensing and synchronized recording of the variations in consumed oxygen, released CO2 and emitted ethylene along with the pictorial variations in terms of colour, texture and firmness. A customized storage enclosure of 40 L (Fig. 1a) with an integrated sensor suite consisting of CO2, oxygen, ethylene, temperature and humidity sensors along with high resolution cameras, installed in multiple directions, were used for data collection. The temperature, humidity, light intensity and ventilation were controlled and maintained at different storage conditions. Mangoes were kept in this enclosure and variation in sensory parameters were recorded over time at a given storage condition.

Fig. 1
figure 1

a IoT and Cloud enabled sensing system and temperature controlled custom enclosure. b Schematic representation of the overall framework

Full size image

The sensed voltage signals from the select sensors (CO2 and O2) recorded at periodic intervals were passed through antialiasing filters to remove unwanted high frequency noise signals and pre-processed by signal conditioning unit. Analog to Digital Converter (ADC) then converted these analog signals to digital signals which are then processed by a microcontroller unit and interfaced with laptop/iPad. Alarms are generated by the microcontroller in case of system malfunctions like temperature overload and improper ventilation which in-turn triggers the actuators to bring the system back to equilibrium conditions. The time series sensory data and images collected at periodic intervals were annotated with respect to number of days. The robust models were trained on these annotated data under different storage conditions to predict shelf life.

The schematic of the overall framework designed to predict shelf life of ‘Kesar’ cultivar of mangoes is presented in Fig. 1b.

2.2 Theoretical considerations

CO2 respiration rate was calculated (RRCO2) by following the principles of enzyme kinetics and Michaelis–Menten (MM) equation which can be used to find RRCO2 of any fresh produce like climacteric fruits and vegetables. There are various types of respiration rate models presented in literature such as linear, polynomial, exponential and MM [15,16,17]. MM equation is formulated based on inhibition role of CO2 which can be classified into four categories namely (1) no inhibition (2) competitive inhibition (3) uncompetitive inhibition (4) non-competitive inhibition are represented below [16]

$$R \, = \, \left( {V_{m} \times y_{O2} } \right)/\left( {K_{m} + \, y_{O2} } \right)$$
(1)
$$R = \left( {V_{m} \times y_{O2} } \right)/\left( {K_{m} \times \, (1 \, + y_{CO2} /K_{ic} ) + y_{O2} } \right)$$
(2)
$$R = \left( {V_{m} \times y_{O2} } \right)/\left( {K_{m} + (1 + y_{CO2} /K_{iu} } \right) \times y_{O2} )$$
(3)
$$R = \left( {V_{m} \times y_{O2} } \right)/\left( {K_{m} + \, y_{O2} ) \times (1 \, + y_{CO2} /K_{in} } \right)$$
(4)

R is the role of CO2 in respiration mediated via inhibition mechanisms, yO2 and yCO2 are the oxygen and carbon di-oxide concentrations recorded using the custom enclosure, Vm is the maximum oxygen consumption rate, Km is the Michaelis constant for oxygen consumption, Kic is the Michaelis constant for the competitive inhibition of oxygen consumption by carbon di-oxide, Kiu is the Michaelis constant for the uncompetitive inhibition of oxygen consumption by carbon di-oxide and Kin is the Michaelis constant for the non-competitive inhibition of oxygen consumption by carbon di-oxide. It has been reported by researchers that role of CO2 (R) in respiration, calculated using the MM equation based on uncompetitive inhibition of carbon di-oxide, is the most accurate as compared to other MM equations. Hence, in this paper RRCO2 has been calculated using uncompetitive MM equation to study the variation of respiration rate in naturally ripened ‘Kesar’ cultivar of mangoes under different storage conditions.

RRCO2 varies during the natural ripening of ‘Kesar’ mangoes under different storage temperatures. However, the temperature factor is not considered in the uncompetitive MM equation. Hence, Arrhenius Eq. (5) was used to find the variation of MM equation coefficients under different storage temperatures [15, 18].

$$R_{m} = R_{p} \exp \left[ { - E_{a} /RT} \right]$$
(5)

where Rm represents the MM equation coefficients, Rp is pre-exponential factor, Ea is the activation energy, T is the storage temperature in Kelvin and R is the Universal gas constant (8.314 kJ/Kg mole K). The variation in RRCO2 was calculated over a duration of 24 h and was normalized by the volume of the custom enclosure and weight of the mango. The overall Eq. (6) used for computing RRCO2 is shown below [12]

$$RR_{CO2} = \left[ {\left\{ {R\left( t \right){-}R\left( {t - 1} \right)} \right\}/\Delta t} \right]V/W$$
(6)

RRCO2 is the CO2 respiration rate calculated in ml Kg−1 h−1, R(t) and R(t-1) are the role of CO2 in respiration mediated via inhibition mechanisms at time instance t and t − 1 respectively, ∆t is the periodic interval at which R is measured, V is the volume of the custom enclosure (40 L) and W is the weight of the mango which was calculated to be 257 g.

2.3 Computational methods

The coefficients of MM equation for uncompetitive inhibition namely Vm, Km and Kiu were calculated by fitting a non-linear regression equation on the experimentally recorded gas concentrations [14]. These MM equation coefficients were further correlated to different storage temperatures using Arrhenius Eq. (5) [15, 18]. MM equation and Arrhenius equation were combinedly used to calculate RRCO2 under different storage temperatures, consumed O2 and released CO2.

Weights of different deep Convolutional Neural Networks (CNN) namely, ResNet-50 [19], Inception-v3 [20, 21] and VGG-16 [22] without the top fully connected layer were considered for further fine tuning. A dense fully connected layer was added and trained on four ripeness levels of ‘Kesar’ mangoes namely ‘unripe’; ‘early-ripe’; ‘partially-ripe’; ‘ideally-ripe’ using ‘softmax’ function to generate a fine tuned CNN model. The computational architecture for fine-tuning of CNNs is presented in Fig. 2a. All these CNN networks are pretrained on natural image dataset named ‘Imagenet’ [23]. Transfer learning and Data Augmentation were employed to overcome the lack of huge annotated dataset. Data augmentation dramatically expanded training and validation image dataset and improved robustness of CNN network for classification of ripeness level of ‘Kesar’ mangoes. The open source Tensor Flow [24] was used to implement and train the models on GPU. During training, the original input images were cropped and resized to the target input size of [224 × 224 × 3] with scale and aspect ratio augmentation [20, 21]. The network was trained using ‘SGD’ optimizer [25] with a momentum of 0.9, learning rate of 0.0004 and batch size of 8. Other classification methods were tried to classify the ripeness level of ‘Kesar’ mangoes. However the pretrained CNN networks namely ResNet-50 [19], VGG-16 [20, 21] and Inception-v3 [22] performed significantly better with improved classification accuracy as compared to other classification methods as they exploit hand-crafted features resulting in limited performance [26, 27].

Fig. 2
figure 2

Computational architecture for a fine-tuning of CNN, b prediction of shelf life

Full size image

The fine-tuned CNN model predicted the ripeness level of an unknown mango image. A correlation was established between ripeness level and CO2 respiration rate (RRCO2) under specified storage environment. Different ML based regression methods were employed to predict the shelf life of ‘Kesar’ cultivar of mangoes based on current ripeness level and RRCO2 under different storage conditions. The corresponding computational architecture for prediction of shelf life is presented in Fig. 2b. The Support Vector Regression (SVR) method performed better than other regression methods like decision tree. SVR was used instead of Support Vector Machine (SVM) Classifier to minimize error. The entire annotated dataset of 1524 samples was divided into training, validation and test data with 70%: 20%: 10% split, respectively.

The model was trained over training dataset for 20 epochs until validation accuracy and training accuracy converged and validation and training loss converged. The model was further used to calculate the prediction accuracies over training, validation and test dataset.

3 Results

‘Kesar’ cultivar of mangoes, classified into four ripening stages namely ‘unripe’; ‘early-ripe’; ‘partially-ripe’; ‘ideally-ripe’ were kept at 18 °C, 20 °C, 25 °C and 30 °C storage temperatures for calculating the shelf life. The custom enclosure with the integrated sensor suite recorded the synchronized variations in the intake of O2 and release of CO2 at periodic intervals under different storage temperatures mentioned above. The coefficients of MM equation computed for uncompetitive inhibition namely, Vm, Km and Kiu at different storage temperatures are presented in Table 1.

Table 1 MM equation coefficients calculated at 18 °C, 20 °C, 25 °C and 30 °C
Full size table

RRCO2 was finally calculated under different storage conditions with respect to the consumed O2 and released CO2 using the Eq. 6. The variation of CO2 respiration rate over time (in mL kg−1 h−1) under different storage conditions is presented in Fig. 3.

Fig. 3
figure 3

Variation of CO2 rate over time under storage temperatures of 18 °C, 20 °C, 25 °C and 30 °C

Full size image

The ‘ideally-ripe’ mangoes were further kept at 4 °C, similar to the temperature observed in household refrigerators and the same is presented in Fig. 4.

Fig. 4
figure 4

Variation of CO2 rate over time a under different storage temperatures a 18 °C, b 20 °C, c 25 °C, d 30 °C followed by storage at 4 °C

Full size image

Deep CNN networks namely ‘VGG-16’, ‘Inception-v3’ and ‘ResNEt-50’ were fine-tuned and trained for classification of ‘Kesar’ cultivar of mangoes into four different ripeness levels and the same is presented in Fig. 5a. 1524 images of ‘Kesar’ mangoes were collected and annotated for four ripeness levels ‘unripe’; ‘early-ripe’; ‘partially-ripe’ and ‘unripe’ based on average RGB scores of 84:102:24; 139:154:36; 179:154:29 and 225:145:7, respectively. CNN networks were trained on these annotated images. To avoid data over-fitting original dataset was extended by data augmentation including translation and reflection in vertical and horizontal directions. After data augmentation images were resized to 224 × 224 pixels for ‘VGG-16’ and ‘ResNet-50’ and to 299 × 299 pixels for ‘Inception-v3’ network. The training mechanism involved redefining of the proposed networks by back-propagation mechanism [24] which calculates the minimized square difference between the classified ground truth and predicted output iteratively. The fine-grained classification performance on different CNN networks is presented in Table 2.

Fig. 5
figure 5

a Pictorial Representation of ripeness level of ‘Kesar’ cultivar of mangoes, b classification accuracy of ripeness level of mangoes on using ‘VGG-16’ network

Full size image
Table 2 Fine-grained classification performance on different networks
Full size table

The variation in training and validation loss and training and validation accuracy over 20 epochs on using ‘VGG-16’ is graphically represented in Fig. 5b.

The CO2 respiration rate (RRCO2) of ‘Kesar’ cultivar of mangoes were further correlated to the CNN predicted ripeness level under storage temperatures of 18 °C, 20 °C, 25 °C and 30 °C and the same is presented in Fig. 6.

Fig. 6
figure 6

Correlation of ripeness level and CO2 respiration rate of ‘Kesar’ mangoes under different storage temperatures

Full size image

Support Vector Regression (SVR) methods were further employed to predict the shelf life of ‘Kesar’ cultivar of mangoes based on current ripeness level and RRCO2 under different storage conditions and the same is presented in Fig. 7. The regression model was further validated with the experimental data under storage temperatures of 18 °C, 20 °C, 25 °C and 30 °C.

Fig. 7
figure 7

Shelf life prediction of ‘Kesar’ cultivar of mangoes at a 18 °C, b 20 °C, c 25 °C, d 30 °C

Full size image

4 Discussion

The observed release rate of CO2 as shown in Fig. 3, increases up to a certain point and then decreases as the fruit respires. The rate of release of CO2 increases with the increase of temperature, thus increasing the natural ripening rate of mangoes [15]. The CO2 respiration rate increases until the unripe mangoes naturally ripen to ‘ideally ripe’ stage, beyond which the rate decreases indicating over-ripening of mangoes. At the peak of CO2 release rate the mangoes are ideally ripe for consumption and most optimal in taste and flavour [16]. Also, as indicated by Fig. 3 the CO2 respiration rate is maximum at 30 °C and minimum at 18 °C and it takes 4 days and 10 days respectively, for hard green unripe mangoes to reach ‘ideally-ripe’ stage. Hence, it is clear that the mangoes take sufficiently longer time to ripen when kept at lower temperature as compared to mangoes kept at elevated temperatures. So, based on the necessity of the stake holder, storage temperature can be altered to get the desired ripening rate.

As depicted in Fig. 4, it was observed that the mangoes maintained their freshness at ‘ideally-ripe’ stage for 4–5 days before being wrinkled and over-ripened. It can be noticed, that mangoes when preserved at 18 °C, 20 °C, 25 °C and 30 °C take 10, 8, 6 and 4 days, respectively to reach from ‘unripe’ to ‘ideally-ripe’ stage and beyond this duration they start to over ripen. However, when ‘ideally-ripe’ mangoes are kept at 4 °C they retain their freshness with ideal taste, flavour and nutritional content for additional 4–5 days. For example, ‘unripe’ mango when maintained at a constant temperature of 18 °C, becomes over ripe after 10 days. Instead, if these mangoes are stored at 4 °C on the 10th day when they are ‘ideally ripe’, the mangoes remain fresh for additional 5 days at the ‘ideally-ripe’ stage before becoming over ripe. This is due to the reduction in the CO2 respiration rate and emission of ethylene at lower temperature which decelerates the natural ripening process.

As presented in Table 2 ‘VGG-16’ showed the best performance as compared to the other networks. The graph in Fig. 5b confirms that there is no data over-fitting. Training accuracy increase from 8.41 to 98.31% and training loss reduces from 4.3593 to 0.1339 after 20 epochs. Validation accuracy increase from 30.41 to 98.96% following similar trend of training accuracy and converges with the training accuracy after 15th epoch. Validation loss also reduces from 3.6693 to 0.1187 after 20 epochs. The prediction accuracy using ‘VGG-16’ on training dataset was 99%, validation dataset was 99% and test data set was 96.2%.

As depicted in Fig. 6, the ripeness levels are numbered from 1 to 4 representing four ripeness levels ‘unripe’, ‘early-ripe’, ‘partially-ripe’ and ‘ideally-ripe’, respectively. It is evident that with the increase in CO2 respiration rate, ripeness level also increases which further corresponds to the increase in natural ripening rate. Avoiding the boundary conditions, it can be observed that change in CO2 respiration rate (in mL/kg/h) between 20 to 30; 45 to 55; and 60 to 70 changes the ripeness level from ‘unripe’ to ‘early-ripe’; ‘early-ripe’ to ‘partially-ripe’; and ‘partially-ripe’ to ‘ideally-ripe’ respectively. Further, when the CO2 respiration rate is beyond 80 ml/kg/hr mangoes can be considered to be over-ripe.

As depicted in Fig. 7, the prediction accuracy of SVR model on test data was maximum at 20 °C (94.5%) and minimum at 30 °C (78.3%) and the training losses in the methods were 0.1123 and 0.6584, respectively. The prediction accuracy at 18 °C and 25 °C were 83.6% and 90.3% with a training loss of 0.4573 and 0.2219, respectively. The developed SVR model has better prediction accuracy in the temperature range of 18–25 °C as compared to higher temperature of 30 °C.

From the graphs in Fig. 7 it is evident that an increase in temperature proportionally increases the release rate of CO2 from climacteric fruits like mango which in turn increases the natural ripening rate of climacteric fruits and in turn decreases the shelf life. The graphs in Fig. 7 thus presents the shelf life until which the ‘Kesar’ mangoes can be consumed with respect to its current ripeness level when maintained at a specified storage temperature. Beyond this predicted duration, the mangoes tend to become over ripe, wrinkled and unfit for consumption when maintained at the same storage temperature. For example, as shown in Fig. 7a, the shelf life of ‘unripe’ ‘Kesar’ mango releasing 18.6 mL/kg/h CO2 at ripeness level ‘1’ was experimentally found to be 12 days and predicted shelf life was 11 days when kept at 18 °C temperature. The predicted shelf life compares well in accordance with the experimentally observed shelf-life under all the four storage conditions. Most importantly, as observed in Fig. 7, the predicted shelf life by SVR model is less than or equal to the experimentally observed shelf life under storage temperatures of 18 °C, 20 °C, 25 °C and 30 °C.

Our work helps retailers, farmers, and distributors to take dynamic decisions on repricing and rerouting of mangoes based on the remaining ripening duration such that the stock can be cleared in time and wastage can be minimized. Further, this work assists in controlling and altering the modified atmospheric packaging (MAP) conditions of the packaged mangoes in bulk to get the desired ripening rate and accordingly enhance shelf life of packaged fruits. Moreover, in case of large consignments when bulk of mangoes are kept together it is extremely difficult to get images of individual mango especially when they are transported in trucks. However, in such scenarios inexpensive CO2 and O2 sensors can be easily deployed and CO2 respiration rate can be measured. Once the respiration rate is measured in real-time, it can be correlated to the ripeness levels and dynamic decisions can be taken on the consignment to maximize profit and reduce wastage based on the ripeness level of mangoes. In order to use these soft sensors for field applications, a handheld device is further being developed to facilitate end users to easily ascertain the storage temperature, released CO2 and consumed O2 by the fruits. This portable device will be IoT enabled and will thus facilitate real-time remote monitoring of shelf life which in turn will assist the stake holders to take dynamic decision well in advance.

5 Conclusion

In this paper we have presented the prediction of shelf-life of ‘Kesar’ cultivar of mangoes based on their respiration rate and ripeness level under different storage conditions. The methodology presented for prediction can be used for prediction of shelf life of any climacteric fruits. Prediction of shelf-life involved recording of time series variation in emission of CO2 and consumption of O2 along with the variation of images using the custom designed infrastructure developed in-house. AI models (CNN and SVR) were trained on these collected data and images to predict shelf life of mangoes. These cloud based AI models acts as soft sensors for real life supply chain scenarios. Thus, it is implied that the predicted shelf life being less than or equal to the actual shelf life under different storage temperatures, will assist stake holders to take dynamic decisions like rerouting, repurposing, recycling and repricing in advance or at correct time thus contributing to reduction of wastage of mangoes and corresponding economic losses.