Introduction

The food supply chain is an essential link in the modern food industry, defined as a series of stages from production to food consumption (Fig. 1). It consists of economic stakeholders in multiple stages, including production, packaging, transportation, storage, and consumption [1]. According to the statistical yearbook of the United Nations Food and Agriculture Organization (FAO) 2021, the total food production of world food, which is dominated by maize, rice, and fresh fruit, is currently more than 1,162, 756, and 38 million tons, respectively. The large volume and variety of food products make detecting and controlling large-scale food quality and safety at all stages of the supply chain a significant challenge. Therefore, rapid, non-destructive, and efficient testing of food products is one of the critical research issues in the food industry. In the production segment of the supply chain, the type of produce and the environment in which it is grown directly affect its overall quality due to natural and human factors [2–4]. Depending on consumer preferences and food quality, similar products from different origins can vary significantly in price. Adulterating food products by unscrupulous traders for profit has led to food fraud which has become one of the threats in the domestic and global market. Therefore, food products at the supply chain source should be authenticated, and traceability tested to ensure food quality and safety. Apart from the production chain, food products throughout the supply chain are continuously subjected to external factors such as temperature, humidity, and microorganisms, and their quality is deteriorating. In particular, fresh fruits and vegetables, meat and seafood, because they contain high moisture, short shelf life, are easy to bump, prone to make microorganisms and bacteria reproduction, in the storage and transportation process of food surface may appear abnormal lesions and pests, the internal will also produce the corresponding toxic substances, causing harm to the human body [5]. In conclusion, the food industry needs not only cooling [6, 7], drying [8–10] and sterilization [11] technologies to improve the storage and transportation conditions of food products, but also adequate quality and safety testing technologies to monitor and evaluate the indicators in real-time before the supply chain reaches the consumer, so that the staff of each link can make timely adjustments to ensure food safety and reduce economic losses.

Fig. 1
figure 1

Basic segments in the food supply chain

Full size image

Traditional food quality and safety testing methods include high-performance liquid chromatography, gas chromatographic method, polymerase chain reaction, etc., which can be used to extract information related to the composition, structure, physical and chemical properties, and sensory characteristics of food products by adapting to the requirements under laboratory conditions [12]. However, most conventional detection techniques are subjective, destructive, non-real-time, labor-intensive, not suitable for online food analysis, and also make food information in the supply chain with some delay. Therefore, developing fast, non-destructive, and efficient detection techniques for the food supply chain is essential.

Machine learning based on the red, green, and blue (RGB) color model was one of the first techniques used for non-destructive testing (NDT) of food products, allowing rapid detection of ripeness and mechanical damage by extracting sensory features such as color, texture, and morphology of RGB images of food products [13]. However, this technique cannot detect the internal compositional changes of food products. It cannot confirm the scientific validity of non-destructive testing techniques from the perspective of chemical composition. With the rapid development of computer technology and instrumentation engineering, this research gap is complemented by the emerging spectroscopic techniques, which can provide information on the interaction of electromagnetic radiation with atoms and molecules to rapidly and nondestructively determine a wide range of quality parameters in food products [14], such as near-infrared spectroscopy (NIRS), hyperspectral imaging (HSI), Raman imaging, fluorescence imaging, laser light backscattering, and magnetic resonance imaging, etc. Among the techniques above, HSI techniques, as a novel alternative to traditional NDT techniques, can acquire spectral and spatial information about the inspected objects in specific spectral regions [15, 16]. Compared with the machine learning model based on RGB images, the combination of the HSI technique and machine learning can obtain more diversified food features, which can be used to quantitatively and qualitatively analyze the physical and chemical properties of food by identifying the spectral features of target molecules and the spatial features based on sensory qualities and can obtain better detection results. Therefore, machine learning and HSI techniques show strong potential for quality and safety testing in food engineering and are now widely used for physical properties, chemical composition, microbial content, authenticity, and traceability testing of food products [17–21].

Due to technological constraints, a modern information technology platform for the food supply chain is not yet in place, making it difficult for relevant inspection authorities to access reliable food information in real-time. Thanks to the efforts of many researchers, machine learning and HSI technologies are working to fill this gap. The technology has strong potential for application in sorting, packaging, transportation, storage, and sales of the food supply chain. However, no relevant literature reviews the application of machine learning and HSI technology in various segments from the perspective of the food supply chain, which may be the key to optimizing the food sorting, packaging, transportation and storage, and sales segments. Therefore, this paper aims to (1) Briefly describe the imaging process of HSI technology and the data processing process of machine learning. (2) Focus on the research progress of machine learning and HSI technology in the food supply chain from 2017–2022. (3) Discuss the application advantages, research gaps, and future development trends of this technology in the food supply chain.

Hyperspectral Imaging

HSI technology is a technique that can acquire spectral and spatial information from the measured sample. The technique probes various properties in a substance based on the interaction in the incident light and the measured substance. It can acquire the spectral and spatial characteristics of the detected object at 300–2600 nm, which was initially equipped on satellites and airborne platforms as remote sensing applications [22]. In recent years, with the improvement and maturation of spectroscopic techniques, HSI technology has made excellent research progress in the fields of agriculture [23], water resources protection [24], medical diagnosis [25] and food quality control [26]. Currently, the spectral range accessible by HSI devices in the market is usually in the visible and near-infrared (VNIR) (400–1000 nm) and near-infrared (NIR) (900–1700 nm) bands. The VNIR presents the content of compounds such as chlorophylls and anthocyanins, mainly used for non-destructive detection of sensory characteristics such as color, shape, hardness, and mechanical damage [27]. While the NIR band mainly reflects the overtones generated by chemical bonds such as O–H, C-H, N–H, and combined molecular vibrations, the content of organic compounds such as water, fat, and amino acids are presented in this range. Most organic compounds can be quantitatively and qualitatively analyzed in this band by suitable methods, so it is commonly used to identify food types, authenticity, and traceability [28, 29].

The imaging process of hyperspectral images is shown in Fig. 2 (in the case of kiwi fruit), which is presented as a whole as a hypercube containing two-dimensional spatial information and one-dimensional spectral information (Fig. 2(b)). In the spectral dimension of the hypercube, each dimension is a two-dimensional grayscale image of a single wavelength. The spatial information of the hyperspectral image is reflected as the pixel intensity of the two-dimensional grayscale image at a specific wavelength, and the spectral information is reflected as the intensity of each pixel point in the image at different wavelengths. Due to the differences in internal chemical composition or surface physical properties in food, the spectral information will show different trends, which lays a theoretical foundation for nondestructive testing of food.

Fig. 2
figure 2

a Hyperspectral imaging system. b The acquired hyperspectral image (with kiwi fruit as an example), each waveband presents a two-dimensional grayscale image, and the green part of the figure is the region of interest. c The average spectral curve of the region of interest

Full size image

Since hyperspectral images are susceptible to noise brought by the external environment and operators during the imaging process, noise not only interferes with the distribution of spectral features but also blurs or distorts the spatial information of the images, techniques such as median filtering, Savitzkye—Golay filter, and radiometric correction is usually used in research to reduce image noise and unevenness due to the morphology of food samples and during the imaging process spectral shifts caused by illumination [30]. In addition, background removal by thresholding or region of interest (ROI) selection is usually used in the study to remove the irrelevant target regions. Researchers usually use Unscrambler, ENVI, etc., to extract pixel points in ROI or average spectra of hyperspectral images as pixel-level or object-level data for analysis and processing.

Machine Learning and Data Processing Processes

Machine learning is an essential branch of artificial intelligence dedicated to exploring computers to simulate or implement human learning behaviors, reorganize existing knowledge structures and continuously improve their performance, and use the performance to predict or classify other unknown data. Therefore, machine learning training models usually have samples randomly divided into three parts: training set, validation set, and test set. Among them, the training set is used to establish the model, the validation set is used to adjust the parameters that control the complexity of the model and monitor whether the model appears to be overfitted, and the test set verifies the generalization ability of the optimal model. Machine learning contains subsets of traditional machine learning, chemometrics, artificial neural network (ANN), and deep learning [31], and the interrelationships among the subsets are shown in Fig. 3.

Fig. 3
figure 3

Types of algorithms included in machine learning and the relationship between them

Full size image

Traditional machine learning techniques can be categorized into algorithms based on the learning method, represented by clustering as unsupervised learning, supervised learning with classification and regression as the task, and reinforcement learning with environmental feedback as the model input. The main traditional machine learning algorithms used in food research such as partial least squares regression (PLSR), multiple linear regression (MLR), K-nearest neighbor (KNN), partial least square discriminant analysis (PLS-DA), random forest (RF), soft independence modeling of class analogy (SIMCA), and support vector machine (SVM), etc. Traditional machine learning algorithms are highly explanatory, have short modeling times, can be adapted to data sets of various sizes, and can exhibit good model performance, especially in small sample data sets. However, such algorithms follow standard procedures to solve problems, usually requiring splitting the problem into multiple parts. Therefore, this class of algorithms adds feature selection and extraction algorithms from chemometrics to the data processing step and selects or extracts features before feeding them into a machine learning model for training.

Chemometrics is the discipline that explores the connection between the measured values of chemical systems and the state of the system through statistical or mathematical methods, mainly using mathematics, statistics, and computer science to select the optimal experimental design and measurement methods to maximize the extraction of state features that contain helpful information. Chemometrics can therefore also be classified as a subset of machine learning. Due to the continuity between the spectral data of hyperspectral images, too much redundant data can significantly degrade the model's performance, i.e., the "curse of dimensionality" phenomenon [32]. Therefore, feature selection and extraction techniques are commonly used to reduce the dimensionality of the data. Among them, the core of feature selection is to create a subset of wavelengths in the spectral data based on a specific objective function, which reduces the data's dimensionality while preserving the original wavelengths' spectral characteristics. The commonly used algorithms are competitive adaptive reweighting sampling (CARS), iteration retaining information variables (IRIV), variable iterative space shrinkage approach (VISSA), PLS regression coefficients (RCs), and successive projections algorithm (SPA), etc. The feature extraction algorithm is a mathematical transformation method for information synthesis and enhancement in a spectral space or subspace, which mainly maps the main features of the original data to a low-dimensional space and combines the features with a specific mathematical model. The commonly used feature selection algorithms in research are principal component analysis (PCA), multidimensional scaling (MDS), isometric mapping (ISOMAP), locally linear embedding (LLE), and t-distributed stochastic neighbor embedding (t-SNE), etc.

ANN and deep learning are at the forefront of machine learning technologies. They have achieved many achievements in areas such as image processing, speech recognition, and biometric recognition. Traditional artificial neural networks, also known as perceptrons [33], can only be used to solve linear classification problems. Therefore, Rumelhart et al. [34] proposed a backpropagation algorithm for training data to adjust the weights between neurons by optimizing the iteration error rate, which improved the generalization ability of neural networks and laid the foundation for the development of deep learning. Deep learning is a form of machine learning that uses ANN as the underlying architecture with multiple hidden layers. At present, the well-developed network structures include deep neural networks (DNN), convolutional neural networks (CNN), and recurrent neural networks (RNN). As a multilayer nonlinear neural network model, deep learning has powerful feature learning (automatic feature extraction), classification, and regression capabilities, so this class of algorithms solves problems in a centralized manner without problem splitting and has advantages over traditional machine learning algorithms and chemometric-based feature selection and feature extraction algorithms in large sample data sets.

All of the above techniques are used to construct prediction or classification models in the food supply chain, and the primary data processing flow is shown in Fig. 4. The performance of the regression model is evaluated by determination coefficient, root mean square error (RMSE), standard error of prediction (SEP), bias factor, and residual predictive deviation (RPD). Furthermore, the performance of the classification model is evaluated by correct classification rate (CCR), precision, recall, and specificity. The evaluation criteria of the models are shown in Table 1.

Fig. 4
figure 4

Basic processing flow of food hyperspectral data

Full size image
Table 1 Evaluation criteria for regression and classification models
Full size table

Applications of Machine Learning and HSI in the Food Supply Chain

This paper provides an objective and comprehensive review of research advances in machine learning and HSI technologies in the food supply chain in sorting, packaging, transportation, storage, and sales.

Food Sorting

Food sorting is a necessary unit operation that directly affects the effectiveness of subsequent packaging, transportation, storage, and sales and is necessary to standardize the supply chain at the source and meet market surveillance needs. During the harvest season, rapidly picking crops by hand or using large agricultural machinery can lead to variable food quality, especially for fruits and vegetables, which have high moisture content, are easily bumped and perishable, and pose challenges for transportation and storage. Therefore, food sorting is the first step in food quality and safety testing. The application of machine learning and HSI technology in food sorting is shown in Table 2, which is mainly divided into three tasks based on sensory feature classification, quantitative and qualitative analysis of the chemical composition, and identification of food adulteration. In addition, when training machine learning models, the data types commonly used in research include extracting the food samples' average spectral curve of the ROI as object-level data or extracting each pixel of the ROI as pixel-level data, both of which are shown in the "Sample type" column of the Tables 2, 3, 4 and 5 in subsequent sections.

Table 2 Application of machine learning and HSI in food sorting
Full size table

Classification Based on Sensory Characteristics

Sensory characteristics are the most intuitive test indicators to describe and judge the quality of food, reflecting the regulatory authority's regulations on the characteristic quality and edibility of the food and comprehensive response to the pursuit of food safety. HSI technology in the sensory classification of food detection targets is generally color, hardness, flavor, and degree of spoilage to measure the maturity of food traits and mechanical damage from the outside, internal bruising, and colony contamination. The linear mapping model PLSR in machine learning is commonly used in studies to distinguish the ripeness of banana, cherry, and peach fruits and predict the spatial distribution of ripeness traits measured [38–36]. The reduction of redundant information based on the chemometric wavelength selection algorithm is further investigated in the above study to remove redundant information between adjacent wavelengths and simplify the model with the potential for online application. Moreover, PLS-DA and SVM can also be used in the current study to establish spectra with the classification of ripeness of easy-to-ripen fruits such as banana, strawberry, and durian, and the classification accuracy of the models established based on full-spectrum data is over 95% [37–46]. In addition, in the harvesting process of easy-to-bump fruit-based food, as the fruit is affected by external factors such as impact, extrusion, or abrasion, its surface will be bruised. As the fruit matures, the internal bruised tissue will rapidly oxidize and rot, resulting in food loss and waste. In the study, traditional machine learning algorithms such as PLSR and SVM combined with wavelength selection algorithms were used to detect and visualize the mechanical damage of blueberries [84], apples [85], and Lingwu long dates [86], respectively, all of which achieved good accuracy. To further study the extent of fruit bruising, [42] proposed a deep learning-based fully convolutional network (FCN) framework to extract bruised tissues inside blueberries, segmenting bruised tissues, healthy tissues, and calyx ends to improve the system monitoring accuracy, with an accuracy of 81.2%, which is better than SVM overall and can be used to study the mechanical damage resistance of fruits and provide guidance for fruit packaging and transportation.

Quantitative and Aualitative Analysis of Chemical Composition

The chemical composition of food products meets human needs primarily, and the chemical content directly affects human health. Therefore, the content of chemical components is regarded as an essential indicator for assessing the quality and safety of food products [87]. In research, food ingredients are usually classified into endogenous and exogenous substance components, with endogenous referring to the intrinsic ingredients of food and exogenous referring to the ingredients ingested during the whole food process from processing to ingestion. Traditional techniques for chemical composition analysis may yield different assessment results and conclude with subjectivity and uncertainty due to differences in detection tools and operator skills. Therefore, researchers have introduced machine learning and HSI techniques for the quantitative and qualitative analysis of chemical food components, which have been widely used to determine endogenous components such as soluble solids content (SSC), chlorophyll, total acidity, and fatty acid [56–47, 54]. MLR and PLSR are commonly used in studies to establish the relationship between the content and spectra of the above compounds. These linear models can predict elements with relatively high content in foods, and although good prediction accuracy has been achieved, the spatial features are still not exploited. In addition, the small sample size involved in the study failed to examine the effects of external factors such as type and region on the food products' attributes. To fully utilize the hyperspectral image features, Yu et al. [51], Shen et al. [52], and Yang et al. [53] used stacked autoencoder (SAE) to extract deep features in pears, plums, and peaches, respectively. SAE is an unsupervised feature extraction technique with deep learning and data dimensionality reduction capabilities. The study used this technique to extract fused spectral and spatial features, significantly improving the model feature representation capability. However, due to the multiple back-propagation in the SAE learning process, the training time is too long compared to traditional machine learning algorithms. The correlation between the model output and the deep hyperspectral features is ignored. Therefore, Tian et al. [48] proposed a stacked weighted auto-encoders (SWAE) algorithm for predicting SSC in apples, extracting the connection between model output prediction and deep spectral features. The experimental results show that SWAE improves the overall prediction performance of the model compared with SAE, and the decision coefficient of the model reaches 0.9436, which promotes deep learning as a new development trend in food NDT. In addition, to further reduce the training time of the model, Su et al. [55] attempted to build a one-dimensional convolutional neural network (1D-CNN) and three-dimensional convolutional neural network (3D-CNN) frameworks to process multidimensional information in images. The results showed that 1D-CNN could distinguish strawberry maturity and predict its SSC content in a small number of sample data sets. With the increase of samples, deep learning can fully exploit its potential in extensive data analysis.

Exogenous food ingredients mainly include food additives and contaminants, which account for a small percentage of the total amount of food but can directly affect food safety. Exogenous food ingredients studied by HSI technology mainly include pesticide residues, heavy metals, and microplastic contamination. Pesticides are like double-edged swords. They can prevent pests and diseases in agricultural products. However, pesticide residues in agricultural products can pose a significant safety risk to human health if misused. Feature selection algorithms such as CARS, SPA, stepwise regression, and RCs were used in the study to select the wavelengths of food samples. Finally, traditional machine learning models such as PLSR, MLR, PLS-DA, and SVM were built to predict the concentration of pesticide residues based on the selected wavelengths and visualize the pesticide residue distribution [57, 58]. However, all the above studies detected a single pesticide. Considering the prevalence of using multiple pesticides in agriculture, He et al. [59] proposed a deep learning model based on 1D-CNN to identify pesticide residues on leek leaves and predicted mixed pesticide residue samples based on this model with reasonable accuracy. The impact of heavy metals on food should not be underestimated either, and heavy metal pollution in the soil is becoming increasingly severe due to the exploitation of mineral resources. Crops will absorb heavy metals due to atmospheric deposition and rainfall. Excess heavy metals will affect crops' physiological and biochemical activities, inhibit plant growth, and finally be absorbed by the human body and accumulate in the blood to damage the nervous and digestive systems. The current research directions in heavy metal pollution are divided into predicting the concentration of heavy metals in crops and classifying different classes of heavy metals for detection. The model developed in the above study has achieved good accuracy [88, 60]. However, all the above studies established traditional machine learning models based on the spectral features of crops. To further explore the degree of heavy metal contamination in soil, Wang et al. [61] proposed an algorithm combining data augmentation (DA) and a deep learning network to classify the degree of chromium contamination in soil. The accuracy of DA-DNN reached 96.25%, proving that HSI technology combined with deep learning can realize large-scale detection of soil heavy metal pollution. In addition, microplastics are also a new type of pollutant based on modern industrial development, referring to plastic fragments and particles less than 5 mm in diameter in general. In recent years, different microplastic particles have entered the soil environment through wastewater irrigation, plastic waste disposal, and atmospheric deposition, causing severe terrestrial environmental pollution. Therefore, Ai et al. [62] employed CNN to classify various microplastic polymers and their mixtures in soil. The experimental results proved that deep learning has significant advantages in the classification accuracy of microplastics.

Identification of Food Adulteration

Food adulteration refers to intentionally adding non-inherent ingredients to food to reduce costs and increase sales profits. In recent years, the ordinary means of food adulteration include adulteration, substitution, and forgery, and there is no clear distinction between the boundaries. Among them, adulteration refers to the illegal adulteration of food with substances similar to food in physical form; substitution refers to the adulteration of food with non-identical or inferior products of the same kind; forgery refers to the imitation of food with other substances and selling them in the market. Adulteration is mainly found in costly commodities such as spices, powdered products, and nutritional products, which disrupt the food market. The adulterated substances in them are likely to cause severe food safety problems. Food products adulterated with different concentrations of impurities are usually classified in studies, and then regression models are used to predict the concentration of adulteration in food products, such as almond flour [66], tapioca flour [68], whole wheat flour [64], cumin powder [70], and minced pork [67] for contamination analysis, and traditional classifiers can all achieve good classification results. However, as the concentration of food adulteration increases, the penetration ability of spectral imaging is limited, and the accuracy of algorithm identification decreases. Therefore, ANN with nonlinear mapping was also used in the study for classification, such as adulteration classification of chocolate powder [90], nutmeg [65], red chili powder [69], colla corii asini [63], marine fishmeal [71], and cereal [72]. The results demonstrate the superiority of the nonlinear mapping model. Although there is no significant improvement in accuracy compared to traditional classifiers, the model's specificity is significantly enhanced, and the training and prediction speed is significantly better than that of traditional classifiers. The substitution phenomenon is mainly found in agricultural products such as meat, tea, and cereals that have essential effects on the overall attributes due to factors such as variety, grade, and origin, and their prices and popularity can vary greatly depending on the characteristics and consumer preferences of the food products [2, 4]. Traditional classifiers such as SVM and PLS-DA primarily identify the varieties, origins, and grades of agricultural products such as tea [74, 73, 76], coffee beans [78], cocoa beans [79], and chia seed [83] for adulteration and the accuracy of the models are above 90%. However, the sample size for establishing the traditional classification model is relatively small. Although it is used extensively, the model's generalization ability is relatively weak and not stable enough. Therefore, deep learning algorithms are introduced in the study for adulteration detection of agricultural products. Most pixel-level data are used in the study of different varieties and grades of, for example, tea [77, 75], grains [81, 80], and meat [82]. The study results demonstrate that the deep learning algorithm performs stably compared with the traditional model and alleviates the more serious overfitting problem in the traditional model. In addition, there is no research related to machine learning and HSI technologies for food forgery, so it can be one of the future research directions.

In summary, the linear mapping models PLSR and MLR are mainly used to differentiate the ripeness of fruits, predict the chemical content and visualize their spatial distribution. The number of food samples involved in the studies above is usually between 100 and 300, with the common problem that the training data set is too small, and most of the studies focus on finding the association between a single factor of the food and its spectrum. Nonlinear models such as DT, RF, and SVM are usually used for studies when lower levels of chemicals in food products are detected or when no clear linear relationship between the measured parameters and the spectra appears. Nonlinear models apply to data sets of various sizes and are commonly applied in food chemical content prediction and adulteration identification. In the above studies, the original models are usually simplified using wavelength selection and feature extraction algorithms to shorten the training time and make the simplified models have the potential for online applications. Deep learning models are more suitable for a large order of magnitude samples. The models are usually trained using pixel-level data in studies involving samples usually above 1000, which can be applied to all classification or prediction tasks in food sorting. Deep learning models mostly use transfer learning to expand the training samples, so the network looks for the connection between multiple factors and food spectral features. Also, an attempt can be made to build sorting production lines to evaluate the potential of deep learning models in practical applications.

Food Package

Food packages play an essential part in controlling product quality, especially pickled, fermented, fried, and air-dried foods usually require high-quality sealed packaging to meet the need to extend the shelf life of food. Packaging seal integrity and cleanliness are critical to food safety. Suppose sealed food in processing, storage, transportation, or shelf during the breakage or air leakage, or sealed packaging contains impurities. In that case, it will lead to food contamination, moisture, and deterioration, shortening the shelf life and directly affecting the food quality in the bag. It has been demonstrated that package seal integrity and food contamination during processing and packaging are the leading causes of food waste in the supply chain, with one-third of the food in the production line being discarded or wasted due to package contamination. Traditional testing methods cannot identify sealing problems due to material transparency and chemical composition. Therefore, HSI technology has also been introduced to detect food packaging materials with similar absorption spectral properties for control detection, as shown in Table 3. The packaging segment usually involves large data sample sets and production line design, so deep learning algorithms are usually used in research for food quality detection. Initially, researchers proposed a principal component analysis network (PCANet) to detect food tray sealing faults, using the network to learn the spectral and spatial characteristics of 11 harmful impurities in packaging trays and classify them [91]. PCANet is mainly used for spatial and spectral data fusion and output feature vectors, KNN and SVM as output classifiers with the accuracy of 89% and 90%, respectively, and there is space for improvement in both accuracy and detection speed. Benouis et al. [92] proposed a deep learning-based fault detection method for food tray sealing to improve the model's generalization ability. To reduce the amount of data for training the network, two pixel-level fusion algorithms, spatial and transform image fusion, were used in the study to process the data. The fused data were fed into deep belief networks (DBN), extreme learning machine (ELM), SAE, and CNN for training, and CNN achieved the best classification performance among all fusion rules. Medina et al. [93] further proposed using CNN as a classifier for heat-sealed food packages. In their study, they established a system that can perform the work of sealing contamination of food trays in the production line. The contaminated trays were automatically discarded from the production line. The study results show that most harmful impurities can be detected correctly (error less than 8%), except for polarized plastics and hair, which cannot be captured due to reflective properties and imaging equipment limitations. In addition, in addition to sealing tray contamination, another critical aspect of the packaging operation is quality control, where machine learning algorithms can automatically detect anomalies and reject non-compliant products on the production line. Banus et al. [94] proposed a computer vision system based on the CNN algorithm to automate package sealing and tight sealing inspection to satisfy the production needs. In the study, pizza packages were experimented with for closure and sealing to ensure food storage conditions. A production chain was established to automatically classify packages for acceptance or rejection, with a model accuracy of 99.87% and a conveyor speed faster than manual twice as fast, demonstrating the superiority of machine learning and HSI technologies in food packages.

Table 3 Application of machine learning and HSI in food package
Full size table

Food Transportation and Storage

Compared to other transportation industries, food transportation is characterized by the need for a high degree of cleanliness and hygiene, strict delivery times, and a stable storage environment. Conventional food transportation techniques lead to the waste of about 20% of the gross food product due to spoiled food discarded in China. Especially fresh food such as fruits, vegetables, and seafood, most of which cannot be kept fresh for a long time and spoil during transportation. Therefore, continuous logistics and specific storage conditions must be established during transportation to ensure food safety. The most commonly used food storage technologies are low-temperature refrigeration, dry dehydration, etc. The applications of machine learning and HSI technology in food transportation and storage are shown in Table 4.

Table 4 Application of machine learning and HSI in food transportation and storage
Full size table

Low Temperature Refrigeration

Low-temperature refrigeration can inhibit the reproduction of microorganisms in food, and the slower the microorganisms reproduce, the slower the flavor, color, and nutrient content of the food will change. Generally speaking, the lower the refrigeration temperature, the longer the shelf life of the food, and the freshness will also be reduced. Freshness is one of the most critical evaluation criteria for food quality and safety. Standard testing tasks include storage time, total viable count (TVC), total volatile basic nitrogen (TVB-N), etc.

Storage time is the most direct measure of food freshness. Since fruit and vegetable foods have high water content and rich nutrients and are highly susceptible to fungi, these foods are usually placed on ambient shelves or low-temperature freezers, where decay is relatively slow under low-temperature storage. The increase in storage time affects the sensory characteristics and chemical composition of fruits and vegetables, such as hardness, SSC, and pH, which reflect the overall quality of food during storage. Traditional algorithms are usually used in research to establish the connection between spectra and storage time, such as nondestructive detection of storage time of fresh spinach [43], strawberries [89], tea leaves [96] and sausages [95], all of which have obtained good accuracy, proving that the HSI technology and machine learning in identifying food good performance in storage time. However, the general problem of the study is that the data processing process is cumbersome and ignores the effect of factors such as temperature and humidity on the storage time to assess the algorithm's reliability. Chemometric-based feature extraction and selection algorithms were added in some studies to reduce further the model's training parameters, which added additional computational effort but significantly improved the model's running speed and robustness. In addition, storage of grains is an important food safety topic, as unsaturated fatty acids and toxic substances (alcohols, aldehydes, ketones, etc.) in grains increase with age, and the content of protein, starch, and SSC decreases year by year, with subsequent loss of seed viability. The traditional algorithms PCA and SIMCA were used to identify wheat seeds from 2007 to 2012 and analyze the spectral differences due to different storage times [97]. The experimental results showed that the traditional model was 97.05% accurate for adjacent years and 82.5% accurate for mixed classification of 6 years, which proved that the traditional machine learning combined with HSI technology could respond to the differences of seeds in different storage times. However, the overall classification accuracy was not high and ignored other factors such as origin and species. Pang et al. [98] used a deep learning framework to fully use spectral and spatial information to identify and predict maize seed vigor. 1D-CNN classified the four seed vigor levels with 99.98% accuracy. Two-dimensional convolutional neural network (2D-CNN) converged faster, and the accuracy reached 99.96%, proving that deep learning has better recognition ability.

Meat and seafood products are rich in protein and suitable for propagating many pathogens and spoilage microorganisms, so TVC has also become one of the crucial indicators of hygienic quality and food safety of animal foods. In the study, SVM, least squares SVM (LS-SVM), and PLSR were used to predict the microbial content of pork [101], rainbow trout fillets [103] and cooked beef [100], respectively. The prediction accuracy of the nonlinear models was all the better than the linear model PLSR, indicating that the nonlinear models have higher robustness and more vital generalization ability. To further explore the relationship between TVC and spectral information, Yu et al. [99] proposed a deep learning framework based on SAE and fully connected neural network (FNN) composition to predict TVC during the storage of Pacific white shrimp. The results showed that although the accuracy of SAE-FNN was similar to LS-SVM in the training set, its correlation coefficient was significantly higher than that of PLSR and LS-SVM in the prediction set, demonstrating that the deep learning framework has strong potential for application in the content prediction of TVC.

TVB-N refers to the alkaline nitrogenous substances such as ammonia and amines produced by the decomposition of proteins in animal food during spoilage. These substances are volatile, and higher levels indicate more amino acid destruction in the food, so their levels can be used to determine the freshness of animal foods. The linear model PLSR is commonly used to predict the TVB-N content of animal foods such as pork [108], oysters [106], and tilapia fillets [104] and to generate chemical images to visualize the distribution of the content. In addition, the nonlinear models LS-SVM [107] and RF [110] are also commonly used for TVB-N content prediction and freshness classification. The above models based on spatial and spectral fusion datasets have good performance. Moosavi-Nasab et al. [105] used a linear deep neural network (LDNN) coupled to predict the TVB-N content of rainbow trout fillets, and the study's results showed that the deep learning framework achieved 90% classification of fillet freshness. However, the performance in predicting TVB-N content was inferior to PLSR and LS-SVM. In conclusion, there is no significant linear and nonlinear relationship between spatial and spectral fusion data and TVB-N content. In addition, to further validate the feasibility of the HSI technique for predicting TVB-N, other techniques were used in some studies to build prediction models by fusing with the HSI technique. Liu et al. [109] used a combination of electronic nose and HSI technique and proposed an input-modified convolution neural network (IMCNN) to predict the TVB-N content of lamb. The experimental results demonstrate that the fusion of e-nose and HSI techniques can achieve a more accurate TVB-N prediction of lamb, and IMCNN has good feature extraction and modeling capability for one-dimensional vector e-nose sensing data.

Food Drying

Food drying can reduce the moisture content in food and make perishable microorganisms lack the conditions to grow and reproduce, thus extending the storage life of food and reducing the costs of packaging, transportation, and storage in the supply chain. Commonly used food drying methods include hot air drying, microwave drying, and freeze-drying. Food products shrink during the drying process due to structural changes, and it has been proved that there is a linear relationship between shrinkage and moisture content of agricultural products. Hence, the moisture content is an essential indicator for measuring the drying degree. PLSR is usually established in research to predict moisture distribution [115–114], compound content [117], and textural properties [113] of dried foods, and all of them show good prediction ability. To further investigate the mapping relationship between drying techniques and food moisture content, shrinkage, and spectra, SVM is commonly used in studies to predict and classify food moisture, texture [112], and quality [118]. The results show that the prediction accuracy of SVM is better than that of PLSR, which is commonly used in studies. In addition, drying techniques can be evaluated by analyzing the moisture distribution of samples. PLSR is commonly used in studies to establish the link between samples and spectra to evaluate the drying effectiveness of standard techniques such as hot air, microwave, and combined (hot air and microwave) by predicting the moisture content, rehydration rate [119], and moisture distribution [111] of food products. The study showed that combined drying resulted in a more uniform moisture content distribution of dehydrated foods, reduced drying time, and improved color quality.

Food Sales

The indicator for ensuring food safety in the distribution process is the food shelf life, which refers to the period during which the food can be guaranteed its sensory, physicochemical, and microbiological properties under the manufacturer's recommended storage conditions. HSI technology usually extracts the spectral profiles of food samples to establish the association between the spectra and their physicochemical parameters and microbiological properties such as color parameters, pH, TVC and TVB-N, etc. The applications of machine learning and HSI technology in food sales are shown in Table 5. Siripatrawan and Makino [120] and Sricharoonratana et al. [121] used PLSR to predict packaged sausage and cakes' quality attributes and spoilage level, respectively achieved good predictions. In the second study, PLS-DA was further used to develop a classification model to distinguish whether a cake is expired or not, and its accuracy reached 91.3%. To investigate the potential of nonlinear models in distinguishing food shelf life, Saleem et al. [122] developed SVM to classify PCA-treated bakery product data. This technique can detect microbial spoilage before visual changes occur in food products with an accuracy of 98.3%. However, for components that are present at low levels in food products, ANN combined with spectral data is commonly used in studies for prediction, such as [123] to predict the rate of chlorophyll degradation in broccoli, which can be further graded based on the shelf life of the product. To further validate the feasibility of the HSI technique for predicting food products' shelf life, multivariate accelerated shelf-life tests (MASLT) combined with spectral information have been used in some studies to develop shelf-life prediction models. Cruz-Tirado et al. [124] and Chaudhry et al. [125] studied chia seeds and rocket leaves, respectively, hypothesizing that spectral changes in food products may reflect chemical changes due to temperature alterations during temperature storage, and incorporated PCA and PLS into MASLT, respectively. Compared to traditional MASLT, MASLT built on PCA can capture the most valuable information in the model. The PLS-based MASLT has potential for industrial applications because it is more robust and allows the addition of new samples during the calibration process.

Table 5 Application of machine learning and HSI in food sales
Full size table

In summary, the common problem of the models built using traditional machine learning algorithms in each session is that the training data set samples are too small, and most of the studies only focus on the effect of a single factor of food on its spectrum, so the models built also do not have good generalization ability. In addition, most of the above studies use object-level data as experimental samples and adopt the average spectral curve of ROI to train the model. Therefore, this significantly reduces the training parameters of the model and enables the model to learn the spectral features of multiple samples. At the same time, the spatial features are lost, which may lower the model's accuracy. Research in the packaging segment primarily used deep learning networks for package sealing and contamination detection and usually used pixel-level data to train and test the networks. The large amount of data in the network leads to a strong dependence of the training model on the hardware platform.

The Potential of Machine Learning and HSI in the Supply Chain

Machine learning and HSI techniques for non-destructive food inspection have become one of the hot inspection techniques in the food industry in recent years. They are mainly used for food quality and safety inspection in sorting, packaging, transportation, storage, and sales. It has the advantage that little or no chemical processing of the sample is required and that spectral and spatial features can be extracted by imaging the sample alone, thus offering potential applications in the online inspection. Secondly, machine learning can take different algorithms for reading and processing according to the data characteristics. Each type of algorithm in machine learning has its advantages, such as linear regression models MLR and PLSR, which have the characteristics of easy fitting and accessible interpretation, can show good prediction accuracy in small data sample sets, and are commonly used for chemical composition analysis in sorting and transportation storage links. Nonlinear models are more suitable for the analytical processing of such problems when there is no apparent linear relationship between the measured parameters and the spectral features in the sample. For example, SVM can balance the complexity and generalization ability of the model in a small sample data set and is often used for some chemical content prediction and adulteration detection of expensive food products. Deep learning can show strong nonlinear approximation ability in big data sets and automatically learn hyperspectral deep features. Many studies using deep learning for non-destructive food inspection have not preprocessed the spectra, and the models still maintain good performance. Therefore, this technique can be used for inspection involving the production line's sorting, packaging, and sales stages. In summary, the technology can be applied to real-life applications with the help of hardware platforms and integrated software, which general practitioners can use to test the quality and safety of food products at various supply chain stages.

However, some non-negligible limitations of the technique make it bottlenecked in developing practical applications. First, during the imaging process, the data are subject to external influences such as acquisition method, detector type, exposure time, operator, etc. External noise interferes with the distribution of spectral features and blurs or distorts the spatial information of the image. Secondly, a unified standard has not yet emerged to evaluate the extracted features during data processing. Only the accuracy of the model built based on the extracted features can be compared with the original data. Therefore, when selecting algorithms to simplify the model, researchers must constantly compare and trial and error various extraction algorithms. In addition, a common problem in the modeling process is the small sample of food products. Many works focus on studying only a single attribute of a food product, ignoring the influence of other factors. Especially for deep learning techniques, the performance is closely related to the size of the dataset. However, current hyperspectral image databases for food inspection are unavailable, and researchers cannot access databases from other supply chain segments for application extension. Moreover, deep learning is also hardware platform-dependent, and its model building requires a large number of data operations and thus requires sufficient hardware resources for support. Finally, in the link involving the production line, the speed of the conveyor belt transporting the products also needs further consideration. Suppose the transport speed of the product changes drastically due to external factors. In that case, the sampling frequency of the hyperspectral camera should also be changed to avoid distortion of the image morphology. The spatial features learned by the network will also change, so the network needs to be retrained.

Despite the limitations of the technology, with the rapid development of computer technology and related hardware and software, it is believed that fast and efficient related applications will soon emerge in the food supply chain. From the perspective of developing software, the research needs to optimize the corresponding algorithm according to different imaging environments (such as uneven light sources in imaging, environmental scattering, and background clutter). Industrial inspection standards also have higher requirements for model accuracy and running speed, so many studies currently focus on reducing training data to reduce the running time of the network while ensuring higher model accuracy. Regarding the problem of a single variable and small amount of data for food research, specialized organizations are needed to build hyperspectral datasets of various foods for sharing. Ideally, future research efforts would be to develop deep learning algorithms based on the entire hyperspectral image database to improve the accuracy and generalization capabilities of the network. However, a balance between model accuracy and runtime must be considered. Only if the model accuracy is always maintained at a high level can the ability to extend the model's functionality using the entire hyperspectral image database be achieved. Moreover, future work will be reality-based and focus on analyzing the impact of multiple factors on various segments of the food supply chain, leading to more generalized solutions.

Conclusion

Ensuring the quality and safety of the food supply chain is an essential issue for the modern food industry. Machine learning and HSI technologies have been widely used in food quality and safety inspection in supply chains in recent years. Based on relevant literature from 2017 to 2022, this paper provides an objective and comprehensive overview of the application of machine learning and HSI technologies in the supply chain for sorting (sensory characteristics, chemical composition, food adulteration), packaging, transportation, and storage (low-temperature refrigeration and food drying), and sales, and draws the following conclusions: (1) The technology has shown excellent qualitative and quantitative analysis capabilities in the sorting segment. (2) Deep learning has initially demonstrated its powerful spectral-spatial information processing capability in the packaging segment. Studies have been conducted to build production lines and obtain good automated inspection results. (3) The current bottlenecks of the technology in the production line and in-line applications are the lack of hyperspectral image datasets, setting production line transport speed, measuring training time and model accuracy, and other limitations. (4) Advancing the technology in terms of hardware platform, integration software, algorithms, standardization, and data set sharing to promote the application of the technology in real life. In recent years, due to the widespread outbreak of COVID-19, all segments of the food supply chain have been severely affected, with reduced labor and transportation changes causing food to stall in storage and transportation, resulting in a significant increase in food losses. Therefore, relevant authorities need to make timely adjustments and control the affected parts of the supply chain to minimize the food waste caused by the outbreak. Machine learning and HSI technologies can potentially reduce human involvement and damage to food caused by the external environment, storage time, mechanical damage, and microorganisms through online quality inspection and control of food in production lines and transportation processes.