Introduction

Recently, thermal imaging (TI) technology has become a useful diagnostic method due to a non-contact of the sensors with objects and potentially fast applications. This technique obtains thermal image or thermogram of the object. The image contains information that could become useful by applying appropriate analyses methods.

Thermal imaging technique has been applied by many researchers for product identification and defect detection in agriculture and food industries [13].

Thermal imaging can become a specification method to detect areas of interest by means of appropriate classification methods [3]. Classification is a kind of data mining technique to distinguish different categories of the observed data [11].

Until now, several classification methods such as linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), support vector machines (SVM), and artificial neural networks (ANN) were applied on thermal imaging data for several goals [2, 7–9]. For example, Chelladurai et al. [2] used thermal imaging technique for classification of healthy and fungal-infected wheat kernels. They classified the healthy and fungal-infected wheat by LDA and QDA methods, and the results showed that the QDA method gave better performance than the LDA method.

Manickavasagan et al. [8, 9] classified wheat classes using temperature gradients obtained from thermograms of different Canadian wheat classes by QDA and PROC STEPDISC in the Statistical Analysis System (SAS) environment. They reported correct classification rates (CCR) from 54 to 88 % for different class identifications [8, 9].

Vadivambal et al. [12] analyzed infrared thermal data from images of healthy and sprout‐damaged wheat kernels using LDA, QDA, and ANN methods. They reported the highest classification accuracies by the ANN method as 99.4 and 91.7 % for detection of healthy and sprout‐damaged wheat kernels, respectively [12]. A system was developed for monitoring the condition of vibro test machines using log-polar mapping method based on thermal imaging by Wong et al. [14]. They obtained more than 94 % accuracy using max-product fuzzy neural network classifier. Baranowski et al. [1] applied soft independent modeling of class analogy (SIMCA), LDA, and SVM on thermal imaging data to detect early bruises in apple fruits. They obtained the best CCR by LDA method as 95 % [1].

The goal of this paper is to develop and evaluate a new algorithm to classify healthy and fungal-infected pistachio kernels by fungi at different infection stages using thermal images.

Materials and Methods

Sample Preparation

Pistachio nuts were selected randomly from Akbari variety, Rafsanjan, Kerman, Iran. Firstly, pistachio kernels were sterilized using an autoclave at 121 °C for 20 min; around 10 % of the sterilized pistachio kernels were separated as healthy specimens and the rest were used for inoculation by the selected isolates of fungus Aspergillus flavus which has many isolates capable of producing aflatoxin in oilseeds. This toxin can be formed in oilseed crops in farms or stores [6].

To study the feasibility of detection of fungal infection by thermal imaging, a half of the sterilized pistachio kernels were infected by A. flavus isolate R5 and the other lot of kernels by the isolate KK11. The R5 is an aflatoxin-producing isolate of A. flavus, whereas KK11 is unable to produce aflatoxin. The inoculated kernels were put on several layers of sterilized paper towel kept in sterilized Petri dishes. The Petri dishes were then placed in freezer plastic bags which were evacuated of air manually, tied, and then incubated at 30 °C for a week. To investigate infection stages, the infected kernels were taken to acquire thermal images after each day (i.e., infection stage 1 (one day after putting infected kernels in incubator), infection stage 2, …. and infection stage 7).

Thermogram Acquisition

A ULIRvision thermal imaging camera (Model: TI160, Zhejiang ULIRvision Technology Co, China) was connected to a Pinnacle frame grabber (Model: 500-PCI, Pinnacle Co., China) installed in a personal computer (PC) to record real-time pseudo-image videos of healthy and fungal-infected pistachio kernels. The spectral range of the camera was 800–1400 nm with 160 × 120 pixels resolution. The thermograms were obtained using Video-to-Frame software [4].

In this research, more than 20 replications [10] were investigated for each pistachio class. Firstly, the emissivity of the thermal imaging camera was adjusted to 0.95 [5], and then three thermograms were acquired for each replication:

Thermogram 1:

obtained from pistachio kernels before heating

Thermogram 2:

obtained from pistachio kernels immediately after heating at 90 °C for 90 s

Thermogram 3:

obtained from pistachio kernels after cooling at room temperature for 10 s

Feature Extraction

An algorithm was programmed in MATLAB 2010A software to extract thermogram features. From each thermogram, the following statistical features were extracted: maximum, mean, minimum, standard deviation, coefficient of variation, median and mode.

The extracted features were used to calculate new features based on equations (1 and 2):

$$ F_{\text{n1}} = F_{ 2} - F_{ 1} $$
(1)
$$ F_{\text{n2}} = F_{ 2} - F_{ 3} $$
(2)

where F n1 and F n2 are new features, F 1 is a feature of thermogram 1, F 2 is feature of a thermogram 2, and F 3 is a feature of thermogram 3. F n1 and F n2 were calculated for the statistical features extracted from all thermograms and then were used for classification of healthy and fungal-infected pistachio kernels.

TBC Algorithm

In this research, a new algorithm was developed based on threshold in MATLAB 2010A software for classification of healthy and fungal-infected pistachio kernels using thermal images. The TBC algorithm is shown in Fig. 1 and is described in the following sections.

Fig. 1
figure 1

TBC algorithm

Full size image

Reading Inputs, Desired Files and Class Number

After reading of inputs and desired dataset, the number of classes should be specified for the algorithm.

For example, the file in Table 1 contains three features which could be extracted from raw thermograms for five different samples for each of the three classes.

Table 1 Input data (left) and desired class (right)
Full size table

Maximum and Minimum of Each Feature

In the next step, algorithm found the maximum and minimum of each feature of all the classes. These data were considered to establish the thresholds. Maximum and minimum for each feature in Table 1 are given in Table 2.

Table 2 Maximum and minimum of each feature for all classes
Full size table

Establishing the Thresholds

The algorithm calculated the thresholds for all classes except the first class and the last class, as follows (3, 4):

$$ {\text{Th}}_{ \hbox{min} } \left( i \right) = \left( {{\text{Mx}}\left( {i - 1} \right) + {\text{Mn}}\left( i \right)} \right)/ 2 $$
(3)
$$ {\text{Th}}_{ \hbox{max} } \left( i \right) = \left( {{\text{Mx}}\left( i \right) + {\text{Mn}}\left( {i + 1} \right)} \right)/ 2 $$
(4)

where Thmin(i) is the under limit of i th class, Thmax(i) is the upper limit of i th class, Mx(i − 1) is the maximum of i − 1th class, Mn(i) is the minimum of i th class, Mx(i) is the maximum of i th class, and Mn(i + 1) is the minimum of i + 1th class.

In this algorithm, Thmin(1) was minimum of class 1 and Thmax(3) was maximum of last class. For the example in Table 1, the calculated thresholds from entries in Table 2 are given in Table 3.

Table 3 Threshold file for all features in all classes
Full size table

Test

After finding the thresholds, the input data given in Table 1 were used for testing.

Comparing Inputs with Thresholds

In this step, each feature in each column of input data was compared with all thresholds of corresponding column in the threshold file (Table 3).

In the example in Table 1, each feature in column F1 was compared with all thresholds in column Thresholds 1 (Table 2), and so on. Results of each comparison were placed in a file as output file. The number of columns and rows in this file was equal to that of input data. The results of comparison for the example in Table 1 are shown in three left columns of Table 4.

Table 4 Output of comparing the inputs with the thresholds (three left columns) and final product (right column)
Full size table

Obtaining the Final Product

In the next step, the final product was obtained. In this step, each row of data was assigned to a class that had the maximum repetition in the row. If the repetition of two classes in a row is equal, the row was assigned to the lower class. The result is shown in the right column of Table 4.

Comparing Final Product with the Desired File

In the next step, final product was compared with the desired file (Table 5). For example, the final product (left column) was compared to the desired class (middle column) and the result of this comparison is shown as result column (right).

Table 5 Final product, desired class and result class for the example in Table 1
Full size table

Report the CCR

In the final step, the correct classification rate (CCR) of the algorithm was calculated and then reported. For example:

$$ {\text{CCR}} = \, \left( { 1 4/ 1 5} \right)* 100 = 9 3. 3 3 $$
(5)

The TBC was such designed to be applied to classify inputs with unlimited number of classes and features. Also TBC was developed for real time utilities by developing a TBCsim algorithm. This algorithm was similar to TBC, but it compared the new input data with thresholds.

To study the effect of the thresholds, section “Establishing the Thresholds” was removed from TBC algorithm and new algorithm, TBC2, threshold based classification 2, was created. In this algorithm, the inputs data were compared to their maximum and minimums.

Evaluation of TBC Algorithm

Thermal images of healthy and fungal-infected pistachio kernels were classified by the TBC and TBC2 algorithms. The corresponding CCRs for classifying the extracted features from the thermograms were computed and compared.

Results and Discussions

The classification results of the healthy and fungal-infected pistachio kernels at different fungal growing stages are given in Tables 6, 7, and 8 for KK11, R5, and combination of both isolates, respectively. The explanations of these tables are as follows:

Table 6 The correct classification rate (CCR) of classifying healthy and fungal-infected pistachio kernels by KK11 isolate of A. flavus at different stages
Full size table
Table 7 The correct classification rate (CCR) of classifying healthy and fungal-infected pistachio kernels by R5 isolate of A. flavus at different stages
Full size table
Table 8 The correct classification rate (CCR) of classifying healthy and fungal-infected pistachio kernels by two isolates (KK11 and R5 of A. flavus) at different stages
Full size table
  1. (1)

    Two-way classification: detecting; 1. healthy pistachio and 2. 1-day, 2-day, … and 7-day after infecting.

  2. (2)

    Three-way classification: detecting; 1. healthy pistachio, 2. 1-day, 3. 2-day, 3-day, … and 7-day after infecting.

  3. (3)

    Four-way: detecting; 1. healthy pistachio, 2. 1-day, 3. 2-day and 3-day, and 4. 4-day, 5-day, 6-day, and 7-day after infecting.

  4. (4)

    Eight-way classification: detecting all of eight pistachio classes, separately.

As seen in the Tables 6, 7, 8, the results of TBC and TBC2 algorithm were close to each other. The CCR of the TBC and TBC2 algorithm were 100 % to classify healthy and fungal-infected pistachio kernels using two-way model (Tables 6, 7, 8). The results of TBC algorithm for three-way classification in Tables 7 and 8 were higher than those of TBC2. The TBC model showed its high ability in classifying healthy, 1-day, and later infected kernels (92.33 %).

Conclusions

A new TBC algorithm was developed and evaluated for classification of thermal imaging data of healthy and fungal-infected pistachio kernels. The results of the TBC algorithm were compared with the results obtained by TBC2 algorithm. The TBC gave best results and can detect fungal infection of pistachio kernels with 100 and 92.33 % accuracy in a two class model consisting of healthy and infected classes and for classifying healthy, 1-day, and later infected kernels, respectively.