Introduction

Fish consumption is motivated by its taste, health, quality, and freshness. The demand for high-quality and immune fish products has increased over the past few years due to recent changes in consumer lifestyles [23, 53, 54]. There are several factors associated with fish quality parameters that range from harvesting to consumption, such as safety, nutrition, availability, and freshness, which are affected by storage and processing methods [49]. A variety of factors may negatively affect fish quality and freshness, including production, transportation, sale, domestic storage, and final preparation of food [11, 32]. Fish provides essential nutrients, vitamins, and proteins for human wellness [12, 37].

Fish freshness may be determined using a variety of techniques. Furthermore, with greater technical advancements, there have been attempts to create a way of measuring and assessing more dependable freshness. The criteria used to determine freshness include sensory, physical, chemical, and microbiological. Rapid protein liquid chromatography and hyperspectral imaging techniques are also considered [37, 53, 54]. It is thought that a fish's eye region has a strong relationship between its coloration and the period during which they are stored. For fish to maintain its highest quality after harvest, it must be kept at a specific temperature for a specific time [11].

A multi-nutrient food that provides proteins, omega-3 fatty acids, and vitamins (health benefits) [14]. The fresher the fish, the more nutritious it is. Consequently, it can be challenging for most consumers to determine whether a fish is fresh while they are shopping. By touching and squeezing the fish's body to determine its flexibility, you may quickly determine how fresh it is [53, 54]. Normally fresh fish has higher elasticity. Unfortunately, using this method can contaminate food with germs, harming fish and leading to food-borne illnesses [17, 38, 39].

Fish spoilage is the process through which the quality of fish deteriorates, altering its color, odor, smell, flavor, and flesh texture [4, 14]. There are two primary sources of decomposition once it starts: biological spoilage and chemical spoiling [16]. Microorganisms invade the fish's body via the gills and commence the process of decomposition. Due to chemical interactions, chemical spoilage causes a disagreeable odor and also affects the flavor [42].

The changeover from fresh to stale is indicated by a change in the color of the gills from brilliant pink to dark red or yellowish red. A faded color will be detected if the discoloration of the skin is overall. It is easy to tell a fresh fish by the shine and brightness of its skin, as opposed to a not-so-fresh fish by its dullness and faded colors. Depending on the color of the flesh of the fish, freshness can be determined by colored flesh ranging from cream to yellow-orange to brown to blue-orange [45, 53, 54]. A fisheye's appearance was also considered [46]. When a fish gets spoiled, the areas around its pupils become opaque, which creates the illusion that it is spoiled [48].

The evaluation of fish freshness and quality is the study’s key objective. This is achieved by utilizing deep learning models for extracting features. Training and testing of freshness are performed with images by cross-validation. The CNN classification algorithms InceptionV3, SqueezeNet, VGG16, and VGG19 were all utilized. For each algorithm separately machine learning will be applied for predicting the freshness of the image dataset. The algorithms (k-NN, SVM, ANN, LR, and RF) were selected to be examined on the dataset and discuss the obtained results. This study utilized a combination of deep learning techniques and machine learning methods to classify the freshness of fish. The most successful classification algorithm will be determined. After this stage, a mobile application will be developed based on the most successful models. By using pre-trained deep learning models like SqueezeNet for feature extraction, you can often achieve better results than handcrafted feature extraction techniques. This is because deep learning models have learned to extract relevant features automatically from large amounts of data, which can be difficult to do manually.

Related works

In this section previously implemented methods for fish freshness classification were reviewed. The reviewed areas were models in deep learning, fish freshness, machine learning algorithms, and prediction with a mobile application. The corresponding articles are listed below.

Deep learning models for fish freshness classification

In their study, Mohammadi Lalabadi et al. proposed the use of color degradation in the various color areas of fisheyes and gills as a basis for the segmentation of fisheyes and gills. This study utilized digital images to analyze the color characteristics of fisheyes and gills during a 10-day ice storage period. The left and right eyes and gills were scanned to determine their condition, with fish being considered fresh only if both eyes and gills were in good condition. Artificial neural networks (ANNs) and support vector machines (SVMs) were employed for feature extraction from the color spaces to classify the ice-storage periods. The accuracy of the ANN for extracting eye features was 84%, while the accuracy for extracting gill features was 96% [32].

Prasetyo et al., presented a DSC Bottleneck Multiplier in their first contribution for detecting freshness features of fisheyes. Using Residual Transition, as suggested in the second contribution, it is possible to bridge current feature maps by using Residual Transition, but by using Residual Transition it is possible to skip connecting the current feature maps to the next convolution block. Thirdly, MobileNetV1 Bottleneck with Expansion (MB-BE) was proposed as a method of classifying fisheyes based on their freshness. At last, a 63.21% accuracy percentage was obtained [38, 39].

Fang et al., developed a method for monitoring fish freshness using color-coded labels that utilize red cabbage anthocyanin and a back propagation neural network. The labels change color based on the fish's pH level, indicating its freshness. A BP neural network trained with red cabbage anthocyanin labels was able to predict fish freshness with an accuracy of 92.6%. This technology was integrated into a smartphone app, providing an efficient system for identifying fish freshness in real time and improving food quality control throughout the supply chain [13].

Taheri-Garavand et al., proposed a new and reliable fish freshness detecting method, and to identify fresh carps, a deep convolutional neural network (CNN) was used. The VGG-16 model's multiple convolutional layers were utilized to extract various filters from the 672 images, which were used for four classes. 98.21% classification accuracy was demonstrated by the outcomes [46].

Additional deep learning techniques, such as the use of an electronic nose and chemometric techniques [29], Electronic noses (E-noses), tongues (E-tongues), and colorimeters paired with several machine learning techniques, and a data fusion approach (artificial neural network, support vector regression, Random Forests regression, extreme gradient boosting) [28] used to determine the freshness of fish. Even though they depend on semiconductor gas sensors, and fish quality detectors [42].

An approach to determine freshness based on deep learning was executed by DF Anas et al., for three marine fish species, utilizing a small version of the YOLOv2 algorithm called tiny YOLOv2. The assessment of fish freshness levels was improved using this algorithm, and the levels were classified as good, medium, or poor quality. The experiment achieved an average precision of 72.9%, an average recall of 57.5%, and an average accuracy of 57.5% [8].

Machine learning algorithms for fish freshness classification

Yudhana et al., used k-Nearest Neighbor (k-NN) and Naïve Bayes (NB) classification techniques based on fish-eye images to determine fish freshness. The categorization process employed RGB and GLCM features. The study involved fish collection, image acquisition, classification, ROI identification, preprocessing, feature extraction, dataset separation, and classification. Results showed that the k-NN technique outperformed the NB method, with an average accuracy, precision, recall, specificity, and AUC of 0.97, 0.97, 0.97, and 0.97 [55].

Banwari et al. used 75 fish samples in total to create a framework and demonstrate the relationship between freshness and retrieved characteristics using the training data. A framework is created and constructed using the training data to use the relationship between the amount of freshness and the extracted characteristics. An accuracy of 96.67% is achieved in identifying the proper amount of freshness [5].

To identify seafood freshness, Jarmin et al., proposed a study to compare RGB color parameters with fish freshness meters. Three different species of fish were tested with the Torrymeter sensor in this study, which measures the RGB color of their eyes and gills. Four classes were obtained (Very fresh, Fresh, Average, and Spoiled) from 90 samples of fresh fish [22].

Huang et al. utilized two different methods, computer vision and near-infrared spectroscopy (NIR spectroscopy), for detection. An integrated approach combining both methods was used, and it was found that computer vision technology performed better than NIR spectroscopy for the BP-ANN model. The study demonstrated that computer vision technology outperforms NIR spectroscopy in terms of accuracy when analyzing oil and gas samples. The BP-ANN model achieved a result of 94.17%, with a prediction rate of 90.0%, whereas the accuracy rate for the NIR spectroscopy model was 86.67%, with a prediction rate of 80.0% [19].

In another study which was proposed by Alaimahal et al., fish freshness was detected by using several image processing filters such as the Gaussian filtering method and reshaping. Segmentation was then carried out utilizing the watershed transformation approach. Feature extraction comes next, and then classification is done using a K-NN classifier. The detection rate was recorded as 90% of correctness [3].

Based on retrieved image features, Nguyen et al. suggested new threshold-based with neural network-based fish freshness classification approaches. These properties are determined by the physiological parameters of fisheyes in freshness and stale conditions, nine suggested models were developed from 49 fisheye photos taken from two primary groups of the four Crucian carp fish. The accuracy results for both the training and testing sets were 100% [48].

Prediction with mobile applications

In this study, Iswari et al. developed a system for categorizing the freshness of fish using fish images was created. The classification approach based on the summarization of the colors in fish images was K-Nearest Neighbour (kNN). According to the KNN classification findings, the three fish were classified with an average accuracy of 91.36% [30].

Navotas et al. created an Android application that can identify three of the most commonly consumed fish in the Philippines, namely milkfish, round scad, and tilapia. The application determines its freshness level using RGB values of the eyes and gills, which are rated from 1 (stale) to 5 (fresh). The software was developed using iterative learning with 30 fish samples per species, resulting in 800 images of the eyes and gills of each fish. The test demonstrated that the application correctly identified Milkfish, Round Skad, and Tilapia with accuracies of 90%, 93.33%, and 100%, respectively [34].

Previously proposed studies on detecting the freshness of fish that are mentioned in the study were compiled into a tabular as shown in Table 1. In the table, as noted in the articles the number of images used, categorized into how many classes, the applied methods and the obtained classification accuracy ratio was presented.

Table 1 Summary of related studies on the detection of fish freshness
Full size table

Materials and methods

In this study, deep learning (DL) and machine learning (ML) algorithms were used to classify the freshness of fish from the Freshness of the Fish Eyes (FFE) dataset [38, 39]). Our model starts with implementing the deep learning models first, each model of SqueezeNet, and VGG-19 was processed separately for feature extraction.

In the second step of the model, we apply five different machine learning algorithms for each deep learning model mentioned previously to classify the fish freshness again separately. The workflow of the study is shown in Fig. 1. In the next sections, details are presented. The final step is the mobile application we have developed to predict the fish freshness on time by taking a photo of a fisheye or asking to classify a previously stored image in the mobile gallery. The dataset was used for SqueezeNet first to detect the fish’s images, the images were categorized into three classes of freshness level. After that with the cross-validation, five ML algorithms were tried to compare the prediction outcomes. This implementation was re-implemented again for VGG19.

Fig. 1
figure 1

Flow diagram of the combined model

Full size image

Freshness of fish eyes (FFE) dataset

For this research, a dataset comprising 4390 sRGB images was utilized, which were captured using a Xiaomi Redmi Note 4 smartphone [38, 39]) obtained from the Mendeley data website. The images were separated into three classes containing eight species of fish. The level of freshness was (Highly fresh, Fresh, and Not fresh). One thousand seven hundred sixty-four images were highly fresh fisheyes, 1320 of them were fresh fisheyes, and 1306 images of non-fresh fisheyes [38, 39].

The dataset underwent a modification, wherein the mirroring technique was applied prior to any feature extraction and classification, resulting in a total of 8780 images in the dataset. We augmented the dataset because the classifiers were experiencing misclassifications between the highly fresh and fresh classes.

Table 2 displays the distribution of images in the FFE dataset after modification according to classes. In the highly fresh class, the fish has a shelf life of 1 or 2 days, 3–4 days assigned as fresh, and 5–6 days classified as not fresh. The staleness of the fisheye starts in 2–3 days after catching and transporting it to storage, and as the days pass, the staleness increases [1]. This can be detected in the eye color and the blurriness appearing in the fisheye. A gray color will cover the fisheye when it is totally staled.

Table 2 FFE dataset image distribution by freshness level
Full size table

Feature extraction with deep learning

CNNs are a subset of artificial neural networks that are primarily used for image analysis. Each number represents one pixel in the image, which is seen by the computer as a matrix of numbers. Even when the image is sent as an input to the network, the connection between the pixels (values) should be preserved. To turn the input into the output, many mathematical procedures are stacked on top of each other layer [10]. In CNN, the convolution process is utilized to identify features in the input images. We decided to choose two models for the feature selection process. Namely SqueezeNet, and VGG19. A brief explanation is given for each model.

SqueezeNet: is a 2016 computer vision network introduced by Iandola et al. [20, 25, 35] SqueezeNet is employed for efficient feature extraction, providing a compact yet powerful architecture that effectively captures essential information from input data [7, 33]. In this extraction from each image 1000 features were extracted and then prepared for the ML algorithms.

Feature extraction with deep learning models, such as SqueezeNet, involves using pre-trained neural networks to extract relevant features from input data. SqueezeNet is a popular deep-learning model that is often used for image classification tasks. In this process, the SqueezeNet model is first pre-trained on a large dataset such as ImageNet, which contains millions of images and their associated labels [7].

VGG19: The VGG19 technique for deep learning was developed by the Visual Geometry Group at the University of Oxford as an extension of VGG16. It consists of 16 convolutional layers and 3 fully connected layers, making a total of 41 layers [31, 35]. Using the VGG19 feature extraction technique, 4096 features were obtained from each image and then preprocessed for the machine learning algorithms. VGG19 is utilized for feature extraction, leveraging its deep and intricate architecture to extract rich and detailed features from input data.

SqueezeNet and VGG19 were chosen as feature extractors in this study due to their respective strengths: SqueezeNet for its efficiency and compact architecture, and VGG19 for its deep and intricate structure, both contributing to effective and comprehensive feature extraction from the dataset.

The SqueezeNet model is implemented as a small and fast model [20] for image recognition, originally trained on the ImageNet dataset. The model achieves accuracy comparable to AlexNet but with significantly fewer parameters (50× fewer). The SqueezeNet model is re-implemented, and weights from the author's pre-trained model [20] are utilized. The activations from the pre-softmax layer (flatten10) are employed as an embedding.

The VGG19 model is implemented as a robust 19-layer image recognition model originally trained on the ImageNet dataset [43]. For feature extraction purposes, the activations from the penultimate layer, specifically fc7, are employed as an embedding. This allows the model to capture and represent essential information from the input data [43].

Machine learning algorithms

In this study, five different algorithms were chosen and applied for the classification process. To predict the images as freshness level. This section provides an explanation for each algorithm and information regarding the techniques utilized is elucidated.

k-NN

The K-Nearest Neighbors Classifier (k-NN) categorizes a new sample in a feature area based on the training samples closest to its location. It assigns test data to point to the class with the highest frequency among its k nearest neighbors [2, 36]. The KNN algorithm is that it can work well with both linear and non-linear data, and it can be used for multi-class classification tasks. However, it can be sensitive to the choice of K and the distance metric used for calculating the distances between data points. For k-Nearest Neighbors (kNN), the algorithm is configured with a parameter setting of ten neighbors, Euclidean distance as the metric, and uniform weighting.

SVM

SVM is a robust classifier effective in both linear and non-linear tasks, optimizing a hyperplane to maximize class margin for enhanced accuracy and generalization. The kernel trick facilitates non-linear problem-solving by transforming features into a higher-dimensional space, where support vectors play a crucial role in defining the decision boundary. The regularization parameter (C) balances margin width and misclassification tolerance [2, 36]. SVM parameters include an SVM Cost (C) of 1.00, Regression loss epsilon (ɛ) of 0.10, kernel set to RBF with g(auto), and optimization parameters specifying a numerical tolerance of 0.0001, along with an iteration limit of 1000.

ANN

Artificial Neural Networks (ANNs) are computer networks that are designed to replicate the structure and functionality of the human brain. In a supervised artificial neural network, there are three layers: input layer, hidden layer, and output layer. These networks can be used for a broad range of problems that involve supervised learning techniques. They process information and collaborate to train and then make a prediction [51]. In the Artificial Neural Network (ANN), there are 100 neurons in the hidden layers with ReLU activation, utilizing the Adam solver. The network incorporates regularization with an α value of 0.0001 and is configured for a maximum of 500 iterations. The training process is set to be replicable (yes).

Logistic regression

In binary classification tasks (when the target is categorical), logistic regression is an additional strong supervised machine learning approach. An effective way to understand logistic regression is to think of it as a classification-specific form of linear regression [18]. Logistic Regression utilizes the sigmoid function, which is characterized by an S-shaped curve that transforms any real-valued input to a probability value within the range of 0 and 1. The logistic function’s parameters, namely the weights, and biases, are acquired by employing the maximum likelihood estimation technique during the training stage. In Logistic Regression (LR), Lasso (L1) regularization with a strength parameter (C) set to 1 is employed.

Random forests

In the field of machine learning and statistical analysis, random forests are used for prediction. The building blocks of RF were formed from the tree-based model. There are several forms of random forests, but all trees in the forest depend on the values of an independently selected random vector that is spread uniformly over all trees [6]. Random Forests (RF) is configured with 500 trees, considering 5 attributes at each split, and opting not to split subsets smaller than 5. For classification tasks, each tree “votes” for a class, and the class with the majority of votes is assigned. For regression tasks, the predictions of individual trees are averaged. Random Forests typically use decision trees as their base learners. Decision trees are simple, non-linear models that recursively split the data based on features to create a tree-like structure.

Confusion matrix

Confusion matrixes are used to store information about the actual and predicted classifications made by machine learning classification models. A matrix usually contains the data used to evaluate the performance of such a model. This model was evaluated based on the following criteria: accuracy, precision, recall, and F1-score [53], from the obtained confusion matrix metrics, were calculated. In Table 3 the metrics used in this study are presented with their formula [40, 44, 50].

Table 3 Performance metrics formulas
Full size table

Classification models are evaluated using various performance metrics, and accuracy is one of them. The accuracy is determined by dividing the number of correct predictions by the total number of predictions made. In the case of binary classification, the accuracy can also be measured by calculating true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) [24, 26], as illustrated in Table 3.

To assess the performance of classification models, accuracy alone may not be sufficient. Therefore, precision and recall are two commonly used performance metrics, which complement accuracy. These metrics are derived from true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) that are summarized in a confusion matrix [21]. Precision is a performance metric that calculates the proportion of relevant instances in the retrieved instances. Its purpose is to evaluate the model's accuracy when it makes a positive prediction [52]. A recall is a performance metric that quantifies the proportion of relevant instances retrieved out of the total number of relevant instances. Its purpose is to evaluate how well the model captures positive cases, measuring the model's completeness. Precision and recall are frequently employed in combination to balance the compromise between the quality and quantity of predictions made by a model. A high precision implies that the model is highly specific, but it may overlook some positive cases. Conversely, a high recall implies that the model is highly sensitive, but it may include some false positives. Depending on the problem and the cost of errors, one may favor either higher precision or higher recall. A common approach to combine precision and recall is to use the F1-score, which represents the harmonic mean of both metrics [15, 27].

Table 4 shows the confusion matrix according to the dataset. True Highly Fresh (THF), True Fresh (TF), True Not-Fresh (TNF), False Highly fresh (FHF), False Fresh (FF), and False Not-Fresh (FNF) are the long form for the values mentioned in Table 4. THF–HF is correctly identified as Highly Fresh and it is Highly Fresh actually, TF–F is predicted as Fresh and it is Fresh actually, TNF–NF predicted as Not-Fresh and actually it is Not-Fresh. FHF–F refered to the misclassified classes that were Highly Fresh but predicted as Fresh. FHF–NF referee to the misclassified classes which were Highly Fresh but predicted as Not-Fresh. FF–HF is predicted as Highly Fresh but, it is Fresh actually, FNF–HF is predicted as Highly Fresh, and it is Not-Fresh actually. FNF–F referees to the misclassified classes which were Not-Fresh but predicted as Fresh.

Table 4 FFE confusion matrix based on three classes
Full size table

Cross validation

In cross-validation, the data are split into subsets (called folds) and the training and validation are rotated among these folds. Cross-validation is used for evaluating learning models. When training a model, splitting a dataset into two segments is a conventional practice, a training set and a testing set. A training model gets only examples from its training set, whereas testing instances are simulated by its testing set. Cross-validation and K-fold are interchangeable terms. A K-fold simply describes how many folds you want your dataset to be split into. Using Cross-validation, it is possible to estimate model performance using unseen data that is not used during training [41, 47]. In this study, the dataset is split into ten folds. In Fig. 2, you can see how cross-validation is performed.

Fig. 2
figure 2

FFE dataset cross-validation performance

Full size image

Experimental results

From the dataset used in this study, the confusion matrix for each model is given independently. First, a data mining design was created, by importing the FFE dataset each model of (SqueezeNet, and VGG-19) was implemented. Following the preprocessing phases and feature extraction, the Training-Test data distribution step has been executed. In this step, the 10-layer cross-validation approach is utilized. At this level, classification operations have been completed. The classifiers were listed as k-nearest neighbors, support vector machines, Random Forests, logistic regression, and Artificial neural networks.

VGG19 was implemented for extracting the features after the feature extraction the k-NN was examined to make the classification. According to Fig. 3, the VGG19 model classified 2265 images as highly fresh fish and 1263 misclassified. With the k-NN machine learning method. While was classified correctly for 1814 images as fresh fish and 826 were misclassified. Not fresh fish were correctly classified as 1449 images and 1163 were misclassified. Figure 3 shows the outcomes obtained by utilizing the confusion matrix. While in the SVM method, the highly fresh fish were truly classified as 1869 with 1659 incorrectly classified. The most successful method can be assigned as ANN since the truly classified was 2758 highly fresh fish class and 770 misclassified. The model classified 1795 images correctly as fresh and 845 images by mistake. In addition, the not fresh class was correctly classified as 1852 images with 760 images by mistake. The other results are also presented in Fig. 3 for the other machine learning as LR, and RF.

Fig. 3
figure 3

Confusion matrix for FFE dataset with VGG19

Full size image

SqueezeNet model was applied, and we got the result of a total of 2469 images classified correctly for highly fresh fish and 1059 classifieds incorrectly with the K-NN algorithm. Moreover, 1949 images were classified as fresh and they were fresh, on the other hand, 691 were incorrectly classified. For not fresh class 1531 were predicted correctly according to the actual, besides that 1081 were classified incorrectly. Other tables in Fig. 4 show the outcomes for each SVM, ANN, LR, and RF model

Fig. 4
figure 4

Confusion matrix for FFE dataset with SqueezeNet

Full size image

In the proposed study Orange data mining [9] was used for the deep learning model designs. A computer with Windows 11 Enterprise version 22H2 with Intel® core ™ i7-870H, a CPU @ 2.20 GHz, and 16 GB RAM was used, the memory of the computer was 2TB HDD and 250 GB SSD and with NVIDIA GeForce GTX 1050 Ti graphic card. The learning parameters that were used in this study are shown in (Table 5). Orange data mining desktop software was used for creating the classification model. For each ML algorithm different parameters were used, all the values are presented in the table. The machine learning parameters utilized in the research were selected based on the default parameters available in the Orange Data Mining tool.

Table 5 Machine learning parameters in Orange Data Mining Software
Full size table

As mentioned in Table 6, VGG19 applied, with the ANN model achieved the highest success ratio compared to the other model. The precision obtained in ANN was also higher than others as well as with the recall and F1 score. The ANN gave a higher classification ratio. While the results achieved by other ML algorithms (k-NN, SVM, RF, ANN, and LR) were lower. The results indicate that VGG19 performs optimally when applied to the FFE dataset using an ANN.

Table 6 Performance metric algorithms for FFE dataset with VGG19
Full size table

An overview of the results obtained when using deep learning techniques to train the convolutional neural network architectures VGG-19 (Table 6), and SqueezeNet (Table 7). Additionally, to see which performance gives the highest accurate percentage on the FFE dataset, the accuracy results have been presented in Table 8 in percentage value.

Table 7 Performance metric algorithms for FFE dataset with SqueezeNet
Full size table
Table 8 The classification accuracy for the models
Full size table

The ability to automatically determine fish freshness using images has several real-world implications and applications. First, it can improve quality control throughout the supply chain, which has significant economic benefits. In addition to preventing the sale of spoiled fish, suppliers can reduce losses caused by customer complaints and improve overall consumer satisfaction by quickly identifying fish freshness. Disposing of spoiled fish can also reduce food waste and minimize environmental impacts. By preventing contaminated or spoiled fish from reaching the consumer, this technology can improve food safety.

Mobile application

Mobile apps can serve a variety of purposes, we suggest that having an application for this purpose will help fish consumers find fresh fish and be sure about the healthiness of the fish. Within the study context, a mobile application is developed to detect the freshness of fish. This application is meant to detect the freshness of fish on fisheye. If a consumer wants to verify the freshness of fish at any time and from any place, he/she may use his/her smartphone and take a photo of the fish, and the application will determine whether the level of freshness (highly fresh, fresh, or not fresh). This program incorporated artificial intelligence to predict based on deep learning model algorithms and machine learning algorithms. The application is currently only available for Android devices. Here are the interfaces for the mobile program:

In Fig. 5 the first screen is shown while opening the application which is named as splash screen, this screen is followed by a screen after 3 s which is the main page of the application. For starting the fish freshness prediction, the user should tap the “Is it fresh” button. Then it displays the prediction ways, either the user will take a photo of the fish’s eye and see the prediction result or press the “Select Image” button to upload an image from the mobile gallery. After uploading the image, the result of the freshness will be shown to the user.

Fig. 5
figure 5

Screen shots for the mobile application

Full size image

We conducted thorough assessments of the mobile application's functionality to guarantee its strong performance. The outcomes of these examinations are illustrated in Fig. 6.

Fig. 6
figure 6

Fish freshness detector application test images

Full size image

Conclusion

From the prepared study, a fish freshness dataset was used, and the dataset was classified into three classes of fisheyes. Through the achieved results it is obtained that the freshness of the fish couldn’t be determined only by detecting the fisheye with this dataset images. The results were not particularly outstanding since the image's resolution quality was not very precise and because we were unable to locate our dataset or other suitable images. A total of 8780 images of 3 based classes were used in the proposed model. These images were of three classes of freshness highly fresh, fresh, and not fresh fish. A mixed model of deep learning and machine learning models was used to classify the freshness of the fish. The machine learning models were operated as (k-NN, SVM, LR, RF, and ANN) for each algorithm of (VGG19, and SqueezeNet) and the highest outcomes in ANN were 77.3% and 72.9% for each deep learning method. Prasetyo et al. used the same FFE dataset and implemented MobileNetV1 combined with DSC-BE, RT, and MB-BE, and 63.21% accuracy were obtained [38, 39]. The comparison point of the study is using the same dataset could achieve a better success ratio while using different methods other than MobileNetV1, which was used for this dataset. From the outcomes, we can conclude that using the VGG19 model for feature selection and ANN for classification can give the highest success percentage of 77.3% for the FFE dataset.

There are many proposed studies on the detection of fish and classification of its freshness either using chemical or biological methods or through diverse kinds of sensors. Moreover, studies about detecting freshness through using deep learning algorithms were present.