Skip to main content
SpringerLink
Log in

Computer Vision Detection of Salmon Muscle Gaping Using Convolutional Neural Network Features

  • Published:
Food Analytical Methods Aims and scope Submit manuscript

Abstract

Salmon muscle gaping will lead to the irregular voids or undesirable lace-like appearance in the final product. This study was carried out to develop an automatic imaging analysis method for rapid, accurate, and non-invasive detection of gaping blemishes on salmon carcasses. Salmon fillets could be classified as wholesome or defective samples based on the number of candidate gaping regions in the preliminary step applying local adaptive thresholding. Supervised classification results were compared between using histograms of oriented gradients (HOG) and convolutional neural network (CNN) feature extractors. It was shown that CNN features outperformed HOG features with correct classification rates (CCRs) of 0.927 and 0.916 for cross validation and test data set, respectively. Relieff was then applied to select important feature attributes by reducing the 4096-dimensional to 239-dimensional vector. Simplified CNN model also yielded good classification performance with CCR of 0.925 for cross validation. Therefore, CNNs were used to extract features from candidate regions and then reduced features to the 239-dimensional vector. The resultant vector was fed to the simplified CNN model to make a final decision. The prediction maps for visualizing the classification result on salmon fillet were subsequently generated. The overall results confirmed that this proposed method is effective and suitable for the muscle gaping detection. Future work will be focused on applying this method in packing plants where fish fillets are progressing rapidly, and promising results will allow the identification of critical points in the supply chain that impact upon product quality.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Ahmad I, Jeenanunta C (2015) Application of support vector classification algorithms for the prediction of quality level of frozen shrimps (Litopenaeus vannamei) suitable for sensor-based time-temperature monitoring. Food Bioprocess Technol 8(1):134–147

  • Al-Bayati M, El-Zaart A (2013) Automatic thresholding techniques for optical images. Signal Image Processing 4(3):1

    Google Scholar 

  • Andersen CM, Wold JP (2003) Fluorescence of muscle and connective tissue from cod and salmon. J Agric Food Chem 51(2):470–476

    Article  CAS  Google Scholar 

  • Andersen B, Steinsholt K, Stroemsnes A, Thomassen M (1994) Fillet gaping in farmed Atlantic salmon (Salmo salar). Nor J Agric Sci

  • Ariana DP, Lu R, Guyer DE (2006) Near-infrared hyperspectral reflectance imaging for detection of bruises on pickling cucumbers. Computers and Electronics in Agriculture 53(1):60–70

  • Balaban MO, Ünal Şengör GF, Soriano MG, Ruiz EG (2011) Quantification of gaping, bruising, and blood spots in salmon fillets using image analysis. J Food Sci 76(3):E291–E2E7

    Article  CAS  Google Scholar 

  • Borderías AJ, Sánchez-Alonso I (2011) First processing steps and the quality of wild and farmed fish. J Food Sci 76(1):R1–R5

    Article  Google Scholar 

  • Borderías AJ, Gómez-Guillén MC, Hurtado O, Montero P (1999) Use of image analysis to determine fat and connective tissue in salmon muscle. Eur Food Res Technol 209(2):104–107

    Article  Google Scholar 

  • Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. Proceedings of the Fifth Annual Workshop on Computational Learning Theory. ACM, p 144–52

  • Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2014) Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:1412.7062

  • Cheng J-H, Sun D-W, Pu H (2016) Combining the genetic algorithm and successive projection algorithm for the selection of feature wavelengths to evaluate exudative characteristics in frozenthawed fish muscle. Food Chem 197:855–863 Part: A

  • Cheng L, Sun D-W, Zhu Z, Zhang Z (2017) Emerging techniques for assisting and accelerating food freezing processes: A review of recent research progresses. Critical Rev Food Sci Nutr 57(4):769–781

  • Chow C, Kaneko T (1972) Automatic boundary detection of the left ventricle from cineangiograms. Comput Biomed Res 5(4):388–410

    Article  CAS  Google Scholar 

  • Cubero S, Lee WS, Aleixos N, Albert F, Blasco J (2016) Automated systems based on machine vision for inspecting citrus fruits from the field to postharvest—a review. Food Bioprocess Technol 9(10):1623–1639

  • Cui Z-W, Sun L-J, Chen W, Sun D-W (2008) Preparation of dry honey by microwave-vacuum drying. J Food Eng 84(4):582–590

  • Du CJ, Sun D-W (2005) Pizza sauce spread classification using colour vision and support vector machines. J Food Eng 66(2):137–145

  • Du C-J, Sun D-W (2009) Retrospective shading correction of confocal laser scanning microscopy beef images for three-dimensional visualization. Food Bioprocess Technol 2(2):167–176

  • Guttormsen E, Toldnes B, Bondø M, Eilertsen A, Gravdahl JT, Mathiassen JR (2016) A machine vision system for robust sorting of herring fractions. Food Bioprocess Technol 9(11):1893–1900

  • He K, Zhang X, Ren S, Sun J. 2015. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. Proceedings of the IEEE International Conference on Computer Vision, p 1026–1034

  • Indergård E, Tolstorebrov I, Larsen H, Eikevik T (2014) The influence of long-term storage, temperature and type of packaging materials on the quality characteristics of frozen farmed Atlantic Salmon (Salmo salar). Int J Refrig 41:27–36

    Article  Google Scholar 

  • Jackman P, Sun D-W, Allen P (2009) Automatic segmentation of beef longissimus dorsi muscle and marbling by an adaptable algorithm. Meat Sci 83(2):187–194

  • Jackman P, Sun D-W, Allen P (2011) Recent advances in the use of computer vision technology in the quality assessment of fresh meats. Trends Food Sci Technol 22(4):185–197

  • Kiani H, Zhang Z, Delgado A, Sun D-W (2011) Ultrasound assisted nucleation of some liquid and solid model foods during freezing. Food Res Int 44(9):2915–2921

  • Kira K, Rendell LA (1992) A practical approach to feature selection. Proceedings of the Ninth International Workshop on Machine Learning p249-56

  • Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Proces Syst 1097–105

  • Lavety J, Afolabi O, Love R (1988) The connective tissues of fish. Int J Food Sci Technol 23(1):23–30

    Article  Google Scholar 

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444

    Article  CAS  Google Scholar 

  • Lee J-S (1983) Digital image smoothing and the sigma filter. Comput Vision Graph Image Process 24(2):255–269

    Article  Google Scholar 

  • Leon K, Mery D, Pedreschi F, Leon J (2006) Color measurement in L∗ a∗ b∗ units from RGB digital images. Food Research International 39(10):1084–1091

  • Lin P, Chen Y, He Y (2012) Identification of broken rice kernels using image analysis techniques combined with velocity representation method. Food Bioprocess Technol 5(2):796–802

  • Liu D, Sun D-W, Zeng X-A (2014) Recent advances in wavelength selection techniques for hyperspectral image processing in the food industry. Food Bioprocess Technol 7(2):307–323

  • Lunadei L, Ruiz-Garcia L, Bodria L, Guidetti R (2012) Automatic identification of defects on eggshell through a multispectral vision system. Food Bioprocess Technol 5(8):3042–3050

  • Ma J, Pu H, Sun D-W, Gao W, Qu J-H, Kai-Yue M (2015) Application of Vis-NIR hyperspectral imaging in classification between fresh and frozen-thawed pork Longissimus Dorsi muscles. Int J Refrig-Rev Int Froid 50:10–18

  • Matiacevich SB, Mery D, Pedreschi F ( 2012) Prediction of mechanical properties of corn and tortilla chips by using computer vision. Food Bioprocess Technol 5(5):2025–30

  • McDonald K, Sun D-W, Kenny T (2000) Comparison of the quality of cooked beef products cooled by vacuum cooling and by conventional cooling. Lebensm Wiss Technol-Food Sci Technol 33(1):21–29

  • Michie I (2001) Causes of downgrading in the salmon farming industry. Farmed fish quality, p 129–136

  • Misimi E (2007) Computer vision for quality grading in fish processing. Fakultet for informasjonsteknologi, matematikk og elektroteknikk

  • Misimi E, Erikson U, Digre H, Skavhaug A, Mathiassen J (2008) Computer vision-based evaluation of pre-and postrigor changes in size and shape of Atlantic cod (Gadus morhua) and Atlantic salmon (Salmo salar) fillets during rigor mortis and ice storage: effects of perimortem handling stress. J Food Sci 73(2):E57–E68

    Article  CAS  Google Scholar 

  • Moscetti R, Saeys W, Keresztes JC, Goodarzi M, Cecchini M, Danilo M, Massantini R (2015) Hazelnut quality sorting using high dynamic range short-wave infrared hyperspectral imaging. Food Bioprocess Technol 8(7):1593–1604

  • Otsu N (1975) A threshold selection method from gray-level histograms. Automatica 11(285–296):23–27

    Google Scholar 

  • Pittman K, Grigory V, Brandebourg T (2013) Bridging the gap to sustainable salmon farming: overcoming the gaping problem. J Fish Livest Production 2013

  • Pu H, Sun D-W, Ma J, Cheng J-H (2015) Classification of fresh and frozen-thawed pork muscles using visible and near infrared hyperspectral imaging and textural analysis. Meat Sci 99:81–88

  • Pu Y-Y, Sun D-W (2016) Prediction of moisture content uniformity of microwave-vacuum dried mangoes as affected by different shapes using NIR hyperspectral imaging. Innov Food Sci Emerg Technol 33:348–356

  • Quevedo R, Aguilera J (2010) Computer vision and stereoscopy for estimating firmness in the salmon (Salmon salar) fillets. Food Bioprocess Technol 3(4):561–567

    Article  Google Scholar 

  • Quevedo R, Aguilera J, Pedreschi F (2010) Color of salmon fillets by computer vision and sensory panel. Food Bioprocess Technol 3(5):637–643

    Article  Google Scholar 

  • Saenz C, Hernandez B, Beriain M, Lizaso G (2005) Meat color in retail displays with fluorescent illumination. Color Research & Application 30(4):304–11

  • Schölkopf B, Tsuda K, Vert J-P (2004) Kernel methods in computational biology. MIT press

  • Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2013) Overfeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229

  • Singh OI, Sinam T, James O, Singh TR (2012) Local contrast and mean thresholding in image binarization. Int J Comput Appl 51(6)

  • Sun D-W (1997) Solar powered combined ejector vapour compression cycle for air conditioning and refrigeration. Energy Convers Manage 38(51):479–491

  • Sun D-W, Brosnan T (1999) Extension of the vase life of cut daffodil flowers by rapid vacuum cooling. Int J Refrig Rev Int Froid 22(6):472–478

  • Sun D-W, Brosnan T (2003) Pizza quality evaluation using computer vision - Part 2 - Pizza topping analysis. J Food Eng 57(1):91–95

  • Sun D-W, Hu ZH (2003) CFD simulation of coupled heat and mass transfer through porous foods during vacuum cooling process. Int J Refrigeration-Revue Internationale du Froid 26(1):19–27

  • Sun D-W, Wang LJ (2000) Heat transfer characteristics of cooked meats using different cooling methods. Int J Refrig-Rev Int du Froid 23(7):508–516

  • Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p 1–9

  • Teye E, Huang X, Dai H, Chen Q (2013) Rapid differentiation of Ghana cocoa beans by FT-NIR spectroscopy coupled with multivariate classification. Spectrochim Acta A Mol Biomol Spectrosc 114:183–189

    Article  CAS  Google Scholar 

  • Tong S, Koller D (2001) Support vector machine active learning with applications to text classification. J Mach Learn Res 2(Nov):45–66

    Google Scholar 

  • Valous NA, Mendoza F, Sun D-W, Allen P (2009) Colour calibration of a laboratory computer vision system for quality evaluation of pre-sliced hams. Meat Sci 81(1):132–141

    Article  Google Scholar 

  • Wang LJ, Sun D-W (2004) Effect of operating conditions of a vacuum cooler on cooling performance for large cooked meat joints. J Food Eng 61(2):231–240

  • Wang LJ, Sun D-W Modelling vacuum cooling process of cooked meat-part 1: analysis of vacuum cooling system. Int J (2002) Refrig-Rev Int du Froid 25(7):854–861

  • Wang LJ, Sun D-W Modelling vacuum cooling process of cooked meat-part 2: mass and heat transfer of cooked meat under vacuum pressure. Int J (2002) Refrig-Rev Int du Froid 25(7):862–871

  • Xie A, Sun D-W, Xu Z, Zhu Z (2015) Rapid detection of frozen pork quality without thawing by Vis-NIR hyperspectral imaging technique. Talanta 139:208–215

  • Xie A, Sun D-W, Zhu Z, Pu H (2016) Nondestructive Measurements of Freezing Parameters of Frozen Porcine Meat by NIR Hyperspectral Imaging. Food Bioprocess Technol 9(9):1444–1454

  • Xu J-L, Sun D-W (2017) Identification of freezer burn on frozen salmon surface using hyperspectral imaging and computer vision combined with machine learning algorithm. Int J Refrig 74:149–162

    Google Scholar 

  • Xu J-L, Riccioli C, Sun D-W (2017) Comparison of hyperspectral imaging and computer vision for automatic differentiation of organically and conventionally farmed salmon. J Food Eng 196:170–182

    Article  CAS  Google Scholar 

  • Yang Q, Sun D-W, Cheng W (2017) Development of simplified models for nondestructive hyperspectral imaging monitoring of TVB-N contents in cured meat during drying process. J Food Eng 192:53–60

  • Zheng LY, Sun D-W (2004) Vacuum cooling for the food industry - a review of recent research advances. Trends Food Sci Technol 15(12):555–568

Download references

Acknowledgements

The authors would like to acknowledge the UCD-CSC Scholarship Scheme supported by University College Dublin (UCD) and China Scholarship Council (CSC) for financial support of this study.

Author information

Authors and Affiliations

  1. Food Refrigeration and Computerized Food Technology (FRCFT), School of Biosystems and Food Engineering, National University of Ireland, Agriculture and Food Science Centre, University College Dublin, Belfield, Dublin 4, Ireland

    Jun-Li Xu & Da-Wen Sun

Authors
  1. Jun-Li Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Da-Wen Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Da-Wen Sun.

Ethics declarations

Conflict of Interest

Junli Xu declares that she has no conflict of interest. Da-Wen Sun declares that he has no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants performed by any of the authors.

Informed Consent

Not applicable.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, JL., Sun, DW. Computer Vision Detection of Salmon Muscle Gaping Using Convolutional Neural Network Features. Food Anal. Methods 11, 34–47 (2018). https://doi.org/10.1007/s12161-017-0957-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12161-017-0957-4

Keywords

Search

Navigation