Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Classification of foodborne bacteria using hyperspectral microscope imaging technology coupled with convolutional neural networks


Foodborne pathogens have become ongoing threats in the food industry, whereas their rapid detection and classification at an early stage are still challenging. To address early and rapid detection, hyperspectral microscope imaging (HMI) technology combined with convolutional neural networks (CNN) was proposed to classify foodborne bacterial species at the cellular level. HMI technology can simultaneously obtain both spatial and spectral information of different live bacterial cells, while two CNN frameworks, U-Net and one-dimensional CNN (1D-CNN), were employed to accelerate the data analysis process. U-Net was used for automating cellular regions of interest (ROI) segmentation, which generated accurate cell-ROI masks in a shorter timeframe than the conventional Otsu or Watershed methods. The 1D-CNN was employed for classifying the spectral profiles extracted from cell-ROI and resulted in a higher accuracy (90%) than k-nearest neighbor (81%) and support vector machine (81%). Overall, the CNN-assisted HMI technology showed potential for foodborne bacteria detection.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5


  1. Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185. https://doi.org/10.1080/00031305.1992.10475879

  2. Bertani FR, Botti E, Ferrari L, Mussi V, Costanzo A, D’Alessandro M, Cilloco F, Selci S (2016) Label-free and non-invasive discrimination of HaCaT and melanoma cells in a co-culture model by hyperspectral confocal reflectance microscopy. J Biophotonics 9(6):619–625. https://doi.org/10.1002/jbio.201500122

  3. CDC (Centers for Disease Control and Prevention) (2019), Foodborne illnesses and germs. https://www.cdc.gov/foodsafety/foodborne-germs.html [Accessed 4 August 2019]

  4. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1007/BF00994018

  5. Eady M, Setia G, Park B (2019) Detection of Salmonella from chicken rinsate with visible/near-infrared hyperspectral microscope imaging compared against RT-PCR. Talanta 195:313–319. https://doi.org/10.1016/j.talanta.2018.11.071

  6. Eady MB, Park B, Yoon S-C, Haidekker MA, Lawrence KC (2018) Methods for hyperspectral microscope calibration and spectra normalization from images of bacteria cells. Trans ASABE 61(2):438–448. https://doi.org/10.13031/trans.12222

  7. Falk T, Mai D, Bensch R, Çiçek Ö, Abdulkadir A, Marrakchi Y, Böhm A, Deubner J, Jäckel Z, Seiwald K (2019) U-Net: deep learning for cell counting, detection, and morphometry. Nat Methods 16(1):67. https://doi.org/10.1038/s41592-018-0261-2

  8. Gracias KS, McKillip JL (2004) A review of conventional detection and enumeration methods for pathogenic bacteria in food. Can J Microbiol 50(11):883–890. https://doi.org/10.1139/w04-080

  9. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778. https://doi.org/10.1109/cvpr.2016.90

  10. Huang D-Y, Wang C-H (2009) Optimal multi-level thresholding using a two-stage Otsu optimization approach. Pattern Recogn Lett 30(3):275–284. https://doi.org/10.1016/j.patrec.2008.10.003

  11. Kammies T-L, Manley M, Gouws PA, Williams PJ (2016) Differentiation of foodborne bacteria using NIR hyperspectral imaging and multivariate data analysis. Appl Microbiol Biotechnol 100(21):9305–9320. https://doi.org/10.1007/s00253-016-7801-4

  12. Kang R, Park B, Chen K (2020) Identifying non-O157 Shiga toxin-producing Escherichia coli (STEC) using deep learning methods with hyperspectral microscope images. Spectrochim Acta A 224:117386. https://doi.org/10.1016/j.saa.2019.117386

  13. Law JW-F, Ab Mutalib N-S, Chan K-G, Lee L-H (2015) Rapid methods for the detection of foodborne bacterial pathogens: principles, applications, advantages and limitations. Front Microbiol 5:770. https://doi.org/10.3389/fmicb.2014.00770

  14. Leavesley SJ, Annamdevula N, Boni J, Stocker S, Grant K, Troyanovsky B, Rich TC, Alvarez DF (2012) Hyperspectral imaging microscopy for identification and quantitative analysis of fluorescently-labeled cells in highly autofluorescent tissue. J Biophotonics 5(1):67–84. https://doi.org/10.1002/jbio.201100066

  15. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436. https://doi.org/10.1038/nature14539

  16. Legland D, Arganda-Carreras I, Andrey PJB (2016) MorphoLibJ: integrated library and plugins for mathematical morphology with ImageJ. Bioinfomatics 32(22):3532–3534. https://doi.org/10.1093/bioinformatics/btw413

  17. Lewis, M., Yarats, D., Dauphin, Y. N., Parikh, D., & Batra, D. J. (2017) Deal or no deal? end-to-end learning for negotiation dialogues. https://arxiv.xilesou.top/abs/1706.05125

  18. Livne M, Rieger J, Aydin OU, Taha AA, Akay EM, Kossen T, Sobesky J, Kelleher JD, Hildebrand K, Frey D, Madai VI (2019) A U-Net deep learning framework for high performance vessel segmentation in patients with cerebrovascular disease. Front Neurosci 13(97). https://doi.org/10.3389/fnins.2019.00097

  19. Maaten LVD, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(Nov):2579–2605 http://www.jmlr.org/papers/v9/vandermaaten08a.html

  20. Naumann D, Helm D, Labischinski H (1991) Microbiological characterizations by FT-IR spectroscopy. Nature 351(6321):81–82. https://doi.org/10.1038/351081a0

  21. Norman B, Pedoia V, Majumdar S (2018) Use of 2D U-Net convolutional neural networks for automated cartilage and meniscus segmentation of knee MR imaging data to determine relaxometry and morphometry. Radiology 288(1):177–185. https://doi.org/10.1148/radiol.2018172322

  22. Otsu N (1979) A threshold selection method from gray-level histograms. In: IEEE transactions on systems, man, and cybernetics 9(1):62–66. https://doi.org/10.1109/TSMC.1979.4310076

  23. Park B, Yoon S, Lee S, Sundaram J, Windham W, Hinton A Jr, Lawrence K (2012) Acousto-optic tunable filter hyperspectral microscope imaging method for characterizing spectra from foodborne pathogens. Trans ASABE 55(5):1997–2006. https://doi.org/10.13031/2013.42345

  24. Park B, Seo Y, Yoon S-C, Hinton JA, Windham WR, Lawrence KC (2015) Hyperspectral microscope imaging methods to classify Gram-positive and Gram-negative foodborne pathogenic bacteria. Trans ASABE 58(1):5–16. https://doi.org/10.13031/trans.58.10832

  25. Qu K, Guo F, Liu X, Lin Y, Zou Q (2019) Application of Machine Learning in Microbiology. Front Microbiol 10(827). https://doi.org/10.3389/fmicb.2019.00827

  26. Rahman MA, Wang Y (2016) Optimizing intersection-over-union in deep neural networks for image segmentation. In: International symposium on visual computing. Springer, pp 234–244. https://doi.org/10.1007/978-3-319-50835-1_22

  27. Reza AW, Eswaran C, Dimyati K (2011) Diagnosis of diabetic retinopathy: automatic extraction of optic disc and exudates from retinal images using marker-controlled watershed transformation. J Med Syst 35(6):1491–1501. https://doi.org/10.1007/s10916-009-9426-y

  28. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241. https://doi.org/10.1007/978-3-319-24574-4_28

  29. Santurkar S, Tsipras D, Ilyas A, Madry A (2018) How does batch normalization help optimization? In: Advances in Neural Information Processing Systems, pp 2483–2493 https://arxiv.org/abs/1805.11604

  30. Scallan E, Hoekstra RM, Angulo FJ, Tauxe RV, Widdowson M-A, Roy SL, Jones JL, Griffin PM (2011) Foodborne illness acquired in the United States—major pathogens. Emerg Infect Dis 17(1):7. https://doi.org/10.3201/eid1701.09-1101p1

  31. Valderrama WB, Dudley EG, Doores S, Cutter CN (2016) Commercially available rapid methods for detection of selected food-borne pathogens. Crit Rev Food Sci Nutr 56(9):1519–1531. https://doi.org/10.1080/10408398.2013.775567

  32. Williams PJ, Geladi P, Britz TJ, Manley M (2012) Investigation of fungal development in maize kernels using NIR hyperspectral imaging and multivariate data analysis. J Cereal Sci 55(3):272–278. https://doi.org/10.1016/j.jcs.2011.12.003

  33. Yoon S-C, Lawrence KC, Park B (2015) Automatic counting and classification of bacterial colonies using hyperspectral imaging. Food Bioproc Technol 8(10):2047–2065. https://doi.org/10.1007/s11947-015-1555-3

Download references


The authors would like to thank Dr. Nasreen Bano of the Quality and Safety Assessment Research Unit in Athens, Georgia, for assistance in this research.


Mr. Rui Kang received financial support from the China Scholarship Council for his study in the USDA, ARS laboratory in Athens, Georgia.

Author information

Correspondence to Bosoon Park.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material


(PDF 684 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kang, R., Park, B., Eady, M. et al. Classification of foodborne bacteria using hyperspectral microscope imaging technology coupled with convolutional neural networks. Appl Microbiol Biotechnol (2020). https://doi.org/10.1007/s00253-020-10387-4

Download citation


  • Hyperspectral microscopy
  • Foodborne pathogen
  • Rapid classification
  • Food safety
  • Machine learning
  • Convolutional neural network