Skip to main content

Allergen30: Detecting Food Items with Possible Allergens Using Deep Learning-Based Computer Vision

Food Analytical Methods volume 15, pages 3045–3078 (2022)Cite this article

Abstract

Food allergies impose a significant health concern on the community. A small number of certain food items can cause an allergic reaction within the human body. The symptoms can range from mild hives or itchiness to life-threatening anaphylaxis. In most cases, such reactions can be prevented by simply being aware of the allergen-based food items and avoiding the consumption of the same. We are among the first research attempts to train a deep learning–based object detection model to detect the presence of such food items within an image. We introduce our Allergen30 dataset, which hosts more than 6,000 annotated images of 30 commonly used food items that can trigger an adverse reaction. We report the comparison of multiple variants of the current state-of-art object detection methods, YOLOv5 and YOLOR. Furthermore, we qualitatively analyzed the performance of these methods by surveying the predictions made on the test dataset images.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22

Data Availability

All the data used in the manuscript are available in the tables and figures.

Code Availability

Not applicable.

References

  • Abrams EM, Sicherer SH (2016) Diagnosis and management of food allergy. CMAJ 188:1087–1093. https://doi.org/10.1503/cmaj.160124

    Article  PubMed  PubMed Central  Google Scholar 

  • Al-Sarayreh M, M. Reis M, Qi Yan W, Klette R (2018) Detection of red-meat adulteration by deep spectral–spatial features in hyperspectral images. J. Imaging 4

  • Arora M, Mangipudi P, Dutta MK (2021) Deep learning neural networks for acrylamide identification in potato chips using transfer learning approach. J Ambient Intell Humaniz Comput. https://doi.org/10.1007/s12652-020-02867-2

    Article  Google Scholar 

  • Arslan B, Memiş S, Sönmez EB, Batur OZ (2022) Fine-grained food classification methods on the UEC FOOD-100 Database. IEEE Trans Artif Intell 3:238–243. https://doi.org/10.1109/TAI.2021.3108126

    Article  Google Scholar 

  • Azizah LM, Umayah SF, Riyadi S, et al (2017) Deep learning implementation using convolutional neural network in mangosteen surface defect detection. In: 2017 7th IEEE International Conference on Control System, Computing and Engineering (ICCSCE). pp 242–246

  • B.S. M, Shinde S, Bhavsar K, et al (2018) Non-destructive method to detect artificially ripened banana using hyperspectral sensing and RGB imaging. In: Proc.SPIE

  • Baenkler H-W (2008) Salicylate intolerance: pathophysiology, clinical spectrum, diagnosis and treatment. Dtsch Arztebl Int 105:137–142. https://doi.org/10.3238/arztebl.2008.0137

    Article  PubMed  PubMed Central  Google Scholar 

  • Bisgin H, Bera T, Ding H et al (2018) Comparing SVM and ANN based machine learning methods for species identification of food contaminating beetles. Sci Rep 8:6532. https://doi.org/10.1038/s41598-018-24926-7

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Bochkovskiy A, Wang C-Y, Liao H (2020) YOLOv4: optimal speed and accuracy of object detection

  • Bossard L, Guillaumin M, Van Gool L (2014) Food-101 – mining discriminative components with random forests BT - computer vision – ECCV 2014. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds). Springer International Publishing, Cham, pp 446–461

  • Bousquet J, Björkstén B, Bruijnzeel-Koomen CA et al (1998) Scientific criteria and the selection of allergenic foods for product labelling. Allergy 53:3–21. https://doi.org/10.1111/j.1398-9995.1998.tb04987.x

    Article  CAS  PubMed  Google Scholar 

  • Boye JI (2012) Food allergies in developing and emerging economies: need for comprehensive data on prevalence rates. Clin Transl Allergy 2:25. https://doi.org/10.1186/2045-7022-2-25

    Article  PubMed  PubMed Central  Google Scholar 

  • Bush RK, Hefle SL (1996) Food allergens. Crit Rev Food Sci Nutr 36:119–163. https://doi.org/10.1080/10408399609527762

    Article  Google Scholar 

  • Chen C-H, Karvela M, Sohbati M et al (2018) PERSON-Personalized Expert Recommendation System for Optimized Nutrition. IEEE Trans Biomed Circuits Syst 12:151–160. https://doi.org/10.1109/TBCAS.2017.2760504

    Article  PubMed  Google Scholar 

  • Chen J, Ngo C (2016) Deep-based ingredient recognition for cooking recipe retrieval. In: Proceedings of the 24th ACM International Conference on Multimedia. Association for Computing Machinery, New York, NY, USA, pp 32–41

  • Chen X, Zhu Y, Zhou H, et al (2017) ChineseFoodNet: a large-scale image dataset for Chinese food recognition

  • Ciocca G, Napoletano P, Schettini R (2018) CNN-based features for retrieval and classification of food images. Comput vis Image Underst 176–177:70–77. https://doi.org/10.1016/j.cviu.2018.09.001

    Article  Google Scholar 

  • Ciocca G, Napoletano P, Schettini R (2017a) Food recognition: a new dataset, experiments, and results. IEEE J Biomed Heal Informatics 21:588–598. https://doi.org/10.1109/JBHI.2016.2636441

    Article  Google Scholar 

  • Ciocca G, Napoletano P, Schettini R (2017b) Learning CNN-based features for retrieval of food images BT - new trends in image analysis and processing – ICIAP 2017b. In: Battiato S, Farinella GM, Leo M, Gallo G (eds). Springer International Publishing, Cham, pp 426–434

  • Comas-Basté O, Sánchez-Pérez S, Veciana-Nogués MT, et al (2020) Histamine intolerance: the current state of the art. Biomolecules 10: https://doi.org/10.3390/biom10081181

  • da Costa AZ, Figueroa HEH, Fracarolli JA (2020) Computer vision based detection of external defects on tomatoes using deep learning. Biosyst Eng 190:131–144. https://doi.org/10.1016/j.biosystemseng.2019.12.003

    Article  Google Scholar 

  • Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. IEEE Comput Soc Conf Comput vis Pattern Recognit (CVP’05) 1:886–893. https://doi.org/10.1109/CVPR.2005.177

    Article  Google Scholar 

  • Donadello I, Dragoni M (2019) Ontology-driven food category classification in images BT - image analysis and processing – ICIAP 2019. In: Ricci E, Rota Bulò S, Snoek C, et al. (eds). Springer International Publishing, Cham, pp 607–617

  • Ege T, YANAI K, (2018) Image-based food calorie estimation using recipe information. IEICE Trans Inf Syst E01.D:1333–1341. https://doi.org/10.1587/transinf.2017MVP0027

    Article  Google Scholar 

  • Fan S, Li J, Zhang Y et al (2020) On line detection of defective apples using computer vision system combined with deep learning methods. J Food Eng 286:110102. https://doi.org/10.1016/j.jfoodeng.2020.110102

    Article  Google Scholar 

  • Farinella G, Allegra D, Stanco F (2014) A benchmark dataset to study the representation of food images

  • Farinella GM, Allegra D, Moltisanti M et al (2016) Retrieval and classification of food images. Comput Biol Med 77:23–39. https://doi.org/10.1016/j.compbiomed.2016.07.006

    Article  PubMed  Google Scholar 

  • Fu Z, Chen D, Li H (2017a) ChinFood1000: a large benchmark dataset for Chinese food recognition. In: Bevilacqua V, Premaratne P, Gupta P (eds) Intelligent Computing Theories and Application. ICIC 2017a. Lecture Notes in Computer Science. Springer

  • Fu Z, Chen D, Li H (2017b) ChinFood1000: a large benchmark dataset for Chinese food recognition BT - intelligent computing theories and application. In: Bevilacqua V, Premaratne P, Gupta P (eds) Huang D-S. Springer International Publishing, Cham, pp 273–281

    Google Scholar 

  • Gallo I, Ria G, Landro N, Grassa RL (2020) Image and text fusion for UPMC Food-101 using BERT and CNNs. In: 2020 35th International Conference on Image and Vision Computing New Zealand (IVCNZ). pp 1–6

  • Gc S, Saidul MdB, Zhang Y et al (2021) Using deep learning neural network in artificial intelligence technology to classify beef cuts. Front Sensors 2:5. https://doi.org/10.3389/fsens.2021.654357

    Article  Google Scholar 

  • Geng Z, Shang D, Han Y, Zhong Y (2019) Early warning modeling and analysis based on a deep radial basis function neural network integrating an analytic hierarchy process: a case study for food safety. Food Control 96:329–342. https://doi.org/10.1016/j.foodcont.2018.09.027

    Article  Google Scholar 

  • Gupta RS, Warren CM, Smith BM et al (2019) Prevalence and severity of food allergies among US adults. JAMA Netw Open 2:e185630. https://doi.org/10.1001/jamanetworkopen.2018.5630

    Article  PubMed  PubMed Central  Google Scholar 

  • Han Y, Liu Z, Khoshelham K, Bai SH (2021) Quality estimation of nuts using deep learning classification of hyperspectral imagery. Comput Electron Agric 180:105868. https://doi.org/10.1016/j.compag.2020.105868

    Article  Google Scholar 

  • Hao W, Zhili S (2020) Improved mosaic: algorithms for more complex images. J Phys Conf Ser 1684:12094. https://doi.org/10.1088/1742-6596/1684/1/012094

    Article  Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 770–778

  • He K, Zhang X, Ren S, Sun J (2014) Spatial pyramid pooling in deep convolutional networks for visual recognition BT - computer vision – ECCV 2014. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds). Springer International Publishing, Cham, pp 346–361

  • Hinton G, Deng L, Yu D et al (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29:82–97. https://doi.org/10.1109/MSP.2012.2205597

    Article  Google Scholar 

  • Hinton GE, Mcclelland JL, Rumelhart DE (1986) Distributed representations (memory storage). Parallel Distrib Process Explor Microstruct Cogn 77–109

  • Hinton GE, Osindero S, Teh Y-W (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554. https://doi.org/10.1162/neco.2006.18.7.1527

    Article  PubMed  Google Scholar 

  • Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9:1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  CAS  PubMed  Google Scholar 

  • Hu J, Zhao D, Zhang Y et al (2021) Real-time nondestructive fish behavior detecting in mixed polyculture system using deep-learning and low-cost devices. Expert Syst Appl 178:115051

    Article  Google Scholar 

  • Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 2261–2269

  • Islam SMM, Rahman A, Prasad N, et al (2019) Identity authentication system using a support vector machine (SVM) on radar respiration measurements. In: 2019 93rd ARFTG Microwave Measurement Conference (ARFTG). pp 1–5

  • Jagtap S, Bhatt C, Thik J, Rahimifard S (2019) Monitoring potato waste in food manufacturing using image processing and internet of things approach. Sustain. 11

  • Jahani Heravi E, Habibi Aghdam H, Puig D (2018) An optimized convolutional neural network with bottleneck and spatial pyramid pooling layers for classification of foods. Pattern Recognit Lett 105:50–58. https://doi.org/10.1016/j.patrec.2017.12.007

    Article  Google Scholar 

  • Jia W, Li Y, Qu R et al (2019) Automatic food detection in egocentric images using artificial intelligence technology. Public Health Nutr 22:1168–1179. https://doi.org/10.1017/S1368980018000538

    Article  PubMed  Google Scholar 

  • Jiang B, He J, Yang S et al (2019) Fusion of machine vision technology and AlexNet-CNNs deep learning network for the detection of postharvest apple pesticide residues. Artif Intell Agric 1:1–8. https://doi.org/10.1016/j.aiia.2019.02.001

    Article  CAS  Google Scholar 

  • Kaur P, Sikka K, Wang W, et al (2019) FoodX-251: a dataset for fine-grained food classification

  • Kawano Y, Yanai K (2014) Food image recognition with deep convolutional features. 589–593

  • Kawano Y, Yanai K (2015) Automatic expansion of a food image dataset leveraging existing categories with domain adaptation BT - computer vision - ECCV 2014 workshops. In: Bronstein MM, Rother C (eds) Agapito L. Springer International Publishing, Cham, pp 3–17

    Google Scholar 

  • Khaki S, Pham H, Han Y et al (2021) DeepCorn: a semi-supervised deep learning method for high-throughput image-based corn kernel counting and yield estimation. Knowledge-Based Syst 218:106874. https://doi.org/10.1016/j.knosys.2021.106874

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  • LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539

    Article  CAS  PubMed  Google Scholar 

  • LeCun Y, Boser B, Denker JS et al (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541–551

    Article  Google Scholar 

  • Lee MC, Chiu SY, Chang JW (2017) A deep convolutional neural network based Chinese menu recognition app. Inf Process Lett 128:14–20. https://doi.org/10.1016/j.ipl.2017.07.010

    Article  Google Scholar 

  • Lin T-Y, Goyal P, Girshick R et al (2020) Focal loss for dense object detection. IEEE Trans Pattern Anal Mach Intell 42:318–327. https://doi.org/10.1109/TPAMI.2018.2858826

    Article  PubMed  Google Scholar 

  • Liu C, Cao Y, Luo Y et al (2018a) A new deep learning-based food recognition system for dietary assessment on an edge computing service infrastructure. IEEE Trans Serv Comput 11:249–261. https://doi.org/10.1109/TSC.2017.2662008

    Article  Google Scholar 

  • Liu C, Cao Y, Luo Y, et al (2016a) DeepFood: deep learning-based food image recognition for computer-aided dietary assessment

  • Liu J-H, Sun X, Young JM et al (2018b) Predicting pork loin intramuscular fat using computer vision system. Meat Sci 143:18–23. https://doi.org/10.1016/j.meatsci.2018.03.020

    Article  PubMed  Google Scholar 

  • Liu W, Anguelov D, Erhan D, et al (2016b) SSD: single shot MultiBox detector BT - computer vision – ECCV 2016b. In: Leibe B, Matas J, Sebe N, Welling M (eds). Springer International Publishing, Cham, pp 21–37

  • López-Pedrouso M, Lorenzo JM, Gagaoua M, Franco D (2020) Current trends in proteomic advances for food allergen analysis. Biology 9(9):247. https://doi.org/10.3390/biology9090247

    Article  PubMed Central  Google Scholar 

  • Maintz L, Novak N (2007) Histamine and histamine intolerance. Am J Clin Nutr 85:1185–1196. https://doi.org/10.1093/ajcn/85.5.1185

    Article  CAS  PubMed  Google Scholar 

  • Mao D, Wang F, Hao Z, Li H (2018) Credit evaluation system based on blockchain for multiple stakeholders in the food supply Chain. Int J Environ Res Public Health 15:1627. https://doi.org/10.3390/ijerph15081627

    Article  PubMed Central  Google Scholar 

  • Martinel N, Foresti GL, Micheloni C (2018) Wide-slice residual networks for food recognition. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). pp 567–576

  • McAllister P (2018) Deep learning-based food image classification and crowdsourcing-based calorie estimation approach to support dietary management

  • McAllister P, Zheng H, Bond R, Moorhead A (2018) Combining deep residual network features with supervised machine learning algorithms to classify diverse food image datasets. Comput Biol Med 95. https://doi.org/10.1016/j.compbiomed.2018.02.008

  • McCulloch WS, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133. https://doi.org/10.1007/BF02478259

    Article  Google Scholar 

  • Mekori YA (1996) Introduction to allergic diseases. Crit Rev Food Sci Nutr 36(Suppl):S1-18. https://doi.org/10.1080/10408399609527756

    Article  CAS  PubMed  Google Scholar 

  • Mezgec S, Koroušić Seljak B (2017) NutriNet: a deep learning food and drink image recognition system for dietary assessment. Nutrients 9: https://doi.org/10.3390/nu9070657

  • Mezgec S, Seljak BK (2019) Using deep learning for food and beverage image recognition. In: 2019 IEEE International Conference on Big Data (Big Data). pp 5149–5151

  • Min W, Liu L, Wang Z, et al (2020) ISIA Food-500: a dataset for large-scale food recognition via stacked global-local attention network

  • Min W, Wang Z, Liu Y, et al (2021) Large scale visual food recognition

  • Muthukumar J, Selvasekaran P, Lokanadham M, Chidambaram R (2020) Food and food products associated with food allergy and food intolerance - an overview. Food Res Int 138:109780. https://doi.org/10.1016/j.foodres.2020.109780

    Article  CAS  PubMed  Google Scholar 

  • Myers A, Johnston N, Rathod V, et al (2015) Im2Calories: towards an automated mobile vision food diary. In: 2015 IEEE International Conference on Computer Vision (ICCV). pp 1233–1241

  • Naritomi S, Tanno R, Ege T, Yanai K (2018) FoodChangeLens: CNN-based food transformation on HoloLens. In: 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). pp 197–199

  • Nasiri A, Omid M, Taheri-Garavand A (2020) An automatic sorting system for unwashed eggs using deep learning. J Food Eng 283:110036. https://doi.org/10.1016/j.jfoodeng.2020.110036

    Article  Google Scholar 

  • Nowak-Węgrzyn A, Chehade M, Groetch ME et al (2017) International consensus guidelines for the diagnosis and management of food protein-induced enterocolitis syndrome: executive summary-Workgroup Report of the Adverse Reactions to Foods Committee, American Academy of Allergy, Asthma & Immunology. J Allergy Clin Immunol 139:1111-1126.e4. https://doi.org/10.1016/j.jaci.2016.12.966

    Article  PubMed  Google Scholar 

  • Nwaru BI, Hickstein L, Panesar SS et al (2014) Prevalence of common food allergies in Europe: a systematic review and meta-analysis. Allergy 69:992–1007. https://doi.org/10.1111/all.12423

    Article  CAS  PubMed  Google Scholar 

  • Ortolani C, Pastorello EA (2006) Food allergies and food intolerances. Best Pract Res Clin Gastroenterol 20:467–483. https://doi.org/10.1016/j.bpg.2005.11.010

    Article  CAS  PubMed  Google Scholar 

  • Pandey P, Deepthi A, Mandal B, Puhan NB (2017) FoodNet: recognizing foods using ensemble of deep networks. IEEE Signal Process Lett 24:1758–1762. https://doi.org/10.1109/LSP.2017.2758862

    Article  Google Scholar 

  • Pereira B, Venter C, Grundy J et al (2005) Prevalence of sensitization to food allergens, reported adverse reaction to foods, food avoidance, and food hypersensitivity among teenagers. J Allergy Clin Immunol 116:884–892. https://doi.org/10.1016/j.jaci.2005.05.047

    Article  PubMed  Google Scholar 

  • Pfisterer KJ, Amelard R, Chung AG, Wong A (2018) A new take on measuring relative nutritional density: the feasibility of using a deep neural network to assess commercially-prepared puréed food concentrations. J Food Eng 223:220–235. https://doi.org/10.1016/j.jfoodeng.2017.10.016

    Article  Google Scholar 

  • Ramos RP, Gomes JS, Prates RM et al (2021) Non-invasive setup for grape maturation classification using deep learning. J Sci Food Agric 101:2042–2051. https://doi.org/10.1002/jsfa.10824

    Article  CAS  PubMed  Google Scholar 

  • Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection

  • Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger

  • Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement

  • Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1. MIT Press, Cambridge, MA, USA, pp 91–99

  • Rich J, Haddadi H, Hospedales T (2016) Towards bottom-up analysis of social food

  • Rodríguez FJ, García A, Pardo PJ et al (2018) Study and classification of plum varieties using image analysis and deep learning techniques. Prog Artif Intell 7:119–127. https://doi.org/10.1007/s13748-017-0137-1

    Article  Google Scholar 

  • Rong D, Xie L, Ying Y (2019) Computer vision detection of foreign objects in walnuts using deep learning. Comput Electron Agric 162:1001–1010. https://doi.org/10.1016/j.compag.2019.05.019

    Article  Google Scholar 

  • Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65:386–408. https://doi.org/10.1037/h0042519

    Article  CAS  PubMed  Google Scholar 

  • Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323:533–536. https://doi.org/10.1038/323533a0

    Article  Google Scholar 

  • Sahoo D, Hao W, Ke S, et al (2019) FoodAI: food image recognition via deep learning for smart food logging. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Association for Computing Machinery, New York, NY, USA, pp 2260–2268

  • Sergi C, Villanacci V, Carroccio A (2021) Non-celiac wheat sensitivity: rationality and irrationality of a gluten-free diet in individuals affected with non-celiac disease: a review. BMC Gastroenterol 21:5. https://doi.org/10.1186/s12876-020-01568-6

    Article  PubMed  PubMed Central  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv 14091556

  • Singla A, Yuan L, Ebrahimi T (2016) Food/non-food image classification and food categorization using pre-trained GoogLeNet model. In: Proceedings of the 2nd International Workshop on Multimedia Assisted Dietary Management. Association for Computing Machinery, New York, NY, USA, pp 3–11

  • Song Q, Zheng Y-J, Xue Y et al (2017) An evolutionary deep neural network for predicting morbidity of gastrointestinal infections by food contamination. Neurocomputing 226:16–22. https://doi.org/10.1016/j.neucom.2016.11.018

    Article  Google Scholar 

  • Soni A, Al-Sarayreh M, Reis MM, Brightwell G (2021) Hyperspectral imaging and deep learning for quantification of Clostridium sporogenes spores in food products using 1D- convolutional neural networks and random forest model. Food Res Int 110577. https://doi.org/10.1016/j.foodres.2021.110577

  • Stadelman WJ (2003) EGGS | dietary importance. Encyclopedia of Food Sciences and Nutrition (Second Edition). Academic Press, USA, pp 2009–2012

    Chapter  Google Scholar 

  • Sun X, Young J, Liu J-H, Newman D (2018a) Prediction of pork loin quality using online computer vision system and artificial intelligence model. Meat Sci 140:72–77. https://doi.org/10.1016/j.meatsci.2018a.03.005

    Article  PubMed  Google Scholar 

  • Sun Y, Wei K, Liu Q, et al (2018b) Classification and discrimination of different fungal diseases of three infection levels on peaches using hyperspectral reflectance imaging analysis. Sensors (Basel) 18. https://doi.org/10.3390/s18041295

  • Szegedy C, Liu W, Jia Y, et al (2015) Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  • Szilagyi A, Ishayek N (2018) Lactose intolerance, dairy avoidance, and treatment options. Nutrients 10:1994. https://doi.org/10.3390/nu10121994

    Article  CAS  PubMed Central  Google Scholar 

  • Taheri-Garavand A, Nasiri A, Banan A, Zhang Y-D (2020) Smart deep learning-based approach for non-destructive freshness diagnosis of common carp fish. J Food Eng 278:109930. https://doi.org/10.1016/j.jfoodeng.2020.109930

    Article  Google Scholar 

  • Tan W, Zhao C, Wu H (2016) Intelligent alerting for fruit-melon lesion image based on momentum deep learning. Multimed Tools Appl 75:16741–16761. https://doi.org/10.1007/s11042-015-2940-7

    Article  Google Scholar 

  • Tatsuma A, Aono M (2016) Food image recognition using covariance of convolutional layer feature maps. IEICE Trans Inf Syst 99-D:1711–1715

    Article  Google Scholar 

  • Taylor S, Hefle S (2001) Food allergies and other food sensitivities. Food Technol 55

  • Temple JL, Bernard C, Lipshultz SE et al (2017) The safety of ingested caffeine: a comprehensive review. Front Psychiatry 8:80. https://doi.org/10.3389/fpsyt.2017.00080

    Article  PubMed  PubMed Central  Google Scholar 

  • Termritthikun C, Muneesawang P, Kanprachar S (2017) NU-InNet: Thai food image recognition using convolutional neural networks on smartphone. J Telecommun Electron Comput Eng 9:63–67

    Google Scholar 

  • Tompson J, Jain A, LeCun Y, Bregler C (2014) Joint training of a convolutional network and a graphical model for human pose estimation. In: Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 1. MIT Press, Cambridge, MA, USA, pp 1799–1807

  • Viola P, Jones M (2001) "Rapid object detection using a boosted cascade of simple features," Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, 2001, pp. I-I. https://doi.org/10.1109/CVPR.2001.990517

  • Wang C-Y, Liao H-YM, Yeh I-H et al (2020) CSPNet: a new backbone that can enhance learning capability of CNN. IEEE/CVF Conf Comput vis Pattern Recognit Work 2020:1571–1580

    Google Scholar 

  • Wang C-Y, Yeh I-H, Liao H (2021) You only learn one representation: unified network for multiple tasks

  • Wang Z, Hu M, Zhai G (2018) Application of deep learning architectures for accurate and rapid detection of internal mechanical damage of blueberry using hyperspectral transmittance data. Sensors 18

  • Widrow B, Hoff ME (1988) Adaptive switching circuits. Neurocomputing: Foundations of Research. MIT Press, Cambridge, MA, USA, pp 123–134

    Google Scholar 

  • Wu N, Zhang C, Bai X, et al (2018) Discrimination of Chrysanthemum varieties using hyperspectral imaging combined with a deep convolutional neural network. Molecules 23. https://doi.org/10.3390/molecules23112831

  • Xiao G, Wu Q, Chen H et al (2020) A deep transfer learning solution for food material recognition using electronic scales. IEEE Trans Ind Informatics 16:2290–2300. https://doi.org/10.1109/TII.2019.2931148

    Article  Google Scholar 

  • Yadav S, Sengar N, Singh A et al (2021) Identification of disease using deep learning and evaluation of bacteriosis in peach leaf. Ecol Inform 61:101247. https://doi.org/10.1016/j.ecoinf.2021.101247

    Article  Google Scholar 

  • Yanai K, Kawano Y (2015) Food image recognition using deep convolutional network with pre-training and fine-tuning. In: 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW). pp 1–6

  • Yu X, Lu H, Wu D (2018a) Development of deep learning method for predicting firmness and soluble solid content of postharvest Korla fragrant pear using Vis/NIR hyperspectral reflectance imaging. Postharvest Biol Technol 141:39–49. https://doi.org/10.1016/j.postharvbio.2018a.02.013

    Article  Google Scholar 

  • Yu X, Tang L, Wu X, Lu H (2018b) Nondestructive freshness discriminating of shrimp using visible/near-infrared hyperspectral imaging technique and deep learning algorithm. Food Anal Methods 11:768–780. https://doi.org/10.1007/s12161-017-1050-8

    Article  Google Scholar 

  • Yu X, Wang J, Wen S et al (2019) A deep learning based feature extraction method on hyperspectral images for nondestructive prediction of TVB-N content in Pacific white shrimp (Litopenaeus vannamei). Biosyst Eng 178:244–255. https://doi.org/10.1016/j.biosystemseng.2018.11.018

    Article  Google Scholar 

  • Zaidi SSA, Ansari MS, Aslam A et al (2022) A survey of modern deep learning based object detection models. Digit Signal Process 126:103514. https://doi.org/10.1016/j.dsp.2022.103514

    Article  Google Scholar 

  • Zhang J, Dai L, Cheng F (2021) Identification of corn seeds with different freezing damage degree based on hyperspectral reflectance imaging and deep learning method. Food Anal Methods 14:389–400. https://doi.org/10.1007/s12161-020-01871-8

    Article  CAS  Google Scholar 

  • Zhang W, Zhang Y, Zhai J et al (2018) Multi-source data fusion using deep learning for smart refrigerators. Comput Ind 95:15–21. https://doi.org/10.1016/j.compind.2017.09.001

    Article  Google Scholar 

  • Zheng J, Zou L, Wang ZJ (2018) Mid-level deep food part mining for food image recognition. IET Comput vis 12:298–304. https://doi.org/10.1049/iet-cvi.2016.0335

    Article  Google Scholar 

  • Zhou X, Sun J, Tian Y et al (2020) Development of deep learning method for lead content prediction of lettuce leaf using hyperspectral images. Int J Remote Sens 41:2263–2276. https://doi.org/10.1080/01431161.2019.1685721

    Article  Google Scholar 

  • Zhou X, Yao C, Wen H, et al (2017) EAST: an efficient and accurate scene text detector

Download references

Acknowledgements

The authors thank GAIN (Axencia Galega de Innovación) for supporting this research (grant number IN607A2019/01). We acknowledge the faculty members of Department of Food Processing Technology and Sri Snehashis Guha, PIC Malda Polytechnic, Malda, for their support to conduct this study.

Author information

Author notes

    Authors and Affiliations

    1. Informatics Cluster, School of Computer Science, University of Petroleum and Energy Studies, Dehradun, Uttarakhand, 248007, India

      Mayank Mishra, Tanupriya Choudhury & Nikunj Bansal

    2. Department of Food Processing Technology, Government of West Bengal, Malda Polytechnic, Bengal State Council of Technical Education, Malda, 732102, West Bengal , India

      Tanmay Sarkar

    3. Laboratory of Microorganisms and Biomolecules of the Center of Biotechnology of Sfax, 3018, Sfax, Tunisia

      Slim Smaoui

    4. Department of Scientific Research, V. M. Gorbatov Federal Research Center for Food Systems, 26 Talalikhin st, Moscow, 109316, Russian Federation

      Maksim Rebezov

    5. Biophotonics Center, Prokhorov General Physics Institute of the Russian Academy of Science, 38 Vavilov st, Moscow, 119991, Russian Federation

      Maksim Rebezov

    6. Department of Scientific Research, K. G. Razumovsky Moscow State University of Technologies and management (The First Cossack University), 73 Zemlyanoy Val, Moscow, 109004, Russian Federation

      Mohammad Ali Shariati

    7. Centro Tecnológico de La Carne de Galicia, Parque Tecnológico de Galicia, Avd. Galicia nº 4, San Cibrao das Viñas, 32900, Ourense, Spain

      Jose Manuel Lorenzo

    8. Área de Tecnología de los Alimentos, Facultad de Ciencias de Ourense, Universidade de Vigo, 32004, Ourense, Spain

      Jose Manuel Lorenzo

    Authors
    1. Mayank Mishra
      View author publications

      You can also search for this author in PubMed Google Scholar

    2. Tanmay Sarkar
      View author publications

      You can also search for this author in PubMed Google Scholar

    3. Tanupriya Choudhury
      View author publications

      You can also search for this author in PubMed Google Scholar

    4. Nikunj Bansal
      View author publications

      You can also search for this author in PubMed Google Scholar

    5. Slim Smaoui
      View author publications

      You can also search for this author in PubMed Google Scholar

    6. Maksim Rebezov
      View author publications

      You can also search for this author in PubMed Google Scholar

    7. Mohammad Ali Shariati
      View author publications

      You can also search for this author in PubMed Google Scholar

    8. Jose Manuel Lorenzo
      View author publications

      You can also search for this author in PubMed Google Scholar

    Contributions

    M.M.: conceptualization, methodology, investigation, validation, formal analysis, writing—original draft preparation; T.C.: methodology, investigation, validation, formal analysis, contribution in writing; T.S.: conceptualization, methodology, investigation, validation, formal analysis, writing—original draft preparation; N.B.: methodology, investigation, validation, formal analysis, contribution in writing; S.S.: methodology, investigation, validation, formal analysis and contribution in writing in relevant section; M.R.: data analysis; writing—review and editing; final draft supervision and monitoring; M.A.S.: review and editing, final draft supervision and monitoring. J.M.L.: review and editing, final draft supervision and monitoring. All authors read and approved the final manuscript.

    Corresponding authors

    Correspondence to Tanmay Sarkar or Jose Manuel Lorenzo.

    Ethics declarations

    Ethics Approval

    Not applicable.

    Consent to Participate

    All authors have given their full consent to participate.

    Consent for Publication

    All authors have given their full consent for publication.

    Conflict of Interest

    Mayank Mishra declares that he has no conflict of interest. Tanmay Sarkar declares that he has no conflict of interest. Tanupriya Choudhury declares that he has no conflict of interest. Nikunj Bansal declares that he has no conflict of interest. Slim Smaoui declares that he has no conflict of interest. Maksim Rebezov declares that he has no conflict of interest. Mohammad Ali Shariati declares that he has no conflict of interest. Jose Manuel Lorenzo declares that he has no conflict of interest.

    Additional information

    Publisher's Note

    Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

    Mayank Mishra, Tanmay Sarkar, Tanupriya Choudhury and Nikunj Bansal contributed equally to this work.

    Rights and permissions

    Reprints and Permissions

    About this article

    Verify currency and authenticity via CrossMark

    Cite this article

    Mishra, M., Sarkar, T., Choudhury, T. et al. Allergen30: Detecting Food Items with Possible Allergens Using Deep Learning-Based Computer Vision. Food Anal. Methods 15, 3045–3078 (2022). https://doi.org/10.1007/s12161-022-02353-9

    Download citation

    • Received:

    • Accepted:

    • Published:

    • Issue Date:

    • DOI: https://doi.org/10.1007/s12161-022-02353-9

    Keywords

    • Food allergy
    • Food dataset
    • Deep learning
    • Computer vision
    • Object detection