Skip to main content

Advertisement

SpringerLink
Log in

Automatic Prediction of Glycemic Index Category from Food Images Using Machine Learning Approaches

  • Research Article-Computer Engineering and Computer Science
  • Published:
Arabian Journal for Science and Engineering Aims and scope Submit manuscript

Abstract

The dietary glycemic index (GI) is a mechanism, whereby carbohydrate-containing foods are assigned a number according to how much each food raises blood sugar. It plays a very important role in the health of a person as this affects significantly metabolic disorders especially when counts for diabetes and obesity. Serious interventions are therefore required for GI in terms of medical perspective. There are a series of breakthroughs, which attempt to improvise the classification of food types from food images using machine learning (ML) techniques. However, no research is reported for the classification of food type based on the food GI. Therefore, the proposed work investigates food image classification based on their GI index category. The work is the first of its kind. The proposed framework is employed on food images from foodpics_extended databases that are based on international GI tables of three food categories for low, medium, and high. An extensive range of texture, statistical, and shape features was extracted from the images. Thereafter, experiments were performed with five different categories of classifiers, viz. AdaBoost with random forest (RF), J48 decision tree, k-nearest-neighbor (KNN) classifier, Naive Bayes classifier, and sequential minimal optimization (SMO)-based support vector machine (SVM) classifier. Statistical analysis was conducted to compare the effectiveness of the different methods. Though no method performs significantly better than others, we observed that the overall performance of the AdaBoost (RF) ensemble model provided better classification accuracy in finding the category of food image GI index.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Forster, H.; Walsh, M.C.; Gibney, M.J.; Brennan, L.; Gibney, E.R.: Personalised nutrition: the role of new dietary assessment methods. Proc. Nutr. Soc. 75, 96–105 (2016)

    Article  Google Scholar 

  2. Coughlin, S.S.; Whitehead, M.; Sheats, J.Q.; Mastromonico, J.; Hardy, D.; Smith, S.A.: Smartphone applications for promoting healthy diet and nutrition: a literature review. Jacobs J. Food Nutr. 2, 21 (2015)

    Google Scholar 

  3. Stumbo, P.J.: New technology in dietary assessment: a review of digital methods in improving food record accuracy. Proc. Nutr. Soc. 72, 70–76 (2013)

    Article  Google Scholar 

  4. Sakshi, V.K.: A retrospective study on handwritten mathematical symbols and expressions: classification and recognition. Eng. Appl. Artif. Intell. 103, 104292 (2021). https://doi.org/10.1016/j.engappai.2021.104292

    Article  Google Scholar 

  5. Fallaize, R.; Forster, H.; Macready, A.L.; Walsh, M.C.; Mathers, J.C.; Brennan, L.; Gibney, E.R.; Gibney, M.J.; Lovegrove, J.A.: Online dietary intake estimation: reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J. Med. Internet Res. 16, e190 (2014)

    Article  Google Scholar 

  6. Kukreja, V.; Kumar, D.; Kaur, A.: GAN-based synthetic data augmentation for increased CNN performance in vehicle number plate recognition. In: 2020 4th International Conference on Electronics, Communication and Aerospace Technology. IEEE, pp. 1190–1195 (2020)

  7. Sharp, D.B.; Allman-Farinelli, M.: Feasibility and validity of mobile phones to assess dietary intake. Nutrition 30, 1257–1266 (2014)

    Article  Google Scholar 

  8. Free, C.; Phillips, G.; Galli, L.; Watson, L.; Felix, L.; Edwards, P.; Patel, V.; Haines, A.: The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: a systematic review. PLoS Med. 10, e1001362 (2013)

    Article  Google Scholar 

  9. Carter, M.C.; Burley, V.J.; Nykjaer, C.; Cade, J.E.: ‘My Meal Mate’(MMM): validation of the diet measures captured on a smartphone application to facilitate weight loss. Br. J. Nutr. 109, 539–546 (2013)

    Article  Google Scholar 

  10. Kukreja, V.; Dhiman, P.: A Deep Neural Network based disease detection scheme for Citrus fruits. In: 2020 International Conference on Smart Electronics and Communication. IEEE, pp. 97–101 (2020)

  11. Lamba, S.; Saini, P.; Kukreja, V.; Sharma, B.: Role of Mathematics in Machine Learning, Available SSRN 3833931 (2021)

  12. Otto, H.; Bleyer, G.; Pennartz, M.; Sabin, G.; Schauberger, G.; Spaethe, R.: Diet in diabetes mellitus (1973)

  13. Jenkins, D.J.A.; Wolever, T.M.S.; Jenkins, A.L.: Starchy foods and glycemic index. Diabetes Care 11, 149–159 (1988)

    Article  Google Scholar 

  14. He, Y.; Xu, C.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Food image analysis: Segmentation, identification and weight estimation. In: 2013 IEEE International Conference on Multimedia and Expo. IEEE, pp. 1–6 (2013)

  15. Aston, L.M.; Jackson, D.; Monsheimer, S.; Whybrow, S.; Handjieva-Darlenska, T.; Kreutzer, M.; Kohl, A.; Papadaki, A.; Martinez, J.A.; Kunova, V.: Developing a methodology for assigning glycaemic index values to foods consumed across Europe. Obes. Rev. 11, 92–100 (2010)

    Article  Google Scholar 

  16. Dehais, J.; Anthimopoulos, M.; Mougiakakou, S.: Dish detection and segmentation for dietary assessment on smartphones. In: International Conference on Image Analysis and Processing. Springer, pp. 433–440 (2015)

  17. Zhu, F.; Bosch, M.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Multiple hypotheses image segmentation and classification with application to dietary assessment. IEEE J. Biomed. Heal. Inform. 19, 377–388 (2014)

    Article  Google Scholar 

  18. Shroff, G.; Smailagic, A.; Siewiorek, D.P.: Wearable context-aware food recognition for calorie monitoring. In: 2008 12th IEEE International Symposium on Wearable Computers. IEEE, pp. 119–120 (2008)

  19. He, Y.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Image segmentation for image-based dietary assessment: a comparative study. In: International Symposium on Signals, Circuits and Systems, ISSCS2013. IEEE, pp. 1–4 (2013)

  20. Shi, J.; Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)

    Article  Google Scholar 

  21. Felzenszwalb, P.F.; Huttenlocher, D.P.: Efficient graph-based image segmentation. Int. J. Comput. Vis. 59, 167–181 (2004)

    Article  Google Scholar 

  22. Matsuda, Y.; Hoashi, H.; Yanai, K.: Recognition of multiple-food images by detecting candidate regions. In: 2012 IEEE International Conference on Multimedia and Expo. IEEE, pp. 25–30 (2012)

  23. Bettadapura, V.; Thomaz, E.; Parnami, A.; Abowd, G.D.; Essa, I.: Leveraging context to support automated food recognition in restaurants. In: 2015 IEEE Winter Conference on Applications of Computer Vision. IEEE, pp. 580–587 (2015)

  24. Rother, C.; Kolmogorov, V.; Blake, A.: “ GrabCut” interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. 23, 309–314 (2004)

    Article  Google Scholar 

  25. Kawano, Y.; Yanai, K.: Foodcam: a real-time food recognition system on a smartphone. Multimed. Tools Appl. 74, 5263–5287 (2015)

    Article  Google Scholar 

  26. Puri, M.; Zhu, Z.; Yu, Q.; Divakaran, A.; Sawhney, H.: Recognition and volume estimation of food intake using a mobile device. In: 2009 Workshop on Applications of Computer Vision (WACV). IEEE, pp. 1–8 (2009)

  27. Anthimopoulos, M.; Dehais, J.; Diem, P.; Mougiakakou, S.: Segmentation and recognition of multi-food meal images for carbohydrate counting. In: 13th IEEE International Conference on BioInformatics and BioEngineering. IEEE, pp. 1–4 (2013)

  28. Rashid, M.; Khan, M.A.; Sharif, M.; Raza, M.; Sarfraz, M.M.; Afza, F.: Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features. Multimed. Tools Appl. 78, 15751–15777 (2019)

    Article  Google Scholar 

  29. Qin, C.; Sun, M.; Chang, C.-C.: Perceptual hashing for color images based on hybrid extraction of structural features. Signal Process. 142, 194–205 (2018)

    Article  Google Scholar 

  30. He, Y.; Xu, C.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Context based food image analysis. In: 2013 IEEE International Conference on Image Processing. IEEE, pp. 2748–2752 (2013)

  31. He, Y.; Xu, C.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Analysis of food images: features and classification. In: 2014 IEEE International Conference on Image Processing. IEEE, pp. 2744–2748 (2014)

  32. Bosch, M.; Zhu, F.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Combining global and local features for food identification in dietary assessment. In: 2011 18th IEEE International Conference on Image Processing. IEEE, pp. 1789–1792 (2011)

  33. Felzenszwalb, P.F.; Girshick, R.B.; McAllester, D.; Ramanan, D.: Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1627–1645 (2009)

    Article  Google Scholar 

  34. Sluzek, A.; Ganzha, M.; Maciaszek, L.A.; Paprzycki, M.: Machine vision in food recognition: attempts to enhance CBVIR tools. In: FedCSIS Position Paper, pp. 57–61 (2016)

  35. Farinella, G.M.; Allegra, D.; Stanco, F.: A benchmark dataset to study the representation of food images. In: European Conference on Computer Vision. Springer, pp. 584–599 (2014)

  36. Mezgec, S.; Koroušić Seljak, B.: NutriNet: a deep learning food and drink image recognition system for dietary assessment. Nutrients 9, 657 (2017)

    Article  Google Scholar 

  37. Sun, M.; Burke, L.E.; Mao, Z.-H.; Chen, Y.; Chen, H.-C.; Bai, Y.; Li, Y.; Li, C.; Jia, W.: eButton: a wearable computer for health monitoring and personal assistance. In: Proceedings of the 51st Annual Design Automation Conference, pp. 1–6 (2014)

  38. O’Loughlin, G.; Cullen, S.J.; McGoldrick, A.; O’Connor, S.; Blain, R.; O’Malley, S.; Warrington, G.D.: Using a wearable camera to increase the accuracy of dietary analysis. Am. J. Prev. Med. 44, 297–301 (2013)

    Article  Google Scholar 

  39. Kong, F.; Tan, J.: DietCam: automatic dietary assessment with mobile camera phones. Pervasive Mob. Comput. 8, 147–163 (2012)

    Article  Google Scholar 

  40. Pouladzadeh, P.; Shirmohammadi, S.; Al-Maghrabi, R.: Measuring calorie and nutrition from food image. IEEE Trans. Instrum. Meas. 63, 1947–1956 (2014)

    Article  Google Scholar 

  41. Zhu, F.; Bosch, M.; Woo, I.; Kim, S.; Boushey, C.J.; Ebert, D.S.; Delp, E.J.: The use of mobile devices in aiding dietary assessment and evaluation. IEEE J. Sel. Top. Signal Process. 4, 756–766 (2010)

    Article  Google Scholar 

  42. Zhu, F.; Bosch, M.; Schap, T.; Khanna, N.; Ebert, D.S.; Boushey, C.J.; Delp, E.J.: Segmentation assisted food classification for dietary assessment. In: Computational Imaging IX, International Society for Optics and Photonics, p. 78730B (2011)

  43. Wang, Y.; He, Y.; Boushey, C.J.; Zhu, F.; Delp, E.J.: Context based image analysis with application in dietary assessment and evaluation. Multimed. Tools Appl. 77, 19769–19794 (2018)

    Article  Google Scholar 

  44. Meyers, A.; Johnston, N.; Rathod, V.; Korattikara, A.; Gorban, A.; Silberman, N.; Guadarrama, S.; Papandreou, G.; Huang, J.; Murphy, K.P.: Im2Calories: towards an automated mobile vision food diary. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1233–1241 (2015)

  45. McFee, B.; Galleguillos, C.; Lanckriet, G.: Contextual object localization with multiple kernel nearest neighbor. IEEE Trans. Image Process. 20, 570–585 (2010)

    Article  MathSciNet  Google Scholar 

  46. Ma, W.Y.; Deng, Y.; Manjunath, B.S.: Tools for texture-and color-based search of images. In: Human Vision & Electronic Imaging II. International Society for Optics and Photonics, pp. 496–507 (1997)

  47. Bosch, M.; Zhu, F.; Khanna, N.; Boushey, C.J.; Delp, E.J.: Food texture descriptors based on fractal and local gradient information. In: 2011 19th European Signal Processing Conference. IEEE, pp. 764–768 (2011)

  48. Farinella, G.M.; Moltisanti, M.; Battiato, S.: Classifying food images represented as bag of textons. In: 2014 IEEE International Conference on Image Processing. IEEE, pp. 5212–5216 (2014)

  49. Anthimopoulos, M.; Gianola, L.; Scarnato, L.; Diem, P.; Mougiakakou, S.G.: A food recognition system for diabetic patients based on an optimized bag-of-features model. IEEE J. Biomed. Heal. Inform. 18, 1261–1271 (2014)

    Article  Google Scholar 

  50. Dalakleidi, K.; Sarantea, M.; Nikita, K.S.: A modified all-and-one classification algorithm combined with the bag-of-features model to address the food recognition task. In: HEALTHINF, pp. 284–290 (2017)

  51. Attokaren, D.J.; Fernandes, I.G.; Sriram, A.; Murthy, Y.V.S.; Koolagudi, S.G.: Food classification from images using convolutional neural networks. In: TENCON 2017–2017 IEEE Region 10 Conference. IEEE, pp. 2801–2806 (2017)

  52. Luo, Y.; Ling, C.; Ao, S.: Mobile-based food classification for type-2 diabetes using nutrient and textual features. In: 2014 International Conference on Data Science and Advanced Analytics. IEEE, pp. 563–569 (2014)

  53. Aizawa, K.; Maruyama, Y.; Li, H.; Morikawa, C.: Food balance estimation by using personal dietary tendencies in a multimedia food log. IEEE Trans. Multimed. 15, 2176–2185 (2013)

    Article  Google Scholar 

  54. McAllister, P.; Zheng, H.; Bond, R.; Moorhead, A.: Towards personalised training of machine learning algorithms for food image classification using a smartphone camera. In: International Conference on Ubiquitous Computing and Ambient Intelligence. Springer, pp. 178–190 (2016)

  55. Joutou, T.; Yanai, K.: A food image recognition system with multiple kernel learning. In: 2009 16th IEEE International Conference on Image Processing. IEEE, pp. 285–288 (2009)

  56. Yang, S.; Chen, M.; Pomerleau, D.; Sukthankar, R.: Food recognition using statistics of pairwise local features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, pp. 2249–2256 (2010)

  57. Chen, M.; Dhingra, K.; Wu, W.; Yang, L.; Sukthankar, R.; Yang, J.: PFID: Pittsburgh fast-food image dataset. In: 2009 16th IEEE International Conference on Image Processing. IEEE, pp. 289–292 (2009)

  58. Baxter, J.: Food recognition using ingredient-level features, Ηλεκτρονικό]. http://Jaybaxter.Net/6869_food_project.pdf. (2012)

  59. Singla, A.; Yuan, L.; Ebrahimi, T.: Food/non-food image classification and food categorization using pre-trained googlenet model. In: Proceedings of the 2nd International Workshop on Multimedia Assisted Dietary Management, pp. 3–11 (2016)

  60. Wang, X.; Kumar, D.; Thome, N.; Cord, M.; Precioso, F.: Recipe recognition with large multimodal food dataset. In: 2015 IEEE International Conference on Multimedia & Expo Workshops. IEEE, pp. 1–6 (2015)

  61. Chokr, M.; Elbassuoni, S.: Calories prediction from food images. In: Twenty-Ninth IAAI Conference (2017)

  62. Ciocca, G.; Napoletano, P.; Schettini, R.: CNN-based features for retrieval and classification of food images. Comput. Vis. Image Underst. 176, 70–77 (2018)

    Article  Google Scholar 

  63. Pandey, P.; Deepthi, A.; Mandal, B.; Puhan, N.B.: FoodNet: recognizing foods using ensemble of deep networks. IEEE Signal Process. Lett. 24, 1758–1762 (2017)

    Article  Google Scholar 

  64. Bossard, L.; Guillaumin, M.; Van Gool, L.: Food-101–mining discriminative components with random forests. In: European Conference on Computer Vision. Springer, pp. 446–461 (2014)

  65. Pan, L.; Pouyanfar, S.; Chen, H.; Qin, J.; Chen, S.-C.: Deepfood: Automatic multi-class classification of food ingredients using deep learning. In: 2017 IEEE 3rd International Conference on Collaboration and Internet Computing. IEEE, pp. 181–189 (2017)

  66. Burkapalli, V.C.; Patil, P.C.: Segmentation and Identification of Indian food items from Images (n.d.)

  67. Sun, F.; Gu, Z.; Feng, B.: Yelp food identification via image feature extraction and classification. arXiv Preprint arXiv:1902.05413 (2019)

  68. Asghar, N.: Yelp dataset challenge: review rating prediction. arXiv Preprint arXiv:1605.05362 (2016)

  69. Kaur, P.; Sikka, K.; Wang, W.; Belongie, S.; Divakaran, A.: FoodX-251: A dataset for fine-grained food classification. arXiv Preprint arXiv:1907.06167 (2019)

  70. De Bonis, M.; Amato, G.; Falchi, F.; Gennaro, C.; Manghi, P.: Deep learning techniques for visual food recognition on a Mobile App. In: International Conference on Multimedia & Network Information Systems. Springer, pp. 303–312 (2018)

  71. Atkinson, F.S.; Foster-Powell, K.; Brand-Miller, J.C.: Glycemic index (GI) and glycemic load (GL) values determined in subjects with normal glucose tolerance: 2008. Int. Tables Glycemic Index Glycemic Load Values, pp. 77–81 (2008)

  72. Foster-Powell, K.; Holt, S.H.A.; Brand-Miller, J.C.: International table of glycemic index and glycemic load values: 2002. Am. J. Clin. Nutr. 76, 5–56 (2002)

    Article  Google Scholar 

  73. Blechert, J.; Lender, A.; Polk, S.; Busch, N.; Ohla, K.: Food-pics_extended—an image database for experimental research on eating and appetite: additional images, normative ratings and an updated review. Front. Psychol. 10, 307 (2019)

    Article  Google Scholar 

  74. Gautam, S.; Sharma, C.; Kukreja, V.: Handwritten mathematical symbols classification using WEKA. In: Applied Artificial Intelligence Machine Learning. Springer, pp. 33–41 (2021)

  75. Olendzki, B.C.; Ma, Y.; Culver, A.L.; Ockene, I.S.; Griffith, J.A.; Hafner, A.R.; Hebert, J.R.: Methodology for adding glycemic index and glycemic load values to 24-hour dietary recall database. Nutrition 22, 1087–1095 (2006)

    Article  Google Scholar 

  76. Dhanachandra, N.; Manglem, K.; Chanu, Y.J.: Image segmentation using K-means clustering algorithm and subtractive clustering algorithm. Procedia Comput. Sci. 54, 764–771 (2015)

    Article  Google Scholar 

  77. Hall, M.A.; Witten, I.H.: The WEKA Workbench, Online Appendix Data Mining. Practical Machine Learning Tools and Techniques, 4th edn. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

  78. Hastie, T.; Tibshirani, R.; Friedman, J.: Random forests. In: The Elements of Statistical Learning. Springer, pp 587–604 (2009)

  79. Wyner, A.J.; Olson, M.; Bleich, J.; Mease, D.: Explaining the success of adaboost and random forests as interpolating classifiers. J. Mach. Learn. Res. 18, 1558–1590 (2017)

    MathSciNet  MATH  Google Scholar 

  80. Valencia, X.B.; Torres, D.B.; Rodriguez, C.P.; Peluffo-Ordóñez, D.H.; Becerra, M.A.; Castro-Ospina, A.E.: Case-based reasoning systems for medical applications with improved adaptation and recovery stages. In: International Conference on Bioinformatics and Biomedical Engineering. Springer, pp. 26–38 (2018)

  81. Bin Tariq, O.; Lazarescu, M.T.; Iqbal, J.; Lavagno, L.: Performance of machine learning classifiers for indoor person localization with capacitive sensors. IEEE Access 5, 12913–12926 (2017)

    Article  Google Scholar 

  82. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This research is carried out in the Embedded System and Design lab, Department of Electronics and Communication Engineering, National Institute of Technology Raipur, CG, India. The authors acknowledge Dr. Ajay Singh Raghuwanshi and Dr. Subhojit Ghosh and Dr. Bikesh Singh of National Institute of Technology Raipur, CG, India, for their assistance in this research.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology, Raipur, India

    Mohammad Imroze Khan & Bibhudendra Acharya

  2. Department of Electronics and Communication Engineering, Maulana Azad National Institute of Technology, Bhopal, India

    Rahul Kumar Chaurasiya

Authors
  1. Mohammad Imroze Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bibhudendra Acharya
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rahul Kumar Chaurasiya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Imroze Khan.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, M.I., Acharya, B. & Chaurasiya, R.K. Automatic Prediction of Glycemic Index Category from Food Images Using Machine Learning Approaches. Arab J Sci Eng 47, 10823–10846 (2022). https://doi.org/10.1007/s13369-022-06754-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13369-022-06754-0

Keywords

Search

Navigation