Abstract
Tracking dietary intake is an important task for health management especially for chronic diseases such as obesity, diabetes, and cardiovascular diseases. Given the popularity of personal hand-held devices, mobile applications provide a promising low-cost solution to tackle the key risk factor by diet monitoring. In this work, we propose a photo based dietary tracking system that employs deep-based image recognition algorithms to recognize food and analyze nutrition. The system is beneficial for patients to manage their dietary and nutrition intake, and for the medical institutions to intervene and treat the chronic diseases. To the best of our knowledge, there are no popular applications in the market that provide a high-performance food photo recognition like ours, which is more convenient and intuitive to enter food than textual typing. We conducted experiments on evaluating the recognition accuracy on laboratory data and real user data on Singapore local food, which shed light on uplifting lab trained image recognition models in real applications. In addition, we have conducted user study to verify that our proposed method has the potential to foster higher user engagement rate as compared to existing apps based dietary tracking approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Fatsecret platform api. https://platform.fatsecret.com/api/
Integrating caffe2 on ios/android. https://caffe2.ai/docs/mobile-integration.html
Every day big data statistics 2.5 quintillion bytes of data created daily (2015). http://www.vcloudnews.com/every-day-big-data-statistics-2-5-quintillion-bytes-of-data-created-daily/
Singapore Health Fact: Disease Burden. Principal Causes of Death, Ministry of Health Singapore (2016)
Aizawa, K., Ogawa, M.: FoodLog: multimedia tool for healthcare applications. IEEE Multimed. 22(2), 4–8 (2015)
Bettadapura, V., Thomaz, E., Parnami, A., Abowd, G.D., Essa, I.: Leveraging context to support automated food recognition in restaurants. In: 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 580–587. IEEE (2015)
Bossard, L., Guillaumin, M., Van Gool, L.: Food-101 – mining discriminative components with random forests. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 446–461. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_29
Brunstrom, J.M.: Mind over platter: pre-meal planning and the control of meal size in humans. Int. J. Obes. 38(Suppl 1), S9 (2014)
Chen, J., Ngo, C.-W.: Deep-based ingredient recognition for cooking recipe retrieval. In: Proceedings of the 2016 ACM on Multimedia Conference, pp. 32–41. ACM (2016)
Ege, T., Yanai, K.: Simultaneous estimation of food categories and calories with multi-task CNN. In: 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA), pp. 198–201. IEEE (2017)
Forde, C.G., Almiron-Roig, E., Brunstrom, J.M.: Expected satiety: application to weight management and understanding energy selection in humans. Curr. Obes. Rep. 4(1), 131–140 (2015)
Hassannejad, H., Matrella, G., Ciampolini, P., De Munari, I., Mordonini, M., Cagnoni, S.: Food image recognition using very deep convolutional networks. In: Proceedings of the 2nd International Workshop on Multimedia Assisted Dietary Management, pp. 41–49. ACM (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hingle, M., Patrick, H.: There are thousands of apps for that: navigating mobile technology for nutrition education and behavior. J. Nutr. Educ. Behav. 48(3), 213.e1–218.e1 (2016)
Hoashi, H., Joutou, T., Yanai, K.: Image recognition of 85 food categories by feature fusion. In: 2010 IEEE International Symposium on Multimedia (ISM), pp. 296–301. IEEE (2010)
Kawano, Y., Yanai, K.: Food image recognition with deep convolutional features. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 589–593. ACM (2014)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Lau, E., Goh, H.J., Quek, R., Lim, S.W., Henry, J.: Rapid estimation of the energy content of composite foods: the application of the calorie answer. Asia Pac. J. Clin. Nutr. 25(1), 18–25 (2016)
Lim, B.Y., Chng, X., Zhao, S.: Trade-off between automation and accuracy in mobile photo recognition food logging. In: Proceedings of the Fifth International Symposium of Chinese CHI 2017, pp. 53–59. ACM, New York (2017)
Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., Ma, Y.: DeepFood: deep learning-based food image recognition for computer-aided dietary assessment. In: Chang, C.K., Chiari, L., Cao, Y., Jin, H., Mokhtari, M., Aloulou, H. (eds.) ICOST 2016. LNCS, vol. 9677, pp. 37–48. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39601-9_4
Martinel, N., Foresti, G.L., Micheloni, C.: Wide-slice residual networks for food recognition. arXiv preprint arXiv:1612.06543 (2016)
Matsuda, Y., Hoashi, H., Yanai, K.: Recognition of multiple-food images by detecting candidate regions. In: 2012 IEEE International Conference on Multimedia and Expo, pp. 25–30. IEEE, July 2012
Matsuda, Y., Yanai, K.: Multiple-food recognition considering co-occurrence employing manifold ranking. In: 2012 21st International Conference on Pattern Recognition (ICPR), pp. 2017–2020. IEEE (2012)
Myers, A., Johnston, N., Rathod, V., Korattikara, A., Gorban, A., Silberman, N., Guadarrama, S., Papandreou, G., Huang, J., Murphy, K.: Im2Calories: towards an automated mobile vision food diary. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1233–1241. IEEE, December 2015
World Health Organization: preventing chronic diseases: a vital investment. World Health Organization (2005)
Health Promotion Board Singapore: energy and nutrition composition of food. http://focos.hpb.gov.sg/eservices/ENCF/. Accessed 18 July 2017
Waki, K., Aizawa, K., Kato, S., Fujita, H., Lee, H., Kobayashi, H., Ogawa, M., Mouri, K., Kadowaki, T., Ohe, K.: Dialbetics with a multimedia food recording tool, foodlog: smartphone-based self-management for type 2 diabetes. J. Diab. Sci. Technol. 9(3), 534–540 (2015)
Willett, W.C., Koplan, J.P., Nugent, R., Dusenbury, C., Puska, P., Gaziano, T.A.: Prevention of chronic disease by means of diet and lifestyle changes (2006)
Yanai, K., Kawano, Y.: Food image recognition using deep convolutional network with pre-training and fine-tuning. In: 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1–6. IEEE (2015)
Acknowledgments
This research is part of NExT++ project, supported by the National Research Foundation, Prime Minister’s Office, Singapore under its IRC@Singapore Funding Initiative.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Ming, ZY., Chen, J., Cao, Y., Forde, C., Ngo, CW., Chua, T.S. (2018). Food Photo Recognition for Dietary Tracking: System and Experiment. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10705. Springer, Cham. https://doi.org/10.1007/978-3-319-73600-6_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-73600-6_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73599-3
Online ISBN: 978-3-319-73600-6
eBook Packages: Computer ScienceComputer Science (R0)