A Multimedia Database for Automatic Meal Assessment Systems

  • Dario Allegra
  • Marios Anthimopoulos
  • Joachim Dehais
  • Ya Lu
  • Filippo Stanco
  • Giovanni Maria Farinella
  • Stavroula MougiakakouEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10590)


A healthy diet is crucial for maintaining overall health and for controlling food-related chronic diseases, like diabetes and obesity. Proper diet management however, relies on the rather challenging task of food intake assessment and monitoring. To facilitate this procedure, several systems have been recently proposed for automatic meal assessment on mobile devices using computer vision methods. The development and validation of these systems requires large amounts of data and although some public datasets already exist, they don’t cover the entire spectrum of inputs and/or uses. In this paper, we introduce a database, which contains RGB images of meals together with the corresponding depth maps, 3D models, segmentation and recognition maps, weights and volumes. We also present a number of experiments on the new database to provide baselines performances in the context of food segmentation, depth and volume estimation.


  1. 1.
    Merler, M., et al.: Snap, Eat, RepEat: a food recognition engine for dietary logging. In: Proceedings of the MADiMa 2016, pp. 31–40 (2016)Google Scholar
  2. 2.
    Myers, A., et al.: Im2Calories: towards an automated mobile vision food diary. In: Proceedings of the ICCV 2015, pp. 1233–124 (2015)Google Scholar
  3. 3.
    Anthimopoulos, M., et al.: Computer vision-based carbohydrate estimation for type 1 patients with diabetes using smartphones. J. Diabetes Sci. Technol. 3, 507–515 (2015)CrossRefGoogle Scholar
  4. 4.
    Zhu, F., et al.: The use of mobile devices in aiding dietary assessment and evaluation. IEEE J-STSP 4(4), 756–766 (2010)Google Scholar
  5. 5.
    Miyazaki, T., et al.: Image-based calorie content estimation for dietary assessment. In: Proceedings of the IEEE ISM (2011)Google Scholar
  6. 6.
    Bally, L., et al.: Carbohydrate estimation supported by the GoCARB system in individuals with type 1 diabetes – a randomized prospective pilot study. Diabetes Care 40(2), e6–e7 (2016). dc162173CrossRefGoogle Scholar
  7. 7.
    Rhyner, D., et al.: Carbohydrate estimation by a mobile phone-based system versus self-estimations of individuals with type 1 diabetes mellitus: a comparative study. J. Med. Internet Res. 18(5), e101 (2016)CrossRefGoogle Scholar
  8. 8.
    Chen, M., et al.: PFID: Pittsburgh fast-food image dataset. In: Proceedings of the ICIP 2009, pp. 289–292 (2009)Google Scholar
  9. 9.
    Matsuda, Y., et al.: Recognition of multiple-food images by detecting candidate regions. In: Proceedings of the ICME 2012, pp. 25–30 (2012)Google Scholar
  10. 10.
    Kawano, Y., et al.: Automatic expansion of a food image dataset leveraging existing categories with domain adaptation. ECCV 2014(8927), 3–17 (2014)Google Scholar
  11. 11.
    Chen, M.-Y., et al.: Automatic chinese food identification and quantity estimation. In: Proceedings of the SIGGRAPH Asia Technical Briefs, pp. 1–4 (2012)Google Scholar
  12. 12.
    Farinella, G.M., et al.: A benchmark dataset to study the representation of food images. ACVR 2014(8927), 584–599 (2014)Google Scholar
  13. 13.
    Farinella, G.M., et al.: Retrieval and classification of food images. Comput. Biol. Med. 77, 23–39 (2016)CrossRefGoogle Scholar
  14. 14.
    Farinella, G.M., et al.: On the exploitation of one class classification to distinguish food vs non-food images. MADiMa 2015(9281), 375–383 (2015)MathSciNetGoogle Scholar
  15. 15.
    Bossard, L., et al.: Food-101 - mining discriminative components with random forests. ECCV 8694, 446–461 (2014)Google Scholar
  16. 16.
    Wang, X., et al.: Recipe recognition with large multimodal food dataset. In: Proceedings of the IEEE ICMEW 2015, pp, 1–6 (2015)Google Scholar
  17. 17.
    Pouladzadeh, P., et al.: FooDD: food detection dataset for calorie measurement using food images. MADiMa 2015(9281), 441–448 (2015)MathSciNetGoogle Scholar
  18. 18.
    Herranz, L., et al.: A probabilistic model for food image recognition in restaurants. In: Proceedings of the ICME 2015, pp. 1–6 (2015)Google Scholar
  19. 19.
    Dehais, J., et al.: Food image segmentation for dietary assessment. In: Proceedings of the MADiMa 2016, pp. 23–28 (2016)Google Scholar
  20. 20.
    Badrinarayanan, V., et al.: SegNet: a deep convolutional encoder-decoder architecture for scene segmentation. In: Proceedings of the IEEE TPAMI (2017)Google Scholar
  21. 21.
    Ronneberger, O., et al.: U-Net: convolutional networks for biomedical image segmentation. MICCAI 9351, 234–241 (2015)Google Scholar
  22. 22.
    Dehais, J., et al.: Two-View 3D reconstruction for food volume estimation. IEEE TMM 19(5), 1090–1099 (2017)Google Scholar
  23. 23.
    Eigen, D., et. al.: Depth map prediction from a single image using a multi-scale deep network. In: Proceedings of the NIPS 2014, pp. 2366–2374 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Dario Allegra
    • 1
  • Marios Anthimopoulos
    • 2
    • 3
  • Joachim Dehais
    • 2
  • Ya Lu
    • 2
  • Filippo Stanco
    • 1
  • Giovanni Maria Farinella
    • 1
  • Stavroula Mougiakakou
    • 2
    • 4
    Email author
  1. 1.Department of Mathematics and Computer ScienceUniversity of CataniaCataniaItaly
  2. 2.ARTORG Center for Biomedical Engineering ResearchUniversity of BernBernSwitzerland
  3. 3.Department of Emergency MedicineBern University HospitalBernSwitzerland
  4. 4.Department of Endocrinology, Diabetes and Clinical NutritionBern University HospitalBernSwitzerland

Personalised recommendations