Discriminative Region Guided Deep Neural Network Towards Food Image Classification

  • Yali Chen
  • Yanping Yang
  • Qing Fang
  • Xiaoyu Yao
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 772)


Food image classification plays an important role in smart health management, such as, diet analysis and food recommendation. Due to the similar appearance and shape between different foods, it is quite challenging to distinguish various food categories from their images. To address this issue, we propose a discriminative region guided deep neural network to classify the food images. More specifically, a saliency map based pooling strategy is applied to the input image to preserve the category aware discriminative regions. Meanwhile, the multi-scale fusion scheme is employed in our deep neural network to describe the discriminative regions across different resolutions. Experimental results on a large-scale Chinese food database show that, the average accuracy the proposed method is as high as 91.18%, and outperforms the baseline by 2.58%.


Image classification CNN Deep learning Discriminative regions 


  1. 1.
    Salvador, A., Hynes, N., Aytar, Y., Marin, J., Ofli, F., Weber, I., Torralba, A.: Learning cross-modal embeddings for cooking recipes and food images. In: CVPR (2017)Google Scholar
  2. 2.
    Angelova, A., Zhu, S.: Efficient object detection and segmentation for fine-grained recognition. In: CVPR (2013)Google Scholar
  3. 3.
    Beijbom, O., Joshi, N., Morris, D., Saponas, T.S., Khullar, S.: Menu-match: Restaurant-specific food logging from images. In: Computer Vision (2015)Google Scholar
  4. 4.
    Bossard, L., Guillaumin, M., Gool, L.V.: Food-101: mining discriminative components with random forests. In: ECCV (2014)Google Scholar
  5. 5.
    Ciocca, G., Napoletano, P., Schettini, R.: Food recognition: a new dataset, experiments and results. IEEE J. Biomed. Health Inform. 21(3), 588–598 (2016)CrossRefGoogle Scholar
  6. 6.
    Cordeiro, F., Bales, E., Cherry, E., Fogarty, J.: Rethinking the mobile food journal: exploring opportunities for lightweight photo-based capture. In: ACM Conference on Human Factors in Computing Systems (2015)Google Scholar
  7. 7.
    Fan, J., Gao, Y., Luo, H., Jain, R.: Mining multilevel image semantics via hierarchical classification. IEEE Trans. Multimedia 10(2), 167–184 (2008)CrossRefGoogle Scholar
  8. 8.
    Farooq, M., Sazonov, E.: A novel wearable device for food intake and physical activity recognition. Sensors 16(7), 1067 (2016)CrossRefGoogle Scholar
  9. 9.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR (2014)Google Scholar
  10. 10.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  11. 11.
    Herranz, L., Jiang, S., Xu, R.: Modeling restaurant context for food recognition. IEEE Trans. Multimedia 19(2), 430–440 (2017)CrossRefGoogle Scholar
  12. 12.
    Huang, C., Li, H., Xie, Y., Wu, Q., Luo, B.: PBC: polygon-based classifier for fine-grained categorization. IEEE Trans. Multimedia 19(4), 673–684 (2016)CrossRefGoogle Scholar
  13. 13.
    Huang, C., Meng, F., Luo, W., Zhu, S.: Bird breed classification and annotation using saliency based graphical model. J. VCIR 25(6), 1299–1307 (2014)Google Scholar
  14. 14.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J.: Caffe: convolutional architecture for fast feature embedding. Eprint arXiv (2014)Google Scholar
  15. 15.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)Google Scholar
  16. 16.
    Lee, G., Tai, Y.W., Kim, J.: Deep saliency with encoded low level distance map and high level features. In: CVPR (2016)Google Scholar
  17. 17.
    Lin, M., Chen, Q., Yan, S.: Network in network. Comput. Sci. (2013)Google Scholar
  18. 18.
    Lin, Z., Hua, G., Davis, L.S.: Multi-scale shared features for cascade object detection. In: ICIP (2013)Google Scholar
  19. 19.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)Google Scholar
  20. 20.
    Martin, C.K., Correa, J.B., Han, H., Allen, H.R., Rood, J.C., Champagne, C.M., Gunturk, B.K., Bray, G.A.: Validity of the remote food photography method (rfpm) for estimating energy and nutrient intake in near real time. Obesity 20(4), 891–899 (2012)CrossRefGoogle Scholar
  21. 21.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2015)Google Scholar
  22. 22.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: CVPR (2015)Google Scholar
  23. 23.
    Tammachat, N., Pantuwong, N.: Calories analysis of food intake using image recognition. In: ICITEE (2014)Google Scholar
  24. 24.
    Yanai, K., Kawano, Y.: Food image recognition using deep convolutional network with pre-training and fine-tuning. In: ICME (2015)Google Scholar
  25. 25.
    Yang, S., Chen, M., Pomerleau, D., Sukthankar, R.: Food recognition using statistics of pairwise local features. In: CVPR (2010)Google Scholar
  26. 26.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). Google Scholar
  27. 27.
    Zhang, J., Sclaroff, S., Lin, Z., Shen, X., Price, B., Mech, R.: Minimum barrier salient object detection at 80 fps. In: ICCV (2016)Google Scholar
  28. 28.
    Zhang, X., Xiong, H., Zhou, W., Lin, W., Tian, Q.: Picking deep filter responses for fine-grained image recognition. In: CVPR (2016)Google Scholar
  29. 29.
    Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: CVPR (2016)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2017

Authors and Affiliations

  • Yali Chen
    • 1
  • Yanping Yang
    • 1
  • Qing Fang
    • 1
  • Xiaoyu Yao
    • 1
  1. 1.School of Electronic EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina

Personalised recommendations