Graph-Based Scale-Aware Network for Human Parsing

  • Beibei Yang
  • Changqian Yu
  • Jiahui Liu
  • Changxin GaoEmail author
  • Nong Sang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11858)


Recent work has made considerable progress in exploring contextual information for human parsing with the Fully Convolutional Network framework. However, there still exist two challenges: (1) inherent relative relationships between parts; (2) scale variation of human parts. To tackle both problems, we propose a Graph-Based Scale-Aware Network for human parsing. First, we embed a Graph-Based Part Reasoning Layer into the backbone network to reason the relative relationship between human parts. Then we construct a Scale-Aware Context Embedding Layer, which consists of two branches to capture scale-specific contextual information, with different receptive fields and scale-specific supervisions. In addition, we adopt an edge supervision to further improve the performance. Extensive experimental evaluations demonstrate that the proposed model performs favorably against the state-of-the-art human parsing methods. More specifically, our algorithm achieves 53.32% (mIoU) on the LIP dataset.


Human parsing Segmentation Graph-based reasoning Scale-aware embedding 



This work was supported by the Project of the National Natural Science Foundation of China (No. 61876210), and Natural Science Foundation of Hubei Province (No. 2018CFB426).


  1. 1.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. TPAMI 39(12), 2481–2495 (2017)CrossRefGoogle Scholar
  2. 2.
    Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: CVPR (2005)Google Scholar
  3. 3.
    Chen, L.-C., Barron, J.T., Papandreou, G., Murphy, K., Yuille, A.L.: Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. In: CVPR (2016)Google Scholar
  4. 4.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. In: ICLR (2015)Google Scholar
  5. 5.
    Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. TPAMI 40(4), 834–848 (2017)CrossRefGoogle Scholar
  6. 6.
    Chen, L.-C., Yang, Y., Wang, J., Xu, W., Yuille, A.L.: Attention to scale: scale-aware semantic image segmentation. In CVPR (2016)Google Scholar
  7. 7.
    Chen, Y., Rohrbach, M., Yan, Z., Yan, S., Kalantidis, Y.: Graph-based global reasoning networks. In: CVPR (2019)Google Scholar
  8. 8.
    Fu, J., Zheng, H., Mei, T.: Look closer to see better: recurrent attention convolutional neural network for fine-grained image recognition. In: CVPR (2017)Google Scholar
  9. 9.
    Fu, J., Liu, J., Tian, H., Fang, Z., Lu, H.: Dual attention network for scene segmentation. In: CVPR (2019)Google Scholar
  10. 10.
    Gan, C., Lin, M., Yang, Y., de Melo, G., Hauptmann, A.G.: Concepts not alone: exploring pairwise relationships for zero-shot video activity recognition. AAAI Press (2016)Google Scholar
  11. 11.
    Gong, K., Liang, X., Zhang, D., Shen, X., Lin, L.: Look into person: self-supervised structure-sensitive learning and a new benchmark for human parsing. In: CVPR (2017)Google Scholar
  12. 12.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
  13. 13.
    Jégou, S., Drozdzal, M., Vazquez, D., Romero, A., Bengio, Y.: The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation. In: CVPR (2017)Google Scholar
  14. 14.
    Liang, X., Gong, K., Shen, X., Lin, L.: Look into person: joint body parsing & pose estimation network and a new benchmark. TPAMI 41(4), 871–885 (2018)CrossRefGoogle Scholar
  15. 15.
    Liang, X., Lin, L., Wei, Y., Shen, X., Yang, J., Yan, S.: Proposal-free network for instance-level object segmentation. TPAMI 40(12), 2978–2991 (2017)CrossRefGoogle Scholar
  16. 16.
    Liang, X., et al.: Human parsing with contextualized convolutional neural network. In: ICCV (2015)Google Scholar
  17. 17.
    Lin, G., Milan, A., Shen, C., Reid, I.: RefineNet: multi-path refinement networks for high-resolution semantic segmentation. In: CVPR (2017)Google Scholar
  18. 18.
    Liu, T., et al.: Devil in the details: towards accurate single and multiple human parsing. In: AAAI (2019)Google Scholar
  19. 19.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)Google Scholar
  20. 20.
    Mostajabi, M., Yadollahpour, P., Shakhnarovich, G.: Feedforward semantic segmentation with zoom-out features. In: CVPR (2015)Google Scholar
  21. 21.
    Park, S., Nie, B.X., Zhu, S.-C.: Attribute and-or grammar for joint parsing of human pose, parts and attributes. TPAMI 40(7), 1555–1569 (2017)CrossRefGoogle Scholar
  22. 22.
    Paszke, A., et al.: Automatic differentiation in PyTorch. In: NIPS (2017)Google Scholar
  23. 23.
    Wang, X., Girshick, R., Gupta, A., He, K.: Non-local neural networks. In: CVPR (2018)Google Scholar
  24. 24.
    Xia, F., Wang, P., Chen, X., Yuille, A.L.: Joint multi-person pose estimation and semantic part segmentation. In: CVPR (2017)Google Scholar
  25. 25.
    Xia, F., Zhu, J., Wang, P., Yuille, A.L.: Pose-guided human parsing by an and/or graph using pose-context features. In: AAAI (2016)Google Scholar
  26. 26.
    Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: ECCV (2018)Google Scholar
  27. 27.
    Yu, C., Wang, J., Peng, C., Gao, C., Yu, G., Sang, N.: Learning a discriminative feature network for semantic segmentation. In: CVPR (2018)Google Scholar
  28. 28.
    Zhao, H., Shi, J., Qi, X., Wang, X., Jia, J.: Pyramid scene parsing network. In: CVPR (2017)Google Scholar
  29. 29.
    Zhao, J., et al.: Self-supervised neural aggregation networks for human parsing. In: CVPR (2017)Google Scholar
  30. 30.
    Zhu, S., Urtasun, R., Fidler, S., Lin, D., Change Loy, C.: Be your own prada: fashion synthesis with structural coherence. In ICCV (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Key Laboratory of Ministry of Education for Image Processing and Intelligent Control, School of Artificial Intelligence and AutomationHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations