Advertisement

Wuhan University Journal of Natural Sciences

, Volume 24, Issue 4, pp 360–368 | Cite as

3D Object Detection Incorporating Instance Segmentation and Image Restoration

  • Bo HuangEmail author
  • Man Huang
  • Yongbin Gao
  • Yuxin Yu
  • Xiaoyan Jiang
  • Juan Zhang
Computer Science
  • 40 Downloads

Abstract

Nowadays, 3D object detection, which uses the color and depth information to find object localization in the 3D world and estimate their physical size and pose, is one of the most important 3D perception tasks in the field of computer vision. In order to solve the problem of mixed segmentation results when multiple instances appear in one frustum in the F-PointNet method and in the occlusion that leads to the loss of depth information, a 3D object detection approach based on instance segmentation and image restoration is proposed in this paper. Firstly, instance segmentation with Mask R-CNN on an RGB image is used to avoid mixed segmentation results. Secondly, for the detected occluded objects, we remove the occluding object first in the depth map and then restore the empty pixel region by utilizing the Criminisi Algorithm to recover the missing depth information of the object. The experimental results show that the proposed method improves the average precision score compared with the F-PointNet method.

Key words

image processing 3D object detection instance segmentation depth information image restoration 

CLC number

TP 314 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    Hoiem D, Efros A A, Hebert M. Geometric context from a single image [C] // IEE International Conference on Computer Vision 2005. Washington D C: IEEE, 2005, 1: 654–661.Google Scholar
  2. [2]
    Hoiem D, Efros A A, Hebert M. Putting objects in perspective[J]. International Journal Computer Vision, 2008, 80(1): 3–15.CrossRefGoogle Scholar
  3. [3]
    Malisiewicz T, Gupta A, Efros A A. Ensemble of exemplar-SVMs for object detection and beyond [C]// IEE International Conference on Computer Vision. Washington D C: IEEE, 2011, 1(2): 6.Google Scholar
  4. [4]
    Gupta S, Girshick R, Arbeláez P, et al. Learning rich features from RGB-D images for object detection and segmentation [C]// 13th European Conference on Computer Vision. Berlin: Springer-Verlag, 2014, 8695: 345–360.Google Scholar
  5. [5]
    Deng Z, Latecki L J. Amodal detection of 3D objects: Inferring 3D bounding boxed from 2D ones in RGB-depth images [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2017: 1063–6919.Google Scholar
  6. [6]
    Song S, Xiao J. Deep sliding shapes for amodal 3D object detection in RGB-D images[J]. Computer Science, 2015, 139(2): 808–816.Google Scholar
  7. [7]
    Qi C R, Liu W, Wu C, et al. Frustum pointnets for 3D object detection from RGB-D data [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2018: 918–927.Google Scholar
  8. [8]
    He K, Gkioxari G, Dollar P, et al. Mask R-CNN [C]// Proceedings of the IEEE International Conference on Computer Vision. Washington D C: IEEE, 2017: 2961–2969.Google Scholar
  9. [9]
    Chen X, Kundu K, Zhang Z, et al. Monocular 3D object detection for autonomous driving[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2016, 1: 2147–2156.Google Scholar
  10. [10]
    Girshick R, Donahue J, Darrell T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2014: 580–587.Google Scholar
  11. [11]
    Chen X, Ma H, Wan J, et al. Multi-view 3D object detection network for autonomous driving [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2017: 1907–1915.Google Scholar
  12. [12]
    Lin D, Fidler S, Urtasun R. Holistic scene understanding for 3D object detection with RGB-D cameras [C]// Proceedings of the IEEE International Conference on Computer Vision. Washington D C: IEEE, 2013, 1: 1417–1424.Google Scholar
  13. [13]
    Kim B S, Xu S, Savarese S. Accurate localization of 3D objects from RGB-D data using segmentation hypotheses [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2013, 1: 3182–3189.Google Scholar
  14. [14]
    Ren Z, Sudderth E B. Three-dimensional object detection and layout prediction using clouds of oriented gradients [C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Washington D C: IEEE, 2016: 1525–1533.Google Scholar
  15. [15]
    Criminisi A, Pérez P, Toyama K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9): 1200–1212.CrossRefPubMedGoogle Scholar
  16. [16]
    Qi C R, Su H, Mo K, et al. Pointnet: Deep learning on point sets for 3D classification and segmentation[J]. Proc Computer Vision and Pattern Recognition (CVPR), 2017, 1(2): 4.Google Scholar
  17. [17]
    Song S, Xiao J. Sliding shapes for 3D object detection in depth images [C]// 2014 European Conference on Computer Vision. Washington D C: IEEE, 2014: 634–651.Google Scholar
  18. [18]
    Chen X, Kundu K, Zhu Y, et al. 3D object proposals for accurate object class detection [C]// Advances in Neural Information Processing Systems. Massachusetts: MIT Press, 2015: 424–432.Google Scholar
  19. [19]
    Li B. 3D fully convolutional network for vehicle detection in point cloud [C]//Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on IEEE. Washington D C: IEEE, 2017: 1513–1518.CrossRefGoogle Scholar
  20. [20]
    Qi C R, Yi L, Su H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems. Massachusetts: MIT Press, 2017: 5099–5108.Google Scholar

Copyright information

© Wuhan University and Springer-Verlag GmbH Germany 2019

Authors and Affiliations

  • Bo Huang
    • 1
    • 2
    Email author
  • Man Huang
    • 1
  • Yongbin Gao
    • 1
  • Yuxin Yu
    • 3
  • Xiaoyan Jiang
    • 1
  • Juan Zhang
    • 1
  1. 1.Department of Electrical and Electronic EngineeringShanghai University of Engineering ScienceShanghaiChina
  2. 2.Collaborative Innovation Center for Economics Crime Investigation and Prevention TechnologyNanchang, JiangxiChina
  3. 3.School of Economics and FinanceShanghai International Studies UniversityShanghaiChina

Personalised recommendations