A Category-Level 3D Object Dataset: Putting the Kinect to Work

  • Allison Janoch
  • Sergey Karayev
  • Yangqing Jia
  • Jonathan T. Barron
  • Mario Fritz
  • Kate Saenko
  • Trevor Darrell
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)

Abstract

The recent proliferation of the Microsoft Kinect, a cheap but quality depth sensor, has brought the need for a challenging category-level 3D object detection dataset to the forefront. Such a dataset can be used for object recognition in a spirit usually reserved for the large collections of intensity images typically collected from the Internet. Here, we will review current 3D datasets and find them lacking in variation of scene, category, instance, and viewpoint. The Berkeley 3D Object Dataset (B3DO), which contains color and depth image pairs gathered in read domestic and office environments will be presented. Baseline object recognition performance in a PASCAL VOC-style detection task is established, and two ways that inferred world size of the object van be used to improve detection are suggested. In an effort to make more significant performance progress, the problem of extracting useful features from range images is addressed. There has been much success in using the histogram of oriented gradients (HOG) as a global descriptor for object detection in intensity images. There are also many proposed descriptors designed specifically for depth data (spin images, shape context, etc.), but these are often focused on the local, not global descriptor paradigm. We explore the failures of gradient-based descriptors when applied to depth, and propose that the proper global descriptor in the realm of 3D should be based on curvature, not gradients.

References

  1. 1.
    Ford campus vision and Lidar dataset. http://robots.engin.umich.edu/Downloads
  2. 2.
  3. 3.
  4. 4.
    Introducing Kinect for Xbox 360. http://www.xbox.com/en-US/Kinect/ (2011)
  5. 5.
    Alexe, B., Deselaers, T., Ferrari, V.: What is an object? In: IEEE Conference on Computer Vision and Pattern Recognition (2010) Google Scholar
  6. 6.
    Besl, P.J., Jain, R.C.: Segmentation through variable-order surface fitting. IEEE Trans. Pattern Anal. Mach. Intell. 10 (1988). doi:10.1109/34.3881
  7. 7.
    Bradski, G.: The OpenCV library. Dr. Dobb’s Journal of Software Tools (2000) Google Scholar
  8. 8.
    Browatzki, B., Fischer, J., Birgit, G., Bulthoff, H., Wallraven, C.: Going into depth: evaluating 2d and 3d cues for object classification on a new, large-scale object dataset. In: International Conference on Computer Vision—Workshop on Consumer Depth Cameras for Computer Vision (2011) Google Scholar
  9. 9.
  10. 10.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Conference on Computer Vision and Pattern Recognition (2005) Google Scholar
  11. 11.
    Ess, A., Schindler, K., Leibe, B., Gool, L.V.: Object detection and tracking for autonomous navigation in dynamic environments. Int. J. Robot. Res. (2010). doi:10.1177/0278364910365417 Google Scholar
  12. 12.
    Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2010 (VOC2010) Results. http://www.pascal-network.org/challenges/VOC/voc2010/workshop/index.html
  13. 13.
    Felzenszwalb, P.F., Girschick, R.B., McAllester, D.: Cascade object detection with deformable part models. In: IEEE Conference on Computer Vision and Pattern Recognition (2010) Google Scholar
  14. 14.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. (2009). doi:10.1109/TPAMI.2009.167 Google Scholar
  15. 15.
    Fritz, M., Saenko, K., Darrell, T.: Size matters: metric visual search constraints from monocular metadata. In: Advances in Neural Information Processing Systems (2010) Google Scholar
  16. 16.
    Frome, A., Huber, D., Kolluri, R., Bulow, T., Malik, J.: Recognizing objects in range data using regional point descriptors. In: European Conference on Computer Vision (2004) Google Scholar
  17. 17.
    Gould, S., Baumstarck, P., Quigley, M., Ng, A.Y., Koller, D.: Integrating visual and range data for robotic object detection. In: European Conference on Computer Vision—Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (2008) Google Scholar
  18. 18.
    Grimson, W.: From Images to Surfaces: A Computational Study of the Human Early Visual System. MIT Press, Cambridge (1981) Google Scholar
  19. 19.
    Hattori, H., Seki, A., Nishiyama, M., Watanabe, T.: Stereo-based pedestrian detection using multiple patterns. In: British Machine Vision Conference (2009) Google Scholar
  20. 20.
    Helmer, S., Meger, D., Muja, M., Little, J.J., Lowe, D.G.: Multiple viewpoint recognition and localization. In: Asian Conference on Computer Vision (2010) Google Scholar
  21. 21.
    Janoch, A., Karayev, S., Jia, Y., Barron, J.T., Fritz, M., Saenko, K., Darrell, T.: A category-level 3-D object dataset: putting the Kinect to work. In: International Conference on Computer Vision—Workshop on Consumer Depth Cameras for Computer Vision (2011) Google Scholar
  22. 22.
    Johnson, A., Hebert, M.: Using spin images for efficient object recognition in cluttered 3d scenes. IEEE Trans. Pattern Anal. Mach. Intell. 21(5), 433–449 (1999) CrossRefGoogle Scholar
  23. 23.
    Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: International Conference on Robotics and Automation (2011) Google Scholar
  24. 24.
    Rohrbach, M., Enzweiler, M., Gavrila, D.M.: High-level fusion of depth and intensity for pedestrian classification. In: Annual Symposium of German Association for Pattern Recognition (2009) Google Scholar
  25. 25.
    Rusu, R.B., Bradski, G., Thibaux, R., Hsu, J.: Fast 3d recognition and pose using the viewpoint feature histogram. In: International Conference on Intelligent Robots and Systems (2010) Google Scholar
  26. 26.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: IEEE Conference on Computer Vision and Pattern Recognition (2011) Google Scholar
  27. 27.
    Silberman, N., Fergus, R.: Indoor scene segmentation using a structured light sensor. In: International Conference on Computer Vision—Workshop on 3D Representation and Recognition (2011) Google Scholar
  28. 28.
    Sun, M., Bradski, G., Xu, B.X., Savarese, S.: Depth-encoded hough voting for joint object detection and shape recovery. In: European Conference on Computer Vision (2010) Google Scholar
  29. 29.
    Walk, S., Schindler, K., Schiele, B.: Disparity statistics for pedestrian detection: combining appearance, motion and stereo. In: European Conference on Computer Vision (2010) Google Scholar
  30. 30.
    Wang, X., Han, T.X., Yan, S.: An HOG-LBP human detector with partial occlusion handling. In: International Conference on Computer Vision (2009) Google Scholar
  31. 31.
    Woodford, O., Torr, P., Reid, I., Fitzgibbon, A.: Global stereo reconstruction under second-order smoothness priors. IEEE Trans. Pattern Anal. Mach. Intell. (2009). doi:10.1109/ICCV.2009.5459207 Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Allison Janoch
    • 1
  • Sergey Karayev
    • 1
  • Yangqing Jia
    • 1
  • Jonathan T. Barron
    • 1
  • Mario Fritz
    • 2
  • Kate Saenko
    • 1
  • Trevor Darrell
    • 1
  1. 1.University of California at BerkeleyBerkeleyUSA
  2. 2.Max Plank Institute for InformaticsSaarbrückenGermany

Personalised recommendations