Automatic Image Collection of Objects with Similar Function by Learning Human Grasping Forms

  • Shinya Morioka
  • Tadashi MatsuoEmail author
  • Yasuhiro Hiramoto
  • Nobutaka Shimada
  • Yoshiaki Shirai
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8869)


This paper proposes an automatic functional object segmentation method based on modeling the relationship between grasping hand form and the object appearance. First the relationship among a representative grasping pattern and a position and pose of a object relative to the hand is learned based on a few typical functional objects. By learning local features from the hand grasping various tools with various way to hold them, the proposed method can estimate the position, scale, direction of the hand and the region of the grasped object. By some experiments, we demonstrate that the proposed method can detect them in cluttered backgrounds.


  1. 1.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (surf). Comput. Vis. Image Underst. 110(3), 346–359 (2008). CrossRefGoogle Scholar
  2. 2.
    Bub, D., Masson, M.: Gestural knowledge evoked by objects as part of conceptual representations. Aphasiology 20(9), 1112–1124 (2006). CrossRefGoogle Scholar
  3. 3.
    Filipovych, R., Ribeiro, E.: Recognizing primitive interactions by exploring actor-object states. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–7 (2008)Google Scholar
  4. 4.
    Gibson, J.J.: The Ecological Approach to Visual Perception. Lawrence Erlbaum Associates, Hillsdale (1986). Google Scholar
  5. 5.
    Grabner, H., Gall, J., Van Gool, L.: What makes a chair a chair? In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1529–1536 (2011)Google Scholar
  6. 6.
    Gupta, A., Kembhavi, A., Davis, L.S.: Observing human-object interactions: using spatial and functional compatibility for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(10), 1775–1789 (2009). CrossRefGoogle Scholar
  7. 7.
    Jiang, Y., Lim, M., Saxena, A.: Learning object arrangements in 3D scenes using human context. In: Proceedings of the 29th International Conference on Machine Learning (ICML 2012), pp. 1543–1550 (2012)Google Scholar
  8. 8.
    Kamakura, N., Matsuo, M., Ishii, H., Mitsuboshi, F., Miura, Y.: Patterns of static prehension in normal hands. Am. J. Occup. Ther. 34(7), 437–445 (1980). CrossRefGoogle Scholar
  9. 9.
    Leibe, B., Leonardis, A., Schiele, B.: Combined object categorization and segmentation with an implicit shape model. In: ECCV Workshop on Statistical Learning in Computer Vision, pp. 17–32 (2004).
  10. 10.
    Lepetit, V., Fua, P.: Keypoint recognition using randomized trees. IEEE Trans. Pattern Anal. Mach. Intell. 28(9), 1465–1479 (2006)CrossRefGoogle Scholar
  11. 11.
    Madry, M., Afkham, H.M., Ek, C.H., Carlsson, S., Kragic, D.: Extracting essential local object characteristics for 3D object categorization. In: 2013 Proceedings of IEEE International Conference on Intelligent Robots and Systems, TuAT4.5 2013 (2013)Google Scholar
  12. 12.
    Napier, J.R.: The prehensile movements of the human hand. J. Bone Joint Surg. 38(4), 902–913 (1956)Google Scholar
  13. 13.
    Pieropan, A., Ek, C.H., Kjellström, H.: Functional object descriptors for human activity modeling. In: 2013 Proceedings of International Conference on Robotics and Automation, pp. 1282–1289 (2013)Google Scholar
  14. 14.
    Rivlin, E., Dickinson, S., Rosenfeld, A.: Recognition by functional parts [function-based object recognition]. In: 1994 Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 1994, pp. 267–274 (1994)Google Scholar
  15. 15.
    Schölkopf, B., Platt, J.C., Shawe-Taylor, J.C., Smola, A.J., Williamson, R.C.: Estimating the support of a high-dimensional distribution. Neural Comput. 13(7), 1443–1471 (2001). CrossRefzbMATHGoogle Scholar
  16. 16.
    Sgorbissa, A., Verda, D.: Structure-based object representation and classification in mobile robotics through a microsoft kinect. Robot. Auton. Syst. 61(12), 1665–1679 (2013)CrossRefGoogle Scholar
  17. 17.
    Shotton, J., Blake, A., Cipolla, R.: Contour-based learning for object detection. In: 2005 Tenth IEEE International Conference on Computer Vision, ICCV 2005, vol. 1, pp. 503–510. IEEE (2005)Google Scholar
  18. 18.
    Winn, J., Shotton, J.: The layout consistent random field for recognizing and segmenting partially occluded objects. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 37–44. IEEE (2006)Google Scholar
  19. 19.
    Wu, B., Nevatia, R.: Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors. In: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV 2005), vol. 1, pp. 90–97. IEEE Computer Society, Washington, DC (2005).
  20. 20.
    Wu, J., Osuntogun, A., Choudhury, T., Philipose, M., Rehg, J.: A scalable approach to activity recognition based on object use. In: 2007 IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–8 (2007)Google Scholar
  21. 21.
    Zhang, H., Bai, X., Zhou, J., Cheng, J., Zhao, H.: Object detection via structural feature selection and shape model. IEEE Trans. Image Process. 22(12), 4984–4995 (2013)CrossRefMathSciNetGoogle Scholar
  22. 22.
    Zhang, X., Yang, Y.H., Han, Z., Wang, H., Gao, C.: Object class detection: a survey. ACM Comput. Surv. 46(1), 10:1–10:53 (2013). CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Shinya Morioka
    • 1
  • Tadashi Matsuo
    • 1
    Email author
  • Yasuhiro Hiramoto
    • 1
  • Nobutaka Shimada
    • 1
  • Yoshiaki Shirai
    • 1
  1. 1.Ritsumeikan UniversityShigaJapan

Personalised recommendations