Geometric and Photometric Analysis for Interactively Recognizing Multicolor or Partially Occluded Objects

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3804)


An effective human-robot interaction is essential for wide penetration of service robots into the market. Such robots need vision systems to recognize objects. It is, however, difficult to realize vision systems that can work in various conditions. More robust techniques of object recognition and image segmentation are essential. Thus, we have proposed to use the human user’s assistance for objects recognition through speech. Our previous system assumes that it can segment images without failure. However, if there are occluded objects and/or objects composed of multicolor parts, segmentation failures cannot be avoided. This paper presents an extended system that can recognize objects in occlusion and/or multicolor cases using geometric and photometric analysis of images. If the robot is not sure about the segmentation results, it asks questions of the user by appropriate expressions depending on the certainty to remove the ambiguity.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ehrenmann, M., Zollner, R., Rogalla, O., Dillmann, R.: Programming service tasks in household environments by human demonstration. In: ROMAN 2002, pp. 460–467 (2002)Google Scholar
  2. 2.
    Hans, M., Graf, B., Schraft, R.D.: Robotics home assistant care-o-bot: past-present-future. In: ROMAN 2002, pp. 380–385 (2002)Google Scholar
  3. 3.
    Berry, G.A., Pavlovic, V., Huang, T.S.: BattleView: a multimodal HCI research application. In: Workshop on Perceptual User Interfaces, pp. 67–70 (1998)Google Scholar
  4. 4.
    Raisamo, R.: A multimodal user interface for public information kiosks. In: Workshop on Perceptual User Interfaces, pp. 7–12 (1998)Google Scholar
  5. 5.
    Takahashi, T., Nakanishi, S., Kuno, Y., Shirai, Y.: Human-robot interface by verbal and nonverbal communication. In: IROS 1998, pp. 924–929 (1998)Google Scholar
  6. 6.
    Yoshizaki, M., Kuno, Y., Nakamura, A.: Mutual assistance between speech and vision for human-robot interface. In: IROS 2002, pp. 1308–1313 (2002)Google Scholar
  7. 7.
    Kurnia, R., Hossain, M.A., Nakamura, A., Kuno, Y.: Object recognition through human-robot interaction by speech. In: ROMAN 2004, pp. 619–624 (2004)Google Scholar
  8. 8.
    Takizawa, M., Makihara, Y., Shimada, N., Miura, J., Shirai, Y.: A service robot with interactive vision- objects recognition using dialog with user. In: First International Workshop on Language Understanding and Agents for Real World Interaction, Hokkaido (2003)Google Scholar
  9. 9.
    Inamura, T., Inaba, M., Inoue, H.: Dialogue control for task achievement based on evaluation of situational vagueness and stochastic representation of experiences. In: International Conference on Intelligent Robots and Systems, Sendai, pp. 2861–2866 (2004)Google Scholar
  10. 10.
    Cremers, A.: Object reference in task-oriented keyboard dialogues, multimodal human-computer communication: system, techniques and experiments, pp. 279–293. Springer, Heidelberg (1998)Google Scholar
  11. 11.
    Winograd, T.: Understanding natural language. Academic Press, New York (1972)Google Scholar
  12. 12.
    Roy, D., Schiele, B., Pentland, A.: Learning audio-visual associations using mutual information. In: ICCV, Workshop on Integrating Speech and Image Understanding, Greece (1999)Google Scholar
  13. 13.
    Hossain, M.A., Kurnia, R., Nakamura, A., Kuno, Y.: Color objects segmentation for helper robot. In: ICECE 2004, pp. 206–209 (2004)Google Scholar
  14. 14.
    Nayar, S.K., Bolle, R.M.: Reflectance based object recognition. Inter. Journal of Computer Vision 17(3), 219–240 (1996)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  1. 1.Department of Information and Computer SciencesSaitama UniversitySaitamaJapan

Personalised recommendations