Personal and Ubiquitous Computing

, Volume 11, Issue 4, pp 265–271 | Cite as

Collaborative capturing, interpreting, and sharing of experiences

  • Yasuyuki SumiEmail author
  • Sadanori Ito
  • Tetsuya Matsuguchi
  • Sidney Fels
  • Shoichiro Iwasawa
  • Kenji Mase
  • Kiyoshi Kogure
  • Norihiro Hagita
Original Article


This paper proposes a notion of interaction corpus, a captured collection of human behaviors and interactions among humans and artifacts. Digital multimedia and ubiquitous sensor technologies create a venue to capture and store interactions that are automatically annotated. A very large-scale accumulated corpus provides an important infrastructure for a future digital society for both humans and computers to understand verbal/non-verbal mechanisms of human interactions. The interaction corpus can also be used as a well-structured stored experience, which is shared with other people for communication and creation of further experiences. Our approach employs wearable and ubiquitous sensors, such as video cameras, microphones, and tracking tags, to capture all of the events from multiple viewpoints simultaneously. We demonstrate an application of generating a video-based experience summary that is reconfigured automatically from the interaction corpus.


Interaction corpus Experience capturing Ubiquitous sensors 



We thank our colleagues at ATR for their valuable discussion and help on the experiments described in this paper. Valuable contributions to the systems described in this paper were made by Tetsushi Yamamoto and Atsushi Nakahara. We also would like to thank Yasuyhiro Katagiri for his continuing support of our research. This research was supported in part by National Institute of Information and Communications Technology.


  1. 1.
    Weiser M (1991) The computer for the 21st century. Sci Am 265(30):94–104CrossRefGoogle Scholar
  2. 2.
    Stiefelhagen R, Yang J, Waibel A (1999) Modeling focus of attention for meeting indexing. In: ACM multimedia ’99. ACM, New York, pp 3–10Google Scholar
  3. 3.
    Kanda T, Ishiguro H, Imai M, Ono T, Mase K (2002) A constructive approach for developing interactive humanoid robots. In: 2002 IEEE/RSJ international conference on intelligent robots and systems (IROS 2002), pp 1265–1270Google Scholar
  4. 4.
    Pentland A (1996) Smart rooms. Sci Am 274(4):68–76CrossRefGoogle Scholar
  5. 5.
    Brooks RA, Coen M, Dang D, De Bonet J, Kramer J, Lozano-Pérez T, Mellor J, Pook P, Stauffer C, Stein L, Torrance M, Wessler M (1997) The intelligent room project. In: Proceedings of the 2nd international cognitive technology conference (CT’97). IEEE, New York, pp 271–278Google Scholar
  6. 6.
    Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt E, Startner TE, Newstetter W (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of CoBuild’99. Springer LNCS1670, pp 190–197Google Scholar
  7. 7.
    Bobick AF, Intille SS, Davis JW, Baird F, Pinhanez CS, Campbell LW, Ivanov YA, Schütte A, Wilson A (1999) The KidsRoom: a perceptually-based interactive and immersive story environment. Presence 8(4):369–393CrossRefGoogle Scholar
  8. 8.
    Brumitt B, Meyers B, Krumm J, Kern A, Shafer S (2000) EasyLiving: technologies for intelligent environments. In: Proceedings of HUC 2000. Springer LNCS1927, pp 12–29Google Scholar
  9. 9.
    Mann S (1998) Humanistic computing: “WearComp” as a new framework for intelligence signal processing. Proc IEEE 86(11):2123–2151CrossRefGoogle Scholar
  10. 10.
    Kawamura T, Kono Y, Kidode M (2002) Wearable interfaces for a video diary: towards memory retrieval, exchange, and transportation. In: The 6th international symposium on wearable computers (ISWC2002). IEEE, New York, pp 31–38Google Scholar
  11. 11.
    Chiu P, Kapuskar A, Reitmeier S, Wilcox L (1999) Meeting capture in a media enriched conference room. In: Proceedings of CoBuild’99. Springer LNCS1670, pp 79–88Google Scholar

Copyright information

© Springer-Verlag London Limited 2006

Authors and Affiliations

  • Yasuyuki Sumi
    • 1
    • 2
    Email author
  • Sadanori Ito
    • 2
    • 3
    • 4
  • Tetsuya Matsuguchi
    • 5
  • Sidney Fels
    • 6
  • Shoichiro Iwasawa
    • 2
    • 4
  • Kenji Mase
    • 2
    • 4
    • 7
  • Kiyoshi Kogure
    • 2
  • Norihiro Hagita
    • 4
  1. 1.Graduate School of InfomaticsKyoto UniversityKyotoJapan
  2. 2.ATR Media Information Science LaboratoriesKyotoJapan
  3. 3.Graduate School of EngineeringTokyo University of Agriculture and TechnologyTokyoJapan
  4. 4.ATR Intelligent Robotics and Communication LaboratoriesKyotoJapan
  5. 5.University of CaliforniaSan FranciscoUSA
  6. 6.The University of British ColumbiaVancouverCanada
  7. 7.Information Technology CenterNagoya UniversityNagoyaJapan

Personalised recommendations