Activity Recognition Using an Egocentric Perspective of Everyday Objects

  • Dipak Surie
  • Thomas Pederson
  • Fabien Lagriffoul
  • Lars-Erik Janlert
  • Daniel Sjölie
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4611)


This paper presents an activity recognition approach based on the tracking of a specific human actor’s current object manipulation actions, complemented by two kinds of situational information: 1) the set of objects that are visually observable (inside the “observable space”) and 2) technically graspable (inside the “manipulable space”). This “egocentric” model is inspired by situated action theory and offers the advantage of not depending on technology for absolute positioning of neither the human nor the objects. Applied in an immersive Virtual Reality environment, the proposed activity recognition approach shows a recognition precision of 89% on the activity-level and 76% on the action-level among 10 everyday home activities.


Activity Recognition Information Channel Object Manipulation Virtual Reality Environment Observable Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Starner, T.: The Challenges of Wearable Computing: Part 1 & 2. IEEE Micro 21(4), 44-52, 54–67 (2001)CrossRefGoogle Scholar
  2. 2.
    Nardi, B. (ed.): Context and Consciousness: Activity Theory and Human-Computer Interaction. MIT Press, Cambridge (1995)Google Scholar
  3. 3.
    Bao, L., Intille, S.: ctivity Recognition from User-Annotated Acceleration Data. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 1–17. Springer, Heidelberg (2004)Google Scholar
  4. 4.
    Lee, S.W., Mase, K.: Activity and location recognition using wearable sensors. IEEE Pervasive Computing 1(3), 24–32 (2002)CrossRefGoogle Scholar
  5. 5.
    Kern, N., Schiele, B., Schmidt, A.: Multi-Sensor Activity Context Detection for Wearable Computing. In: European Symposium on Ambient Intelligence (2003)Google Scholar
  6. 6.
    Chen, J., Kam, A., Zhang, J., Liu, N., Shue, L.: Bathroom Activity Monitoring Based on Sound. In: Gellersen, H.-W., Want, R., Schmidt, A. (eds.) PERVASIVE 2005. LNCS, vol. 3468, pp. 47–61. Springer, Heidelberg (2005)Google Scholar
  7. 7.
    Haritaoglu, I., Harwood, D., Davis, L.: W4: Real-Time Surveillance of People and Their Activities. IEEE Transactions on Pattern Analysis and Machine Intelligence 22(8) (2000)Google Scholar
  8. 8.
    Lukowicz, P., Ward, J., Junker, H., Stäger, M., Tröster, G., Atrash, A., Starner, T.: Recognizing Workshop Activity Using Body Worn Microphones and Accelerometers. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 18–32. Springer, Heidelberg (2004)Google Scholar
  9. 9.
    Lester, J., Choudhury, T., Kern, N., Borriello, G., Hannaford, B.: A Hybrid Discriminative/Generative Approach for Modeling Human Activities. In: 19th International Joint Conference on Artificial Intelligence (2005)Google Scholar
  10. 10.
    Philipose, M., Fishkin, K., Perkowitz, M., Patterson, D., Fox, D., Kautz, H., Hähnel, D.: Inferring Activities from Interactions with Objects. IEEE Pervasive Computing, 50–57 (2004)Google Scholar
  11. 11.
    Patterson, D., Fox, D., Kautz, H., Philipose, M.: Fine-Grained Activity Recognition by Aggregating Abstract Object Usage. In: Ninth IEEE International Symposium on Wearable Computers (2005)Google Scholar
  12. 12.
    Suchman, L.: Plans and situated actions: the problem of human machine interaction. Cambridge University Press, Cambridge (1987)Google Scholar
  13. 13.
    Janlert, L.-E.: Putting Pictures in Context. In: Proceedings of AVI2006, pp. 463–466. ACM Press, New York (2006)Google Scholar
  14. 14.
    Tapia, E., Intille, S., Larson, K.: Activity Recognition in the Home Using Simple and Ubiquitous Sensors. In: Ferscha, A., Mattern, F. (eds.) PERVASIVE 2004. LNCS, vol. 3001, pp. 158–175. Springer, Heidelberg (2004)Google Scholar
  15. 15.
    Pederson, T.: From Conceptual Links to Causal Relations — Physical-Virtual Artefacts in Mixed-Reality Space. PhD thesis, Dept. of Computing Science, Umeå university, report UMINF-03.14 (2003) ISBN 91-7305-556-5Google Scholar
  16. 16.
    Pederson, T.: Magic Touch: A Simple Object Location Tracking System Enabling the Development of Physical-Virtual Artefacts in Office Environments. Journal of Personal and Ubiquitous Computing 5, 54–57 (2001)CrossRefGoogle Scholar
  17. 17.
    Backman, A.: Colosseum3D – Authoring Framework for Virtual Environments. In: Proceedings of EUROGRAPHICS Workshop IPT & EGVE Workshop, pp. 225–226 (2005)Google Scholar
  18. 18.
    Fritzke, B.: A growing neural gas network learns topologies. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems 7, pp. 625–632. MIT Press, Cambridge, MA (1995)Google Scholar
  19. 19.
    Rabiner, L.: A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. Proceedings of the IEEE 77(2) (1989)Google Scholar
  20. 20.
    AMPS. (2nd January, 2007),
  21. 21.
    Finkenzeller, K.: RFID Handbook, 2nd edn. John Wiley and Sons, New York, USA (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Dipak Surie
    • 1
  • Thomas Pederson
    • 1
  • Fabien Lagriffoul
    • 1
  • Lars-Erik Janlert
    • 1
  • Daniel Sjölie
    • 2
  1. 1.Department of Computing Science, Umeå University, SE-901 87 UmeåSweden
  2. 2.VRlab / HPC2N, Umeå University, SE-901 87 UmeåSweden

Personalised recommendations