Skip to main content

Advertisement

SpringerLink
Log in

Multi-sensor system for detection and classification of human activities

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

This paper describes a novel system for detecting and classifying human activities based on a multi-sensor approach. The aim of this research is to create a loosely structured environment, where activity is constantly monitored and automatically classified, transparently to the subjects who are observed. The system uses four calibrated cameras installed in the room which is being monitored and a body-mounted wireless accelerometer on each person, exploiting the features of different sensors to maximize recognition accuracy, improve scalability and reliability. The algorithms on which the system is based, as well as its structure, are aimed at analyzing and classifying complex movements (like walking, sitting, jumping, running, falling, etc.) of potentially multiple people at the same time. Here, we describe a preliminary application, in which action classification is mostly aimed at detecting falls. Several instances of a hybrid classifier based on Support Vector Machines and Hierarchical Temporal Memories, a recent bio-inspired computational paradigm, are used to detect potentially dangerous activities of each person in the environment. If such an activity is detected and if the person “in danger” is wearing the accelerometer, the system localizes and activates it to receive data and then performs a more reliable fall detection using a specifically trained classifier. The opportunity to turn on the accelerometer on-demand makes it possible to extend its battery life. Besides and beyond surveillance, this system could also be used for the assessment of the degree of independence of elderly people or, in rehabilitation, to assist patients during recovery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. http://www.ptgrey.com/products/fireflymv/fireflymv.pdf equipped with lenses with an aperture of F1.4 and focal length of 2.8 mm.

  2. The calibration toolbox is available at http://www.vision.caltech.edu/bouguetj/calib_doc.

  3. Wireless Sensor Network Platform: see http://wisnp.henesis.eu.

References

  • Burges CJ (1998) A tutorial on Support Vector Machines for pattern recognition. Data Min Knowl Discov 2:121–167

    Article  Google Scholar 

  • Csapo A, Baranyi P, Tikk D (2007) Object categorization using VFA-generated nodemaps and Hierarchical Temporal Memories. In: IEEE International Conference on Computational Cybernetics, ICCC 2007, pp 257–262

  • Cucchiara R, Grana C, Piccardi M, Prati A, Sirotti S (2001) Improving shadow suppression in moving object detection with HSV color information. In: Proceedings of IEEE Intelligent Transportation Systems, 2001, pp 334–339

  • Farahmand N, Dezfoulian M, GhiasiRad H, Mokhtari A, Nouri A (2009) Online temporal pattern learning. In: Proceedings of International Joint Conference on Neural Networks, IJCNN 2009, pp 797–802

  • George D, Hawkins J (2009) Towards a mathematical theory of cortical micro-circuits. PLoS Comput Biol 5(10). doi:10.1371/journal.pcbi.1000532

  • George D, Jaros B (2007) The HTM learning algorithms. Technical report, Numenta inc. http://www.numenta.com/htm-overview/education/Numenta_HTM_Learning_Algos.pdf. Accessed 16th Aug 2011

  • Grossmann R, Blumenthal J, Golatowski F, Timmermann D (2007) Localization in Zigbee-based sensor networks. In: Proceedings of 1st European ZigBee Developers Conference, EuZDC 2007

  • Hawkins J, Blakeslee S (2004) On intelligence. In: Times books, New York

  • Hsu CW, Chang CC, Lin CJ (2003) A practical guide to support vector classification. Technical report. Department of Computer Science, National Taiwan University, Taipei

  • Ji X, Liu H (2010) Advances in view-invariant human motion analysis: a review. IEEE Trans Syst Man Cybern Part C 40:13–24

    Google Scholar 

  • Kellokumpu V, Pietikinen M, Heikkil J (2005) Human activity recognition using sequences of postures. In: Proceedings of IAPR Conference on Machine Vision Applications. MVA 2005, pp 570–573

  • Kern N, Schiele B, Schmidt A (2003) Multi-sensor activity context detection for wearable computing. In: Proceedings of EUSAI, pp 220–232

  • Khan A, Lee YK, Lee S, Kim TS (2010) A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans Inf Technol Biomed 14(5):1166–1172

    Article  Google Scholar 

  • Kim Y, Ling H (2009) Human activity classification based on micro-Doppler signatures using a support vector machine. IEEE Trans Geosci Remote Sens 47(5):1328–1337

    Article  Google Scholar 

  • Leone A, Diraco G, Distante C, Siciliano P, Malfatti M, Gonzo L, Grassi M, Lombardi A, Rescio G, Malcovati P, Libal V, Huang J, Potamianos G (2008) A multi-sensor approach for people fall detection in home environment. In: Cavallaro A, Aghajan H (eds) Workshop on multi-camera and multi-modal sensor fusion algorithms and applications, M2SFA2 2008, Marseille France

  • Lester J, Choudhury T, Kern N, Borriello G, Hannaford B (2005) A hybrid discriminative/generative approach for modeling human activities. In: Proceedings of International Joint Conference on Artificial Intelligence, IJCAI, pp 766–772

  • Lihan M, Tsuchiya T, Koyanagi K (2008) Orientation-aware indoor localization path loss prediction model for wireless sensor networks. In: Network-based information systems, Springer, pp 169–178

  • Luo R, Chen O, Lin CW (2010) Indoor human monitoring system using wireless and pyroelectric sensory fusion system. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2010, pp 1507–1512

  • Monekosso D, Remagnino P, Kuno Y (eds) (2009) Intelligent environments—methods, algorithms and applications. Springer, Berlin

  • Mountcastle V (1978) An organizing principle for cerebral function: the unit model and the distributed system. In: Edelman G, Mountcastle V (eds) The mindful brain. MIT Press, MA

  • Niebles JC, Fei-Fei L (2007) A hierarchical model of shape and appearance for human action classification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE Computer Society, Los Alamitos, CA, pp 1–8

  • Rashidi P, Cook D, Holder L, Schmitter-Edgecombe M (2011) Discovering activities to recognize and track in a smart environment. IEEE Trans Knowl Data Eng 23(4):527–539

    Article  Google Scholar 

  • Rutishauser U, Joller J, Douglas R (2005) Control and learning of ambience by an intelligent building. IEEE Trans Syst Man Cybern Part A Syst Humans 35(1):121–132

    Article  Google Scholar 

  • Sassi F, Ascari L, Cagnoni S (2009) Classifying human body acceleration patterns using a hierarchical temporal memory. In: Proceedings of the XIth International Conference of the Italian Association for Artificial Intelligence: emergent perspectives in artificial intelligence. Springer, AI*IA ’09, pp 496–505

  • Schuldt C, Laptev I, Caputo B (2004) Recognizing human actions: a local SVM approach. In: Proceedings of the 17th International Conference on Pattern Recognition, ICPR 2004, vol 3, pp 32–36

  • Shi J, Tomasi C (1994) Good Features to Track. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR94), pp 593–600

  • Turaga P, Chellappa R, Subrahmanian VS, Udrea O (2008) Machine recognition of human activities: a survey. IEEE Trans Circuits Syst Video Technol 18(11):1473–1488

    Article  Google Scholar 

  • van Doremalen J, Lou B (2008) Spoken digit recognition using a hierarchical temporal memory. In: Proceedings of Interspeech 2008. Brisbane, Australia, pp 2566–2569

  • Ward JA, Lukowicz P, Troster G, Starner TE (2006) Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans Pattern Anal Mach Intell 28:1553–1567

    Article  Google Scholar 

  • Wu X, Ou Y, Qian H, Xu Y (2005) A detection system for human abnormal behavior. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2005, pp 1204–1208

  • Zhang S, Ang M, Xiao W, Tham C (2008) Detection of activities for daily life surveillance: eating and drinking. In: 10th International Conference on e-health Networking, Applications and Services, HealthCom 2008, pp 171–176

  • Zhu C, Sheng W (2009) Multi-sensor fusion for human daily activity recognition in robot-assisted living. In: Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, ACM, New York, NY, USA, HRI ’09, pp 303–304

Download references

Acknowledgments

We would like to thank Gabriele Camellini for the great effort spent in the hardware setup of the environment and for the precious work on the wireless module-based localization. Moreover, we would like to thank Henesis s.r.l. for providing us with the wireless modules and the fellow VisLab laboratory for the cameras.

Author information

Authors and Affiliations

  1. Department of Information Engineering, Intelligent Bio-Inspired Systems (IBIS) Laboratory, University of Parma, Via G.P. Usberti 181/a, 43100, Parma, Italy

    Roberto Ugolotti, Federico Sassi, Monica Mordonini & Stefano Cagnoni

  2. Henesis s.r.l., Viale Dei Mille, 108, 43125, Parma, Italy

    Federico Sassi

Authors
  1. Roberto Ugolotti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Federico Sassi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Monica Mordonini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stefano Cagnoni
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Federico Sassi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ugolotti, R., Sassi, F., Mordonini, M. et al. Multi-sensor system for detection and classification of human activities. J Ambient Intell Human Comput 4, 27–41 (2013). https://doi.org/10.1007/s12652-011-0065-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-011-0065-z

Keywords

Search

Navigation