Skip to main content
SpringerLink
Log in

Infrequent Non-speech Gestural Activity Recognition Using Smart Jewelry: Challenges and Opportunities for Large-Scale Adaptation

  • Chapter
  • First Online:
Handbook of Large-Scale Distributed Computing in Smart Healthcare

Part of the book series: Scalable Computing and Communications ((SCC))

  • 2066 Accesses

Abstract

Wearable Body Area Network (BAN) based activity recognition is one of the fastest growing research areas in activity recognition and context reasoning. However, wearable physical sensor based Infrequent Non-Speech Gestural Activity (IGA) recognition is not well studied problem because IGAs are not directly observable from BAN sensor devices. Due to the recent proliferation of smart jewelries capable of monitoring locomotive and physiological signals from certain specific human body positions which are currently hitherto impossible to measure by traditional fitness and smart wristwatch devices opens up unprecedented research and development opportunities in anatomical gestural activity recognition. Inspired by this, we propose a new wearable smart earring based framework which is capable of differentiating IGAs in a daily environment with a single integrated accelerometer sensor. The natural gestures associated with the first portion of the human alimentary canal, i.e., human mouth can broadly be categorized in two types; frequent (talking, silence etc.) or infrequent (coughing, deglutition, yawning) gestures. Infrequent Gestural Activities (IGAs) help create an abrupt but distinct change in accelerometer sensor signal streams of an earring pertaining to specific activities. Mining and classifying the abrupt changes in sensor signal streams require high sampling frequency which in turn depletes the limited battery life of any smart ornaments. Extending the battery life of smartened designer jewelry requires probing those devices less which in turn prohibits of achieving high precision and recall for non-frequent gestural activity discovery and recognition. In this book chapter, we propose a novel data segmentation technique that harnesses the power of change-point detection algorithm to detect and quantify any abrupt changes in sensor data streams of smart earrings. This helps to distinguish between frequent and infrequent gestural acclivities at a high precision with a low sampling frequency, energy, and computational overhead. Experimental evaluation on one real-time and two publicly available benchmark datasets attests the scalability and adaptation of our techniques for both IGAs and postural activities in large-scale participatory sensing health applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mohammad Arif Ul Alam, Nirmalya Roy, Michelle Petruska, Andrea Zemp, Smart-Energy Group Anomaly Based Behavioral Abnormality Detection, IEEE Wireless Health Conference 2016, WH

    Google Scholar 

  2. Nicholas D. Lane, Petko Georgiev, Lorena Qendro: DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning. UbiComp 2015: 283–294

    Google Scholar 

  3. F. S. Wei, L. C. D. Silva, Speech based emotion classification, International Conference on Electrical Electronic Technology, 2001, 1, 297–301

    Google Scholar 

  4. Oh-Wook Kwon, Kwokleung Chan, Jiucang Hao, Te-Won Lee, Emotion Recognition by Speech Signals, In proceeding of: 8th European Conference on Speech Communication and Technology, EUROSPEECH, 2003, 125–128

    Google Scholar 

  5. T. Rahman, A. Adams, E. Carroll, B. Zhou, H. Peng, Mi Zhang, T. Choudhury, BodyBeat: A Mobile System for Sensing Non-Speech Body Sounds, International Conference on Mobile Systems, Applications and Services (MobiSys), 2014

    Google Scholar 

  6. R. Cowie, E Douglas-Cowie, Automatic statistical analysis of the signal and prosodic signs of emotion in speech, Fourth International Conference on Spoken Language, 1996. ICSLP 96. Proceedings, 1996, 3, 1989–1992

    Google Scholar 

  7. Xia Mao, Bing Zhang, Yi Luo, Multi-level Speech Emotion Recognition Based on HMM and ANN, WRI World Congress on Computer Science and Information Engineering, 2009, 7, 225–229

    Article  Google Scholar 

  8. Xin Min Cheng, Pei Ying Cheng,Li Zhao, A Study on Emotional Feature Analysis and Recognition in Speech Signal, Proceedings of the International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), 2009, 1, 418–420

    Article  Google Scholar 

  9. K. H. Kim, S. W. Bang, S. R. Kim, Emotion recognition system using short-term monitoring of physiological signals, Medical and Biological Engineering and Computing, 2004, 42, 419–427

    Google Scholar 

  10. R. W. Picard , E. Vyzas , J. Healey, Toward machine emotional intelligence: analysis of affective physiological state, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23, 1175–1191

    Article  Google Scholar 

  11. I.A. Essa, A.P. Pentland, Coding, analysis, interpretation, and recognition of facial expressions, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997, 19, 757–763

    Google Scholar 

  12. Ying-Li Tian, T. Kanade, J.F. Cohn, Recognizing action units for facial expression analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23, 97–115

    Google Scholar 

  13. J. Fogarty, C. Au, and S.E. Hudson. Sensing from the basement: a feasibility study of unobtrusive and low-cost home activity recognition. In Proceedings of the 19th annual ACM symposium on User interface software and technology, UIST ’06, pages 91–100, New York, NY, USA, 2006. ACM

    Google Scholar 

  14. J. Rowan and E.D. Mynatt. Digital family portrait field trial: Support for aging in place. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’05, pages 521–530, New York, NY, USA, 2005. ACM

    Google Scholar 

  15. T.V. Duong, H.H. Bui, D.Q. Phung, and S. Venkatesh. Activity recognition and abnormality detection with the switching hidden semi-markov model. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 838–845, June 2005

    Google Scholar 

  16. Mohammad Arif Ul Alam, Nirmalya Roy, GeSmart: A Gestural Activity Recognition Model for Predicting Behavioral Health, International Conference on Smart Computing, SmartComp 2014, Hong Kong

    Google Scholar 

  17. Mohammad Arif Ul Alam, Nirmalya Roy, Aryya Gangopadhyay, Elizabeth Galik, A Smart Segmentation Technique Towards Improved Infrequent Non-Speech Gestural Activity Recognition Model, Pervasive and Mobile Computing (PMC) Special Issue on Gerontechnology

    Google Scholar 

  18. Drew Williams, Mohammad Arif Ul Alam, Sheikh Iqbal Ahamed, William Chu, Considerations in Designing Human-Computer Interfaces for Elderly People, International Conference on Quality Software, 2013, QSIC/SQHE

    Google Scholar 

  19. A. Parate, M. Chiu, C. Chadowitz, D. Ganesan, E. Kalogerakis: RisQ: recognizing smoking gestures with inertial sensors on a wristband. MobiSys 2014

    Google Scholar 

  20. Sougata Sen, Vigneshwaran Subbaraju, Archan Misra, Rajesh Krishna Balan, Youngki Lee: The case for smartwatch-based diet monitoring. PerCom Workshops, WristSense, 2015

    Google Scholar 

  21. Mohammad Arif Ul Alam, Nilavra Pathak, Nirmalya Roy, Mobeacon: An iBeacon-Assisted smart-phone-Based Real Time Activity Recognition Framework, 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, Mobiquitous 2015

    Google Scholar 

  22. Mohammad Arif Ul Alam, Nirmalya Roy, Sarah Holmes, Aryya Gangopadhyay, Elizabeth Galik, Automated Functional and Behavioral Health Assessment of Older Adults with Dementia, IEEE Conference on Connected Health: Applications, Systems and Engineering Technologies, CHASE 2016

    Google Scholar 

  23. Mohammad Arif Ul Alam, Nirmalya Roy, Archan Misra, Joseph Taylor, CACE: Exploiting Behavioral Interactions for Improved Activity Recognition in Multi-Inhabitant Smart Homes, 36th International Conference on Distributed Computing Systems, ICDCS 2016

    Google Scholar 

  24. Mohammad Arif Ul Alam, Nilavra Pathak, Nirmalya Roy, Mobeacon: An iBeacon-Assisted smart-phone-Based Real Time Activity Recognition Framework, 12th International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services, EAI Endorsed Transactions on Ubiquitous Environments, January, 2016

    Google Scholar 

  25. M. Coulson, Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence, Journal of Nonverbal Behavior, 1992, 28, 2, 117–139

    Article  Google Scholar 

  26. H. Gunes, M. Piccardi, Bi-modal emotion recognition from expressive face and body gestures, Journal of Network and Computer Applications, 2007, 30, 4, 1334–1345

    Article  Google Scholar 

  27. Ginevra Castellano, Loic Kessous, George Caridakis, Multimodal emotion recognition from expressive faces, body gestures and speech, Artificial Intelligence and Innovations 2007: from Theory to Applications IFIP The International Federation for Information Processing, 2007, 247, 375–388

    Google Scholar 

  28. L.C De Silva, Pei Chi Ng, Bimodal emotion recognition, Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000. Proceedings, 2000, 332–335

    Google Scholar 

  29. Mohammad Arif Ul Alam, Weiqiang Wang, Sheikh Iqbal Ahamed, William Chu, Elderly Safety: A Smartphone Based Real Time Approach, International Conference On Smart homes and health Telematics, ICOST, 2013

    Google Scholar 

  30. L. Bao and S.S. Intille. Activity recognition from user-annotated acceleration data. pages 1–17. Springer, 2004

    Google Scholar 

  31. Alaiad, A. Zhou, L. Patient Behavioural Intention toward Adopting Healthcare Robots. The 19th Americas Conference on Information Systems (AMCIS), Chicago, USA, 2013

    Google Scholar 

  32. Alaiad, A., & Zhou, L. (2015, January). Patients’ Behavioral Intentions toward Using WSN Based Smart Home Healthcare Systems: An Empirical Investigation. In System Sciences (HICSS), 2015 48th Hawaii International Conference on (pp. 824–833). IEEE. doi:10.1109/HICSS.2015.104

  33. T. Gu, Z. Wu, X. Tao, H.K. Pung, and J. Lu. Epsicar: An emerging patterns based approach to sequential, interleaved and concurrent activity recognition. In Pervasive Computing and Communications, IEEE International Conference on, 2009

    Google Scholar 

  34. J.R. Kwapitz, G.M. Weiss, and S. Moore. Activity recognition using cell phone accelerometers. SIGKDD, 12(2):74–82, 2010

    Google Scholar 

  35. K. Grauman. Efficient activity detection with max-subgraph search. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), CVPR ’12, IEEE Computer Society, 2012

    Google Scholar 

  36. Patrick S. Hamilton and E.P. Limited. Open Source ECG Analysis Software Documentation. 2002

    Google Scholar 

  37. Andreas Krause, Matthias Ihmig, and et al. Trading off prediction accuracy and power consumption for context-aware wearable computing, ISWC 2005

    Google Scholar 

  38. David Chu, Nicholas D.Lane, and et al. Balancing energy, latency and accuracy for mobile sensor data classification. In Proc. of Sensys 2011

    Google Scholar 

  39. Song Liu, Makoto Yamada, Nigel Collier, Masashi Sugiyama, Change-point detection in time-series data by relative density-ratio estimation, Neural Networks. 2013; 72–83

    Google Scholar 

  40. Y. Kawahara, T. Yairi, and K. Machida. Change-point detection in time-series data based on subspace identification. In Proceedings of the 7th IEEE International Conference on Data Mining, pages 559–564, 2007

    Google Scholar 

  41. S.J. Bae, B.M. Mun, and K.Y. Kim. Change-point detection in failure intensity: A case study with repairable artillery systems. Computers and Industrial Engineering, 64:11–18, January 2013

    Google Scholar 

  42. A.G. Tartakovsky, B.L. Rozovskii, R.B. Blazek, and H. Kim. A novel approach to detection of intrusions in computer networks via adaptive sequential and batch sequential change-point detection methods. IEEE Transactions on Signal Processing, 54:3372–3382, September 2006

    Google Scholar 

  43. M. Staudacher, S. Telserb, A. Amannc, H. Hinterhuberb, and M. Ritsch-Marte. A new method for change-point detection developed for on-line analysis of the heart beat variability during sleep. Statistical Mechanics and its Applications, 349:582–596, April 2005

    Google Scholar 

  44. Ni-Chun Wang and Hudson, R.E. and Lee Ngee Tan and Taylor, C.E. and Alwan, A. and Rung Yao, Change point detection methodology used for segmenting bird songs, Signal and Information Processing (ChinaSIP), 2013 IEEE China Summit International Conference on, 2013

    Google Scholar 

  45. Zhang, Chi and Hansen, John HL, Effective segmentation based on vocal effort change point detection, Speech Analysis and Processing for Knowledge Discovery, 2008

    Google Scholar 

  46. James. R. Anderson, Pauline Meno, Psychological Influences on Yawning in Children, Current psychology letters, 2003

    Google Scholar 

  47. McGarvey LP, Patterns of cough in the clinic, Pulm Pharmacol Ther. 2011 Jun;24(3):300-3

    Google Scholar 

  48. Chronos: http://processors.wiki.ti.com/index.php/EZ430-Chronos

  49. Bosch: http://www.bosch-sensortec.com/de/homepage/products_3/3_axis_sensors/acceleration_sensors/bma250_1/bma250

  50. Hayley Hung, Gwenn Englebienn, Jeroen Kools, Classifying social actions with a single accelerometer, UbiComp ’13 Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing, 2013, 207–210

    Google Scholar 

  51. Michael Mason, Physical activity recognition of free-living data using change-point detection algorithms and hidden Markov models, Masters Thesis, Oregon State University, 2013

    Google Scholar 

  52. Hall, M. A. Correlation-based Feature Selection for Machine Learning. PhD Thesis (April 1999)

    Google Scholar 

  53. Source code: http://www.makotoyamada-ml.com/RuLSIF.html

  54. J. Platt: Fast Training of Support Vector Machines using Sequential Minimal Optimization. In B. Schoelkopf and C. Burges and A. Smola, editors, Advances in Kernel Methods—Support Vector Learning, 1998

    Google Scholar 

  55. HMMWeka, Marco Gillies, http://doc.gold.ac.uk/~mas02mg/software/hmmweka/

  56. Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjrgaard, Anind Dey, Tobias Sonne, and Mads Moller Jensen, Smart Devices are Different: Assessing and Mitigating Mobile Sensing Heterogeneities for Activity Recognition, SenSys 2015

    Google Scholar 

  57. O. Banos, M. A. Toth, M. Damas, H. Pomares, I. Rojas,Dealing with the effects of sensor displacement in wearable activity recognition. Sensors vol. 14, no. 6, 2014

    Google Scholar 

  58. T. L. Chartrand and J. A. Bargh, The chameleon effect: the perception-behavior link and social interaction, Journal of Personality and Social Psychology, 1999, 76, 6, 893–910

    Article  Google Scholar 

  59. A. Kendon, Conducting Interaction: Patterns of Behavior in Focused Encounters, Cambridge University Press, 1990

    Google Scholar 

  60. D. McNeill, Language and Gesture, Cambridge University Press New York, 2000

    Book  Google Scholar 

Download references

Acknowledgements

The work is supported in part by the National Science Foundation (NSF) under grants CNS-1344990, CNS-1544687, and IIP-1559752; the ONR under grant N00014-15-1-2229; Constellation: Energy to Educate; and the University of Maryland Baltimore-University of Maryland Baltimore County (UMB-UMBC) Research and Innovation Partnership grant.

Author information

Authors and Affiliations

  1. University of Maryland Baltimore County, Baltimore, MD, USA

    Mohammad Arif Ul Alam, Nirmalya Roy, Aryya Gangopadhyay & Elizabeth Galik

Authors
  1. Mohammad Arif Ul Alam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nirmalya Roy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aryya Gangopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Elizabeth Galik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Arif Ul Alam .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer Engineering, North Dakota State University , Fargo, North Dakota, USA

    Samee U. Khan

  2. School of Information Technologies, University of Sydney, Sydney, New South Wales, Australia

    Albert Y. Zomaya

  3. Department of Computer Science, COMSATS Institute of Information Technology, Islamabad, Pakistan

    Assad Abbas

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Alam, M.A.U., Roy, N., Gangopadhyay, A., Galik, E. (2017). Infrequent Non-speech Gestural Activity Recognition Using Smart Jewelry: Challenges and Opportunities for Large-Scale Adaptation. In: Khan, S., Zomaya, A., Abbas, A. (eds) Handbook of Large-Scale Distributed Computing in Smart Healthcare. Scalable Computing and Communications. Springer, Cham. https://doi.org/10.1007/978-3-319-58280-1_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-58280-1_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-58279-5

  • Online ISBN: 978-3-319-58280-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation