Adaptive Activity and Context Recognition Using Multimodal Sensors in Smart Devices

  • Sébastien Faye
  • Raphael Frank
  • Thomas Engel
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 162)


The continuous development of new technologies has led to the creation of a wide range of personal devices embedded with an ever increasing number of miniature sensors. With accelerometers and technologies such as Bluetooth and Wi-Fi, today’s smartphones have the potential to monitor and record a complete history of their owners’ movements as well as the context in which they occur. In this article, we focus on four complementary aspects related to the understanding of human behaviour. First, the use of smartwatches in combination with smartphones in order to detect different activities and associated physiological patterns. Next, the use of a scalable and energy-efficient data structure that can represent the detected signal shapes. Then, the use of a supervised classifier (i.e. Support Vector Machine) in parallel with a quantitative survey involving a dozen participants to achieve a deeper understanding of the influence of each collected metric and its use in detecting user activities and contexts. Finally, the use of novel representations to visualize the activities and social interactions of all the users, allowing the creation of quick and easy-to-understand comparisons. The tools used in this article are freely available online under a MIT licence.


Sensing system Wearable computing Activity detection 


  1. 1.
    Xu, C., Pathak, P.H., Mohapatra, P.: Finger-writing with smartwatch: a case for finger and hand gesture recognition using smartwatch. In: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, pp. 9–14. ACM (2015)Google Scholar
  2. 2.
    Tilenius, S.: Will An App A Day Keep The Doctor Away? The Coming Health Revolution. Forbes CIO Network (2013)Google Scholar
  3. 3.
    Campbell, A.T., Eisenman, S.B., Lane, N.D., Miluzzo, E., Peterson, R.A.: People-centric urban sensing. In: Proceedings of the 2nd annual international workshop on Wireless internet, p. 18. ACM (2006)Google Scholar
  4. 4.
    Burke, J.A., Estrin, D., Hansen, M., Parker, A., Ramanathan, N., Reddy, S., Srivastava, M.B.: Participatory sensing. In: Center for Embedded Network Sensing (2006)Google Scholar
  5. 5.
    Lane, N.D., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., Campbell, A.T.: A survey of mobile phone sensing. IEEE Commun. Mag. 48(9), 140–150 (2010)CrossRefGoogle Scholar
  6. 6.
    Yang, F., Wang, S., Li, S., Pan, G., Huang, R.: MagicWatch: interacting & segueing. In: Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 315–318. ACM (2014)Google Scholar
  7. 7.
    Faye, S., Frank, R.: Demo: using wearables to learn from human dynamics. In: Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services, pp. 445–445. ACM (2015)Google Scholar
  8. 8.
    Zheng, X., Ordieres-Meré, J.: Development of a human movement monitoring system based on wearable devices. In: The International Conference on Electronics, Signal Processing and Communication Systems (ESPCO 2014) (2014)Google Scholar
  9. 9.
    Bluetooth, S.: Bluetooth specification version 4.0. In: Bluetooth SIG (2010).
  10. 10.
    Rodrigues, J.G., Aguiar, A., Barros, J.: SenseMyCity: Crowdsourcing an Urban Sensor. arXiv preprint arxiv:1412.2070 (2014)
  11. 11.
    Cuervo, E., Balasubramanian, A., Cho, D.-K., Wolman, A., Saroiu, S., Chandra, R., Bahl, P.: MAUI: making smartphones last longer with code offload. In: Proceedings of the 8th International Conference on Mobile Systems, Applications, and Services, pp. 49–62. ACM (2010)Google Scholar
  12. 12.
    Honicky, R., Brewer, E.A., Paulos, E., White, R.: N-smarts: networked suite of mobile atmospheric real-time sensors. In: Proceedings of the Second ACM SIGCOMM Workshop on Networked Systems for Developing Regions, pp. 25–30. ACM (2008)Google Scholar
  13. 13.
    Hussain, S., Bang, J.H., Han, M., Ahmed, M.I., Amin, M.B., Lee, S., Nugent, C., McClean, S., Scotney, B., Parr, G.: Behavior life style analysis for mobile sensory data in cloud computing through MapReduce. Sensors 14(11), 22001–22020 (2014)CrossRefGoogle Scholar
  14. 14.
    Han, M., Lee, Y.-K., Lee, S., et al.: Comprehensive context recognizer based on multimodal sensors in a smartphone. Sensors 12(9), 12588–12605 (2012)CrossRefGoogle Scholar
  15. 15.
    Lu, H., Pan, W., Lane, N.D., Choudhury, T., Campbell, A.T.: Sound- Sense: scalable sound sensing for people-centric applications on mobile phones. In: Proceedings of the 7th International Conference on Mobile systems, Applications, and Services, pp. 165–178. ACM (2009)Google Scholar
  16. 16.
    Ma, L., Smith, D., Milner, B.: Environmental noise classification for context-aware applications. In: Mařík, V., Štěpánková, O., Retschitzegger, W. (eds.) DEXA 2003. LNCS, vol. 2736, pp. 360–370. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  17. 17.
    Sharma, V., Mankodiya, K., De La Torre, F., Zhang, A., Ryan, N., Ton, T.G.N., Gandhi, R., Jain, S.: SPARK: personalized parkinson disease interventions through synergy between a smartphone and a smartwatch. In: Marcus, A. (ed.) DUXU 2014, Part III. LNCS, vol. 8519, pp. 103–114. Springer, Heidelberg (2014)Google Scholar
  18. 18.
    Shin, D., Shin, D., Shin, D.: Ubiquitous health management system with watch-type monitoring device for dementia patients. J. Appl. Math. 2014(2014), Article ID 878741, 8 (2014).
  19. 19.
    Porzi, L., Messelodi, S., Modena, C.M., Ricci, E.: A smart watch-based gesture recognition system for assisting people with visual impairments. In: Proceedings of the 3rd ACM International Workshop on Interactive Multimedia on Mobile & Portable Devices, pp. 19–24. ACM (2013)Google Scholar
  20. 20.
    He, Z., Liu, Z., Jin, L., Zhen, L.-X., Huang, J.-C.: Weightlessness feature–a novel feature for single tri-axial accelerometer based activity recognition. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)Google Scholar
  21. 21.
    Kao, T.-P., Lin, C.-W., Wang, J.-S.: Development of a portable activity detector for daily activity recognition. In: IEEE International Symposium on Industrial Electronics, ISIE 2009, pp. 115–120 (2009)Google Scholar
  22. 22.
    Qian, H., Mao, Y., Xiang, W., Wang, Z.: Recognition of human activities using SVM multi-class classifier. Pattern Recogn. Lett. 31(2), 100–111 (2010)CrossRefGoogle Scholar
  23. 23.
    Wu, J., Pan, G., Zhang, D., Qi, G., Li, Shijian: Gesture recognition with 3-D accelerometer. In: Zhang, D., Portmann, M., Tan, A.-H., Indulska, J. (eds.) UIC 2009. LNCS, vol. 5585, pp. 25–38. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  24. 24.
    Jain, A.K.: Data clustering: 50 years beyond K-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)CrossRefGoogle Scholar
  25. 25.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)zbMATHGoogle Scholar
  26. 26.
    Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., Weingessel, A.: e1071: Misc Functions of the Department of Statistics (e1071), TU Wien (2011).
  27. 27.
    Bergstra, J., Bengio, Y.: Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13(1), 281–305 (2012)MathSciNetzbMATHGoogle Scholar

Copyright information

© Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2015

Authors and Affiliations

  1. 1.Interdisciplinary Centre for Security, Reliability and TrustUniversity of LuxembourgLuxembourgLuxembourg

Personalised recommendations