An Adaptive Hidden Markov Model for Activity Recognition Based on a Wearable Multi-Sensor Device

  • Zhen Li
  • Zhiqiang Wei
  • Yaofeng Yue
  • Hao Wang
  • Wenyan Jia
  • Lora E. Burke
  • Thomas Baranowski
  • Mingui Sun
Mobile Systems
Part of the following topical collections:
  1. Mobile Systems


Human activity recognition is important in the study of personal health, wellness and lifestyle. In order to acquire human activity information from the personal space, many wearable multi-sensor devices have been developed. In this paper, a novel technique for automatic activity recognition based on multi-sensor data is presented. In order to utilize these data efficiently and overcome the big data problem, an offline adaptive-Hidden Markov Model (HMM) is proposed. A sensor selection scheme is implemented based on an improved Viterbi algorithm. A new method is proposed that incorporates personal experience into the HMM model as a priori information. Experiments are conducted using a personal wearable computer eButton consisting of multiple sensors. Our comparative study with the standard HMM and other alternative methods in processing the eButton data have shown that our method is more robust and efficient, providing a useful tool to evaluate human activity and lifestyle.


Hidden Markov model Activity recognition Wearable device Big data Machine learning Personal health 



This work was supported by National Institutes of Health Grants No. R01CA165255, R21CA172864, and P30 AG024827 of the United States, and the National Natural Science Foundation No. 61402428 of China.


  1. 1.
    Touati, F., and Tabish, R., U-healthcare system: state-of-the-art review and challenges. J. Med. Syst. 37:1–20, 2013.CrossRefGoogle Scholar
  2. 2.
    Gyllensten, I. C., and Bonomi, A. G., Identifying types of physical activity with a single accelerometer: evaluating laboratory-trained algorithms in daily life. IEEE Trans. Biomed. Eng. 58:2656–2663, 2011.CrossRefGoogle Scholar
  3. 3.
    Khan, A. M., Lee, Y.-K. L. Y.-K., Lee, S. Y., and Kim, T.-S. K. T.-S., A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer. IEEE Trans. Inf. Technol. Biomed. 14:1166–1172, 2010.CrossRefGoogle Scholar
  4. 4.
    Gao, L., Bourke, A. K., Nelson, J., Activity recognition using dynamic multiple sensor fusion in body sensor networks. In: 2012 Annu. Int. Conf. IEEE Eng. Med. Biol. Soc. 49:1077–1080, 2012.Google Scholar
  5. 5.
    Chang, S.-Y., Lai, C.-F., Chao, H.-C., et al., An environmental-adaptive fall detection system on mobile device. J. Med. Syst. 35:1299–1312, 2011.CrossRefGoogle Scholar
  6. 6.
    Jin, G. H., Lee, S. B., and Lee, T. S., Context awareness of human motion states using accelerometer. J. Med. Syst. 32:93–100, 2008.CrossRefGoogle Scholar
  7. 7.
    Arab, L., and Winter, A., Automated camera-phone experience with the frequency of imaging necessary to capture diet. J. Am. Diet. Assoc. 110:1238–1241, 2010.CrossRefGoogle Scholar
  8. 8.
    Kwapisz, J. R., Weiss, G. M., and Moore, S. A., Activity recognition using cell phone accelerometers. ACM SIGKDD Explor. Newsl. 12:74–82, 2011.CrossRefGoogle Scholar
  9. 9.
    Doherty, A. R., Ó Conaire, C., Blighe, M., et al., Combining image descriptors to effectively retrieve events from visual lifelogs. In: Proc. 1st ACM Int. Conf. Multimed. Inf. Retr. 10–17, 2008.Google Scholar
  10. 10.
    Chatzichristofis, S. A., Boutalis, Y. S., CEDD: color and edge directivity descriptor . A compact descriptor for image indexing and retrieval. In: Proc. 6th Int. Conf. Comput. Vis. Syst. 312–322, 2008.Google Scholar
  11. 11.
    Lowe, D. G., Object recognition from local scale-invariant features. In: Proc. Seventh IEEE Int. Conf. Comput. Vis. 2:1150–1157, 1999.Google Scholar
  12. 12.
    Duan, L., Xu, D., Tsang, I. W.-H., and Luo, J., Visual event recognition in videos by learning from Web data. IEEE Trans. Pattern Anal. Mach. Intell. 34:1667–1680, 2012.CrossRefGoogle Scholar
  13. 13.
    Bebars, A. A., Hemayed, E. E., Comparative study for feature detectors in human activity recognition. In: 2013 9th Int. Comput. Eng. Conf. 19–24, 2013.Google Scholar
  14. 14.
    Hassan, S. M., Al-Sadek, A. F., Hemayed, E. E., Rule-based approach for enhancing the motion trajectories in human activity recognition. In: 2010 10th Int. Conf. Intell. Syst. Des. Appl. 829–834, 2010.Google Scholar
  15. 15.
    Boutell, M., and Brown, C., Pictures are not taken in a vacuum - an overview of exploiting context for semantic scene content understanding. IEEE Signal Process. Mag. 23:101–114, 2006.Google Scholar
  16. 16.
    Arif, M., Bilal, M., Kattan, A., and Ahamed, S. I., Better physical activity classification using smartphone acceleration sensor. J. Med. Syst. 38:1–10, 2014.CrossRefGoogle Scholar
  17. 17.
    Yin, J., Yang, Q., and Pan, J. J., Sensor-based abnormal human-activity detection. IEEE Trans. Knowl. Data Eng. 20:1082–1090, 2008.CrossRefGoogle Scholar
  18. 18.
    Giansanti, D., Macellari, V., and Maccioni, G., New neural network classifier of fall-risk based on the Mahalanobis distance and kinematic parameters assessed by a wearable device. Physiol. Meas. 29:N11–N19, 2008.CrossRefGoogle Scholar
  19. 19.
    Cao, L. C. L., Luo, J. L. J., Kautz, H., and Huang, T. S., Image annotation within the context of personal photo collections using hierarchical event and scene models. IEEE Trans. Multimed. 11:208–219, 2009.CrossRefGoogle Scholar
  20. 20.
    Hughes, G., On the mean accuracy of statistical pattern recognizers. IEEE Trans. Inf. Theory 14:55–63, 1968.CrossRefGoogle Scholar
  21. 21.
    Zhang, W., Jia, W., Sun, M., Segmentation for efficient browsing of chronical video recorded by a wearable device. In: Proc 2010 I.E. 36th Annu. Northeast. Bioeng. Conf. 1–2, 2010.Google Scholar
  22. 22.
    Manjunath, B. S., Ohm, J.-R., Vasudevan, V. V., and Yamada, A., Color and texture descriptors. IEEE Trans. Circ. Syst. Video Technol. 11:703–715, 2001.CrossRefGoogle Scholar
  23. 23.
    Csurka, G., Dance, C., Fan, L., et al., Visual categorization with bags of keypoints. In: Work. Stat. Learn. Comput. Vision ECCV. 1–2, 2004.Google Scholar
  24. 24.
    Peng, H., Long, F., and Ding, C., Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27:1226–1238, 2005.CrossRefGoogle Scholar
  25. 25.
    Powers, D. M., Evaluation: from precision, recall and F-measure to ROC, informedness, markedness & correlation. J. Mach. Learn. Technol. 2:37–63, 2011.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  • Zhen Li
    • 1
    • 2
  • Zhiqiang Wei
    • 1
  • Yaofeng Yue
    • 2
    • 3
  • Hao Wang
    • 2
    • 3
  • Wenyan Jia
    • 2
    • 3
  • Lora E. Burke
    • 4
  • Thomas Baranowski
    • 5
  • Mingui Sun
    • 2
    • 3
    • 2
  1. 1.Department of Computer ScienceOcean University of ChinaQingdaoChina
  2. 2.Department of NeurosurgeryUniversity of PittsburghPittsburghUSA
  3. 3.Department of Electrical & Computer EngineeringUniversity of PittsburghPittsburghUSA
  4. 4.Department of Health and Community SystemsUniversity of PittsburghPittsburghUSA
  5. 5.Department of PediatricsBaylor College of MedicineHoustonUSA

Personalised recommendations