Skip to main content
SpringerLink
Log in

PERACTIV: Personalized Activity Monitoring - Ask My Hands

  • Conference paper
  • First Online:
Distributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13326))

Included in the following conference series:

Abstract

Medication adherence is a major problem in the healthcare industry: it has a major impact on an individual’s health and is a major expense on the healthcare system. We note that much of human activity involves using our hands, often in conjunction with objects. Camera-based wearables for tracking human activities have sparked a lot of attention in the past few years. These technologies have the potential to track human behavior anytime, any place. This paper proposes a paradigm for medication adherence employing innovative wrist-worn camera technology. We discuss how the device was built, various experiments to demonstrate feasibility and how the device could be deployed to detect the micro-activities involved in pill taking so as to ensure medication adherence.

Supported by National Science Foundation under Grant No. 1828010, Greater Phoenix Economic Council (GPEC), The Global Sport Institute at Arizona State University (GSI), and Arizona State University.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Almasi, M., Riera, J, Boza, S.: Undestanding human motions from ego-camera videos. https://doi.org/10.13140/RG.2.2.31884.54409

  2. Attal, F., Mohammed, S., Dedabrishvili, M., Chamroukhi, F., Oukhellou, L., Amirat, Y.: Physical human activity recognition using wearable sensors. Sensors. 15(12), 31314–31338 (2015)

    Article  Google Scholar 

  3. Bambach, S., Lee, S., Crandall, D., Yu, C.: Lending a hand: detecting hands and recognizing activities in complex egocentric interactions. In: 2015 IEEE International Conference on Computer Vision (ICCV) (2015)

    Google Scholar 

  4. Barabas, J., Bednar, T., Vychlopen, M.: Kinect-based platform for movement monitoring and fall-detection of elderly people. In: 2019 12th International Conference on Measurement (2019)

    Google Scholar 

  5. Baritz, M., Cotoros, D., Singer, C.: Thermographic analysis of hand structure when subjected to controlled effort. In: 2013 E-Health and Bioengineering Conference. EHB 2013, pp. 1–4 (2013)

    Google Scholar 

  6. Barsoum, E.: Articulated hand pose estimation review. arXiv preprint arXiv:1604.06195 (2016)

  7. Blum, M., Pentland, A., Troster, G.: InSense: interest-based life logging. IEEE Multim. 13(4), 40–48 (2006)

    Article  Google Scholar 

  8. Carreira, J., Zisserman, A.: Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset, pp. 4724–4733 (2017). https://doi.org/10.1109/CVPR.2017.502

  9. Chan, C.-S., Chen, S.-Z., Xie, P.-X., Chang, C.-C., Sun, M.: Recognition from hand cameras: a revisit with deep learning. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 505–521. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_31

    Chapter  Google Scholar 

  10. Chao, Y., Scherer, Y., Montgomery, C.: Effects of using Nintendo Wii\(^{\rm TM}\) Exergames in older adults. J. Aging Health 27(3), 379–402 (2014)

    Article  Google Scholar 

  11. Chatzis, T., Stergioulas, A., Konstantinidis, D., Dimitropoulos, K., Daras, P.: A comprehensive study on deep learning-based 3D hand pose estimation methods. Appl. Sci.. 10(19), 6850 (2020)

    Google Scholar 

  12. Clark, N.P.: Role of the anticoagulant monitoring service in 2018: beyond warfarin. Hematol. Am. Soc. Hematol. Educ. Program. 2018(1), 348–352 (2018)

    Google Scholar 

  13. Clarkson, B., Mase, K., Pentland, A.: Recognizing user’s context from wearable sensors: baseline system. J. Neurol. Sci. 248 (1999)

    Google Scholar 

  14. Kim, D., et al.: Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In: Proceedings of the 25th annual ACM symposium on User Interface Software and Technology, pp. 167–176. Association for Computing Machinery, New York (2012)

    Google Scholar 

  15. Doshi, V., et al.: An IoT based smart medicine box. Int. J. Adv.Res. Ideas Innov. Technol. 5(1), 205–207 (2019)

    Google Scholar 

  16. Tavakolizadeh, F., Gu, J., Saket, B.: Traceband: locating missing items by visual remembrance. In: Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST 2014 Adjunct), pp. 109–110. Association for Computing Machinery, New York (2014)

    Google Scholar 

  17. Feng, W., Liu, R., Zhu, M.: Fall detection for elderly person care in a vision-based home surveillance environment using a monocular camera. Signal Image Video Process. 8(6), 1129–1138 (2014). https://doi.org/10.1007/s11760-014-0645-4

    Article  Google Scholar 

  18. Ahmad, F., Musilek, P.: A keystroke and pointer control input interface for wearable computers. In: Fourth Annual IEEE International Conference on Pervasive Computing and Communications (PERCOM 2006), pp. 10–11 (2006)

    Google Scholar 

  19. Fernandez-Cervantes, V., Neubauer, N., Hunter, B., Stroulia, E., Liu, L.: VirtualGym: a kinect-based system for seniors exercising at home. Entertain. Comput. 27, 60–72 (2018)

    Article  Google Scholar 

  20. Fitzpatrick, P., Kemp, C.: Shoes as a platform for vision. In: Proceedings Seventh IEEE International Symposium on Wearable Computers, 2003. (n.d.)

    Google Scholar 

  21. Automatic Pill Dispenser - How the Hero Dispenser Works!. https://herohealth.com/our-product/. Accessed 24 Feb 2022

  22. Khusainov, R., et al.: Real-time human ambulation, activity, and physiological monitoring: taxonomy of issues, techniques, applications, challenges and limitations. Sensors 13(10), pp. 12852–12902 (2013)

    Google Scholar 

  23. Kim, S., Ko, M., Lee, K., Kim, M., Kim, K.: 3D fall detection for single camera surveillance systems on the street. In: 2018 IEEE Sensors Applications Symposium (SAS) (2008)

    Google Scholar 

  24. Lee, J., Lee, J., Lim, I., Kim, Y., Hyun-Namgung, Lee, J.: Kinect-based monitoring system to prevent seniors who live alone from solitary death. In: Computational Science and Its Applications, UCCSA 2014. ICCSA 2014. LNCS, vol. 8582, pp. 709–719. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09147-1_51

  25. Medication Dispensing Service: Philips Lifeline. https://www.lifeline.philips.com/business/medicationdispensing. Accessed 24 Feb 2022

  26. Maekawa, T., Kishino, Y., Yanagisawa, Y., Sakurai, Y.: WristSense: wrist-worn sensor device with camera for daily activity recognition. In: 2012 IEEE International Conference on Pervasive Computing and Communications Workshops (2012)

    Google Scholar 

  27. Muhamada, A.W., Mohammed, A.A.: Review on recent computer vision methods for human action recognition. Adv. Distrib. Comput. Artif. Intell. J 10(4), 361–379 (2022)

    Google Scholar 

  28. Pharmadva MedaCube\(^{\rm TM}\). MedaCube. https://www.medacube.com/. Accessed 24 Feb 2022

  29. e-Pill MedSmart Voice. https://www.epill.com/medsmart-voice.html. Accessed 24 Feb 2022

  30. El-Sheimy, N., Hou, H., Niu, X.: Analysis and modeling of inertial sensors using Allan variance. IEEE Trans. Instrum. Measur. 57(1), 140–149 (2008)

    Article  Google Scholar 

  31. Nelson, D.L., Mitchell, M.A., Groszewski, P.G., Pennick, S.L., Manske, P.R.: Wrist Range of motion in activities of daily living. In: Schuind, F., An, K.N., Cooney, W.P., Garcia-Elias, M. (eds.) Advances in the Biomechanics of the Hand and Wrist, pp. 329–334. Springer, Boston (1994). https://doi.org/10.1007/978-1-4757-9107-5

    Chapter  Google Scholar 

  32. Nguyen, T., Nebel, J., Florez-Revuelta, F.: Recognition of activities of daily living with egocentric vision: a review. Sensors 16(1), 72 (2016)

    Google Scholar 

  33. Núñez-Marcos, A., Azkune, G., Arganda-Carreras, I.: Egocentric vision-based action recognition: a survey. Neurocomputing 472, 175–197 (2022)

    Article  Google Scholar 

  34. How to Use Your Auto Pill Dispenser: Medication Management: Pria. Pria. https://www.okpria.com/How-it-works. Accessed 24 Feb 2022

  35. Rusu, L., Mocanu, I.G., Jecan, S., Sitar, D.S.: Monitoring adaptive exergame for seniors. J. Inf. Syst. Oper. Manag. 10, 336–343 (2016)

    Google Scholar 

  36. Kido, S., Miyasaka, T., Tanaka, T., Shimizu, T., Saga, T.: Fall detection in toilet rooms using thermal imaging sensors. In: IEEE/SICE International Symposium on System Integration (SII) 2009, pp. 83–88 (2009)

    Google Scholar 

  37. Stone, E., Skubic, M.: Evaluation of an inexpensive depth camera for in-home gait assessment. J. Ambi. Intell. Smart Environ. 3(4), 349–361 (2011)

    Google Scholar 

  38. Stone, E., Skubic, M.: Evaluation of an inexpensive depth camera for passive in-home fall risk assessment. In: Proceedings of the 5th International ICST Conference on Pervasive Computing Technologies for Healthcare (2011)

    Google Scholar 

  39. Tavakolizadeh, F., Gu, J., Saket, B.: Traceband. In: Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology – UIST 2014 Adjunct (2014)

    Google Scholar 

  40. Tran, D., Bourdev, L., Fergus, R., Torresani, L., Paluri, M.: Learning Spatiotemporal Features with 3D Convolutional Networks. In: Conference: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 4489–4497 (2015)

    Google Scholar 

  41. Ueoka, T., Kawamura, T., Kono, Y., Kidode, M.: I’m Here!: a Wearable object remembrance support system. In: Proceedings of 5th International Symposium on the Human-Computer Interaction with Mobile Devices and Services, Mobile HCI 2003, Udine, Italy, 8–11 September 2003, pp. 422–427 (2003). https://doi.org/10.1007/978-3-540-45233-1_40

  42. Van Onzenoort, H.A., Verberk, W.J., Kroon, A.A., et al.: Electronic monitoring of adherence, treatment of hypertension, and blood pressure control. Am. J. Hypertens. 25, 54e59 (2012)

    Google Scholar 

  43. Vardy, A., Robinson, J., Cheng, L.T.: The WristCam as input device. Digest of papers. In: Third International Symposium on Wearable Computers (n.d.)

    Google Scholar 

  44. Watanabe, J., McInnis, T., Hirsch, J.: Cost of prescription drug-related morbidity and Mortality. Ann. Pharmacother 52(9), 829–837 (2018)

    Article  Google Scholar 

  45. Wu, D., Sharma, N., Blumenstein, M.: Recent advances in video-based human action recognition using deep learning: a review. In: 2017 International Joint Conference on Neural Networks (IJCNN). IEEE (2017)

    Google Scholar 

  46. Yamato, J., Ohya, J., Ishii, K.: Recognizing human action in time-sequential images using hidden Markov model. In: Proceedings/CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Conference on Computer Vision and Pattern Recognition J76-D-II, pp. 379–385 (1992). https://doi.org/10.1109/CVPR.1992.223161

  47. Yang, C., Chen Hsieh, J., Chen, Y., Yang, S., Lin, H.: Effects of Kinect exergames on balance training among community older adults. Medicine 99(28), e21228 (2020)

    Google Scholar 

  48. Yang, L., Ren, Y., Zhang, W.: 3D depth image analysis for indoor fall detection of elderly people. Digit. Commun. Netw. 2(1), 24–34 (2016)

    Google Scholar 

  49. Zhang, C., Tian, Y., Capezuti, E.: Privacy preserving automatic fall detection for elderly using RGBD cameras. In: Miesenberger, K., Karshmer, A., Penaz, P., Zagler, W. (eds.) ICCHP 2012. LNCS, vol. 7382, pp. 625–633. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31522-0_95

  50. Zullig, L., Deschodt, M., Liska, J., Bosworth, H., De Geest, S.: Moving from the trial to the real world: improving medication adherence using insights of implementation science. Ann. Rev. Pharmacol. Toxicol. 59(1), 423–445 (2019)

    Google Scholar 

Download references

Acknowledgements

This paper was supported by funding from National Science Foundation under Grant No. 1828010, Greater Phoenix Economic Council (GPEC), The Global Sport Institute at Arizona State University (GSI), and Arizona State University.

The authors thank partner facility, Mirabella, at ASU for helping in recruiting participants for interviews. The authors also thank Joshua Chang for his help in sketching Figs. 1, 2, 4, 9 and Abhik Chowdhury for his help in developing the device.

Author information

Authors and Affiliations

  1. Arizona State University, Tempe, USA

    Vishnu Kakaraparthi, Troy McDaniel, Hemanth Venkateswara & Morris Goldberg

Authors
  1. Vishnu Kakaraparthi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Troy McDaniel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hemanth Venkateswara
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Morris Goldberg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vishnu Kakaraparthi .

Editor information

Editors and Affiliations

  1. Smart Future Initiative, Frankfurt am Main, Germany

    Norbert A. Streitz

  2. Kyushu University, Fukuoka, Japan

    Shin'ichi Konomi

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kakaraparthi, V., McDaniel, T., Venkateswara, H., Goldberg, M. (2022). PERACTIV: Personalized Activity Monitoring - Ask My Hands. In: Streitz, N.A., Konomi, S. (eds) Distributed, Ambient and Pervasive Interactions. Smart Living, Learning, Well-being and Health, Art and Creativity. HCII 2022. Lecture Notes in Computer Science, vol 13326. Springer, Cham. https://doi.org/10.1007/978-3-031-05431-0_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05431-0_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05430-3

  • Online ISBN: 978-3-031-05431-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation