Skip to main content
SpringerLink
Log in

In-Home Sleep Monitoring using Edge Intelligence

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

We spend about one-third of our life either sleeping or attempting to do so. Sleeping is a key aspect for most human body processes, affecting physical and mental health and the ability to fight diseases, develop immunity and control metabolism. Therefore, monitoring human sleep quality is extremely important for the detection of possible sleep disorders. Several technologies exist to achieve this goal, however, most of them are expensive proprietary systems, some require hospitalization and many use intrusive equipment that can, by itself, affect sleep quality. This paper presents an intelligent system, a complete low-cost hardware and software solution, for monitoring the sleep quality of an individual in a home environment. User privacy is guaranteed as all processing is done at the edge and no audio or video is stored. This system monitors several fundamental aspects of sleeping periods in real-time using a low cost single-board computer for processing, a camera for body motion detection (MD module) and for eye/sleep status detection (SSD module), and a microphone for audio recognition (AUDR module) of breath pattern analysis and snore detection. It can be strategically placed near the bed to avoid interfering with the natural sleep pattern. For each sleeping period, the system produces a final report that can be a valuable aid for improving the sleeping health of the monitored person. Functional unitary tests were carried successfully on the selected, low-cost, hardware platform (Raspberry Pi). The entire process was validated by an expert clinical psychologist, ensuring the reliability and effectiveness of the system. The visual and sound modules use sophisticated computer vision and machine learning techniques suitable for edge computing devices. Each of the system’s features have been independently tested, using properly organized audio and video datasets and the well established metrics of precision, recall and F1 score, to evaluate the binary classifiers in each of the three modules. The accuracy values obtained where 90.2% (MD), 79.1% (SSD) and 81.3% (AUDR), demonstrating the great application potential of our solution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Ohayon M, Wickwire EM, Hirshkowitz M, Albert SM, Avidan A, Daly FJ, et al. National sleep foundation’s sleep quality recommendations: first report. Sleep Health. 2017;3(1):6–19. https://doi.org/10.1016/j.sleh.2016.11.006.

    Article  Google Scholar 

  2. Larsen LH, Lauritzen MH, Sinkjaer M, Kjaer TW. A multi-component intervention to affect physical activity, sleep length and stress levels in office workers. Smart Health. 2021;22: 100219. https://doi.org/10.1016/j.smhl.2021.100219.

    Article  Google Scholar 

  3. Buysse DJ. Sleep health: can we define it? does it matter? Sleep. 2014;37(1):9–17. https://doi.org/10.5665/sleep.3298.

    Article  Google Scholar 

  4. National Heart Lung Blood Institute. Sleep apnea. U.S. Department of Health and Human Services. https://www.nhlbi.nih.gov/health-topics/sleep-apnea.

  5. Ntenta PK, Vavougios GD, Zarogiannis SG, Gourgoulianis KI. Obstructive sleep apnea syndrome comorbidity phenotypes in primary health care patients in Northern Greece. Healthcare. 2022. https://doi.org/10.3390/healthcare10020338.

    Article  Google Scholar 

  6. Camci B, Kahveci AY, Arnrich B, Ersoy C. Sleep apnea detection via smart phones. In: 25th Signal processing and communications applications conference. SIU 2017; 2017. p. 16–19. https://doi.org/10.1109/SIU.2017.7960484.

  7. Ramesh J, Keeran N, Sagahyroon A, Aloul F. Towards validating the effectiveness of obstructive sleep apnea classification from electronic health records using machine learning. Healthcare. 2021. https://doi.org/10.3390/healthcare9111450.

    Article  Google Scholar 

  8. Boulemtafes A, Khemissa H, Derki MS, Amira A, Djedjig N. Deep learning in pervasive health monitoring, design goals, applications, and architectures: an overview and a brief synthesis. Smart Health. 2021;22: 100221. https://doi.org/10.1016/j.smhl.2021.100221.

    Article  Google Scholar 

  9. Almarshad MA, Islam MS, Al-Ahmadi S, BaHammam AS. Diagnostic features and potential applications of PPG signal in healthcare: a systematic review. Healthcare. 2022. https://doi.org/10.3390/healthcare10030547.

    Article  Google Scholar 

  10. Nandakumar R, Gollakota S, Watson N. Contactless sleep apnea detection on smartphones. In: MobiSys 2015—proceedings of the 13th annual international conference on mobile systems, applications, and services; 2015. p. 45–57. https://doi.org/10.1145/2742647.2742674.

  11. Gu W, Shangguan L, Yang Z, Liu Y. Sleep hunter: towards fine grained sleep stage tracking with smartphones. IEEE Trans Mob Comput. 2016;6(15):1514–27. https://doi.org/10.1109/TMC.2015.2462812.

    Article  Google Scholar 

  12. Behar J, Roebuck A, Shahid M, Daly J, Hallack A, Palmius N, et al. SleepAp: an automated obstructive sleep apnoea screening application for smartphones. IEEE J Biomed Health Inform. 2015;1(19):325–31. https://doi.org/10.1109/JBHI.2014.2307913.

    Article  Google Scholar 

  13. Lazazzera R, Laguna P, Gil E, Carrault G. Proposal for a home sleep monitoring platform employing a smart glove. Sensors. 2021. https://doi.org/10.3390/s21237976.

    Article  Google Scholar 

  14. Ibrahim DM, Hammoudeh MAA, Ambreen S, Mohammadi S. Raspberry Pi-based smart infant monitoring system. Int J Eng Res Technol. 2019;12(10):1723–9.

    Google Scholar 

  15. Lui B. BabbyCam. https://www.babbycam.com/. Accessed: 24 April 2021.

  16. Islam MZ, Nahiyan KMT, Kiber MA. A motion detection algorithm for video-polysomnography to diagnose sleep disorder. In: 2015 18th International conference on computer and information technology, ICCIT 2015; 2016. p. 272–5. https://doi.org/10.1109/ICCITechn.2015.7488081.

  17. Fei J, Pavlidis I, Murthy J. Thermal vision for sleep apnea monitoring. In: Yang GZ, Hawkes D, Rueckert D, Noble A, Taylor C, editors. Medical image computing and computer-assisted intervention—MICCAI 2009. Berlin Heidelberg: Springer; 2009. p. 1084–91.

    Google Scholar 

  18. Pratyasha P, Gupta S. Early recognition of dynamic sleeping patterns associated with rapid eyeball movement sleep behavior disorder of apnea patients using neural network techniques. In: Next generation healthcare systems using soft computing techniques. Boca Raton: CRC Press; 2022. p. 55–69.

    Chapter  Google Scholar 

  19. Dafna E, Tarasiuk A, Zigel Y. Sleep-quality assessment from full night audio recordings of sleep apnea patients. In: Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, EMBS’; 2012. p. 3660–63. https://doi.org/10.1109/EMBC.2012.6346760.

  20. Rosenwein T, Dafna E, Tarasiuk A, Zigel Y. Detection of breathing sounds during sleep using non-contact audio recordings. In: 36th Annual international conference of the IEEE engineering in medicine and biology society. EMBC 2014; 2014. p. 1489–92. https://doi.org/10.1109/EMBC.2014.6943883.

  21. Breiman L. Random forests. Mach Learn. 2001;45(1):5–32. https://doi.org/10.1023/A:1010933404324.

    Article  Google Scholar 

  22. Karci E, Dogrusoz YS, Ciloglu T. Detection of post apnea sounds and apnea periods from sleep sounds. In: Proceedings of the annual international conference of the IEEE engineering in medicine and biology society, EMBS; 2011. p. 6075–8. https://doi.org/10.1109/IEMBS.2011.6091501.

  23. Khan T. A deep learning model for snoring detection and vibration notification using a smartwearable gadget. Electronics (Switzerland). 2019;8:1–19. https://doi.org/10.3390/electronics8090987.

    Article  Google Scholar 

  24. Miao Y, Zhang Z, Jia F, Dai M. Treatment pillow for relieving snoring symptoms based on SNORE RECOGNITION. In: 2018 25th International conference on mechatronics and machine vision in practice (M2VIP); 2018.https://doi.org/10.1109/m2vip.2018.8600841.

  25. Lin X, Cheng H, Lu Y, Luo H, Li H, Qian Y, et al. Contactless sleep apnea detection in snoring signals using hybrid deep neural networks targeted for embedded hardware platform with real-time applications. Biomed Signal Process Control. 2022;77: 103765. https://doi.org/10.1016/j.bspc.2022.103765.

    Article  Google Scholar 

  26. OpenCV. https://opencv.org/. Accessed 30 June 2022.

  27. King D. Dlib-ml: a machine learning toolkit. J Mach Learn Res. 2009;07(10):1755–8. https://doi.org/10.1145/1577069.1755843.

    Article  Google Scholar 

  28. Tensorflow. Tensorflow. https://www.tensorflow.org/. Accessed 25 Jan 2024.

  29. Raspberry Pi Foundation. Raspberry Pi 4 Model B specifications. https://www.raspberrypi.com/products/raspberry-pi-4-model-b/specifications/. Accessed 25 Jan 2024.

  30. Rice T. Real time inference on Raspberry PI 4 (30 FPS!). PyTorch. https://pytorch.org/tutorials/intermediate/realtime_rpi.html. Accessed 25 Jan 2024.

  31. Raspberry Pi Foundation. Raspberry Pi Camera Module 3 NoIR. https://www.raspberrypi.com/products/camera-module-3/. Accessed 25 Jan 2024.

  32. Cao Z, Simon T, Wei SE, Sheikh Y. Realtime multi-person 2D pose estimation using part affinity fields. In: CVPR; 2017.

  33. Sagonas C, Antonakos E, Tzimiropoulos G, Zafeiriou S, Pantic M. 300 faces in-the-wild challenge: database and results. Image Vis Comput. 2016;47:3–18. https://doi.org/10.1016/j.imavis.2016.01.002.

    Article  Google Scholar 

  34. Cech J, Soukupova T. Real-time eye blink detection using facial landmarks. Prague: Center for Machine Perception, Department of Cybernetics, Faculty of Electrical Engineering, Czech Technical University; 2016. p. 1–8.

    Google Scholar 

  35. Tensorflow. Yamnet—audio event classification. Tensorflow. https://tfhub.dev/google/yamnet/1. Accessed 25 Jan 2024.

  36. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al. MobileNets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861.

  37. Gemmeke JF, Ellis DPW, Freedman D, Jansen A, Lawrence W, Moore RC, et al. Audio Set: an ontology and human-labeled dataset for audio events. In: Proc. IEEE ICASSP 2017. New Orleans, LA; 2017.

  38. Piczak KJ. ESC-50: dataset for environmental sound classification. Karolpiczak. https://github.com/karolpiczak/ESC-50. Accessed 25 Jan 2024.

  39. HomeAssistant. Home assistant open source home automation platform. Nabu Casa, Inc. https://www.home-assistant.io. Accessed 25 Jan 2024.

  40. Oliveira SR. Intelligent sleep monitoring system. https://github.com/csoares/IntelligentSleepMonitoringSystem.git. Accessed 25 Jan 2024.

  41. Piczak KJ. ESC: Dataset for Environmental Sound Classification. In: Proceedings of the 23rd annual ACM conference on multimedia. ACM Press; 2015. p. 1015–8. http://dl.acm.org/citation.cfm?doid=2733373.2806390.

  42. Mariano VY, Min J, Park JH, Kasturi R, Mihalcik D, Li H, et al. Performance evaluation of object detection algorithms. Proc Int Conf Pattern Recogn. 2002;16(3):965–9. https://doi.org/10.1109/icpr.2002.1048198.

    Article  Google Scholar 

  43. Tzutalin. LabelImg: graphical image annotation tool for labeling object bounding boxes in images. Tzutalin. https://github.com/tzutalin/labelImg. Accessed 25 Jan 2024.

  44. Tensorboard. https://www.tensorflow.org/tensorboard. Accessed 25 Jan 2024.

Download references

Acknowledgements

Vera Almeida, PhD, Professor in Health Psychology (ORCID: 0000-0003-2803-8038), Affiliations: UCIBIO, REQUIMTE, Laboratório de Tecnologia Farmacêutica, Faculdade de Farmácia, Universidade do Porto, Rua de Jorge Viterbo Ferreira, 228, 4050-313 Porto, Portugal; UNIPRO-Unidade de Investigação em Patologia e Reabilitação Oral, Instituto Universitário de Ciências da Saúde (IUCS), CESPU, 4585-116 Gandra, Portugal.

Funding

This work was partially supported by Base Funding-UIDB/00027/2020 of the Artificial Intelligence and Computer Science Laboratory-LIACC-funded by national funds through the FCT/MCTES (PIDDAC).

Author information

Authors and Affiliations

  1. ISUS-Intelligent Sensing and Ubiquitous Systems Group, University Fernando Pessoa, Praça 9 de Abril, Porto, 4249-004, Portugal

    José Manuel Torres, Sara Oliveira, Pedro Sobral, Rui Silva Moreira & Christophe Soares

  2. LIACC-Artificial Intelligence and Computer Science Laboratory, University of Porto, Porto, Portugal

    José Manuel Torres, Pedro Sobral, Rui Silva Moreira & Christophe Soares

Authors
  1. José Manuel Torres
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sara Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pedro Sobral
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Silva Moreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christophe Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José Manuel Torres.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Torres, J.M., Oliveira, S., Sobral, P. et al. In-Home Sleep Monitoring using Edge Intelligence. SN COMPUT. SCI. 5, 538 (2024). https://doi.org/10.1007/s42979-024-02928-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-02928-9

Keywords

Search

Navigation