A novel depth image analysis for sleep posture estimation

  • Maryam S. Rasouli DEmail author
  • Shahram Payandeh
Original Research


Recognition of sleep posture and its changes are related to information monitoring in a number of health-related applications such as apnea prevention and elderly care. This paper uses a less privacy-invading approach to classify sleep postures of a person in various configurations including side and supine postures. In order to accomplish this, a single depth sensor has been utilized to collect selective depth signals and populated a dataset associated with the depth data. The data is then analyzed by a novel frequency-based feature selection approach. These extracted features were then correlated in order to rank their information content in various 2D scans from the 3D point cloud in order to train a support vector machine (SVM). The data of subjects are collected under two conditions. First when they were covered with a thin blanket and second without any blanket. In order to reduce the dimensionality of the feature space, a T-test approach is employed to determine the most dominant set of features in the frequency domain. The proposed recognition approach based on the frequency domain is also compared with an approach using feature vector defined based on skeleton joints. The comparative studies are performed given various scenarios and by a variety of datasets. Through our study, it is shown that our proposed method offers better performance to that of the joint-based method.


Sleep posture estimation Human posture estimation Machine learning Depth data 



  1. Barbara P, Ancoli-Israelb S (2001) Sleep disorders in the elderly. Sleep Med 2:99–114CrossRefGoogle Scholar
  2. Bixler EO, Vgontzas AN, Have TT, Tyson K, Kales A (1998) Effects of age on sleep apnea in men: I. Prevalence and severity. Am J Respir Crit Care Med 157:144–148CrossRefGoogle Scholar
  3. Boulay B, Brémond F, Thonnat M (2006) Applying 3d human model in a posture recognition system. Pattern Recognit Lett 27:1788–1796CrossRefGoogle Scholar
  4. Dal Mutto C, Zanuttigh P, Cortelazzo GM (2012) Time-of-flight cameras and Microsoft KinectTM. Springer Science & Business Media, BerlinCrossRefGoogle Scholar
  5. Falie D, Ichim M, David L (2008) Respiratory motion visualization and the sleep apnea diagnosis with the time of flight (ToF) camera. In: International Conference on Visualization, Imaging and Simulation, WSEAS, pp 179–184Google Scholar
  6. Fujiyoshi H, Lipton AJ, Kanade T (2004) Real-time human motion analysis by image skeletonization. IEICE Trans Inf Syst 87:113–120Google Scholar
  7. Gordon S, Grimmer K, Patricia T (2007) Understanding sleep quality and waking cervico-thoracic symptoms. Internet J Allied Health Sci Pract 5:1–12Google Scholar
  8. Hao T, Xing G, Zhou G (2013) iSleep: unobtrusive sleep quality monitoring using smartphones. In: Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems. ACM, 2013Google Scholar
  9. Karlsen K, Herlofson K, Larsen JP, Tandberg M (1999) Influence of clinical and demographic variables on quality of life in patients with Parkinson’s disease. J Neurol Neurosurg Psychiatry 66:431–435CrossRefGoogle Scholar
  10. Lachat E, Macher H, Mittet M, Landes T, Grussenmeyer P (2015) First experiences with Kinect v2 sensor for close range 3D modelling. Int Arch Photogramm Remote Sens Spatial Inf Sci 40:93CrossRefGoogle Scholar
  11. Lee J, Hong M, Ryu S (2015) Sleep monitoring system using kinect sensor. Int J Distrib Sens Netw 11:205Google Scholar
  12. Liu JJ, Xu W, Huang M-C, Alshurafa N, Sarrafzadeh M, Raut N, Yadegar B (2013) A dense pressure sensitive bedsheet design for unobtrusive sleep posture monitoring. In: 2013 IEEE international conference on pervasive computing and communications (PerCom). IEEE, pp 207–215Google Scholar
  13. Luštrek M, Kaluža B (2009) Fall detection and activity recognition with machine learning. Informatica 33:197–204Google Scholar
  14. Manzi A, Cavallo F, Dario PA (2016) Neural network approach to human posture classification and fall detection using RGB-D camera. In: Italian Forum of Ambient Assisted LivingGoogle Scholar
  15. Menon A, Kumar M (2013) Influence of body position on severity of obstructive sleep apnea: a systematic review. ISRN Otolaryngol 2013:670381Google Scholar
  16. Metsis V, Kosmopoulos D, Athitsos V, Makedon F (2014) Non-invasive analysis of sleep patterns via multimodal sensor input. Pers Ubiquitous Comput 18:19–26CrossRefGoogle Scholar
  17. Mongkolnam P, Booranrom Y, Watanapa B, Visutarrom T, Chan JH, Nukoolkit C (2017) Smart bedroom for the elderly with gesture and posture analyses using Kinect. Maejo Int J Sci Technol 11:1Google Scholar
  18. Oksenberg A, Silverberg DS (1998) The effect of body posture on sleep-related breathing disorders: facts and therapeutic implications. Sleep Med Rev 2:139–162CrossRefGoogle Scholar
  19. Oksenberg A, Dynia A, Nasser K, Gadoth N (2012) Obstructive sleep apnoea in adults: body postures and weight changes interactions. J Sleep Res 21:402–409CrossRefGoogle Scholar
  20. Panini L, Cucchiara R (2003) A machine learning approach for human posture detection in domotics applications. In: Proceedings of the 12th international conference on image analysis and processing, 2003. IEEE, pp 103–108Google Scholar
  21. Pellegrini S, Iocchi L (2007) Human posture tracking and classification through stereo vision and 3D model matching. EURASIP J Image Video Process 2008:476151Google Scholar
  22. Shotton J et al (2013) Real-time human pose recognition in parts from single depth images. Commun ACM 56:116–124CrossRefGoogle Scholar
  23. Sleep Position Gives Personality Clue (2003) BBC. Accessed 20 Sept 2016
  24. Sutherland K, Cistulli PA (2015) Recent advances in obstructive sleep apnea pathophysiology and treatment. Sleep Biol Rhythms 13:26–40CrossRefGoogle Scholar
  25. Theodoridis S, Koutroumbas K (1999) Pattern recognition. Academic Press, New YorkzbMATHGoogle Scholar
  26. Torres C, Hammond SD, Fried JC, Manjunath B (2015) Sleep pose recognition in an ICU using multimodal data and environmental feedback. In: International conference on computer vision systems. Springer, pp 56–66Google Scholar
  27. Torres C, Fragoso V, Hammond SD (2016) Eye-CU: sleep pose classification for healthcare using multimodal multiview data. In: 2016 IEEE Winter conference on applications of computer vision (WACV)Google Scholar
  28. Wachs J, Goshorn D, Kölsch M (2009) Recognizing human postures and poses in monocular still Images. IPCV 9:665–671Google Scholar
  29. Xiao Z, Mengyin F, Yi Y, Ningyi L (2012) 3D human postures recognition using kinect. In: 2012 4th international conference on intelligent human–machine systems and cybernetics (IHMSC). IEEE, pp 344–347Google Scholar
  30. Yang C, Mao Y, Cheung G, Stankovic V, Chan K (2014) Graph-based depth video denoising and event detection for sleep monitoring. In: 2014 IEEE 16th international workshop on multimedia signal processing (MMSP). IEEE, pp 1–6Google Scholar
  31. Yao K-W, Cheng C (2008) Relationships between personal, depression and social network factors and sleep quality in community-dwelling older adults. J Nurs Res 16:131–139CrossRefGoogle Scholar
  32. Yoshino A, Nishimura H (2016) Study of posture estimation system using infrared camera. In: International conference on human–computer interaction. Springer, pp 553–558Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Networked Robotics and Sensing Laboratory, School of Engineering ScienceSimon Fraser UniversityBurnabyCanada

Personalised recommendations