International Journal of Computer Vision

, Volume 122, Issue 2, pp 212–227 | Cite as

Automatic Sleep System Recommendation by Multi-modal RBG-Depth-Pressure Anthropometric Analysis

  • Cristina Palmero
  • Jordi Esquirol
  • Vanessa Bayo
  • Miquel Àngel Cos
  • Pouya Ahmadmonfared
  • Joan Salabert
  • David Sánchez
  • Sergio Escalera


This paper presents a novel system for automatic sleep system recommendation using RGB, depth and pressure information. It consists of a validated clinical knowledge-based model that, along with a set of prescription variables extracted automatically, obtains a personalized bed design recommendation. The automatic process starts by performing multi-part human body RGB-D segmentation combining GrabCut, 3D Shape Context descriptor and Thin Plate Splines, to then extract a set of anthropometric landmark points by applying orthogonal plates to the segmented human body. The extracted variables are introduced to the computerized clinical model to calculate body circumferences, weight, morphotype and Body Mass Index categorization. Furthermore, pressure image analysis is performed to extract pressure values and at-risk points, which are also introduced to the model to eventually obtain the final prescription of mattress, topper, and pillow. We validate the complete system in a set of 200 subjects, showing accurate category classification and high correlation results with respect to manual measures.


Sleep system recommendation RGB-Depth data Pressure imaging Anthropometric landmark extraction Multi-part human body segmentation 



This work has been partially supported by Spanish Project TIN2013-43478-P and®.


  1. Allen, B., Curless, B., & Popović, Z. (2004). Exploring the space of human body shapes: Data-driven synthesis under anthropometric control. Tech. rep., SAE Technical PaperGoogle Scholar
  2. Azouz, Z. B., Shu, C., & Mantel, A. (2006). Automatic locating of anthropometric landmarks on 3d human models. In: Third International Symposium on 3D Data Processing, Visualization, and Transmission, pp. 750–757. IEEE.Google Scholar
  3. Bain, D., Ferguson-Pell, M., & McLeod, A. (2003). Evaluation of mattresses using interface pressure mapping. Journal of Wound Care, 12(6), 231–235.CrossRefGoogle Scholar
  4. Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4), 509–522.CrossRefGoogle Scholar
  5. Benjumea, A. C. (2001). Datos antropométricos de la población laboral española. Prevención, trabajo y salud: Revista del Instituto Nacional de Seguridad e Higiene en el Trabajo, 14, 22–30.Google Scholar
  6. Bookstein, F. L. (1989). Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis & Machine Intelligence, 6, 567–585.CrossRefMATHGoogle Scholar
  7. Boykov, Y., & Funka-Lea, G. (2006). Graph cuts and efficient n-d image segmentation. International Journal of Computer Vision, 70(2), 109–131.CrossRefGoogle Scholar
  8. Canda, A. S., & de Deportes, C. S. (2012). Variables antropométricas de la población deportista española. Consejo Superior de Deportes, Servicio de Documentación y Publicaciones.Google Scholar
  9. Cippitelli, E., Gasparrini, S., Spinsante, S., & Gambi, E. (2015). Kinect as a tool for gait analysis: Validation of a real-time joint extraction algorithm working in side view. Sensors, 15(1), 1417–1434.CrossRefGoogle Scholar
  10. Clarkson, S., Wheat, J., Heller, B., & Choppin, S. (2014). Assessing the suitability of the microsoft kinect for calculating person specific body segment parameters. In Computer Vision-ECCV 2014 Workshops, pp. 372–385. Springer.Google Scholar
  11. de Sanidad y Consumo, M. (2008). Estudio antropométrico de la población femenina española.Google Scholar
  12. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39, 1–38.MathSciNetMATHGoogle Scholar
  13. Determann, R. M., Wolthuis, E. K., Spronk, P. E., Kuiper, M. A., Korevaar, J. C., Vroom, M. B., et al. (2007). Reliability of height and weight estimates in patients acutely admitted to intensive care units. Critical Care Nurse, 27(5), 48–55.Google Scholar
  14. DeVocht, J. W., Wilder, D. G., Bandstra, E. R., & Spratt, K. F. (2006). Biomechanical evaluation of four different mattresses. Applied Ergonomics, 37(3), 297–304.CrossRefGoogle Scholar
  15. Espitia-Contreras, A., Sanchez-Caiman, P., & Uribe-Quevedo, A. (2014). Development of a kinect-based anthropometric measurement application. In 2014 IEEE Virtual Reality (VR), pp. 71–72. IEEE.Google Scholar
  16. Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.MathSciNetCrossRefGoogle Scholar
  17. Foubert, N. (2010). Posture recognition and postural transition detection using bed-based pressure sensor. PhD thesis, Carleton University Ottawa.Google Scholar
  18. Gordon, S. J., & Grimmer-Somers, K. (2011). Your pillow may not guarantee a good night’s sleep or symptom-free waking. Physiotherapy Canada, 63(2), 183–190.CrossRefGoogle Scholar
  19. Gordon, S. J., Grimmer-Somers, K., & Trott, P. (2009). Pillow use: The behaviour of cervical pain, sleep quality and pillow comfort in side sleepers. Manual Therapy, 14(6), 671–678.CrossRefGoogle Scholar
  20. Gordon, S. J., Grimmer-Somers, K. A., & Trott, P. H. (2010). Pillow use: The behavior of cervical stiffness, headache and scapular/arm pain. Journal of Pain Research, 3, 137.Google Scholar
  21. Gupta, S., Markey, M. K., & Bovik, A. C. (2010). Anthropometric 3d face recognition. International Journal of Computer Vision, 90(3), 331–349.CrossRefGoogle Scholar
  22. Harada, T., Mori, T., Nishida, Y., Yoshimi, T., & Sato, T. (1999). Body parts positions and posture estimation system based on pressure distribution image. In Proceedings of the 1999 IEEE international conference on robotics and automation (Vol. 2, pp. 968–975). IEEE.Google Scholar
  23. Harada, T., Sato, T., & Mori, T. (2001). Pressure distribution image based human motion tracking system using skeleton and surface integration model. In Proceedings 2001 ICRA IEEE international conference on robotics and automation (Vol. 4, pp. 3201–3207). IEEE.Google Scholar
  24. Huang, S. H., & Pan, Y. C. (2014). Ergonomic job rotation strategy based on an automated rgb-d anthropometric measuring system. Journal of Manufacturing Systems, 33(4), 699–710.CrossRefGoogle Scholar
  25. Huang, W., Wai, A.A.P., Foo, S.F., Biswas, J., Hsia, C.C., & Liou, K. (2010). Multimodal sleeping posture classification. In 2010 20th international conference on pattern recognition (ICPR), pp. 4336–4339. IEEE.Google Scholar
  26. Joint, F., Organization, W. H., et al. (1985). Energy and protein requirements: report of a joint fa.Google Scholar
  27. Jonker, R., & Volgenant, A. (1987). A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing, 38(4), 325–340.MathSciNetCrossRefMATHGoogle Scholar
  28. Kim, H. S., Park, K. H., & Jeoung, J. W. (2013). Can we measure the intraocular pressure when the eyeball is against the pillow in the lateral decubitus position? Acta Ophthalmologica, 91(7), e502–e505.CrossRefGoogle Scholar
  29. Lazzaro, E., Mallick, A., Singh, M., Reich, I., Elmann, S., Stefanov, D. G., et al. (2014). The effect of positional changes on intraocular pressure during sleep in patients with and without glaucoma. Journal of Glaucoma, 23(5), 282–287.CrossRefGoogle Scholar
  30. Leilnahari, K., Fatouraee, N., Khodalotfi, M., Sadeghein, M. A., & Kashani, Y. A. (2011). Spine alignment in men during lateral sleep position: experimental study and modeling. Biomedical Engineering Online, 10(1), 103.CrossRefGoogle Scholar
  31. Liu, J.J., Xu, W., Huang, M.C., Alshurafa, N., Sarrafzadeh, M., Raut, N., & Yadegar, B. (2013). A dense pressure sensitive bedsheet design for unobtrusive sleep posture monitoring. In 2013 IEEE international conference on pervasive computing and communications (PerCom), pp 207–215. IEEE.Google Scholar
  32. Liu, J. J., Xu, W., Huang, M. C., Alshurafa, N., Sarrafzadeh, M., Raut, N., et al. (2014). Sleep posture analysis using a dense pressure sensitive bedsheet. Pervasive and Mobile Computing, 10, 34–50.CrossRefGoogle Scholar
  33. López-Torres, M., Porcar, R., Solaz, J., & Romero, T. (2008). Objective firmness, average pressure and subjective perception in mattresses for the elderly. Applied Ergonomics, 39(1), 123–130.CrossRefGoogle Scholar
  34. Lorenz, M. W., Graf, M., Henke, C., Hermans, M., Ziemann, U., Sitzer, M., et al. (2007). Anthropometric approximation of body weight in unresponsive stroke patients. Journal of Neurology, Neurosurgery & Psychiatry, 78(12), 1331–1336.CrossRefGoogle Scholar
  35. Lorenzo-Navarro, J., Castrillón-Santana, M., & Hernández-Sosa, D. (2013). On the use of simple geometric descriptors provided by rgb-d sensors for re-identification. Sensors, 13(7), 8222–8238.CrossRefGoogle Scholar
  36. Madadi, M., Escalera, S., Gonzalez, J., Roca, F. X., & Lumbreras, F. (2015). Multi-part body segmentation based on depth maps for soft biometry analysis. Pattern Recognition Letters, 56, 14–21.CrossRefGoogle Scholar
  37. Martinez, M., & Stiefelhagen, R. (2014). Kinect unbiased. In 2014 IEEE International Conference on Image Processing (ICIP), pp. 5791–5795. IEEE.Google Scholar
  38. Matsuo, J., Sugama, J., Sanada, H., Okuwa, M., Nakatani, T., Konya, C., et al. (2011). Development and validity of a new model for assessing pressure redistribution properties of support surfaces. Journal of Tissue Viability, 20(2), 55–66.CrossRefGoogle Scholar
  39. Metsis, V., Kosmopoulos, D., Athitsos, V., & Makedon, F. (2014). Non-invasive analysis of sleep patterns via multimodal sensor input. Personal and Ubiquitous Computing, 18(1), 19–26.CrossRefGoogle Scholar
  40. Miller, S., Parker, M., Blasiole, N., Beinlich, N., & Fulton, J. (2013). A prospective, in vivo evaluation of two pressure-redistribution surfaces in healthy volunteers using pressure mapping as a quality control instrument. Ostomy/Wound Management, 59(2), 44–48.Google Scholar
  41. Mogelmose, A., Bahnsen, C., Moeslund, T.B., Clapés, A., & Escalera, S. (2013). Tri-modal person re-identification with rgb, depth and thermal features. In 2013 IEEE conference on computer vision and pattern recognition workshops (CVPRW), pp 301–307. IEEE.Google Scholar
  42. Nechala, P., Mahoney, J., & Farkas, L. G. (1999). Digital two-dimensional photogrammetry: A comparison of three techniques of obtaining digital photographs. Plastic and Reconstructive Surgery, 103(7), 1819–1825.CrossRefGoogle Scholar
  43. Nguyen, T. V., Feng, J., & Yan, S. (2014). Seeing human weight from a single rgb-d image. Journal of Computer Science and Technology, 29(5), 777–784.CrossRefGoogle Scholar
  44. Reyes, M., Clapés, A., Ramírez, J., Revilla, J. R., & Escalera, S. (2013). Automatic digital biometry analysis based on depth maps. Computers in Industry, 64(9), 1316–1325.CrossRefGoogle Scholar
  45. Romero Collazos, J. (2008). Análisis de la forma y la proporcionalidad. antropometria aplicada a la nutrición.Google Scholar
  46. Rother, C., Kolmogorov, V., & Blake, A. (2004). Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (TOG), 23(3), 309–314.CrossRefGoogle Scholar
  47. Rusu, R. B. (2013). Semantic 3D object maps for everyday robot manipulation (Vol. 85). Heidelberg: Springer.Google Scholar
  48. Sáenz, Z. L., Arias, A., Guzmán, E. C., & Arias, d L. (2011). Analysis of ergonomics conditions of a brand of mattress and pillows. University-industry project, medellin-colombia. Work (Reading, Mass), 41, 1281–1287.Google Scholar
  49. Suzuki, K., Horiba, I., & Sugie, N. (2003). Linear-time connected-component labeling based on sequential local operations. Computer Vision and Image Understanding, 89(1), 1–23.Google Scholar
  50. Uhm, T., Park, H., & Park, J. I. (2015). Fully vision-based automatic human body measurement system for apparel application. Measurement, 61, 169–179.CrossRefGoogle Scholar
  51. Verhaert, V., Druyts, H., Van Deun, D., Berckmans, D., Verbraecken, J., Vandekerckhove, M., Haex, B., & Vander Sloten, J. (2011a). The use of a generic human model to personalize bed design. In: Proceedings of 1st international symposium on digital human modeling, vol 2202. Lyon, June, paper ID.Google Scholar
  52. Verhaert, V., Druyts, H., Van Deun, D., De Wilde, T., Van Brussel, K., Haex, B., et al. (2011b). Modeling human-bed interaction: The predictive value of anthropometric models in choosing the correct bed support. Work (Reading, Mass), 41, 2268–2273.Google Scholar
  53. Verhaert, V., Haex, B., Wilde, T. D., Berckmans, D., Verbraecken, J., Valck, Ed, et al. (2011c). Ergonomics in bed design: the effect of spinal alignment on sleep parameters. Ergonomics, 54(2), 169–178.Google Scholar
  54. Wang, Q., Jagadeesh, V., Ressler, B., & Piramuthu, R. (2014). Im2fit: Fast 3d model fitting and anthropometrics using single consumer depth camera and synthetic data. arXiv:1410.0745.
  55. Wong, M., Lai, A., Singh, M., & Chew, P. (2013). Sleeping posture and intraocular pressure. Singapore Medical Journal, 54(3), 146–148.CrossRefGoogle Scholar
  56. Yang, C., Mao, Y., Cheung, G., Stankovic, V., & Chan, K. L. (2014). Graph-based depth video denoising and event detection for sleep monitoring. In 2014 IEEE 16th international workshop on multimedia signal processing (MMSP), IEEE, pp 1–6.Google Scholar
  57. Yousefi, R., Ostadabbas, S., Faezipour, M., Nourani, M., Tamil, L., & Pompeo, M. (2011). Posture and limb detection for pressure ulcer prevention. In Proceedings of the Southern Biomedical Engineering Conference (SBEC).Google Scholar
  58. Yu, M. C., Wu, H., Liou, J. L., Lee, M. S., & Hung, Y. P. (2012). Breath and position monitoring during sleeping with a depth camera. In HEALTHINF, pp 12–22.Google Scholar
  59. Zuberi, N. A., Rekab, K., & Nguyen, H. V. (2004). Sleep apnea avoidance pillow effects on obstructive sleep apnea syndrome and snoring. Sleep and Breathing, 8(4), 201–207.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Cristina Palmero
    • 1
  • Jordi Esquirol
    • 2
  • Vanessa Bayo
    • 2
  • Miquel Àngel Cos
    • 2
  • Pouya Ahmadmonfared
    • 1
  • Joan Salabert
    • 3
  • David Sánchez
    • 3
  • Sergio Escalera
    • 1
    • 4
  1. 1.Computer Vision CenterCerdanyola del VallèsSpain
  2. 2.Servei Universitari de Recerca en Fisioteràpia (S.U.R.F), Escola Universitària GimbernatSant Cugat del VallèsSpain
  3.®Sant Cugat del VallèsSpain
  4. 4.Dept. Matemàtica Aplicada i Anàlisi, UBBarcelonaSpain

Personalised recommendations