Vision-based patient monitoring: a comprehensive review of algorithms and technologies

  • Supriya Sathyanarayana
  • Ravi Kumar Satzoda
  • Suchitra Sathyanarayana
  • Srikanthan Thambipillai
Original Research

Abstract

Vision-based monitoring for assisted living is gaining increasing attention, especially in multi-modal monitoring systems owing to the several advantages of vision-based sensors. In this paper, a detailed survey of some of the important vision-based patient monitoring applications is presented, namely (a) fall detection (b) action and activity monitoring (c) sleep monitoring (d) respiration and apnea monitoring (e) epilepsy monitoring (f) vital signs monitoring and (g) facial expression monitoring. The challenges and state-of-art technologies in each of these applications is presented. This is the first work to present such a comprehensive survey with the focus on a set of seven most common applications pertaining to patient monitoring. Potential future directions are presented while also considering practical large scale deployment of vision-based systems in patient monitoring. One of the important conclusions drawn is that rather than applying generic algorithms, use of the application context of patient monitoring can be a useful way to develop novel techniques that are robust and yet cost-effective.

Keywords

Survey Patient monitoring Elderly care Remote monitoring Computer vision 

References

  1. AL-Khalidi F, Saatchi R, Burke D, Elphick H (2010) Tracking human face features in thermal images for respiration monitoring. In: 2010 IEEE/ACS Intl. Conf. on Computer Systems and Applications (AICCSA), pp 1–6Google Scholar
  2. AL-Khalidi F, Saatchi R, Burke D, Elphick H, Tan S (2011) Respiration rate monitoring methods: a review. Pediatr Pulmonol 46(6):523–529CrossRefGoogle Scholar
  3. Alghowinem S, Goecke R, Wagner M, Parker G, Breakspear M (2013a) Eye movement analysis for depression detection. In: 2013 20th IEEE Intl Conf on Image Processing (ICIP), pp 4220–4224Google Scholar
  4. Alghowinem S, Goecke R, Wagner M, Parkerx G, Breakspear M (2013b) Head Pose and Movement Analysis as an Indicator of Depression. In: 2013 Humaine Association Conf. on Affective Computing and Intelligent Interaction (ACII), pp 283–288Google Scholar
  5. Alkali A, Saatchi R, Elphick H, Burke D (2013) Facial Tracking in Thermal Images for Real-Time Noncontact Respiration Rate Monitoring. In: Modelling Symposium (EMS), 2013 European, pp 265–270Google Scholar
  6. Alvino C, Kohler C, Barrett F, Gur RE, Gur RC, Verma R (2007) Computerized measurement of facial expression of emotions in schizophrenia. J Neurosci Methods 163(2):350–361CrossRefGoogle Scholar
  7. Amoretti M, Copelli S, Wientapper F, Furfari F, Lenzi S, Chessa S (2011) Sensor data fusion for activity monitoring in the PERSONA ambient assisted living project. J Ambient Intell Human Comput 4(1):67–84CrossRefGoogle Scholar
  8. Anderson D, Luke RH, Keller JM, Skubic M, Rantz M, Aud M (2009) Linguistic Summarization of Video for Fall Detection Using Voxel Person and Fuzzy Logic. Comput Vis Image Underst 113(1):80–89CrossRefGoogle Scholar
  9. Aoki H, Koshiji K, Nakamura H, Takemura Y, Nakajima M (2005) Study on respiration monitoring method using near-infrared multiple slit-lights projection. In: 2005 IEEE Intl. Symposium on Micro-NanoMechatronics and Human Science, pp 291–296Google Scholar
  10. Ashraf AB, Prkachin K, Chen T, Lucey S, Solomon P, Ambadar Z, Cohn JF, Theobald BJ (2007) The painful face—pain expression recognition using active appearance models. In: In 9th Intl Conf on Multimodal Interfaces ICMI, pp 9–14Google Scholar
  11. Ashraf AB, Lucey S, Cohn JF, Chen T, Ambadar Z, Prkachin KM, Solomon PE (2009) The painful face—pain expression recognition using active appearance models. Image Vis Comput 27(12):1788–1796CrossRefGoogle Scholar
  12. Auvinet E, Reveret L, St-Arnaud A, Rousseau J, Meunier J (2008) Fall detection using multiple cameras. In: 30th Annual Intl Conf of the IEEE Engineering in Medicine and Biology Society, 2008. EMBS 2008, pp 2554–2557Google Scholar
  13. Auvinet E, Rougier C, Meunier J, St-Arnaud A, Rousseau J (2010) Multiple cameras fall dataset. DIRO-Université de Montréal, Tech Rep 1350Google Scholar
  14. Auvinet E, Multon F, Saint-Arnaud A, Rousseau J, Meunier J (2011) Fall detection with multiple cameras: an occlusion-resistant method based on 3-D silhouette vertical distribution. IEEE Trans Inf Technol Biomed 15(2):290–300CrossRefGoogle Scholar
  15. Bai YW, Li WT, Chen YW (2010) Design and implementation of an embedded monitor system for detection of a patient’s breath by double Webcams. In: 2010 IEEE Intl Workshop on Medical Measurements and Applications Proceedings (MeMeA), pp 171–176Google Scholar
  16. Bai YW, Tsai CL, Wu SC (2012) Design of a breath detection system with multiple remotely enhanced hand-computer interaction devices. In: 2012 IEEE 16th Intl. Symposium on Consumer Electronics (ISCE), pp 1–5Google Scholar
  17. Balakrishnan G, Durand F, Guttag J (2013) Detecting Pulse from Head Motions in Video. In: 2013 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp 3430–3437Google Scholar
  18. Banerjee T, Keller J, Skubic M, Stone E (2014) Day or night activity recognition from video using fuzzy clustering techniques. IEEE Trans Fuzzy Syst 22(3):483–493CrossRefGoogle Scholar
  19. Belbachir A, Schraml S, Nowakowska A (2011) Event-driven stereo vision for fall detection. In: 2011 IEEE Computer Society Conf on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 78–83Google Scholar
  20. Belbachir A, Litzenberger M, Schraml S, Hofstatter M, Bauer D, Schon P, Humenberger M, Sulzbachner C, Lunden T, Merne M (2012) CARE: A dynamic stereo vision sensor system for fall detection. In: 2012 IEEE Intl Symposium on Circuits and Systems (ISCAS), pp 731–734Google Scholar
  21. Bevilacqua V, D’Ambruoso D, Mandolino G, Suma M (2011) A new tool to support diagnosis of neurological disorders by means of facial expressions. In: 2011 IEEE Intl. Workshop on Medical Measurements and Applications Proc. (MeMeA), pp 544–549Google Scholar
  22. Bieber G, Hoffmeyer A, Gutzeit E, Peter C, Urban B (2009) Activity Monitoring by Fusion of Optical and Mechanical Tracking Technologies for User Behavior Analysis. In: Proc. of the 2Nd Intl. Conf. on PErvasive Technologies Related to Assistive Environments, ACM, PETRA ’09, pp 45:1–45:6Google Scholar
  23. Bin Mansor MN, Yaacob S, Nagarajan R, Hariharan M (2010) Detection of facial changes for hospital ICU patients. In: 2010 6th Intl. Colloquium on Signal Processing and Its Applications (CSPA), pp 1–5Google Scholar
  24. Brulin D, Benezeth Y, Courtial E (2012) Posture recognition based on fuzzy logic for home monitoring of the elderly. IEEE Trans Inf Technol Biomed 16(5):974–982CrossRefGoogle Scholar
  25. Brusco N, Paviotti A (2005) 3d quantification of facial edemas. In: Proc. of the 4th Intl. Symposium on Image and Signal Processing and Analysis, 2005. ISPA 2005, pp 191–196Google Scholar
  26. Calhoun DA (2010) Sleep and hypertension. CHEST J 138(2):434CrossRefGoogle Scholar
  27. Chaaraoui AA, Climent-Prez P, Flrez-Revuelta F (2012) A review on vision techniques applied to human behaviour analysis for ambient-assisted living. Expert Syst Appl 39(12):10873–10888CrossRefGoogle Scholar
  28. Charfi I, Miteran J, Dubois J, Atri M, Tourki R (2012) Definition and Performance Evaluation of a Robust SVM Based Fall Detection Solution. In: 2012 Eighth Intl. Conf. on Signal Image Technology and Internet Based Systems (SITIS), pp 218–224Google Scholar
  29. Charfi I, Miteran J, Dubois J, Atri M, Tourki R (2013) Optimized spatio-temporal descriptors for real-time fall detection: comparison of support vector machine and Adaboost-based classification. J Electron Imaging 22(4):041106CrossRefGoogle Scholar
  30. Chekmenev SY, Rara H, Farag AA (2005) Non-contact, wavelet-based measurement of vital signs using thermal imaging. In: The first Intl. Conf on graphics, vision, and image processing (GVIP), pp 107–112Google Scholar
  31. Chen LL, Chen KW, Hung YP (2014) A sleep monitoring system based on audio, video and depth information for detecting sleep events. In: 2014 IEEE Intl. Conf. on Multimedia and Expo (ICME), pp 1–6Google Scholar
  32. Chen MY (2010) Long Term Activity Analysis in Surveillance Video Archives. Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA, USAGoogle Scholar
  33. Cheng H, Liu Z, Zhao Y, Ye G (2011) Real world activity summary for senior home monitoring. In: 2011 IEEE Intl. Conf. on Multimedia and Expo (ICME), pp 1–4Google Scholar
  34. Chung PC, Liu CD (2008) A daily behavior enabled hidden Markov model for human behavior understanding. Pattern Recognit 41(5):1572–1580MATHCrossRefGoogle Scholar
  35. Cohn J, Kruez T, Matthews I, Yang Y, Nguyen MH, Padilla M, Zhou F, De la Torre F (2009) Detecting depression from facial actions and vocal prosody. In: 3rd Intl. Conf. on Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009, pp 1–7Google Scholar
  36. Crispim C, Bathrinarayanan V, Fosty B, Konig A, Romdhane R, Thonnat M, Bremond F (2013) Evaluation of a monitoring system for event recognition of older people. In: 2013 10th IEEE Intl. Conf. on Advanced Video and Signal Based Surveillance (AVSS), pp 165–170Google Scholar
  37. Cucchiara R, Prati A, Vezzani R (2007) A multi-camera vision system for fall detection and alarm generation. Expert Syst 24(5):334–345CrossRefGoogle Scholar
  38. Cuppens K, Lagae L, Vanrumste B (2009) Towards automatic detection of movement during sleep in pediatric patients with epilepsy by means of video recordings and the optical flow algorithm. In: 4th European Conference of the International Federation for Medical and Biological Engineering, Springer, pp 784–789Google Scholar
  39. Cuppens K, Lagae L, Ceulemans B, Huffel SV, Vanrumste B (2010) Automatic video detection of body movement during sleep based on optical flow in pediatric patients with epilepsy. Med Biol Eng Comput 48(9):923–931CrossRefGoogle Scholar
  40. Cuppens K, Chen CW, Wong KBY, Van de Vel A, Lagae L, Ceulemans B, Tuytelaars T, Van Huffel S, Vanrumste B, Aghajan H (2012) Using Spatio-Temporal Interest Points (STIP) for myoclonic jerk detection in nocturnal video. Conf Proc IEEE Eng Med Biol Soc 2012:4454–4457Google Scholar
  41. Dai Y, Shibata Y, Ishii T, Hashimoto K, Katamachi K, Noguchi K, Kakizaki N, Cai D (2001) An associate memory model of facial expressions and its application in facial expression recognition of patients on bed. In: IEEE Intl. Conf. on Multimedia and Expo, 2001. ICME 2001, pp 591–594, 00014Google Scholar
  42. Dickinson P, Hunter A (2008) Using Inactivity to Detect Unusual behavior. In: IEEE Workshop on Motion and video Computing, 2008. WMVC 2008, pp 1–6Google Scholar
  43. Diraco G, Leone A, Siciliano P (2010) An active vision system for fall detection and posture recognition in elderly healthcare. In: Design, Automation Test in Europe Conf Exhibition (DATE), 2010, pp 1536–1541Google Scholar
  44. Eguchi K, Hoshide S, Ishikawa S, Shimada K, Kario K (2010) Short sleep duration is an independent predictor of stroke events in elderly hypertensive patients. J Am Soc Hypertens 4(5):255–262CrossRefGoogle Scholar
  45. Ekman P, Friesen W (1978) Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto, CAGoogle Scholar
  46. Ellgring H (1989) Non verbal Communication in Depression. Cambridge University Press, Cambridge, EnglandGoogle Scholar
  47. Engel J, Epilepsy Intl League Against, (ILAE), (2001) A proposed diagnostic scheme for people with epileptic seizures and with epilepsy: report of the ILAE Task Force on Classification and Terminology. Epilepsia 42(6):796–803Google Scholar
  48. Espa F, Ondze B, Deglise P, Billiard M, Besset A (2000) Sleep architecture, slow wave activity, and sleep spindles in adult patients with sleepwalking and sleep terrors. Clin Neurophysiol 111(5):929–939CrossRefGoogle Scholar
  49. Falie D, Ichim M, David L (2008) Respiratory Motion Visualization and the Sleep Apnea Diagnosis with the Time of Flight (TOF) CameraGoogle Scholar
  50. Falie D, David L, Ichim M (2009) Statistical algorithm for detection and screening sleep apnea. In: Intl. Symposium on Signals, Circuits and Systems, 2009. ISSCS 2009, pp 1–4Google Scholar
  51. Fei J, Pavlidis I (2006) Analysis of breathing air flow patterns in thermal imaging. In: Engineering in Medicine and Biology Society, 2006. EMBS’06. 28th Annual International Conference of the IEEE, IEEE, pp 946–952Google Scholar
  52. Fei J, Pavlidis I, Murthy J (2009) Thermal vision for sleep apnea monitoring. Med Image Comput Comput Assist Interv 12(Pt 2):1084–1091Google Scholar
  53. Fleck S, Strasser W (2008) Smart camera based monitoring system and its application to assisted living. Proc IEEE 96(10):1698–1714CrossRefGoogle Scholar
  54. Foo Siang Fook V, Thang PV, Htwe TM, Qiang Q, Wai AAP, Jayachandran M, Biswas J, Yap P (2007) Automated Recognition of Complex Agitation Behavior of Dementia Patients Using Video Camera. 2007 9th Intl. Conf. on e-Health Networking, Application and Services, pp 68–73Google Scholar
  55. Frigola M, Amat J, Pagès J (2002) Vision Based Respiratory Monitoring System. In: Proceedings of the 10th Mediterranean Conference on Control and AutomationMED2002 Lisbon, PortugalGoogle Scholar
  56. Fu Z, Delbruck T, Lichtsteiner P, Culurciello E (2008) An address-event fall detector for assisted living applications. IEEE Trans Biomed Circuits Syst 2(2):88–96CrossRefGoogle Scholar
  57. Gade R, Moeslund TB (2013) Thermal cameras and applications: a survey. Mach Vis Appl 25(1):245–262CrossRefGoogle Scholar
  58. Gault T, Farag A (2013) A Fully Automatic Method to Extract the Heart Rate from Thermal Video. In: 2013 IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 336–341Google Scholar
  59. Gault T, Blumenthal N, Farag A, Starr T (2010) Extraction of the superficial facial vasculature, vital signs waveforms and rates using thermal imaging. In: 2010 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 1–8Google Scholar
  60. Gholami B, Haddad WM, Tannenbaum AR (2009) Agitation and pain assessment using digital imaging. Conf Proc IEEE Eng Med Biol Soc 2009:2176–2179Google Scholar
  61. Gilbert CA, Lilley CM, Craig KD, McGrath PJ, Court CA, Bennett SM, Montgomery CJ (1999) Postoperative pain expression in preschool children: validation of the child facial coding system. Clin J Pain 15(3):192–200CrossRefGoogle Scholar
  62. Gohil B, Gholamhhosseini H, Harrison MJ, Lowe A, Al-Jumaily A (2007) Intelligent monitoring of critical pathological events during anesthesia. Conf Proc IEEE Eng Med Biol Soc 2007:4343–4346Google Scholar
  63. Grassi M, Lombardi A, Rescio G, Malcovati P, Malfatti M, Gonzo L, Leone A, Diraco G, Distante C, Siciliano P, Libal V, Huang J, Potamianos G (2008) A hardware-software framework for high-reliability people fall detection. In: 2008 IEEE Sensors, pp 1328–1331Google Scholar
  64. Hamm J, Kohler CG, Gur RC, Verma R (2011) Automated Facial Action Coding System for Dynamic Analysis of Facial Expressions in Neuropsychiatric Disorders. J Neurosci Methods 200(2):237–256CrossRefGoogle Scholar
  65. Hammal Z, Cohn JF (2012) Automatic Detection of Pain Intensity. In: Proc. of the 14th ACM Intl. Conf. on Multimodal Interaction, ACM, pp 47–52Google Scholar
  66. Hammal Z, Kunz M (2012) Pain monitoring: a dynamic and context-sensitive system. Pattern Recognit 45(4):1265–1280CrossRefGoogle Scholar
  67. Hawley-Hague H, Boulton E, Hall A, Pfeiffer K, Todd C (2014) Older adults perceptions of technologies aimed at falls prevention, detection or monitoring: a systematic review. Intl J Med Inf 83(6):416–426CrossRefGoogle Scholar
  68. Hazelhoff L, Han J, With PH (2008) Video-Based Fall Detection in the Home Using Principal Component Analysis. In: Proceedings of the 10th Intl Conf on Advanced Concepts for Intelligent Vision Systems, Springer-Verlag, Berlin, Heidelberg, ACIVS ’08, pp 298–309Google Scholar
  69. Hijaz F, Afzal N, Ahmad T, Hasan O (2010) Survey of fall detection and daily activity monitoring techniques. In: 2010 Intl Conf on Information and Emerging Technologies (ICIET), pp 1–6Google Scholar
  70. Hirayama M, Nakamura T, Hori N, Koike Y, Sobue G (2008) The video images of sleep attacks in Parkinson’s disease. Mov Disord 23(2):288–290CrossRefGoogle Scholar
  71. Horgas A, Miller L (2008) Pain assessment in people with dementia. AJN Am J Nurs 108(7):62–70CrossRefGoogle Scholar
  72. Humenberger M, Schraml S, Sulzbachner C, Belbachir A, Srp A, Vajda F (2012) Embedded fall detection with a neural network and bio-inspired stereo vision. In: 2012 IEEE Computer Society Conf on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 60–67Google Scholar
  73. Iasemidis L (2003) Epileptic seizure prediction and control. IEEE Trans Biomed Eng 50(5):549–558CrossRefGoogle Scholar
  74. Igual R, Medrano C, Plaza I (2013) Challenges, issues and trends in fall detection systems. BioMed Eng OnLine 12(1):66CrossRefGoogle Scholar
  75. Jansen B, Deklerck R (2006) Context aware inactivity recognition for visual fall detection. Pervasive Health Conf Workshops 2006:1–4Google Scholar
  76. Jansen B, Temmermans F, Deklerck R (2007) 3d human pose recognition for home monitoring of elderly. In: 29th Annual Intl Conf of the IEEE Engineering in Medicine and Biology Society, 2007. EMBS 2007, pp 4049–4051Google Scholar
  77. Jansen B, Rebel S, Deklerck R, Mets T, Schelkens P (2008) Detection of activity pattern changes among elderly with 3d camera technology. In: Photonics Europe, International Society for Optics and Photonics, pp 70,000O–70,000OGoogle Scholar
  78. Jiang G, Kang L (2007) Character Analysis of Facial Expression Thermal Image. In: IEEE/ICME Intl. Conf. on Complex Medical Engineering, 2007. CME 2007, pp 824–827Google Scholar
  79. Johnson KD, Winkelman C, Burant CJ, Dolansky M, Totten V (2014) The factors that affect the frequency of vital sign monitoring in the emergency department. J Emerg Nurs 40(1):27–35CrossRefGoogle Scholar
  80. Johnson ML, Price PA, Jovanov E (2007) A new method for the quantification of breathing. In: Engineering in Medicine and Biology Society, 2007. EMBS 2007. 29th Annual International Conference of the IEEE, IEEE, pp 4568–4571Google Scholar
  81. Joshi J, Dhall A, Goecke R, Breakspear M, Parker G (2012) Neural-net classification for spatio-temporal descriptor based depression analysis. In: 2012 21st Intl. Conf. on Pattern Recognition (ICPR), pp 2634–2638Google Scholar
  82. Joshi J, Goecke R, Alghowinem S, Dhall A, Wagner M, Epps J, Parker G, Breakspear M (2013a) Multimodal assistive technologies for depression diagnosis and monitoring. J Multimodal User Interfaces 7(3):217–228CrossRefGoogle Scholar
  83. Joshi J, Goecke R, Parker G, Breakspear M (2013b) Can body expressions contribute to automatic depression analysis? In: 2013 10th IEEE Intl. Conf. and Workshops on Automatic Face and Gesture Recognition (FG), pp 1–7Google Scholar
  84. Joumier V, Romdhane R, Bremond F, Thonnat M, Mulin E, Robert P, Derreumaux A, Piano J, Lee J (2011) Video Activity Recognition Framework for assessing motor behavioural disorders in Alzheimer Disease Patients. In: International Workshop on Behaviour Analysis and Video Understanding (ICVS 2011), p 9Google Scholar
  85. Junior CFC, Bremond F, Joumier V (2012) A Multi-Sensor Approach for Activity Recognition in Older Patients. In: Second International Conference on Ambient Computing, Applications, Services and TechnologiesGoogle Scholar
  86. Kalitzin S, Petkov G, Velis D, Vledder B, Lopes da Silva F (2012) Automatic segmentation of episodes containing epileptic clonic seizures in video sequences. IEEE Trans Biomed Eng 59(12):3379–3385CrossRefGoogle Scholar
  87. Kanade T, Cohn J, Tian Y (2000) Comprehensive database for facial expression analysis. In: Fourth IEEE Intl. Conf. on Automatic Face and Gesture Recognition, 2000. Proc., pp 46–53Google Scholar
  88. Karaman S, Benois-Pineau J, Mgret R, Dovgalecs V, Dartigues JF, Gastel Y (2010) Human Daily Activities Indexing in Videos from Wearable Cameras for Monitoring of Patients with Dementia Diseases. In: 2010 20th Intl. Conf. on Pattern Recognition (ICPR), pp 4113–4116Google Scholar
  89. Kayyali HA, Weimer S, Frederick C, Martin C, Basa D, Juguilon JA, Jugilioni F (2008) Remotely attended home monitoring of sleep disorders. Telemed J E Health 14(4):371–374CrossRefGoogle Scholar
  90. Khan R, Meyer A, Konik H, Bouakaz S (2013) Pain detection through shape and appearance features. In: 2013 IEEE Intl. Conf. on Multimedia and Expo (ICME), pp 1–6Google Scholar
  91. Khan Z, Sohn W (2011) Abnormal human activity recognition system based on R-transform and kernel discriminant technique for elderly home care. IEEE Trans Consum Electron 57(4):1843–1850CrossRefGoogle Scholar
  92. Kosmopoulos D, Antonakaki P, Valasoulis K, Katsoulas D (2008) Monitoring Human Behavior in an Assistive Environment Using Multiple Views. In: Proc. of the 1st Intl. Conf. on PErvasive Technologies Related to Assistive Environments, ACM, New York, NY, USA, PETRA ’08, pp 32:1–32:6Google Scholar
  93. Kranjec J, Begu S, Gerak G, Drnovek J (2014) Non-contact heart rate and heart rate variability measurements: a review. Biomed Signal Process Control 13:102–112CrossRefGoogle Scholar
  94. Kroutil J, Laposa A, Husak M (2011) Respiration Monitoring During Sleeping. In: Proc. of the 4th Intl. Symposium on Applied Sciences in Biomedical and Communication Technologies, ACM, New York, NY, USA, ISABEL ’11, pp 33:1–33:5Google Scholar
  95. Kunz M, Scharmann S, Hemmeter U, Schepelmann K, Lautenbacher S (2007) The facial expression of pain in patients with dementia. PAIN 133(1–3):221–228CrossRefGoogle Scholar
  96. Kuo YM, Lee JS, Pc Chung (2010) A visual context-awareness-based sleeping-respiration measurement system. IEEE Trans Inf Technol Biomed 14(2):255–265CrossRefGoogle Scholar
  97. Kwolek B, Kepski M (2014) Human fall detection on embedded platform using depth maps and wireless accelerometer. Comput Methods Programs Biomed 117(3):489–501CrossRefGoogle Scholar
  98. Kwon S, Kim H, Park KS (2012) Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. In: 2012 Annual Intl. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 2174–2177Google Scholar
  99. Leone A, Diraco G, Siciliano P (2011) Detecting falls with 3d range camera in ambient assisted living applications: a preliminary study. Med Eng Phys 33(6):770–781CrossRefGoogle Scholar
  100. Lewandowska M, Ruminski J, Kocejko T, Nowak J (2011) Measuring pulse rate with a webcam; A non-contact method for evaluating cardiac activity. In: 2011 Federated Conf. on Computer Science and Information Systems (FedCSIS), pp 405–410Google Scholar
  101. Li X, Chen J, Zhao G, Pietikainen M (2014) Remote Heart Rate Measurement from Face Videos under Realistic Situations. In: Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, IEEE, pp 4264–4271Google Scholar
  102. Li Z, da Silva A, Cunha J (2002) Movement quantification in epileptic seizures: a new approach to video-EEG analysis. IEEE Trans Biomed Eng 49(6):565–573CrossRefGoogle Scholar
  103. Liao W, Zhang W, Zhu Z, Ji Q (2005) A Real-Time Human Stress Monitoring System Using Dynamic Bayesian Network. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition—Workshops, 2005. CVPR Workshops, pp 70–70Google Scholar
  104. Liao WH, Yang CM (2008) Video-based activity and movement pattern analysis in overnight sleep studies. In: 19th Intl Conf on Pattern Recognition, 2008. ICPR 2008, pp 1–4Google Scholar
  105. Littlewort GC, Bartlett MS, Lee K (2007) Faces of Pain: Automated Measurement of Spontaneousallfacial Expressions of Genuine and Posed Pain. In: Proc. of the 9th Intl. Conf. on Multimodal Interfaces, ACM, New York, NY, USA, ICMI ’07, pp 15–21Google Scholar
  106. Liu Q, Sun M, Sclabassi R (2004) Illumination-invariant change detection model for patient monitoring video. In: 26th Annual Intl. Conf. of the IEEE Engineering in Medicine and Biology Society, 2004. IEMBS ’04, vol 1, pp 1782–1785Google Scholar
  107. Lu H, Pan Y, Mandal B, Eng HL, Guan C, Chan DWS (2013) Quantifying limb movements in epileptic seizures through color-based video analysis. IEEE Trans Biomed Eng 60(2):461–469CrossRefGoogle Scholar
  108. Lucey P, Cohn J, Lucey S, Matthews I, Sridharan S, Prkachin KM (2009) Automatically detecting pain using facial actions. Int Conf Affect Comput Intell Interact Workshops 2009:1–8Google Scholar
  109. Lucey P, Cohn J, Matthews I, Lucey S, Sridharan S, Howlett J, Prkachin K (2011a) Automatically detecting pain in video through facial action units. IEEE Trans Syst Man Cybern Part B: Cybern 41(3):664–674CrossRefGoogle Scholar
  110. Lucey P, Cohn JF, Prkachin KM, Solomon PE, Matthews I (2011b) Painful data: The unbc-mcmaster shoulder pain expression archive database. In: Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, IEEE, pp 57–64Google Scholar
  111. Lucey P, Cohn JF, Prkachin KM, Solomon PE, Chew S, Matthews I (2012) Painful monitoring: automatic pain monitoring using the UNBC-McMaster shoulder pain expression archive database. Image Vis Comput 30(3):197–205CrossRefGoogle Scholar
  112. Makantasis K, Protopapadakis E, Doulamis A, Grammatikopoulos L, Stentoumis C (2012) Monocular Camera Fall Detection System Exploiting 3d Measures: A Semi-supervised Learning Approach. In: Fusiello A, Murino V, Cucchiara R (eds) Computer Vision ECCV 2012. Workshops and Demonstrations, no. 7585 in Lecture Notes in Computer Science, Springer Berlin Heidelberg, pp 81–90Google Scholar
  113. Malakuti K, Albu A (2010) Towards an Intelligent Bed Sensor: Non-intrusive Monitoring of Sleep Irregularities with Computer Vision Techniques. In: 2010 20th Intl Conf on Pattern Recognition (ICPR), pp 4004–4007Google Scholar
  114. Mandal MK, Pandey R, Prasad AB (1998) Facial expressions of emotions and schizophrenia: a review. Schizophr Bull 24(3):399–412CrossRefGoogle Scholar
  115. Manfredi PL, Breuer B, Meier DE, Libow L (2003) Pain assessment in elderly patients with severe dementia. J Pain Symptom Manag 25(1):48–52CrossRefGoogle Scholar
  116. Martinez M, Stiefelhagen R (2012) Breath rate monitoring during sleep using near-ir imagery and PCA. In: 2012 21st Intl. Conf. on Pattern Recognition (ICPR), pp 3472–3475Google Scholar
  117. Mastorakis G, Makris D (2012) Fall detection system using Kinects infrared sensor. J Real-Time Image Proc pp 1–12Google Scholar
  118. Matic A, Mehta P, Rehg J, Osmani V, Mayora O (2010) AID-ME: Automatic identification of dressing failures through monitoring of patients and activity Evaluation. In: Pervasive Computing Technologies for Healthcare (PervasiveHealth), 2010 4th Intl. Conf. on-NO PERMISSIONS, pp 1–8Google Scholar
  119. McDuff D, El Kaliouby R, Senechal T, Amr M, Cohn J, Picard R (2013) Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected In-the-Wild. In: 2013 IEEE Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 881–888Google Scholar
  120. McIntyre G, Gocke R, Hyett M, Green M, Breakspear M (2009) An approach for automatically measuring facial activity in depressed subjects. In: 3rd Intl. Conf. on Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009, pp 1–8Google Scholar
  121. Mihailidis A, Carmichael B, Boger J (2004) The use of computer vision in an intelligent environment to support aging-in-place, safety, and independence in the home. IEEE Trans Inf Technol Biomed 8(3):238–247CrossRefGoogle Scholar
  122. Mirmahboub B, Samavi S, Karimi N, Shirani S (2013) Automatic monocular system for human fall detection based on variations in silhouette area. IEEE Trans Biomed Eng 60(2):427–436CrossRefGoogle Scholar
  123. Mishima K, Sugahara T (2009) Analysis methods for facial motion. Jpn Dental Sci Rev 45(1):4–13CrossRefGoogle Scholar
  124. Mubashir M, Shao L, Seed L (2013) A survey on fall detection: principles and approaches. Neurocomputing 100:144–152CrossRefGoogle Scholar
  125. Murthy R, Pavlidis I, Tsiamyrtzis P (2004) Touchless monitoring of breathing function. Conf Proc IEEE Eng Med Biol Soc 2:1196–1199Google Scholar
  126. Nait-Charif H, McKenna S (2004) Activity summarisation and fall detection in a supportive home environment. In: Proceedings of the 17th Intl Conf on Pattern Recognition, 2004. ICPR 2004, vol 4, pp 323–326 Vol.4Google Scholar
  127. Nakajim K, Matsumoto Y, Tamura T (2001) Development of real-time image sequence analysis for evaluating posture change and respiratory rate of a subject in bed. Physiol Meas 22(3):N21–28CrossRefGoogle Scholar
  128. Nakajima K, Matsumoto Y, Tamura T (2000) A monitor for posture changes and respiration in bed using real time image sequence analysis. In: Proc. of the 22nd Annual Intl. Conf. of the IEEE Engineering in Medicine and Biology Society, 2000, vol 1, pp 51–54 vol.1Google Scholar
  129. Nanni L, Brahnam S, Lumini A (2010) A local approach based on a local binary patterns variant texture descriptor for classifying pain states. Expert Syst Appl 37(12):7888–7894CrossRefGoogle Scholar
  130. Nasution A, Emmanuel S (2007) Intelligent Video Surveillance for Monitoring Elderly in Home Environments. In: IEEE 9th Workshop on Multimedia Signal Processing, 2007. MMSP 2007, pp 203–206Google Scholar
  131. Naufal Mansor M, Yaacob S, Nagarajan R, Che LS, Hariharan M, Ezanuddin M (2010) Detection of facial changes for ICU patients using KNN classifier. In: 2010 Intl. Conf. on Intelligent and Advanced Systems (ICIAS), pp 1–5Google Scholar
  132. Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K (2002) Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 346(22):1715–1722CrossRefGoogle Scholar
  133. Neubauer DN (1999) Sleep problems in the elderly. Am Fam Phys 59(9):2551–2558, 2559–2560Google Scholar
  134. Ni B, Dat NC, Moulin P (2012) RGBD-camera based get-up event detection for hospital fall prevention. In: 2012 IEEE Intl Conf on Acoustics, Speech and Signal Processing (ICASSP), pp 1405–1408Google Scholar
  135. Nicolas Thome SM (2008) A real-time, multi-view fall detection system: a LHMM-based approach. IEEE Trans Circuits Syst Video Technol 18:1522–1532CrossRefGoogle Scholar
  136. Noachtar S, Peters AS (2009) Semiology of epileptic seizures: a critical review. Epilepsy Behav 15(1):2–9CrossRefGoogle Scholar
  137. Noury N, Fleury A, Rumeau P, Bourke A, Laighin G, Rialle V, Lundy JE (2007) Fall detection—Principles and Methods. In: 29th Annual Intl Conf of the IEEE Engineering in Medicine and Biology Society, 2007. EMBS 2007, pp 1663–1666Google Scholar
  138. Obdrzalek S, Kurillo G, Ofli F, Bajcsy R, Seto E, Jimison H, Pavel M (2012) Accuracy and robustness of Kinect pose estimation in the context of coaching of elderly population. In: 2012 Annual Intl. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 1188–1193Google Scholar
  139. Olivieri DN, Gmez Conde I, Vila Sobrino XA (2012) Eigenspace-based fall detection and activity recognition from motion templates and machine learning. Expert Syst Appl 39(5):5935–5945CrossRefGoogle Scholar
  140. Pantelopoulos A, Bourbakis NG (2010) A survey on wearable sensor-based systems for health monitoring and prognosis. IEEE Trans Syst Man Cybern Part C-Appl Rev 40(1):1–12CrossRefGoogle Scholar
  141. Paradiso R (2003) Wearable health care system for vital signs monitoring. In: 4th Intl. IEEE EMBS Special Topic Conf. on Information Technology Applications in Biomedicine, 2003, pp 283–286Google Scholar
  142. Pediaditis M, Tsiknakis M, Vorgia P, Kafetzopoulos D, Danilatou V, Fotiadis D (2010) Vision-based human motion analysis in epilepsy—Methods and challenges. In: 2010 10th IEEE Intl. Conf. on Information Technology and Applications in Biomedicine (ITAB), pp 1–5Google Scholar
  143. Pediaditis M, Tsiknakis M, Koumakis L, Karachaliou M, Voutoufianakis S, Vorgia P (2012) Vision-based absence seizure detection. Conf Proc IEEE Eng Med Biol Soc 2012:65–68Google Scholar
  144. Peng YT, Lin CY, Sun MT, Feng MW (2006) Sleep condition inferencing using simple multimodality sensors. In: Circuits and Systems, 2006. ISCAS 2006. Proceedings. 2006 IEEE International Symposium on, IEEE, pp 4–ppGoogle Scholar
  145. Planinc R, Kampel M (2013) Introducing the use of depth data for fall detection. Pers Ubiquit Comput 17(6):1063–1072CrossRefGoogle Scholar
  146. Poh MZ, McDuff DJ, Picard RW (2010) Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt Express 18(10):10762–10774CrossRefGoogle Scholar
  147. Prkachin KM (1992) The consistency of facial expressions of pain: a comparison across modalities. Pain 51(3):297–306CrossRefGoogle Scholar
  148. Prkachin KM (2009) Assessing pain by facial expression: facial expression as nexus. Pain Res Manag 14(1):53–58CrossRefGoogle Scholar
  149. Rashidi P, Mihailidis A (2013) A survey on ambient-assisted living tools for older adults. IEEE J Biomed Health Inf 17(3):579–590CrossRefGoogle Scholar
  150. Rodrigues J, Estevao M, Malaquias J, Santos P, Gouveia G, Simoes J (2007) SleepAtHome—Portable Home Based System for Pediatric Sleep Apnoea Diagnosis. In: IEEE Intl. Conf. on Portable Information Devices, 2007. PORTABLE 07, pp 1–4Google Scholar
  151. Rogers CR, Schmidt KL, VanSwearingen JM, Cohn JF, Wachtman GS, Manders EK, Deleyiannis FWB (2007) Automated facial image analysis: detecting improvement in abnormal facial movement after treatment with botulinum toxin A. Ann Plast Surg 58(1):39–47CrossRefGoogle Scholar
  152. Rougier C, Auvinet E, Rousseau J, Mignotte M, Meunier J (2011a) Fall Detection from Depth Map Video Sequences. In: Abdulrazak B, Giroux S, Bouchard B, Pigot H, Mokhtari M (eds) Toward Useful Services for Elderly and People with Disabilities, no. 6719 in Lecture Notes in Computer Science, Springer Berlin Heidelberg, pp 121–128Google Scholar
  153. Rougier C, Meunier J, St-Arnaud A, Rousseau J (2011b) Robust video surveillance for fall detection based on human shape deformation. IEEE Trans Circuits Syst Video Technol 21(5):611–622CrossRefGoogle Scholar
  154. Rougier C, Meunier J, St-Arnaud A, Rousseau J (2013) 3d head tracking for fall detection using a single calibrated camera. Image Vis Comput 31(3):246–254CrossRefGoogle Scholar
  155. Sathyanarayana S, Satzoda RK, Sathyanarayana S, Thambipillai S (2014) Block-Based Search Space Reduction Technique for Face Detection Using Shoulder and Head Curves. In: Image and Video Technology, no. 8333 in Lecture Notes in Computer Science, Springer Berlin Heidelberg, pp 385–396Google Scholar
  156. Sathyanarayana S, Satzoda R, Sathyanarayana S, Thambipillai S (2015a) Identifying epileptic seizures based on a template-based eyeball detection technique. In: 2015 IEEE Intl Conf on Image Processing (ICIP), p [in Press]Google Scholar
  157. Sathyanarayana S, Satzoda R, Sathyanarayana S, Thambipillai S (2015b) WellCam: Dataset for Vision-based Patient Wellness Monitoring. CVPR Workshop on The Future of Datasets in VisionGoogle Scholar
  158. Sato I, Nakajima M (2005) Non-contact Breath Motion Monitor ing System in Full Automation. In: Engineering in Medicine and Biology Society, 2005. IEEE-EMBS 2005. 27th Annual Intl. Conf. of the, pp 3448–3451Google Scholar
  159. Seki H (2009) Fuzzy inference based non-daily behavior pattern detection for elderly people monitoring system. Conf Proc IEEE Eng Med Biol Soc 2009:6187–6192Google Scholar
  160. Sikka K, Dhall A, Bartlett MS (2014) Classification and weakly supervised pain localization using multiple segment representation. Image Vis Comput 32(10):659–670CrossRefGoogle Scholar
  161. Somers VK, White DP, Amin R, Abraham WT, Costa F, Culebras A, Daniels S, Floras JS, Hunt CE, Olson LJ, Pickering TG, Russell R, Woo M, Young T (2008) Sleep apnea and cardiovascular disease. J Am Coll Cardiol 52(8):686–717CrossRefGoogle Scholar
  162. Stevens S, Bharucha A (2003) Caremedia: Automated video and sensor analysis for geriatric care. Carnegie Mellon UniversityGoogle Scholar
  163. Sun N, Garbey M, Merla A, Pavlidis I (2005) Imaging the cardiovascular pulse. In: IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005. CVPR 2005, vol 2, pp 416–421 vol. 2Google Scholar
  164. Tabar AM, Keshavarz A, Aghajan H (2006) Smart Home Care Network Using Sensor Fusion and Distributed Vision-based Reasoning. In: Proceedings of the 4th ACM Intl Workshop on Video Surveillance and Sensor Networks, ACM, New York, NY, USA, VSSN ’06, pp 145–154Google Scholar
  165. Takemura Y, Jy Sato, Nakajima M (2005) A respiratory movement monitoring system using fiber-grating vision sensor for diagnosing sleep apnea syndrome. OPT REV 12(1):46–53CrossRefGoogle Scholar
  166. Toreyin B, Dedeoglu Y, Cetin A (2006) HMM Based Falling Person Detection Using Both Audio and Video. In: Signal Processing and Communications Applications, 2006 IEEE 14th, pp 1–4Google Scholar
  167. UN (2012) World population ageing 1950–2050. Department of Economic and Social Affairs, Population Division, United Nations, http://www.un.org/esa/population/publications/worldageing19502050/
  168. Van Kasteren TLM (2011) Activity Recognition for Health Monitoring Elderly Using Temporal Probabilistic Models. ASCIGoogle Scholar
  169. Viola P, Jones M (2001) Robust real-time object detection. Int J Comput Vis 4:51–52Google Scholar
  170. Vishwakarma S, Agrawal A (2012) A survey on activity recognition and behavior understanding in video surveillance. Vis Comput 29(10):983–1009CrossRefGoogle Scholar
  171. Wang CW, Ahmed A, Hunter A (2006) Vision analysis in detecting abnormal breathing activity in application to diagnosis of obstructive sleep apnoea. Conf Proc IEEE Eng Med Biol Soc 1:4469–4473Google Scholar
  172. Wang CW, Ahmed A, Hunter A (2007) Locating the upper body of covered humans in application to diagnosis of obstructive sleep apnea. In: World Congress on Engineering, pp 662–667Google Scholar
  173. Wang CW, Hunter A, Gravill N, Matusiewicz S (2010) Real time pose recognition of covered human for diagnosis of sleep apnoea. Comput Med Imaging Graph 34(6):523–533CrossRefGoogle Scholar
  174. Wang CW, Hunter A, Gravill N, Matusiewicz S (2014) Unconstrained video monitoring of breathing behavior and application to diagnosis of sleep apnea. IEEE Trans Biomed Eng 61(2):396–404CrossRefGoogle Scholar
  175. Wang F, Stone E, Dai W, Banerjee T, Giger J, Krampe J, Rantz M, Skubic M (2009) Testing an in-home gait assessment tool for older adults. Conf Proc IEEE Eng Med Biol Soc 2009:6147–6150Google Scholar
  176. Wang P, Barrett F, Martin E, Milanova M, Gur RE, Gur RC, Kohler C, Verma R (2008) Automated video based facial expression analysis of neuropsychiatric disorders. J Neurosci Methods 168(1):224–238CrossRefGoogle Scholar
  177. Wang S, Xu Z, Yang Y, Li X, Pang C, Haumptmann AG (2013) Fall Detection in Multi-camera Surveillance Videos: Experimentations and Observations. In: Proceedings of the 1st ACM Intl Workshop on Multimedia Indexing and Information Retrieval for Healthcare, ACM, New York, NY, USA, MIIRH ’13, pp 33–38Google Scholar
  178. Werner P, Al-Hamadi A, Niese R (2012) Pain recognition and intensity rating based on Comparative Learning. In: 2012 19th IEEE Intl. Conf. on Image Processing (ICIP), pp 2313–2316Google Scholar
  179. Werner P, Al-Hamadi A, Niese R, Walter S, Gruss S, Traue HC (2013) Towards Pain Monitoring: Facial Expression, Head Pose, a new Database, an Automatic System and Remaining Challenges. In: Proceedings of the British Machine Vision ConferenceGoogle Scholar
  180. Wiesner S, Yaniv Z (2007) Monitoring patient respiration using a single optical camera. Conf Proc IEEE Eng Med Biol Soc 2007:2740–2743Google Scholar
  181. Yang FC, Kuo CH, Tsai MY, Huang SC (2003) Image-based sleep motion recognition using artificial neural networks. In: 2003 Intl. Conf on Machine Learning and Cybernetics, vol 5, pp 2775–2780 Vol.5Google Scholar
  182. Yu M, Rhuma A, Naqvi S, Wang L, Chambers J (2012) A posture recognition-based fall detection system for monitoring an elderly person in a smart home environment. IEEE Trans Inf Technol Biomed 16(6):1274–1286CrossRefGoogle Scholar
  183. Yu MC, Wu H, Liou JL, Lee MS, Hung YP (2013) Multiparameter Sleep Monitoring Using a Depth Camera. In: Gabriel J, Schier J, Huffel SV, Conchon E, Correia C, Fred A, Gamboa H (eds) Biomedical Engineering Systems and Technologies, no. 357 in Communications in Computer and Information Science, Springer Berlin Heidelberg, pp 311–325Google Scholar
  184. Yu X (2008) Approaches and principles of fall detection for elderly and patient. In: 10th Intl Conf on e-health Networking, Applications and Services, 2008. HealthCom 2008, pp 42–47Google Scholar
  185. Zambanini S, Machajdik J, Kampel M (2010) Detecting falls at homes using a network of low-resolution cameras. In: 2010 10th IEEE Intl Conf on Information Technology and Applications in Biomedicine (ITAB), pp 1–4Google Scholar
  186. Zhan K, Ramos F, Faux S (2012) Activity recognition from a wearable camera. In: 2012 12th Intl. Conf. on Control Automation Robotics Vision (ICARCV), pp 365–370Google Scholar
  187. Zhang C, Tian Y (2012) Rgb-d camera-based daily living activity recognition. J Comput Vis Image Process 2(4):12Google Scholar
  188. Zhang Z, Kapoor U, Narayanan M, Lovell N, Redmond S (2011) Design of an unobtrusive wireless sensor network for nighttime falls detection. In: 2011 Annual Intl Conf of the IEEE Engineering in Medicine and Biology Society,EMBC, pp 5275–5278Google Scholar
  189. Zhong H, Shi J, Visontai M (2004) Detecting unusual activity in video. In: Proc. of the 2004 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2004. CVPR 2004, vol 2, pp II–819–II–826 Vol.2Google Scholar
  190. Zhou Z, Dai W, Eggert J, Giger J, Keller J, Rantz M, He Z (2009) A real-time system for in-home activity monitoring of elders. In: Annual Intl. Conf. of the IEEE Engineering in Medicine and Biology Society, 2009. EMBC 2009, pp 6115–6118Google Scholar
  191. Zhu Z, Fei J, Pavlidis I (2005) Tracking human breath in infrared imaging. In: Fifth IEEE Symposium on Bioinformatics and Bioengineering, 2005. BIBE 2005, pp 227–231Google Scholar
  192. Zouba N, Bremond F, Thonnat M, Vu VT (2007) Multi-sensors Analysis for Everyday Activity Monitoring. Proc of SETIT pp 25–29Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.Nanyang Technological UniversitySingaporeSingapore
  2. 2.University of CaliforniaSan DiegoUSA

Personalised recommendations