Pattern Analysis and Applications

, Volume 7, Issue 4, pp 386–401 | Cite as

Summarising contextual activity and detecting unusual inactivity in a supportive home environment

  • Stephen J. McKenna
  • Hammadi Nait Charif
Theoretical Advances


Interpretation of human activity and the detection of associated events are eased if appropriate models of context are available. A method is presented for automatically learning a context-specific spatial model in terms of semantic regions, specifically inactivity zones and entry zones. Maximium a posteriori estimation of Gaussian mixtures is used in conjunction with minumum description length for selection of the number of mixture components. Learning is performed using expectation-maximisation algorithms to maximise penalised likelihood functions that incorporate prior knowledge of the size and shape of the semantic regions. This encourages a one-to-one correspondence between the Gaussian mixture components and the regions. The resulting contextual model enables human-readable summaries of activity to be produced and unusual inactivity to be detected. Results are presented using overhead camera sequences tracked using a particle filter. The method is developed and described within the context of supportive home environments which have as their aim the extension of independent, quality living for older people.


Learning spatial context Inactivity detection Fall detection Supportive home environments Gaussian mixture models Expectation-maximisation 



Dr. Nait Charif was supported by UK EPSRC EQUAL grant GR/R27419/01. The authors are grateful to the reviewers for helpful comments on an earlier version of this paper.


  1. 1.
    Barnes NM, Edwards NH, Rose DAD, Garner P (1998) Lifestyle monitoring: technology for supported independence. IEE Comput Control Eng J 9:169-174Google Scholar
  2. 2.
    Bonner S (1998) Assisted interactive dwelling house: Edinvar housing association smart technology demonstrator and evaluation site. In: Porrero IP, Ballabio E (eds) Improving the quality of life for the European citizen (TIDE: technology for inclusive design and equality). IOS Press, Amsterdam, pp 396–400Google Scholar
  3. 3.
    Marquis-Faulkes F, McKenna SJ, Gregor P, Newell AF (2003) Gathering the requirements for a fall monitor using drama and video with older people. Technol Disabil (in press)Google Scholar
  4. 4.
    Porteus J, Brownsell S (2000) Using telecare: exploring technologies for independent living for older people. Report on the Anchor Trust/BT telecare research project, Anchor TrustGoogle Scholar
  5. 5.
    Kidd CD, Orr R, Abowd GD, Atkeson CG, Essa IA, MacIntyre B, Mynatt ED, Starner T, Newstetter W (1999) The aware home: a living laboratory for ubiquitous computing research. In: Proceedings of the 2nd international workshop on cooperative buildings, integrating information, organization and architecture,Pittsburgh, pp 191–198Google Scholar
  6. 6.
    Scuffham P, Chaplin S, Legood R (2003) Incidence and costs of unintentional falls in older people in the United Kingdom. J Epidemiol Community Health 57:740–744PubMedGoogle Scholar
  7. 7.
    Doughty K (2000) Fall prevention and management strategies based on intelligent detection monitoring and assessment. In: New technologies in medicine for the elderly, Charing Cross hospital, LondonGoogle Scholar
  8. 8.
    Chan M, Bocquet H, Campo E, Pous J (1998) Remote monitoring system to measure indoors mobility and transfer of the elderly. In: Porrero IP, Ballabio E (eds) Improving the quality of life for the European citizen (TIDE: Technology for Inclusive Design and Equality), pp 379–383Google Scholar
  9. 9.
    SeniorWatch (2001) Fall detector. A case study of the European IST Seniorwatch project IST-1999-29086
  10. 10.
    McKenna SJ, Marquis-Faulkes F, Newell AF, Gregor P (2004) Using drama for requirements gathering: a case study on advanced sensors in supportive environments for the elderly. Int J Hum Comput Stud (Submitted)Google Scholar
  11. 11.
    Bromiley PA, Courtney P, Thacker NA (2002) Design of a visual system for detecting natural events by the use of an independent visual estimate: a human fall detector. In: Christensen HI, Philips PJ (eds) Empirical evaluation methods in computer vision. World Scientific Publishing , 50 Machine Perception and Artificial IntelligenceGoogle Scholar
  12. 12.
    Sixsmith A, Johnson N (2002) SIMBAD: smart inactivity monitor using array-based detector. Gerontechnology 2(1):110Google Scholar
  13. 13.
    Buxton H, Gong S (1995) Visual surveillance in a dynamic and uncertain world. Artif Intell 78(1–2):431–459Google Scholar
  14. 14.
    Howarth RJ, Buxton H (1992) An analogical representation of space and time. Image Vis Comput 10(7):467–478Google Scholar
  15. 15.
    Ayers D, Shah M (2001) Monitoring human behavior from video taken in an office environment. Image Vis Comput 19(12):833–846Google Scholar
  16. 16.
    Nguyen N, Bui H, Venkatesh S, West G (2003) Recognising and monitoring high-level behaviours in complex spatial environments. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Madison, pp 620–625Google Scholar
  17. 17.
    Fernyhough JH, Cohn AG, Hogg D (1996) Generation of semantic regions from image sequences. In: Proceedings of the European conference on computer vision, Cambridge, vol 2, pp 475–484Google Scholar
  18. 18.
    Johnson N, Hogg D (1996) Learning the distribution of object trajectories for event recognition. Image Vis Comput 14(8):609–615Google Scholar
  19. 19.
    Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Machine Intell 22(8):747–757Google Scholar
  20. 20.
    Makris D, Ellis T (2002) Path detection in video surveillance. Image Vis Comput 20(12):895–903Google Scholar
  21. 21.
    Makris D, Ellis T (2003) Automatic learning of an activity-based semantic scene model. In: Proceedings of the IEEE conference on advanced video and signal based surveillance, MiamiGoogle Scholar
  22. 22.
    Makris D, Ellis T (2002) Spatial and probabilistic modelling of pedestrian behaviour. In: British machine vision conference, Cardiff, vol 2, pp 557-566Google Scholar
  23. 23.
    Brand M, Kettnaker V (2000) Discovery and segmentation of activities in video. IEEE Trans Pattern Anal Machine Intell 22(8):844–851Google Scholar
  24. 24.
    Ferryman J (Ed.) (2002) Proceedings of the 3rd IEEE international workshop on performance evaluation of tracking and surveillance, CopenhagenGoogle Scholar
  25. 25.
    Comaniciu D, Ramesh V, Meer P (2003) Kernel-based object tracking. IEEE Trans Pattern Anal Machine Intell 25(5):564–575Google Scholar
  26. 26.
    Fuentes LM, Velastin SA (2001) People tracking in indoor surveillance applications. In: Proceedings of the 2nd IEEE workshop on performance evaluation of tracking and surveillance, HawaiiGoogle Scholar
  27. 27.
    Needham CJ, Boyle RD (2001) Tracking multiple sports players through occlusion congestion and scale. In: Proceedings of the British machine vision conference, Manchester, vol 1, pp 93–102Google Scholar
  28. 28.
    Bobick A, Intille S, Davis J, Baird F, Pinhanez C, Campbell L, Ivanov Y, Schutte A, Wilson A (1999) The KidsRoom: a perceptually-based interactive and immersive story environment. PRESENCE: Teleop Virtual Environ 8(4):367–391Google Scholar
  29. 29.
    Yoda I, Hosotani D, Sakaue K (2004) Ubiquitous stereo vision for controlling safety on platforms in railroad stations. In: Proceedings of the 6th Asian conference on computer vision, Jeju, vol 2, pp 770-776Google Scholar
  30. 30.
    Kim J-W, Choi K-S, Choi B-D, Ko S-J (2003) Real-time system for counting the number of passing people using a single camera. Pattern Recognition Symposium (DAGM), Magdeburg, Lecture notes in computer science, vol 2781, pp 466–473Google Scholar
  31. 31.
    Chen X, Yang J (2002) Towards monitoring human activities using an omnidirectional camera. In: Proceedings of the international conference on multimodal interfaces, Pittsburgh, pp 423–428Google Scholar
  32. 32.
    Krumm J, Harris S, Meyers B, Brummitt B, Hale M, Shafer S (2000) Multi-camera multi-person tracking for EasyLiving. In: Proceedings of the 3rd IEEE workshop on visual surveillance, Dublin, pp 3–10Google Scholar
  33. 33.
    Birchfield S (1998) Elliptical head tracking using intensity gradients and color histograms. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Santa BarbaraGoogle Scholar
  34. 34.
    Isard M, Blake A (1998) ICondensation: unifying low-level and high-level tracking in a stochastic framework. In: Proceedings of the European conference on computer vision, vol 1, pp 893–908Google Scholar
  35. 35.
    Liu F, Lin X, Li SZ, Shi Y (2003) Multi-modal face tracking using Bayesian network. In: Proceedings of the IEEE international workshop on analysis and modeling of faces and gestures, NiceGoogle Scholar
  36. 36.
    Nait-Charif H, McKenna SJ (2003) Head tracking and action recognition in a smart meeting room. In: Proceedings of the 4th IEEE international workshop on performance evaluation of tracking and surveillance (PETS-ICVS), GrazGoogle Scholar
  37. 37.
    Wren CR, Azarbayejani A, Darrell T, Pentland A (1997) Pfinder: real-time tracking of the human body. IEEE Trans Pattern Anal Machine Intell 19(7):780–785Google Scholar
  38. 38.
    Blake A, Isard M (2000) Active contours. Springer, Berlin Heidelberg New YorkGoogle Scholar
  39. 39.
    McKenna SJ, Jabri S, Duric Z, Rosenfeld A, Wechsler H (2000) Tracking groups of people. Comput Vis Image Understanding 80(1):42–56Google Scholar
  40. 40.
    Isard M, Blake A (1996) Contour tracking by stochastic propagation of conditional density. In: Proceedings of the European conference on computer vision, vol 1, pp 343–356Google Scholar
  41. 41.
    Nait-Charif H, McKenna SJ (2004) Tracking poorly modelled motion using particle filters with iterated likelihood weighting. In: Proceedings of the Asian conference on computer vision (ACCV), Jeju Island, pp 156–161Google Scholar
  42. 42.
    Pitt M, Shephard N (1999) Filtering via simulation: auxiliary particle filters. J Amer Statist Assoc 94(446):590–599MathSciNetGoogle Scholar
  43. 43.
    Roberts SJ (1997) Parametric and non-parametric unsupervised cluster analysis. Pattern Recognit 30(2):261–272Google Scholar
  44. 44.
    Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39:1–38Google Scholar
  45. 45.
    Roberts S, Husmeier D, Rezek I, Penny W (1998) Bayesian approaches to Gaussian mixture modelling. IEEE Trans Pattern Anal Machine Intell 20(11):1133–1142Google Scholar
  46. 46.
    Rissanen J (1978) Modelling by shortest data description. Automatica 14:465–471Google Scholar
  47. 47.
    Sawhney HS, Ayer S (1996) Compact representations of videos through dominant and multiple motion estimation. IEEE Trans Pattern Anal Machine Intell 18(8):814–830Google Scholar
  48. 48.
    Walter M, Psarrou A, Gong S (2001) Data driven gesture model acquisition using minimum description length. In: Proceedings of the British machine vision conference, ManchesterGoogle Scholar
  49. 49.
    Carson C, Belongie S, Greenspan H, Malik J (2002) Blobworld: color- and texture-based image segmentation using EM and its application to image querying and classification. IEEE Trans Pattern Anal Machine Intell 24(8):1026–1038Google Scholar
  50. 50.
    Greenspan H, Goldberger J, Mayer A (2002) A probabilistic framework for spatio-temporal video representation and indexing. In: Proceedings of the European conference on computer vision, CopenhagenGoogle Scholar
  51. 51.
    Hansen MH, Yu B (2001) Model selection and the principle of minimum description length. J Amer Statist Assoc 96(454):746–774MathSciNetGoogle Scholar
  52. 52.
    Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464Google Scholar
  53. 53.
    Gauvain J-L, Lee C-H (1994) Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains. IEEE Trans Speech Audio Process 2(2):291–298Google Scholar
  54. 54.
    Ormoneit D, Tresp V (1998) Averaging maximum penalized likelihood and Bayesian estimation for improving Gaussian mixture probability density estimates. IEEE Trans Neural Netw 9(4):639–650Google Scholar
  55. 55.
    Cheeseman P, Stutz J (1996) Bayesian classification (AutoClass): theory and results. In: Fayyad UM, Piatetsky-Shapiro G, Smyth P, Uthurusamy R (eds) Advances in knowledge discovery and data mining. AAAI Press, Menlo Park, pp 153–180Google Scholar
  56. 56.
    Geiger D, Heckerman D (2002) Parameter priors for directed acyclic graphical models and the characterization of several probability distributions. Ann Stat 30:1412-1440Google Scholar
  57. 57.
    Wallace CS, Dowe DL (2000) MML clustering of multi-state, Poisson, von Mises circular and Gaussian distributions. Stat Comput 10(1):73–83Google Scholar
  58. 58.
    Figueiredo MAT, Leitao JMN, Jain AK (1999) On fitting mixture models. In: Hancock E, Pellilo M (eds) Energy minimization methods in computer vision and pattern recognition. Springer, Berlin Heidelberg New York, pp 54–69Google Scholar
  59. 59.
    Hue C, Le Cadre J-P, Pérez P (2002) Sequential Monte Carlo methods for multiple target tracking and data fusion. IEEE Trans Signal Process 50(2):309–325Google Scholar
  60. 60.
    Isard M, MacCormick J (2001) BraMBLe: a Bayesian multiple-blob tracker. In: Proceedings of the 8th IEEE international conference on computer vision, Vancouver, vol 2, pp 34–41Google Scholar
  61. 61.
    Khan Z, Balch T, Dellaert F (2004) An MCMC-based particle filter for tracking multiple interacting targets. In: Proceedings of the European conference on computer vision, Prague, vol 4, pp 279–290Google Scholar
  62. 62.
    MacCormick J, Blake A (1999) A probabilistic exclusion principle for tracking multiple objects. In: Proceedings of the IEEE international conference on computer vision, Kerkyra, pp 572–587Google Scholar

Copyright information

© Springer-Verlag London Limited 2005

Authors and Affiliations

  1. 1.Division of Applied ComputingUniversity of DundeeScotland

Personalised recommendations