Skip to main content
SpringerLink
Log in

An approach to provide dynamic, illustrative, video-based guidance within a goal-driven smart home

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The global population is aging in a never-before seen way, introducing an increasing ageing-related cognitive ailments, such as dementia. This aging is coupled with a reduction in the global support ratio, reducing the availability of formal and informal support and therefore capacity to care for those suffering these aging related ailments. Assistive Smart Homes (SH) are a promising form of technology enabling assistance with activities of daily living, providing support of suffers of cognitive ailments and increasing their independence and quality of life. Traditional SH systems have deficiencies that have been partially addressed by through goal-driven SH systems. Goal-driven SHs incorporate flexible activity models, goals, which partially address some of these issues. Goals may be combined to provide assistance with dynamic and variable activities. This paradigm-shift, however, introduces the need to provide dynamic assistance within such SHs. This study presents a novel approach to achieve this through video based content analysis and a mechanism to facilitate matching analysed videos to dynamic activities/goals. The mechanism behind this approach is detailed and followed by the presentation of an evaluation where showing promising results were shown.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Acampora G et al (2013) A survey on ambient intelligence in health care. In: Proceedings of the IEEE. Institute of Electrical and Electronics Engineers, vol 101(12), pp 2470–2494. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=3890262&tool=pmcentrez&rendertype=abstract

  • Ballan L et al (2010) Event detection and recognition for semantic annotation of video. Multimed Tools Appl 51(1):279–302. doi:10.1007/s11042-010-0643-7. Accessed 2 June 2014

  • Chan M et al (2008) A review of smart homes—present state and future challenges. Comput Methods Programs Biomed 91(1):55–81. http://www.ncbi.nlm.nih.gov/pubmed/18367286. Accessed 20 July 2012

  • Chatterjee M, Leuski A (2015) A novel statistical approach for image and video retrieval and its adaption for active learning. Proceedings of the 23rd ACM international conference on multimedia. ACM, Brisbane, pp 935–938

    Chapter  Google Scholar 

  • Chen L, Hoey J et al (2012) Sensor-based activity recognition. IEEE Trans Syst Man Cybern Part C (Appl Rev) 1–19. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6208895

  • Chen W, Ananthakrishnan S (2013) ASR error detection in a conversational spoken language translation system. In: 2013 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 7418–7422. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6639104. Accessed 28 May 2014

  • Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24(6):961–974. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5710936

  • Choe TE et al (2013) Semantic video-to-video search using sub-graph grouping and matching. Proc IEEE Int Conf Comput Vis 1:787–794

    Google Scholar 

  • Cook DJ, Das SK (2007) How smart are our environments? An updated look at the state of the art. Pervasive Mobile Comput 3(2):53–73. http://linkinghub.elsevier.com/retrieve/pii/S1574119206000642. Accessed 5 October 2012

  • De Luca, d’Alessandro E, Bonacci S, Giraldi G (2011) Aging populations: the health and quality of life of the elderly. La Clin Terapeut 162(1):e13

  • European Commission (2016) The ambient assisted living (AAL) joint programme. http://ec.europa.eu/information_society/activities/einclusion/docs/ageing/aal_overview.pdf

  • Filippova K, Hall K (2011) Improved video categorization from text metadata and user comments. In: SIGIR’11 proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval, pp 835–842. http://dl.acm.org/citation.cfm?id=2010028. Accessed 10 June 2014

  • Gaüzère B et al (2015) Semantic web technologies for object tracking and video analytics. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), vol 9475, pp 574–585. https://www.scopus.com/inward/record.uri?eid=2-s2.0-84952801857&partnerID=40&md5=36ddc1d67dedd953b3f41a0abf4fcf1c

  • Google (2016) Google speech API. http://www.google.com/speech-api/v1/recognize

  • Greco L et al (2016) Abnormal event recognition: a hybrid approach using semantic web. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 58–65

  • Hentschel C, Blümel I, Sack H (2013) Automatic annotation of scientific video material based on visual concept detection. In: Proceedings of the 13th international conference on knowledge management and knowledge technologies, i-Know’13, pp 1–8. http://dl.acm.org/citation.cfm?doid=2494188.2494213

  • Hwang A, Hoey J (2012) Smart home, the next generation: closing the gap between users and technology. In: AAAI fall symposium on gerontechnology, Arlington, pp 14–21. http://www.aaai.org/ocs/index.php/FSS/FSS12/paper/viewPDFInterstitial/5549/5784. Accessed 6 March 2014

  • Jones S, Shao L (2013) Content-based retrieval of human actions from realistic video databases. Inf Sci 236:56–65. doi:10.1016/j.ins.2013.02.018

  • Lawton M, Brody E (1988) Instrumental activities of daily living scale (IADL). https://www.abramsoncenter.org/PRI/documents/IADL.pdf. Accessed 5 March 2014

  • Lowthian JA et al (2011) The challenges of population ageing: accelerating demand for emergency ambulance services by older patients, 1995–2015. Med J Austr 194(11):574–578. http://www.ncbi.nlm.nih.gov/pubmed/21644869

  • Maratea A, Petrosino A, Manzo M (2013) Generation of description metadata for video files. In: Proceedings of the 14th international conference on computer systems and technologies—CompSysTech’13, vol 767, pp 262–269. http://www.scopus.com/inward/record.url?eid=2-s2.0-84889596966&partnerID=tZOtx3y1

  • Matejka J, Grossman T, Fitzmaurice G (2014) Video lens: rapid playback and exploration of large video collections and associated metadata. In: Proceedings of the 27th annual ACM symposium on user interface software and technology, UIST’14, ACM, New York, pp 541–550. doi:10.1145/2642918.2647366

  • Mazloom M et al (2013) Querying for video events by semantic signatures from few examples. In: Proceedings of the 21st ACM international conference on multimedia, pp 609–612

  • McCloskey S, Davalos P (2012) Activity detection in the wild using video metadata. In: Pattern recognition (ICPR), pp 3140–3143. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6460830. Accessed 2 June 2014

  • McGinnis S, Moore J (2006) The impact of the aging population on the health workforce in the United States: summary of key findings. In: Cahiers de sociologie et de démographie. http://cat.inist.fr/?aModele=afficheN&cpsidt=17965740. Accessed 20 May 2013

  • Mehla R, Aggarwal R (2014) Automatic speech recognition: a survey. Int J Adv Res Comput Sci Electron Eng 3(1):45–53. http://www.ijarcsee.org/index.php/IJARCSEE/article/view/440. Accessed 28 May 2014

  • Metze F et al (2013) Beyond audio and video retrieval: topic-oriented multimedia summarization. Int J Multimed Inf Retriev 2(2):131–144. doi:10.1007/s13735-012-0028-y. Accessed 13 November 2013

  • Mihailidis A et al (2008) The COACH prompting system to assist older adults with dementia through handwashing: an efficacy study. BMC Geriatr 8:28. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=2588599&tool=pmcentrez&rendertype=abstract. Accessed 6 November 2012

  • NLTK Project (2016) Natural language toolkit. http://www.nltk.org/

  • O’Neill SA et al (2010) Video reminders as cognitive prosthetics for people with dementia. Ageing Int 36(2):267–282. doi:10.1007/s12126-010-9089-5. Accessed 6 March 2014

  • Panchal P, Merchant S, Patel N (2012) Scene detection and retrieval of video using motion vector and occurrence rate of shot boundaries. In: 2012 Nirma University international conference on engineering (NUiCONE), pp 1–6

  • Papadopoulos DP et al (2013) Automatic summarization and annotation of videos with lack of metadata information. Expert Syst Appl 40(14):5765–5778. http://linkinghub.elsevier.com/retrieve/pii/S0957417413001322

  • Patel BV, Meshram BB (2012) Content based video retrieval systems. Int J UbiComp 3(2):13–30

    Article  Google Scholar 

  • Perea-Ortega JM et al (2013) Semantic tagging of video ASR transcripts using the web as a source of knowledge. Comput Stand Interfaces 35(5):519–528. http://linkinghub.elsevier.com/retrieve/pii/S0920548912000888. Accessed 2 June 2014

  • Princeton University (2010) About WordNet. WordNet, Princeton University. http://wordnet.princeton.edu

  • Rafferty J et al (2014a) Automatic summarization of activities depicted in instructional videos by use of speech analysis. In: Pecchia L et al (eds) Ambient assisted living and daily activities. Lecture notes in computer science. Springer, New York, pp 123–130. doi:10.1007/978-3-319-13105-4_20

  • Rafferty J et al (2014b) NFC based provisioning of instructional videos to assist with instrumental activities of daily living. In: 2014 36th annual international conference of the IEEE engineering in medicine and biology society, EMBC 2014, pp 4131–4134

  • Rafferty J, Chen L et al (2015a) Goal lifecycles and ontological models for intention based assistive living within smart environments. Comput Syst Sci Eng 30(1):7–18

    Google Scholar 

  • Rafferty J, Nugent C et al (2015b) A mechanism for nominating video clips to provide assistance for instrumental activities of daily living

  • Rafferty J, Nugent C et al (2015c) Automatic metadata generation through analysis of narration within instructional videos. J Med Syst 39(9):1–7. doi:10.1007/s10916-015-0295-2

  • SIL (2016) American English homophones. http://www-01.sil.org/linguistics/wordlists/english/

  • Shabani AH, Zelek JS, Clausi DA (2013) Multiple scale-specific representations for improved human action recognition. Pattern Recognit Lett 34(15):1771–1779. doi:10.1016/j.patrec.2012.12.013

  • United Nations (2010) World population ageing 2009 (population studies series), Pap/Cdr edn

  • United Nations (2014) Concise report on the world population situation in 2014

  • van de Kaa DJ; Population Reference Bureau Inc., W.D.C. (1987) Europe’s second demographic transition. Distributed by ERIC Clearinghouse, Washington, D.C.

  • Veltkamp R, Burkhardt H, Kriegel H-P (2013) State-of-the-art in content-based image and video retrieval. Springer, New York

  • Yang H, Meinel C (2014) Content based lecture video retrieval using speech and video text information. IEEE Trans Learn Technol 7(2):142–154

    Article  Google Scholar 

Download references

Acknowledgement

Funding was provided by Department of Education and Learning, Northern Ireland.

Author information

Authors and Affiliations

  1. School of Computing and Mathematics, Ulster University, Jordanstown, Northern Ireland, UK

    Joseph Rafferty, Chris Nugent & Jun Liu

  2. School Computer Science and Informatics, De Montfort University, Leicester, England, UK

    Liming Chen

Authors
  1. Joseph Rafferty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chris Nugent
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Joseph Rafferty.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rafferty, J., Nugent, C., Liu, J. et al. An approach to provide dynamic, illustrative, video-based guidance within a goal-driven smart home. J Ambient Intell Human Comput 11, 3045–3056 (2020). https://doi.org/10.1007/s12652-016-0421-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-016-0421-0

Keywords

Search

Navigation