Multimedia Tools and Applications

, Volume 73, Issue 2, pp 1029–1051 | Cite as

An experimental evaluation of ontology-based user profiles

  • Frank Hopfgartner
  • Joemon M. Jose


In recent years, a number of research works have been carried out to improve the information retrieval process by exploiting external knowledge, e.g. by employing ontologies. Even though ontologies seem to be a promising technique to improve the retrieval process, hardly any study has been performed to evaluate the use of ontologies over a longer time period to model user interests. In this work we introduce an ontology based video recommender system that exploits implicit relevance feedback to capture users’ evolving information needs. The system exploits a generic ontology to organise users’ interests. We evaluate the recommendations by performing a user-centred multiple time-series study where participants were asked to include the system into their daily news gathering routine. The results of this study suggest that the system can be successfully employed to improve personal information seeking tasks in news domain.


Video retrieval Multiple time series study Personalisation 


  1. 1.
    Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans on Knowl and Data Eng 17(6):734–749CrossRefGoogle Scholar
  2. 2.
    Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives ZG (2007) DBpedia: a nucleus for a web of open data. In: Proc. 6th int. semantic web conf. Springer, Berlin/Heidelberg, pp 722–735Google Scholar
  3. 3.
    Belkin NJ (2008) Some(what) grand challenges for information retrieval. SIGIR Forum 42(1):47–54CrossRefGoogle Scholar
  4. 4.
    Bharat K, Kamba T, Albers M (1998) Personalized, interactive news on the web. Mult Syst 6(5):349–358CrossRefGoogle Scholar
  5. 5.
    Borlund P (2003) The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Inf Res 8(3)
  6. 6.
    Campbell DT, Stanley JC (1963) Experimental and quasi-experimental design for research, 1st edn. Wadsworth Publishing, MontereyGoogle Scholar
  7. 7.
    Chen L, Sycara K (1998) WebMate: a personal agent for browsing and searching. In: Sycara KP, Wooldridge M (eds) Proc. agents’98, vol 9–13. ACM Press, New York, pp 132–139Google Scholar
  8. 8.
    Christel MG (2007) Establishing the utility of non-text search for news video retrieval with real world users. In: MULTIMEDIA ’07: proceedings of the 15th international conference on multimedia. ACM, New York, pp 707–716Google Scholar
  9. 9.
    Christel MG (2007) Examining user interactions with video retrieval systems. In: SPIE’06: proceedings of SPIE, multimedia content access: algorithms and systems, vol 6506Google Scholar
  10. 10.
    Dudev M, Elbassuoni S, Luxenburger J, Ramanath M, Weikum G (2008) Personalizing the search for knowledge. In: Proc. PersDBGoogle Scholar
  11. 11.
    Fernández M, López V, Sabou M, Uren V, Vallet D, Motta E, Castells P (2009) Using TREC for cross-comparison between classic IR and ontology-based search models at a web scale. In: SemSearch’09Google Scholar
  12. 12.
    Fernández N, Blázquez JM, Fisteus JA, Sánchez L, Sintek M, Bernardi A, Fuentes M, Marrara A, Ben-Asher Z (2006) NEWS: bringing semantic web technologies into news agencies. In: Proc. ISWC, pp 778–791Google Scholar
  13. 13.
    Hopfgartner F (2011) Adaptive interactive news video recommendation: an example system. SEMAIS’11 - second international workshop on semantic models for adaptive interactive systems, vol 2. Palo Alto, CA, USA, pp 21–25Google Scholar
  14. 14.
    Hopfgartner F, Jose JM (2010) Semantic user modelling for personal news video retrieval. MMM’10 - 16th international conference on multimedia modeling, vol 1. Chongqing, China, Springer Verlag, pp 336–349Google Scholar
  15. 15.
    Hopfgartner F, Jose JM (2010) Semantic user profiling techniques for personalised multimedia recommendation. ACM/Springer multimedia systems, vol 16, issue 4. Springer-Verlag New York, Inc., Secaucus, NJ, USA, pp 255–274Google Scholar
  16. 16.
    Hopfgartner F, Jose JM (2011) Development of a test collection for studying long-term user modelling. In: LWA’11: proceedings of the workshop information retrieval at Lernen, Wissen, AdaptivitaetGoogle Scholar
  17. 17.
    Hopfgartner F, Vallet D, Halvey M, Jose JM (2008) Search trails using user feedback to improve video search. In: Proc. of the ACM int. conf. on multimedia, pp 339–348Google Scholar
  18. 18.
    Kelly D (2004) Understanding implicit feedback and document preference: a naturalistic user study. PhD thesis, Rutgers UniversityGoogle Scholar
  19. 19.
    Kelly D, Dumais ST, Pedersen JO (2009) Evaluation challenges and directions for information-seeking support systems. IEEE Comput 42(3):60–66CrossRefGoogle Scholar
  20. 20.
    Liu J, Belkin NJ (2010) Personalizing information retrieval for multi-session tasks: the roles of task stage and task type. In: Proceedings of the 33rd international ACM SIGIR conference on research and development in information retrieval, SIGIR ’10. ACM, New York, pp 26–33Google Scholar
  21. 21.
    Misra H, Hopfgartner F, Goyal A, Punitha P, Jose JM (2010) News video story segmentation based on semantic coherence and content similarity. MMM’10 - 16th international conference on multimedia modeling, vol 1. Chongqing, China, Springer Verlag, pp 347-357Google Scholar
  22. 22.
    Pérez-Agüera JR, Arroyo J, Greenberg J, Perez Iglesias J, Fresno V (2010) Using bm25f for semantic search. In: Semantic search 2010 workshopGoogle Scholar
  23. 23.
    Ruthven I, Kelly D (2011) Interactive information seeking behaviour and retrieval. Facet Press, MontereyGoogle Scholar
  24. 24.
    Shneiderman B, Plaisant C (2006) Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies. In: BELIV’06, pp 1–7Google Scholar
  25. 25.
    Vallet D, Hopfgartner F, Jose JM, Castells P (2011) Effects of usage-based feedback on video retrieval: a simulation-based study. ACM Trans Inf Syst 29(2):11:1–11:32CrossRefGoogle Scholar
  26. 26.
    Zaragoza H, Craswell N, Taylor MJ, Saria S, Robertson SE (2004) Microsoft Cambridge at TREC 13: web and hard tracks. In: TREC’04: proceedings of the 13th text retrieval conferenceGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.International Computer Science InstituteBerkeleyUSA
  2. 2.Department of Computing ScienceUniversity of GlasgowGlasgowScotland

Personalised recommendations