Knowledge and Information Systems

, Volume 30, Issue 2, pp 319–340 | Cite as

What is the difference? A cognitive dissimilarity measure for information retrieval result sets

  • Carsten KeßlerEmail author
Regular Paper


Result rankings from context-aware information retrieval are inherently dynamic, as the same query can lead to significantly different outcomes in different contexts. For example, the search term Digital Camera will lead to different—albeit potentially overlapping—results in the contexts customer reviews and shops, respectively. The comparison of such result rankings can provide useful insights into the effects of context changes on the information retrieval results. In particular, the impact of single aspects of the context in complex applications can be analyzed to identify the most (and least) influential context parameters. While a multitude of methods exists for assessing the relevance of a result ranking with respect to a given query, the question how different two result rankings are from a user’s point of view has not been tackled so far. This paper introduces DIR, a cognitively plausible dissimilarity measure for information retrieval result sets that is based solely on the results and thus applicable independently of the retrieval method. Unlike statistical correlation measures, this dissimilarity measure reflects how human users quantify the changes in information retrieval result rankings. The DIR measure supports cognitive engineering tasks for information retrieval, such as work flow and interface design: using the measure, developers can identify which aspects of context heavily influence the outcome of the retrieval task and should therefore be in the focus of the user’s interaction with the system. The cognitive plausibility of DIR has been evaluated in two human participants tests, which demonstrate a strong correlation with user judgments.


Cognitive information retrieval Human–computer interaction Context awareness 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agichtein E, Brill E, Dumais S, Ragno R (2006) Learning user interaction models for predicting web search result preferences. In: Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval. ACM Press, New York, NY, USA, pp 3–10Google Scholar
  2. 2.
    Albertoni R, De Martino M (2008) Asymmetric and context-dependent semantic similarity among ontology instances. J Data Semant Lect Notes Comput Sci 4900: 1–30CrossRefGoogle Scholar
  3. 3.
    Baeza-Yates R, Ribeiro-Neto B (1999) Modern information retrieval. Addison Wesley, BostonGoogle Scholar
  4. 4.
    Bazire M, Brézillon P (2005) Understanding context before using it. In: Dey AK, Kokinov B, Leake D, Turner R (eds) Modeling and using context—5th international and interdisciplinary conference (CONTEXT 2005), Paris, France. Lecture notes in computer science, vol 3554. Springer, Berlin, pp 29–40Google Scholar
  5. 5.
    Bikakis A, Antoniou G, Hasapis P (2010) Strategies for contextual reasoning with conflicts in ambient intelligence. Knowl Info SystGoogle Scholar
  6. 6.
    Brown PJ, Jones GJF (2001) Context-aware retrieval: exploring a new environment for information retrieval and information filtering. Pers Ubiquit Comput 5: 253–263CrossRefGoogle Scholar
  7. 7.
    Dey A (2001) Understanding and using Context. Pers Ubiquit Comput 5(1): 4–7CrossRefGoogle Scholar
  8. 8.
    Efron M (2009) Using multiple query aspects to build test collections without human relevance judgments. In: Boughanem M, Berrut C, Mothe J, Soule-Dupuy C (eds) ECIR ’09: proceedings of the 31th European conference on IR research on advances in information retrieval. Lecture notes in computer science, vol 5478. Springer, pp 276–287Google Scholar
  9. 9.
    Finkelstein L, Gabrilovich E, Matias Y, Rivlin E, Solan Z, Wolfman G, Ruppin E (2001) Placing search in context: the concept revisited. In: WWW ’01: proceedings of the 10th international conference on World Wide Web. ACM Press, New York, NY, USA, pp 406–414Google Scholar
  10. 10.
    Gärdenfors P (2000) Conceptual spaces: the geometry of thought. MIT Press, CambridgeGoogle Scholar
  11. 11.
    Goldstone R, Son J (2004) Similarity. In: Holyoak K, Morrison R (eds) Cambridge handbook of thinking and reasoning. Cambridge University Press, CambridgeGoogle Scholar
  12. 12.
    Harrison S (1995) A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM Press/Addison- Wesley Publishing Co., New York, NY, USA, pp 82–89Google Scholar
  13. 13.
    Hu J, Chan P (2008) Personalized web search by using learned user profiles in re-ranking. In: Workshop on knowledge discovery on the web, KDD conference. pp 84–97Google Scholar
  14. 14.
    Janowicz K (2008) Kinds of contexts and their impact on semantic similarity measurement. In: 5th IEEE workshop on context modeling and reasoning (CoMoRea) at the 6th IEEE international conference on pervasive computing and communication (PerComa’08)Google Scholar
  15. 15.
    Janowicz K, Keßler C, Panov I, Wilkes M, Espeter M, Schwarz M (2008) A study on the cognitive plausibility of SIM-DL similarity rankings for geographic feature types. In: Bernard L, Friis-Christensen A, Pundt H (eds) The European information society—taking geoinformation science one step further (AGILE 2008 proceedings). Lecture notes in geoinformation and cartography. Springer, Berlin, pp 115–134Google Scholar
  16. 16.
    Janowicz K, Keßler C, Schwarz M, Wilkes M, Panov I, Espeter M, Bäumer B (2007) Algorithm, implementation and application of the SIM-DL similarity server. In: Fonseca F, Rodríguez M (eds) Second international conference on geoSpatial semantics, GeoS 2007. Lecture notes in computer science, vol 4853. Springer, Berlin, pp 128–145Google Scholar
  17. 17.
    Kendall MG (1938) A new measure of rank correlation. Biometrika 30(12)Google Scholar
  18. 18.
    Kent A, Berry MM, Fred J, Luehrs U, Perry JW (1955) Machine literature searching VIII. Operational criteria for designing information retrieval systems. Am Document 6(2): 93–101CrossRefGoogle Scholar
  19. 19.
    Keßler C (2007) Similarity measurement in context. In: Kokinov B, Richardson D, Roth-Berghofer T, Vieu L (eds) 6th international and interdisciplinary conference, CONTEXT 2007, Roskilde, Denmark. Lecture notes in artificial intelligence, vol 4635. Springer, Berlin, pp 277–290Google Scholar
  20. 20.
    Keßler C, Raubal M, Janowicz K (2007) The effect of context on semantic similarity measurement. In: Meersman R, Tari Z, Herrero P (eds) On the move—OTM 2007 workshops, part II. Lecture notes in computer science, vol 4806. Springer, Berlin, pp 1274–1284Google Scholar
  21. 21.
    Keßler C, Raubal M, Wosniok C (2009) Semantic rules for context-aware geographical information retrieval. In: Barnaghi P, Moessner K, Presser M, Meissner S (eds) Smart sensing and context, 4th European conference, EuroSSC 2009, Guildford, UK, September 2009. Lecture notes in computer science, vol 5741. Springer-Verlag, Berlin, pp 77–92Google Scholar
  22. 22.
    Kim HR, Chan PK (2008) Learning implicit user interest hierarchy for context in personalization. Appl Intell 28(2): 153–166CrossRefGoogle Scholar
  23. 23.
    Kokinov B, Richardson D, Roth-Berghofer T, Vieu L (eds) (2007) Modeling and using context 6th international and interdisciplinary conference CONTEXT 2007, Roskilde, Denmark, 20–24, August, 2007, proceedings. Lecture notes in artificial intelligence, vol 4635. Springer, BerlinGoogle Scholar
  24. 24.
    Kraft R, Chang CC, Maghoul F, Kumar R (2006) Searching with context. In: WWW ’06: proceedings of the 15th international conference on World Wide Web. ACM Press, New York, NY, USA, pp 477–486Google Scholar
  25. 25.
    Leonidis A, Baryannis G, Fafoutis X, Korozi M, Gazoni N, Dimitriou M, Koutsogiannaki M, Boutsika A, Papadakis M, Papagiannakis H, Tesseris G, Voskakis E, Bikakis A, Antoniou G (2009) Alertme: a semantics-based context-aware notification system. In: 33rd annual IEEE international computer software and applications conference, pp 200–205Google Scholar
  26. 26.
    Manjunath BS, Ohm JR, Vasudevan VV, Yamada A (2001) Color and texture descriptors. IEEE Trans Circuit Syst Video Technol 11(6): 703–715CrossRefGoogle Scholar
  27. 27.
    Marchionini G (2006) Toward human-computer information retrieval. June/July 2006 bulletin of the American society for information science, available online at
  28. 28.
    Medin D, Goldstone R, Gentner D (1993) Respects for similarity. Psychol Rev 100(2): 254–278CrossRefGoogle Scholar
  29. 29.
    Melucci M (2008) A basis for information retrieval in context. ACM Trans Info Syst 26(3): 1–41CrossRefGoogle Scholar
  30. 30.
    Melucci M, Pretto M (2007) PageRank: when order changes. Lecture notes in computer science. Springer, Berlin, pp, pp 581–588Google Scholar
  31. 31.
    Meza BA, Halaschek C, Arpinar BI, Sheth A (2003) Context-aware semantic association ranking. In: Semantic web and databases workshop proceedings. Berlin, Germany, pp 33–50Google Scholar
  32. 32.
    Miller GA (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol Rev 63: 81–97CrossRefGoogle Scholar
  33. 33.
    Nedas KA, Egenhofer MJ (2008) Integral vs. separable attributes in spatial similarity assessments. In: Proceedings of the international conference on spatial cognition VI. Springer, Berlin, pp 295–310Google Scholar
  34. 34.
    Page L, Brin S, Motwani R, Winograd T (1998) The PageRank citation ranking: bringing order to the web. Technical report. Digital Library Technologies Project, StanfordGoogle Scholar
  35. 35.
    Pfitzner D, Leibbrandt R, Powers D (2009) Characterization and evaluation of similarity measures for pairs of clusterings. Knowl Info Syst 19(3): 361–394CrossRefGoogle Scholar
  36. 36.
    Raubal M (2004) Formalizing conceptual spaces. In: Vieu LVA (eds) Formal ontology in information systems, proceedings of the 3rd international conference (FOIS 2004), Frontiers in artificial intelligence and applications. IOS Press, Amsterdam, NL, pp 153–164Google Scholar
  37. 37.
    Rissland E (2006) AI and similarity. IEEE Intell Syst 21(3): 39–49CrossRefGoogle Scholar
  38. 38.
    Robertson SE (1997) The probability ranking principle in IR. pp 281–286Google Scholar
  39. 39.
    Rodgers JL, Nicewanderer WA (1988) Thirteen ways to look at the correlation coefficient. Am Stat 42(1): 59–66CrossRefGoogle Scholar
  40. 40.
    Rodríguez A, Egenhofer MJ (2004) Comparing geospatial entity classes: an asymmetric and context-dependent similarity measure. Int J Geo Info Sci 18(3): 229–256CrossRefGoogle Scholar
  41. 41.
    Rose DE, Levinson D (2004) Understanding user goals in web search. In: Feldman S, Uretsky M (eds) WWW ’04: Proceedings of the 13th international conference on World Wide Web. ACM Press, pp 13–19Google Scholar
  42. 42.
    Rosset S, Perlich C, Zadrozny B (2007) Ranking-based evaluation of regression models. Knowl Info Syst 12(3): 331–353CrossRefGoogle Scholar
  43. 43.
    Schwering A (2008) Approaches to semantic similarity measurement for Geo-spatial data—a survey. Trans GIS 12(1): 5–12CrossRefGoogle Scholar
  44. 44.
    Spearman C (1904) The proof and measurement of association between two things. Am J psychol 15: 72–101CrossRefGoogle Scholar
  45. 45.
    Spink, A, Cole, C (eds) (2005) New directions in cognitive information retrieval. Springer, NetherlandszbMATHGoogle Scholar
  46. 46.
    Strang T, Linnhoff-Popien C (2004) A context modeling survey. In: First international workshop on advanced context modelling, reasoning and management at UbiComp 2004, Nottingham, England, 7 September, 2004Google Scholar
  47. 47.
    Strube G (1992) The role of cognitive science in knowledge engineering. In: Proceedings of the first joint workshop on contemporary knowledge engineering and cognition. Lecture notes in computer science, vol 622. Springer, pp 161–174Google Scholar
  48. 48.
    Tamine-Lechani L, Boughanem M, Daoud M (2009) Evaluation of contextual information retrieval effectiveness: overview of issues and research. Knowl Info SystGoogle Scholar
  49. 49.
    Ukkonen A, Castillo C, Donato D, Gionis A (2008) Searching the wikipedia with contextual information. In: CIKM ’08: proceeding of the 17th ACM conference on information and knowledge mining. ACM press, New York, NY, USA, pp 1351–1352Google Scholar
  50. 50.
    van Rijsbergen CJ (1979) Information retrieval, 2 edn. Butterworth.Google Scholar
  51. 51.
    Wang D, Tse Q, Zhou Y (2009) A decentralized search engine for dynamic web communities. Knowl Info SystGoogle Scholar
  52. 52.
    Wang J (2009) Mean-Variance Analysis: A New Document Ranking Theory in Information Retrieval. In: Boughanem M, Berrut C, Mothe J, Soule-Dupuy C (eds) ECIR ’09: Proceedings of the 31th European conference on IR research on advances in information retrieval. Lecture notes in computer science, vol 5478. Springer, pp 4–16Google Scholar
  53. 53.
    Weerkamp W, Balog K, de Rijke M (2009) Using contextual information to improve search in email archives. In: Advances in information retrieval. 31st European conference on information retrieval conference (ECIR 2009). pp 400–411Google Scholar
  54. 54.
    Wilkes M (2008) A graph-based alignment approach to context-sensitive similarity between climbing routes. Diploma thesis, Institute for Geoinformatics, University of Münster, GermanyGoogle Scholar
  55. 55.
    Wu G, Chang EY, Panda N (2005) Formulating context-dependent similarity functions. In: MULTIMEDIA ’05: proceedings of the 13th annual ACM international conference on multimedia. ACM press, New York, NY, USA, pp 725–734Google Scholar
  56. 56.
    Yang J, Cheung W, Chen X (2009) Learning element similarity matrix for semi-structured document analysis. Knowl Info Syst 19(1): 53–78CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Limited 2011

Authors and Affiliations

  1. 1.Institute for GeoinformaticsUniversity of MünsterMünsterGermany

Personalised recommendations