BetterRelations: Collecting Association Strengths for Linked Data Triples with a Game

  • Jörn Hees
  • Thomas Roth-Berghofer
  • Ralf Biedert
  • Benjamin Adrian
  • Andreas Dengel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7538)


The simulation of human thinking is one of the long term goals of the Artificial Intelligence community. In recent years, the adoption of Semantic Web technologies and the ongoing sharing of Linked Data has generated one of the world’s largest knowledge bases, bringing us closer to this dream than ever. Nevertheless, while associations in the human memory have different strengths, such explicit association strengths (edge weights) are missing in Linked Data. Hence, finding good heuristics which can estimate human-like association strengths for Linked Data facts (triples) is of major interest to us. In order to evaluate existing approaches with respect to human-like association strengths, we need a collection of such explicit edge weights for Linked Data triples.

In this chapter we first provide an overview of existing approaches to rate Linked Data triples which could be valuable candidates for good heuristics. We then present a web-game prototype which can help with the collection of a ground truth of edge weights for triples. We explain the game’s concept, summarize Linked Data related implementation aspects, and include a detailed evaluation of the game.


Mean Square Error Semantic Relatedness Link Data Association Strength Naming Authority 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Berners-Lee, T., Hendler, J., Lassila, O.: The Semantic Web: A new form of Web content that is meaningful to computers will unleash a revolution of new possibilities. Scientific American 284(5), 34–43 (2001)CrossRefGoogle Scholar
  2. 2.
    Bizer, C., Heath, T., Berners-Lee, T.: Linked Data - The Story So Far. International Journal on Semantic Web and Information Systems 5(3), 1–22 (2009)CrossRefGoogle Scholar
  3. 3.
    Bizer, C., Lehmann, J., Kobilarov, G., Auer, S., Becker, C., Cyganiak, R., Hellmann, S.: DBpedia - A crystallization point for the Web of Data. Web Semantics: Science, Services and Agents on the World Wide Web 7(3), 154–165 (2009)CrossRefGoogle Scholar
  4. 4.
    van Elst, L., Abecker, A.: Ontologies for information management: balancing formality, stability, and sharing scope. Expert Systems with Applications 23(4), 357–366 (2002)CrossRefGoogle Scholar
  5. 5.
    Crestani, F.: Application of Spreading Activation Techniques in Information Retrieval. Artificial Intelligence Review 11(6), 453–482 (1997)CrossRefGoogle Scholar
  6. 6.
    Schumacher, K., Sintek, M., Sauermann, L.: Combining Fact and Document Retrieval with Spreading Activation for Semantic Desktop Search. In: Bechhofer, S., Hauswirth, M., Hoffmann, J., Koubarakis, M. (eds.) ESWC 2008. LNCS, vol. 5021, pp. 569–583. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  7. 7.
    Brin, S., Page, L.: The Anatomy of a Large-Scale Hypertextual Web Search Engine. Computer Networks and ISDN Systems 30(1-7), 107–117 (1998)CrossRefGoogle Scholar
  8. 8.
    Kleinberg, J.M.: Authoritative sources in a hyperlinked environment. Journal of the ACM 46(5), 604–632 (1999)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Hees, J., Roth-Berghofer, T., Dengel, A.: Linked Data Games: Simulating Human Association with Linked Data. In: LWA 2010, Kassel, Germany (2010)Google Scholar
  10. 10.
    Hees, J., Roth-Berghofer, T., Biedert, R., Adrian, B., Dengel, A.: BetterRelations: Using a Game to Rate Linked Data Triples. In: Bach, J., Edelkamp, S. (eds.) KI 2011. LNCS, vol. 7006, pp. 134–138. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  11. 11.
    von Ahn, L., Dabbish, L.: Designing games with a purpose. Communications of the ACM 51(8), 58–67 (2008)Google Scholar
  12. 12.
    Balmin, A., Hristidis, V., Papakonstantinou, Y.: ObjectRank: Authority-Based Keyword Search in Databases. In: Proc. of the 13th International Conference on Very Large Data Bases, pp. 564–575. VLDB Endowment (2004)Google Scholar
  13. 13.
    Ding, L., Pan, R., Finin, T., Joshi, A., Peng, Y., Kolari, P.: Finding and Ranking Knowledge on the Semantic Web. In: Gil, Y., Motta, E., Benjamins, V.R., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, pp. 156–170. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  14. 14.
    Harth, A., Kinsella, S., Decker, S.: Using Naming Authority to Rank Data and Ontologies for Web Search. In: Bernstein, A., Karger, D.R., Heath, T., Feigenbaum, L., Maynard, D., Motta, E., Thirunarayan, K. (eds.) ISWC 2009. LNCS, vol. 5823, pp. 277–292. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Delbru, R., Toupikov, N., Catasta, M., Tummarello, G., Decker, S.: Hierarchical Link Analysis for Ranking Web Data. In: Aroyo, L., Antoniou, G., Hyvönen, E., ten Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) ESWC 2010, Part II. LNCS, vol. 6089, pp. 225–239. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Tummarello, G., Delbru, R., Oren, E.: Weaving the Open Linked Data. In: Aberer, K., Choi, K.-S., Noy, N., Allemang, D., Lee, K.-I., Nixon, L.J.B., Golbeck, J., Mika, P., Maynard, D., Mizoguchi, R., Schreiber, G., Cudré-Mauroux, P. (eds.) ASWC/ISWC 2007. LNCS, vol. 4825, pp. 552–565. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Franz, T., Schultz, A., Sizov, S., Staab, S.: TripleRank: Ranking Semantic Web Data by Tensor Decomposition. In: Bernstein, A., Karger, D.R., Heath, T., Feigenbaum, L., Maynard, D., Motta, E., Thirunarayan, K. (eds.) ISWC 2009. LNCS, vol. 5823, pp. 213–228. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  18. 18.
    Anyanwu, K., Maduko, A., Sheth, A.P.: SemRank: Ranking Complex Relationship Search Results on the Semantic Web. In: Proc. of the WWW 2005, Chiba, Japan (2005)Google Scholar
  19. 19.
    Ell, B., Vrandečić, D., Simperl, E.: Labels in the Web of Data. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011, Part I. LNCS, vol. 7031, pp. 162–176. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  20. 20.
    Fellbaum, C. (ed.): WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)zbMATHGoogle Scholar
  21. 21.
    Budanitsky, A., Hirst, G.: Evaluating WordNet-based Measures of Lexical Semantic Relatedness. Computational Linguistics 32(1), 13–47 (2006)zbMATHCrossRefGoogle Scholar
  22. 22.
    Strube, M., Ponzetto, S.P.: WikiRelate! Computing Semantic Relatedness Using Wikipedia. In: Proc. of the AAAI 2006, pp. 1419–1424. AAAI Press, Boston (2006)Google Scholar
  23. 23.
    Cilibrasi, R.L., Vitányi, P.M.B.: The Google Similarity Distance. IEEE Trans. Knowledge and Data Engineering 19(3), 370–383 (2007)CrossRefGoogle Scholar
  24. 24.
    Cattuto, C., Benz, D., Hotho, A., Stumme, G.: Semantic Grounding of Tag Relatedness in Social Bookmarking Systems. In: Sheth, A.P., Staab, S., Dean, M., Paolucci, M., Maynard, D., Finin, T., Thirunarayan, K. (eds.) ISWC 2008. LNCS, vol. 5318, pp. 615–631. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  25. 25.
    Agirre, E., Alfonseca, E., Hall, K., Kravalova, J., Pasça, M., Soroa, A.: A Study on Similarity and Relatedness Using Distributional and WordNet-based Approaches. In: Proc. of the NAACL 2009, pp. 19–27. Association for Computational Linguistics, Boulder (2009)CrossRefGoogle Scholar
  26. 26.
    Mirizzi, R., Ragone, A., Di Noia, T., Di Sciascio, E.: Ranking the Linked Data: The Case of DBpedia. In: Benatallah, B., Casati, F., Kappel, G., Rossi, G. (eds.) ICWE 2010. LNCS, vol. 6189, pp. 337–354. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  27. 27.
    Waitelonis, J., Sack, H.: Towards Exploratory Video Search Using Linked Data. In: Proc. of the IEEE International Symposium on Multimedia (ISM) 2009, pp. 540–545. IEEE, San Diego (2009)CrossRefGoogle Scholar
  28. 28.
    Hacker, S., von Ahn, L.: Matchin: Eliciting User Preferences with an Online Game. In: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1207–1216. ACM, Boston (2009)Google Scholar
  29. 29.
    Siorpaes, K., Hepp, M.: OntoGame: Towards Overcoming the Incentive Bottleneck in Ontology Building. In: Meersman, R., Tari, Z. (eds.) OTM-WS 2007, Part II. LNCS, vol. 4806, pp. 1222–1232. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  30. 30.
    Kny, E., Kölle, S., Töpper, G., Wittmers, E.: WhoKnows? (October 2010)Google Scholar
  31. 31.
    Singh, P.: The Open Mind Common Sense Project. (January 2002)Google Scholar
  32. 32.
    Herbrich, R., Minka, T., Graepel, T.: TrueSkill(TM): A Bayesian Skill Rating System. In: Schölkopf, B., Platt, J., Hoffmann, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 569–576. MIT Press, Cambridge (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jörn Hees
    • 1
    • 2
  • Thomas Roth-Berghofer
    • 2
    • 3
  • Ralf Biedert
    • 2
  • Benjamin Adrian
    • 2
  • Andreas Dengel
    • 1
    • 2
  1. 1.Computer Science DepartmentUniversity of KaiserslauternGermany
  2. 2.Knowledge Management DepartmentDFKI GmbHKaiserslauternGermany
  3. 3.School of Computing and TechnologyUniversity of West LondonUK

Personalised recommendations