ISWC 2007, ASWC 2007: The Semantic Web pp 423-437 | Cite as

Evaluating the Semantic Web: A Task-Based Approach

  • Marta Sabou
  • Jorge Gracia
  • Sofia Angeletou
  • Mathieu d’Aquin
  • Enrico Motta
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4825)

Abstract

The increased availability of online knowledge has led to the design of several algorithms that solve a variety of tasks by harvesting the Semantic Web, i.e., by dynamically selecting and exploring a multitude of online ontologies. Our hypothesis is that the performance of such novel algorithms implicitly provides an insight into the quality of the used ontologies and thus opens the way to a task-based evaluation of the Semantic Web. We have investigated this hypothesis by studying the lessons learnt about online ontologies when used to solve three tasks: ontology matching, folksonomy enrichment, and word sense disambiguation. Our analysis leads to a suit of conclusions about the status of the Semantic Web, which highlight a number of strengths and weaknesses of the semantic information available online and complement the findings of other analysis of the Semantic Web landscape.

Keywords

Word Sense Word Sense Disambiguation Ontology Match Ontology Learning Online Knowledge 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

References

  1. 1.
    Alani, H.: Position Paper: Ontology Construction from Online Ontologies. In: Proc. of WWW (2006)Google Scholar
  2. 2.
    Alani, H., Brewster, C., Shadbolt, N.: Ranking Ontologies with AKTiveRank. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L. (eds.) ISWC 2006. LNCS, vol. 4273, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  3. 3.
    Aleksovski, Z., Klein, M., ten Katen, W., van Harmelen, F.: Matching Unstructured Vocabularies using a Background Ontology. In: Staab, S., Svátek, V. (eds.) EKAW 2006. LNCS (LNAI), vol. 4248, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  4. 4.
    Angeletou, S., Sabou, M., Specia, L., Motta, E.: Bridging the Gap Between Folksonomies and the Semantic Web: An Experience Report. In: Proc. of the ESWC Bridging the Gap between Semantic Web and Web 2.0 Wokshop (2007)Google Scholar
  5. 5.
    Bechhofer, S., Volz, R.: Patching Syntax in OWL ontologies. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, Springer, Heidelberg (2004)Google Scholar
  6. 6.
    Brank, J., Grobelnik, M., Mladenic, D.: A survey of ontology evaluation techniques. In: Proc. of the Conf. on Data Mining and Data Warehouses (2005)Google Scholar
  7. 7.
    Buitelaar, P., Eigner, T., Declerck, T.: OntoSelect: A Dynamic Ontology Library with Support for Ontology Selection. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, Springer, Heidelberg (2004)Google Scholar
  8. 8.
    d’Aquin, M., Baldassarre, C., Gridinoc, L., Angeletou, S., Sabou, M., Motta, E.: Characterizig Knowledge on the Semantic Web with WATSON, (submitted for peer review) (2007)Google Scholar
  9. 9.
    d’Aquin, M., Sabou, M., Dzbor, M., Baldassarre, C., Gridinoc, L., Angeletou, S., Motta, E.: WATSON: A Gateway for the Semantic Web. In: ESWC Poster (2007)Google Scholar
  10. 10.
    Ding, L., Finin, T.: Characterizing the Semantic Web on the Web. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L. (eds.) ISWC 2006. LNCS, vol. 4273, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  11. 11.
    Ding, L., Pan, R., Finin, T., Joshi, A., Peng, Y., Kolari, P.: Finding and Ranking Knowledge on the Semantic Web. In: Gil, Y., Motta, E., Benjamins, V.R., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  12. 12.
    Begelman, G., Keller, P., Smadja, F.: Automated Tag Clustering: Improving search and exploration in the tag space. In: WWW 2006. Proc. of the Collaborative Web Tagging Workshop (2006)Google Scholar
  13. 13.
    Giunchiglia, F., Shvaiko, P., Yatskevich, M.: Discovering Missing Background Knowledge in Ontology Matching. In: ECAI 2006. LNCS (LNAI), vol. 4386, Springer, Heidelberg (2006)Google Scholar
  14. 14.
    Gracia, J., Trillo, R., Espinoza, M., Mena, E.: Querying the Web: A Multiontology Disambiguation Method. In: Proc. of ICWE (2006)Google Scholar
  15. 15.
    Guarino, N., Welty, C.A.: An Overview of OntoClean. In: Staab, S., Studer, R. (eds.) Handbook on Ontologies, Springer, Heidelberg (2004)Google Scholar
  16. 16.
    Hartmann, J., Sure, Y., Giboin, A., Maynard, D., Suarez-Figueroa, M.C., Cuel, R.: Methods for ontology evaluation. Knowledge Web Deliverable. In: Methods for ontology evaluation. Knowledge Web Deliverable D1.2.3 (2005)Google Scholar
  17. 17.
    Hassell, J., Aleman-Meza, B., Arpinar, I.B.: Ontology-Driven Automatic Entity Disambiguation in Unstructured Text. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L. (eds.) ISWC 2006. LNCS, vol. 4273, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  18. 18.
    Hotho, A., Jaschke, R., Schmitz, C., Stumme, G.: Information Retrieval in Folksonomies: Search and Ranking. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  19. 19.
    Lee, J., Goodwin, R.: The Semantic Webscape: a View of the Semantic Web. IBM Research Report (2004)Google Scholar
  20. 20.
    Motta, E., Sabou, M.: Language Technologies and the Evolution of the Semantic Web. In: Proc. of the Int. Conf. on Language Resources and Evaluation (2006)Google Scholar
  21. 21.
    Motta, E., Sabou, M.: Next Generation Semantic Web Applications. In: Mizoguchi, R., Shi, Z., Giunchiglia, F. (eds.) ASWC 2006. LNCS, vol. 4185, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  22. 22.
    Navigli, R., Velardi, P.: Structural Semantic Interconnections: A Knowledge-Based Approach to Word Sense Disambiguation. IEEE Trans. Pattern Anal. Mach. Intell. 27(7), 1075–1086 (2005)CrossRefGoogle Scholar
  23. 23.
    Patel, C., Supekar, K., Lee, Y., Park, E.K.: OntoKhoj: A Semantic Web Portal for Ontology Searching, Ranking and Classification. In: Proc. of WIDM (2003)Google Scholar
  24. 24.
    Pedersen, T., Banerjee, S., Patwardhan, S.: Maximizing Semantic Relatedness to Perform Word Sense Disambiguation, 2005. Research Report UMSI 2005/25Google Scholar
  25. 25.
    Porzel, R., Malaka, R.: A Task-based Approach for Ontology Evaluation. In: Proc. of the ECAI Workshop on Ontology Learning and Population (2004)Google Scholar
  26. 26.
    Sabou, M., d’Aquin, M., Motta, E.: Using the Semantic Web as Background Knowledge for Ontology Mapping. In: Proc. of the Ontology Matching WS (2006)Google Scholar
  27. 27.
    Sabou, M.: M. d’Aquin, and E. Motta. Using the Semantic Web as Background Knowledge for Ontology Matching: Investigating a New Paradigm (submitted for peer review, 2007)Google Scholar
  28. 28.
    Sabou, M., Lopez, V., Motta, E., Uren, V.: Ontology Selection: Ontology Evaluation on the Real Semantic Web. In: Proc. of the EON Workshop (2006)Google Scholar
  29. 29.
    Schmitz, P.: Inducing Ontology from Flickr Tags. In: WWW 2006. Proc. of the Collaborative Web Tagging Workshop (2006)Google Scholar
  30. 30.
    Shvaiko, P., Euzenat, J.: A Survey of Schema-based Matching Approaches. Journal on Data Semantics IV (2005)Google Scholar
  31. 31.
    Specia, L., Motta, E.: Integrating Folksonomies with the Semantic Web. In: ESWC 2007. LNCS, vol. 4519, Springer, Heidelberg (2007)Google Scholar
  32. 32.
    van Hage, W., Katrenko, S., Schreiber, G.: A Method to Combine Linguistic Ontology-Mapping Techniques. In: Gil, Y., Motta, E., Benjamins, V.R., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  33. 33.
    Völker, J., Vrandecic, D., Sure, Y.: Automatic Evaluation of Ontologies (AEON). In: Gil, Y., Motta, E., Benjamins, V.R., Musen, M.A. (eds.) ISWC 2005. LNCS, vol. 3729, Springer, Heidelberg (2005)CrossRefGoogle Scholar
  34. 34.
    Wang, T.D., Parsia, B., Hendler, J.A.: A Survey of the Web Ontology Landscape. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L. (eds.) ISWC 2006. LNCS, vol. 4273, Springer, Heidelberg (2006)Google Scholar
  35. 35.
    Wu, X., Zhang, L., Yu, Y.: Exploring Social Annotations for the Semantic Web. In: Proc. of WWW (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Marta Sabou
    • 1
  • Jorge Gracia
    • 2
  • Sofia Angeletou
    • 1
  • Mathieu d’Aquin
    • 1
  • Enrico Motta
    • 1
  1. 1.Knowledge Media Institute (KMi), The Open UniversityUnited Kingdom
  2. 2.IIS Department,University of ZaragozaSpain

Personalised recommendations