Skip to main content
SpringerLink
Log in

Human Intelligence in the Process of Semantic Content Creation

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Despite significant progress over the last years the large-scale adoption of semantic technologies is still to come. One of the reasons for this state of affairs is assumed to be the lack of useful semantic content, a prerequisite for almost every IT system or application using semantics. Through its very nature, this content can not be created fully automatically, but requires, to a certain degree, human contribution. The interest of Internet users in semantics, and in particular in creating semantic content, is, however, low. This is understandable if we think of several characteristics exposed by many of the most prominent semantic technologies, and the applications thereof. One of these characteristics is the high barrier of entry imposed. Interacting with semantic technologies today requires specific skills and expertise on subjects which are not part of the mainstream IT knowledge portfolio. A second characteristic are the incentives that are largely missing in the design of most semantic applications. The benefits of using machine-understandable content are in most applications fully decoupled from the effort of creating and maintaining this content. In other words, users do not have a motivation to contribute to the process. Initiatives in the areas of the Social Semantic Web acknowledged this problem, and identified mechanisms to motivate users to dedicate more of their time and resources to participate in the semantic content creation process. Still, even if incentives are theoretically in place, available human labor is limited and must only be used for those tasks that are heavily dependent on human intervention, and cannot be reliably automated. In this article, we concentrate on this step in between. As a first contribution, we analyze the process of semantic content creation in order to identify those tasks that are inherently human-driven. When building semantic applications involving these specific tasks, one has to install incentive schemes that are likely to encourage users to perform exactly these tasks that crucially rely on manual input. As a second contribution of the article, we propose incentives or incentive-driven tools that can be used to increase user interest in semantic content creation tasks. We hope that our findings will be adopted as recommendations for establishing a fundamentally new form of design of semantic applications by the semantic technologies community.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Akkiraju, R., Farrell, J., Miller, J., Nagarajan, M., Schmidt, M.T., Sheth, A., Verma, K.: Web service semantics—wsdl-s. Technical Report, W3C Member Submission (2005)

  2. Asirelli, P., Little, S., Martinelli, M., Salvetti, O.: Mulimedia metadata management: a proposal for an infrastructure. In: Proceedings of SWAP 2006 (2006)

  3. Aussenac-Gilles, N., Despres, S., Szulman, S.: Ontology learning and population: bridging the gap between text and knowledge. Chapter: The TERMINAE method and platform for ontology engineering from texts, pp. 199–223. IOS, Amsterdam (2008)

    Google Scholar 

  4. Barras, C., Geoffrois, E., Wu, Z., Liberman, M.,: Transcriber: development and use of a tool for assisting speech corpora production. Speech Communication Special Issue on Speech Annotation and Corpus Tools 33(1–2) (2000)

  5. Bernaras, A., Laresgoiti, I., Corera J.: Building and reusing ontologies for electrical network applications. In: European Conference on Articifial Intelligence (ECAI’96) (1996)

  6. Bloehdorn, S., Petridis, K., Saathoff, C., Simou, N., Tzouvaras, V., Avrithis, Y., Handschuh, S., Kompatsiaris, Y., Staab, S., Strintzis, M.G.: Semantic annotation of images and videos for multimedia analysis. In: European Semantic Web Conference (ESWC). Springer LNCS, New York (2005)

    Google Scholar 

  7. Bontcheva, K., Kiryakov, A., Cunningham, H., Popov, B., Dimitrov, M.: Semantic web enabled, open source language technology. In: EACL Workshop on Language Technology and the Semantic Web: NLP and XML, Budapest, Hungary (2003)

  8. Brank, J., Grobelnik, M., Dunja M.: A survey of ontology evaluation techniques. In: Conference on Data Mining and Data Warehouses (SiKDD 2005) (2005)

  9. Braun, S., Schmidt, A., Walter, A., Nagypal, G., Zacharias, V.: Ontology maturing: a collaborative web 2.0 approach to ontology engineering. In: Workshop on Social and Collaborative Construction of Structured Knowledge at the 16th International World Wide Web Conference (WWW2007), Banff, Canada, 8 May 2007

  10. Buerger, T., Ammendola, C.: A user centered annotation methodology for multimedia content. In: Poster Session at ESWC 2008. CEUR, vol. 367 (2008)

  11. Campanini, S.E., Castagna, P., Tazzoli, R.: Platypus wiki: a semantic wiki wiki web. In: International Semantic Web Conference (ISWC), pp. 1–6, 10 December 2004

  12. Cantador, I., Fernández, M., Castells, P.: Improving ontology recommendation and reuse in WebCORE by collaborative assessments. In: Proceedings of the 1st International Workshop on Social and Collaborative Construction of Structured Knowledge CKC 2007 at the at the 16th International World Wide Web Conference WWW 2007 (2007)

  13. Chakarvarthy, A., Ciravegna, F., Lanfranchi, V.: Cross-media document annotation and enrichment. In: 1st Semantic Authoring and Annotation Workshop (SAAW2006) (2006)

  14. Cimiano, P.: Ontology Learning and Population from Text: Algorithms, Evaluation and Applications. Springer, New York (2006)

    Google Scholar 

  15. Cimiano, P., Handschuh, S., Staab, S.: Towards the self-annotating web. In: Thirteenth International Conference on World Wide Web (2004)

  16. Ciravegna, F., Wilks, Y.: Designing Adaptive Information Extraction for the Semantic web in Amilcare, Volume Annotation for the Semantic Web. IOS, Amsterdam (2003)

    Google Scholar 

  17. Ciravegna, F., Dingli, A., Petrelli, D., Wilks, Y.: User-system cooperation in document annotation based on information. In: 13th International Conference on Knowledge Engineering and KM (EKAW02) (2002)

  18. De Moor, A., De Leenheer, P., Meersmann, R.: Dogma-mess: A meaning evolution support system for interorganizational ontology engineering. In: 14th International Conference on Conceptual Structures (ICCS 2006). Springer LNCS, New York (2006)

    Google Scholar 

  19. Dellschaft, K., Staab, S.: Ontology learning and population: briding the gap between text and knowledge. Chapter: Strategies for the Evaluation of Ontology Learning, pp. 253–272. IOS, Amsterdam (2008)

    Google Scholar 

  20. Dill, S., Gibson, N., Gruhl, D., Guha, R.V., Jhingran, A., Kanungo, T., Rajagopalan, S., Tomkins, A., Tomlin, J.A., Zien, J.Y.: Semtag and seeker: bootstrapping the semantic web via automated semantic annotation. In: Twelfth International World Wide Web Conference (2003)

  21. Dimitrov, M., Simov, A., Momtchev, V., Konstantinov, M.: WSMO studio — a semantic web services modelling environment for WSMO (system description). In: European Semantic Web Conference (ESWC 2007), Innsbruck, Austria (2007)

  22. Dingli, A., Ciravegna, F., Wilks, Y.: Automatic semantic annotation using unsupervised information extraction and integration. In: K-CAP 2003 Workshop on Knowledge Markup and Semantic Annotation (2003)

  23. Domingue, J., Dzbor, M., Motta, E.: Magpie: Supporting browsing and navigation on the semantic web. In: ACM Conference on Intelligent User Interfaces (IUI) (2004)

  24. Ehrig, M., Sure, Y.: Ontology mapping—an integrated approach. Technical Report, AIFB Karlsruhe (2004)

  25. Euzenat, J., Shvaiko, P.: Ontology Matching. Springer, New York (2007)

    MATH  Google Scholar 

  26. Euzenat, J., Mocan, A., Scharffe, F.: Ontology alignments. In: Semantic Web and Beyond, vol. 6. Springer, New York (2007)

    Google Scholar 

  27. Falconer, S.M., Storey, M.-A.: A cognitive support framework for ontology mapping. In: Asian Semantic Web Conference (ASWC 2007) (2007)

  28. Farrell, J., Lausen, H.: Semantic annotations for wsdl and xml schema. Technical Report, W3C Recommendation (2007)

  29. Fellbaum, C.: Wordnet: An Electronic Lexical Database. MIT, Cambridge (1998)

    MATH  Google Scholar 

  30. Fensel, D., Lausen, H., Polleres, A., De, J., Bruijn, Stollberg, M., Roman, D., Domingue, J.: Enabling Semantic Web Services: The Web Service Modeling Ontology. Springer, New York (2006)

    Google Scholar 

  31. Fernandez-Lopez, M., Gomez-Perez, A., Juristo, N.: Methontology: from ontological art towards ontological engineering. In: AAAI97 Spring Symposium Series on Ontological Engineering, Stanford, USA (1997)

  32. Gangemi, A., Catenacci, C., Ciaramita, M., Lehmann, J.: Modeling ontology evaluation and validation. In: Proceedings of the European Semantic Web Conference (ESWC 2006), pp. 140–154. Springer, New York (2006)

    Google Scholar 

  33. Gómez-Pérez, A.: Evaluation of ontologies. Int. J. Intell. Syst. 16(3), 391–409 (2001)

    Article  MATH  Google Scholar 

  34. Gomez-Perez, A.: A survey of ontology learning methods and techniques. Technical Report, OntoWeb Deliverable 1.5 (2003)

  35. Gomez-Perez, A., Fernandez-Lopez, M., Corcho, O.: Ontological engineering. In: Advanced Information and Knowledge Processing. Springer, New York (2004)

    Google Scholar 

  36. Gruber, T.R.: Toward principles for the design of ontologies used for knowledge sharing. Int. J. Human-Comput. Stud. 43, 907–928 (1995)

    Article  Google Scholar 

  37. Gruninger, M., Fox, M.S.: Methodology for the design and evaluation of ontologies. In: Workshop on Basic Ontological Issues in Knowledge Sharing, Montreal, Canada 1995

  38. Guarino, N., Welty, C.A.: Evaluating ontological decisions with ontoclean. Commun. ACM 45(2), 61–65 (2002)

    Article  Google Scholar 

  39. Handschuh, S., Staab, S., Ciravegna, F.: S-cream—semi-automatic creation of metadata. In: SAAKM 2002—Semantic Authoring, Annotation & Knowledge Markup (2002)

  40. Hemetsberger, A.: When consumers produce on the internet: the relationship between cognitive-affective, socially-based, and behavioral involvement of prosumers. J. Soc. Psychol. (2003)

  41. Holsapple, C.W., Joshi, K.D.: A collaborative approach to ontology design. Commun. ACM 45(2), 42–47 (2002)

    Article  Google Scholar 

  42. Jarrar, M., Meersman, R.: Formal ontology engineering in the dogma approach. In: On the Move to Meaningful Internet Systems 2002: CoopIS, DOA, and ODBASE (2002)

  43. Kerrigan, M., Mocan, A., Simperl, E., Fensel, D.: Modeling semantic web services with the web service modeling toolkit. Technical Report, Semantic Technology Institute (STI) (2008)

  44. Kerrigan, M., Mocan, A., Tanler, M., Fensel, D.: The web service modeling toolkit — an integrated development environment for semantic web services (system description). In: European Semantic Web Conference (ESWC 2007), Innsbruck, Austria (2007)

  45. Kiryakov, A., Ognyanov, D., Manov, D.: Owlim—a pragmatic semantic repository for owl. In: Proceedings of International Workshop on Scalable Semantic Web Knowledge Base Systems (SSWS 2005), WISE 2005, New York City, USA, 20 November 2005

  46. Kogut, P., Holmes, W.: Applying information extraction to generate daml annotations from web pages. In: First International Conference on Knowledge Capture 2001

  47. Kotis, K., Vouros, G.A.: Human-centered ontology engineering: the hcome methodology. Knowl. Inf. Syst. 10(1), 109–131 (2005)

    Article  Google Scholar 

  48. Kuznetsov, S.: Motivations of contributors to wikipedia. ACM SIGCAS Comput. Soc. 36(2) (2006)

  49. Lenat, D.B., Guha, R.V.: Building Large Knowledge-based Systems: Representation and Inference in the Cyc Project. Addison-Wesley, Reading (1990)

    Google Scholar 

  50. Lozano-Tello, A., Gomez-PerezL, A.: Ontometric: a method to choose the appropriate ontology. J. Database Manage. 15(2) (2004)

  51. Lux, M., Becker, J., Krottmaier, H.: Caliph & Emir: semantic annotation and retrieval in personal digital photo libraries. In: CAISE Forum at the 15th Conference on Advanced Information Systems Engineering (2003)

  52. Maedche, A., Staab, S.: Ontology learning for the semantic web. IEEE Intell. Syst. 16(2), 72–79 (2001)

    Article  Google Scholar 

  53. Marlow, C., Naaman, M., Boyd, D., Davis, M.: Position paper, tagging, taxonomy, flickr, article, toread. In: In Proceedings of the World Wide Web Conference (WWW2006), Edinburgh, Scotland, 22–26 May 2006. ACM (2006)

  54. Martin, D., Burstein, M., Denker, G., Hobbs, J., Kagal, L., Lassila, O., McDermott, D., McIlraith, S., Paolucci, M., Parsia, B., Payne, T., Sabou, M., Sirin, E., Solanki, M., Srinivasan, N., Sycara, K.: Owl-s 1.0 release (2004)

  55. Mika, P.: Ontologies are us: a unified model of social networks and semantics. Journal of Web Semantics (Elsevier) 5(1), 5–15 (2007)

    Article  MathSciNet  Google Scholar 

  56. Nicola, A.D., Navigli, R., Missikoff, M.: Building an eprocurement ontology with upon methodology. In: Proceedings of the 15th e-Challenges Conferences, Lublijana, Slovenia (2005)

    Google Scholar 

  57. Noy, D.L., McGuinness, N.: Ontology development 101: a guide to creating your first ontology. Technical Report, Stanford University, Stanford (2000)

  58. Noy, N., Musen, M.: Anchor-prompt: using non-logical context for semantic matching. In: IJCAI Workshop on Ontologies and Information Sharing, pp. 63–70. Seattle (WA US) (2001)

  59. Noy, N.F., Musen, M.: The prompt suite: interactive tools for ontology merging and mapping. Int. J. Human-Comput. Stud. 59(6):983–1024 (2003)

    Article  Google Scholar 

  60. Patil, A., Oundhakar, S., Sheth, A., Verma, K.: Meteor-s web service annotation framework. In: World Wide Web Conference (WWW 2004) (2004)

  61. Petridis, K., Anastasopoulos, D., Saathoff, C., Timmermann, N., Kompatasiaris, I., Staab, S.: M-ontomat-annotizer: image annotation. Linking ontologies and multimedia low-level features. In: Engineered Applications of Semantic Web Session (SWEA) at the 10th International Conference on Knowledge-Based & Intelligent Information & Engineering Systems (KES 2006) (2006)

  62. Pinto, H.S., Martins, J.P.: A methodology for ontology integration. In: International Conference on Knowledge Capture (K-CAP), pp. 131–138. ACM, New York (2001)

    Chapter  Google Scholar 

  63. Popov, B., Kiryakov, A., Kirilov, A., Manov, D., Ognyanoff, D., Goranov, M.: Kim—semantic annotation platform. In: Symposium on Applied Computing, 20–23 October 2005. LNAI, vol. 2870, pp. 834–849. Springer (2005)

  64. Reeve, L., Han, H.: Survey of Semantic Annotation Platforms, pp. 1634–1638. ACM, New York (2005)

    Google Scholar 

  65. Schaffert, S.: Ikewiki: a semantic wiki for collaborative knowledge management. In: 1st International Workshop on Semantic Technologies in Collaborative Applications STICA 06 (2006)

  66. Simperl, E., Tempich, C., Vrandecic, D.: Ontology learning and population: bridging the gap between text and knowledge. Chapter: A Methodology for Ontology Learning, pp. 225–249. IOS, Amsterdam (2008)

    Google Scholar 

  67. Siorpaes, K., Hepp, M.: Games with a purpose for the semantic web. IEEE Intell. Syst. 23(3), 50–60 (2008)

    Article  Google Scholar 

  68. Siorpaes, K., Hepp, M.: OntoGame: weaving the semantic web by online games. In: European Semantic Web Conference (ESWC 2008), Teneriffe, Spain. Springer LNCS (2008)

  69. Siorpaes, K., Hepp, M., Klotz, A., Hackl, M.: Myontology: tapping the wisdom of crowds for building ontologies. Technical report, STI Innsbruck Technical Report (2008)

  70. Souzis, A.: Building a semantic wiki. IEEE Intell. Syst. 20(5), 87–91 (2005)

    Article  Google Scholar 

  71. Specia, L., Motta, E.: Integrating folksonomies with the semantic web. In: ESWC 2007. Springer, New York (2007)

    Google Scholar 

  72. Spyns, P., Tang, Y., Meersman, R.: A model theory inspired collaborative ontology engineering methodology. Technical Report, VUB StarLab (2007)

  73. Staab, S., Schnurr, H.P., Studer, R., Sure, Y.: Knowledge processes and ontologies. IEEE Intell. Syst. 16(1), 26–34 (2001)

    Article  Google Scholar 

  74. Sure, Y.: Methodology, tools and case studies for ontology based knowledge management. PhD thesis, University of Karlsruhe (2003)

  75. Uren, V., Cimiano, P., Iria, J., Handschuh, S., Vargas-Vera, M., Motta, E., Ciravegna, F.: Semantic annotation for knowledge management: requirements and a survey of the state of the art. Web Semantics: Science, Services and Agents on the World Wide Web 4(1), 14–28 (2006)

    Article  Google Scholar 

  76. Uschold, M., King, M.: Towards a methodology for building ontologies. In: Workshop on Basic Ontological Issues in Knowledge Sharing, Montreal, Canada (1995)

  77. Van Damme, C., Hepp, M., Siorpaes, K.: Folksontology: An integrated approach for turning folksonomies into ontologies. In: Proceedings of the Workshop Bridging the Gap between Semantic Web and Web 2.0 at the ESWC 2007, Innsbruck, Austria, 7 June 2007, p. 15. LNCS Springer (2007)

  78. Vargas-Vera, M., Motta, E., Domingue, J., Lanzoni, M., Stutt, A., Ciravegna, F.: Mnm: ontology driven semi-automatic and automatic support for semantic markup. In: 13th International Conference on Knowledge Engineering and Management (EKAW2002) (2002)

  79. Völkel, M., Krötzsch, M., Vrandecic, D., Haller, H.: Semantic Wikipedia. 23–26 May 2006

  80. Von Ahn, L., Dabbish, L.: Labeling images with a computer game. In: CHI 2004. ACM, New York (2004)

    Google Scholar 

  81. von Ahn, L., Dabbish, L.,: Designing games with a purpose. Commun. ACM 51(8), 58–67 (2008)

    Google Scholar 

  82. Vrandecic, D., Pinto, S., Tempich, C., Sure, Y.: The diligent knowledge process. J. Knowl. Manag. 9(5), 85–96 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Semantic Technology Institute (STI) Innsbruck, University Innsbruck, Innsbruck, Austria

    Katharina Siorpaes & Elena Simperl

Authors
  1. Katharina Siorpaes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elena Simperl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katharina Siorpaes.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Siorpaes, K., Simperl, E. Human Intelligence in the Process of Semantic Content Creation. World Wide Web 13, 33–59 (2010). https://doi.org/10.1007/s11280-009-0078-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-009-0078-0

Keywords

Search

Navigation