Semantic Publishing Challenge – Assessing the Quality of Scientific Output in Its Ecosystem

Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 641)


The Semantic Publishing Challenge aims to involve participants in extracting data from heterogeneous sources on scholarly publications, and produce Linked Data which can be exploited by the community itself. The 2014 edition was the first attempt to organize a challenge to enable the assessment of the quality of scientific output. The 2015 edition was more explicit regarding the potential techniques, i.e., information extraction and interlinking. The current 2016 edition focuses on the multiple dimensions of scientific quality and the great potential impact of producing Linked Data for this purpose. In this paper, we discuss the overall structure of the Semantic Publishing Challenge, as it is for the 2016 edition, as well as the submitted solutions and their evaluation.


Linked data Information extraction Challenge 


Part of this research has been funded by the European Union under grant agreement no. 643410 (OpenAIRE2020).


  1. 1.
    Ahmad, R., Afzal, M.T., Qadir, M.A.: Information extraction for PDF sources based on rule-based system using integrated formats. In: Sack et al. [12], pp. 293–308Google Scholar
  2. 2.
    Bryl, V., Birukou, A., Eckert, K., Kessler, M.: What’s in the proceedings? Combining publisher’s and researcher’s perspectives. In: García Castro, A., Lange, C., Lord, P., Stevens, R. (eds.) 4th Workshop on Semantic Publishing (SePublica). CEUR Workshop Proceedings, vol. 1155, Aachen (2014)Google Scholar
  3. 3.
    Di Iorio, A., Lange, C., Dimou, A., Vahdati, S.: Semantic publishing challenge - assessing the quality of scientific output by information extraction and interlinking. In: Gandon et al. [4], pp. 65–80Google Scholar
  4. 4.
    Gandon, F., et al. (eds.): SemWebEval 2015. CCIS, vol. 548. Springer, Heidelberg (2015)Google Scholar
  5. 5.
    Klampfl, S., Kern, R.: Machine learning techniques for automatically extracting contextual information from scientific publications. In: Gandon et al. [4], pp. 105–116Google Scholar
  6. 6.
    Klampfl, S., Kern, R.: Reconstructing the logical structure of a scientific publication using machine learning. In: Sack et al. [12], pp. 255–268Google Scholar
  7. 7.
    Lange, C., Di Iorio, A.: Semantic publishing challenge – assessing the quality of scientific output. In: Presutti, V., Stankovic, M., Cambria, E., Cantador, I., Di Iorio, A., Di Noia, T., Lange, C., Reforgiato Recupero, D., Tordai, A. (eds.) SemWebEval 2014. CCIS, vol. 475, pp. 61–76. Springer, Heidelberg (2014)Google Scholar
  8. 8.
    Milicka, M., Burget, R.: Information extraction from web sources based on multi-aspect content analysis. In: Gandon et al. [4]Google Scholar
  9. 9.
    Nuzzolese, A.G., Peroni, S., Recupero, D.R.: MACJa: metadata and citations jailbreaker. In: Gandon et al. [4], pp. 117–128Google Scholar
  10. 10.
    Nuzzolese, A.G., Peroni, S., Recupero, D.R.: ACM: article content miner for assessing the quality of scientific output. In: Sack et al. [12], pp. 281–292Google Scholar
  11. 11.
    Ramesh, S.H., Dhar, A., Kumar, R.R., Anjaly, V., Sarath, K., Pearce, J., Sundaresan, K.: Automatically identify and label sections in scientific journals using conditional random fields. In: Sack et al. [12], pp. 269–280Google Scholar
  12. 12.
    Sack, H., Dietze, S., Tordai, A., Lange, C.: SemWebEval 2016. CCIS, vol. 641. Springer, Heidelberg (2016)Google Scholar
  13. 13.
    Sateli, B., Witte, R.: Automatic construction of a semantic knowledge base from CEUR workshop proceedings. In: Gandon et al. [4], pp. 129–141Google Scholar
  14. 14.
    Sateli, B., Witte, R.: An automatic workflow for the formalization of scholarly articles’ structural and semantic elements. In: Sack et al. [12], pp. 309–320Google Scholar
  15. 15.
    Shotton, D.: Semantic publishing: the coming revolution in scientific journal publishing. Learn. Publish. 22(2), 85–94 (2009)CrossRefGoogle Scholar
  16. 16.
    Shotton, D., Portwin, K., Klyne, G., Miles, A.: Adventures in semantic publishing: exemplar semantic enhancements of a research article. PLoS Comput. Biol. 5(4), e1000361 (2009)CrossRefGoogle Scholar
  17. 17.
    Tkaczyk, D., Bolikowski, L.: Extracting contextual information from scientific literature using CERMINE system. In: Gandon et al. [4]Google Scholar
  18. 18.
    Vahdati, S., Dimou, A., Lange, C., Di Iorio, A.: Semantic publishing challenge: bootstrapping a value chain for scientific data. In: Gonzalez-Beltran, A., Osborne, F., Peroni, S. (eds.) Semantics, Analytics, Visualisation: Enhancing Scholarly Data. LNCS. Springer, Heidelberg (2016)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Ghent University - iMindsGhentBelgium
  2. 2.Università di BolognaBolognaItaly
  3. 3.University of BonnBonnGermany
  4. 4.Fraunhofer IAISSankt AugustinGermany

Personalised recommendations