LOD Lab: Scalable Linked Data Processing

  • Wouter Beek
  • Laurens Rietveld
  • Filip Ilievski
  • Stefan SchlobachEmail author
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9885)


With tens if not hundreds of billions of logical statements, the Linked Open Data (LOD) is one of the biggest knowledge bases ever built. As such it is a gigantic source of information for applications in various domains, but also given its size an ideal test-bed for knowledge representation and reasoning, heterogeneous nature, and complexity.

However, making use of this unique resource has proven next to impossible in the past due to a number of problems, including data collection, quality, accessibility, scalability, availability and findability. The LOD Laundromat and LOD Lab are recent infrastructures that addresses these problems in a systematic way, by automatically crawling, cleaning, indexing, analysing and republishing data in a unified way. Given a family of simple tools, LOD Lab allows researchers to query, access, analyse and manipulate hundreds of thousands of data documents seamlessly, e.g. facilitating experiments (e.g. for reasoning) over hundreds of thousands of (possibly integrated) datasets based on content and meta-data.

This chapter provides the theoretical basis and practical skills required for making ideal use of this large scale experimental platform. First we study the problems that make it so hard to work with Semantic Web data in its current form. We’ll also propose generic solutions and introduce the tools the reader needs to get started with their own experiments on the LOD Cloud.


SPARQL Query Data Document Link Open Data Unique IRIs Metadata Description 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Hogan, A., Harth, A., Passant, A., Decker, S., Polleres, A.: Weaving the pedantic web. In: Linked Data on the Web Workshop (2010)Google Scholar
  2. 2.
    Hogan, A., Umbrich, J., Harth, A., Cyganiak, R., Polleres, A., Decker, S.: An empirical survey of linked data conformance. Web Semant.: Sci. Serv. Agents World Wide Web 14, 14–44 (2012)CrossRefGoogle Scholar
  3. 3.
    Beek, W., Rietveld, L., Bazoobandi, H.R., Wielemaker, J., Schlobach, S.: LOD laundromat: a uniform way of publishing other people’s dirty data. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 213–228. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11964-9_14 Google Scholar
  4. 4.
    Rietveld, L., Verborgh, R., Beek, W., Vander Sande, M., Schlobach, S.: Linkeddata-as-a-service: the semantic web redeployed. In: Gandon, F., Sabou, M., Sack, H., d’Amato, C., Cudré-Mauroux, P., Zimmermann, A. (eds.) ESWC 2015. LNCS, vol. 9088, pp. 471–487. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-18818-8_29 CrossRefGoogle Scholar
  5. 5.
    Verborgh, R., et al.: Querying datasets on the web with high availability. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 180–196. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11964-9_12 Google Scholar
  6. 6.
    Ermilov, I., Martin, M., Lehmann, J., Auer, S.: Linked open data statistics: collection and exploitation. In: Klinov, P., Mouromtsev, D. (eds.) KESW 2013. CCIS, vol. 394, pp. 242–249. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-41360-5_19 CrossRefGoogle Scholar
  7. 7.
    Auer, S., Demter, J., Martin, M., Lehmann, J.: LODStats – an extensible framework for high-performance dataset analytics. In: Teije, A., Völker, J., Handschuh, S., Stuckenschmidt, H., d’Acquin, M., Nikolov, A., Aussenac-Gilles, N., Hernandez, N. (eds.) EKAW 2012. LNCS (LNAI), vol. 7603, pp. 353–362. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33876-2_31 CrossRefGoogle Scholar
  8. 8.
    Buil-Aranda, C., Hogan, A., Umbrich, J., Vandenbussche, P.-Y.: SPARQL web-querying infrastructure: ready for action? In: Alani, H., et al. (eds.) ISWC 2013. LNCS, vol. 8219, pp. 277–293. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-41338-4_18 CrossRefGoogle Scholar
  9. 9.
    Cheng, G., Gong, S., Qu, Y.: An empirical study of vocabulary relatedness and its application to recommender systems. In: Aroyo, L., Welty, C., Alani, H., Taylor, J., Bernstein, A., Kagal, L., Noy, N., Blomqvist, E. (eds.) ISWC 2011. LNCS, vol. 7031, pp. 98–113. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-25073-6_7 CrossRefGoogle Scholar
  10. 10.
    Ge, W., Chen, J., Hu, W., Qu, Y.: Object link structure in the semantic web. In: Aroyo, L., Antoniou, G., Hyvönen, E., Teije, A., Stuckenschmidt, H., Cabral, L., Tudorache, T. (eds.) ESWC 2010. LNCS, vol. 6089, pp. 257–271. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-13489-0_18 CrossRefGoogle Scholar
  11. 11.
    Alexander, K., Cyganiak, R., Hausenbals, M., Zhao, J.: Describing linked datasets with the VoID vocabulary, March 2011.
  12. 12.
    Millard, I., Glaser, H., Salvadores, M., Shadbolt, N.: Consuming multiple Linked Data sources: challenges and experiences. In: First International Workshop on Consuming Linked Data (COLD), November 2010Google Scholar
  13. 13.
    Prud’hommeaux, E., Buil-Aranda, C.: SPARQL 1.1 Federated Query (2013).
  14. 14.
    Fernández, J.D., Martínez-Prieto, M.A., Gutiérrez, C., Polleres, A., Arias, M.: Binary RDF representation for publication and exchange (HDT). Web Seman.: Sci. Serv. Agents World Wide Web 19, 22–41 (2013)CrossRefGoogle Scholar
  15. 15.
    Mäkelä, E.: Aether – generating and viewing extended void statistical descriptions of RDF datasets. In: Presutti, V., Blomqvist, E., Troncy, R., Sack, H., Papadakis, I., Tordai, A. (eds.) ESWC 2014. LNCS, vol. 8798, pp. 429–433. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11955-7_61 Google Scholar
  16. 16.
    Callahan, A., Cruz-Toledo, J., Ansell, P., Dumontier, M.: Bio2RDF release 2: improved coverage, interoperability and provenance of life science linked data. In: Cimiano, P., Corcho, O., Presutti, V., Hollink, L., Rudolph, S. (eds.) ESWC 2013. LNCS, vol. 7882, pp. 200–212. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38288-8_14 CrossRefGoogle Scholar
  17. 17.
    Bazoobandi, H.R., Rooij, S., Urbani, J., Teije, A., Harmelen, F., Bal, H.: A compact in-memory dictionary for RDF data. In: Gandon, F., Sabou, M., Sack, H., d’Amato, C., Cudré-Mauroux, P., Zimmermann, A. (eds.) ESWC 2015. LNCS, vol. 9088, pp. 205–220. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-18818-8_13 CrossRefGoogle Scholar
  18. 18.
    Christophides, V., Efthymiou, V., Stefanidis, K.: Entity Resolution in the Web of Data. Morgan & Claypool Publishers, San Rafael (2015)Google Scholar
  19. 19.
    Rietveld, L., Beek, W., Schlobach, S.: LOD lab: experiments at LOD scale. In: Arenas, M., et al. (eds.) ISWC 2015. LNCS, vol. 9367, pp. 339–355. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-25010-6_23 CrossRefGoogle Scholar
  20. 20.
    Schmachtenberg, M., Bizer, C., Paulheim, H.: Adoption of the linked data best practices in different topical domains. In: Mika, P., et al. (eds.) ISWC 2014. LNCS, vol. 8796, pp. 245–260. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-11964-9_16 Google Scholar
  21. 21.
    Isele, R., Umbrich, J., Bizer, C., Harth, A.: LDSpider: an open-source crawling framework for the Web of Linked Data. In: 9th International Semantic Web Conference. Citeseer (2010)Google Scholar
  22. 22.
    Harris, S., Seaborne, A.: SPARQL 1.1 query language, March 2013Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Wouter Beek
    • 1
  • Laurens Rietveld
    • 1
  • Filip Ilievski
    • 1
  • Stefan Schlobach
    • 1
    Email author
  1. 1.Department of Computer ScienceVU University AmsterdamAmsterdamNetherlands

Personalised recommendations