Processing RDF Using Hadoop

Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 177)


The basic inspiration of the Semantic Web is to broaden the existing human-readable web by encoding some of the semantics of resources in a machine-understandable form. There are various formats and technologies that help in making it possible. These technologies comprise of the Resource Description Framework (RDF), an assortment of data interchange formats like RDF/XML, N3, N-Triples, and representations such as RDF Schema (RDFS) and Web Ontology Language (OWL), all of which help in providing a proper description of concepts, terms and associations in a particular knowledge domain. Presently, there are some existing frameworks for semantic web technologies but they have limitations for large RDF graphs. Thus storing and efficiently querying a large number of RDF triples is a challenging and important problem. We propose a framework which is constructed using Hadoop to store and retrieve massive numbers of RDF triples by taking advantage of the cloud computing paradigm. Hadoop permits the development of reliable, scalable, proficient, cost-effective and distributed computing using very simple Java interfaces. Hadoop comprises of a distributed file system HDFS to stock up RDF data. Hadoop Map Reduce framework is used to answer the queries. MapReduce job divides the input data-set into independent units which are processed in parallel by the map tasks , which then serve as inputs to the reduce tasks. This framework takes care of task scheduling, supervising them and re-execution of the failed tasks. Uniqueness of our approach is its efficient, automatic allocation of data and work across machines and in turn exploiting the fundamental parallelism of the CPU cores. Results confirm that our proposed framework offers multi-fold efficiencies and benefits which include on-demand processing, operational scalability, competence, cost efficiency and local access to enormous data, contrasting the various traditional approaches.


Semantic Web Distributed Computing Map-Reduce Programming SPARQL Graph Data Performance Evaluation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Amazon. Amazon EC2 Instance Types (2010),
  2. 2.
    Berners-Lee, T., Hendler, J., Lassila, O.: The Semantic Web. Scientific American Magazine (May 17, 2001)Google Scholar
  3. 3.
    Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. In: Proceedings of the USENIX Symposium on Operating Systems Design & Implementation, OSDI, pp. 137–147 (2004)Google Scholar
  4. 4.
    DeWitt, D., Stonebraker, M.: MapReduce: A major step backwards,, (retrieved August 28, 2010)
  5. 5.
    Guo, Y., Pan, Z., Heflin, J.: LUBM: A benchmark for OWL knowledge base systems. Journal of Web Semantics 3(2), 158–182 (2005)CrossRefGoogle Scholar
  6. 6.
    Grigoris, A., van Harmelen, F.: A Semantic Web Primer, 2nd edn. The MIT Press (2008)Google Scholar
  7. 7.
    Urbani, J., Kotoulas, S., Oren, E., van Harmelen, F.: Scalable Distributed Reasoning Using MapReduce. In: Bernstein, A., Karger, D.R., Heath, T., Feigenbaum, L., Maynard, D., Motta, E., Thirunarayan, K. (eds.) ISWC 2009. LNCS, vol. 5823, pp. 634–649. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Hendler, J.: Web 3.0: The Dawn of Semantic Search. IEEE Computer (January 2010)Google Scholar
  9. 9.
    Kolas, D., Emmons, I., Dean, M.: Efficient Linked-List RDF Indexing in Parliament. In: The Proceedings of the Scalable Semantic Web (SSWS) Workshop of ISWC 2009 (2009)Google Scholar
  10. 10.
    Li, P., Zeng, Y., Kotoulas, S., Urbani, J., Zhong, N.: The Quest for Parallel Reasoning on the Semantic Web. In: Liu, J., Wu, J., Yao, Y., Nishida, T. (eds.) AMT 2009. LNCS, vol. 5820, pp. 430–441. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
  12. 12.
    Mika, P., Tummarello, G.: Web Semantics in the Clouds. IEEE Intelligent Systems 23(5), 82–87 (2008)CrossRefGoogle Scholar
  13. 13.
    Husain, M., McGlothlin, J., Masud, M.M., Khan, L., Thuraisingham, B.: Heuristics Based Query Processing for Large RDF Graphs Using Cloud Computing. Journal of Latex Class Files 6(1) (January 2007)Google Scholar
  14. 14.
    Project Voldemort (2010),
  15. 15.
    RDF. Resource Description Framework (RDF) (2010),
  16. 16.
    Rohloff, K., Schantz, R.: High-Performance, Massively Scalable Distributed Systems using the MapReduce Software Framework: The SHARD Triple-Store. In: International Workshop on Programming Support Innovations for Emerging Distributed Applications, PSIEtA (2010)Google Scholar
  17. 17.
    Rohloff, K., Dean, M., Emmons, I., Ryder, D., Sumner, J.: Evaluation of Triple-Store Technologies for Large Data Stores. In: 3rd International Workshop on Scalable Semantic Web Knowledge Base Systems, SSWS 2007, Vilamoura, Portugal (2007)Google Scholar
  18. 18.
    SPARQL. SPARQL Query Language for RDF (2010),

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.MS (Software Engineering)VIT UniversityVelloreIndia
  2. 2.VIT UniversityVelloreIndia

Personalised recommendations