Semantics Based Intelligent Search in Large Digital Repositories Using Hadoop MapReduce

  • Muhammad Idris
  • Shujaat Hussain
  • Taqdir Ali
  • Byeong Ho Kang
  • Sungyoung Lee
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8867)


Information contained in large digital repositories consisting of billions of documents represented in various formats make it difficult to retrieve the desired information. It is necessary to develop techniques that are accurate and fast enough to retrieve the desired information from hay stack of online digital repositories. On one hand, Keyword based systems and techniques have high recall and performance, however, they have low precision. On the other hand, semantics based systems have high precision and good recall, however, their performance decreases with data growth. Therefore, to improve precision and performance, we propose semantics based searching framework using Hadoop MapReduce to process the data at large scale. We apply semantic techniques to extract required information from digital documents and MapReduce programming model to apply these techniques. Application of semantic techniques using MapReduce distributed model will result in high precision and good performance of user query result.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Manyika, J., Chui, M., Brown, B., Bughin, J., Dobbs, R., Roxburgh, C., Byers, A.H.: Big data: The next frontier for innovation, competition and productivityGoogle Scholar
  2. 2.
    Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic inquiry and word count: Liwc 2001, vol. 71, p. 2001. Lawrence Erlbaum Associates, Mahway (2001)Google Scholar
  3. 3.
    Moffat, A., Zobel, J.: Self-indexing inverted files for fast text retrieval. ACM Transactions on Information Systems (TOIS) 14, 349–379 (1996)CrossRefGoogle Scholar
  4. 4.
    IEEE-org: IEEE digital library,
  5. 5.
    ACM-Org: ACM digital library,
  6. 6.
    National Library of Medicine.: Medline,
  7. 7.
    Khattaka, A.: Context-aware search in dynamic repositories of digital documentsGoogle Scholar
  8. 8.
    Bonino, D., Corno, F., Farinetti, L., Bosca, A.: Ontology driven semantic search. WSEAS Transaction on Information Science and Application 1, 1597–1605 (2004)Google Scholar
  9. 9.
    Rodríguez, E.A.: Determining semantic similarity among entity classes from different ontologies. IEEE Transactions on Knowledge and Data EngineeringGoogle Scholar
  10. 10.
    Laclavík, M., Šeleng, M., Hluchý, L.: Towards large scale semantic annotation built on mapReduce architecture. In: Bubak, M., van Albada, G.D., Dongarra, J., Sloot, P.M.A. (eds.) ICCS 2008, Part III. LNCS, vol. 5103, pp. 331–338. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Ghemawat, S., Gobioff, H., Leung, S.T.: The google file system. ACM SIGOPS Operating Systems Review 37, 29–43 (2003)CrossRefGoogle Scholar
  12. 12.
    Borthakur, D.: Facebook has the worlds largest hadoop clusterGoogle Scholar
  13. 13.
    Yuan, P., Sha, C., Wang, X., Yang, B., Zhou, A., Yang, S.: XML structural similarity search using mapReduce. In: Chen, L., Tang, C., Yang, J., Gao, Y. (eds.) WAIM 2010. LNCS, vol. 6184, pp. 169–181. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  14. 14.
    Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. Communications of the ACM 51, 107–113 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Muhammad Idris
    • 1
  • Shujaat Hussain
    • 1
  • Taqdir Ali
    • 1
  • Byeong Ho Kang
    • 2
  • Sungyoung Lee
    • 1
  1. 1.Department of Computer EngineeringKyung Hee UniversityKorea
  2. 2.Dept. of ScienceEngineering and Technology UoTAustralia

Personalised recommendations