Abstract
Apart from the well-established facility of searching for research articles, the modern academic search engines also provide information regarding the scientists themselves. Until recently, this information was limited to include the articles each scientist has authored, accompanied by their corresponding citations. Presently, the most popular scientific databases have enriched this information by including scientometrics, that is, metrics which evaluate the research activity of a scientist. Although the computation of scientometrics is relatively easy when dealing with small data sets, in larger scales the problem becomes more challenging since the involved data is huge and cannot be handled efficiently by a single workstation. In this paper we attempt to address this interesting problem by employing MapReduce, a distributed, fault-tolerant framework used to solve problems in large scales without considering complex network programming details. We demonstrate that by setting the problem in a manner that is compatible to MapReduce, we can achieve an effective and scalable solution. We propose four algorithms which exploit the features of the framework and we compare their efficiency by conducting experiments on a large dataset comprised of roughly 1.8 million scientific documents.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
CiteSeerX Data, http://csxstatic.ist.psu.edu/about/data
Abouzeid, A., Bajda-Pawlikowski, K., Abadi, D., Silberschatz, A., Rasin, A.: HadoopDB: An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads. Proceedings of the VLDB Endowment 2(1), 922–933 (2009)
Borthakur, D.: The Hadoop distributed file system: Architecture and design (2007)
Dean, J., Ghemawat, S.: MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM 51(1), 107–113 (2008)
Egghe, L.: Theory and Practise of the g-index. Scientometrics 69(1), 131–152 (2006)
Elsayed, T., Lin, J., Oard, D.: Pairwise document similarity in large collections with MapReduce. In: Proceedings of 46th Annual Meeting of the Association for Computational Linguistics on Human Language Technologies, pp. 265–268 (2008)
Ghemawat, S., Dean, J.: MapReduce: Simplified Data Processing on Large Clusters. In: Symposium on Operating System Design and Implementation (OSDI 2004), San Francisco, California, USA, pp. 137–150 (2004)
Ghemawat, S., Gobioff, H., Leung, S.: The Google file system. ACM SIGOPS Operating Systems Review 37, 29–43 (2003)
Hirsch, J.: An Index to Quantify an Individual’s Scientific Research Output. Proceedings of the National Academy of Sciences 102(46), 16569 (2005)
Katsaros, D., Akritidis, L., Bozanis, P.: The f index: Quantifying the Impact of Coterminal Citations on Scientists’ Ranking. Journal of the American Society for Information Science and Technology 60(5), 1051–1056 (2009)
Lin, J.: Scalable language processing algorithms for the masses: A case study in computing word co-occurrence matrices with MapReduce. In: Proceedings of the Conference on Empirical Methods in Language Processing, pp. 419–428 (2008)
Lin, J., Dyer, C.: Data-intensive Text Processing with MapReduce. Synthesis Lectures on Human Language Technologies 3(1), 1–177 (2010)
McCreadie, R., Macdonald, C., Ounis, I.: On single-pass indexing with MapReduce. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 742–743 (2009)
Sidiropoulos, A., Katsaros, D., Manolopoulos, Y.: Generalized Hirsch h-index for Disclosing Latent Facts in Citation Networks. Scientometrics 72(2), 253–280
Sidiropoulos, A., Manolopoulos, Y.: A Citation-Based System to Assist Prize Awarding. ACM SIGMOD Record 34(4), 60 (2005)
Yang, H., Dasdan, A., Hsiao, R., Parker, D.: Map-reduce-merge: simplified relational data processing on large clusters. In: Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pp. 1029–1040 (2007)
Malewicz, G., Austern, M.H., Bik, A.J.C., Dehnert, J.C., Horn, I., Leiser, N., Czajkowski, G.: Pregel: A System for Large-Scale Graph Processing. In: Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, pp. 135–146 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Akritidis, L., Bozanis, P. (2012). Computing Scientometrics in Large-Scale Academic Search Engines with MapReduce. In: Wang, X.S., Cruz, I., Delis, A., Huang, G. (eds) Web Information Systems Engineering - WISE 2012. WISE 2012. Lecture Notes in Computer Science, vol 7651. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-35063-4_44
Download citation
DOI: https://doi.org/10.1007/978-3-642-35063-4_44
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-35062-7
Online ISBN: 978-3-642-35063-4
eBook Packages: Computer ScienceComputer Science (R0)