Abstract
In research, influence is often synonymous with importance; the researcher that is judged to be influential is often chosen for the grants, distinctions and promotions that serve as fuel for research programs. The influence of a researcher is often measured by how often he or she is cited, yet as a measure of influence, we show that citation frequency is only weakly correlated with influence ratings collected from peers. In this paper, we use machine learning to enable a new system that provides a better measure of researcher influence. This system predicts the influence of one researcher on another via a range of novel social, linguistic, psychological, and bibliometric features. To collect data for training and testing this approach, we conducted a survey of 74 researchers in the field of computational linguistics, and collected thousands of influence ratings. Our results on this data show that our approach significantly outperforms measures based on citations alone, improving prediction accuracy by 56%. We also perform a detailed analysis of the key features in our model, and make some important observations about the scientific and non-scientific factors that most predict researcher influence.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aral, S., Walker, D.: Identifying influential and susceptible members of social networks. Science 337(6092), 337–341 (2012)
Tang, T.Y., Winoto, P., McCalla, G.I.: Further thoughts on context aware paper recommendations for education. In: Manouselis, N., et al. (eds.) Recommender Systems for Technology Enhanced Learning, 16 p. Springer (in press, 2013)
Hirsch, J.E.: An index to quantify an individual’s scientific research output. Proceedings of the National academy of Sciences of the United States of America 102(46), 16569–16572 (2005)
Bornmann, L., Daniel, H.D.: What do citation counts measure? A review of studies on citing behavior. Journal of Documentation 64(1), 45–80 (2008)
Van Raan, A.F.: Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics 67(3), 491–502 (2006)
Radev, D.R., Muthukrishnan, P., Qazvinian, V.: The ACL anthology network corpus. In: Proc. ACL Workshop on Natural Language Processing and Information Retrieval for Digital Libraries, pp. 54–61 (2009)
Turney, P.D., Pantel, P.: From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research 37(1), 141–188 (2010)
Lawani, S.M., Bayer, A.E.: Validity of citation criteria for assessing the influence of scientific publications: New evidence with peer assessment. Journal of the American Society for Information Science 34(1), 59–66 (1983)
Joachims, T.: Optimizing search engines using clickthrough data. In: Proc. KDD, pp. 133–142 (2002)
Cialdini, R.B.: Influence: The Psychology of Persuasion. HarperCollins (2007)
Milgram, S.: Behavioral study of obedience. The Journal of Abnormal and Social Psychology 67(4), 371–378 (1963)
Liu, X., Bollen, J., Nelson, M.L., Van de Sompel, H.: Co-authorship networks in the digital library research community. Information Processing & Management 41(6), 1462–1480 (2005)
Bergsma, S., Dredze, M., Van Durme, B., Wilson, T., Yarowsky, D.: Broadly improving user classification via communication-based name and location clustering on twitter. In: Proc. NAACL-HLT, pp. 1010–1019 (2013)
Bergsma, S., Lin, D.: Bootstrapping path-based pronoun resolution. In: Proc. Coling-ACL, pp. 33–40 (2006)
Dubin, D.: The most influential paper Gerard Salton never wrote. Library Trends 52(4), 748–764 (2004)
Zhu, X., Turney, P., Lemire, D., Vellino, A.: Measuring academic influence: Not all citations are equal. Journal of the American Society for Information Science and Technology, JASIST (to appear, 2013)
Joachims, T.: Training linear SVMs in linear time. In: Proc. KDD (2006)
Hall, D., Jurafsky, D., Manning, C.D.: Studying the history of ideas using topic models. In: Proc. EMNLP, pp. 363–371 (2008)
Gerrish, S., Blei, D.M.: A language-based approach to measuring scholarly impact. In: Proc. ICML, pp. 375–382 (2010)
Radev, D.R., Joseph, M.T., Gibson, B., Muthukrishnan, P.: A bibliometric and network analysis of the field of computational linguistics. Journal of the American Society for Information Science and Technology, JASIST (2009)
Johri, N., Ramage, D., McFarland, D., Jurafsky, D.: A study of academic collaborations in computational linguistics using a latent mixture of authors model. In: Proc. 5th ACL-HLT Workshop on Language Technology for Cultural Heritage, Social Sciences, and Humanities, pp. 124–132 (2011)
Liben-Nowell, D., Kleinberg, J.: The link prediction problem for social networks. In: Proc. CIKM, pp. 556–559 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Bergsma, S., Mandryk, R.L., McCalla, G. (2014). Learning to Measure Influence in a Scientific Social Network. In: Sokolova, M., van Beek, P. (eds) Advances in Artificial Intelligence. Canadian AI 2014. Lecture Notes in Computer Science(), vol 8436. Springer, Cham. https://doi.org/10.1007/978-3-319-06483-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-06483-3_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-06482-6
Online ISBN: 978-3-319-06483-3
eBook Packages: Computer ScienceComputer Science (R0)