Advertisement

Scientometrics

, Volume 116, Issue 3, pp 1867–1886 | Cite as

Evaluating author name disambiguation for digital libraries: a case of DBLP

  • Jinseok KimEmail author
Article

Abstract

Author name ambiguity in a digital library may affect the findings of research that mines authorship data of the library. This study evaluates author name disambiguation in DBLP, a widely used but insufficiently evaluated digital library for its disambiguation performance. In doing so, this study takes a triangulation approach that author name disambiguation for a digital library can be better evaluated when its performance is assessed on multiple labeled datasets with comparison to baselines. Tested on three types of labeled data containing 5000 to 6 M disambiguated names, DBLP is shown to assign author names quite accurately to distinct authors, resulting in pairwise precision, recall, and F1 measures around 0.90 or above overall. DBLP’s author name disambiguation performs well even on large ambiguous name blocks but deficiently on distinguishing authors with the same names. Compared to other disambiguation algorithms, DBLP’s disambiguation performance is quite competitive, possibly due to its hybrid disambiguation approach combining algorithmic disambiguation and manual error correction. A discussion follows on strengths and weaknesses of labeled datasets used in this study for future efforts to evaluate author name disambiguation on a digital library scale.

Keywords

Author name disambiguation Digital library Triangulation Disambiguation evaluation DBLP 

Notes

Acknowledgements

I would like to thank Florian Reitz (Leibniz Center for Informatics, Schloss Dagstuhl, Germany) for providing the list of synonyms in DBLP and Alan Filipe Santana (Departamento de Ciência da Computação, Universidade Federal de Minas Gerais, Brazil) for sharing the raw KISTI dataset. I am also thankful to anonymous reviewers for their comments. This work was supported by grants from the National Science Foundation (Grants #1561687 and #1535370), the Alfred P. Sloan Foundation, and the Ewing Marion Kauffman Foundation.

References

  1. Abbott, A., Cyranoski, D., Jones, N., Maher, B., Schiermeier, Q., & Van Noorden, R. (2010). Do metrics matter? Nature, 465(7300), 860–862.CrossRefGoogle Scholar
  2. Bilder, G. (2011). Disambiguation without de-duplication: Modeling authority and trust in the ORCID system. Retrieved from https://www.crossref.org/wp/labs/whitepapers/disambiguation-deduplication-wp-v4.pdf.
  3. Bornmann, L., & Mutz, R. (2015). Growth rates of modern science: A bibliometric analysis based on the number of publications and cited references. Journal of the Association for Information Science and Technology, 66(11), 2215–2222.CrossRefGoogle Scholar
  4. Cota, R. G., Ferreira, A. A., Nascimento, C., Goncalves, M. A., & Laender, A. H. F. (2010). An unsupervised heuristic-based hierarchical method for name disambiguation in bibliographic citations. Journal of the American Society for Information Science and Technology, 61(9), 1853–1870.CrossRefGoogle Scholar
  5. D’Angelo, C. A., Giuffrida, C., & Abramo, G. (2011). A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments. Journal of the American Society for Information Science and Technology, 62(2), 257–269.CrossRefGoogle Scholar
  6. Erman, N., & Todorovski, L. (2015). The effects of measurement error in case of scientific network analysis. Scientometrics, 104(2), 453–473.CrossRefGoogle Scholar
  7. Fegley, B. D., & Torvik, V. I. (2013). Has large-scale named-entity network analysis been resting on a flawed assumption? PLoS ONE, 8(7), 670299.CrossRefGoogle Scholar
  8. Ferreira, A. A., Goncalves, M. A., & Laender, A. H. F. (2012). A brief survey of automatic methods for author name disambiguation. Sigmod Record, 41(2), 15–26.CrossRefGoogle Scholar
  9. Franceschet, M. (2011). Collaboration in computer science: A network science approach. Journal of the American Society for Information Science and Technology, 62(10), 1992–2012.CrossRefGoogle Scholar
  10. Haak, L. L., Fenner, M., Paglione, L., Pentz, E., & Ratner, H. (2012). ORCID: A system to uniquely identify researchers. Learned Publishing, 25(4), 259–264.CrossRefGoogle Scholar
  11. Han, H., Zha, H. Y., & Giles, C. L. (2005). Name disambiguation spectral in author citations using a K-way clustering method. In Proceedings of the 5th ACM/IEEE joint conference on digital libraries, proceedings, pp. 334–343.Google Scholar
  12. Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41(2), 251–261.CrossRefGoogle Scholar
  13. Ioannidis, J. P. A., Boyack, K. W., & Klavans, R. (2014). Estimates of the continuously publishing core in the scientific workforce. PLoS ONE, 9(7), 6101698.CrossRefGoogle Scholar
  14. Kang, I. S., Kim, P., Lee, S., Jung, H., & You, B. J. (2011). Construction of a large-scale test set for author disambiguation. Information Processing and Management, 47(3), 452–465.CrossRefGoogle Scholar
  15. Kawashima, H., & Tomizawa, H. (2015). Accuracy evaluation of Scopus author ID based on the largest funding database in Japan. Scientometrics, 103(3), 1061–1071.CrossRefGoogle Scholar
  16. Kim, J., & Diesner, J. (2015). The effect of data pre-processing on understanding the evolution of collaboration networks. Journal of Informetrics, 9(1), 226–236.CrossRefGoogle Scholar
  17. Kim, J., & Diesner, J. (2016). Distortive effects of initial-based name disambiguation on measurements of large-scale coauthorship networks. Journal of the Association for Information Science and Technology, 67(6), 1446–1461.CrossRefGoogle Scholar
  18. Lerchenmueller, M. J., & Sorenson, O. (2016). Author disambiguation in PubMed: Evidence on the precision and recall of authority among NIH-funded scientists. PLoS ONE, 11(7), e0158731.CrossRefGoogle Scholar
  19. Levin, M., Krawczyk, S., Bethard, S., & Jurafsky, D. (2012). Citation-based bootstrapping for large-scale author disambiguation. Journal of the American Society for Information Science and Technology, 63(5), 1030–1047.CrossRefGoogle Scholar
  20. Ley, M. (2002). The DBLP computer science bibliography: Evolution, research issues, Perspectives. In A. F. Laender & A. Oliveira (Eds.), String processing and information retrieval (Vol. 2476, pp. 1–10). Berlin: Springer.CrossRefGoogle Scholar
  21. Ley, M. (2009). DBLP: Some lessons learned. Proceedings of the VLDB Endowment, 2(2), 1493–1500.CrossRefGoogle Scholar
  22. Liu, W., Islamaj Dogan, R., Kim, S., Comeau, D. C., Kim, W., Yeganova, L., et al. (2014). Author name disambiguation for PubMed. Journal of the Association for Information Science and Technology, 65(4), 765–781.CrossRefGoogle Scholar
  23. Louppe, G., Al-Natsheh, H. T., Susik, M., & Maguire, E. J. (2016). Ethnicity sensitive author disambiguation using semi-supervised learning. Knowledge Engineering and Semantic Web, Kesw, 2016(649), 272–287.CrossRefGoogle Scholar
  24. Martin, T., Ball, B., Karrer, B., & Newman, M. E. J. (2013). Coauthorship and citation patterns in the physical review. Physical Review E, 88(1), 012814.CrossRefGoogle Scholar
  25. Menestrina, D., Whang, S. E., & Garcia-Molina, H. (2010). Evaluating entity resolution results. Proceedings of the VLDB Endowment, 3(1–2), 208–219.CrossRefGoogle Scholar
  26. Milojević, S. (2013). Accuracy of simple, initials-based methods for author name disambiguation. Journal of Informetrics, 7(4), 767–773.CrossRefGoogle Scholar
  27. Moed, H. F., Aisati, M., & Plume, A. (2013). Studying scientific migration in Scopus. Scientometrics, 94(3), 929–942.CrossRefGoogle Scholar
  28. Müller, M. C., Reitz, F., & Roy, N. (2017). Data sets for author name disambiguation: an empirical analysis and a new resource. Scientometrics, 111(3), 1467–1500.CrossRefGoogle Scholar
  29. Newman, M. E. J. (2001). The structure of scientific collaboration networks. Proceedings of the National Academy of Sciences USA, 98(2), 404–409.MathSciNetCrossRefGoogle Scholar
  30. On, B. W., Lee, D., Kang, J., & Mitra, P. (2005). Comparative study of name disambiguation problem using a scalable blocking-based framework. In Proceedings of the 5th ACM/IEEE joint conference on digital libraries, proceedings, pp. 344–353.Google Scholar
  31. Qian, Y., Zheng, Q., Sakai, T., Ye, J., & Liu, J. (2015). Dynamic author name disambiguation for growing digital libraries. Information Retrieval Journal, 18(5), 379–412.CrossRefGoogle Scholar
  32. Reijnhoudt, L., Costas, R., Noyons, E., Borner, K., & Scharnhorst, A. (2014). ‘Seed plus expand’: A general methodology for detecting publication oeuvres of individual researchers. Scientometrics, 101(2), 1403–1417.CrossRefGoogle Scholar
  33. Reitz, F., & Hoffmann, O. (2013). Learning from the past: An analysis of person name corrections in the DBLP collection and social network properties of affected entities. In T. Özyer, J. Rokne, G. Wagner, & A. H. P. Reuser (Eds.), The influence of technology on social network analysis and mining (pp. 427–453). Vienna: Springer.CrossRefGoogle Scholar
  34. Santana, A. F., Goncalves, M. A., Laender, A. H. F., & Ferreira, A. A. (2015). On the combination of domain-specific heuristics for author name disambiguation: The nearest cluster method. International Journal on Digital Libraries, 16(3–4), 229–246.CrossRefGoogle Scholar
  35. Schulz, C., Mazloumian, A., Petersen, A. M., Penner, O., & Helbing, D. (2014). Exploiting citation networks for large-scale author name disambiguation. EPJ Data Science, 3(1), 11.CrossRefGoogle Scholar
  36. Shin, D., Kim, T., Choi, J., & Kim, J. (2014). Author name disambiguation using a graph model with node splitting and merging based on bibliographic information. Scientometrics, 100(1), 15–50.CrossRefGoogle Scholar
  37. Sinatra, R., Wang, D., Deville, P., Song, C. M., & Barabasi, A. L. (2016). Quantifying the evolution of individual scientific impact. Science, 354(6312), aaf5239.CrossRefGoogle Scholar
  38. Smalheiser, N. R., & Torvik, V. I. (2009). Author name disambiguation. Annual Review of Information Science and Technology, 43, 287–313.CrossRefGoogle Scholar
  39. Strotmann, A., & Zhao, D. Z. (2012). Author name disambiguation: What difference does it make in author-based citation analysis? Journal of the American Society for Information Science and Technology, 63(9), 1820–1833.CrossRefGoogle Scholar
  40. Tang, J., Fong, A. C. M., Wang, B., & Zhang, J. (2012). A unified probabilistic framework for name disambiguation in digital library. IEEE Transactions on Knowledge and Data Engineering, 24(6), 975–987.CrossRefGoogle Scholar
  41. Torvik, V. I., & Smalheiser, N. R. (2009). Author name disambiguation in MEDLINE. ACM Transactions on Knowledge Discovery from Data, 3(3), 11.CrossRefGoogle Scholar
  42. Wang, D. J., Shi, X. L., McFarland, D. A., & Leskovec, J. (2012). Measurement error in network data: A re-classification. Social Networks, 34(4), 396–409.CrossRefGoogle Scholar
  43. Wang, X., Tang, J., Cheng, H., & Yu, P. S. (2011). ADANA: Active name disambiguation. In Paper presented at the 2011 IEEE 11th international conference on data mining. http://ieeexplore.ieee.org/document/6137284/.

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2018

Authors and Affiliations

  1. 1.Institute for Research on Innovation and Science, University of MichiganAnn ArborUSA

Personalised recommendations