, Volume 110, Issue 3, pp 1495–1504 | Cite as

Citation algorithms for identifying research milestones driving biomedical innovation

  • Jordan A. Comins
  • Loet Leydesdorff


Scientific activity plays a major role in innovation for biomedicine and healthcare. For instance, fundamental research on disease pathologies and mechanisms can generate potential targets for drug therapy. This co-evolution is punctuated by papers which provide new perspectives and open new domains. Despite the relationship between scientific discovery and biomedical advancement, identifying these research milestones that truly impact biomedical innovation can be difficult and is largely based solely on the opinions of subject matter experts. Here, we consider whether a new class of citation algorithms that identify seminal scientific works in a field, Reference Publication Year Spectroscopy (RPYS) and multi-RPYS, can identify the connections between innovation (e.g., therapeutic treatments) and the foundational research underlying them. Specifically, we assess whether the results of these analytic techniques converge with expert opinions on research milestones driving biomedical innovation in the treatment of Basal Cell Carcinoma. Our results show that these algorithms successfully identify the majority of milestone papers detailed by experts (Wong and Dlugosz in J Investig Dermatol 134(e1):E18–E22, 2014)—thereby validating the power of these algorithms to converge on independent opinions of seminal scientific works derived by subject matter experts. These advances offer an opportunity to identify scientific activities enabling innovation in biomedicine.


Reference Publication Year Spectroscopy Citation analysis Algorithmic historiography Bibliometrics 


  1. Agarwal, P., & Searls, D. B. (2009). Can literature analysis identify innovation drivers in drug discovery? Nature Reviews Drug Discovery, 8(11), 865–878.CrossRefGoogle Scholar
  2. Bornmann, L., & Mutz, R. (2016). Growth rates of modern science: A bibliometric analysis based on number of publications and cited references. Journal of the Associaton for Information Science and Techology, 66(11), 2215–2222.CrossRefGoogle Scholar
  3. Comins, J. A., & Hussey, T. W. (2015a). Compressing multiple scales of impact detection by Reference Publication Year Spectroscopy. Journal of Informetrics, 9(3), 449–454. doi: 10.1016/j.joi.2015.03.003.CrossRefGoogle Scholar
  4. Comins, J. A., & Hussey, T. W. (2015b). Detecting seminal research contributions to the development and use of the global positioning system by reference publication year spectroscopy. Scientometrics, 104(2), 575–580. doi: 10.1007/s11192-015-1598-2.CrossRefGoogle Scholar
  5. Comins, J. A., & Leydesdorff, L. (2016a). RPYS i/o: software demonstration of a web-based tool for the historiography and visualization of citation classics, sleeping beauties and research fronts. Scientometrics, 107(3), 1509–1517.CrossRefGoogle Scholar
  6. Comins, J. A., & Leydesdorff, L. (2016b). Identification of long-term concept-symbols among citations: Can documents be clustered in terms of common intellectual histories? (pp. 1–20) arXiv:1601.00288.
  7. de Solla Price, D. J. (1965). Networks of scientific papers. Science, 149(3683), 510–515. Accessed 16 Dec 2014.
  8. Elango, B., Bornmann, L., & Kannan, G. (2016). Detecting the historical roots of tribology research: A bibliometric analysis. arXiv:1601.00141.
  9. Garfield, E., Malin, M. V., & Small, H. (1978). Citation data as science indicators. In Y. Elkana, J. Lederberg, R. K. Merton, A. Thackray, & H. Zuckerman (Eds.), Toward a metric of science: The advent of science indicators (pp. 179–207). New York: Wiley.Google Scholar
  10. Garfield, E., Sher, I. H., & Torpie, R. J. (1964). The use of citation data in writing the history of science. Philadelphia, PA: Institute for Scientific Information.Google Scholar
  11. Kostoff, R., & Shlesinger, M. (2005). CAB: Citation-assisted background. Scientometrics, 62, 199–212. Accessed 16 Dec 2014.
  12. Leydesdorff, L., Bornmann, L., Marx, W., & Milojević, S. (2014). Referenced Publication Years Spectroscopy applied to iMetrics: Scientometrics, Journal of Informetrics, and a relevant subset of JASIST. Journal of Informetrics, 1901, 1–34. Accessed 23 Sept 2014.
  13. Leydesdorff, L., & Rafols, I. (2011). How do emerging technologies conquer the world? An exploration of patterns of diffusion and network formation. Journal of the American Society for Information Science and Technology, 62(5), 846–860.CrossRefGoogle Scholar
  14. Marx, W., & Bornmann, L. (2013). Tracing the origin of a scientific legend by reference publication year spectroscopy (RPYS): The legend of the Darwin finches. Scientometrics, 99(3), 839–844. doi: 10.1007/s11192-013-1200-8.CrossRefGoogle Scholar
  15. Marx, W., Bornmann, L., Barth, A., & Leydesdorff, L. (2014). Detecting the historical roots of research fields by reference publication year spectroscopy (RPYS). Journal of the Association for Information Science and Technology, 65(4), 751–764. doi: 10.1002/asi.23089.CrossRefGoogle Scholar
  16. Mina, A., Ramlogan, R., Tampubolon, G., & Metcalfe, J. S. (2007). Mapping evolutionary trajectories: Applications to the growth and transformation of medical knowledge. Research Policy, 36(5), 789–806.CrossRefGoogle Scholar
  17. Mogoutov, A., Cambrosio, A., Keating, P., & Mustar, P. (2008). Biomedical innovation at the laboratory, clinical and commercial interface: A new method for mapping research projects, publications and patents in the field of microarrays. Journal of Informetrics, 2(4), 341–353.CrossRefGoogle Scholar
  18. Nelson, R. R., Buterbaugh, K., Perl, M., & Gelijns, A. (2011). How medical know-how progresses. Research Policy, 40(10), 1339–1344.CrossRefGoogle Scholar
  19. Petersen, A., Rotolo, D., & Leydesdorff, L. (2016). A triple helix model of medical innovations: Supply, demand, and technological capabilities in terms of medical subject headings. Research Policy, 45(3), 666–681. doi: 10.1016/j.respol.2015.12.004.CrossRefGoogle Scholar
  20. Radicchi, F., Fortuno, S., & Castellano, C. (2008). Universality of citation distributions: Toward an objective measure of scientific impact. Proceedings of the National Academy of Sciences, 105(45), 17268–17272. Accessed 16 Dec 2014.
  21. Swanson, D. R. (1990). Medical literature as a potential source of new knowledge. Bulletin of the Medical Library Association, 78, 29–37.Google Scholar
  22. Thor, A., Marx, W., Leydesdorff, L., & Bornmann, L. (2016). Introducing CitedReferencesExplorer (CRExplorer): A program for Reference Publication Year Spectroscopy with Cited References Disambiguation. Journal of Informetrics, 10(2), 503–515. doi: 10.1016/j.joi.2016.02.005.CrossRefGoogle Scholar
  23. Van Raan, A. (2000). On growth, ageing, and fractal differentiation of science. Scientometrics, 47, 347–362. Accessed 16 Dec 2014.
  24. Von Hippel, E. (1988). The sources of innovation. Oxford: Oxford University Press.Google Scholar
  25. Williams, R. S., Lotia, S., Holloway, A. K., & Pico, A. R. (2015). From scientific discovery to cures: Bright stars within a galaxy. Cell, 163, 21–23.CrossRefGoogle Scholar
  26. Wong, S. Y., & Dlugosz, A. A. (2014). Basal cell carcinoma, Hedgehog signaling, and targeted therapeutics: The long and winding road. The Journal of Investigative Dermatology, 134(e1), E18–E22. doi: 10.1038/skinbio.2014.4.CrossRefGoogle Scholar
  27. Wray, K. B., & Bornmann, L. (2015). Philosophy of science viewed through the lense of “Referenced Publication Years Spectroscopy” (RPYS). Scientometrics, 102(3), 1987–1996.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.Center for Applied Information ScienceVirginia Tech Applied Research CorporationArlingtonUSA
  2. 2.Amsterdam School of Communication Research (ASCoR)University of AmsterdamAmsterdamThe Netherlands

Personalised recommendations