Skip to main content
Log in

Citation algorithms for identifying research milestones driving biomedical innovation

  • Published:
Scientometrics Aims and scope Submit manuscript


Scientific activity plays a major role in innovation for biomedicine and healthcare. For instance, fundamental research on disease pathologies and mechanisms can generate potential targets for drug therapy. This co-evolution is punctuated by papers which provide new perspectives and open new domains. Despite the relationship between scientific discovery and biomedical advancement, identifying these research milestones that truly impact biomedical innovation can be difficult and is largely based solely on the opinions of subject matter experts. Here, we consider whether a new class of citation algorithms that identify seminal scientific works in a field, Reference Publication Year Spectroscopy (RPYS) and multi-RPYS, can identify the connections between innovation (e.g., therapeutic treatments) and the foundational research underlying them. Specifically, we assess whether the results of these analytic techniques converge with expert opinions on research milestones driving biomedical innovation in the treatment of Basal Cell Carcinoma. Our results show that these algorithms successfully identify the majority of milestone papers detailed by experts (Wong and Dlugosz in J Investig Dermatol 134(e1):E18–E22, 2014)—thereby validating the power of these algorithms to converge on independent opinions of seminal scientific works derived by subject matter experts. These advances offer an opportunity to identify scientific activities enabling innovation in biomedicine.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others


  1. Implementations of RPYS can be retrieved at and CRExplorer (the first option) offers the additional option of disambiguation of the cited references (Thor et al. 2016), while RPYS i/o (the second option) offers in addition to standard RPYS, Mult-RPYS as a graphically rich visualization (Comins and Leydesdorff 2016a). A pilot version of this same technique (rpys.exe) can be found at (Marx et al. 2014).


  • Agarwal, P., & Searls, D. B. (2009). Can literature analysis identify innovation drivers in drug discovery? Nature Reviews Drug Discovery, 8(11), 865–878.

    Article  Google Scholar 

  • Bornmann, L., & Mutz, R. (2016). Growth rates of modern science: A bibliometric analysis based on number of publications and cited references. Journal of the Associaton for Information Science and Techology, 66(11), 2215–2222.

    Article  Google Scholar 

  • Comins, J. A., & Hussey, T. W. (2015a). Compressing multiple scales of impact detection by Reference Publication Year Spectroscopy. Journal of Informetrics, 9(3), 449–454. doi:10.1016/j.joi.2015.03.003.

    Article  Google Scholar 

  • Comins, J. A., & Hussey, T. W. (2015b). Detecting seminal research contributions to the development and use of the global positioning system by reference publication year spectroscopy. Scientometrics, 104(2), 575–580. doi:10.1007/s11192-015-1598-2.

    Article  Google Scholar 

  • Comins, J. A., & Leydesdorff, L. (2016a). RPYS i/o: software demonstration of a web-based tool for the historiography and visualization of citation classics, sleeping beauties and research fronts. Scientometrics, 107(3), 1509–1517.

    Article  Google Scholar 

  • Comins, J. A., & Leydesdorff, L. (2016b). Identification of long-term concept-symbols among citations: Can documents be clustered in terms of common intellectual histories? (pp. 1–20) arXiv:1601.00288.

  • de Solla Price, D. J. (1965). Networks of scientific papers. Science, 149(3683), 510–515. Accessed 16 Dec 2014.

  • Elango, B., Bornmann, L., & Kannan, G. (2016). Detecting the historical roots of tribology research: A bibliometric analysis. arXiv:1601.00141.

  • Garfield, E., Malin, M. V., & Small, H. (1978). Citation data as science indicators. In Y. Elkana, J. Lederberg, R. K. Merton, A. Thackray, & H. Zuckerman (Eds.), Toward a metric of science: The advent of science indicators (pp. 179–207). New York: Wiley.

    Google Scholar 

  • Garfield, E., Sher, I. H., & Torpie, R. J. (1964). The use of citation data in writing the history of science. Philadelphia, PA: Institute for Scientific Information.

    Google Scholar 

  • Kostoff, R., & Shlesinger, M. (2005). CAB: Citation-assisted background. Scientometrics, 62, 199–212. Accessed 16 Dec 2014.

  • Leydesdorff, L., Bornmann, L., Marx, W., & Milojević, S. (2014). Referenced Publication Years Spectroscopy applied to iMetrics: Scientometrics, Journal of Informetrics, and a relevant subset of JASIST. Journal of Informetrics, 1901, 1–34. Accessed 23 Sept 2014.

  • Leydesdorff, L., & Rafols, I. (2011). How do emerging technologies conquer the world? An exploration of patterns of diffusion and network formation. Journal of the American Society for Information Science and Technology, 62(5), 846–860.

    Article  Google Scholar 

  • Marx, W., & Bornmann, L. (2013). Tracing the origin of a scientific legend by reference publication year spectroscopy (RPYS): The legend of the Darwin finches. Scientometrics, 99(3), 839–844. doi:10.1007/s11192-013-1200-8.

    Article  Google Scholar 

  • Marx, W., Bornmann, L., Barth, A., & Leydesdorff, L. (2014). Detecting the historical roots of research fields by reference publication year spectroscopy (RPYS). Journal of the Association for Information Science and Technology, 65(4), 751–764. doi:10.1002/asi.23089.

    Article  Google Scholar 

  • Mina, A., Ramlogan, R., Tampubolon, G., & Metcalfe, J. S. (2007). Mapping evolutionary trajectories: Applications to the growth and transformation of medical knowledge. Research Policy, 36(5), 789–806.

    Article  Google Scholar 

  • Mogoutov, A., Cambrosio, A., Keating, P., & Mustar, P. (2008). Biomedical innovation at the laboratory, clinical and commercial interface: A new method for mapping research projects, publications and patents in the field of microarrays. Journal of Informetrics, 2(4), 341–353.

    Article  Google Scholar 

  • Nelson, R. R., Buterbaugh, K., Perl, M., & Gelijns, A. (2011). How medical know-how progresses. Research Policy, 40(10), 1339–1344.

    Article  Google Scholar 

  • Petersen, A., Rotolo, D., & Leydesdorff, L. (2016). A triple helix model of medical innovations: Supply, demand, and technological capabilities in terms of medical subject headings. Research Policy, 45(3), 666–681. doi:10.1016/j.respol.2015.12.004.

    Article  Google Scholar 

  • Radicchi, F., Fortuno, S., & Castellano, C. (2008). Universality of citation distributions: Toward an objective measure of scientific impact. Proceedings of the National Academy of Sciences, 105(45), 17268–17272. Accessed 16 Dec 2014.

  • Swanson, D. R. (1990). Medical literature as a potential source of new knowledge. Bulletin of the Medical Library Association, 78, 29–37.

    Google Scholar 

  • Thor, A., Marx, W., Leydesdorff, L., & Bornmann, L. (2016). Introducing CitedReferencesExplorer (CRExplorer): A program for Reference Publication Year Spectroscopy with Cited References Disambiguation. Journal of Informetrics, 10(2), 503–515. doi:10.1016/j.joi.2016.02.005.

    Article  Google Scholar 

  • Van Raan, A. (2000). On growth, ageing, and fractal differentiation of science. Scientometrics, 47, 347–362. Accessed 16 Dec 2014.

  • Von Hippel, E. (1988). The sources of innovation. Oxford: Oxford University Press.

    Google Scholar 

  • Williams, R. S., Lotia, S., Holloway, A. K., & Pico, A. R. (2015). From scientific discovery to cures: Bright stars within a galaxy. Cell, 163, 21–23.

    Article  Google Scholar 

  • Wong, S. Y., & Dlugosz, A. A. (2014). Basal cell carcinoma, Hedgehog signaling, and targeted therapeutics: The long and winding road. The Journal of Investigative Dermatology, 134(e1), E18–E22. doi:10.1038/skinbio.2014.4.

    Article  Google Scholar 

  • Wray, K. B., & Bornmann, L. (2015). Philosophy of science viewed through the lense of “Referenced Publication Years Spectroscopy” (RPYS). Scientometrics, 102(3), 1987–1996.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Jordan A. Comins.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Comins, J.A., Leydesdorff, L. Citation algorithms for identifying research milestones driving biomedical innovation. Scientometrics 110, 1495–1504 (2017).

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: