Skip to main content
SpringerLink
Log in

Quantifying the scientific output of new researchers using the zp-index

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Despite a high level of interest in quantifying the scientific output of established researchers, there has been less of a focus on quantifying the performance of junior researchers. The available metrics that quantify a scientist’s research output all utilize citation information, which often takes a number of years to accrue and thus would disadvantage newer researchers (e.g., graduate students, post-doctoral members, new professors). Based on this critical limitation of existing metrics, we created a new metric of scientific output, the zp-index, which remedies this issue by utilizing the journal quality rather than citation count in calculating an index of scientific output. Additionally, the zp-index also takes authorship position into account by allocating empirically derived weights to each authorship position, so that first authorship publications receive more credit than later authorship positions (Study 1). Furthermore, the zp-index has equal predictive validity as a measure of the number of publications but does a better job of discriminating researcher’s scientific output and may provide different information than the number of publications (Study 2). Therefore, use of the zp-index in conjunction with the number of publications can provide a more accurate assessment of a new scientist’s academic achievements.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. It might be noted here that because the zp-index’s sensitivity can increase differentiation when it is used to rank scientists, what appear to be significant differences in ranking may arise as a consequence of smaller and potentially more random factors. The consideration of other markers of academic success, such as research funding, teaching ability and general administrative and societal contributions should help guard against potential over-weighting of ranking differences.

References

  • Abramo, G., D’Angelo, C. A., & Costa, F. D. (2010). Testing the trade-off between productivity and quality in research activities. Journal of the American Society for Information Science and Technology, 61(1), 132–140.

    Article  Google Scholar 

  • Anauati, M. V., Galiani, S., Gálvez, R. H. (2014). Quantifying the life cycle of scholarly articles across fields of economic research. Available at SSRN: http://ssrn.com/abstract=2523078.

  • APA Science Student Council. (2006). A graduate student’s guide to determining authorship credit and authorship order. Retrieved 15 Oct 2014. http://www.apa.org/science/leadership/students/authorship-paper.pdf.

  • Ball, P. (2005). Index aims for fair ranking of scientists. Nature, 436(7053), 900.

    Article  Google Scholar 

  • Bateman, T. S., & Hess, A. M. (2015). Different personal propensities among scientists relate to deeper vs. broader knowledge contributions. Proceedings of the National Academy of Sciences, 112(12), 3653–3658.

    Google Scholar 

  • Bohannon, J. (2013). Who’s afraid of peer review? Science, 342(6154), 60–65.

    Article  Google Scholar 

  • Bordons, M., Fernández, M., & Gómez, I. (2002). Advantages and limitations in the use of impact factor measures for the assessment of research performance. Scientometrics, 53(2), 195–206.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H. D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.

    Article  Google Scholar 

  • Bornmann, L., & Daniel, H. D. (2009). The state of h index research. EMBO Reports, 10(1), 2–6.

    Article  Google Scholar 

  • Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H. D. (2008). Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8(1), 93–102.

    Article  Google Scholar 

  • Cacioppo, J. T. (2008). Metrics of science. Observer, 21. http://www.psychologicalscience.org/index.php/publications/observer/2008/january-08/metrics-ofscience.html.

  • Egghe, L. (2006). Theory and practise of the g-index. Scientometrics, 69(1), 131–152.

    Article  MathSciNet  Google Scholar 

  • Garfield, E. (2006). The history and meaning of the journal impact factor. JAMA, 295(1), 90–93.

    Article  Google Scholar 

  • González-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of informetrics, 4(3), 379–391.

    Article  Google Scholar 

  • Google Scholar Blog. (2011). Google scholar citations open to all, Google.

  • Gookins, J. L. (2012). The essentials of oral abstract presentation. Retrieved 15 Oct 2014. http://www.lib.ncsu.edu/documents/vetmed/research/slides/gookin_presenting_data_2012.pdf.

  • Halonen, J. S. (2011). Are there too many psychology majors (White paper). Florida: State University System of Florida Board of Governors, University of West Florida.

    Google Scholar 

  • Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

    Article  Google Scholar 

  • Hirsch, J. E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Sciences, 104(49), 19193–19198.

    Article  Google Scholar 

  • Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4–6.

    Google Scholar 

  • Laband, D. N. (1985). An evaluation of 50” Ranked” economics departments: By quantity and quality of faculty publications and graduate student placement and research success. Southern Economic Journal, 52, 216–240.

    Article  Google Scholar 

  • Lawrence, P. A. (2007). The mismeasurement of science. Current Biology, 17(15), 583–585.

    Article  Google Scholar 

  • Lorden, J. F., Kuh, C.V., Voytuk, J. A. (2011). National Research Council (US) an assessment of research-doctorate programs: Panel on the biomedical sciences; Research-Doctorate Programs in the biomedical sciences: Selected findings from the NRC Assessment. Washington (DC): National Academies Press (US). Supplementary Materials.

  • Mazloumian, A. (2012). Predicting scholars’ scientific impact. PLoS One, 7(11), e49246. doi:10.1371/journal.pone.0049246.

    Article  Google Scholar 

  • Rezek, I., McDonald, R. J., & Kallmes, D. F. (2011). Is the h-index predictive of greater NIH funding success among academic radiologists? Academic Radiology, 18(11), 1337–1340.

    Article  Google Scholar 

  • Rohawani, M. (2013). Publishing in conference proceedings use and abuse. Professors Behaving Badly [web log]. Retrieved 15 Oct 2014. http://professorsbehavingbadly.blogspot.ca/2013/06/publishing-in-conference-proceedings.html.

  • Saha, S., Saint, S., & Christakis, D. A. (2003). Impact factor: A valid measure of journal quality? Journal of the Medical Library Association: JMLA, 91(1), 42.

    Google Scholar 

  • Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 497.

    Article  Google Scholar 

  • Sekercioglu, C. H. (2008). Quantifying coauthor contributions. Science, 322(5900), 371.

    Article  Google Scholar 

  • Simonton, D. K. (1997). Creative productivity: A predictive and explanatory model of career trajectories and landmarks. Psychological Review, 104(1), 66.

    Article  Google Scholar 

  • Tscharntke, T., Hochberg, M. E., Rand, T. A., Resh, V. H., & Krauss, J. (2007). Author sequence and credit for contributions in multiauthored publications. PLoS Biology, 5(1), e18.

    Article  Google Scholar 

  • Valla, J. M. (2010). Getting hired: Publications, post-docs, and the path to professorship. Observer, 23(7). http://www.psychologicalscience.org/index.php/publications/observer/2010/september-10/getting-hired.html.

  • Venkatraman, V. (2010,April). Conventions of scientific authorship. Career Magazine. http://sciencecareers.sciencemag.org/career_magazine/previous_issues/articles/2010_04_16/caredit.a1000039.

  • Wren, J. D., Kozak, K. Z., Johnson, K. R., Deakyne, S. J., Schilling, L. M., & Dellavalle, R. P. (2007). The write position. EMBO Reports, 8(11), 988–991.

    Article  Google Scholar 

  • Zhang, C. T. (2009). A proposal for calculating weighted citations based on author rank. EMBO Reports, 10(5), 416–417.

    Article  Google Scholar 

Download references

Acknowledgments

We would like to thank multiple people who have contributed to the completion of this manuscript. First, we would like to thank the numberous research asssistants from the Peterson Lab for their careful collection and coding of the data. In particular, we would like to thank Julia Tsui for her coordination with the project. Second, we would like to thank Sabrina Thai, Jessical Maxwell, Xiaowen Xu, Kate Guan, Dimitry Besson, and Christian Poole for their helpful comments on previous versions of the paper. Finally, we would like to thank the editors for their helpful comments in revising the manuscript.

Author information

Authors and Affiliations

  1. University of Toronto, 100 St. George St. 4th Floor, Sidney Smith Hall, Toronto, ON, M5S 3G3, Canada

    Christopher Zou & Jordan B. Peterson

Authors
  1. Christopher Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jordan B. Peterson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christopher Zou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zou, C., Peterson, J.B. Quantifying the scientific output of new researchers using the zp-index. Scientometrics 106, 901–916 (2016). https://doi.org/10.1007/s11192-015-1807-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-015-1807-z

Keywords

Search

Navigation