Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted?

  • 344 Accesses

  • 16 Citations

Abstract

Quantifying the relative performance of individual scholars has become an integral part of decision-making in research policy. The objective of the present study was to evaluate if the scholarship rank of Brazilian Council for Scientific and Technological Development (CNPq) researchers in Medicine is consistent with their scientific productivity. The Lattes curricula of 411 researchers (2006–2008) were included in the study. Scholarship category was the variable of interest. Other variables analyzed were: time since receiving the doctorate, teaching activity (undergraduate, master’s and doctoral students), number of articles published, and number of papers indexed by the Institute for Scientific Information (ISI) and Scopus databases. Additional performance indicators included were: citations, h-index, and m-index. There was a significant difference among scholarship categories regarding number of papers per year, considering the entire scientific career (P < 0.001) or the last 5 years (P < 0.001). There was no significant difference among scholarship categories regarding the number of citations per article in the ISI (Thomson Reuters) database (P = 0.23). There was a significant difference in h-index among scholarship categories in both databases, i.e. (P < 0.001) and Scopus (P < 0.001). Regarding the m-index, there was a significant difference among categories only in the ISI database (P = 0.012). According to our findings, a better instrument for qualitative and quantitative indicators is needed to identify researchers with outstanding scientific output.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. Abramo, G., D’Angelo, C. A., & Costa, F. (2011a). National research assessment exercises: The effects of changing the rules of the game during the game. Scientometrics, 88, 229–238.

  2. Abramo, G., D’Angelo, C. A., & Costa, F. (2011b). Research productivity: Are higher academic ranks more productive than lower ones? Scientometrics. doi:10.1007/s11192-011-0426-6.

  3. Allen, L., Jones, C., Dolby, K., Lynn, D., & Walport, M. (2009). Looking for landmarks: The role of expert review and bibliometric analysis in evaluating scientific publication outputs. PLoS One, 4(6), e5910. doi:10.1371/journal.pone.0005910.

  4. Arruda, D., Bezerra, F., Neris, V. A., Toro, P. R., & Wainer, J. (2009). Brazilian computer science research: Gender and regional distributions. Scientometrics, 79(3), 651–665.

  5. Barata, R. B., & Goldbaum, M. (2003). A profile of researchers in public health with productivity grants from the Brazilian National Research Council (CNPq). Cad Saude Publica, 19(6), 1863–1876.

  6. Boell, S. K., & Wilson, C. S. (2010). Journal impact factors for evaluating scientific performance: Use of h-like indicators. Scientometrics, 82, 613–626.

  7. Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index? Journal of the American Society for Information Science, 58, 1381–1385.

  8. Bornmann, L., Mutz, R., & Daniel, H.-D. (2008a). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science & Technology, 59(5), 830–837.

  9. Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H.-D. (2008b). Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8, 93–102.

  10. Bornmann, L., Wallon, G., & Ledin, A. (2008c). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data. Research Evaluation, 17(2), 149–156.

  11. Browman, H. I., & Stergiou, K. I. (2008). Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely. Ethics in Science and Environmental Politics, 8, 1–3.

  12. Burrell, Q. L. (2007a). Hirsch index or Hirsch rate? Some thoughts arising from Liang’s data. Scientometrics, 73(1), 19–28.

  13. Burrell, Q. L. (2007b). Hirsch’s h-index: A stochastic model. Journal of Informetrics, 1(1), 16–25.

  14. Cavalcante, R. A., Barbosa, D. R., Bonan, P. R. F., Pires, M. B. O., & Martelli-Junior, H. (2008). Perfil dos pesquisadores da área de odontologia no Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq). Revista Brasileira de Epidemiologia, 11(1), 106–113.

  15. CNPq, Ministério de Ciência e Tecnologia. (2010). Critérios de Julgamento—CA-MD. http://www.cnpq.br/cas/ca-md.htm#criterios. Accessed 4th Aug 2010.

  16. Franceschini, F., & Maisano, D. (2011). Proposals for evaluating the regularity of a scientist’s research output. Scientometrics, 88, 279–295.

  17. Geuna, A., & Martin, B. R. (2003). University research evaluation and funding: An international comparison. Minerva, 41, 277–304.

  18. Glanzel, W. (2006). On the h-index: A mathematical approach to a new measure of publication activity and citation impact. Scientometrics, 67, 315–321.

  19. Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56, 357–367.

  20. Haeffner-Cavaillon, N., & Graillot-Gak, C. (2009). The use of bibliometric indicators to help peer-review assessment. Archivum Immunologiae et Therapiae Experimentalis (Warsz), 57(1), 33–38.

  21. Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572.

  22. Jeang, K. T. (2008). H-index, mentoring-index, highly-cited and highly-accessed: How to evaluate scientists? Retrovirology, 5, 106.

  23. Lane, J. (2010). Let’s make science metrics more scientific. Nature, 464(7288), 488–489.

  24. Lehmann, S., Jackson, A. D., & Lautrup, B. E. (2006). Measures for measures. Nature, 444(7122), 1003–1004.

  25. Leite, P., Mugnaini, R., & Leta, J. (2011). A new indicator for international visibility: Exploring Brazilian scientific community. Scientometrics, 88, 311–319.

  26. Martelli-Junior, H., Martelli, D. R., Quirino, I. G., Oliveira, M. C., Lima, L. S., & Oliveira, E. A. (2010). CNPq researchers in medicine: A comparative study of research areas. Revista Da Associacao Medica Brasileira, 56(4), 478–483.

  27. Mendes, P. H. C., Martelli, D. R., Souza, W. P., Filho, S. Q., & Martelli Junior, H. (2010). Perfil dos pesquisadores bolsistas de produtividade científica na medicina no CNPq, Brasil. Revista Brasileira de Educação Médica (in press).

  28. Moed, H. F. (2009). New developments in the use of citation analysis in research evaluation. Arch Immunol Ther Exp (Warsz), 57(1), 13–18. doi:10.1007/s00005-009-0001-5.

  29. Montgomery, D. C. (2008). Design and analysis of experiments (Vol. 1, 7th ed.). Boston: Wiley.

  30. Oliveira, E. A., Pecoits-Filho, R., Quirino, I. G., Oliveira, M. C. L., Martelli, D. R., Lima, L. S., et al. (2011a). Perfil e produção científica dos pesquisadores do CNPq nas áreas de Nefrologia e Urologia. Jornal Brasileiro de Nefrologia, 33, 31–37.

  31. Oliveira, E. A., Ribeiro, A. L. P., Quirino, I. G., Oliveira, M. C., Martelli, D. R., Lima, L. S., et al. (2011b). Perfil e produção científica dos pesquisadores do Conselho Nacional de Desenvolvimento Científico e Tecnológico na área de Cardiologia. Arquivos Brasileiro de Cardiologia. doi:10.1590/S0066-782X2011005000086

  32. Panaretos, J., & Malesios, C. (2009). Assessing scientific research performance and impact with single indices. Scientometrics, 81(3), 635–670.

  33. Petherick, A. (2010). High hopes for Brazilian science. Nature, 465(7299), 674–675.

  34. Podlubny, I. (2005). Comparison of scientific impact expressed by the number of citations in different fields of science. Scientometrics, 64, 95–99.

  35. Regalado, A. (2010). Science in Brazil. Brazilian science: Riding a gusher. Science, 330(6009), 1306–1312. doi:10.1126/science.330.6009.1306.

  36. Santos, N. C. F., Candido, L. F. O., & Kuppens, C. L. (2010). Produtividade em pesquisa do CNPq: Análise do perfil dos pesquisadores da química. Quimica Nova, 33(2), 489–495.

  37. Santos, S. M. C., Lima, L. S., Martelli, D. R. B., & Martelli Junior, H. (2009). Perfil dos pesquisadores da Saúde Coletiva no Conselho Nacional de Desenvolvimento Científico e Tecnológico. Physis Revista de Saúde Coletiva, 19(3), 761–775.

  38. Shibayama, S. (2011). Distribution of academic research funds: A case of Japanese national research grant. Scientometrics, 88, 43–60.

  39. van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67, 491–502.

Download references

Acknowledgements

This study was partially supported by CNPq (National Council for Scientific and Technological Development) and FAPEMIG (Research Support Foundation of Minas Gerais). Eduardo A. Oliveira and Ana Cristina Simões e Silva were the recipients of CNPq scholarships Category 2 in the area of Medicine. Hercílio Martelli-Júnior and Enrico A. Colosimo were the recipients of CNPq scholarships Category 2 in the areas of Dentistry and Mathematics, respectively.

Author information

Correspondence to Eduardo A. Oliveira.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Oliveira, E.A., Colosimo, E.A., Martelli, D.R. et al. Comparison of Brazilian researchers in clinical medicine: are criteria for ranking well-adjusted?. Scientometrics 90, 429–443 (2012). https://doi.org/10.1007/s11192-011-0492-9

Download citation

Keywords

  • Scientific publication indicators
  • Clinical medicine
  • Health postgraduate programs
  • Health sciences
  • H index
  • Scopus

JEL Classification

  • I23